diff --git a/COPYING b/COPYING new file mode 100644 index 00000000..94a9ed02 --- /dev/null +++ b/COPYING @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. diff --git a/Makefile.am b/Makefile.am new file mode 100644 index 00000000..3d663c0a --- /dev/null +++ b/Makefile.am @@ -0,0 +1,224 @@ +ACLOCAL_AMFLAGS = -I build-aux/m4 +SUBDIRS = src +.PHONY: deploy FORCE + +GZIP_ENV="-9n" + +BITCOIND_BIN=$(top_builddir)/src/bitcoind$(EXEEXT) +BITCOIN_QT_BIN=$(top_builddir)/src/qt/bitcoin-qt$(EXEEXT) +BITCOIN_CLI_BIN=$(top_builddir)/src/bitcoin-cli$(EXEEXT) +BITCOIN_WIN_INSTALLER=$(PACKAGE)-$(PACKAGE_VERSION)-win$(WINDOWS_BITS)-setup$(EXEEXT) + +# MCHN START + +MULTICHAIND_BIN=$(top_builddir)/src/multichaind$(EXEEXT) +MULTICHAIN_CLI_BIN=$(top_builddir)/src/multichain-cli$(EXEEXT) +MULTICHAIN_UTIL_BIN=$(top_builddir)/src/multichain-util$(EXEEXT) + +# MCHN END + +OSX_APP=Bitcoin-Qt.app +OSX_DMG=Bitcoin-Qt.dmg +OSX_DEPLOY_SCRIPT=$(top_srcdir)/contrib/macdeploy/macdeployqtplus +OSX_FANCY_PLIST=$(top_srcdir)/contrib/macdeploy/fancy.plist +OSX_INSTALLER_ICONS=$(top_srcdir)/src/qt/res/icons/bitcoin.icns +OSX_PLIST=$(top_srcdir)/share/qt/Info.plist #not installed +OSX_QT_TRANSLATIONS = da,de,es,hu,ru,uk,zh_CN,zh_TW + +DIST_DOCS = $(wildcard doc/*.md) $(wildcard doc/release-notes/*.md) + +WINDOWS_PACKAGING = $(top_srcdir)/share/pixmaps/bitcoin.ico \ + $(top_srcdir)/share/pixmaps/nsis-header.bmp \ + $(top_srcdir)/share/pixmaps/nsis-wizard.bmp \ + $(top_srcdir)/doc/README_windows.txt + +OSX_PACKAGING = $(OSX_DEPLOY_SCRIPT) $(OSX_FANCY_PLIST) $(OSX_INSTALLER_ICONS) \ + $(top_srcdir)/contrib/macdeploy/background.png \ + $(top_srcdir)/contrib/macdeploy/DS_Store \ + $(top_srcdir)/contrib/macdeploy/detached-sig-apply.sh \ + $(top_srcdir)/contrib/macdeploy/detached-sig-create.sh + +COVERAGE_INFO = baseline_filtered_combined.info baseline.info block_test.info \ + leveldb_baseline.info test_bitcoin_filtered.info total_coverage.info \ + baseline_filtered.info block_test_filtered.info \ + leveldb_baseline_filtered.info test_bitcoin_coverage.info test_bitcoin.info + +dist-hook: + -$(MAKE) -C $(top_distdir)/src/leveldb clean + -$(MAKE) -C $(top_distdir)/src/secp256k1 distclean + -$(GIT) archive --format=tar HEAD -- src/clientversion.cpp | $(AMTAR) -C $(top_distdir) -xf - + +distcheck-hook: + $(MKDIR_P) $(top_distdir)/_build/src/leveldb + cp -rf $(top_srcdir)/src/leveldb/* $(top_distdir)/_build/src/leveldb/ + -$(MAKE) -C $(top_distdir)/_build/src/leveldb clean + +distcleancheck: + @: + +$(BITCOIN_WIN_INSTALLER): all-recursive + $(MKDIR_P) $(top_builddir)/release + STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIND_BIN) $(top_builddir)/release + STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIN_QT_BIN) $(top_builddir)/release + STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIN_CLI_BIN) $(top_builddir)/release + STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(MULTICHAIND_BIN) $(top_builddir)/release # MCHN + STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(MULTICHAI_CLI_BIN) $(top_builddir)/release # MCHN + STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(MULTICHAI_UTIL_BIN) $(top_builddir)/release # MCHN + @test -f $(MAKENSIS) && $(MAKENSIS) $(top_builddir)/share/setup.nsi || \ + echo error: could not build $@ + +$(if $(findstring src/,$(MAKECMDGOALS)),$(MAKECMDGOALS), none): FORCE + $(MAKE) -C src $(patsubst src/%,%,$@) + +$(OSX_APP)/Contents/PkgInfo: + $(MKDIR_P) $(@D) + @echo "APPL????" > $@ + +$(OSX_APP)/Contents/Resources/empty.lproj: + $(MKDIR_P) $(@D) + @touch $@ + +$(OSX_APP)/Contents/Info.plist: $(OSX_PLIST) + $(MKDIR_P) $(@D) + $(INSTALL_DATA) $< $@ + +$(OSX_APP)/Contents/Resources/bitcoin.icns: $(OSX_INSTALLER_ICONS) + $(MKDIR_P) $(@D) + $(INSTALL_DATA) $< $@ + +$(OSX_APP)/Contents/MacOS/Bitcoin-Qt: $(BITCOIN_QT_BIN) + $(MKDIR_P) $(@D) + STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $< $@ + +OSX_APP_BUILT=$(OSX_APP)/Contents/PkgInfo $(OSX_APP)/Contents/Resources/empty.lproj \ + $(OSX_APP)/Contents/Resources/bitcoin.icns $(OSX_APP)/Contents/Info.plist \ + $(OSX_APP)/Contents/MacOS/Bitcoin-Qt + +if BUILD_DARWIN +$(OSX_DMG): $(OSX_APP_BUILT) $(OSX_PACKAGING) + $(OSX_DEPLOY_SCRIPT) $(OSX_APP) -add-qt-tr $(OSX_QT_TRANSLATIONS) -translations-dir=$(QT_TRANSLATION_DIR) -dmg -fancy $(OSX_FANCY_PLIST) -verbose 2 + +deploydir: $(OSX_DMG) +else +APP_DIST_DIR=$(top_builddir)/dist +APP_DIST_EXTRAS=$(APP_DIST_DIR)/.background/background.png $(APP_DIST_DIR)/.DS_Store $(APP_DIST_DIR)/Applications + +$(APP_DIST_DIR)/Applications: + @rm -f $@ + @cd $(@D); $(LN_S) /Applications $(@F) + +$(APP_DIST_EXTRAS): $(APP_DIST_DIR)/$(OSX_APP)/Contents/MacOS/Bitcoin-Qt + +$(OSX_DMG): $(APP_DIST_EXTRAS) + $(GENISOIMAGE) -no-cache-inodes -D -l -probe -V "Bitcoin-Qt" -no-pad -r -apple -o $@ dist + +$(APP_DIST_DIR)/.background/background.png: + $(MKDIR_P) $(@D) + $(INSTALL) $(top_srcdir)/contrib/macdeploy/background.png $@ +$(APP_DIST_DIR)/.DS_Store: + $(INSTALL) $(top_srcdir)/contrib/macdeploy/DS_Store $@ + +$(APP_DIST_DIR)/$(OSX_APP)/Contents/MacOS/Bitcoin-Qt: $(OSX_APP_BUILT) $(OSX_PACKAGING) + INSTALLNAMETOOL=$(INSTALLNAMETOOL) OTOOL=$(OTOOL) STRIP=$(STRIP) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) -translations-dir=$(QT_TRANSLATION_DIR) -add-qt-tr $(OSX_QT_TRANSLATIONS) -verbose 2 + +deploydir: $(APP_DIST_EXTRAS) +endif + +if TARGET_DARWIN +appbundle: $(OSX_APP_BUILT) +deploy: $(OSX_DMG) +endif +if TARGET_WINDOWS +deploy: $(BITCOIN_WIN_INSTALLER) +endif + +$(BITCOIN_QT_BIN): FORCE + $(MAKE) -C src qt/$(@F) + +$(BITCOIND_BIN): FORCE + $(MAKE) -C src $(@F) + +$(BITCOIN_CLI_BIN): FORCE + $(MAKE) -C src $(@F) + +# MCHN START + +$(MULTICHAIND_BIN): FORCE + $(MAKE) -C src $(@F) + +$(MULTICHAIN_CLI_BIN): FORCE + $(MAKE) -C src $(@F) + +$(MULTICHAIN_UTIL_BIN): FORCE + $(MAKE) -C src $(@F) + +# MCHN END + +if USE_LCOV + +baseline.info: + $(LCOV) -c -i -d $(abs_builddir)/src -o $@ + +baseline_filtered.info: baseline.info + $(LCOV) -r $< "/usr/include/*" -o $@ + +leveldb_baseline.info: baseline_filtered.info + $(LCOV) -c -i -d $(abs_builddir)/src/leveldb -b $(abs_builddir)/src/leveldb -o $@ + +leveldb_baseline_filtered.info: leveldb_baseline.info + $(LCOV) -r $< "/usr/include/*" -o $@ + +baseline_filtered_combined.info: leveldb_baseline_filtered.info baseline_filtered.info + $(LCOV) -a leveldb_baseline_filtered.info -a baseline_filtered.info -o $@ + +test_bitcoin.info: baseline_filtered_combined.info + $(MAKE) -C src/ check + $(LCOV) -c -d $(abs_builddir)/src -t test_bitcoin -o $@ + $(LCOV) -z -d $(abs_builddir)/src + $(LCOV) -z -d $(abs_builddir)/src/leveldb + +test_bitcoin_filtered.info: test_bitcoin.info + $(LCOV) -r $< "/usr/include/*" -o $@ + +block_test.info: test_bitcoin_filtered.info + $(MKDIR_P) qa/tmp + -@TIMEOUT=15 qa/pull-tester/run-bitcoind-for-test.sh $(JAVA) -jar $(JAVA_COMPARISON_TOOL) qa/tmp/compTool 0 + $(LCOV) -c -d $(abs_builddir)/src --t BitcoinJBlockTest -o $@ + $(LCOV) -z -d $(abs_builddir)/src + $(LCOV) -z -d $(abs_builddir)/src/leveldb + +block_test_filtered.info: block_test.info + $(LCOV) -r $< "/usr/include/*" -o $@ + +test_bitcoin_coverage.info: baseline_filtered_combined.info test_bitcoin_filtered.info + $(LCOV) -a baseline_filtered.info -a leveldb_baseline_filtered.info -a test_bitcoin_filtered.info -o $@ + +total_coverage.info: baseline_filtered_combined.info test_bitcoin_filtered.info block_test_filtered.info + $(LCOV) -a baseline_filtered.info -a leveldb_baseline_filtered.info -a test_bitcoin_filtered.info -a block_test_filtered.info -o $@ | $(GREP) "\%" | $(AWK) '{ print substr($$3,2,50) "/" $$5 }' > coverage_percent.txt + +test_bitcoin.coverage/.dirstamp: test_bitcoin_coverage.info + $(GENHTML) -s $< -o $(@D) + @touch $@ + +total.coverage/.dirstamp: total_coverage.info + $(GENHTML) -s $< -o $(@D) + @touch $@ + +cov: test_bitcoin.coverage/.dirstamp total.coverage/.dirstamp + +endif + +if USE_COMPARISON_TOOL +check-local: + $(MKDIR_P) qa/tmp + @qa/pull-tester/run-bitcoind-for-test.sh $(JAVA) -jar $(JAVA_COMPARISON_TOOL) qa/tmp/compTool $(COMPARISON_TOOL_REORG_TESTS) 2>&1 +endif + +EXTRA_DIST = $(top_srcdir)/share/genbuild.sh qa/pull-tester/rpc-tests.sh qa/pull-tester/run-bitcoin-cli qa/rpc-tests $(DIST_DOCS) $(WINDOWS_PACKAGING) $(OSX_PACKAGING) + +CLEANFILES = $(OSX_DMG) $(BITCOIN_WIN_INSTALLER) + +.INTERMEDIATE: $(COVERAGE_INFO) + +clean-local: + rm -rf test_bitcoin.coverage/ total.coverage/ $(OSX_APP) diff --git a/README.md b/README.md new file mode 100644 index 00000000..41d1fb26 --- /dev/null +++ b/README.md @@ -0,0 +1,72 @@ +Building MultiChain +===================== + +Linux Build Notes +===================== + +System requirements +-------------------- + +C++ compilers are memory-hungry. It is recommended to have at least 1 GB of +memory available when compiling MultiChain. With 512MB of memory or less +compilation will take much longer due to swap thrashing. + +Dependency Build Instructions: Ubuntu +---------------------------------------------- + +sudo apt-get update +sudo apt-get install build-essential libtool autotools-dev automake pkg-config libssl-dev libevent-dev bsdmainutils +sudo apt-get install libboost-all-dev +sudo apt-get install git +sudo apt-get install software-properties-common +sudo add-apt-repository ppa:bitcoin/bitcoin +sudo apt-get update +sudo apt-get install libdb4.8-dev libdb4.8++-dev + + +To Build +--------------------- + +./autogen.sh +./configure +make + +Notes +----- + +This will build multichaind, multichain-cli and multitchain-util in ./src + +The release is built with GCC and then "strip multichaind" to strip the debug +symbols, which reduces the executable size by about 90%. + + +Windows Build Notes +===================== + +Dependency Build Instructions: Ubuntu +---------------------------------------------- + +sudo apt-get update +sudo apt-get install build-essential libtool autotools-dev automake pkg-config libssl-dev libevent-dev bsdmainutils +sudo apt-get install g++-mingw-w64-i686 mingw-w64-i686-dev g++-mingw-w64-x86-64 mingw-w64-x86-64-dev curl +sudo apt-get install libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-program-options-dev libboost-test-dev libboost-thread-dev +sudo apt-get install git +sudo add-apt-repository ppa:bitcoin/bitcoin +sudo apt-get update +sudo apt-get install libdb4.8-dev libdb4.8++-dev + +To Build +--------------------- + +./autogen.sh +cd depends +make HOST=x86_64-w64-mingw32 -j4 +cd .. +./configure --prefix=`pwd`/depends/x86_64-w64-mingw32 --enable-cxx --disable-shared --enable-static --with-pic +make + +Notes +----- + +This will build multichaind.exe, multichain-cli.exe and multitchain-util.exe in ./src + diff --git a/TODO b/TODO new file mode 100644 index 00000000..ac3b57c5 --- /dev/null +++ b/TODO @@ -0,0 +1,17 @@ +TO DO BEFORE MULTICHAIN 1.0 BETA +-------------------------------- +* Automatic checkpointing for admin/miner permission changes +* Add parameter for minimum transactions per block +* Increase maximum data size per transaction from 8 MB to 32+ MB +* Add monotonically increasing version number to getinfo +* Allow metadata and P2SH addresses in exchange APIs +* Fix APIs for performing external mining +* Reactivate transaction notifications mechanism in new wallet +* Add getruntimeparams command to retrieve runtime parameters +* Add parameters to clearmempool and setlastblock to support rollback +* Review and organize error codes +* Review online and built-in documentation for consistency +* Review upstream security patches since Bitcoin Core 0.10 +* Review and reduce log and other file sizes +* Add interactive mode to multichain-cli for Windows +* Port to Mac OS diff --git a/autogen.sh b/autogen.sh new file mode 100755 index 00000000..3e26a183 --- /dev/null +++ b/autogen.sh @@ -0,0 +1,9 @@ +#!/bin/sh +set -e +srcdir="$(dirname $0)" +cd "$srcdir" +if [ -z ${LIBTOOLIZE} ] && GLIBTOOLIZE="`which glibtoolize 2>/dev/null`"; then + LIBTOOLIZE="${GLIBTOOLIZE}" + export LIBTOOLIZE +fi +autoreconf --install --force --warnings=all diff --git a/build-aux/m4/ax_boost_base.m4 b/build-aux/m4/ax_boost_base.m4 new file mode 100644 index 00000000..3f24d5dd --- /dev/null +++ b/build-aux/m4/ax_boost_base.m4 @@ -0,0 +1,281 @@ +# =========================================================================== +# http://www.gnu.org/software/autoconf-archive/ax_boost_base.html +# =========================================================================== +# +# SYNOPSIS +# +# AX_BOOST_BASE([MINIMUM-VERSION], [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND]) +# +# DESCRIPTION +# +# Test for the Boost C++ libraries of a particular version (or newer) +# +# If no path to the installed boost library is given the macro searchs +# under /usr, /usr/local, /opt and /opt/local and evaluates the +# $BOOST_ROOT environment variable. Further documentation is available at +# . +# +# This macro calls: +# +# AC_SUBST(BOOST_CPPFLAGS) / AC_SUBST(BOOST_LDFLAGS) +# +# And sets: +# +# HAVE_BOOST +# +# LICENSE +# +# Copyright (c) 2008 Thomas Porschberg +# Copyright (c) 2009 Peter Adolphs +# +# Copying and distribution of this file, with or without modification, are +# permitted in any medium without royalty provided the copyright notice +# and this notice are preserved. This file is offered as-is, without any +# warranty. + +#serial 23 + +AC_DEFUN([AX_BOOST_BASE], +[ +AC_ARG_WITH([boost], + [AS_HELP_STRING([--with-boost@<:@=ARG@:>@], + [use Boost library from a standard location (ARG=yes), + from the specified location (ARG=), + or disable it (ARG=no) + @<:@ARG=yes@:>@ ])], + [ + if test "$withval" = "no"; then + want_boost="no" + elif test "$withval" = "yes"; then + want_boost="yes" + ac_boost_path="" + else + want_boost="yes" + ac_boost_path="$withval" + fi + ], + [want_boost="yes"]) + + +AC_ARG_WITH([boost-libdir], + AS_HELP_STRING([--with-boost-libdir=LIB_DIR], + [Force given directory for boost libraries. Note that this will override library path detection, so use this parameter only if default library detection fails and you know exactly where your boost libraries are located.]), + [ + if test -d "$withval" + then + ac_boost_lib_path="$withval" + else + AC_MSG_ERROR(--with-boost-libdir expected directory name) + fi + ], + [ac_boost_lib_path=""] +) + +if test "x$want_boost" = "xyes"; then + boost_lib_version_req=ifelse([$1], ,1.20.0,$1) + boost_lib_version_req_shorten=`expr $boost_lib_version_req : '\([[0-9]]*\.[[0-9]]*\)'` + boost_lib_version_req_major=`expr $boost_lib_version_req : '\([[0-9]]*\)'` + boost_lib_version_req_minor=`expr $boost_lib_version_req : '[[0-9]]*\.\([[0-9]]*\)'` + boost_lib_version_req_sub_minor=`expr $boost_lib_version_req : '[[0-9]]*\.[[0-9]]*\.\([[0-9]]*\)'` + if test "x$boost_lib_version_req_sub_minor" = "x" ; then + boost_lib_version_req_sub_minor="0" + fi + WANT_BOOST_VERSION=`expr $boost_lib_version_req_major \* 100000 \+ $boost_lib_version_req_minor \* 100 \+ $boost_lib_version_req_sub_minor` + AC_MSG_CHECKING(for boostlib >= $boost_lib_version_req) + succeeded=no + + dnl On 64-bit systems check for system libraries in both lib64 and lib. + dnl The former is specified by FHS, but e.g. Debian does not adhere to + dnl this (as it rises problems for generic multi-arch support). + dnl The last entry in the list is chosen by default when no libraries + dnl are found, e.g. when only header-only libraries are installed! + libsubdirs="lib" + ax_arch=`uname -m` + case $ax_arch in + x86_64) + libsubdirs="lib64 libx32 lib lib64" + ;; + ppc64|s390x|sparc64|aarch64) + libsubdirs="lib64 lib lib64" + ;; + esac + + dnl allow for real multi-arch paths e.g. /usr/lib/x86_64-linux-gnu. Give + dnl them priority over the other paths since, if libs are found there, they + dnl are almost assuredly the ones desired. + AC_REQUIRE([AC_CANONICAL_HOST]) + libsubdirs="lib/${host_cpu}-${host_os} $libsubdirs" + + case ${host_cpu} in + i?86) + libsubdirs="lib/i386-${host_os} $libsubdirs" + ;; + esac + + dnl some arches may advertise a cpu type that doesn't line up with their + dnl prefix's cpu type. For example, uname may report armv7l while libs are + dnl installed to /usr/lib/arm-linux-gnueabihf. Try getting the compiler's + dnl value for an extra chance of finding the correct path. + libsubdirs="lib/`$CXX -dumpmachine 2>/dev/null` $libsubdirs" + + dnl first we check the system location for boost libraries + dnl this location ist chosen if boost libraries are installed with the --layout=system option + dnl or if you install boost with RPM + if test "$ac_boost_path" != ""; then + BOOST_CPPFLAGS="-I$ac_boost_path/include" + for ac_boost_path_tmp in $libsubdirs; do + if test -d "$ac_boost_path"/"$ac_boost_path_tmp" ; then + BOOST_LDFLAGS="-L$ac_boost_path/$ac_boost_path_tmp" + break + fi + done + elif test "$cross_compiling" != yes; then + for ac_boost_path_tmp in /usr /usr/local /opt /opt/local ; do + if test -d "$ac_boost_path_tmp/include/boost" && test -r "$ac_boost_path_tmp/include/boost"; then + for libsubdir in $libsubdirs ; do + if ls "$ac_boost_path_tmp/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi + done + BOOST_LDFLAGS="-L$ac_boost_path_tmp/$libsubdir" + BOOST_CPPFLAGS="-I$ac_boost_path_tmp/include" + break; + fi + done + fi + + dnl overwrite ld flags if we have required special directory with + dnl --with-boost-libdir parameter + if test "$ac_boost_lib_path" != ""; then + BOOST_LDFLAGS="-L$ac_boost_lib_path" + fi + + CPPFLAGS_SAVED="$CPPFLAGS" + CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" + export CPPFLAGS + + LDFLAGS_SAVED="$LDFLAGS" + LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" + export LDFLAGS + + AC_REQUIRE([AC_PROG_CXX]) + AC_LANG_PUSH(C++) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ + @%:@include + ]], [[ + #if BOOST_VERSION >= $WANT_BOOST_VERSION + // Everything is okay + #else + # error Boost version is too old + #endif + ]])],[ + AC_MSG_RESULT(yes) + succeeded=yes + found_system=yes + ],[: + ]) + AC_LANG_POP([C++]) + + + + dnl if we found no boost with system layout we search for boost libraries + dnl built and installed without the --layout=system option or for a staged(not installed) version + if test "x$succeeded" != "xyes"; then + _version=0 + if test "$ac_boost_path" != ""; then + if test -d "$ac_boost_path" && test -r "$ac_boost_path"; then + for i in `ls -d $ac_boost_path/include/boost-* 2>/dev/null`; do + _version_tmp=`echo $i | sed "s#$ac_boost_path##" | sed 's/\/include\/boost-//' | sed 's/_/./'` + V_CHECK=`expr $_version_tmp \> $_version` + if test "$V_CHECK" = "1" ; then + _version=$_version_tmp + fi + VERSION_UNDERSCORE=`echo $_version | sed 's/\./_/'` + BOOST_CPPFLAGS="-I$ac_boost_path/include/boost-$VERSION_UNDERSCORE" + done + fi + else + if test "$cross_compiling" != yes; then + for ac_boost_path in /usr /usr/local /opt /opt/local ; do + if test -d "$ac_boost_path" && test -r "$ac_boost_path"; then + for i in `ls -d $ac_boost_path/include/boost-* 2>/dev/null`; do + _version_tmp=`echo $i | sed "s#$ac_boost_path##" | sed 's/\/include\/boost-//' | sed 's/_/./'` + V_CHECK=`expr $_version_tmp \> $_version` + if test "$V_CHECK" = "1" ; then + _version=$_version_tmp + best_path=$ac_boost_path + fi + done + fi + done + + VERSION_UNDERSCORE=`echo $_version | sed 's/\./_/'` + BOOST_CPPFLAGS="-I$best_path/include/boost-$VERSION_UNDERSCORE" + if test "$ac_boost_lib_path" = ""; then + for libsubdir in $libsubdirs ; do + if ls "$best_path/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi + done + BOOST_LDFLAGS="-L$best_path/$libsubdir" + fi + fi + + if test "x$BOOST_ROOT" != "x"; then + for libsubdir in $libsubdirs ; do + if ls "$BOOST_ROOT/stage/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi + done + if test -d "$BOOST_ROOT" && test -r "$BOOST_ROOT" && test -d "$BOOST_ROOT/stage/$libsubdir" && test -r "$BOOST_ROOT/stage/$libsubdir"; then + version_dir=`expr //$BOOST_ROOT : '.*/\(.*\)'` + stage_version=`echo $version_dir | sed 's/boost_//' | sed 's/_/./g'` + stage_version_shorten=`expr $stage_version : '\([[0-9]]*\.[[0-9]]*\)'` + V_CHECK=`expr $stage_version_shorten \>\= $_version` + if test "$V_CHECK" = "1" -a "$ac_boost_lib_path" = "" ; then + AC_MSG_NOTICE(We will use a staged boost library from $BOOST_ROOT) + BOOST_CPPFLAGS="-I$BOOST_ROOT" + BOOST_LDFLAGS="-L$BOOST_ROOT/stage/$libsubdir" + fi + fi + fi + fi + + CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" + export CPPFLAGS + LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" + export LDFLAGS + + AC_LANG_PUSH(C++) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ + @%:@include + ]], [[ + #if BOOST_VERSION >= $WANT_BOOST_VERSION + // Everything is okay + #else + # error Boost version is too old + #endif + ]])],[ + AC_MSG_RESULT(yes) + succeeded=yes + found_system=yes + ],[: + ]) + AC_LANG_POP([C++]) + fi + + if test "$succeeded" != "yes" ; then + if test "$_version" = "0" ; then + AC_MSG_NOTICE([[We could not detect the boost libraries (version $boost_lib_version_req_shorten or higher). If you have a staged boost library (still not installed) please specify \$BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in . See http://randspringer.de/boost for more documentation.]]) + else + AC_MSG_NOTICE([Your boost libraries seems to old (version $_version).]) + fi + # execute ACTION-IF-NOT-FOUND (if present): + ifelse([$3], , :, [$3]) + else + AC_SUBST(BOOST_CPPFLAGS) + AC_SUBST(BOOST_LDFLAGS) + AC_DEFINE(HAVE_BOOST,,[define if the Boost library is available]) + # execute ACTION-IF-FOUND (if present): + ifelse([$2], , :, [$2]) + fi + + CPPFLAGS="$CPPFLAGS_SAVED" + LDFLAGS="$LDFLAGS_SAVED" +fi + +]) diff --git a/build-aux/m4/ax_boost_chrono.m4 b/build-aux/m4/ax_boost_chrono.m4 new file mode 100644 index 00000000..318ecea1 --- /dev/null +++ b/build-aux/m4/ax_boost_chrono.m4 @@ -0,0 +1,119 @@ +# =========================================================================== +# http://www.gnu.org/software/autoconf-archive/ax_boost_chrono.html +# =========================================================================== +# +# SYNOPSIS +# +# AX_BOOST_CHRONO +# +# DESCRIPTION +# +# Test for System library from the Boost C++ libraries. The macro requires +# a preceding call to AX_BOOST_BASE. Further documentation is available at +# . +# +# This macro calls: +# +# AC_SUBST(BOOST_CHRONO_LIB) +# +# And sets: +# +# HAVE_BOOST_CHRONO +# +# LICENSE +# +# Copyright (c) 2012 Xiyue Deng +# +# Copying and distribution of this file, with or without modification, are +# permitted in any medium without royalty provided the copyright notice +# and this notice are preserved. This file is offered as-is, without any +# warranty. + +#serial 1 + +AC_DEFUN([AX_BOOST_CHRONO], +[ + AC_ARG_WITH([boost-chrono], + AS_HELP_STRING([--with-boost-chrono@<:@=special-lib@:>@], + [use the Chrono library from boost - it is possible to specify a certain library for the linker + e.g. --with-boost-chrono=boost_chrono-gcc-mt ]), + [ + if test "$withval" = "no"; then + want_boost="no" + elif test "$withval" = "yes"; then + want_boost="yes" + ax_boost_user_chrono_lib="" + else + want_boost="yes" + ax_boost_user_chrono_lib="$withval" + fi + ], + [want_boost="yes"] + ) + + if test "x$want_boost" = "xyes"; then + AC_REQUIRE([AC_PROG_CC]) + AC_REQUIRE([AC_CANONICAL_BUILD]) + CPPFLAGS_SAVED="$CPPFLAGS" + CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" + export CPPFLAGS + + LDFLAGS_SAVED="$LDFLAGS" + LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" + export LDFLAGS + + AC_CACHE_CHECK(whether the Boost::Chrono library is available, + ax_cv_boost_chrono, + [AC_LANG_PUSH([C++]) + CXXFLAGS_SAVE=$CXXFLAGS + + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include ]], + [[boost::chrono::system_clock::time_point time;]])], + ax_cv_boost_chrono=yes, ax_cv_boost_chrono=no) + CXXFLAGS=$CXXFLAGS_SAVE + AC_LANG_POP([C++]) + ]) + if test "x$ax_cv_boost_chrono" = "xyes"; then + AC_SUBST(BOOST_CPPFLAGS) + + AC_DEFINE(HAVE_BOOST_CHRONO,,[define if the Boost::Chrono library is available]) + BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'` + + LDFLAGS_SAVE=$LDFLAGS + if test "x$ax_boost_user_chrono_lib" = "x"; then + ax_lib= + for libextension in `ls $BOOSTLIBDIR/libboost_chrono*.so* $BOOSTLIBDIR/libboost_chrono*.dylib* $BOOSTLIBDIR/libboost_chrono*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\(boost_chrono.*\)\.so.*$;\1;' -e 's;^lib\(boost_chrono.*\)\.dylib.*$;\1;' -e 's;^lib\(boost_chrono.*\)\.a.*$;\1;'` ; do + ax_lib=${libextension} + AC_CHECK_LIB($ax_lib, exit, + [BOOST_CHRONO_LIB="-l$ax_lib"; AC_SUBST(BOOST_CHRONO_LIB) link_chrono="yes"; break], + [link_chrono="no"]) + done + if test "x$link_chrono" != "xyes"; then + for libextension in `ls $BOOSTLIBDIR/boost_chrono*.dll* $BOOSTLIBDIR/boost_chrono*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\(boost_chrono.*\)\.dll.*$;\1;' -e 's;^\(boost_chrono.*\)\.a.*$;\1;'` ; do + ax_lib=${libextension} + AC_CHECK_LIB($ax_lib, exit, + [BOOST_CHRONO_LIB="-l$ax_lib"; AC_SUBST(BOOST_CHRONO_LIB) link_chrono="yes"; break], + [link_chrono="no"]) + done + fi + + else + for ax_lib in $ax_boost_user_chrono_lib boost_chrono-$ax_boost_user_chrono_lib; do + AC_CHECK_LIB($ax_lib, exit, + [BOOST_CHRONO_LIB="-l$ax_lib"; AC_SUBST(BOOST_CHRONO_LIB) link_chrono="yes"; break], + [link_chrono="no"]) + done + + fi + if test "x$ax_lib" = "x"; then + AC_MSG_ERROR(Could not find a version of the boost_chrono library!) + fi + if test "x$link_chrono" = "xno"; then + AC_MSG_ERROR(Could not link against $ax_lib !) + fi + fi + + CPPFLAGS="$CPPFLAGS_SAVED" + LDFLAGS="$LDFLAGS_SAVED" + fi +]) diff --git a/build-aux/m4/ax_boost_filesystem.m4 b/build-aux/m4/ax_boost_filesystem.m4 new file mode 100644 index 00000000..f5c9d564 --- /dev/null +++ b/build-aux/m4/ax_boost_filesystem.m4 @@ -0,0 +1,119 @@ +# =========================================================================== +# http://www.gnu.org/software/autoconf-archive/ax_boost_filesystem.html +# =========================================================================== +# +# SYNOPSIS +# +# AX_BOOST_FILESYSTEM +# +# DESCRIPTION +# +# Test for Filesystem library from the Boost C++ libraries. The macro +# requires a preceding call to AX_BOOST_BASE. Further documentation is +# available at . +# +# This macro calls: +# +# AC_SUBST(BOOST_FILESYSTEM_LIB) +# +# And sets: +# +# HAVE_BOOST_FILESYSTEM +# +# LICENSE +# +# Copyright (c) 2009 Thomas Porschberg +# Copyright (c) 2009 Michael Tindal +# Copyright (c) 2009 Roman Rybalko +# +# Copying and distribution of this file, with or without modification, are +# permitted in any medium without royalty provided the copyright notice +# and this notice are preserved. This file is offered as-is, without any +# warranty. + +#serial 26 + +AC_DEFUN([AX_BOOST_FILESYSTEM], +[ + AC_ARG_WITH([boost-filesystem], + AS_HELP_STRING([--with-boost-filesystem@<:@=special-lib@:>@], + [use the Filesystem library from boost - it is possible to specify a certain library for the linker + e.g. --with-boost-filesystem=boost_filesystem-gcc-mt ]), + [ + if test "$withval" = "no"; then + want_boost="no" + elif test "$withval" = "yes"; then + want_boost="yes" + ax_boost_user_filesystem_lib="" + else + want_boost="yes" + ax_boost_user_filesystem_lib="$withval" + fi + ], + [want_boost="yes"] + ) + + if test "x$want_boost" = "xyes"; then + AC_REQUIRE([AC_PROG_CC]) + CPPFLAGS_SAVED="$CPPFLAGS" + CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" + export CPPFLAGS + + LDFLAGS_SAVED="$LDFLAGS" + LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" + export LDFLAGS + + LIBS_SAVED=$LIBS + LIBS="$LIBS $BOOST_SYSTEM_LIB" + export LIBS + + AC_CACHE_CHECK(whether the Boost::Filesystem library is available, + ax_cv_boost_filesystem, + [AC_LANG_PUSH([C++]) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include ]], + [[using namespace boost::filesystem; + path my_path( "foo/bar/data.txt" ); + return 0;]])], + ax_cv_boost_filesystem=yes, ax_cv_boost_filesystem=no) + AC_LANG_POP([C++]) + ]) + if test "x$ax_cv_boost_filesystem" = "xyes"; then + AC_DEFINE(HAVE_BOOST_FILESYSTEM,,[define if the Boost::Filesystem library is available]) + BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'` + ax_lib= + if test "x$ax_boost_user_filesystem_lib" = "x"; then + for libextension in `ls -r $BOOSTLIBDIR/libboost_filesystem* 2>/dev/null | sed 's,.*/lib,,' | sed 's,\..*,,'` ; do + ax_lib=${libextension} + AC_CHECK_LIB($ax_lib, exit, + [BOOST_FILESYSTEM_LIB="-l$ax_lib"; AC_SUBST(BOOST_FILESYSTEM_LIB) link_filesystem="yes"; break], + [link_filesystem="no"]) + done + if test "x$link_filesystem" != "xyes"; then + for libextension in `ls -r $BOOSTLIBDIR/boost_filesystem* 2>/dev/null | sed 's,.*/,,' | sed -e 's,\..*,,'` ; do + ax_lib=${libextension} + AC_CHECK_LIB($ax_lib, exit, + [BOOST_FILESYSTEM_LIB="-l$ax_lib"; AC_SUBST(BOOST_FILESYSTEM_LIB) link_filesystem="yes"; break], + [link_filesystem="no"]) + done + fi + else + for ax_lib in $ax_boost_user_filesystem_lib boost_filesystem-$ax_boost_user_filesystem_lib; do + AC_CHECK_LIB($ax_lib, exit, + [BOOST_FILESYSTEM_LIB="-l$ax_lib"; AC_SUBST(BOOST_FILESYSTEM_LIB) link_filesystem="yes"; break], + [link_filesystem="no"]) + done + + fi + if test "x$ax_lib" = "x"; then + AC_MSG_ERROR(Could not find a version of the boost_filesystem library!) + fi + if test "x$link_filesystem" != "xyes"; then + AC_MSG_ERROR(Could not link against $ax_lib !) + fi + fi + + CPPFLAGS="$CPPFLAGS_SAVED" + LDFLAGS="$LDFLAGS_SAVED" + LIBS="$LIBS_SAVED" + fi +]) diff --git a/build-aux/m4/ax_boost_program_options.m4 b/build-aux/m4/ax_boost_program_options.m4 new file mode 100644 index 00000000..f5914418 --- /dev/null +++ b/build-aux/m4/ax_boost_program_options.m4 @@ -0,0 +1,109 @@ +# ============================================================================ +# http://www.gnu.org/software/autoconf-archive/ax_boost_program_options.html +# ============================================================================ +# +# SYNOPSIS +# +# AX_BOOST_PROGRAM_OPTIONS +# +# DESCRIPTION +# +# Test for program options library from the Boost C++ libraries. The macro +# requires a preceding call to AX_BOOST_BASE. Further documentation is +# available at . +# +# This macro calls: +# +# AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB) +# +# And sets: +# +# HAVE_BOOST_PROGRAM_OPTIONS +# +# LICENSE +# +# Copyright (c) 2009 Thomas Porschberg +# +# Copying and distribution of this file, with or without modification, are +# permitted in any medium without royalty provided the copyright notice +# and this notice are preserved. This file is offered as-is, without any +# warranty. + +#serial 22 + +AC_DEFUN([AX_BOOST_PROGRAM_OPTIONS], +[ + AC_ARG_WITH([boost-program-options], + AS_HELP_STRING([--with-boost-program-options@<:@=special-lib@:>@], + [use the program options library from boost - it is possible to specify a certain library for the linker + e.g. --with-boost-program-options=boost_program_options-gcc-mt-1_33_1 ]), + [ + if test "$withval" = "no"; then + want_boost="no" + elif test "$withval" = "yes"; then + want_boost="yes" + ax_boost_user_program_options_lib="" + else + want_boost="yes" + ax_boost_user_program_options_lib="$withval" + fi + ], + [want_boost="yes"] + ) + + if test "x$want_boost" = "xyes"; then + AC_REQUIRE([AC_PROG_CC]) + export want_boost + CPPFLAGS_SAVED="$CPPFLAGS" + CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" + export CPPFLAGS + LDFLAGS_SAVED="$LDFLAGS" + LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" + export LDFLAGS + AC_CACHE_CHECK([whether the Boost::Program_Options library is available], + ax_cv_boost_program_options, + [AC_LANG_PUSH(C++) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include + ]], + [[boost::program_options::options_description generic("Generic options"); + return 0;]])], + ax_cv_boost_program_options=yes, ax_cv_boost_program_options=no) + AC_LANG_POP([C++]) + ]) + if test "$ax_cv_boost_program_options" = yes; then + AC_DEFINE(HAVE_BOOST_PROGRAM_OPTIONS,,[define if the Boost::PROGRAM_OPTIONS library is available]) + BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'` + if test "x$ax_boost_user_program_options_lib" = "x"; then + ax_lib= + for libextension in `ls $BOOSTLIBDIR/libboost_program_options*.so* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\(boost_program_options.*\)\.so.*$;\1;'` `ls $BOOSTLIBDIR/libboost_program_options*.dylib* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\(boost_program_options.*\)\.dylib.*$;\1;'` `ls $BOOSTLIBDIR/libboost_program_options*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\(boost_program_options.*\)\.a.*$;\1;'` ; do + ax_lib=${libextension} + AC_CHECK_LIB($ax_lib, exit, + [BOOST_PROGRAM_OPTIONS_LIB="-l$ax_lib"; AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB) link_program_options="yes"; break], + [link_program_options="no"]) + done + if test "x$link_program_options" != "xyes"; then + for libextension in `ls $BOOSTLIBDIR/boost_program_options*.dll* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\(boost_program_options.*\)\.dll.*$;\1;'` `ls $BOOSTLIBDIR/boost_program_options*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\(boost_program_options.*\)\.a.*$;\1;'` ; do + ax_lib=${libextension} + AC_CHECK_LIB($ax_lib, exit, + [BOOST_PROGRAM_OPTIONS_LIB="-l$ax_lib"; AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB) link_program_options="yes"; break], + [link_program_options="no"]) + done + fi + else + for ax_lib in $ax_boost_user_program_options_lib boost_program_options-$ax_boost_user_program_options_lib; do + AC_CHECK_LIB($ax_lib, main, + [BOOST_PROGRAM_OPTIONS_LIB="-l$ax_lib"; AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB) link_program_options="yes"; break], + [link_program_options="no"]) + done + fi + if test "x$ax_lib" = "x"; then + AC_MSG_ERROR(Could not find a version of the boost_program_options library!) + fi + if test "x$link_program_options" != "xyes"; then + AC_MSG_ERROR([Could not link against [$ax_lib] !]) + fi + fi + CPPFLAGS="$CPPFLAGS_SAVED" + LDFLAGS="$LDFLAGS_SAVED" + fi +]) diff --git a/build-aux/m4/ax_boost_system.m4 b/build-aux/m4/ax_boost_system.m4 new file mode 100644 index 00000000..9c78280f --- /dev/null +++ b/build-aux/m4/ax_boost_system.m4 @@ -0,0 +1,121 @@ +# =========================================================================== +# http://www.gnu.org/software/autoconf-archive/ax_boost_system.html +# =========================================================================== +# +# SYNOPSIS +# +# AX_BOOST_SYSTEM +# +# DESCRIPTION +# +# Test for System library from the Boost C++ libraries. The macro requires +# a preceding call to AX_BOOST_BASE. Further documentation is available at +# . +# +# This macro calls: +# +# AC_SUBST(BOOST_SYSTEM_LIB) +# +# And sets: +# +# HAVE_BOOST_SYSTEM +# +# LICENSE +# +# Copyright (c) 2008 Thomas Porschberg +# Copyright (c) 2008 Michael Tindal +# Copyright (c) 2008 Daniel Casimiro +# +# Copying and distribution of this file, with or without modification, are +# permitted in any medium without royalty provided the copyright notice +# and this notice are preserved. This file is offered as-is, without any +# warranty. + +#serial 17 + +AC_DEFUN([AX_BOOST_SYSTEM], +[ + AC_ARG_WITH([boost-system], + AS_HELP_STRING([--with-boost-system@<:@=special-lib@:>@], + [use the System library from boost - it is possible to specify a certain library for the linker + e.g. --with-boost-system=boost_system-gcc-mt ]), + [ + if test "$withval" = "no"; then + want_boost="no" + elif test "$withval" = "yes"; then + want_boost="yes" + ax_boost_user_system_lib="" + else + want_boost="yes" + ax_boost_user_system_lib="$withval" + fi + ], + [want_boost="yes"] + ) + + if test "x$want_boost" = "xyes"; then + AC_REQUIRE([AC_PROG_CC]) + AC_REQUIRE([AC_CANONICAL_BUILD]) + CPPFLAGS_SAVED="$CPPFLAGS" + CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" + export CPPFLAGS + + LDFLAGS_SAVED="$LDFLAGS" + LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" + export LDFLAGS + + AC_CACHE_CHECK(whether the Boost::System library is available, + ax_cv_boost_system, + [AC_LANG_PUSH([C++]) + CXXFLAGS_SAVE=$CXXFLAGS + + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include ]], + [[boost::system::system_category]])], + ax_cv_boost_system=yes, ax_cv_boost_system=no) + CXXFLAGS=$CXXFLAGS_SAVE + AC_LANG_POP([C++]) + ]) + if test "x$ax_cv_boost_system" = "xyes"; then + AC_SUBST(BOOST_CPPFLAGS) + + AC_DEFINE(HAVE_BOOST_SYSTEM,,[define if the Boost::System library is available]) + BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'` + + LDFLAGS_SAVE=$LDFLAGS + if test "x$ax_boost_user_system_lib" = "x"; then + ax_lib= + for libextension in `ls -r $BOOSTLIBDIR/libboost_system* 2>/dev/null | sed 's,.*/lib,,' | sed 's,\..*,,'` ; do + ax_lib=${libextension} + AC_CHECK_LIB($ax_lib, exit, + [BOOST_SYSTEM_LIB="-l$ax_lib"; AC_SUBST(BOOST_SYSTEM_LIB) link_system="yes"; break], + [link_system="no"]) + done + if test "x$link_system" != "xyes"; then + for libextension in `ls -r $BOOSTLIBDIR/boost_system* 2>/dev/null | sed 's,.*/,,' | sed -e 's,\..*,,'` ; do + ax_lib=${libextension} + AC_CHECK_LIB($ax_lib, exit, + [BOOST_SYSTEM_LIB="-l$ax_lib"; AC_SUBST(BOOST_SYSTEM_LIB) link_system="yes"; break], + [link_system="no"]) + done + fi + + else + for ax_lib in $ax_boost_user_system_lib boost_system-$ax_boost_user_system_lib; do + AC_CHECK_LIB($ax_lib, exit, + [BOOST_SYSTEM_LIB="-l$ax_lib"; AC_SUBST(BOOST_SYSTEM_LIB) link_system="yes"; break], + [link_system="no"]) + done + + fi + if test "x$ax_lib" = "x"; then + AC_MSG_ERROR(Could not find a version of the boost_system library!) + fi + if test "x$link_system" = "xno"; then + AC_MSG_ERROR(Could not link against $ax_lib !) + fi + fi + + CPPFLAGS="$CPPFLAGS_SAVED" + LDFLAGS="$LDFLAGS_SAVED" + fi +]) diff --git a/build-aux/m4/ax_boost_thread.m4 b/build-aux/m4/ax_boost_thread.m4 new file mode 100644 index 00000000..9f0bd0b2 --- /dev/null +++ b/build-aux/m4/ax_boost_thread.m4 @@ -0,0 +1,150 @@ +# =========================================================================== +# http://www.gnu.org/software/autoconf-archive/ax_boost_thread.html +# =========================================================================== +# +# SYNOPSIS +# +# AX_BOOST_THREAD +# +# DESCRIPTION +# +# Test for Thread library from the Boost C++ libraries. The macro requires +# a preceding call to AX_BOOST_BASE. Further documentation is available at +# . +# +# This macro calls: +# +# AC_SUBST(BOOST_THREAD_LIB) +# +# And sets: +# +# HAVE_BOOST_THREAD +# +# LICENSE +# +# Copyright (c) 2009 Thomas Porschberg +# Copyright (c) 2009 Michael Tindal +# +# Copying and distribution of this file, with or without modification, are +# permitted in any medium without royalty provided the copyright notice +# and this notice are preserved. This file is offered as-is, without any +# warranty. + +#serial 27 + +AC_DEFUN([AX_BOOST_THREAD], +[ + AC_ARG_WITH([boost-thread], + AS_HELP_STRING([--with-boost-thread@<:@=special-lib@:>@], + [use the Thread library from boost - it is possible to specify a certain library for the linker + e.g. --with-boost-thread=boost_thread-gcc-mt ]), + [ + if test "$withval" = "no"; then + want_boost="no" + elif test "$withval" = "yes"; then + want_boost="yes" + ax_boost_user_thread_lib="" + else + want_boost="yes" + ax_boost_user_thread_lib="$withval" + fi + ], + [want_boost="yes"] + ) + + if test "x$want_boost" = "xyes"; then + AC_REQUIRE([AC_PROG_CC]) + AC_REQUIRE([AC_CANONICAL_BUILD]) + CPPFLAGS_SAVED="$CPPFLAGS" + CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" + export CPPFLAGS + + LDFLAGS_SAVED="$LDFLAGS" + LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" + export LDFLAGS + + AC_CACHE_CHECK(whether the Boost::Thread library is available, + ax_cv_boost_thread, + [AC_LANG_PUSH([C++]) + CXXFLAGS_SAVE=$CXXFLAGS + + if test "x$host_os" = "xsolaris" ; then + CXXFLAGS="-pthreads $CXXFLAGS" + elif test "x$host_os" = "xmingw32" ; then + CXXFLAGS="-mthreads $CXXFLAGS" + else + CXXFLAGS="-pthread $CXXFLAGS" + fi + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include ]], + [[boost::thread_group thrds; + return 0;]])], + ax_cv_boost_thread=yes, ax_cv_boost_thread=no) + CXXFLAGS=$CXXFLAGS_SAVE + AC_LANG_POP([C++]) + ]) + if test "x$ax_cv_boost_thread" = "xyes"; then + if test "x$host_os" = "xsolaris" ; then + BOOST_CPPFLAGS="-pthreads $BOOST_CPPFLAGS" + elif test "x$host_os" = "xmingw32" ; then + BOOST_CPPFLAGS="-mthreads $BOOST_CPPFLAGS" + else + BOOST_CPPFLAGS="-pthread $BOOST_CPPFLAGS" + fi + + AC_SUBST(BOOST_CPPFLAGS) + + AC_DEFINE(HAVE_BOOST_THREAD,,[define if the Boost::Thread library is available]) + BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'` + + LDFLAGS_SAVE=$LDFLAGS + case "x$host_os" in + *bsd* ) + LDFLAGS="-pthread $LDFLAGS" + break; + ;; + esac + if test "x$ax_boost_user_thread_lib" = "x"; then + ax_lib= + for libextension in `ls -r $BOOSTLIBDIR/libboost_thread* 2>/dev/null | sed 's,.*/lib,,' | sed 's,\..*,,'`; do + ax_lib=${libextension} + AC_CHECK_LIB($ax_lib, exit, + [BOOST_THREAD_LIB="-l$ax_lib"; AC_SUBST(BOOST_THREAD_LIB) link_thread="yes"; break], + [link_thread="no"]) + done + if test "x$link_thread" != "xyes"; then + for libextension in `ls -r $BOOSTLIBDIR/boost_thread* 2>/dev/null | sed 's,.*/,,' | sed 's,\..*,,'`; do + ax_lib=${libextension} + AC_CHECK_LIB($ax_lib, exit, + [BOOST_THREAD_LIB="-l$ax_lib"; AC_SUBST(BOOST_THREAD_LIB) link_thread="yes"; break], + [link_thread="no"]) + done + fi + + else + for ax_lib in $ax_boost_user_thread_lib boost_thread-$ax_boost_user_thread_lib; do + AC_CHECK_LIB($ax_lib, exit, + [BOOST_THREAD_LIB="-l$ax_lib"; AC_SUBST(BOOST_THREAD_LIB) link_thread="yes"; break], + [link_thread="no"]) + done + + fi + if test "x$ax_lib" = "x"; then + AC_MSG_ERROR(Could not find a version of the boost_thread library!) + fi + if test "x$link_thread" = "xno"; then + AC_MSG_ERROR(Could not link against $ax_lib !) + else + case "x$host_os" in + *bsd* ) + BOOST_LDFLAGS="-pthread $BOOST_LDFLAGS" + break; + ;; + esac + + fi + fi + + CPPFLAGS="$CPPFLAGS_SAVED" + LDFLAGS="$LDFLAGS_SAVED" + fi +]) diff --git a/build-aux/m4/ax_boost_unit_test_framework.m4 b/build-aux/m4/ax_boost_unit_test_framework.m4 new file mode 100644 index 00000000..4efd1e2f --- /dev/null +++ b/build-aux/m4/ax_boost_unit_test_framework.m4 @@ -0,0 +1,138 @@ +# ================================================================================ +# http://www.gnu.org/software/autoconf-archive/ax_boost_unit_test_framework.html +# ================================================================================ +# +# SYNOPSIS +# +# AX_BOOST_UNIT_TEST_FRAMEWORK +# +# DESCRIPTION +# +# Test for Unit_Test_Framework library from the Boost C++ libraries. The +# macro requires a preceding call to AX_BOOST_BASE. Further documentation +# is available at . +# +# This macro calls: +# +# AC_SUBST(BOOST_UNIT_TEST_FRAMEWORK_LIB) +# +# And sets: +# +# HAVE_BOOST_UNIT_TEST_FRAMEWORK +# +# LICENSE +# +# Copyright (c) 2008 Thomas Porschberg +# +# Copying and distribution of this file, with or without modification, are +# permitted in any medium without royalty provided the copyright notice +# and this notice are preserved. This file is offered as-is, without any +# warranty. + +#serial 19 + +AC_DEFUN([AX_BOOST_UNIT_TEST_FRAMEWORK], +[ + AC_ARG_WITH([boost-unit-test-framework], + AS_HELP_STRING([--with-boost-unit-test-framework@<:@=special-lib@:>@], + [use the Unit_Test_Framework library from boost - it is possible to specify a certain library for the linker + e.g. --with-boost-unit-test-framework=boost_unit_test_framework-gcc ]), + [ + if test "$withval" = "no"; then + want_boost="no" + elif test "$withval" = "yes"; then + want_boost="yes" + ax_boost_user_unit_test_framework_lib="" + else + want_boost="yes" + ax_boost_user_unit_test_framework_lib="$withval" + fi + ], + [want_boost="yes"] + ) + + if test "x$want_boost" = "xyes"; then + AC_REQUIRE([AC_PROG_CC]) + CPPFLAGS_SAVED="$CPPFLAGS" + CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" + export CPPFLAGS + + LDFLAGS_SAVED="$LDFLAGS" + LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" + export LDFLAGS + + AC_CACHE_CHECK(whether the Boost::Unit_Test_Framework library is available, + ax_cv_boost_unit_test_framework, + [AC_LANG_PUSH([C++]) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include ]], + [[using boost::unit_test::test_suite; + test_suite* test= BOOST_TEST_SUITE( "Unit test example 1" ); return 0;]])], + ax_cv_boost_unit_test_framework=yes, ax_cv_boost_unit_test_framework=no) + AC_LANG_POP([C++]) + ]) + if test "x$ax_cv_boost_unit_test_framework" = "xyes"; then + AC_DEFINE(HAVE_BOOST_UNIT_TEST_FRAMEWORK,,[define if the Boost::Unit_Test_Framework library is available]) + BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'` + + if test "x$ax_boost_user_unit_test_framework_lib" = "x"; then + saved_ldflags="${LDFLAGS}" + ax_lib= + for monitor_library in `ls $BOOSTLIBDIR/libboost_unit_test_framework*.so* $BOOSTLIBDIR/libboost_unit_test_framework*.dylib* $BOOSTLIBDIR/libboost_unit_test_framework*.a* 2>/dev/null` ; do + if test -r $monitor_library ; then + libextension=`echo $monitor_library | sed 's,.*/,,' | sed -e 's;^lib\(boost_unit_test_framework.*\)\.so.*$;\1;' -e 's;^lib\(boost_unit_test_framework.*\)\.dylib.*$;\1;' -e 's;^lib\(boost_unit_test_framework.*\)\.a.*$;\1;'` + ax_lib=${libextension} + link_unit_test_framework="yes" + else + link_unit_test_framework="no" + fi + + if test "x$link_unit_test_framework" = "xyes"; then + BOOST_UNIT_TEST_FRAMEWORK_LIB="-l$ax_lib" + AC_SUBST(BOOST_UNIT_TEST_FRAMEWORK_LIB) + break + fi + done + if test "x$link_unit_test_framework" != "xyes"; then + for libextension in `ls $BOOSTLIBDIR/boost_unit_test_framework*.dll* $BOOSTLIBDIR/boost_unit_test_framework*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\(boost_unit_test_framework.*\)\.dll.*$;\1;' -e 's;^\(boost_unit_test_framework.*\)\.a.*$;\1;'` ; do + ax_lib=${libextension} + AC_CHECK_LIB($ax_lib, exit, + [BOOST_UNIT_TEST_FRAMEWORK_LIB="-l$ax_lib"; AC_SUBST(BOOST_UNIT_TEST_FRAMEWORK_LIB) link_unit_test_framework="yes"; break], + [link_unit_test_framework="no"]) + done + fi + else + link_unit_test_framework="no" + saved_ldflags="${LDFLAGS}" + for ax_lib in boost_unit_test_framework-$ax_boost_user_unit_test_framework_lib $ax_boost_user_unit_test_framework_lib ; do + if test "x$link_unit_test_framework" = "xyes"; then + break; + fi + for unittest_library in `ls $BOOSTLIBDIR/lib${ax_lib}.so* $BOOSTLIBDIR/lib${ax_lib}.a* 2>/dev/null` ; do + if test -r $unittest_library ; then + libextension=`echo $unittest_library | sed 's,.*/,,' | sed -e 's;^lib\(boost_unit_test_framework.*\)\.so.*$;\1;' -e 's;^lib\(boost_unit_test_framework.*\)\.a*$;\1;'` + ax_lib=${libextension} + link_unit_test_framework="yes" + else + link_unit_test_framework="no" + fi + + if test "x$link_unit_test_framework" = "xyes"; then + BOOST_UNIT_TEST_FRAMEWORK_LIB="-l$ax_lib" + AC_SUBST(BOOST_UNIT_TEST_FRAMEWORK_LIB) + break + fi + done + done + fi + if test "x$ax_lib" = "x"; then + AC_MSG_ERROR(Could not find a version of the boost_unit_test_framework library!) + fi + if test "x$link_unit_test_framework" != "xyes"; then + AC_MSG_ERROR(Could not link against $ax_lib !) + fi + fi + + CPPFLAGS="$CPPFLAGS_SAVED" + LDFLAGS="$LDFLAGS_SAVED" + fi +]) diff --git a/build-aux/m4/ax_check_compile_flag.m4 b/build-aux/m4/ax_check_compile_flag.m4 new file mode 100644 index 00000000..c3a8d695 --- /dev/null +++ b/build-aux/m4/ax_check_compile_flag.m4 @@ -0,0 +1,72 @@ +# =========================================================================== +# http://www.gnu.org/software/autoconf-archive/ax_check_compile_flag.html +# =========================================================================== +# +# SYNOPSIS +# +# AX_CHECK_COMPILE_FLAG(FLAG, [ACTION-SUCCESS], [ACTION-FAILURE], [EXTRA-FLAGS]) +# +# DESCRIPTION +# +# Check whether the given FLAG works with the current language's compiler +# or gives an error. (Warnings, however, are ignored) +# +# ACTION-SUCCESS/ACTION-FAILURE are shell commands to execute on +# success/failure. +# +# If EXTRA-FLAGS is defined, it is added to the current language's default +# flags (e.g. CFLAGS) when the check is done. The check is thus made with +# the flags: "CFLAGS EXTRA-FLAGS FLAG". This can for example be used to +# force the compiler to issue an error when a bad flag is given. +# +# NOTE: Implementation based on AX_CFLAGS_GCC_OPTION. Please keep this +# macro in sync with AX_CHECK_{PREPROC,LINK}_FLAG. +# +# LICENSE +# +# Copyright (c) 2008 Guido U. Draheim +# Copyright (c) 2011 Maarten Bosmans +# +# This program is free software: you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by the +# Free Software Foundation, either version 3 of the License, or (at your +# option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program. If not, see . +# +# As a special exception, the respective Autoconf Macro's copyright owner +# gives unlimited permission to copy, distribute and modify the configure +# scripts that are the output of Autoconf when processing the Macro. You +# need not follow the terms of the GNU General Public License when using +# or distributing such scripts, even though portions of the text of the +# Macro appear in them. The GNU General Public License (GPL) does govern +# all other use of the material that constitutes the Autoconf Macro. +# +# This special exception to the GPL applies to versions of the Autoconf +# Macro released by the Autoconf Archive. When you make and distribute a +# modified version of the Autoconf Macro, you may extend this special +# exception to the GPL to apply to your modified version as well. + +#serial 2 + +AC_DEFUN([AX_CHECK_COMPILE_FLAG], +[AC_PREREQ(2.59)dnl for _AC_LANG_PREFIX +AS_VAR_PUSHDEF([CACHEVAR],[ax_cv_check_[]_AC_LANG_ABBREV[]flags_$4_$1])dnl +AC_CACHE_CHECK([whether _AC_LANG compiler accepts $1], CACHEVAR, [ + ax_check_save_flags=$[]_AC_LANG_PREFIX[]FLAGS + _AC_LANG_PREFIX[]FLAGS="$[]_AC_LANG_PREFIX[]FLAGS $4 $1" + AC_COMPILE_IFELSE([AC_LANG_PROGRAM()], + [AS_VAR_SET(CACHEVAR,[yes])], + [AS_VAR_SET(CACHEVAR,[no])]) + _AC_LANG_PREFIX[]FLAGS=$ax_check_save_flags]) +AS_IF([test x"AS_VAR_GET(CACHEVAR)" = xyes], + [m4_default([$2], :)], + [m4_default([$3], :)]) +AS_VAR_POPDEF([CACHEVAR])dnl +])dnl AX_CHECK_COMPILE_FLAGS diff --git a/build-aux/m4/ax_check_link_flag.m4 b/build-aux/m4/ax_check_link_flag.m4 new file mode 100644 index 00000000..e2d0d363 --- /dev/null +++ b/build-aux/m4/ax_check_link_flag.m4 @@ -0,0 +1,71 @@ +# =========================================================================== +# http://www.gnu.org/software/autoconf-archive/ax_check_link_flag.html +# =========================================================================== +# +# SYNOPSIS +# +# AX_CHECK_LINK_FLAG(FLAG, [ACTION-SUCCESS], [ACTION-FAILURE], [EXTRA-FLAGS]) +# +# DESCRIPTION +# +# Check whether the given FLAG works with the linker or gives an error. +# (Warnings, however, are ignored) +# +# ACTION-SUCCESS/ACTION-FAILURE are shell commands to execute on +# success/failure. +# +# If EXTRA-FLAGS is defined, it is added to the linker's default flags +# when the check is done. The check is thus made with the flags: "LDFLAGS +# EXTRA-FLAGS FLAG". This can for example be used to force the linker to +# issue an error when a bad flag is given. +# +# NOTE: Implementation based on AX_CFLAGS_GCC_OPTION. Please keep this +# macro in sync with AX_CHECK_{PREPROC,COMPILE}_FLAG. +# +# LICENSE +# +# Copyright (c) 2008 Guido U. Draheim +# Copyright (c) 2011 Maarten Bosmans +# +# This program is free software: you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by the +# Free Software Foundation, either version 3 of the License, or (at your +# option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program. If not, see . +# +# As a special exception, the respective Autoconf Macro's copyright owner +# gives unlimited permission to copy, distribute and modify the configure +# scripts that are the output of Autoconf when processing the Macro. You +# need not follow the terms of the GNU General Public License when using +# or distributing such scripts, even though portions of the text of the +# Macro appear in them. The GNU General Public License (GPL) does govern +# all other use of the material that constitutes the Autoconf Macro. +# +# This special exception to the GPL applies to versions of the Autoconf +# Macro released by the Autoconf Archive. When you make and distribute a +# modified version of the Autoconf Macro, you may extend this special +# exception to the GPL to apply to your modified version as well. + +#serial 2 + +AC_DEFUN([AX_CHECK_LINK_FLAG], +[AS_VAR_PUSHDEF([CACHEVAR],[ax_cv_check_ldflags_$4_$1])dnl +AC_CACHE_CHECK([whether the linker accepts $1], CACHEVAR, [ + ax_check_save_flags=$LDFLAGS + LDFLAGS="$LDFLAGS $4 $1" + AC_LINK_IFELSE([AC_LANG_PROGRAM()], + [AS_VAR_SET(CACHEVAR,[yes])], + [AS_VAR_SET(CACHEVAR,[no])]) + LDFLAGS=$ax_check_save_flags]) +AS_IF([test x"AS_VAR_GET(CACHEVAR)" = xyes], + [m4_default([$2], :)], + [m4_default([$3], :)]) +AS_VAR_POPDEF([CACHEVAR])dnl +])dnl AX_CHECK_LINK_FLAGS diff --git a/build-aux/m4/ax_check_preproc_flag.m4 b/build-aux/m4/ax_check_preproc_flag.m4 new file mode 100644 index 00000000..b1cfef6b --- /dev/null +++ b/build-aux/m4/ax_check_preproc_flag.m4 @@ -0,0 +1,72 @@ +# =========================================================================== +# http://www.gnu.org/software/autoconf-archive/ax_check_preproc_flag.html +# =========================================================================== +# +# SYNOPSIS +# +# AX_CHECK_PREPROC_FLAG(FLAG, [ACTION-SUCCESS], [ACTION-FAILURE], [EXTRA-FLAGS]) +# +# DESCRIPTION +# +# Check whether the given FLAG works with the current language's +# preprocessor or gives an error. (Warnings, however, are ignored) +# +# ACTION-SUCCESS/ACTION-FAILURE are shell commands to execute on +# success/failure. +# +# If EXTRA-FLAGS is defined, it is added to the preprocessor's default +# flags when the check is done. The check is thus made with the flags: +# "CPPFLAGS EXTRA-FLAGS FLAG". This can for example be used to force the +# preprocessor to issue an error when a bad flag is given. +# +# NOTE: Implementation based on AX_CFLAGS_GCC_OPTION. Please keep this +# macro in sync with AX_CHECK_{COMPILE,LINK}_FLAG. +# +# LICENSE +# +# Copyright (c) 2008 Guido U. Draheim +# Copyright (c) 2011 Maarten Bosmans +# +# This program is free software: you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by the +# Free Software Foundation, either version 3 of the License, or (at your +# option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program. If not, see . +# +# As a special exception, the respective Autoconf Macro's copyright owner +# gives unlimited permission to copy, distribute and modify the configure +# scripts that are the output of Autoconf when processing the Macro. You +# need not follow the terms of the GNU General Public License when using +# or distributing such scripts, even though portions of the text of the +# Macro appear in them. The GNU General Public License (GPL) does govern +# all other use of the material that constitutes the Autoconf Macro. +# +# This special exception to the GPL applies to versions of the Autoconf +# Macro released by the Autoconf Archive. When you make and distribute a +# modified version of the Autoconf Macro, you may extend this special +# exception to the GPL to apply to your modified version as well. + +#serial 2 + +AC_DEFUN([AX_CHECK_PREPROC_FLAG], +[AC_PREREQ(2.59)dnl for _AC_LANG_PREFIX +AS_VAR_PUSHDEF([CACHEVAR],[ax_cv_check_[]_AC_LANG_ABBREV[]cppflags_$4_$1])dnl +AC_CACHE_CHECK([whether _AC_LANG preprocessor accepts $1], CACHEVAR, [ + ax_check_save_flags=$CPPFLAGS + CPPFLAGS="$CPPFLAGS $4 $1" + AC_PREPROC_IFELSE([AC_LANG_PROGRAM()], + [AS_VAR_SET(CACHEVAR,[yes])], + [AS_VAR_SET(CACHEVAR,[no])]) + CPPFLAGS=$ax_check_save_flags]) +AS_IF([test x"AS_VAR_GET(CACHEVAR)" = xyes], + [m4_default([$2], :)], + [m4_default([$3], :)]) +AS_VAR_POPDEF([CACHEVAR])dnl +])dnl AX_CHECK_PREPROC_FLAGS diff --git a/build-aux/m4/ax_gcc_func_attribute.m4 b/build-aux/m4/ax_gcc_func_attribute.m4 new file mode 100644 index 00000000..275ca63a --- /dev/null +++ b/build-aux/m4/ax_gcc_func_attribute.m4 @@ -0,0 +1,217 @@ +# =========================================================================== +# http://www.gnu.org/software/autoconf-archive/ax_gcc_func_attribute.html +# =========================================================================== +# +# SYNOPSIS +# +# AX_GCC_FUNC_ATTRIBUTE(ATTRIBUTE) +# +# DESCRIPTION +# +# This macro checks if the compiler supports one of GCC's function +# attributes; many other compilers also provide function attributes with +# the same syntax. Compiler warnings are used to detect supported +# attributes as unsupported ones are ignored by default so quieting +# warnings when using this macro will yield false positives. +# +# The ATTRIBUTE parameter holds the name of the attribute to be checked. +# +# If ATTRIBUTE is supported define HAVE_FUNC_ATTRIBUTE_. +# +# The macro caches its result in the ax_cv_have_func_attribute_ +# variable. +# +# The macro currently supports the following function attributes: +# +# alias +# aligned +# alloc_size +# always_inline +# artificial +# cold +# const +# constructor +# deprecated +# destructor +# dllexport +# dllimport +# error +# externally_visible +# flatten +# format +# format_arg +# gnu_inline +# hot +# ifunc +# leaf +# malloc +# noclone +# noinline +# nonnull +# noreturn +# nothrow +# optimize +# pure +# unused +# used +# visibility +# warning +# warn_unused_result +# weak +# weakref +# +# Unsuppored function attributes will be tested with a prototype returning +# an int and not accepting any arguments and the result of the check might +# be wrong or meaningless so use with care. +# +# LICENSE +# +# Copyright (c) 2013 Gabriele Svelto +# +# Copying and distribution of this file, with or without modification, are +# permitted in any medium without royalty provided the copyright notice +# and this notice are preserved. This file is offered as-is, without any +# warranty. + +#serial 2 + +AC_DEFUN([AX_GCC_FUNC_ATTRIBUTE], [ + AS_VAR_PUSHDEF([ac_var], [ax_cv_have_func_attribute_$1]) + + AC_CACHE_CHECK([for __attribute__(($1))], [ac_var], [ + AC_LINK_IFELSE([AC_LANG_PROGRAM([ + m4_case([$1], + [alias], [ + int foo( void ) { return 0; } + int bar( void ) __attribute__(($1("foo"))); + ], + [aligned], [ + int foo( void ) __attribute__(($1(32))); + ], + [alloc_size], [ + void *foo(int a) __attribute__(($1(1))); + ], + [always_inline], [ + inline __attribute__(($1)) int foo( void ) { return 0; } + ], + [artificial], [ + inline __attribute__(($1)) int foo( void ) { return 0; } + ], + [cold], [ + int foo( void ) __attribute__(($1)); + ], + [const], [ + int foo( void ) __attribute__(($1)); + ], + [constructor], [ + int foo( void ) __attribute__(($1)); + ], + [deprecated], [ + int foo( void ) __attribute__(($1(""))); + ], + [destructor], [ + int foo( void ) __attribute__(($1)); + ], + [dllexport], [ + __attribute__(($1)) int foo( void ) { return 0; } + ], + [dllimport], [ + int foo( void ) __attribute__(($1)); + ], + [error], [ + int foo( void ) __attribute__(($1(""))); + ], + [externally_visible], [ + int foo( void ) __attribute__(($1)); + ], + [flatten], [ + int foo( void ) __attribute__(($1)); + ], + [format], [ + int foo(const char *p, ...) __attribute__(($1(printf, 1, 2))); + ], + [format_arg], [ + char *foo(const char *p) __attribute__(($1(1))); + ], + [gnu_inline], [ + inline __attribute__(($1)) int foo( void ) { return 0; } + ], + [hot], [ + int foo( void ) __attribute__(($1)); + ], + [ifunc], [ + int my_foo( void ) { return 0; } + static int (*resolve_foo(void))(void) { return my_foo; } + int foo( void ) __attribute__(($1("resolve_foo"))); + ], + [leaf], [ + __attribute__(($1)) int foo( void ) { return 0; } + ], + [malloc], [ + void *foo( void ) __attribute__(($1)); + ], + [noclone], [ + int foo( void ) __attribute__(($1)); + ], + [noinline], [ + __attribute__(($1)) int foo( void ) { return 0; } + ], + [nonnull], [ + int foo(char *p) __attribute__(($1(1))); + ], + [noreturn], [ + void foo( void ) __attribute__(($1)); + ], + [nothrow], [ + int foo( void ) __attribute__(($1)); + ], + [optimize], [ + __attribute__(($1(3))) int foo( void ) { return 0; } + ], + [pure], [ + int foo( void ) __attribute__(($1)); + ], + [unused], [ + int foo( void ) __attribute__(($1)); + ], + [used], [ + int foo( void ) __attribute__(($1)); + ], + [visibility], [ + int foo_def( void ) __attribute__(($1("default"))); + int foo_hid( void ) __attribute__(($1("hidden"))); + ], + [warning], [ + int foo( void ) __attribute__(($1(""))); + ], + [warn_unused_result], [ + int foo( void ) __attribute__(($1)); + ], + [weak], [ + int foo( void ) __attribute__(($1)); + ], + [weakref], [ + static int foo( void ) { return 0; } + static int bar( void ) __attribute__(($1("foo"))); + ], + [ + m4_warn([syntax], [Unsupported attribute $1, the test may fail]) + int foo( void ) __attribute__(($1)); + ] + )], []) + ], + dnl GCC doesn't exit with an error if an unknown attribute is + dnl provided but only outputs a warning, so accept the attribute + dnl only if no warning were issued. + [AS_IF([test -s conftest.err], + [AS_VAR_SET([ac_var], [no])], + [AS_VAR_SET([ac_var], [yes])])], + [AS_VAR_SET([ac_var], [no])]) + ]) + + AS_IF([test yes = AS_VAR_GET([ac_var])], + [AC_DEFINE_UNQUOTED(AS_TR_CPP(HAVE_FUNC_ATTRIBUTE_$1), 1, + [Define to 1 if the system has the `$1' function attribute])], []) + + AS_VAR_POPDEF([ac_var]) +]) diff --git a/build-aux/m4/ax_pthread.m4 b/build-aux/m4/ax_pthread.m4 new file mode 100644 index 00000000..d383ad5c --- /dev/null +++ b/build-aux/m4/ax_pthread.m4 @@ -0,0 +1,332 @@ +# =========================================================================== +# http://www.gnu.org/software/autoconf-archive/ax_pthread.html +# =========================================================================== +# +# SYNOPSIS +# +# AX_PTHREAD([ACTION-IF-FOUND[, ACTION-IF-NOT-FOUND]]) +# +# DESCRIPTION +# +# This macro figures out how to build C programs using POSIX threads. It +# sets the PTHREAD_LIBS output variable to the threads library and linker +# flags, and the PTHREAD_CFLAGS output variable to any special C compiler +# flags that are needed. (The user can also force certain compiler +# flags/libs to be tested by setting these environment variables.) +# +# Also sets PTHREAD_CC to any special C compiler that is needed for +# multi-threaded programs (defaults to the value of CC otherwise). (This +# is necessary on AIX to use the special cc_r compiler alias.) +# +# NOTE: You are assumed to not only compile your program with these flags, +# but also link it with them as well. e.g. you should link with +# $PTHREAD_CC $CFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS +# +# If you are only building threads programs, you may wish to use these +# variables in your default LIBS, CFLAGS, and CC: +# +# LIBS="$PTHREAD_LIBS $LIBS" +# CFLAGS="$CFLAGS $PTHREAD_CFLAGS" +# CC="$PTHREAD_CC" +# +# In addition, if the PTHREAD_CREATE_JOINABLE thread-attribute constant +# has a nonstandard name, defines PTHREAD_CREATE_JOINABLE to that name +# (e.g. PTHREAD_CREATE_UNDETACHED on AIX). +# +# Also HAVE_PTHREAD_PRIO_INHERIT is defined if pthread is found and the +# PTHREAD_PRIO_INHERIT symbol is defined when compiling with +# PTHREAD_CFLAGS. +# +# ACTION-IF-FOUND is a list of shell commands to run if a threads library +# is found, and ACTION-IF-NOT-FOUND is a list of commands to run it if it +# is not found. If ACTION-IF-FOUND is not specified, the default action +# will define HAVE_PTHREAD. +# +# Please let the authors know if this macro fails on any platform, or if +# you have any other suggestions or comments. This macro was based on work +# by SGJ on autoconf scripts for FFTW (http://www.fftw.org/) (with help +# from M. Frigo), as well as ac_pthread and hb_pthread macros posted by +# Alejandro Forero Cuervo to the autoconf macro repository. We are also +# grateful for the helpful feedback of numerous users. +# +# Updated for Autoconf 2.68 by Daniel Richard G. +# +# LICENSE +# +# Copyright (c) 2008 Steven G. Johnson +# Copyright (c) 2011 Daniel Richard G. +# +# This program is free software: you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by the +# Free Software Foundation, either version 3 of the License, or (at your +# option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program. If not, see . +# +# As a special exception, the respective Autoconf Macro's copyright owner +# gives unlimited permission to copy, distribute and modify the configure +# scripts that are the output of Autoconf when processing the Macro. You +# need not follow the terms of the GNU General Public License when using +# or distributing such scripts, even though portions of the text of the +# Macro appear in them. The GNU General Public License (GPL) does govern +# all other use of the material that constitutes the Autoconf Macro. +# +# This special exception to the GPL applies to versions of the Autoconf +# Macro released by the Autoconf Archive. When you make and distribute a +# modified version of the Autoconf Macro, you may extend this special +# exception to the GPL to apply to your modified version as well. + +#serial 21 + +AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD]) +AC_DEFUN([AX_PTHREAD], [ +AC_REQUIRE([AC_CANONICAL_HOST]) +AC_LANG_PUSH([C]) +ax_pthread_ok=no + +# We used to check for pthread.h first, but this fails if pthread.h +# requires special compiler flags (e.g. on True64 or Sequent). +# It gets checked for in the link test anyway. + +# First of all, check if the user has set any of the PTHREAD_LIBS, +# etcetera environment variables, and if threads linking works using +# them: +if test x"$PTHREAD_LIBS$PTHREAD_CFLAGS" != x; then + save_CFLAGS="$CFLAGS" + CFLAGS="$CFLAGS $PTHREAD_CFLAGS" + save_LIBS="$LIBS" + LIBS="$PTHREAD_LIBS $LIBS" + AC_MSG_CHECKING([for pthread_join in LIBS=$PTHREAD_LIBS with CFLAGS=$PTHREAD_CFLAGS]) + AC_TRY_LINK_FUNC([pthread_join], [ax_pthread_ok=yes]) + AC_MSG_RESULT([$ax_pthread_ok]) + if test x"$ax_pthread_ok" = xno; then + PTHREAD_LIBS="" + PTHREAD_CFLAGS="" + fi + LIBS="$save_LIBS" + CFLAGS="$save_CFLAGS" +fi + +# We must check for the threads library under a number of different +# names; the ordering is very important because some systems +# (e.g. DEC) have both -lpthread and -lpthreads, where one of the +# libraries is broken (non-POSIX). + +# Create a list of thread flags to try. Items starting with a "-" are +# C compiler flags, and other items are library names, except for "none" +# which indicates that we try without any flags at all, and "pthread-config" +# which is a program returning the flags for the Pth emulation library. + +ax_pthread_flags="pthreads none -Kthread -kthread lthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config" + +# The ordering *is* (sometimes) important. Some notes on the +# individual items follow: + +# pthreads: AIX (must check this before -lpthread) +# none: in case threads are in libc; should be tried before -Kthread and +# other compiler flags to prevent continual compiler warnings +# -Kthread: Sequent (threads in libc, but -Kthread needed for pthread.h) +# -kthread: FreeBSD kernel threads (preferred to -pthread since SMP-able) +# lthread: LinuxThreads port on FreeBSD (also preferred to -pthread) +# -pthread: Linux/gcc (kernel threads), BSD/gcc (userland threads) +# -pthreads: Solaris/gcc +# -mthreads: Mingw32/gcc, Lynx/gcc +# -mt: Sun Workshop C (may only link SunOS threads [-lthread], but it +# doesn't hurt to check since this sometimes defines pthreads too; +# also defines -D_REENTRANT) +# ... -mt is also the pthreads flag for HP/aCC +# pthread: Linux, etcetera +# --thread-safe: KAI C++ +# pthread-config: use pthread-config program (for GNU Pth library) + +case ${host_os} in + solaris*) + + # On Solaris (at least, for some versions), libc contains stubbed + # (non-functional) versions of the pthreads routines, so link-based + # tests will erroneously succeed. (We need to link with -pthreads/-mt/ + # -lpthread.) (The stubs are missing pthread_cleanup_push, or rather + # a function called by this macro, so we could check for that, but + # who knows whether they'll stub that too in a future libc.) So, + # we'll just look for -pthreads and -lpthread first: + + ax_pthread_flags="-pthreads pthread -mt -pthread $ax_pthread_flags" + ;; + + darwin*) + ax_pthread_flags="-pthread $ax_pthread_flags" + ;; +esac + +# Clang doesn't consider unrecognized options an error unless we specify +# -Werror. We throw in some extra Clang-specific options to ensure that +# this doesn't happen for GCC, which also accepts -Werror. + +AC_MSG_CHECKING([if compiler needs -Werror to reject unknown flags]) +save_CFLAGS="$CFLAGS" +ax_pthread_extra_flags="-Werror" +CFLAGS="$CFLAGS $ax_pthread_extra_flags -Wunknown-warning-option -Wsizeof-array-argument" +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([int foo(void);],[foo()])], + [AC_MSG_RESULT([yes])], + [ax_pthread_extra_flags= + AC_MSG_RESULT([no])]) +CFLAGS="$save_CFLAGS" + +if test x"$ax_pthread_ok" = xno; then +for flag in $ax_pthread_flags; do + + case $flag in + none) + AC_MSG_CHECKING([whether pthreads work without any flags]) + ;; + + -*) + AC_MSG_CHECKING([whether pthreads work with $flag]) + PTHREAD_CFLAGS="$flag" + ;; + + pthread-config) + AC_CHECK_PROG([ax_pthread_config], [pthread-config], [yes], [no]) + if test x"$ax_pthread_config" = xno; then continue; fi + PTHREAD_CFLAGS="`pthread-config --cflags`" + PTHREAD_LIBS="`pthread-config --ldflags` `pthread-config --libs`" + ;; + + *) + AC_MSG_CHECKING([for the pthreads library -l$flag]) + PTHREAD_LIBS="-l$flag" + ;; + esac + + save_LIBS="$LIBS" + save_CFLAGS="$CFLAGS" + LIBS="$PTHREAD_LIBS $LIBS" + CFLAGS="$CFLAGS $PTHREAD_CFLAGS $ax_pthread_extra_flags" + + # Check for various functions. We must include pthread.h, + # since some functions may be macros. (On the Sequent, we + # need a special flag -Kthread to make this header compile.) + # We check for pthread_join because it is in -lpthread on IRIX + # while pthread_create is in libc. We check for pthread_attr_init + # due to DEC craziness with -lpthreads. We check for + # pthread_cleanup_push because it is one of the few pthread + # functions on Solaris that doesn't have a non-functional libc stub. + # We try pthread_create on general principles. + AC_LINK_IFELSE([AC_LANG_PROGRAM([#include + static void routine(void *a) { a = 0; } + static void *start_routine(void *a) { return a; }], + [pthread_t th; pthread_attr_t attr; + pthread_create(&th, 0, start_routine, 0); + pthread_join(th, 0); + pthread_attr_init(&attr); + pthread_cleanup_push(routine, 0); + pthread_cleanup_pop(0) /* ; */])], + [ax_pthread_ok=yes], + []) + + LIBS="$save_LIBS" + CFLAGS="$save_CFLAGS" + + AC_MSG_RESULT([$ax_pthread_ok]) + if test "x$ax_pthread_ok" = xyes; then + break; + fi + + PTHREAD_LIBS="" + PTHREAD_CFLAGS="" +done +fi + +# Various other checks: +if test "x$ax_pthread_ok" = xyes; then + save_LIBS="$LIBS" + LIBS="$PTHREAD_LIBS $LIBS" + save_CFLAGS="$CFLAGS" + CFLAGS="$CFLAGS $PTHREAD_CFLAGS" + + # Detect AIX lossage: JOINABLE attribute is called UNDETACHED. + AC_MSG_CHECKING([for joinable pthread attribute]) + attr_name=unknown + for attr in PTHREAD_CREATE_JOINABLE PTHREAD_CREATE_UNDETACHED; do + AC_LINK_IFELSE([AC_LANG_PROGRAM([#include ], + [int attr = $attr; return attr /* ; */])], + [attr_name=$attr; break], + []) + done + AC_MSG_RESULT([$attr_name]) + if test "$attr_name" != PTHREAD_CREATE_JOINABLE; then + AC_DEFINE_UNQUOTED([PTHREAD_CREATE_JOINABLE], [$attr_name], + [Define to necessary symbol if this constant + uses a non-standard name on your system.]) + fi + + AC_MSG_CHECKING([if more special flags are required for pthreads]) + flag=no + case ${host_os} in + aix* | freebsd* | darwin*) flag="-D_THREAD_SAFE";; + osf* | hpux*) flag="-D_REENTRANT";; + solaris*) + if test "$GCC" = "yes"; then + flag="-D_REENTRANT" + else + # TODO: What about Clang on Solaris? + flag="-mt -D_REENTRANT" + fi + ;; + esac + AC_MSG_RESULT([$flag]) + if test "x$flag" != xno; then + PTHREAD_CFLAGS="$flag $PTHREAD_CFLAGS" + fi + + AC_CACHE_CHECK([for PTHREAD_PRIO_INHERIT], + [ax_cv_PTHREAD_PRIO_INHERIT], [ + AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include ]], + [[int i = PTHREAD_PRIO_INHERIT;]])], + [ax_cv_PTHREAD_PRIO_INHERIT=yes], + [ax_cv_PTHREAD_PRIO_INHERIT=no]) + ]) + AS_IF([test "x$ax_cv_PTHREAD_PRIO_INHERIT" = "xyes"], + [AC_DEFINE([HAVE_PTHREAD_PRIO_INHERIT], [1], [Have PTHREAD_PRIO_INHERIT.])]) + + LIBS="$save_LIBS" + CFLAGS="$save_CFLAGS" + + # More AIX lossage: compile with *_r variant + if test "x$GCC" != xyes; then + case $host_os in + aix*) + AS_CASE(["x/$CC"], + [x*/c89|x*/c89_128|x*/c99|x*/c99_128|x*/cc|x*/cc128|x*/xlc|x*/xlc_v6|x*/xlc128|x*/xlc128_v6], + [#handle absolute path differently from PATH based program lookup + AS_CASE(["x$CC"], + [x/*], + [AS_IF([AS_EXECUTABLE_P([${CC}_r])],[PTHREAD_CC="${CC}_r"])], + [AC_CHECK_PROGS([PTHREAD_CC],[${CC}_r],[$CC])])]) + ;; + esac + fi +fi + +test -n "$PTHREAD_CC" || PTHREAD_CC="$CC" + +AC_SUBST([PTHREAD_LIBS]) +AC_SUBST([PTHREAD_CFLAGS]) +AC_SUBST([PTHREAD_CC]) + +# Finally, execute ACTION-IF-FOUND/ACTION-IF-NOT-FOUND: +if test x"$ax_pthread_ok" = xyes; then + ifelse([$1],,[AC_DEFINE([HAVE_PTHREAD],[1],[Define if you have POSIX threads libraries and header files.])],[$1]) + : +else + ax_pthread_ok=no + $2 +fi +AC_LANG_POP +])dnl AX_PTHREAD diff --git a/build-aux/m4/bitcoin_find_bdb48.m4 b/build-aux/m4/bitcoin_find_bdb48.m4 new file mode 100644 index 00000000..f3b14461 --- /dev/null +++ b/build-aux/m4/bitcoin_find_bdb48.m4 @@ -0,0 +1,66 @@ +AC_DEFUN([BITCOIN_FIND_BDB48],[ + AC_MSG_CHECKING([for Berkeley DB C++ headers]) + BDB_CPPFLAGS= + BDB_LIBS= + bdbpath=X + bdb48path=X + bdbdirlist= + for _vn in 4.8 48 4 5 ''; do + for _pfx in b lib ''; do + bdbdirlist="$bdbdirlist ${_pfx}db${_vn}" + done + done + for searchpath in $bdbdirlist ''; do + test -n "${searchpath}" && searchpath="${searchpath}/" + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ + #include <${searchpath}db_cxx.h> + ]],[[ + #if !((DB_VERSION_MAJOR == 4 && DB_VERSION_MINOR >= 8) || DB_VERSION_MAJOR > 4) + #error "failed to find bdb 4.8+" + #endif + ]])],[ + if test "x$bdbpath" = "xX"; then + bdbpath="${searchpath}" + fi + ],[ + continue + ]) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ + #include <${searchpath}db_cxx.h> + ]],[[ + #if !(DB_VERSION_MAJOR == 4 && DB_VERSION_MINOR == 8) + #error "failed to find bdb 4.8" + #endif + ]])],[ + bdb48path="${searchpath}" + break + ],[]) + done + if test "x$bdbpath" = "xX"; then + AC_MSG_RESULT([no]) + AC_MSG_ERROR(libdb_cxx headers missing) + elif test "x$bdb48path" = "xX"; then + BITCOIN_SUBDIR_TO_INCLUDE(BDB_CPPFLAGS,[${bdbpath}],db_cxx) + AC_ARG_WITH([incompatible-bdb],[AS_HELP_STRING([--with-incompatible-bdb], [allow using a bdb version other than 4.8])],[ + AC_MSG_WARN([Found Berkeley DB other than 4.8; wallets opened by this build will not be portable!]) + ],[ + AC_MSG_ERROR([Found Berkeley DB other than 4.8, required for portable wallets (--with-incompatible-bdb to ignore or --disable-wallet to disable wallet functionality)]) + ]) + else + BITCOIN_SUBDIR_TO_INCLUDE(BDB_CPPFLAGS,[${bdb48path}],db_cxx) + bdbpath="${bdb48path}" + fi + AC_SUBST(BDB_CPPFLAGS) + + # TODO: Ideally this could find the library version and make sure it matches the headers being used + for searchlib in db_cxx-4.8 db_cxx; do + AC_CHECK_LIB([$searchlib],[main],[ + BDB_LIBS="-l${searchlib}" + break + ]) + done + if test "x$BDB_LIBS" = "x"; then + AC_MSG_ERROR([libdb_cxx missing, Bitcoin Core requires this library for wallet functionality (--disable-wallet to disable wallet functionality)]) + fi + AC_SUBST(BDB_LIBS) +]) diff --git a/build-aux/m4/bitcoin_qt.m4 b/build-aux/m4/bitcoin_qt.m4 new file mode 100644 index 00000000..2a722626 --- /dev/null +++ b/build-aux/m4/bitcoin_qt.m4 @@ -0,0 +1,415 @@ +dnl Helper for cases where a qt dependency is not met. +dnl Output: If qt version is auto, set bitcoin_enable_qt to false. Else, exit. +AC_DEFUN([BITCOIN_QT_FAIL],[ + if test "x$bitcoin_qt_want_version" = "xauto" && test x$bitcoin_qt_force != xyes; then + if test x$bitcoin_enable_qt != xno; then + AC_MSG_WARN([$1; bitcoin-qt frontend will not be built]) + fi + bitcoin_enable_qt=no + else + AC_MSG_ERROR([$1]) + fi +]) + +AC_DEFUN([BITCOIN_QT_CHECK],[ + if test "x$bitcoin_enable_qt" != "xno" && test x$bitcoin_qt_want_version != xno; then + true + $1 + else + true + $2 + fi +]) + +dnl BITCOIN_QT_PATH_PROGS([FOO], [foo foo2], [/path/to/search/first], [continue if missing]) +dnl Helper for finding the path of programs needed for Qt. +dnl Inputs: $1: Variable to be set +dnl Inputs: $2: List of programs to search for +dnl Inputs: $3: Look for $2 here before $PATH +dnl Inputs: $4: If "yes", don't fail if $2 is not found. +dnl Output: $1 is set to the path of $2 if found. $2 are searched in order. +AC_DEFUN([BITCOIN_QT_PATH_PROGS],[ + BITCOIN_QT_CHECK([ + if test "x$3" != "x"; then + AC_PATH_PROGS($1,$2,,$3) + else + AC_PATH_PROGS($1,$2) + fi + if test "x$$1" = "x" && test "x$4" != "xyes"; then + BITCOIN_QT_FAIL([$1 not found]) + fi + ]) +]) + +dnl Initialize qt input. +dnl This must be called before any other BITCOIN_QT* macros to ensure that +dnl input variables are set correctly. +dnl CAUTION: Do not use this inside of a conditional. +AC_DEFUN([BITCOIN_QT_INIT],[ + dnl enable qt support + AC_ARG_WITH([gui], + [AS_HELP_STRING([--with-gui@<:@=no|qt4|qt5|auto@:>@], + [build bitcoin-qt GUI (default=auto, qt4 tried first)])], + [ + bitcoin_qt_want_version=$withval + if test x$bitcoin_qt_want_version = xyes; then + bitcoin_qt_force=yes + bitcoin_qt_want_version=auto + fi + ], + [bitcoin_qt_want_version=auto]) + + AC_ARG_WITH([qt-incdir],[AS_HELP_STRING([--with-qt-incdir=INC_DIR],[specify qt include path (overridden by pkgconfig)])], [qt_include_path=$withval], []) + AC_ARG_WITH([qt-libdir],[AS_HELP_STRING([--with-qt-libdir=LIB_DIR],[specify qt lib path (overridden by pkgconfig)])], [qt_lib_path=$withval], []) + AC_ARG_WITH([qt-plugindir],[AS_HELP_STRING([--with-qt-plugindir=PLUGIN_DIR],[specify qt plugin path (overridden by pkgconfig)])], [qt_plugin_path=$withval], []) + AC_ARG_WITH([qt-translationdir],[AS_HELP_STRING([--with-qt-translationdir=PLUGIN_DIR],[specify qt translation path (overridden by pkgconfig)])], [qt_translation_path=$withval], []) + AC_ARG_WITH([qt-bindir],[AS_HELP_STRING([--with-qt-bindir=BIN_DIR],[specify qt bin path])], [qt_bin_path=$withval], []) + + AC_ARG_WITH([qtdbus], + [AS_HELP_STRING([--with-qtdbus], + [enable DBus support (default is yes if qt is enabled and QtDBus is found)])], + [use_dbus=$withval], + [use_dbus=auto]) + + AC_SUBST(QT_TRANSLATION_DIR,$qt_translation_path) +]) + +dnl Find the appropriate version of Qt libraries and includes. +dnl Inputs: $1: Whether or not pkg-config should be used. yes|no. Default: yes. +dnl Inputs: $2: If $1 is "yes" and --with-gui=auto, which qt version should be +dnl tried first. +dnl Outputs: See _BITCOIN_QT_FIND_LIBS_* +dnl Outputs: Sets variables for all qt-related tools. +dnl Outputs: bitcoin_enable_qt, bitcoin_enable_qt_dbus, bitcoin_enable_qt_test +AC_DEFUN([BITCOIN_QT_CONFIGURE],[ + use_pkgconfig=$1 + + if test x$use_pkgconfig = x; then + use_pkgconfig=yes + fi + + if test x$use_pkgconfig = xyes; then + BITCOIN_QT_CHECK([_BITCOIN_QT_FIND_LIBS_WITH_PKGCONFIG([$2])]) + else + BITCOIN_QT_CHECK([_BITCOIN_QT_FIND_LIBS_WITHOUT_PKGCONFIG]) + fi + + dnl This is ugly and complicated. Yuck. Works as follows: + dnl We can't discern whether Qt4 builds are static or not. For Qt5, we can + dnl check a header to find out. When Qt is built statically, some plugins must + dnl be linked into the final binary as well. These plugins have changed between + dnl Qt4 and Qt5. With Qt5, languages moved into core and the WindowsIntegration + dnl plugin was added. Since we can't tell if Qt4 is static or not, it is + dnl assumed for windows builds. + dnl _BITCOIN_QT_CHECK_STATIC_PLUGINS does a quick link-check and appends the + dnl results to QT_LIBS. + BITCOIN_QT_CHECK([ + TEMP_CPPFLAGS=$CPPFLAGS + CPPFLAGS=$QT_INCLUDES + if test x$bitcoin_qt_got_major_vers = x5; then + _BITCOIN_QT_IS_STATIC + if test x$bitcoin_cv_static_qt = xyes; then + AC_DEFINE(QT_STATICPLUGIN, 1, [Define this symbol if qt plugins are static]) + if test x$qt_plugin_path != x; then + QT_LIBS="$QT_LIBS -L$qt_plugin_path/accessible" + QT_LIBS="$QT_LIBS -L$qt_plugin_path/platforms" + fi + if test x$use_pkgconfig = xyes; then + PKG_CHECK_MODULES([QTPLATFORM], [Qt5PlatformSupport], [QT_LIBS="$QTPLATFORM_LIBS $QT_LIBS"]) + fi + _BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(AccessibleFactory)], [-lqtaccessiblewidgets]) + if test x$TARGET_OS = xwindows; then + _BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(QWindowsIntegrationPlugin)],[-lqwindows]) + AC_DEFINE(QT_QPA_PLATFORM_WINDOWS, 1, [Define this symbol if the qt platform is windows]) + elif test x$TARGET_OS = xlinux; then + PKG_CHECK_MODULES([X11XCB], [x11-xcb], [QT_LIBS="$X11XCB_LIBS $QT_LIBS"]) + _BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(QXcbIntegrationPlugin)],[-lqxcb -lxcb-static]) + AC_DEFINE(QT_QPA_PLATFORM_XCB, 1, [Define this symbol if the qt platform is xcb]) + elif test x$TARGET_OS = xdarwin; then + if test x$use_pkgconfig = xyes; then + PKG_CHECK_MODULES([QTPRINT], [Qt5PrintSupport], [QT_LIBS="$QTPRINT_LIBS $QT_LIBS"]) + fi + AX_CHECK_LINK_FLAG([[-framework IOKit]],[QT_LIBS="$QT_LIBS -framework IOKit"],[AC_MSG_ERROR(could not iokit framework)]) + _BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(QCocoaIntegrationPlugin)],[-lqcocoa]) + AC_DEFINE(QT_QPA_PLATFORM_COCOA, 1, [Define this symbol if the qt platform is cocoa]) + fi + fi + else + if test x$TARGET_OS = xwindows; then + AC_DEFINE(QT_STATICPLUGIN, 1, [Define this symbol if qt plugins are static]) + if test x$qt_plugin_path != x; then + QT_LIBS="$QT_LIBS -L$qt_plugin_path/accessible" + QT_LIBS="$QT_LIBS -L$qt_plugin_path/codecs" + fi + _BITCOIN_QT_CHECK_STATIC_PLUGINS([ + Q_IMPORT_PLUGIN(qcncodecs) + Q_IMPORT_PLUGIN(qjpcodecs) + Q_IMPORT_PLUGIN(qtwcodecs) + Q_IMPORT_PLUGIN(qkrcodecs) + Q_IMPORT_PLUGIN(AccessibleFactory)], + [-lqcncodecs -lqjpcodecs -lqtwcodecs -lqkrcodecs -lqtaccessiblewidgets]) + fi + fi + CPPFLAGS=$TEMP_CPPFLAGS + ]) + + if test x$use_pkgconfig$qt_bin_path = xyes; then + if test x$bitcoin_qt_got_major_vers = x5; then + qt_bin_path="`$PKG_CONFIG --variable=host_bins Qt5Core 2>/dev/null`" + fi + fi + + BITCOIN_QT_PATH_PROGS([MOC], [moc-qt${bitcoin_qt_got_major_vers} moc${bitcoin_qt_got_major_vers} moc], $qt_bin_path) + BITCOIN_QT_PATH_PROGS([UIC], [uic-qt${bitcoin_qt_got_major_vers} uic${bitcoin_qt_got_major_vers} uic], $qt_bin_path) + BITCOIN_QT_PATH_PROGS([RCC], [rcc-qt${bitcoin_qt_got_major_vers} rcc${bitcoin_qt_got_major_vers} rcc], $qt_bin_path) + BITCOIN_QT_PATH_PROGS([LRELEASE], [lrelease-qt${bitcoin_qt_got_major_vers} lrelease${bitcoin_qt_got_major_vers} lrelease], $qt_bin_path) + BITCOIN_QT_PATH_PROGS([LUPDATE], [lupdate-qt${bitcoin_qt_got_major_vers} lupdate${bitcoin_qt_got_major_vers} lupdate],$qt_bin_path, yes) + + MOC_DEFS='-DHAVE_CONFIG_H -I$(srcdir)' + case $host in + *darwin*) + BITCOIN_QT_CHECK([ + MOC_DEFS="${MOC_DEFS} -DQ_OS_MAC" + base_frameworks="-framework Foundation -framework ApplicationServices -framework AppKit" + AX_CHECK_LINK_FLAG([[$base_frameworks]],[QT_LIBS="$QT_LIBS $base_frameworks"],[AC_MSG_ERROR(could not find base frameworks)]) + ]) + ;; + *mingw*) + BITCOIN_QT_CHECK([ + AX_CHECK_LINK_FLAG([[-mwindows]],[QT_LDFLAGS="$QT_LDFLAGS -mwindows"],[AC_MSG_WARN(-mwindows linker support not detected)]) + ]) + esac + + + dnl enable qt support + AC_MSG_CHECKING(whether to build Bitcoin Core GUI) + BITCOIN_QT_CHECK([ + bitcoin_enable_qt=yes + bitcoin_enable_qt_test=yes + if test x$have_qt_test = xno; then + bitcoin_enable_qt_test=no + fi + bitcoin_enable_qt_dbus=no + if test x$use_dbus != xno && test x$have_qt_dbus = xyes; then + bitcoin_enable_qt_dbus=yes + fi + if test x$use_dbus = xyes && test x$have_qt_dbus = xno; then + AC_MSG_ERROR("libQtDBus not found. Install libQtDBus or remove --with-qtdbus.") + fi + if test x$LUPDATE = x; then + AC_MSG_WARN("lupdate is required to update qt translations") + fi + ],[ + bitcoin_enable_qt=no + ]) + AC_MSG_RESULT([$bitcoin_enable_qt (Qt${bitcoin_qt_got_major_vers})]) + + AC_SUBST(QT_INCLUDES) + AC_SUBST(QT_LIBS) + AC_SUBST(QT_LDFLAGS) + AC_SUBST(QT_DBUS_INCLUDES) + AC_SUBST(QT_DBUS_LIBS) + AC_SUBST(QT_TEST_INCLUDES) + AC_SUBST(QT_TEST_LIBS) + AC_SUBST(QT_SELECT, qt${bitcoin_qt_got_major_vers}) + AC_SUBST(MOC_DEFS) +]) + +dnl All macros below are internal and should _not_ be used from the main +dnl configure.ac. +dnl ---- + +dnl Internal. Check if the included version of Qt is Qt5. +dnl Requires: INCLUDES must be populated as necessary. +dnl Output: bitcoin_cv_qt5=yes|no +AC_DEFUN([_BITCOIN_QT_CHECK_QT5],[ + AC_CACHE_CHECK(for Qt 5, bitcoin_cv_qt5,[ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM( + [[#include ]], + [[ + #if QT_VERSION < 0x050000 + choke me + #else + return 0; + #endif + ]])], + [bitcoin_cv_qt5=yes], + [bitcoin_cv_qt5=no]) +])]) + +dnl Internal. Check if the linked version of Qt was built as static libs. +dnl Requires: Qt5. This check cannot determine if Qt4 is static. +dnl Requires: INCLUDES and LIBS must be populated as necessary. +dnl Output: bitcoin_cv_static_qt=yes|no +dnl Output: Defines QT_STATICPLUGIN if plugins are static. +AC_DEFUN([_BITCOIN_QT_IS_STATIC],[ + AC_CACHE_CHECK(for static Qt, bitcoin_cv_static_qt,[ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM( + [[#include ]], + [[ + #if defined(QT_STATIC) + return 0; + #else + choke me + #endif + ]])], + [bitcoin_cv_static_qt=yes], + [bitcoin_cv_static_qt=no]) + ]) + if test xbitcoin_cv_static_qt = xyes; then + AC_DEFINE(QT_STATICPLUGIN, 1, [Define this symbol for static Qt plugins]) + fi +]) + +dnl Internal. Check if the link-requirements for static plugins are met. +dnl Requires: INCLUDES and LIBS must be populated as necessary. +dnl Inputs: $1: A series of Q_IMPORT_PLUGIN(). +dnl Inputs: $2: The libraries that resolve $1. +dnl Output: QT_LIBS is prepended or configure exits. +AC_DEFUN([_BITCOIN_QT_CHECK_STATIC_PLUGINS],[ + AC_MSG_CHECKING(for static Qt plugins: $2) + CHECK_STATIC_PLUGINS_TEMP_LIBS="$LIBS" + LIBS="$2 $QT_LIBS $LIBS" + AC_LINK_IFELSE([AC_LANG_PROGRAM([[ + #define QT_STATICPLUGIN + #include + $1]], + [[return 0;]])], + [AC_MSG_RESULT(yes); QT_LIBS="$2 $QT_LIBS"], + [AC_MSG_RESULT(no); BITCOIN_QT_FAIL(Could not resolve: $2)]) + LIBS="$CHECK_STATIC_PLUGINS_TEMP_LIBS" +]) + +dnl Internal. Find Qt libraries using pkg-config. +dnl Inputs: bitcoin_qt_want_version (from --with-gui=). The version to check +dnl first. +dnl Inputs: $1: If bitcoin_qt_want_version is "auto", check for this version +dnl first. +dnl Outputs: All necessary QT_* variables are set. +dnl Outputs: bitcoin_qt_got_major_vers is set to "4" or "5". +dnl Outputs: have_qt_test and have_qt_dbus are set (if applicable) to yes|no. +AC_DEFUN([_BITCOIN_QT_FIND_LIBS_WITH_PKGCONFIG],[ + m4_ifdef([PKG_CHECK_MODULES],[ + auto_priority_version=$1 + if test x$auto_priority_version = x; then + auto_priority_version=qt5 + fi + if test x$bitcoin_qt_want_version = xqt5 || ( test x$bitcoin_qt_want_version = xauto && test x$auto_priority_version = xqt5 ); then + QT_LIB_PREFIX=Qt5 + bitcoin_qt_got_major_vers=5 + else + QT_LIB_PREFIX=Qt + bitcoin_qt_got_major_vers=4 + fi + qt5_modules="Qt5Core Qt5Gui Qt5Network Qt5Widgets" + qt4_modules="QtCore QtGui QtNetwork" + BITCOIN_QT_CHECK([ + if test x$bitcoin_qt_want_version = xqt5 || ( test x$bitcoin_qt_want_version = xauto && test x$auto_priority_version = xqt5 ); then + PKG_CHECK_MODULES([QT], [$qt5_modules], [QT_INCLUDES="$QT_CFLAGS"; have_qt=yes],[have_qt=no]) + elif test x$bitcoin_qt_want_version = xqt4 || ( test x$bitcoin_qt_want_version = xauto && test x$auto_priority_version = xqt4 ); then + PKG_CHECK_MODULES([QT], [$qt4_modules], [QT_INCLUDES="$QT_CFLAGS"; have_qt=yes], [have_qt=no]) + fi + + dnl qt version is set to 'auto' and the preferred version wasn't found. Now try the other. + if test x$have_qt = xno && test x$bitcoin_qt_want_version = xauto; then + if test x$auto_priority_version = x$qt5; then + PKG_CHECK_MODULES([QT], [$qt4_modules], [QT_INCLUDES="$QT_CFLAGS"; have_qt=yes; QT_LIB_PREFIX=Qt; bitcoin_qt_got_major_vers=4], [have_qt=no]) + else + PKG_CHECK_MODULES([QT], [$qt5_modules], [QT_INCLUDES="$QT_CFLAGS"; have_qt=yes; QT_LIB_PREFIX=Qt5; bitcoin_qt_got_major_vers=5], [have_qt=no]) + fi + fi + if test x$have_qt != xyes; then + have_qt=no + BITCOIN_QT_FAIL([Qt dependencies not found]) + fi + ]) + BITCOIN_QT_CHECK([ + PKG_CHECK_MODULES([QT_TEST], [${QT_LIB_PREFIX}Test], [QT_TEST_INCLUDES="$QT_TEST_CFLAGS"; have_qt_test=yes], [have_qt_test=no]) + if test x$use_dbus != xno; then + PKG_CHECK_MODULES([QT_DBUS], [${QT_LIB_PREFIX}DBus], [QT_DBUS_INCLUDES="$QT_DBUS_CFLAGS"; have_qt_dbus=yes], [have_qt_dbus=no]) + fi + ]) + ]) + true; dnl +]) + +dnl Internal. Find Qt libraries without using pkg-config. Version is deduced +dnl from the discovered headers. +dnl Inputs: bitcoin_qt_want_version (from --with-gui=). The version to use. +dnl If "auto", the version will be discovered by _BITCOIN_QT_CHECK_QT5. +dnl Outputs: All necessary QT_* variables are set. +dnl Outputs: bitcoin_qt_got_major_vers is set to "4" or "5". +dnl Outputs: have_qt_test and have_qt_dbus are set (if applicable) to yes|no. +AC_DEFUN([_BITCOIN_QT_FIND_LIBS_WITHOUT_PKGCONFIG],[ + TEMP_CPPFLAGS="$CPPFLAGS" + TEMP_LIBS="$LIBS" + BITCOIN_QT_CHECK([ + if test x$qt_include_path != x; then + QT_INCLUDES="-I$qt_include_path -I$qt_include_path/QtCore -I$qt_include_path/QtGui -I$qt_include_path/QtWidgets -I$qt_include_path/QtNetwork -I$qt_include_path/QtTest -I$qt_include_path/QtDBus" + CPPFLAGS="$QT_INCLUDES $CPPFLAGS" + fi + ]) + + BITCOIN_QT_CHECK([AC_CHECK_HEADER([QtPlugin],,BITCOIN_QT_FAIL(QtCore headers missing))]) + BITCOIN_QT_CHECK([AC_CHECK_HEADER([QApplication],, BITCOIN_QT_FAIL(QtGui headers missing))]) + BITCOIN_QT_CHECK([AC_CHECK_HEADER([QLocalSocket],, BITCOIN_QT_FAIL(QtNetwork headers missing))]) + + BITCOIN_QT_CHECK([ + if test x$bitcoin_qt_want_version = xauto; then + _BITCOIN_QT_CHECK_QT5 + fi + if test x$bitcoin_cv_qt5 = xyes || test x$bitcoin_qt_want_version = xqt5; then + QT_LIB_PREFIX=Qt5 + bitcoin_qt_got_major_vers=5 + else + QT_LIB_PREFIX=Qt + bitcoin_qt_got_major_vers=4 + fi + ]) + + BITCOIN_QT_CHECK([ + LIBS= + if test x$qt_lib_path != x; then + LIBS="$LIBS -L$qt_lib_path" + fi + + if test x$TARGET_OS = xwindows; then + AC_CHECK_LIB([imm32], [main],, BITCOIN_QT_FAIL(libimm32 not found)) + fi + ]) + + BITCOIN_QT_CHECK(AC_CHECK_LIB([z] ,[main],,AC_MSG_WARN([zlib not found. Assuming qt has it built-in]))) + BITCOIN_QT_CHECK(AC_CHECK_LIB([png] ,[main],,AC_MSG_WARN([libpng not found. Assuming qt has it built-in]))) + BITCOIN_QT_CHECK(AC_CHECK_LIB([jpeg] ,[main],,AC_MSG_WARN([libjpeg not found. Assuming qt has it built-in]))) + BITCOIN_QT_CHECK(AC_CHECK_LIB([pcre16] ,[main],,AC_MSG_WARN([libpcre16 not found. Assuming qt has it built-in]))) + BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}Core] ,[main],,BITCOIN_QT_FAIL(lib$QT_LIB_PREFIXCore not found))) + BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}Gui] ,[main],,BITCOIN_QT_FAIL(lib$QT_LIB_PREFIXGui not found))) + BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}Network],[main],,BITCOIN_QT_FAIL(lib$QT_LIB_PREFIXNetwork not found))) + if test x$bitcoin_qt_got_major_vers = x5; then + BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}Widgets],[main],,BITCOIN_QT_FAIL(lib$QT_LIB_PREFIXWidgets not found))) + fi + QT_LIBS="$LIBS" + LIBS="$TEMP_LIBS" + + BITCOIN_QT_CHECK([ + LIBS= + if test x$qt_lib_path != x; then + LIBS="-L$qt_lib_path" + fi + AC_CHECK_LIB([${QT_LIB_PREFIX}Test], [main],, have_qt_test=no) + AC_CHECK_HEADER([QTest],, have_qt_test=no) + QT_TEST_LIBS="$LIBS" + if test x$use_dbus != xno; then + LIBS= + if test x$qt_lib_path != x; then + LIBS="-L$qt_lib_path" + fi + AC_CHECK_LIB([${QT_LIB_PREFIX}DBus], [main],, have_qt_dbus=no) + AC_CHECK_HEADER([QtDBus],, have_qt_dbus=no) + QT_DBUS_LIBS="$LIBS" + fi + ]) + CPPFLAGS="$TEMP_CPPFLAGS" + LIBS="$TEMP_LIBS" +]) + diff --git a/build-aux/m4/bitcoin_subdir_to_include.m4 b/build-aux/m4/bitcoin_subdir_to_include.m4 new file mode 100644 index 00000000..66f106c7 --- /dev/null +++ b/build-aux/m4/bitcoin_subdir_to_include.m4 @@ -0,0 +1,14 @@ +dnl BITCOIN_SUBDIR_TO_INCLUDE([CPPFLAGS-VARIABLE-NAME],[SUBDIRECTORY-NAME],[HEADER-FILE]) +dnl SUBDIRECTORY-NAME must end with a path separator +AC_DEFUN([BITCOIN_SUBDIR_TO_INCLUDE],[ + if test "x$2" = "x"; then + AC_MSG_RESULT([default]) + else + echo "#include <$2$3.h>" >conftest.cpp + newinclpath=`${CXXCPP} ${CPPFLAGS} -M conftest.cpp 2>/dev/null | [ tr -d '\\n\\r\\\\' | sed -e 's/^.*[[:space:]:]\(\/[^[:space:]]*\)]$3[\.h[[:space:]].*$/\1/' -e t -e d`] + AC_MSG_RESULT([${newinclpath}]) + if test "x${newinclpath}" != "x"; then + eval "$1=\"\$$1\"' -I${newinclpath}'" + fi + fi +]) diff --git a/configure.ac b/configure.ac new file mode 100644 index 00000000..224e463f --- /dev/null +++ b/configure.ac @@ -0,0 +1,923 @@ +dnl require autoconf 2.60 (AS_ECHO/AS_ECHO_N) +AC_PREREQ([2.60]) +define(_CLIENT_VERSION_MAJOR, 0) +define(_CLIENT_VERSION_MINOR, 10) +define(_CLIENT_VERSION_REVISION, 0) +define(_CLIENT_VERSION_BUILD, 0) +define(_CLIENT_VERSION_IS_RELEASE, true) +define(_COPYRIGHT_YEAR, 2015) +AC_INIT([Bitcoin Core],[_CLIENT_VERSION_MAJOR._CLIENT_VERSION_MINOR._CLIENT_VERSION_REVISION],[info@bitcoin.org],[bitcoin]) +AC_CONFIG_SRCDIR([src/core/main.cpp]) +AC_CONFIG_HEADERS([src/config/bitcoin-config.h]) +AC_CONFIG_AUX_DIR([build-aux]) +AC_CONFIG_MACRO_DIR([build-aux/m4]) + +AC_CANONICAL_HOST + +AH_TOP([#ifndef BITCOIN_CONFIG_H]) +AH_TOP([#define BITCOIN_CONFIG_H]) +AH_BOTTOM([#endif //BITCOIN_CONFIG_H]) + +dnl faketime breaks configure and is only needed for make. Disable it here. +unset FAKETIME + +dnl Automake init set-up and checks +AM_INIT_AUTOMAKE([no-define subdir-objects foreign]) + +dnl faketime messes with timestamps and causes configure to be re-run. +dnl --disable-maintainer-mode can be used to bypass this. +AM_MAINTAINER_MODE([enable]) + +dnl make the compilation flags quiet unless V=1 is used +m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) + +dnl Compiler checks (here before libtool). +if test "x${CXXFLAGS+set}" = "xset"; then + CXXFLAGS_overridden=yes +else + CXXFLAGS_overridden=no +fi +AC_PROG_CXX +m4_ifdef([AC_PROG_OBJCXX],[AC_PROG_OBJCXX]) + +dnl By default, libtool for mingw refuses to link static libs into a dll for +dnl fear of mixing pic/non-pic objects, and import/export complications. Since +dnl we have those under control, re-enable that functionality. +case $host in + *mingw*) + lt_cv_deplibs_check_method="pass_all" + ;; +esac +dnl Libtool init checks. +LT_INIT([pic-only]) + +dnl Check/return PATH for base programs. +AC_PATH_TOOL(AR, ar) +AC_PATH_TOOL(RANLIB, ranlib) +AC_PATH_TOOL(STRIP, strip) +AC_PATH_TOOL(GCOV, gcov) +AC_PATH_PROG(LCOV, lcov) +AC_PATH_PROG(JAVA, java) +AC_PATH_PROG(GENHTML, genhtml) +AC_PATH_PROG([GIT], [git]) +AC_PATH_PROG(CCACHE,ccache) +AC_PATH_PROG(XGETTEXT,xgettext) +AC_PATH_PROG(HEXDUMP,hexdump) + +# This m4 will only be used if a system copy cannot be found. This is helpful +# on systems where autotools are installed but the pkg-config macros are not in +# a default location. It is currently used for building on OSX where autotools +# are preinstalled but pkg-config comes from macports or homebrew. It should +# probably be removed when building on <= 10.6 is no longer supported. +m4_include([pkg.m4]) + +dnl pkg-config check. +PKG_PROG_PKG_CONFIG + +# Enable wallet +AC_ARG_ENABLE([wallet], + [AS_HELP_STRING([--enable-wallet], + [enable wallet (default is yes)])], + [enable_wallet=$enableval], + [enable_wallet=yes]) + +AC_ARG_WITH([miniupnpc], + [AS_HELP_STRING([--with-miniupnpc], + [enable UPNP (default is yes if libminiupnpc is found)])], + [use_upnp=$withval], + [use_upnp=auto]) + +AC_ARG_ENABLE([upnp-default], + [AS_HELP_STRING([--enable-upnp-default], + [if UPNP is enabled, turn it on at startup (default is no)])], + [use_upnp_default=$enableval], + [use_upnp_default=no]) + +AC_ARG_ENABLE(tests, + AS_HELP_STRING([--enable-tests],[compile tests (default is yes)]), + [use_tests=$enableval], + [use_tests=yes]) + +AC_ARG_WITH([comparison-tool], + AS_HELP_STRING([--with-comparison-tool],[path to java comparison tool (requires --enable-tests)]), + [use_comparison_tool=$withval], + [use_comparison_tool=no]) + +AC_ARG_ENABLE([comparison-tool-reorg-tests], + AS_HELP_STRING([--enable-comparison-tool-reorg-tests],[enable expensive reorg tests in the comparison tool (default no)]), + [use_comparison_tool_reorg_tests=$enableval], + [use_comparison_tool_reorg_tests=no]) + +AC_ARG_WITH([qrencode], + [AS_HELP_STRING([--with-qrencode], + [enable QR code support (default is yes if qt is enabled and libqrencode is found)])], + [use_qr=$withval], + [use_qr=auto]) + +AC_ARG_ENABLE([hardening], + [AS_HELP_STRING([--enable-hardening], + [attempt to harden the resulting executables (default is yes)])], + [use_hardening=$enableval], + [use_hardening=yes]) + +AC_ARG_ENABLE([reduce-exports], + [AS_HELP_STRING([--enable-reduce-exports], + [attempt to reduce exported symbols in the resulting executables (default is yes)])], + [use_reduce_exports=$enableval], + [use_reduce_exports=auto]) + +AC_ARG_ENABLE([ccache], + [AS_HELP_STRING([--enable-ccache], + [use ccache for building (default is yes if ccache is found)])], + [use_ccache=$enableval], + [use_ccache=auto]) + +AC_ARG_ENABLE([lcov], + [AS_HELP_STRING([--enable-lcov], + [enable lcov testing (default is no)])], + [use_lcov=yes], + [use_lcov=no]) + +AC_ARG_ENABLE([glibc-back-compat], + [AS_HELP_STRING([--enable-glibc-back-compat], + [enable backwards compatibility with glibc and libstdc++])], + [use_glibc_compat=$enableval], + [use_glibc_compat=no]) + +AC_ARG_WITH([protoc-bindir],[AS_HELP_STRING([--with-protoc-bindir=BIN_DIR],[specify protoc bin path])], [protoc_bin_path=$withval], []) + +# Enable debug +AC_ARG_ENABLE([debug], + [AS_HELP_STRING([--enable-debug], + [use debug compiler flags and macros (default is no)])], + [enable_debug=$enableval], + [enable_debug=no]) + +if test "x$enable_debug" = xyes; then + if test "x$GCC" = xyes; then + CFLAGS="-g3 -O0 -DDEBUG" + fi + + if test "x$GXX" = xyes; then + CXXFLAGS="-g3 -O0 -DDEBUG" + fi +fi + +## TODO: Remove these hard-coded paths and flags. They are here for the sake of +## compatibility with the legacy buildsystem. +## +if test "x$CXXFLAGS_overridden" = "xno"; then + CXXFLAGS="$CXXFLAGS -Wall -Wextra -Wformat -Wformat-security -Wno-unused-parameter" +fi +CPPFLAGS="$CPPFLAGS -DBOOST_SPIRIT_THREADSAFE -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS" + +AC_ARG_WITH([utils], + [AS_HELP_STRING([--with-utils], + [build bitcoin-cli bitcoin-tx (default=yes)])], + [build_bitcoin_utils=$withval], + [build_bitcoin_utils=yes]) + +AC_ARG_WITH([libs], + [AS_HELP_STRING([--with-libs], + [build libraries (default=yes)])], + [build_bitcoin_libs=$withval], + [build_bitcoin_libs=yes]) + +AC_ARG_WITH([daemon], + [AS_HELP_STRING([--with-daemon], + [build bitcoind daemon (default=yes)])], + [build_bitcoind=$withval], + [build_bitcoind=yes]) + +AC_LANG_PUSH([C++]) + +use_pkgconfig=yes +case $host in + *mingw*) + + #pkgconfig does more harm than good with MinGW + use_pkgconfig=no + + TARGET_OS=windows + AC_CHECK_LIB([mingwthrd], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([kernel32], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([user32], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([gdi32], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([comdlg32], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([winspool], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([winmm], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([shell32], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([comctl32], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([ole32], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([oleaut32], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([uuid], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([rpcrt4], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([advapi32], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([ws2_32], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([mswsock], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([shlwapi], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([iphlpapi], [main],, AC_MSG_ERROR(lib missing)) + AC_CHECK_LIB([crypt32], [main],, AC_MSG_ERROR(lib missing)) + + # -static is interpreted by libtool, where it has a different meaning. + # In libtool-speak, it's -all-static. + AX_CHECK_LINK_FLAG([[-static]],[LIBTOOL_APP_LDFLAGS="$LIBTOOL_APP_LDFLAGS -all-static"]) + + AC_PATH_PROG([MAKENSIS], [makensis], none) + if test x$MAKENSIS = xnone; then + AC_MSG_WARN("makensis not found. Cannot create installer.") + fi + + AC_PATH_TOOL(WINDRES, windres, none) + if test x$WINDRES = xnone; then + AC_MSG_ERROR("windres not found") + fi + + CPPFLAGS="$CPPFLAGS -D_MT -DWIN32 -D_WINDOWS -DBOOST_THREAD_USE_LIB" + LEVELDB_TARGET_FLAGS="TARGET_OS=OS_WINDOWS_CROSSCOMPILE" + if test "x$CXXFLAGS_overridden" = "xno"; then + CXXFLAGS="$CXXFLAGS -w" + fi + case $host in + i?86-*) WINDOWS_BITS=32 ;; + x86_64-*) WINDOWS_BITS=64 ;; + *) AC_MSG_ERROR("Could not determine win32/win64 for installer") ;; + esac + AC_SUBST(WINDOWS_BITS) + + dnl libtool insists upon adding -nostdlib and a list of objects/libs to link against. + dnl That breaks our ability to build dll's with static libgcc/libstdc++/libssp. Override + dnl its command here, with the predeps/postdeps removed, and -static inserted. Postdeps are + dnl also overridden to prevent their insertion later. + dnl This should only affect dll's. + archive_cmds_CXX="\$CC -shared \$libobjs \$deplibs \$compiler_flags -static -o \$output_objdir/\$soname \${wl}--enable-auto-image-base -Xlinker --out-implib -Xlinker \$lib" + postdeps_CXX= + + ;; + *darwin*) + TARGET_OS=darwin + LEVELDB_TARGET_FLAGS="TARGET_OS=Darwin" + if test x$cross_compiling != xyes; then + BUILD_OS=darwin + AC_CHECK_PROG([PORT],port, port) + if test x$PORT = xport; then + dnl add default macports paths + CPPFLAGS="$CPPFLAGS -isystem /opt/local/include" + LIBS="$LIBS -L/opt/local/lib" + if test -d /opt/local/include/db48; then + CPPFLAGS="$CPPFLAGS -I/opt/local/include/db48" + LIBS="$LIBS -L/opt/local/lib/db48" + fi + fi + + AC_CHECK_PROG([BREW],brew, brew) + if test x$BREW = xbrew; then + dnl These Homebrew packages may be keg-only, meaning that they won't be found + dnl in expected paths because they may conflict with system files. Ask + dnl Homebrew where each one is located, then adjust paths accordingly. + dnl It's safe to add these paths even if the functionality is disabled by + dnl the user (--without-wallet or --without-gui for example). + + openssl_prefix=`$BREW --prefix openssl 2>/dev/null` + bdb_prefix=`$BREW --prefix berkeley-db4 2>/dev/null` + qt5_prefix=`$BREW --prefix qt5 2>/dev/null` + if test x$openssl_prefix != x; then + PKG_CONFIG_PATH="$openssl_prefix/lib/pkgconfig:$PKG_CONFIG_PATH" + export PKG_CONFIG_PATH + fi + if test x$bdb_prefix != x; then + CPPFLAGS="$CPPFLAGS -I$bdb_prefix/include" + LIBS="$LIBS -L$bdb_prefix/lib" + fi + if test x$qt5_prefix != x; then + PKG_CONFIG_PATH="$qt5_prefix/lib/pkgconfig:$PKG_CONFIG_PATH" + export PKG_CONFIG_PATH + fi + fi + else + case $build_os in + *darwin*) + BUILD_OS=darwin + ;; + *) + AC_PATH_TOOL([INSTALLNAMETOOL], [install_name_tool], install_name_tool) + AC_PATH_TOOL([OTOOL], [otool], otool) + AC_PATH_PROGS([GENISOIMAGE], [genisoimage mkisofs],genisoimage) + + dnl libtool will try to strip the static lib, which is a problem for + dnl cross-builds because strip attempts to call a hard-coded ld, + dnl which may not exist in the path. Stripping the .a is not + dnl necessary, so just disable it. + old_striplib= + ;; + esac + fi + + AX_CHECK_LINK_FLAG([[-Wl,-headerpad_max_install_names]], [LDFLAGS="$LDFLAGS -Wl,-headerpad_max_install_names"]) + CPPFLAGS="$CPPFLAGS -DMAC_OSX" + ;; + *linux*) + TARGET_OS=linux + ;; + *) + ;; +esac + +if test x$use_comparison_tool != xno; then + AC_SUBST(JAVA_COMPARISON_TOOL, $use_comparison_tool) +fi + +if test x$use_comparison_tool_reorg_tests != xno; then + if test x$use_comparison_tool = x; then + AC_MSG_ERROR("comparison tool reorg tests but comparison tool was not specified") + fi + AC_SUBST(COMPARISON_TOOL_REORG_TESTS, 1) +else + AC_SUBST(COMPARISON_TOOL_REORG_TESTS, 0) +fi + +if test x$use_lcov = xyes; then + if test x$LCOV = x; then + AC_MSG_ERROR("lcov testing requested but lcov not found") + fi + if test x$GCOV = x; then + AC_MSG_ERROR("lcov testing requested but gcov not found") + fi + if test x$JAVA = x; then + AC_MSG_ERROR("lcov testing requested but java not found") + fi + if test x$GENHTML = x; then + AC_MSG_ERROR("lcov testing requested but genhtml not found") + fi + if test x$use_comparison_tool = x; then + AC_MSG_ERROR("lcov testing requested but comparison tool was not specified") + fi + LCOV="$LCOV --gcov-tool=$GCOV" + AX_CHECK_COMPILE_FLAG([--coverage],[CXXFLAGS="$CXXFLAGS --coverage"], + [AC_MSG_ERROR("lcov testing requested but --coverage flag does not work")]) +fi + +dnl Require little endian +AC_C_BIGENDIAN([AC_MSG_ERROR("Big Endian not supported")]) + +dnl Check for pthread compile/link requirements +AX_PTHREAD + +# The following macro will add the necessary defines to bitcoin-config.h, but +# they also need to be passed down to any subprojects. Pull the results out of +# the cache and add them to CPPFLAGS. +AC_SYS_LARGEFILE +# detect POSIX or GNU variant of strerror_r +AC_FUNC_STRERROR_R + +if test x$ac_cv_sys_file_offset_bits != x && + test x$ac_cv_sys_file_offset_bits != xno && + test x$ac_cv_sys_file_offset_bits != xunknown; then + CPPFLAGS="$CPPFLAGS -D_FILE_OFFSET_BITS=$ac_cv_sys_file_offset_bits" +fi + +if test x$ac_cv_sys_large_files != x && + test x$ac_cv_sys_large_files != xno && + test x$ac_cv_sys_large_files != xunknown; then + CPPFLAGS="$CPPFLAGS -D_LARGE_FILES=$ac_cv_sys_large_files" +fi + +AX_CHECK_LINK_FLAG([[-Wl,--large-address-aware]], [LDFLAGS="$LDFLAGS -Wl,--large-address-aware"]) + +AX_GCC_FUNC_ATTRIBUTE([visibility]) +AX_GCC_FUNC_ATTRIBUTE([dllexport]) +AX_GCC_FUNC_ATTRIBUTE([dllimport]) + +if test x$use_glibc_compat != xno; then + + #__fdelt_chk's params and return type have changed from long unsigned int to long int. + # See which one is present here. + AC_MSG_CHECKING(__fdelt_chk type) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#ifdef _FORTIFY_SOURCE + #undef _FORTIFY_SOURCE + #endif + #define _FORTIFY_SOURCE 2 + #include + extern "C" long unsigned int __fdelt_warn(long unsigned int);]],[[]])], + [ fdelt_type="long unsigned int"], + [ fdelt_type="long int"]) + AC_MSG_RESULT($fdelt_type) + AC_DEFINE_UNQUOTED(FDELT_TYPE, $fdelt_type,[parameter and return value type for __fdelt_chk]) + +fi + +if test x$use_hardening != xno; then + AX_CHECK_COMPILE_FLAG([-Wstack-protector],[HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -Wstack-protector"]) + AX_CHECK_COMPILE_FLAG([-fstack-protector-all],[HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -fstack-protector-all"]) + + AX_CHECK_PREPROC_FLAG([-D_FORTIFY_SOURCE=2],[ + AX_CHECK_PREPROC_FLAG([-U_FORTIFY_SOURCE],[ + HARDENED_CPPFLAGS="$HARDENED_CPPFLAGS -U_FORTIFY_SOURCE" + ]) + HARDENED_CPPFLAGS="$HARDENED_CPPFLAGS -D_FORTIFY_SOURCE=2" + ]) + + AX_CHECK_LINK_FLAG([[-Wl,--dynamicbase]], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,--dynamicbase"]) + AX_CHECK_LINK_FLAG([[-Wl,--nxcompat]], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,--nxcompat"]) + AX_CHECK_LINK_FLAG([[-Wl,-z,relro]], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,-z,relro"]) + AX_CHECK_LINK_FLAG([[-Wl,-z,now]], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,-z,now"]) + + if test x$TARGET_OS != xwindows; then + # All windows code is PIC, forcing it on just adds useless compile warnings + AX_CHECK_COMPILE_FLAG([-fPIE],[HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -fPIE"]) + AX_CHECK_LINK_FLAG([[-pie]], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -pie"]) + fi + + case $host in + *mingw*) + AC_CHECK_LIB([ssp], [main],, AC_MSG_ERROR(lib missing)) + ;; + esac + + CXXFLAGS="$CXXFLAGS $HARDENED_CXXFLAGS" + CPPFLAGS="$CPPFLAGS $HARDENED_CPPFLAGS" + LDFLAGS="$LDFLAGS $HARDENED_LDFLAGS" + OBJCXXFLAGS="$CXXFLAGS" +fi + +dnl this flag screws up non-darwin gcc even when the check fails. special-case it. +if test x$TARGET_OS = xdarwin; then + AX_CHECK_LINK_FLAG([[-Wl,-dead_strip]], [LDFLAGS="$LDFLAGS -Wl,-dead_strip"]) +fi + +AC_CHECK_HEADERS([endian.h stdio.h stdlib.h unistd.h strings.h sys/types.h sys/stat.h sys/select.h sys/prctl.h]) +AC_SEARCH_LIBS([getaddrinfo_a], [anl], [AC_DEFINE(HAVE_GETADDRINFO_A, 1, [Define this symbol if you have getaddrinfo_a])]) +AC_SEARCH_LIBS([inet_pton], [nsl resolv], [AC_DEFINE(HAVE_INET_PTON, 1, [Define this symbol if you have inet_pton])]) + +AC_CHECK_DECLS([strnlen]) + +AC_CHECK_DECLS([le32toh, le64toh, htole32, htole64, be32toh, be64toh, htobe32, htobe64],,, + [#if HAVE_ENDIAN_H + #include + #endif]) + +dnl Check for MSG_NOSIGNAL +AC_MSG_CHECKING(for MSG_NOSIGNAL) +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include ]], + [[ int f = MSG_NOSIGNAL; ]])], + [ AC_MSG_RESULT(yes); AC_DEFINE(HAVE_MSG_NOSIGNAL, 1,[Define this symbol if you have MSG_NOSIGNAL]) ], + [ AC_MSG_RESULT(no)] +) + +AC_SEARCH_LIBS([clock_gettime],[rt]) + +AC_MSG_CHECKING([for visibility attribute]) +AC_LINK_IFELSE([AC_LANG_SOURCE([ + int foo_def( void ) __attribute__((visibility("default"))); + int main(){} + ])], + [ + AC_DEFINE(HAVE_VISIBILITY_ATTRIBUTE,1,[Define if the visibility attribute is supported.]) + AC_MSG_RESULT(yes) + ], + [ + AC_MSG_RESULT(no) + if test x$use_reduce_exports = xyes; then + AC_MSG_ERROR([Cannot find a working visibility attribute. Use --disable-reduced-exports.]) + fi + AC_MSG_WARN([Cannot find a working visibility attribute. Disabling reduced exports.]) + use_reduce_exports=no + ] +) + +if test x$use_reduce_exports != xno; then + AX_CHECK_COMPILE_FLAG([-fvisibility=hidden],[RE_CXXFLAGS="-fvisibility=hidden"], + [ + if test x$use_reduce_exports = xyes; then + AC_MSG_ERROR([Cannot set default symbol visibility. Use --disable-reduced-exports.]) + fi + AC_MSG_WARN([Cannot set default symbol visibility. Disabling reduced exports.]) + use_reduce_exports=no + ]) +fi + +LEVELDB_CPPFLAGS= +LIBLEVELDB= +LIBMEMENV= +AM_CONDITIONAL([EMBEDDED_LEVELDB],[true]) +AC_SUBST(LEVELDB_CPPFLAGS) +AC_SUBST(LIBLEVELDB) +AC_SUBST(LIBMEMENV) + +if test x$enable_wallet != xno; then + dnl Check for libdb_cxx only if wallet enabled + BITCOIN_FIND_BDB48 +fi + +dnl Check for libminiupnpc (optional) +if test x$use_upnp != xno; then + AC_CHECK_HEADERS( + [miniupnpc/miniwget.h miniupnpc/miniupnpc.h miniupnpc/upnpcommands.h miniupnpc/upnperrors.h], + [AC_CHECK_LIB([miniupnpc], [main],[MINIUPNPC_LIBS=-lminiupnpc], [have_miniupnpc=no])], + [have_miniupnpc=no] + ) +fi + +BITCOIN_QT_INIT + +dnl sets $bitcoin_enable_qt, $bitcoin_enable_qt_test, $bitcoin_enable_qt_dbus +BITCOIN_QT_CONFIGURE([$use_pkgconfig], [qt4]) + +if test x$build_bitcoin_utils$build_bitcoind$bitcoin_enable_qt$use_tests = xnononono; then + use_boost=no +else + use_boost=yes +fi + +if test x$use_boost = xyes; then + +dnl Check for boost libs +AX_BOOST_BASE +AX_BOOST_SYSTEM +AX_BOOST_FILESYSTEM +AX_BOOST_PROGRAM_OPTIONS +AX_BOOST_THREAD +AX_BOOST_CHRONO + + +if test x$use_reduce_exports != xno; then + AC_MSG_CHECKING([for working boost reduced exports]) + TEMP_CPPFLAGS="$CPPFLAGS" + CPPFLAGS="$BOOST_CPPFLAGS $CPPFLAGS" + AC_PREPROC_IFELSE([AC_LANG_PROGRAM([[ + @%:@include + ]], [[ + #if BOOST_VERSION >= 104900 + // Everything is okay + #else + # error Boost version is too old + #endif + ]])],[ + AC_MSG_RESULT(yes) + ],[: + if test x$use_reduce_exports = xauto; then + use_reduce_exports=no + else + if test x$use_reduce_exports = xyes; then + AC_MSG_ERROR([boost versions < 1.49 are known to be broken with reduced exports. Use --disable-reduced-exports.]) + fi + fi + AC_MSG_RESULT(no) + AC_MSG_WARN([boost versions < 1.49 are known to have symbol visibility issues. Disabling reduced exports.]) + ]) + CPPFLAGS="$TEMP_CPPFLAGS" +fi + +elif test x$use_reduce_exports = xauto; then + use_reduce_exports=yes +fi + +if test x$use_reduce_exports != xno; then + CXXFLAGS="$CXXFLAGS $RE_CXXFLAGS" + AX_CHECK_LINK_FLAG([[-Wl,--exclude-libs,ALL]], [RELDFLAGS="-Wl,--exclude-libs,ALL"]) +fi + +if test x$use_tests = xyes; then + + if test x$HEXDUMP = x; then + AC_MSG_ERROR(hexdump is required for tests) + fi + + + if test x$use_boost = xyes; then + + AX_BOOST_UNIT_TEST_FRAMEWORK + + dnl Determine if -DBOOST_TEST_DYN_LINK is needed + AC_MSG_CHECKING([for dynamic linked boost test]) + TEMP_LIBS="$LIBS" + LIBS="$LIBS $BOOST_LDFLAGS $BOOST_UNIT_TEST_FRAMEWORK_LIB" + TEMP_CPPFLAGS="$CPPFLAGS" + CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" + AC_LINK_IFELSE([AC_LANG_SOURCE([ + #define BOOST_TEST_DYN_LINK + #define BOOST_TEST_MAIN + #include + + ])], + [AC_MSG_RESULT(yes)] + [TESTDEFS="$TESTDEFS -DBOOST_TEST_DYN_LINK"], + [AC_MSG_RESULT(no)]) + LIBS="$TEMP_LIBS" + CPPFLAGS="$TEMP_CPPFLAGS" + + fi +fi + +if test x$use_boost = xyes; then + +BOOST_LIBS="$BOOST_LDFLAGS $BOOST_SYSTEM_LIB $BOOST_FILESYSTEM_LIB $BOOST_PROGRAM_OPTIONS_LIB $BOOST_THREAD_LIB" + +dnl Boost >= 1.50 uses sleep_for rather than the now-deprecated sleep, however +dnl it was broken from 1.50 to 1.52 when backed by nanosleep. Use sleep_for if +dnl a working version is available, else fall back to sleep. sleep was removed +dnl after 1.56. +dnl If neither is available, abort. +dnl If sleep_for is used, boost_chrono becomes a requirement. +if test x$ax_cv_boost_chrono = xyes; then +TEMP_LIBS="$LIBS" +LIBS="$BOOST_LIBS $BOOST_CHRONO_LIB $LIBS" +TEMP_CPPFLAGS="$CPPFLAGS" +CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" +AC_LINK_IFELSE([AC_LANG_PROGRAM([[ + #include + #include + ]],[[ + #if BOOST_VERSION >= 105000 && (!defined(BOOST_HAS_NANOSLEEP) || BOOST_VERSION >= 105200) + boost::this_thread::sleep_for(boost::chrono::milliseconds(0)); + #else + choke me + #endif + ]])], + [boost_sleep=yes; BOOST_LIBS="$BOOST_LIBS $BOOST_CHRONO_LIB"; + AC_DEFINE(HAVE_WORKING_BOOST_SLEEP_FOR, 1, [Define this symbol if boost sleep_for works])], + [boost_sleep=no]) +LIBS="$TEMP_LIBS" +CPPFLAGS="$TEMP_CPPFLAGS" +fi + +if test x$boost_sleep != xyes; then +TEMP_LIBS="$LIBS" +LIBS="$BOOST_LIBS $LIBS" +TEMP_CPPFLAGS="$CPPFLAGS" +CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" +AC_LINK_IFELSE([AC_LANG_PROGRAM([[ + #include + #include + #include + ]],[[ + #if BOOST_VERSION <= 105600 + boost::this_thread::sleep(boost::posix_time::milliseconds(0)); + #else + choke me + #endif + ]])], + [boost_sleep=yes; AC_DEFINE(HAVE_WORKING_BOOST_SLEEP, 1, [Define this symbol if boost sleep works])], + [boost_sleep=no]) +LIBS="$TEMP_LIBS" +CPPFLAGS="$TEMP_CPPFLAGS" +fi + +if test x$boost_sleep != xyes; then + AC_MSG_ERROR(No working boost sleep implementation found.) +fi + +fi + +if test x$use_pkgconfig = xyes; then + + if test x"$PKG_CONFIG" = "x"; then + AC_MSG_ERROR(pkg-config not found.) + fi + + : #NOP + m4_ifdef( + [PKG_CHECK_MODULES], + [ + PKG_CHECK_MODULES([SSL], [libssl],, [AC_MSG_ERROR(openssl not found.)]) + PKG_CHECK_MODULES([CRYPTO], [libcrypto],,[AC_MSG_ERROR(libcrypto not found.)]) + BITCOIN_QT_CHECK([PKG_CHECK_MODULES([PROTOBUF], [protobuf], [have_protobuf=yes], [BITCOIN_QT_FAIL(libprotobuf not found)])]) + if test x$use_qr != xno; then + BITCOIN_QT_CHECK([PKG_CHECK_MODULES([QR], [libqrencode], [have_qrencode=yes], [have_qrencode=no])]) + fi + ] + ) +else + AC_CHECK_HEADER([openssl/crypto.h],,AC_MSG_ERROR(libcrypto headers missing)) + AC_CHECK_LIB([crypto], [main],CRYPTO_LIBS=-lcrypto, AC_MSG_ERROR(libcrypto missing)) + + AC_CHECK_HEADER([openssl/ssl.h],, AC_MSG_ERROR(libssl headers missing),) + AC_CHECK_LIB([ssl], [main],SSL_LIBS=-lssl, AC_MSG_ERROR(libssl missing)) + + BITCOIN_QT_CHECK(AC_CHECK_LIB([protobuf] ,[main],[PROTOBUF_LIBS=-lprotobuf], BITCOIN_QT_FAIL(libprotobuf not found))) + if test x$use_qr != xno; then + BITCOIN_QT_CHECK([AC_CHECK_LIB([qrencode], [main],[QR_LIBS=-lqrencode], [have_qrencode=no])]) + BITCOIN_QT_CHECK([AC_CHECK_HEADER([qrencode.h],, have_qrencode=no)]) + fi +fi + +CFLAGS_TEMP="$CFLAGS" +LIBS_TEMP="$LIBS" +CFLAGS="$CFLAGS $SSL_CFLAGS $CRYPTO_CFLAGS" +LIBS="$LIBS $SSL_LIBS $CRYPTO_LIBS" +AC_CHECK_HEADER([openssl/ec.h],, AC_MSG_ERROR(OpenSSL ec header missing),) +CFLAGS="$CFLAGS_TEMP" +LIBS="$LIBS_TEMP" + +BITCOIN_QT_PATH_PROGS([PROTOC], [protoc],$protoc_bin_path) + +AC_MSG_CHECKING([whether to build bitcoind]) +AM_CONDITIONAL([BUILD_BITCOIND], [test x$build_bitcoind = xyes]) +AC_MSG_RESULT($build_bitcoind) + +AC_MSG_CHECKING([whether to build utils (bitcoin-cli bitcoin-tx)]) +AM_CONDITIONAL([BUILD_BITCOIN_UTILS], [test x$build_bitcoin_utils = xyes]) +AC_MSG_RESULT($build_bitcoin_utils) + +AC_MSG_CHECKING([whether to build libraries]) +AM_CONDITIONAL([BUILD_BITCOIN_LIBS], [test x$build_bitcoin_libs = xyes]) +if test x$build_bitcoin_libs = xyes; then + AC_DEFINE(HAVE_CONSENSUS_LIB, 1, [Define this symbol if the consensus lib has been built]) +fi +AC_MSG_RESULT($build_bitcoin_libs) + +AC_LANG_POP + +if test "x$use_ccache" != "xno"; then + AC_MSG_CHECKING(if ccache should be used) + if test x$CCACHE = x; then + if test "x$use_ccache" = "xyes"; then + AC_MSG_ERROR([ccache not found.]); + else + use_ccache=no + fi + else + use_ccache=yes + CC="$ac_cv_path_CCACHE $CC" + CXX="$ac_cv_path_CCACHE $CXX" + fi + AC_MSG_RESULT($use_ccache) +fi +if test "x$use_ccache" = "xyes"; then + AX_CHECK_PREPROC_FLAG([-Qunused-arguments],[CPPFLAGS="-Qunused-arguments $CPPFLAGS"]) +fi + +dnl enable wallet +AC_MSG_CHECKING([if wallet should be enabled]) +if test x$enable_wallet != xno; then + AC_MSG_RESULT(yes) + AC_DEFINE_UNQUOTED([ENABLE_WALLET],[1],[Define to 1 to enable wallet functions]) + +else + AC_MSG_RESULT(no) +fi + +dnl enable upnp support +AC_MSG_CHECKING([whether to build with support for UPnP]) +if test x$have_miniupnpc = xno; then + if test x$use_upnp = xyes; then + AC_MSG_ERROR("UPnP requested but cannot be built. use --without-miniupnpc") + fi + AC_MSG_RESULT(no) +else + if test x$use_upnp != xno; then + AC_MSG_RESULT(yes) + AC_MSG_CHECKING([whether to build with UPnP enabled by default]) + use_upnp=yes + upnp_setting=0 + if test x$use_upnp_default != xno; then + use_upnp_default=yes + upnp_setting=1 + fi + AC_MSG_RESULT($use_upnp_default) + AC_DEFINE_UNQUOTED([USE_UPNP],[$upnp_setting],[UPnP support not compiled if undefined, otherwise value (0 or 1) determines default state]) + if test x$TARGET_OS = xwindows; then + MINIUPNPC_CPPFLAGS="-DSTATICLIB -DMINIUPNP_STATICLIB" + fi + else + AC_MSG_RESULT(no) + fi +fi + +dnl these are only used when qt is enabled +if test x$bitcoin_enable_qt != xno; then + BUILD_QT=qt + dnl enable dbus support + AC_MSG_CHECKING([whether to build GUI with support for D-Bus]) + if test x$bitcoin_enable_qt_dbus != xno; then + AC_DEFINE([USE_DBUS],[1],[Define if dbus support should be compiled in]) + fi + AC_MSG_RESULT($bitcoin_enable_qt_dbus) + + dnl enable qr support + AC_MSG_CHECKING([whether to build GUI with support for QR codes]) + if test x$have_qrencode = xno; then + if test x$use_qr = xyes; then + AC_MSG_ERROR("QR support requested but cannot be built. use --without-qrencode") + fi + AC_MSG_RESULT(no) + else + if test x$use_qr != xno; then + AC_MSG_RESULT(yes) + AC_DEFINE([USE_QRCODE],[1],[Define if QR support should be compiled in]) + use_qr=yes + else + AC_MSG_RESULT(no) + fi + fi + + if test x$XGETTEXT = x; then + AC_MSG_WARN("xgettext is required to update qt translations") + fi + + AC_MSG_CHECKING([whether to build test_bitcoin-qt]) + if test x$use_tests$bitcoin_enable_qt_test = xyesyes; then + AC_MSG_RESULT([yes]) + BUILD_TEST_QT="test" + else + AC_MSG_RESULT([no]) + fi +fi + +AC_MSG_CHECKING([whether to build test_bitcoin]) +if test x$use_tests = xyes; then + AC_MSG_RESULT([yes]) + BUILD_TEST="test" +else + AC_MSG_RESULT([no]) +fi + +AC_MSG_CHECKING([whether to reduce exports]) +if test x$use_reduce_exports != xno; then + AC_MSG_RESULT([yes]) +else + AC_MSG_RESULT([no]) +fi + +if test x$build_bitcoin_utils$build_bitcoin_libs$build_bitcoind$bitcoin_enable_qt$use_tests = xnonononono; then + AC_MSG_ERROR([No targets! Please specify at least one of: --with-utils --with-libs --with-daemon --with-gui or --enable-tests]) +fi + +AM_CONDITIONAL([TARGET_DARWIN], [test x$TARGET_OS = xdarwin]) +AM_CONDITIONAL([BUILD_DARWIN], [test x$BUILD_OS = xdarwin]) +AM_CONDITIONAL([TARGET_WINDOWS], [test x$TARGET_OS = xwindows]) +AM_CONDITIONAL([ENABLE_WALLET],[test x$enable_wallet = xyes]) +AM_CONDITIONAL([ENABLE_TESTS],[test x$use_tests = xyes]) +AM_CONDITIONAL([ENABLE_QT],[test x$bitcoin_enable_qt = xyes]) +AM_CONDITIONAL([ENABLE_QT_TESTS],[test x$use_tests$bitcoin_enable_qt_test = xyesyes]) +AM_CONDITIONAL([USE_QRCODE], [test x$use_qr = xyes]) +AM_CONDITIONAL([USE_LCOV],[test x$use_lcov = xyes]) +AM_CONDITIONAL([USE_COMPARISON_TOOL],[test x$use_comparison_tool != xno]) +AM_CONDITIONAL([USE_COMPARISON_TOOL_REORG_TESTS],[test x$use_comparison_tool_reorg_test != xno]) +AM_CONDITIONAL([GLIBC_BACK_COMPAT],[test x$use_glibc_compat = xyes]) + +AC_DEFINE(CLIENT_VERSION_MAJOR, _CLIENT_VERSION_MAJOR, [Major version]) +AC_DEFINE(CLIENT_VERSION_MINOR, _CLIENT_VERSION_MINOR, [Minor version]) +AC_DEFINE(CLIENT_VERSION_REVISION, _CLIENT_VERSION_REVISION, [Build revision]) +AC_DEFINE(CLIENT_VERSION_BUILD, _CLIENT_VERSION_BUILD, [Version Build]) +AC_DEFINE(CLIENT_VERSION_IS_RELEASE, _CLIENT_VERSION_IS_RELEASE, [Version is release]) +AC_DEFINE(COPYRIGHT_YEAR, _COPYRIGHT_YEAR, [Version is release]) +AC_SUBST(CLIENT_VERSION_MAJOR, _CLIENT_VERSION_MAJOR) +AC_SUBST(CLIENT_VERSION_MINOR, _CLIENT_VERSION_MINOR) +AC_SUBST(CLIENT_VERSION_REVISION, _CLIENT_VERSION_REVISION) +AC_SUBST(CLIENT_VERSION_BUILD, _CLIENT_VERSION_BUILD) +AC_SUBST(CLIENT_VERSION_IS_RELEASE, _CLIENT_VERSION_IS_RELEASE) +AC_SUBST(COPYRIGHT_YEAR, _COPYRIGHT_YEAR) + +AC_SUBST(RELDFLAGS) +AC_SUBST(LIBTOOL_APP_LDFLAGS) +AC_SUBST(USE_UPNP) +AC_SUBST(USE_QRCODE) +AC_SUBST(BOOST_LIBS) +AC_SUBST(TESTDEFS) +AC_SUBST(LEVELDB_TARGET_FLAGS) +AC_SUBST(BUILD_TEST) +AC_SUBST(BUILD_QT) +AC_SUBST(BUILD_TEST_QT) +AC_SUBST(MINIUPNPC_CPPFLAGS) +AC_SUBST(MINIUPNPC_LIBS) +AC_CONFIG_FILES([Makefile src/Makefile]) + +dnl boost's m4 checks do something really nasty: they export these vars. As a +dnl result, they leak into secp256k1's configure and crazy things happen. +dnl Until this is fixed upstream and we've synced, we'll just un-export them. +CPPFLAGS_TEMP="$CPPFLAGS" +unset CPPFLAGS +CPPFLAGS="$CPPFLAGS_TEMP" + +LDFLAGS_TEMP="$LDFLAGS" +unset LDFLAGS +LDFLAGS="$LDFLAGS_TEMP" + +LIBS_TEMP="$LIBS" +unset LIBS +LIBS="$LIBS_TEMP" + +PKGCONFIG_PATH_TEMP="$PKG_CONFIG_PATH" +unset PKG_CONFIG_PATH +PKG_CONFIG_PATH="$PKGCONFIG_PATH_TEMP" + +PKGCONFIG_LIBDIR_TEMP="$PKG_CONFIG_LIBDIR" +unset PKG_CONFIG_LIBDIR +PKG_CONFIG_LIBDIR="$PKGCONFIG_LIBDIR_TEMP" + +ac_configure_args="${ac_configure_args} --disable-shared --with-pic --enable-module-recovery --with-bignum=no" +AC_CONFIG_SUBDIRS([src/secp256k1]) + +AC_OUTPUT + +dnl Taken from https://wiki.debian.org/RpathIssue +case $host in + *-*-linux-gnu) + AC_MSG_RESULT([Fixing libtool for -rpath problems.]) + sed < libtool > libtool-2 \ + 's/^hardcode_libdir_flag_spec.*$'/'hardcode_libdir_flag_spec=" -D__LIBTOOL_IS_A_FOOL__ "/' + mv libtool-2 libtool + chmod 755 libtool + ;; +esac diff --git a/depends/.gitignore b/depends/.gitignore new file mode 100644 index 00000000..82c48638 --- /dev/null +++ b/depends/.gitignore @@ -0,0 +1,5 @@ +SDKs/ +work/ +built/ +sources/ +config.site diff --git a/depends/Makefile b/depends/Makefile new file mode 100644 index 00000000..fb88c931 --- /dev/null +++ b/depends/Makefile @@ -0,0 +1,136 @@ +.NOTPARALLEL : + +SOURCES_PATH ?= $(BASEDIR)/sources +BASE_CACHE ?= $(BASEDIR)/built +SDK_PATH ?= $(BASEDIR)/SDKs +NO_QT ?= +NO_WALLET ?= +NO_UPNP ?= +USE_LINUX_STATIC_QT5 ?= +FALLBACK_DOWNLOAD_PATH ?= https://bitcoincore.org/depends-sources + +BUILD = $(shell ./config.guess) +HOST ?= $(BUILD) +PATCHES_PATH = $(BASEDIR)/patches +BASEDIR = $(CURDIR) +HASH_LENGTH:=11 +DOWNLOAD_CONNECT_TIMEOUT:=10 +DOWNLOAD_RETRIES:=3 + +host:=$(BUILD) +ifneq ($(HOST),) +host:=$(HOST) +host_toolchain:=$(HOST)- +endif + +ifneq ($(DEBUG),) +release_type=debug +else +release_type=release +endif + +base_build_dir=$(BASEDIR)/work/build +base_staging_dir=$(BASEDIR)/work/staging +base_download_dir=$(BASEDIR)/work/download +canonical_host:=$(shell ./config.sub $(HOST)) +build:=$(shell ./config.sub $(BUILD)) + +build_arch =$(firstword $(subst -, ,$(build))) +build_vendor=$(word 2,$(subst -, ,$(build))) +full_build_os:=$(subst $(build_arch)-$(build_vendor)-,,$(build)) +build_os:=$(findstring linux,$(full_build_os)) +build_os+=$(findstring darwin,$(full_build_os)) +build_os:=$(strip $(build_os)) +ifeq ($(build_os),) +build_os=$(full_build_os) +endif + +host_arch=$(firstword $(subst -, ,$(canonical_host))) +host_vendor=$(word 2,$(subst -, ,$(canonical_host))) +full_host_os:=$(subst $(host_arch)-$(host_vendor)-,,$(canonical_host)) +host_os:=$(findstring linux,$(full_host_os)) +host_os+=$(findstring darwin,$(full_host_os)) +host_os+=$(findstring mingw32,$(full_host_os)) +host_os:=$(strip $(host_os)) +ifeq ($(host_os),) +host_os=$(full_host_os) +endif + +$(host_arch)_$(host_os)_prefix=$(BASEDIR)/$(host) +$(host_arch)_$(host_os)_host=$(host) +host_prefix=$($(host_arch)_$(host_os)_prefix) +build_prefix=$(host_prefix)/native +build_host=$(build) + +AT_$(V):= +AT_:=@ +AT:=$(AT_$(V)) + +all: install + +include hosts/$(host_os).mk +include hosts/default.mk +include builders/$(build_os).mk +include builders/default.mk +include packages/packages.mk + +qt_packages_$(NO_QT) = $(qt_packages) $(qt_$(host_os)_packages) +qt_native_packages_$(NO_QT) = $(qt_native_packages) +wallet_packages_$(NO_WALLET) = $(wallet_packages) +upnp_packages_$(NO_UPNP) = $(upnp_packages) + +#packages += $($(host_arch)_$(host_os)_packages) $($(host_os)_packages) $(qt_packages_) $(wallet_packages_) $(upnp_packages_) +#native_packages += $($(host_arch)_$(host_os)_native_packages) $($(host_os)_native_packages) $(qt_native_packages_) +packages += $($(host_arch)_$(host_os)_packages) $($(host_os)_packages) $(wallet_packages_) $(upnp_packages_) +native_packages += $($(host_arch)_$(host_os)_native_packages) $($(host_os)_native_packages) +all_packages = $(packages) $(native_packages) + +meta_depends = Makefile funcs.mk builders/default.mk hosts/default.mk hosts/$(host_os).mk builders/$(build_os).mk + +$(host_arch)_$(host_os)_native_toolchain?=$($(host_os)_native_toolchain) + +include funcs.mk + +toolchain_path=$($($(host_arch)_$(host_os)_native_toolchain)_prefixbin) +final_build_id_long+=$(shell $(build_SHA256SUM) config.site.in) +final_build_id+=$(shell echo -n $(final_build_id_long) | $(build_SHA256SUM) | cut -c-$(HASH_LENGTH)) +$(host_prefix)/.stamp_$(final_build_id): | $(native_packages) $(packages) + $(AT)rm -rf $(@D) + $(AT)mkdir -p $(@D) + $(AT)echo copying packages: $| + $(AT)echo to: $(@D) + $(AT)cd $(@D); $(foreach package,$|, tar xf $($(package)_cached); ) + $(AT)touch $@ + +$(host_prefix)/share/config.site : config.site.in $(host_prefix)/.stamp_$(final_build_id) + $(AT)@mkdir -p $(@D) + $(AT)sed -e 's|@HOST@|$(host)|' \ + -e 's|@CC@|$(toolchain_path)$(host_CC)|' \ + -e 's|@CXX@|$(toolchain_path)$(host_CXX)|' \ + -e 's|@AR@|$(toolchain_path)$(host_AR)|' \ + -e 's|@RANLIB@|$(toolchain_path)$(host_RANLIB)|' \ + -e 's|@NM@|$(toolchain_path)$(host_NM)|' \ + -e 's|@STRIP@|$(toolchain_path)$(host_STRIP)|' \ + -e 's|@build_os@|$(build_os)|' \ + -e 's|@host_os@|$(host_os)|' \ + -e 's|@CFLAGS@|$(strip $(host_CFLAGS) $(host_$(release_type)_CFLAGS))|' \ + -e 's|@CXXFLAGS@|$(strip $(host_CXXFLAGS) $(host_$(release_type)_CXXFLAGS))|' \ + -e 's|@CPPFLAGS@|$(strip $(host_CPPFLAGS) $(host_$(release_type)_CPPFLAGS))|' \ + -e 's|@LDFLAGS@|$(strip $(host_LDFLAGS) $(host_$(release_type)_LDFLAGS))|' \ + -e 's|@no_qt@|$(NO_QT)|' \ + -e 's|@no_wallet@|$(NO_WALLET)|' \ + -e 's|@no_upnp@|$(NO_UPNP)|' \ + -e 's|@debug@|$(DEBUG)|' \ + $< > $@ + $(AT)touch $@ + +install: $(host_prefix)/share/config.site +download-one: $(all_sources) +download-osx: + @$(MAKE) -s HOST=x86_64-apple-darwin11 download-one +download-linux: + @$(MAKE) -s HOST=x86_64-unknown-linux-gnu download-one +download-win: + @$(MAKE) -s HOST=x86_64-w64-mingw32 download-one +download: download-osx download-linux download-win +.PHONY: install cached download-one download-osx download-linux download-win download diff --git a/depends/README b/depends/README new file mode 100644 index 00000000..fed2f9b5 --- /dev/null +++ b/depends/README @@ -0,0 +1,55 @@ +This is a system of building and caching dependencies necessary for building +Bitcoin. + +There are several features that make it different from most similar systems: + +- It is designed to be builder and host agnostic + +In theory, binaries for any target OS/architecture can be created, from a +builder running any OS/architecture. In practice, build-side tools must be +specified when the defaults don't fit, and packages must be ammended to work +on new hosts. For now, a build architecture of x86_64 is assumed, either on +Linux or OSX. + +- No reliance on timestamps + +File presence is used to determine what needs to be built. This makes the +results distributable and easily digestable by automated builders. + +- Each build only has its specified dependencies available at build-time. + +For each build, the sysroot is wiped and the (recursive) dependencies are +installed. This makes each build deterministic, since there will never be any +unknown files available to cause side-effects. + +- Each package is cached and only rebuilt as needed. + +Before building, a unique build-id is generated for each package. This id +consists of a hash of all files used to build the package (Makefiles, packages, +etc), and as well as a hash of the same data for each recursive dependency. If +any portion of a package's build recipe changes, it will be rebuilt as well as +any other package that depends on it. If any of the main makefiles (Makefile, +funcs.mk, etc) are changed, all packages will be rebuilt. After building, the +results are cached into a tarball that can be re-used and distributed. + +- Package build results are (relatively) deterministic. + +Each package is configured and patched so that it will yield the same +build-results with each consequent build, within a reasonable set of +constraints. Some things like timestamp insertion are unavoidable, and are +beyond the scope of this system. Additionally, the toolchain itself must be +capable of deterministic results. When revisions are properly bumped, a cached +build should represent an exact single payload. + +- Sources are fetched and verified automatically + +Each package must define its source location and checksum. The build will fail +if the fetched source does not match. Sources may be pre-seeded and/or cached +as desired. + +- Self-cleaning + +Build and staging dirs are wiped after use, and any previous version of a +cached result is removed following a successful build. Automated builders +should be able to build each revision and store the results with no further +intervention. diff --git a/depends/README.packages b/depends/README.packages new file mode 100644 index 00000000..5ab7ed7d --- /dev/null +++ b/depends/README.packages @@ -0,0 +1,128 @@ +Each recipe consists of 3 main parts: defining identifiers, setting build +variables, and defining build commands. + +The package "mylib" will be used here as an example + +General tips: +mylib_foo is written as $(package)_foo in order to make recipes more similar. + +Identifiers: +Each package is required to define at least these variables: + $(package)_version: + Version of the upstream library or program. If there is no version, a + placeholder such as 1.0 can be used. + $(package)_download_path: + Location of the upstream source, without the file-name. Usually http or + ftp. + $(package)_file_name: + The upstream source filename available at the download path. + $(package)_sha256_hash: + The sha256 hash of the upstream file + +These variables are optional: + $(package)_build_subdir: + cd to this dir before running configure/build/stage commands. + $(package)_download_file: + The file-name of the upstream source if it differs from how it should be + stored locally. This can be used to avoid storing file-names with strange + characters. + $(package)_dependencies: + Names of any other packages that this one depends on. + $(package)_patches: + Filenames of any patches needed to build the package + + +Build Variables: +After defining the main identifiers, build variables may be added or customized +before running the build commands. They should be added to a function called +$(package)_set_vars. For example: + +define $(package)_set_vars +... +endef + +Most variables can be prefixed with the host, architecture, or both, to make +the modifications specific to that case. For example: + + Universal: $(package)_cc=gcc + Linux only: $(package)_linux_cc=gcc + x86_64 only: $(package)_x86_64_cc = gcc + x86_64 linux only: $(package)_x86_64_linux_cc = gcc + +These variables may be set to override or append their default values. + $(package)_cc + $(package)_cxx + $(package)_objc + $(package)_objcxx + $(package)_ar + $(package)_ranlib + $(package)_libtool + $(package)_nm + $(package)_cflags + $(package)_cxxflags + $(package)_ldflags + $(package)_cppflags + $(package)_config_env + $(package)_build_env + $(package)_stage_env + $(package)_build_opts + $(package)_config_opts + +The *_env variables are used to add environment variables to the respective +commands. + +Many variables respect a debug/release suffix as well, in order to use them for +only the appropriate build config. For example: + $(package)_cflags_release = -O3 + $(package)_cflags_i686_debug = -g + $(package)_config_opts_release = --disable-debug + +These will be used in addition to the options that do not specify +debug/release. All builds are considered to be release unless DEBUG=1 is set by +the user. + +Other variables may be defined as needed. + +Build commands: + + For each build, a unique build dir and staging dir are created. For example, + work/build/mylib/1.0-1adac830f6e and work/staging/mylib/1.0-1adac830f6e. + + The following build commands are available for each recipe: + + $(package)_fetch_cmds: + Runs from: build dir + Fetch the source file. If undefined, it will be fetched and verified + against its hash. + $(package)_extract_cmds: + Runs from: build dir + Verify the source file against its hash and extract it. If undefined, the + source is assumed to be a tarball. + $(package)_preprocess_cmds: + Runs from: build dir/$(package)_build_subdir + Preprocess the source as necessary. If undefined, does nothing. + $(package)_config_cmds: + Runs from: build dir/$(package)_build_subdir + Configure the source. If undefined, does nothing. + $(package)_build_cmds: + Runs from: build dir/$(package)_build_subdir + Build the source. If undefined, does nothing. + $(package)_stage_cmds: + Runs from: build dir/$(package)_build_subdir + Stage the build results. If undefined, does nothing. + + The following variables are available for each recipe: + $(1)_staging_dir: package's destination sysroot path + $(1)_staging_prefix_dir: prefix path inside of the package's staging dir + $(1)_extract_dir: path to the package's extracted sources + $(1)_build_dir: path where configure/build/stage commands will be run + $(1)_patch_dir: path where the package's patches (if any) are found + +Notes on build commands: + +For packages built with autotools, $($(package)_autoconf) can be used in the +configure step to (usually) correctly configure automatically. Any +$($(package)_config_opts) will be appended. + +Most autotools projects can be properly staged using: + $(MAKE) DESTDIR=$($(package)_staging_dir) install diff --git a/depends/README.usage b/depends/README.usage new file mode 100644 index 00000000..f5aa5314 --- /dev/null +++ b/depends/README.usage @@ -0,0 +1,35 @@ +To build dependencies for the current arch+OS: + make +To build for another arch/OS: + make HOST=host-platform-triplet && make HOST=host-platform-triplet + (For example: make HOST=i686-w64-mingw32 -j4) + +A prefix will be generated that's suitable for plugging into Bitcoin's +configure. In the above example, a dir named i686-w64-mingw32 will be +created. To use it for Bitcoin: + +./configure --prefix=`pwd`/depends/i686-w64-mingw32 + +No other options are needed, the paths are automatically configured. + +Dependency Options: +The following can be set when running make: make FOO=bar + +SOURCES_PATH: downloaded sources will be placed here +BASE_CACHE: built packages will be placed here +SDK_PATH: Path where sdk's can be found (used by OSX) +FALLBACK_DOWNLOAD_PATH: If a source file can't be fetched, try here before giving up +NO_QT: Don't download/build/cache qt and its dependencies +NO_WALLET: Don't download/build/cache libs needed to enable the wallet +NO_UPNP: Don't download/build/cache packages needed for enabling upnp +DEBUG: disable some optimizations and enable more runtime checking +USE_LINUX_STATIC_QT5: Build a static qt5 rather than shared qt4. Linux only. + +If some packages are not built, for example 'make NO_WALLET=1', the appropriate +options will be passed to bitcoin's configure. In this case, --disable-wallet. + +Additional targets: +download: run 'make download' to fetch all sources without building them +download-osx: run 'make download-osx' to fetch all sources needed for osx builds +download-win: run 'make download-win' to fetch all sources needed for win builds +download-linux: run 'make download-linux' to fetch all sources needed for linux builds diff --git a/depends/builders/darwin.mk b/depends/builders/darwin.mk new file mode 100644 index 00000000..b366460e --- /dev/null +++ b/depends/builders/darwin.mk @@ -0,0 +1,22 @@ +build_darwin_CC: = $(shell xcrun -f clang) +build_darwin_CXX: = $(shell xcrun -f clang++) +build_darwin_AR: = $(shell xcrun -f ar) +build_darwin_RANLIB: = $(shell xcrun -f ranlib) +build_darwin_STRIP: = $(shell xcrun -f strip) +build_darwin_OTOOL: = $(shell xcrun -f otool) +build_darwin_NM: = $(shell xcrun -f nm) +build_darwin_INSTALL_NAME_TOOL:=$(shell xcrun -f install_name_tool) +build_darwin_SHA256SUM = shasum -a 256 +build_darwin_DOWNLOAD = curl --connect-timeout $(DOWNLOAD_CONNECT_TIMEOUT) --retry $(DOWNLOAD_RETRIES) -L -o + +#darwin host on darwin builder. overrides darwin host preferences. +darwin_CC=$(shell xcrun -f clang) -mmacosx-version-min=$(OSX_MIN_VERSION) +darwin_CXX:=$(shell xcrun -f clang++) -mmacosx-version-min=$(OSX_MIN_VERSION) +darwin_AR:=$(shell xcrun -f ar) +darwin_RANLIB:=$(shell xcrun -f ranlib) +darwin_STRIP:=$(shell xcrun -f strip) +darwin_LIBTOOL:=$(shell xcrun -f libtool) +darwin_OTOOL:=$(shell xcrun -f otool) +darwin_NM:=$(shell xcrun -f nm) +darwin_INSTALL_NAME_TOOL:=$(shell xcrun -f install_name_tool) +darwin_native_toolchain= diff --git a/depends/builders/default.mk b/depends/builders/default.mk new file mode 100644 index 00000000..f097db65 --- /dev/null +++ b/depends/builders/default.mk @@ -0,0 +1,20 @@ +default_build_CC = gcc +default_build_CXX = g++ +default_build_AR = ar +default_build_RANLIB = ranlib +default_build_STRIP = strip +default_build_NM = nm +default_build_OTOOL = otool +default_build_INSTALL_NAME_TOOL = install_name_tool + +define add_build_tool_func +build_$(build_os)_$1 ?= $$(default_build_$1) +build_$(build_arch)_$(build_os)_$1 ?= $$(build_$(build_os)_$1) +build_$1=$$(build_$(build_arch)_$(build_os)_$1) +endef +$(foreach var,CC CXX AR RANLIB NM STRIP SHA256SUM DOWNLOAD OTOOL INSTALL_NAME_TOOL,$(eval $(call add_build_tool_func,$(var)))) +define add_build_flags_func +build_$(build_arch)_$(build_os)_$1 += $(build_$(build_os)_$1) +build_$1=$$(build_$(build_arch)_$(build_os)_$1) +endef +$(foreach flags, CFLAGS CXXFLAGS LDFLAGS, $(eval $(call add_build_flags_func,$(flags)))) diff --git a/depends/builders/linux.mk b/depends/builders/linux.mk new file mode 100644 index 00000000..98d0e9de --- /dev/null +++ b/depends/builders/linux.mk @@ -0,0 +1,2 @@ +build_linux_SHA256SUM = sha256sum +build_linux_DOWNLOAD = wget --timeout=$(DOWNLOAD_CONNECT_TIMEOUT) --tries=$(DOWNLOAD_RETRIES) -nv -O diff --git a/depends/config.guess b/depends/config.guess new file mode 100755 index 00000000..1f5c50c0 --- /dev/null +++ b/depends/config.guess @@ -0,0 +1,1420 @@ +#! /bin/sh +# Attempt to guess a canonical system name. +# Copyright 1992-2014 Free Software Foundation, Inc. + +timestamp='2014-03-23' + +# This file is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, see . +# +# As a special exception to the GNU General Public License, if you +# distribute this file as part of a program that contains a +# configuration script generated by Autoconf, you may include it under +# the same distribution terms that you use for the rest of that +# program. This Exception is an additional permission under section 7 +# of the GNU General Public License, version 3 ("GPLv3"). +# +# Originally written by Per Bothner. +# +# You can get the latest version of this script from: +# http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD +# +# Please send patches with a ChangeLog entry to config-patches@gnu.org. + + +me=`echo "$0" | sed -e 's,.*/,,'` + +usage="\ +Usage: $0 [OPTION] + +Output the configuration name of the system \`$me' is run on. + +Operation modes: + -h, --help print this help, then exit + -t, --time-stamp print date of last modification, then exit + -v, --version print version number, then exit + +Report bugs and patches to ." + +version="\ +GNU config.guess ($timestamp) + +Originally written by Per Bothner. +Copyright 1992-2014 Free Software Foundation, Inc. + +This is free software; see the source for copying conditions. There is NO +warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." + +help=" +Try \`$me --help' for more information." + +# Parse command line +while test $# -gt 0 ; do + case $1 in + --time-stamp | --time* | -t ) + echo "$timestamp" ; exit ;; + --version | -v ) + echo "$version" ; exit ;; + --help | --h* | -h ) + echo "$usage"; exit ;; + -- ) # Stop option processing + shift; break ;; + - ) # Use stdin as input. + break ;; + -* ) + echo "$me: invalid option $1$help" >&2 + exit 1 ;; + * ) + break ;; + esac +done + +if test $# != 0; then + echo "$me: too many arguments$help" >&2 + exit 1 +fi + +trap 'exit 1' 1 2 15 + +# CC_FOR_BUILD -- compiler used by this script. Note that the use of a +# compiler to aid in system detection is discouraged as it requires +# temporary files to be created and, as you can see below, it is a +# headache to deal with in a portable fashion. + +# Historically, `CC_FOR_BUILD' used to be named `HOST_CC'. We still +# use `HOST_CC' if defined, but it is deprecated. + +# Portable tmp directory creation inspired by the Autoconf team. + +set_cc_for_build=' +trap "exitcode=\$?; (rm -f \$tmpfiles 2>/dev/null; rmdir \$tmp 2>/dev/null) && exit \$exitcode" 0 ; +trap "rm -f \$tmpfiles 2>/dev/null; rmdir \$tmp 2>/dev/null; exit 1" 1 2 13 15 ; +: ${TMPDIR=/tmp} ; + { tmp=`(umask 077 && mktemp -d "$TMPDIR/cgXXXXXX") 2>/dev/null` && test -n "$tmp" && test -d "$tmp" ; } || + { test -n "$RANDOM" && tmp=$TMPDIR/cg$$-$RANDOM && (umask 077 && mkdir $tmp) ; } || + { tmp=$TMPDIR/cg-$$ && (umask 077 && mkdir $tmp) && echo "Warning: creating insecure temp directory" >&2 ; } || + { echo "$me: cannot create a temporary directory in $TMPDIR" >&2 ; exit 1 ; } ; +dummy=$tmp/dummy ; +tmpfiles="$dummy.c $dummy.o $dummy.rel $dummy" ; +case $CC_FOR_BUILD,$HOST_CC,$CC in + ,,) echo "int x;" > $dummy.c ; + for c in cc gcc c89 c99 ; do + if ($c -c -o $dummy.o $dummy.c) >/dev/null 2>&1 ; then + CC_FOR_BUILD="$c"; break ; + fi ; + done ; + if test x"$CC_FOR_BUILD" = x ; then + CC_FOR_BUILD=no_compiler_found ; + fi + ;; + ,,*) CC_FOR_BUILD=$CC ;; + ,*,*) CC_FOR_BUILD=$HOST_CC ;; +esac ; set_cc_for_build= ;' + +# This is needed to find uname on a Pyramid OSx when run in the BSD universe. +# (ghazi@noc.rutgers.edu 1994-08-24) +if (test -f /.attbin/uname) >/dev/null 2>&1 ; then + PATH=$PATH:/.attbin ; export PATH +fi + +UNAME_MACHINE=`(uname -m) 2>/dev/null` || UNAME_MACHINE=unknown +UNAME_RELEASE=`(uname -r) 2>/dev/null` || UNAME_RELEASE=unknown +UNAME_SYSTEM=`(uname -s) 2>/dev/null` || UNAME_SYSTEM=unknown +UNAME_VERSION=`(uname -v) 2>/dev/null` || UNAME_VERSION=unknown + +case "${UNAME_SYSTEM}" in +Linux|GNU|GNU/*) + # If the system lacks a compiler, then just pick glibc. + # We could probably try harder. + LIBC=gnu + + eval $set_cc_for_build + cat <<-EOF > $dummy.c + #include + #if defined(__UCLIBC__) + LIBC=uclibc + #elif defined(__dietlibc__) + LIBC=dietlibc + #else + LIBC=gnu + #endif + EOF + eval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep '^LIBC' | sed 's, ,,g'` + ;; +esac + +# Note: order is significant - the case branches are not exclusive. + +case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" in + *:NetBSD:*:*) + # NetBSD (nbsd) targets should (where applicable) match one or + # more of the tuples: *-*-netbsdelf*, *-*-netbsdaout*, + # *-*-netbsdecoff* and *-*-netbsd*. For targets that recently + # switched to ELF, *-*-netbsd* would select the old + # object file format. This provides both forward + # compatibility and a consistent mechanism for selecting the + # object file format. + # + # Note: NetBSD doesn't particularly care about the vendor + # portion of the name. We always set it to "unknown". + sysctl="sysctl -n hw.machine_arch" + UNAME_MACHINE_ARCH=`(/sbin/$sysctl 2>/dev/null || \ + /usr/sbin/$sysctl 2>/dev/null || echo unknown)` + case "${UNAME_MACHINE_ARCH}" in + armeb) machine=armeb-unknown ;; + arm*) machine=arm-unknown ;; + sh3el) machine=shl-unknown ;; + sh3eb) machine=sh-unknown ;; + sh5el) machine=sh5le-unknown ;; + *) machine=${UNAME_MACHINE_ARCH}-unknown ;; + esac + # The Operating System including object format, if it has switched + # to ELF recently, or will in the future. + case "${UNAME_MACHINE_ARCH}" in + arm*|i386|m68k|ns32k|sh3*|sparc|vax) + eval $set_cc_for_build + if echo __ELF__ | $CC_FOR_BUILD -E - 2>/dev/null \ + | grep -q __ELF__ + then + # Once all utilities can be ECOFF (netbsdecoff) or a.out (netbsdaout). + # Return netbsd for either. FIX? + os=netbsd + else + os=netbsdelf + fi + ;; + *) + os=netbsd + ;; + esac + # The OS release + # Debian GNU/NetBSD machines have a different userland, and + # thus, need a distinct triplet. However, they do not need + # kernel version information, so it can be replaced with a + # suitable tag, in the style of linux-gnu. + case "${UNAME_VERSION}" in + Debian*) + release='-gnu' + ;; + *) + release=`echo ${UNAME_RELEASE}|sed -e 's/[-_].*/\./'` + ;; + esac + # Since CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM: + # contains redundant information, the shorter form: + # CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM is used. + echo "${machine}-${os}${release}" + exit ;; + *:Bitrig:*:*) + UNAME_MACHINE_ARCH=`arch | sed 's/Bitrig.//'` + echo ${UNAME_MACHINE_ARCH}-unknown-bitrig${UNAME_RELEASE} + exit ;; + *:OpenBSD:*:*) + UNAME_MACHINE_ARCH=`arch | sed 's/OpenBSD.//'` + echo ${UNAME_MACHINE_ARCH}-unknown-openbsd${UNAME_RELEASE} + exit ;; + *:ekkoBSD:*:*) + echo ${UNAME_MACHINE}-unknown-ekkobsd${UNAME_RELEASE} + exit ;; + *:SolidBSD:*:*) + echo ${UNAME_MACHINE}-unknown-solidbsd${UNAME_RELEASE} + exit ;; + macppc:MirBSD:*:*) + echo powerpc-unknown-mirbsd${UNAME_RELEASE} + exit ;; + *:MirBSD:*:*) + echo ${UNAME_MACHINE}-unknown-mirbsd${UNAME_RELEASE} + exit ;; + alpha:OSF1:*:*) + case $UNAME_RELEASE in + *4.0) + UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $3}'` + ;; + *5.*) + UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $4}'` + ;; + esac + # According to Compaq, /usr/sbin/psrinfo has been available on + # OSF/1 and Tru64 systems produced since 1995. I hope that + # covers most systems running today. This code pipes the CPU + # types through head -n 1, so we only detect the type of CPU 0. + ALPHA_CPU_TYPE=`/usr/sbin/psrinfo -v | sed -n -e 's/^ The alpha \(.*\) processor.*$/\1/p' | head -n 1` + case "$ALPHA_CPU_TYPE" in + "EV4 (21064)") + UNAME_MACHINE="alpha" ;; + "EV4.5 (21064)") + UNAME_MACHINE="alpha" ;; + "LCA4 (21066/21068)") + UNAME_MACHINE="alpha" ;; + "EV5 (21164)") + UNAME_MACHINE="alphaev5" ;; + "EV5.6 (21164A)") + UNAME_MACHINE="alphaev56" ;; + "EV5.6 (21164PC)") + UNAME_MACHINE="alphapca56" ;; + "EV5.7 (21164PC)") + UNAME_MACHINE="alphapca57" ;; + "EV6 (21264)") + UNAME_MACHINE="alphaev6" ;; + "EV6.7 (21264A)") + UNAME_MACHINE="alphaev67" ;; + "EV6.8CB (21264C)") + UNAME_MACHINE="alphaev68" ;; + "EV6.8AL (21264B)") + UNAME_MACHINE="alphaev68" ;; + "EV6.8CX (21264D)") + UNAME_MACHINE="alphaev68" ;; + "EV6.9A (21264/EV69A)") + UNAME_MACHINE="alphaev69" ;; + "EV7 (21364)") + UNAME_MACHINE="alphaev7" ;; + "EV7.9 (21364A)") + UNAME_MACHINE="alphaev79" ;; + esac + # A Pn.n version is a patched version. + # A Vn.n version is a released version. + # A Tn.n version is a released field test version. + # A Xn.n version is an unreleased experimental baselevel. + # 1.2 uses "1.2" for uname -r. + echo ${UNAME_MACHINE}-dec-osf`echo ${UNAME_RELEASE} | sed -e 's/^[PVTX]//' | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + # Reset EXIT trap before exiting to avoid spurious non-zero exit code. + exitcode=$? + trap '' 0 + exit $exitcode ;; + Alpha\ *:Windows_NT*:*) + # How do we know it's Interix rather than the generic POSIX subsystem? + # Should we change UNAME_MACHINE based on the output of uname instead + # of the specific Alpha model? + echo alpha-pc-interix + exit ;; + 21064:Windows_NT:50:3) + echo alpha-dec-winnt3.5 + exit ;; + Amiga*:UNIX_System_V:4.0:*) + echo m68k-unknown-sysv4 + exit ;; + *:[Aa]miga[Oo][Ss]:*:*) + echo ${UNAME_MACHINE}-unknown-amigaos + exit ;; + *:[Mm]orph[Oo][Ss]:*:*) + echo ${UNAME_MACHINE}-unknown-morphos + exit ;; + *:OS/390:*:*) + echo i370-ibm-openedition + exit ;; + *:z/VM:*:*) + echo s390-ibm-zvmoe + exit ;; + *:OS400:*:*) + echo powerpc-ibm-os400 + exit ;; + arm:RISC*:1.[012]*:*|arm:riscix:1.[012]*:*) + echo arm-acorn-riscix${UNAME_RELEASE} + exit ;; + arm*:riscos:*:*|arm*:RISCOS:*:*) + echo arm-unknown-riscos + exit ;; + SR2?01:HI-UX/MPP:*:* | SR8000:HI-UX/MPP:*:*) + echo hppa1.1-hitachi-hiuxmpp + exit ;; + Pyramid*:OSx*:*:* | MIS*:OSx*:*:* | MIS*:SMP_DC-OSx*:*:*) + # akee@wpdis03.wpafb.af.mil (Earle F. Ake) contributed MIS and NILE. + if test "`(/bin/universe) 2>/dev/null`" = att ; then + echo pyramid-pyramid-sysv3 + else + echo pyramid-pyramid-bsd + fi + exit ;; + NILE*:*:*:dcosx) + echo pyramid-pyramid-svr4 + exit ;; + DRS?6000:unix:4.0:6*) + echo sparc-icl-nx6 + exit ;; + DRS?6000:UNIX_SV:4.2*:7* | DRS?6000:isis:4.2*:7*) + case `/usr/bin/uname -p` in + sparc) echo sparc-icl-nx7; exit ;; + esac ;; + s390x:SunOS:*:*) + echo ${UNAME_MACHINE}-ibm-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` + exit ;; + sun4H:SunOS:5.*:*) + echo sparc-hal-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` + exit ;; + sun4*:SunOS:5.*:* | tadpole*:SunOS:5.*:*) + echo sparc-sun-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` + exit ;; + i86pc:AuroraUX:5.*:* | i86xen:AuroraUX:5.*:*) + echo i386-pc-auroraux${UNAME_RELEASE} + exit ;; + i86pc:SunOS:5.*:* | i86xen:SunOS:5.*:*) + eval $set_cc_for_build + SUN_ARCH="i386" + # If there is a compiler, see if it is configured for 64-bit objects. + # Note that the Sun cc does not turn __LP64__ into 1 like gcc does. + # This test works for both compilers. + if [ "$CC_FOR_BUILD" != 'no_compiler_found' ]; then + if (echo '#ifdef __amd64'; echo IS_64BIT_ARCH; echo '#endif') | \ + (CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | \ + grep IS_64BIT_ARCH >/dev/null + then + SUN_ARCH="x86_64" + fi + fi + echo ${SUN_ARCH}-pc-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` + exit ;; + sun4*:SunOS:6*:*) + # According to config.sub, this is the proper way to canonicalize + # SunOS6. Hard to guess exactly what SunOS6 will be like, but + # it's likely to be more like Solaris than SunOS4. + echo sparc-sun-solaris3`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` + exit ;; + sun4*:SunOS:*:*) + case "`/usr/bin/arch -k`" in + Series*|S4*) + UNAME_RELEASE=`uname -v` + ;; + esac + # Japanese Language versions have a version number like `4.1.3-JL'. + echo sparc-sun-sunos`echo ${UNAME_RELEASE}|sed -e 's/-/_/'` + exit ;; + sun3*:SunOS:*:*) + echo m68k-sun-sunos${UNAME_RELEASE} + exit ;; + sun*:*:4.2BSD:*) + UNAME_RELEASE=`(sed 1q /etc/motd | awk '{print substr($5,1,3)}') 2>/dev/null` + test "x${UNAME_RELEASE}" = "x" && UNAME_RELEASE=3 + case "`/bin/arch`" in + sun3) + echo m68k-sun-sunos${UNAME_RELEASE} + ;; + sun4) + echo sparc-sun-sunos${UNAME_RELEASE} + ;; + esac + exit ;; + aushp:SunOS:*:*) + echo sparc-auspex-sunos${UNAME_RELEASE} + exit ;; + # The situation for MiNT is a little confusing. The machine name + # can be virtually everything (everything which is not + # "atarist" or "atariste" at least should have a processor + # > m68000). The system name ranges from "MiNT" over "FreeMiNT" + # to the lowercase version "mint" (or "freemint"). Finally + # the system name "TOS" denotes a system which is actually not + # MiNT. But MiNT is downward compatible to TOS, so this should + # be no problem. + atarist[e]:*MiNT:*:* | atarist[e]:*mint:*:* | atarist[e]:*TOS:*:*) + echo m68k-atari-mint${UNAME_RELEASE} + exit ;; + atari*:*MiNT:*:* | atari*:*mint:*:* | atarist[e]:*TOS:*:*) + echo m68k-atari-mint${UNAME_RELEASE} + exit ;; + *falcon*:*MiNT:*:* | *falcon*:*mint:*:* | *falcon*:*TOS:*:*) + echo m68k-atari-mint${UNAME_RELEASE} + exit ;; + milan*:*MiNT:*:* | milan*:*mint:*:* | *milan*:*TOS:*:*) + echo m68k-milan-mint${UNAME_RELEASE} + exit ;; + hades*:*MiNT:*:* | hades*:*mint:*:* | *hades*:*TOS:*:*) + echo m68k-hades-mint${UNAME_RELEASE} + exit ;; + *:*MiNT:*:* | *:*mint:*:* | *:*TOS:*:*) + echo m68k-unknown-mint${UNAME_RELEASE} + exit ;; + m68k:machten:*:*) + echo m68k-apple-machten${UNAME_RELEASE} + exit ;; + powerpc:machten:*:*) + echo powerpc-apple-machten${UNAME_RELEASE} + exit ;; + RISC*:Mach:*:*) + echo mips-dec-mach_bsd4.3 + exit ;; + RISC*:ULTRIX:*:*) + echo mips-dec-ultrix${UNAME_RELEASE} + exit ;; + VAX*:ULTRIX*:*:*) + echo vax-dec-ultrix${UNAME_RELEASE} + exit ;; + 2020:CLIX:*:* | 2430:CLIX:*:*) + echo clipper-intergraph-clix${UNAME_RELEASE} + exit ;; + mips:*:*:UMIPS | mips:*:*:RISCos) + eval $set_cc_for_build + sed 's/^ //' << EOF >$dummy.c +#ifdef __cplusplus +#include /* for printf() prototype */ + int main (int argc, char *argv[]) { +#else + int main (argc, argv) int argc; char *argv[]; { +#endif + #if defined (host_mips) && defined (MIPSEB) + #if defined (SYSTYPE_SYSV) + printf ("mips-mips-riscos%ssysv\n", argv[1]); exit (0); + #endif + #if defined (SYSTYPE_SVR4) + printf ("mips-mips-riscos%ssvr4\n", argv[1]); exit (0); + #endif + #if defined (SYSTYPE_BSD43) || defined(SYSTYPE_BSD) + printf ("mips-mips-riscos%sbsd\n", argv[1]); exit (0); + #endif + #endif + exit (-1); + } +EOF + $CC_FOR_BUILD -o $dummy $dummy.c && + dummyarg=`echo "${UNAME_RELEASE}" | sed -n 's/\([0-9]*\).*/\1/p'` && + SYSTEM_NAME=`$dummy $dummyarg` && + { echo "$SYSTEM_NAME"; exit; } + echo mips-mips-riscos${UNAME_RELEASE} + exit ;; + Motorola:PowerMAX_OS:*:*) + echo powerpc-motorola-powermax + exit ;; + Motorola:*:4.3:PL8-*) + echo powerpc-harris-powermax + exit ;; + Night_Hawk:*:*:PowerMAX_OS | Synergy:PowerMAX_OS:*:*) + echo powerpc-harris-powermax + exit ;; + Night_Hawk:Power_UNIX:*:*) + echo powerpc-harris-powerunix + exit ;; + m88k:CX/UX:7*:*) + echo m88k-harris-cxux7 + exit ;; + m88k:*:4*:R4*) + echo m88k-motorola-sysv4 + exit ;; + m88k:*:3*:R3*) + echo m88k-motorola-sysv3 + exit ;; + AViiON:dgux:*:*) + # DG/UX returns AViiON for all architectures + UNAME_PROCESSOR=`/usr/bin/uname -p` + if [ $UNAME_PROCESSOR = mc88100 ] || [ $UNAME_PROCESSOR = mc88110 ] + then + if [ ${TARGET_BINARY_INTERFACE}x = m88kdguxelfx ] || \ + [ ${TARGET_BINARY_INTERFACE}x = x ] + then + echo m88k-dg-dgux${UNAME_RELEASE} + else + echo m88k-dg-dguxbcs${UNAME_RELEASE} + fi + else + echo i586-dg-dgux${UNAME_RELEASE} + fi + exit ;; + M88*:DolphinOS:*:*) # DolphinOS (SVR3) + echo m88k-dolphin-sysv3 + exit ;; + M88*:*:R3*:*) + # Delta 88k system running SVR3 + echo m88k-motorola-sysv3 + exit ;; + XD88*:*:*:*) # Tektronix XD88 system running UTekV (SVR3) + echo m88k-tektronix-sysv3 + exit ;; + Tek43[0-9][0-9]:UTek:*:*) # Tektronix 4300 system running UTek (BSD) + echo m68k-tektronix-bsd + exit ;; + *:IRIX*:*:*) + echo mips-sgi-irix`echo ${UNAME_RELEASE}|sed -e 's/-/_/g'` + exit ;; + ????????:AIX?:[12].1:2) # AIX 2.2.1 or AIX 2.1.1 is RT/PC AIX. + echo romp-ibm-aix # uname -m gives an 8 hex-code CPU id + exit ;; # Note that: echo "'`uname -s`'" gives 'AIX ' + i*86:AIX:*:*) + echo i386-ibm-aix + exit ;; + ia64:AIX:*:*) + if [ -x /usr/bin/oslevel ] ; then + IBM_REV=`/usr/bin/oslevel` + else + IBM_REV=${UNAME_VERSION}.${UNAME_RELEASE} + fi + echo ${UNAME_MACHINE}-ibm-aix${IBM_REV} + exit ;; + *:AIX:2:3) + if grep bos325 /usr/include/stdio.h >/dev/null 2>&1; then + eval $set_cc_for_build + sed 's/^ //' << EOF >$dummy.c + #include + + main() + { + if (!__power_pc()) + exit(1); + puts("powerpc-ibm-aix3.2.5"); + exit(0); + } +EOF + if $CC_FOR_BUILD -o $dummy $dummy.c && SYSTEM_NAME=`$dummy` + then + echo "$SYSTEM_NAME" + else + echo rs6000-ibm-aix3.2.5 + fi + elif grep bos324 /usr/include/stdio.h >/dev/null 2>&1; then + echo rs6000-ibm-aix3.2.4 + else + echo rs6000-ibm-aix3.2 + fi + exit ;; + *:AIX:*:[4567]) + IBM_CPU_ID=`/usr/sbin/lsdev -C -c processor -S available | sed 1q | awk '{ print $1 }'` + if /usr/sbin/lsattr -El ${IBM_CPU_ID} | grep ' POWER' >/dev/null 2>&1; then + IBM_ARCH=rs6000 + else + IBM_ARCH=powerpc + fi + if [ -x /usr/bin/oslevel ] ; then + IBM_REV=`/usr/bin/oslevel` + else + IBM_REV=${UNAME_VERSION}.${UNAME_RELEASE} + fi + echo ${IBM_ARCH}-ibm-aix${IBM_REV} + exit ;; + *:AIX:*:*) + echo rs6000-ibm-aix + exit ;; + ibmrt:4.4BSD:*|romp-ibm:BSD:*) + echo romp-ibm-bsd4.4 + exit ;; + ibmrt:*BSD:*|romp-ibm:BSD:*) # covers RT/PC BSD and + echo romp-ibm-bsd${UNAME_RELEASE} # 4.3 with uname added to + exit ;; # report: romp-ibm BSD 4.3 + *:BOSX:*:*) + echo rs6000-bull-bosx + exit ;; + DPX/2?00:B.O.S.:*:*) + echo m68k-bull-sysv3 + exit ;; + 9000/[34]??:4.3bsd:1.*:*) + echo m68k-hp-bsd + exit ;; + hp300:4.4BSD:*:* | 9000/[34]??:4.3bsd:2.*:*) + echo m68k-hp-bsd4.4 + exit ;; + 9000/[34678]??:HP-UX:*:*) + HPUX_REV=`echo ${UNAME_RELEASE}|sed -e 's/[^.]*.[0B]*//'` + case "${UNAME_MACHINE}" in + 9000/31? ) HP_ARCH=m68000 ;; + 9000/[34]?? ) HP_ARCH=m68k ;; + 9000/[678][0-9][0-9]) + if [ -x /usr/bin/getconf ]; then + sc_cpu_version=`/usr/bin/getconf SC_CPU_VERSION 2>/dev/null` + sc_kernel_bits=`/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null` + case "${sc_cpu_version}" in + 523) HP_ARCH="hppa1.0" ;; # CPU_PA_RISC1_0 + 528) HP_ARCH="hppa1.1" ;; # CPU_PA_RISC1_1 + 532) # CPU_PA_RISC2_0 + case "${sc_kernel_bits}" in + 32) HP_ARCH="hppa2.0n" ;; + 64) HP_ARCH="hppa2.0w" ;; + '') HP_ARCH="hppa2.0" ;; # HP-UX 10.20 + esac ;; + esac + fi + if [ "${HP_ARCH}" = "" ]; then + eval $set_cc_for_build + sed 's/^ //' << EOF >$dummy.c + + #define _HPUX_SOURCE + #include + #include + + int main () + { + #if defined(_SC_KERNEL_BITS) + long bits = sysconf(_SC_KERNEL_BITS); + #endif + long cpu = sysconf (_SC_CPU_VERSION); + + switch (cpu) + { + case CPU_PA_RISC1_0: puts ("hppa1.0"); break; + case CPU_PA_RISC1_1: puts ("hppa1.1"); break; + case CPU_PA_RISC2_0: + #if defined(_SC_KERNEL_BITS) + switch (bits) + { + case 64: puts ("hppa2.0w"); break; + case 32: puts ("hppa2.0n"); break; + default: puts ("hppa2.0"); break; + } break; + #else /* !defined(_SC_KERNEL_BITS) */ + puts ("hppa2.0"); break; + #endif + default: puts ("hppa1.0"); break; + } + exit (0); + } +EOF + (CCOPTS= $CC_FOR_BUILD -o $dummy $dummy.c 2>/dev/null) && HP_ARCH=`$dummy` + test -z "$HP_ARCH" && HP_ARCH=hppa + fi ;; + esac + if [ ${HP_ARCH} = "hppa2.0w" ] + then + eval $set_cc_for_build + + # hppa2.0w-hp-hpux* has a 64-bit kernel and a compiler generating + # 32-bit code. hppa64-hp-hpux* has the same kernel and a compiler + # generating 64-bit code. GNU and HP use different nomenclature: + # + # $ CC_FOR_BUILD=cc ./config.guess + # => hppa2.0w-hp-hpux11.23 + # $ CC_FOR_BUILD="cc +DA2.0w" ./config.guess + # => hppa64-hp-hpux11.23 + + if echo __LP64__ | (CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | + grep -q __LP64__ + then + HP_ARCH="hppa2.0w" + else + HP_ARCH="hppa64" + fi + fi + echo ${HP_ARCH}-hp-hpux${HPUX_REV} + exit ;; + ia64:HP-UX:*:*) + HPUX_REV=`echo ${UNAME_RELEASE}|sed -e 's/[^.]*.[0B]*//'` + echo ia64-hp-hpux${HPUX_REV} + exit ;; + 3050*:HI-UX:*:*) + eval $set_cc_for_build + sed 's/^ //' << EOF >$dummy.c + #include + int + main () + { + long cpu = sysconf (_SC_CPU_VERSION); + /* The order matters, because CPU_IS_HP_MC68K erroneously returns + true for CPU_PA_RISC1_0. CPU_IS_PA_RISC returns correct + results, however. */ + if (CPU_IS_PA_RISC (cpu)) + { + switch (cpu) + { + case CPU_PA_RISC1_0: puts ("hppa1.0-hitachi-hiuxwe2"); break; + case CPU_PA_RISC1_1: puts ("hppa1.1-hitachi-hiuxwe2"); break; + case CPU_PA_RISC2_0: puts ("hppa2.0-hitachi-hiuxwe2"); break; + default: puts ("hppa-hitachi-hiuxwe2"); break; + } + } + else if (CPU_IS_HP_MC68K (cpu)) + puts ("m68k-hitachi-hiuxwe2"); + else puts ("unknown-hitachi-hiuxwe2"); + exit (0); + } +EOF + $CC_FOR_BUILD -o $dummy $dummy.c && SYSTEM_NAME=`$dummy` && + { echo "$SYSTEM_NAME"; exit; } + echo unknown-hitachi-hiuxwe2 + exit ;; + 9000/7??:4.3bsd:*:* | 9000/8?[79]:4.3bsd:*:* ) + echo hppa1.1-hp-bsd + exit ;; + 9000/8??:4.3bsd:*:*) + echo hppa1.0-hp-bsd + exit ;; + *9??*:MPE/iX:*:* | *3000*:MPE/iX:*:*) + echo hppa1.0-hp-mpeix + exit ;; + hp7??:OSF1:*:* | hp8?[79]:OSF1:*:* ) + echo hppa1.1-hp-osf + exit ;; + hp8??:OSF1:*:*) + echo hppa1.0-hp-osf + exit ;; + i*86:OSF1:*:*) + if [ -x /usr/sbin/sysversion ] ; then + echo ${UNAME_MACHINE}-unknown-osf1mk + else + echo ${UNAME_MACHINE}-unknown-osf1 + fi + exit ;; + parisc*:Lites*:*:*) + echo hppa1.1-hp-lites + exit ;; + C1*:ConvexOS:*:* | convex:ConvexOS:C1*:*) + echo c1-convex-bsd + exit ;; + C2*:ConvexOS:*:* | convex:ConvexOS:C2*:*) + if getsysinfo -f scalar_acc + then echo c32-convex-bsd + else echo c2-convex-bsd + fi + exit ;; + C34*:ConvexOS:*:* | convex:ConvexOS:C34*:*) + echo c34-convex-bsd + exit ;; + C38*:ConvexOS:*:* | convex:ConvexOS:C38*:*) + echo c38-convex-bsd + exit ;; + C4*:ConvexOS:*:* | convex:ConvexOS:C4*:*) + echo c4-convex-bsd + exit ;; + CRAY*Y-MP:*:*:*) + echo ymp-cray-unicos${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' + exit ;; + CRAY*[A-Z]90:*:*:*) + echo ${UNAME_MACHINE}-cray-unicos${UNAME_RELEASE} \ + | sed -e 's/CRAY.*\([A-Z]90\)/\1/' \ + -e y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/ \ + -e 's/\.[^.]*$/.X/' + exit ;; + CRAY*TS:*:*:*) + echo t90-cray-unicos${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' + exit ;; + CRAY*T3E:*:*:*) + echo alphaev5-cray-unicosmk${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' + exit ;; + CRAY*SV1:*:*:*) + echo sv1-cray-unicos${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' + exit ;; + *:UNICOS/mp:*:*) + echo craynv-cray-unicosmp${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' + exit ;; + F30[01]:UNIX_System_V:*:* | F700:UNIX_System_V:*:*) + FUJITSU_PROC=`uname -m | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'` + FUJITSU_REL=`echo ${UNAME_RELEASE} | sed -e 's/ /_/'` + echo "${FUJITSU_PROC}-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}" + exit ;; + 5000:UNIX_System_V:4.*:*) + FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'` + FUJITSU_REL=`echo ${UNAME_RELEASE} | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/ /_/'` + echo "sparc-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}" + exit ;; + i*86:BSD/386:*:* | i*86:BSD/OS:*:* | *:Ascend\ Embedded/OS:*:*) + echo ${UNAME_MACHINE}-pc-bsdi${UNAME_RELEASE} + exit ;; + sparc*:BSD/OS:*:*) + echo sparc-unknown-bsdi${UNAME_RELEASE} + exit ;; + *:BSD/OS:*:*) + echo ${UNAME_MACHINE}-unknown-bsdi${UNAME_RELEASE} + exit ;; + *:FreeBSD:*:*) + UNAME_PROCESSOR=`/usr/bin/uname -p` + case ${UNAME_PROCESSOR} in + amd64) + echo x86_64-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` ;; + *) + echo ${UNAME_PROCESSOR}-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` ;; + esac + exit ;; + i*:CYGWIN*:*) + echo ${UNAME_MACHINE}-pc-cygwin + exit ;; + *:MINGW64*:*) + echo ${UNAME_MACHINE}-pc-mingw64 + exit ;; + *:MINGW*:*) + echo ${UNAME_MACHINE}-pc-mingw32 + exit ;; + *:MSYS*:*) + echo ${UNAME_MACHINE}-pc-msys + exit ;; + i*:windows32*:*) + # uname -m includes "-pc" on this system. + echo ${UNAME_MACHINE}-mingw32 + exit ;; + i*:PW*:*) + echo ${UNAME_MACHINE}-pc-pw32 + exit ;; + *:Interix*:*) + case ${UNAME_MACHINE} in + x86) + echo i586-pc-interix${UNAME_RELEASE} + exit ;; + authenticamd | genuineintel | EM64T) + echo x86_64-unknown-interix${UNAME_RELEASE} + exit ;; + IA64) + echo ia64-unknown-interix${UNAME_RELEASE} + exit ;; + esac ;; + [345]86:Windows_95:* | [345]86:Windows_98:* | [345]86:Windows_NT:*) + echo i${UNAME_MACHINE}-pc-mks + exit ;; + 8664:Windows_NT:*) + echo x86_64-pc-mks + exit ;; + i*:Windows_NT*:* | Pentium*:Windows_NT*:*) + # How do we know it's Interix rather than the generic POSIX subsystem? + # It also conflicts with pre-2.0 versions of AT&T UWIN. Should we + # UNAME_MACHINE based on the output of uname instead of i386? + echo i586-pc-interix + exit ;; + i*:UWIN*:*) + echo ${UNAME_MACHINE}-pc-uwin + exit ;; + amd64:CYGWIN*:*:* | x86_64:CYGWIN*:*:*) + echo x86_64-unknown-cygwin + exit ;; + p*:CYGWIN*:*) + echo powerpcle-unknown-cygwin + exit ;; + prep*:SunOS:5.*:*) + echo powerpcle-unknown-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` + exit ;; + *:GNU:*:*) + # the GNU system + echo `echo ${UNAME_MACHINE}|sed -e 's,[-/].*$,,'`-unknown-${LIBC}`echo ${UNAME_RELEASE}|sed -e 's,/.*$,,'` + exit ;; + *:GNU/*:*:*) + # other systems with GNU libc and userland + echo ${UNAME_MACHINE}-unknown-`echo ${UNAME_SYSTEM} | sed 's,^[^/]*/,,' | tr '[A-Z]' '[a-z]'``echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`-${LIBC} + exit ;; + i*86:Minix:*:*) + echo ${UNAME_MACHINE}-pc-minix + exit ;; + aarch64:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + aarch64_be:Linux:*:*) + UNAME_MACHINE=aarch64_be + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + alpha:Linux:*:*) + case `sed -n '/^cpu model/s/^.*: \(.*\)/\1/p' < /proc/cpuinfo` in + EV5) UNAME_MACHINE=alphaev5 ;; + EV56) UNAME_MACHINE=alphaev56 ;; + PCA56) UNAME_MACHINE=alphapca56 ;; + PCA57) UNAME_MACHINE=alphapca56 ;; + EV6) UNAME_MACHINE=alphaev6 ;; + EV67) UNAME_MACHINE=alphaev67 ;; + EV68*) UNAME_MACHINE=alphaev68 ;; + esac + objdump --private-headers /bin/sh | grep -q ld.so.1 + if test "$?" = 0 ; then LIBC="gnulibc1" ; fi + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + arc:Linux:*:* | arceb:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + arm*:Linux:*:*) + eval $set_cc_for_build + if echo __ARM_EABI__ | $CC_FOR_BUILD -E - 2>/dev/null \ + | grep -q __ARM_EABI__ + then + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + else + if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \ + | grep -q __ARM_PCS_VFP + then + echo ${UNAME_MACHINE}-unknown-linux-${LIBC}eabi + else + echo ${UNAME_MACHINE}-unknown-linux-${LIBC}eabihf + fi + fi + exit ;; + avr32*:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + cris:Linux:*:*) + echo ${UNAME_MACHINE}-axis-linux-${LIBC} + exit ;; + crisv32:Linux:*:*) + echo ${UNAME_MACHINE}-axis-linux-${LIBC} + exit ;; + frv:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + hexagon:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + i*86:Linux:*:*) + echo ${UNAME_MACHINE}-pc-linux-${LIBC} + exit ;; + ia64:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + m32r*:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + m68*:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + mips:Linux:*:* | mips64:Linux:*:*) + eval $set_cc_for_build + sed 's/^ //' << EOF >$dummy.c + #undef CPU + #undef ${UNAME_MACHINE} + #undef ${UNAME_MACHINE}el + #if defined(__MIPSEL__) || defined(__MIPSEL) || defined(_MIPSEL) || defined(MIPSEL) + CPU=${UNAME_MACHINE}el + #else + #if defined(__MIPSEB__) || defined(__MIPSEB) || defined(_MIPSEB) || defined(MIPSEB) + CPU=${UNAME_MACHINE} + #else + CPU= + #endif + #endif +EOF + eval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep '^CPU'` + test x"${CPU}" != x && { echo "${CPU}-unknown-linux-${LIBC}"; exit; } + ;; + openrisc*:Linux:*:*) + echo or1k-unknown-linux-${LIBC} + exit ;; + or32:Linux:*:* | or1k*:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + padre:Linux:*:*) + echo sparc-unknown-linux-${LIBC} + exit ;; + parisc64:Linux:*:* | hppa64:Linux:*:*) + echo hppa64-unknown-linux-${LIBC} + exit ;; + parisc:Linux:*:* | hppa:Linux:*:*) + # Look for CPU level + case `grep '^cpu[^a-z]*:' /proc/cpuinfo 2>/dev/null | cut -d' ' -f2` in + PA7*) echo hppa1.1-unknown-linux-${LIBC} ;; + PA8*) echo hppa2.0-unknown-linux-${LIBC} ;; + *) echo hppa-unknown-linux-${LIBC} ;; + esac + exit ;; + ppc64:Linux:*:*) + echo powerpc64-unknown-linux-${LIBC} + exit ;; + ppc:Linux:*:*) + echo powerpc-unknown-linux-${LIBC} + exit ;; + ppc64le:Linux:*:*) + echo powerpc64le-unknown-linux-${LIBC} + exit ;; + ppcle:Linux:*:*) + echo powerpcle-unknown-linux-${LIBC} + exit ;; + s390:Linux:*:* | s390x:Linux:*:*) + echo ${UNAME_MACHINE}-ibm-linux-${LIBC} + exit ;; + sh64*:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + sh*:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + sparc:Linux:*:* | sparc64:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + tile*:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + vax:Linux:*:*) + echo ${UNAME_MACHINE}-dec-linux-${LIBC} + exit ;; + x86_64:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + xtensa*:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; + i*86:DYNIX/ptx:4*:*) + # ptx 4.0 does uname -s correctly, with DYNIX/ptx in there. + # earlier versions are messed up and put the nodename in both + # sysname and nodename. + echo i386-sequent-sysv4 + exit ;; + i*86:UNIX_SV:4.2MP:2.*) + # Unixware is an offshoot of SVR4, but it has its own version + # number series starting with 2... + # I am not positive that other SVR4 systems won't match this, + # I just have to hope. -- rms. + # Use sysv4.2uw... so that sysv4* matches it. + echo ${UNAME_MACHINE}-pc-sysv4.2uw${UNAME_VERSION} + exit ;; + i*86:OS/2:*:*) + # If we were able to find `uname', then EMX Unix compatibility + # is probably installed. + echo ${UNAME_MACHINE}-pc-os2-emx + exit ;; + i*86:XTS-300:*:STOP) + echo ${UNAME_MACHINE}-unknown-stop + exit ;; + i*86:atheos:*:*) + echo ${UNAME_MACHINE}-unknown-atheos + exit ;; + i*86:syllable:*:*) + echo ${UNAME_MACHINE}-pc-syllable + exit ;; + i*86:LynxOS:2.*:* | i*86:LynxOS:3.[01]*:* | i*86:LynxOS:4.[02]*:*) + echo i386-unknown-lynxos${UNAME_RELEASE} + exit ;; + i*86:*DOS:*:*) + echo ${UNAME_MACHINE}-pc-msdosdjgpp + exit ;; + i*86:*:4.*:* | i*86:SYSTEM_V:4.*:*) + UNAME_REL=`echo ${UNAME_RELEASE} | sed 's/\/MP$//'` + if grep Novell /usr/include/link.h >/dev/null 2>/dev/null; then + echo ${UNAME_MACHINE}-univel-sysv${UNAME_REL} + else + echo ${UNAME_MACHINE}-pc-sysv${UNAME_REL} + fi + exit ;; + i*86:*:5:[678]*) + # UnixWare 7.x, OpenUNIX and OpenServer 6. + case `/bin/uname -X | grep "^Machine"` in + *486*) UNAME_MACHINE=i486 ;; + *Pentium) UNAME_MACHINE=i586 ;; + *Pent*|*Celeron) UNAME_MACHINE=i686 ;; + esac + echo ${UNAME_MACHINE}-unknown-sysv${UNAME_RELEASE}${UNAME_SYSTEM}${UNAME_VERSION} + exit ;; + i*86:*:3.2:*) + if test -f /usr/options/cb.name; then + UNAME_REL=`sed -n 's/.*Version //p' /dev/null >/dev/null ; then + UNAME_REL=`(/bin/uname -X|grep Release|sed -e 's/.*= //')` + (/bin/uname -X|grep i80486 >/dev/null) && UNAME_MACHINE=i486 + (/bin/uname -X|grep '^Machine.*Pentium' >/dev/null) \ + && UNAME_MACHINE=i586 + (/bin/uname -X|grep '^Machine.*Pent *II' >/dev/null) \ + && UNAME_MACHINE=i686 + (/bin/uname -X|grep '^Machine.*Pentium Pro' >/dev/null) \ + && UNAME_MACHINE=i686 + echo ${UNAME_MACHINE}-pc-sco$UNAME_REL + else + echo ${UNAME_MACHINE}-pc-sysv32 + fi + exit ;; + pc:*:*:*) + # Left here for compatibility: + # uname -m prints for DJGPP always 'pc', but it prints nothing about + # the processor, so we play safe by assuming i586. + # Note: whatever this is, it MUST be the same as what config.sub + # prints for the "djgpp" host, or else GDB configury will decide that + # this is a cross-build. + echo i586-pc-msdosdjgpp + exit ;; + Intel:Mach:3*:*) + echo i386-pc-mach3 + exit ;; + paragon:*:*:*) + echo i860-intel-osf1 + exit ;; + i860:*:4.*:*) # i860-SVR4 + if grep Stardent /usr/include/sys/uadmin.h >/dev/null 2>&1 ; then + echo i860-stardent-sysv${UNAME_RELEASE} # Stardent Vistra i860-SVR4 + else # Add other i860-SVR4 vendors below as they are discovered. + echo i860-unknown-sysv${UNAME_RELEASE} # Unknown i860-SVR4 + fi + exit ;; + mini*:CTIX:SYS*5:*) + # "miniframe" + echo m68010-convergent-sysv + exit ;; + mc68k:UNIX:SYSTEM5:3.51m) + echo m68k-convergent-sysv + exit ;; + M680?0:D-NIX:5.3:*) + echo m68k-diab-dnix + exit ;; + M68*:*:R3V[5678]*:*) + test -r /sysV68 && { echo 'm68k-motorola-sysv'; exit; } ;; + 3[345]??:*:4.0:3.0 | 3[34]??A:*:4.0:3.0 | 3[34]??,*:*:4.0:3.0 | 3[34]??/*:*:4.0:3.0 | 4400:*:4.0:3.0 | 4850:*:4.0:3.0 | SKA40:*:4.0:3.0 | SDS2:*:4.0:3.0 | SHG2:*:4.0:3.0 | S7501*:*:4.0:3.0) + OS_REL='' + test -r /etc/.relid \ + && OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid` + /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ + && { echo i486-ncr-sysv4.3${OS_REL}; exit; } + /bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \ + && { echo i586-ncr-sysv4.3${OS_REL}; exit; } ;; + 3[34]??:*:4.0:* | 3[34]??,*:*:4.0:*) + /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ + && { echo i486-ncr-sysv4; exit; } ;; + NCR*:*:4.2:* | MPRAS*:*:4.2:*) + OS_REL='.3' + test -r /etc/.relid \ + && OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid` + /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ + && { echo i486-ncr-sysv4.3${OS_REL}; exit; } + /bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \ + && { echo i586-ncr-sysv4.3${OS_REL}; exit; } + /bin/uname -p 2>/dev/null | /bin/grep pteron >/dev/null \ + && { echo i586-ncr-sysv4.3${OS_REL}; exit; } ;; + m68*:LynxOS:2.*:* | m68*:LynxOS:3.0*:*) + echo m68k-unknown-lynxos${UNAME_RELEASE} + exit ;; + mc68030:UNIX_System_V:4.*:*) + echo m68k-atari-sysv4 + exit ;; + TSUNAMI:LynxOS:2.*:*) + echo sparc-unknown-lynxos${UNAME_RELEASE} + exit ;; + rs6000:LynxOS:2.*:*) + echo rs6000-unknown-lynxos${UNAME_RELEASE} + exit ;; + PowerPC:LynxOS:2.*:* | PowerPC:LynxOS:3.[01]*:* | PowerPC:LynxOS:4.[02]*:*) + echo powerpc-unknown-lynxos${UNAME_RELEASE} + exit ;; + SM[BE]S:UNIX_SV:*:*) + echo mips-dde-sysv${UNAME_RELEASE} + exit ;; + RM*:ReliantUNIX-*:*:*) + echo mips-sni-sysv4 + exit ;; + RM*:SINIX-*:*:*) + echo mips-sni-sysv4 + exit ;; + *:SINIX-*:*:*) + if uname -p 2>/dev/null >/dev/null ; then + UNAME_MACHINE=`(uname -p) 2>/dev/null` + echo ${UNAME_MACHINE}-sni-sysv4 + else + echo ns32k-sni-sysv + fi + exit ;; + PENTIUM:*:4.0*:*) # Unisys `ClearPath HMP IX 4000' SVR4/MP effort + # says + echo i586-unisys-sysv4 + exit ;; + *:UNIX_System_V:4*:FTX*) + # From Gerald Hewes . + # How about differentiating between stratus architectures? -djm + echo hppa1.1-stratus-sysv4 + exit ;; + *:*:*:FTX*) + # From seanf@swdc.stratus.com. + echo i860-stratus-sysv4 + exit ;; + i*86:VOS:*:*) + # From Paul.Green@stratus.com. + echo ${UNAME_MACHINE}-stratus-vos + exit ;; + *:VOS:*:*) + # From Paul.Green@stratus.com. + echo hppa1.1-stratus-vos + exit ;; + mc68*:A/UX:*:*) + echo m68k-apple-aux${UNAME_RELEASE} + exit ;; + news*:NEWS-OS:6*:*) + echo mips-sony-newsos6 + exit ;; + R[34]000:*System_V*:*:* | R4000:UNIX_SYSV:*:* | R*000:UNIX_SV:*:*) + if [ -d /usr/nec ]; then + echo mips-nec-sysv${UNAME_RELEASE} + else + echo mips-unknown-sysv${UNAME_RELEASE} + fi + exit ;; + BeBox:BeOS:*:*) # BeOS running on hardware made by Be, PPC only. + echo powerpc-be-beos + exit ;; + BeMac:BeOS:*:*) # BeOS running on Mac or Mac clone, PPC only. + echo powerpc-apple-beos + exit ;; + BePC:BeOS:*:*) # BeOS running on Intel PC compatible. + echo i586-pc-beos + exit ;; + BePC:Haiku:*:*) # Haiku running on Intel PC compatible. + echo i586-pc-haiku + exit ;; + x86_64:Haiku:*:*) + echo x86_64-unknown-haiku + exit ;; + SX-4:SUPER-UX:*:*) + echo sx4-nec-superux${UNAME_RELEASE} + exit ;; + SX-5:SUPER-UX:*:*) + echo sx5-nec-superux${UNAME_RELEASE} + exit ;; + SX-6:SUPER-UX:*:*) + echo sx6-nec-superux${UNAME_RELEASE} + exit ;; + SX-7:SUPER-UX:*:*) + echo sx7-nec-superux${UNAME_RELEASE} + exit ;; + SX-8:SUPER-UX:*:*) + echo sx8-nec-superux${UNAME_RELEASE} + exit ;; + SX-8R:SUPER-UX:*:*) + echo sx8r-nec-superux${UNAME_RELEASE} + exit ;; + Power*:Rhapsody:*:*) + echo powerpc-apple-rhapsody${UNAME_RELEASE} + exit ;; + *:Rhapsody:*:*) + echo ${UNAME_MACHINE}-apple-rhapsody${UNAME_RELEASE} + exit ;; + *:Darwin:*:*) + UNAME_PROCESSOR=`uname -p` || UNAME_PROCESSOR=unknown + eval $set_cc_for_build + if test "$UNAME_PROCESSOR" = unknown ; then + UNAME_PROCESSOR=powerpc + fi + if test `echo "$UNAME_RELEASE" | sed -e 's/\..*//'` -le 10 ; then + if [ "$CC_FOR_BUILD" != 'no_compiler_found' ]; then + if (echo '#ifdef __LP64__'; echo IS_64BIT_ARCH; echo '#endif') | \ + (CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | \ + grep IS_64BIT_ARCH >/dev/null + then + case $UNAME_PROCESSOR in + i386) UNAME_PROCESSOR=x86_64 ;; + powerpc) UNAME_PROCESSOR=powerpc64 ;; + esac + fi + fi + elif test "$UNAME_PROCESSOR" = i386 ; then + # Avoid executing cc on OS X 10.9, as it ships with a stub + # that puts up a graphical alert prompting to install + # developer tools. Any system running Mac OS X 10.7 or + # later (Darwin 11 and later) is required to have a 64-bit + # processor. This is not true of the ARM version of Darwin + # that Apple uses in portable devices. + UNAME_PROCESSOR=x86_64 + fi + echo ${UNAME_PROCESSOR}-apple-darwin${UNAME_RELEASE} + exit ;; + *:procnto*:*:* | *:QNX:[0123456789]*:*) + UNAME_PROCESSOR=`uname -p` + if test "$UNAME_PROCESSOR" = "x86"; then + UNAME_PROCESSOR=i386 + UNAME_MACHINE=pc + fi + echo ${UNAME_PROCESSOR}-${UNAME_MACHINE}-nto-qnx${UNAME_RELEASE} + exit ;; + *:QNX:*:4*) + echo i386-pc-qnx + exit ;; + NEO-?:NONSTOP_KERNEL:*:*) + echo neo-tandem-nsk${UNAME_RELEASE} + exit ;; + NSE-*:NONSTOP_KERNEL:*:*) + echo nse-tandem-nsk${UNAME_RELEASE} + exit ;; + NSR-?:NONSTOP_KERNEL:*:*) + echo nsr-tandem-nsk${UNAME_RELEASE} + exit ;; + *:NonStop-UX:*:*) + echo mips-compaq-nonstopux + exit ;; + BS2000:POSIX*:*:*) + echo bs2000-siemens-sysv + exit ;; + DS/*:UNIX_System_V:*:*) + echo ${UNAME_MACHINE}-${UNAME_SYSTEM}-${UNAME_RELEASE} + exit ;; + *:Plan9:*:*) + # "uname -m" is not consistent, so use $cputype instead. 386 + # is converted to i386 for consistency with other x86 + # operating systems. + if test "$cputype" = "386"; then + UNAME_MACHINE=i386 + else + UNAME_MACHINE="$cputype" + fi + echo ${UNAME_MACHINE}-unknown-plan9 + exit ;; + *:TOPS-10:*:*) + echo pdp10-unknown-tops10 + exit ;; + *:TENEX:*:*) + echo pdp10-unknown-tenex + exit ;; + KS10:TOPS-20:*:* | KL10:TOPS-20:*:* | TYPE4:TOPS-20:*:*) + echo pdp10-dec-tops20 + exit ;; + XKL-1:TOPS-20:*:* | TYPE5:TOPS-20:*:*) + echo pdp10-xkl-tops20 + exit ;; + *:TOPS-20:*:*) + echo pdp10-unknown-tops20 + exit ;; + *:ITS:*:*) + echo pdp10-unknown-its + exit ;; + SEI:*:*:SEIUX) + echo mips-sei-seiux${UNAME_RELEASE} + exit ;; + *:DragonFly:*:*) + echo ${UNAME_MACHINE}-unknown-dragonfly`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` + exit ;; + *:*VMS:*:*) + UNAME_MACHINE=`(uname -p) 2>/dev/null` + case "${UNAME_MACHINE}" in + A*) echo alpha-dec-vms ; exit ;; + I*) echo ia64-dec-vms ; exit ;; + V*) echo vax-dec-vms ; exit ;; + esac ;; + *:XENIX:*:SysV) + echo i386-pc-xenix + exit ;; + i*86:skyos:*:*) + echo ${UNAME_MACHINE}-pc-skyos`echo ${UNAME_RELEASE}` | sed -e 's/ .*$//' + exit ;; + i*86:rdos:*:*) + echo ${UNAME_MACHINE}-pc-rdos + exit ;; + i*86:AROS:*:*) + echo ${UNAME_MACHINE}-pc-aros + exit ;; + x86_64:VMkernel:*:*) + echo ${UNAME_MACHINE}-unknown-esx + exit ;; +esac + +cat >&2 < in order to provide the needed +information to handle your system. + +config.guess timestamp = $timestamp + +uname -m = `(uname -m) 2>/dev/null || echo unknown` +uname -r = `(uname -r) 2>/dev/null || echo unknown` +uname -s = `(uname -s) 2>/dev/null || echo unknown` +uname -v = `(uname -v) 2>/dev/null || echo unknown` + +/usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null` +/bin/uname -X = `(/bin/uname -X) 2>/dev/null` + +hostinfo = `(hostinfo) 2>/dev/null` +/bin/universe = `(/bin/universe) 2>/dev/null` +/usr/bin/arch -k = `(/usr/bin/arch -k) 2>/dev/null` +/bin/arch = `(/bin/arch) 2>/dev/null` +/usr/bin/oslevel = `(/usr/bin/oslevel) 2>/dev/null` +/usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null` + +UNAME_MACHINE = ${UNAME_MACHINE} +UNAME_RELEASE = ${UNAME_RELEASE} +UNAME_SYSTEM = ${UNAME_SYSTEM} +UNAME_VERSION = ${UNAME_VERSION} +EOF + +exit 1 + +# Local variables: +# eval: (add-hook 'write-file-hooks 'time-stamp) +# time-stamp-start: "timestamp='" +# time-stamp-format: "%:y-%02m-%02d" +# time-stamp-end: "'" +# End: diff --git a/depends/config.site.in b/depends/config.site.in new file mode 100644 index 00000000..873f6601 --- /dev/null +++ b/depends/config.site.in @@ -0,0 +1,100 @@ +cross_compiling=maybe +host_alias=@HOST@ +ac_tool_prefix=${host_alias}- + +if test -z $with_boost; then + with_boost=$prefix +fi +if test -z $with_qt_plugindir; then + with_qt_plugindir=$prefix/plugins +fi +if test -z $with_qt_translationdir; then + with_qt_translationdir=$prefix/translations +fi +if test -z $with_qt_bindir; then + with_qt_bindir=$prefix/native/bin +fi +if test -z $with_protoc_bindir; then + with_protoc_bindir=$prefix/native/bin +fi +if test -z $with_comparison_tool; then + with_comparison_tool=$prefix/native/share/BitcoindComparisonTool_jar/BitcoindComparisonTool.jar +fi + + +if test -z $enable_wallet && test -n "@no_wallet@"; then + enable_wallet=no +fi + +if test -z $with_miniupnpc && test -n "@no_upnp@"; then + with_miniupnpc=no +fi + +if test -z $with_gui && test -n "@no_qt@"; then + with_gui=no +fi + +if test x@host_os@ = xdarwin; then + BREW=no + PORT=no +fi + +if test x@host_os@ = xmingw32; then + if test -z $with_qt_incdir; then + with_qt_incdir=$prefix/include + fi + if test -z $with_qt_libdir; then + with_qt_libdir=$prefix/lib + fi +fi + +PATH=$prefix/native/bin:$PATH +PKG_CONFIG="`which pkg-config` --static" + +# These two need to remain exported because pkg-config does not see them +# otherwise. That means they must be unexported at the end of configure.ac to +# avoid ruining the cache. Sigh. + +export PKG_CONFIG_LIBDIR=$prefix/lib/pkgconfig +export PKG_CONFIG_PATH=$prefix/share/pkgconfig + +CPPFLAGS="-I$prefix/include/ $CPPFLAGS" +LDFLAGS="-L$prefix/lib $LDFLAGS" + +CC="@CC@" +CXX="@CXX@" +OBJC="${CC}" +OBJCXX="${CXX}" +CCACHE=$prefix/native/bin/ccache + +if test -n "@AR@"; then + AR=@AR@ + ac_cv_path_ac_pt_AR=${AR} +fi + +if test -n "@RANLIB@"; then + RANLIB=@RANLIB@ + ac_cv_path_ac_pt_RANLIB=${RANLIB} +fi + +if test -n "@NM@"; then + NM=@NM@ + ac_cv_path_ac_pt_NM=${NM} +fi + +if test -n "@debug@"; then + enable_reduce_exports=no +fi + +if test -n "@CFLAGS@"; then + CFLAGS="@CFLAGS@ $CFLAGS" +fi +if test -n "@CXXFLAGS@"; then + CXXFLAGS="@CXXFLAGS@ $CXXFLAGS" +fi +if test -n "@CPPFLAGS@"; then + CPPFLAGS="@CPPFLAGS@ $CPPFLAGS" +fi +if test -n "@LDFLAGS@"; then + LDFLAGS="@LDFLAGS@ $LDFLAGS" +fi diff --git a/depends/config.sub b/depends/config.sub new file mode 100755 index 00000000..d654d03c --- /dev/null +++ b/depends/config.sub @@ -0,0 +1,1794 @@ +#! /bin/sh +# Configuration validation subroutine script. +# Copyright 1992-2014 Free Software Foundation, Inc. + +timestamp='2014-05-01' + +# This file is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, see . +# +# As a special exception to the GNU General Public License, if you +# distribute this file as part of a program that contains a +# configuration script generated by Autoconf, you may include it under +# the same distribution terms that you use for the rest of that +# program. This Exception is an additional permission under section 7 +# of the GNU General Public License, version 3 ("GPLv3"). + + +# Please send patches with a ChangeLog entry to config-patches@gnu.org. +# +# Configuration subroutine to validate and canonicalize a configuration type. +# Supply the specified configuration type as an argument. +# If it is invalid, we print an error message on stderr and exit with code 1. +# Otherwise, we print the canonical config type on stdout and succeed. + +# You can get the latest version of this script from: +# http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD + +# This file is supposed to be the same for all GNU packages +# and recognize all the CPU types, system types and aliases +# that are meaningful with *any* GNU software. +# Each package is responsible for reporting which valid configurations +# it does not support. The user should be able to distinguish +# a failure to support a valid configuration from a meaningless +# configuration. + +# The goal of this file is to map all the various variations of a given +# machine specification into a single specification in the form: +# CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM +# or in some cases, the newer four-part form: +# CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM +# It is wrong to echo any other type of specification. + +me=`echo "$0" | sed -e 's,.*/,,'` + +usage="\ +Usage: $0 [OPTION] CPU-MFR-OPSYS + $0 [OPTION] ALIAS + +Canonicalize a configuration name. + +Operation modes: + -h, --help print this help, then exit + -t, --time-stamp print date of last modification, then exit + -v, --version print version number, then exit + +Report bugs and patches to ." + +version="\ +GNU config.sub ($timestamp) + +Copyright 1992-2014 Free Software Foundation, Inc. + +This is free software; see the source for copying conditions. There is NO +warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." + +help=" +Try \`$me --help' for more information." + +# Parse command line +while test $# -gt 0 ; do + case $1 in + --time-stamp | --time* | -t ) + echo "$timestamp" ; exit ;; + --version | -v ) + echo "$version" ; exit ;; + --help | --h* | -h ) + echo "$usage"; exit ;; + -- ) # Stop option processing + shift; break ;; + - ) # Use stdin as input. + break ;; + -* ) + echo "$me: invalid option $1$help" + exit 1 ;; + + *local*) + # First pass through any local machine types. + echo $1 + exit ;; + + * ) + break ;; + esac +done + +case $# in + 0) echo "$me: missing argument$help" >&2 + exit 1;; + 1) ;; + *) echo "$me: too many arguments$help" >&2 + exit 1;; +esac + +# Separate what the user gave into CPU-COMPANY and OS or KERNEL-OS (if any). +# Here we must recognize all the valid KERNEL-OS combinations. +maybe_os=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\2/'` +case $maybe_os in + nto-qnx* | linux-gnu* | linux-android* | linux-dietlibc | linux-newlib* | \ + linux-musl* | linux-uclibc* | uclinux-uclibc* | uclinux-gnu* | kfreebsd*-gnu* | \ + knetbsd*-gnu* | netbsd*-gnu* | \ + kopensolaris*-gnu* | \ + storm-chaos* | os2-emx* | rtmk-nova*) + os=-$maybe_os + basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'` + ;; + android-linux) + os=-linux-android + basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'`-unknown + ;; + *) + basic_machine=`echo $1 | sed 's/-[^-]*$//'` + if [ $basic_machine != $1 ] + then os=`echo $1 | sed 's/.*-/-/'` + else os=; fi + ;; +esac + +### Let's recognize common machines as not being operating systems so +### that things like config.sub decstation-3100 work. We also +### recognize some manufacturers as not being operating systems, so we +### can provide default operating systems below. +case $os in + -sun*os*) + # Prevent following clause from handling this invalid input. + ;; + -dec* | -mips* | -sequent* | -encore* | -pc532* | -sgi* | -sony* | \ + -att* | -7300* | -3300* | -delta* | -motorola* | -sun[234]* | \ + -unicom* | -ibm* | -next | -hp | -isi* | -apollo | -altos* | \ + -convergent* | -ncr* | -news | -32* | -3600* | -3100* | -hitachi* |\ + -c[123]* | -convex* | -sun | -crds | -omron* | -dg | -ultra | -tti* | \ + -harris | -dolphin | -highlevel | -gould | -cbm | -ns | -masscomp | \ + -apple | -axis | -knuth | -cray | -microblaze*) + os= + basic_machine=$1 + ;; + -bluegene*) + os=-cnk + ;; + -sim | -cisco | -oki | -wec | -winbond) + os= + basic_machine=$1 + ;; + -scout) + ;; + -wrs) + os=-vxworks + basic_machine=$1 + ;; + -chorusos*) + os=-chorusos + basic_machine=$1 + ;; + -chorusrdb) + os=-chorusrdb + basic_machine=$1 + ;; + -hiux*) + os=-hiuxwe2 + ;; + -sco6) + os=-sco5v6 + basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` + ;; + -sco5) + os=-sco3.2v5 + basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` + ;; + -sco4) + os=-sco3.2v4 + basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` + ;; + -sco3.2.[4-9]*) + os=`echo $os | sed -e 's/sco3.2./sco3.2v/'` + basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` + ;; + -sco3.2v[4-9]*) + # Don't forget version if it is 3.2v4 or newer. + basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` + ;; + -sco5v6*) + # Don't forget version if it is 3.2v4 or newer. + basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` + ;; + -sco*) + os=-sco3.2v2 + basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` + ;; + -udk*) + basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` + ;; + -isc) + os=-isc2.2 + basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` + ;; + -clix*) + basic_machine=clipper-intergraph + ;; + -isc*) + basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` + ;; + -lynx*178) + os=-lynxos178 + ;; + -lynx*5) + os=-lynxos5 + ;; + -lynx*) + os=-lynxos + ;; + -ptx*) + basic_machine=`echo $1 | sed -e 's/86-.*/86-sequent/'` + ;; + -windowsnt*) + os=`echo $os | sed -e 's/windowsnt/winnt/'` + ;; + -psos*) + os=-psos + ;; + -mint | -mint[0-9]*) + basic_machine=m68k-atari + os=-mint + ;; +esac + +# Decode aliases for certain CPU-COMPANY combinations. +case $basic_machine in + # Recognize the basic CPU types without company name. + # Some are omitted here because they have special meanings below. + 1750a | 580 \ + | a29k \ + | aarch64 | aarch64_be \ + | alpha | alphaev[4-8] | alphaev56 | alphaev6[78] | alphapca5[67] \ + | alpha64 | alpha64ev[4-8] | alpha64ev56 | alpha64ev6[78] | alpha64pca5[67] \ + | am33_2.0 \ + | arc | arceb \ + | arm | arm[bl]e | arme[lb] | armv[2-8] | armv[3-8][lb] | armv7[arm] \ + | avr | avr32 \ + | be32 | be64 \ + | bfin \ + | c4x | c8051 | clipper \ + | d10v | d30v | dlx | dsp16xx \ + | epiphany \ + | fido | fr30 | frv \ + | h8300 | h8500 | hppa | hppa1.[01] | hppa2.0 | hppa2.0[nw] | hppa64 \ + | hexagon \ + | i370 | i860 | i960 | ia64 \ + | ip2k | iq2000 \ + | k1om \ + | le32 | le64 \ + | lm32 \ + | m32c | m32r | m32rle | m68000 | m68k | m88k \ + | maxq | mb | microblaze | microblazeel | mcore | mep | metag \ + | mips | mipsbe | mipseb | mipsel | mipsle \ + | mips16 \ + | mips64 | mips64el \ + | mips64octeon | mips64octeonel \ + | mips64orion | mips64orionel \ + | mips64r5900 | mips64r5900el \ + | mips64vr | mips64vrel \ + | mips64vr4100 | mips64vr4100el \ + | mips64vr4300 | mips64vr4300el \ + | mips64vr5000 | mips64vr5000el \ + | mips64vr5900 | mips64vr5900el \ + | mipsisa32 | mipsisa32el \ + | mipsisa32r2 | mipsisa32r2el \ + | mipsisa32r6 | mipsisa32r6el \ + | mipsisa64 | mipsisa64el \ + | mipsisa64r2 | mipsisa64r2el \ + | mipsisa64r6 | mipsisa64r6el \ + | mipsisa64sb1 | mipsisa64sb1el \ + | mipsisa64sr71k | mipsisa64sr71kel \ + | mipsr5900 | mipsr5900el \ + | mipstx39 | mipstx39el \ + | mn10200 | mn10300 \ + | moxie \ + | mt \ + | msp430 \ + | nds32 | nds32le | nds32be \ + | nios | nios2 | nios2eb | nios2el \ + | ns16k | ns32k \ + | open8 | or1k | or1knd | or32 \ + | pdp10 | pdp11 | pj | pjl \ + | powerpc | powerpc64 | powerpc64le | powerpcle \ + | pyramid \ + | rl78 | rx \ + | score \ + | sh | sh[1234] | sh[24]a | sh[24]aeb | sh[23]e | sh[34]eb | sheb | shbe | shle | sh[1234]le | sh3ele \ + | sh64 | sh64le \ + | sparc | sparc64 | sparc64b | sparc64v | sparc86x | sparclet | sparclite \ + | sparcv8 | sparcv9 | sparcv9b | sparcv9v \ + | spu \ + | tahoe | tic4x | tic54x | tic55x | tic6x | tic80 | tron \ + | ubicom32 \ + | v850 | v850e | v850e1 | v850e2 | v850es | v850e2v3 \ + | we32k \ + | x86 | xc16x | xstormy16 | xtensa \ + | z8k | z80) + basic_machine=$basic_machine-unknown + ;; + c54x) + basic_machine=tic54x-unknown + ;; + c55x) + basic_machine=tic55x-unknown + ;; + c6x) + basic_machine=tic6x-unknown + ;; + m6811 | m68hc11 | m6812 | m68hc12 | m68hcs12x | nvptx | picochip) + basic_machine=$basic_machine-unknown + os=-none + ;; + m88110 | m680[12346]0 | m683?2 | m68360 | m5200 | v70 | w65 | z8k) + ;; + ms1) + basic_machine=mt-unknown + ;; + + strongarm | thumb | xscale) + basic_machine=arm-unknown + ;; + xgate) + basic_machine=$basic_machine-unknown + os=-none + ;; + xscaleeb) + basic_machine=armeb-unknown + ;; + + xscaleel) + basic_machine=armel-unknown + ;; + + # We use `pc' rather than `unknown' + # because (1) that's what they normally are, and + # (2) the word "unknown" tends to confuse beginning users. + i*86 | x86_64) + basic_machine=$basic_machine-pc + ;; + # Object if more than one company name word. + *-*-*) + echo Invalid configuration \`$1\': machine \`$basic_machine\' not recognized 1>&2 + exit 1 + ;; + # Recognize the basic CPU types with company name. + 580-* \ + | a29k-* \ + | aarch64-* | aarch64_be-* \ + | alpha-* | alphaev[4-8]-* | alphaev56-* | alphaev6[78]-* \ + | alpha64-* | alpha64ev[4-8]-* | alpha64ev56-* | alpha64ev6[78]-* \ + | alphapca5[67]-* | alpha64pca5[67]-* | arc-* | arceb-* \ + | arm-* | armbe-* | armle-* | armeb-* | armv*-* \ + | avr-* | avr32-* \ + | be32-* | be64-* \ + | bfin-* | bs2000-* \ + | c[123]* | c30-* | [cjt]90-* | c4x-* \ + | c8051-* | clipper-* | craynv-* | cydra-* \ + | d10v-* | d30v-* | dlx-* \ + | elxsi-* \ + | f30[01]-* | f700-* | fido-* | fr30-* | frv-* | fx80-* \ + | h8300-* | h8500-* \ + | hppa-* | hppa1.[01]-* | hppa2.0-* | hppa2.0[nw]-* | hppa64-* \ + | hexagon-* \ + | i*86-* | i860-* | i960-* | ia64-* \ + | ip2k-* | iq2000-* \ + | k1om-* \ + | le32-* | le64-* \ + | lm32-* \ + | m32c-* | m32r-* | m32rle-* \ + | m68000-* | m680[012346]0-* | m68360-* | m683?2-* | m68k-* \ + | m88110-* | m88k-* | maxq-* | mcore-* | metag-* \ + | microblaze-* | microblazeel-* \ + | mips-* | mipsbe-* | mipseb-* | mipsel-* | mipsle-* \ + | mips16-* \ + | mips64-* | mips64el-* \ + | mips64octeon-* | mips64octeonel-* \ + | mips64orion-* | mips64orionel-* \ + | mips64r5900-* | mips64r5900el-* \ + | mips64vr-* | mips64vrel-* \ + | mips64vr4100-* | mips64vr4100el-* \ + | mips64vr4300-* | mips64vr4300el-* \ + | mips64vr5000-* | mips64vr5000el-* \ + | mips64vr5900-* | mips64vr5900el-* \ + | mipsisa32-* | mipsisa32el-* \ + | mipsisa32r2-* | mipsisa32r2el-* \ + | mipsisa32r6-* | mipsisa32r6el-* \ + | mipsisa64-* | mipsisa64el-* \ + | mipsisa64r2-* | mipsisa64r2el-* \ + | mipsisa64r6-* | mipsisa64r6el-* \ + | mipsisa64sb1-* | mipsisa64sb1el-* \ + | mipsisa64sr71k-* | mipsisa64sr71kel-* \ + | mipsr5900-* | mipsr5900el-* \ + | mipstx39-* | mipstx39el-* \ + | mmix-* \ + | mt-* \ + | msp430-* \ + | nds32-* | nds32le-* | nds32be-* \ + | nios-* | nios2-* | nios2eb-* | nios2el-* \ + | none-* | np1-* | ns16k-* | ns32k-* \ + | open8-* \ + | or1k*-* \ + | orion-* \ + | pdp10-* | pdp11-* | pj-* | pjl-* | pn-* | power-* \ + | powerpc-* | powerpc64-* | powerpc64le-* | powerpcle-* \ + | pyramid-* \ + | rl78-* | romp-* | rs6000-* | rx-* \ + | sh-* | sh[1234]-* | sh[24]a-* | sh[24]aeb-* | sh[23]e-* | sh[34]eb-* | sheb-* | shbe-* \ + | shle-* | sh[1234]le-* | sh3ele-* | sh64-* | sh64le-* \ + | sparc-* | sparc64-* | sparc64b-* | sparc64v-* | sparc86x-* | sparclet-* \ + | sparclite-* \ + | sparcv8-* | sparcv9-* | sparcv9b-* | sparcv9v-* | sv1-* | sx?-* \ + | tahoe-* \ + | tic30-* | tic4x-* | tic54x-* | tic55x-* | tic6x-* | tic80-* \ + | tile*-* \ + | tron-* \ + | ubicom32-* \ + | v850-* | v850e-* | v850e1-* | v850es-* | v850e2-* | v850e2v3-* \ + | vax-* \ + | we32k-* \ + | x86-* | x86_64-* | xc16x-* | xps100-* \ + | xstormy16-* | xtensa*-* \ + | ymp-* \ + | z8k-* | z80-*) + ;; + # Recognize the basic CPU types without company name, with glob match. + xtensa*) + basic_machine=$basic_machine-unknown + ;; + # Recognize the various machine names and aliases which stand + # for a CPU type and a company and sometimes even an OS. + 386bsd) + basic_machine=i386-unknown + os=-bsd + ;; + 3b1 | 7300 | 7300-att | att-7300 | pc7300 | safari | unixpc) + basic_machine=m68000-att + ;; + 3b*) + basic_machine=we32k-att + ;; + a29khif) + basic_machine=a29k-amd + os=-udi + ;; + abacus) + basic_machine=abacus-unknown + ;; + adobe68k) + basic_machine=m68010-adobe + os=-scout + ;; + alliant | fx80) + basic_machine=fx80-alliant + ;; + altos | altos3068) + basic_machine=m68k-altos + ;; + am29k) + basic_machine=a29k-none + os=-bsd + ;; + amd64) + basic_machine=x86_64-pc + ;; + amd64-*) + basic_machine=x86_64-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + amdahl) + basic_machine=580-amdahl + os=-sysv + ;; + amiga | amiga-*) + basic_machine=m68k-unknown + ;; + amigaos | amigados) + basic_machine=m68k-unknown + os=-amigaos + ;; + amigaunix | amix) + basic_machine=m68k-unknown + os=-sysv4 + ;; + apollo68) + basic_machine=m68k-apollo + os=-sysv + ;; + apollo68bsd) + basic_machine=m68k-apollo + os=-bsd + ;; + aros) + basic_machine=i386-pc + os=-aros + ;; + aux) + basic_machine=m68k-apple + os=-aux + ;; + balance) + basic_machine=ns32k-sequent + os=-dynix + ;; + blackfin) + basic_machine=bfin-unknown + os=-linux + ;; + blackfin-*) + basic_machine=bfin-`echo $basic_machine | sed 's/^[^-]*-//'` + os=-linux + ;; + bluegene*) + basic_machine=powerpc-ibm + os=-cnk + ;; + c54x-*) + basic_machine=tic54x-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + c55x-*) + basic_machine=tic55x-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + c6x-*) + basic_machine=tic6x-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + c90) + basic_machine=c90-cray + os=-unicos + ;; + cegcc) + basic_machine=arm-unknown + os=-cegcc + ;; + convex-c1) + basic_machine=c1-convex + os=-bsd + ;; + convex-c2) + basic_machine=c2-convex + os=-bsd + ;; + convex-c32) + basic_machine=c32-convex + os=-bsd + ;; + convex-c34) + basic_machine=c34-convex + os=-bsd + ;; + convex-c38) + basic_machine=c38-convex + os=-bsd + ;; + cray | j90) + basic_machine=j90-cray + os=-unicos + ;; + craynv) + basic_machine=craynv-cray + os=-unicosmp + ;; + cr16 | cr16-*) + basic_machine=cr16-unknown + os=-elf + ;; + crds | unos) + basic_machine=m68k-crds + ;; + crisv32 | crisv32-* | etraxfs*) + basic_machine=crisv32-axis + ;; + cris | cris-* | etrax*) + basic_machine=cris-axis + ;; + crx) + basic_machine=crx-unknown + os=-elf + ;; + da30 | da30-*) + basic_machine=m68k-da30 + ;; + decstation | decstation-3100 | pmax | pmax-* | pmin | dec3100 | decstatn) + basic_machine=mips-dec + ;; + decsystem10* | dec10*) + basic_machine=pdp10-dec + os=-tops10 + ;; + decsystem20* | dec20*) + basic_machine=pdp10-dec + os=-tops20 + ;; + delta | 3300 | motorola-3300 | motorola-delta \ + | 3300-motorola | delta-motorola) + basic_machine=m68k-motorola + ;; + delta88) + basic_machine=m88k-motorola + os=-sysv3 + ;; + dicos) + basic_machine=i686-pc + os=-dicos + ;; + djgpp) + basic_machine=i586-pc + os=-msdosdjgpp + ;; + dpx20 | dpx20-*) + basic_machine=rs6000-bull + os=-bosx + ;; + dpx2* | dpx2*-bull) + basic_machine=m68k-bull + os=-sysv3 + ;; + ebmon29k) + basic_machine=a29k-amd + os=-ebmon + ;; + elxsi) + basic_machine=elxsi-elxsi + os=-bsd + ;; + encore | umax | mmax) + basic_machine=ns32k-encore + ;; + es1800 | OSE68k | ose68k | ose | OSE) + basic_machine=m68k-ericsson + os=-ose + ;; + fx2800) + basic_machine=i860-alliant + ;; + genix) + basic_machine=ns32k-ns + ;; + gmicro) + basic_machine=tron-gmicro + os=-sysv + ;; + go32) + basic_machine=i386-pc + os=-go32 + ;; + h3050r* | hiux*) + basic_machine=hppa1.1-hitachi + os=-hiuxwe2 + ;; + h8300hms) + basic_machine=h8300-hitachi + os=-hms + ;; + h8300xray) + basic_machine=h8300-hitachi + os=-xray + ;; + h8500hms) + basic_machine=h8500-hitachi + os=-hms + ;; + harris) + basic_machine=m88k-harris + os=-sysv3 + ;; + hp300-*) + basic_machine=m68k-hp + ;; + hp300bsd) + basic_machine=m68k-hp + os=-bsd + ;; + hp300hpux) + basic_machine=m68k-hp + os=-hpux + ;; + hp3k9[0-9][0-9] | hp9[0-9][0-9]) + basic_machine=hppa1.0-hp + ;; + hp9k2[0-9][0-9] | hp9k31[0-9]) + basic_machine=m68000-hp + ;; + hp9k3[2-9][0-9]) + basic_machine=m68k-hp + ;; + hp9k6[0-9][0-9] | hp6[0-9][0-9]) + basic_machine=hppa1.0-hp + ;; + hp9k7[0-79][0-9] | hp7[0-79][0-9]) + basic_machine=hppa1.1-hp + ;; + hp9k78[0-9] | hp78[0-9]) + # FIXME: really hppa2.0-hp + basic_machine=hppa1.1-hp + ;; + hp9k8[67]1 | hp8[67]1 | hp9k80[24] | hp80[24] | hp9k8[78]9 | hp8[78]9 | hp9k893 | hp893) + # FIXME: really hppa2.0-hp + basic_machine=hppa1.1-hp + ;; + hp9k8[0-9][13679] | hp8[0-9][13679]) + basic_machine=hppa1.1-hp + ;; + hp9k8[0-9][0-9] | hp8[0-9][0-9]) + basic_machine=hppa1.0-hp + ;; + hppa-next) + os=-nextstep3 + ;; + hppaosf) + basic_machine=hppa1.1-hp + os=-osf + ;; + hppro) + basic_machine=hppa1.1-hp + os=-proelf + ;; + i370-ibm* | ibm*) + basic_machine=i370-ibm + ;; + i*86v32) + basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` + os=-sysv32 + ;; + i*86v4*) + basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` + os=-sysv4 + ;; + i*86v) + basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` + os=-sysv + ;; + i*86sol2) + basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` + os=-solaris2 + ;; + i386mach) + basic_machine=i386-mach + os=-mach + ;; + i386-vsta | vsta) + basic_machine=i386-unknown + os=-vsta + ;; + iris | iris4d) + basic_machine=mips-sgi + case $os in + -irix*) + ;; + *) + os=-irix4 + ;; + esac + ;; + isi68 | isi) + basic_machine=m68k-isi + os=-sysv + ;; + m68knommu) + basic_machine=m68k-unknown + os=-linux + ;; + m68knommu-*) + basic_machine=m68k-`echo $basic_machine | sed 's/^[^-]*-//'` + os=-linux + ;; + m88k-omron*) + basic_machine=m88k-omron + ;; + magnum | m3230) + basic_machine=mips-mips + os=-sysv + ;; + merlin) + basic_machine=ns32k-utek + os=-sysv + ;; + microblaze*) + basic_machine=microblaze-xilinx + ;; + mingw64) + basic_machine=x86_64-pc + os=-mingw64 + ;; + mingw32) + basic_machine=i686-pc + os=-mingw32 + ;; + mingw32ce) + basic_machine=arm-unknown + os=-mingw32ce + ;; + miniframe) + basic_machine=m68000-convergent + ;; + *mint | -mint[0-9]* | *MiNT | *MiNT[0-9]*) + basic_machine=m68k-atari + os=-mint + ;; + mips3*-*) + basic_machine=`echo $basic_machine | sed -e 's/mips3/mips64/'` + ;; + mips3*) + basic_machine=`echo $basic_machine | sed -e 's/mips3/mips64/'`-unknown + ;; + monitor) + basic_machine=m68k-rom68k + os=-coff + ;; + morphos) + basic_machine=powerpc-unknown + os=-morphos + ;; + msdos) + basic_machine=i386-pc + os=-msdos + ;; + ms1-*) + basic_machine=`echo $basic_machine | sed -e 's/ms1-/mt-/'` + ;; + msys) + basic_machine=i686-pc + os=-msys + ;; + mvs) + basic_machine=i370-ibm + os=-mvs + ;; + nacl) + basic_machine=le32-unknown + os=-nacl + ;; + ncr3000) + basic_machine=i486-ncr + os=-sysv4 + ;; + netbsd386) + basic_machine=i386-unknown + os=-netbsd + ;; + netwinder) + basic_machine=armv4l-rebel + os=-linux + ;; + news | news700 | news800 | news900) + basic_machine=m68k-sony + os=-newsos + ;; + news1000) + basic_machine=m68030-sony + os=-newsos + ;; + news-3600 | risc-news) + basic_machine=mips-sony + os=-newsos + ;; + necv70) + basic_machine=v70-nec + os=-sysv + ;; + next | m*-next ) + basic_machine=m68k-next + case $os in + -nextstep* ) + ;; + -ns2*) + os=-nextstep2 + ;; + *) + os=-nextstep3 + ;; + esac + ;; + nh3000) + basic_machine=m68k-harris + os=-cxux + ;; + nh[45]000) + basic_machine=m88k-harris + os=-cxux + ;; + nindy960) + basic_machine=i960-intel + os=-nindy + ;; + mon960) + basic_machine=i960-intel + os=-mon960 + ;; + nonstopux) + basic_machine=mips-compaq + os=-nonstopux + ;; + np1) + basic_machine=np1-gould + ;; + neo-tandem) + basic_machine=neo-tandem + ;; + nse-tandem) + basic_machine=nse-tandem + ;; + nsr-tandem) + basic_machine=nsr-tandem + ;; + op50n-* | op60c-*) + basic_machine=hppa1.1-oki + os=-proelf + ;; + openrisc | openrisc-*) + basic_machine=or32-unknown + ;; + os400) + basic_machine=powerpc-ibm + os=-os400 + ;; + OSE68000 | ose68000) + basic_machine=m68000-ericsson + os=-ose + ;; + os68k) + basic_machine=m68k-none + os=-os68k + ;; + pa-hitachi) + basic_machine=hppa1.1-hitachi + os=-hiuxwe2 + ;; + paragon) + basic_machine=i860-intel + os=-osf + ;; + parisc) + basic_machine=hppa-unknown + os=-linux + ;; + parisc-*) + basic_machine=hppa-`echo $basic_machine | sed 's/^[^-]*-//'` + os=-linux + ;; + pbd) + basic_machine=sparc-tti + ;; + pbb) + basic_machine=m68k-tti + ;; + pc532 | pc532-*) + basic_machine=ns32k-pc532 + ;; + pc98) + basic_machine=i386-pc + ;; + pc98-*) + basic_machine=i386-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + pentium | p5 | k5 | k6 | nexgen | viac3) + basic_machine=i586-pc + ;; + pentiumpro | p6 | 6x86 | athlon | athlon_*) + basic_machine=i686-pc + ;; + pentiumii | pentium2 | pentiumiii | pentium3) + basic_machine=i686-pc + ;; + pentium4) + basic_machine=i786-pc + ;; + pentium-* | p5-* | k5-* | k6-* | nexgen-* | viac3-*) + basic_machine=i586-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + pentiumpro-* | p6-* | 6x86-* | athlon-*) + basic_machine=i686-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + pentiumii-* | pentium2-* | pentiumiii-* | pentium3-*) + basic_machine=i686-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + pentium4-*) + basic_machine=i786-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + pn) + basic_machine=pn-gould + ;; + power) basic_machine=power-ibm + ;; + ppc | ppcbe) basic_machine=powerpc-unknown + ;; + ppc-* | ppcbe-*) + basic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + ppcle | powerpclittle | ppc-le | powerpc-little) + basic_machine=powerpcle-unknown + ;; + ppcle-* | powerpclittle-*) + basic_machine=powerpcle-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + ppc64) basic_machine=powerpc64-unknown + ;; + ppc64-*) basic_machine=powerpc64-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + ppc64le | powerpc64little | ppc64-le | powerpc64-little) + basic_machine=powerpc64le-unknown + ;; + ppc64le-* | powerpc64little-*) + basic_machine=powerpc64le-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + ps2) + basic_machine=i386-ibm + ;; + pw32) + basic_machine=i586-unknown + os=-pw32 + ;; + rdos | rdos64) + basic_machine=x86_64-pc + os=-rdos + ;; + rdos32) + basic_machine=i386-pc + os=-rdos + ;; + rom68k) + basic_machine=m68k-rom68k + os=-coff + ;; + rm[46]00) + basic_machine=mips-siemens + ;; + rtpc | rtpc-*) + basic_machine=romp-ibm + ;; + s390 | s390-*) + basic_machine=s390-ibm + ;; + s390x | s390x-*) + basic_machine=s390x-ibm + ;; + sa29200) + basic_machine=a29k-amd + os=-udi + ;; + sb1) + basic_machine=mipsisa64sb1-unknown + ;; + sb1el) + basic_machine=mipsisa64sb1el-unknown + ;; + sde) + basic_machine=mipsisa32-sde + os=-elf + ;; + sei) + basic_machine=mips-sei + os=-seiux + ;; + sequent) + basic_machine=i386-sequent + ;; + sh) + basic_machine=sh-hitachi + os=-hms + ;; + sh5el) + basic_machine=sh5le-unknown + ;; + sh64) + basic_machine=sh64-unknown + ;; + sparclite-wrs | simso-wrs) + basic_machine=sparclite-wrs + os=-vxworks + ;; + sps7) + basic_machine=m68k-bull + os=-sysv2 + ;; + spur) + basic_machine=spur-unknown + ;; + st2000) + basic_machine=m68k-tandem + ;; + stratus) + basic_machine=i860-stratus + os=-sysv4 + ;; + strongarm-* | thumb-*) + basic_machine=arm-`echo $basic_machine | sed 's/^[^-]*-//'` + ;; + sun2) + basic_machine=m68000-sun + ;; + sun2os3) + basic_machine=m68000-sun + os=-sunos3 + ;; + sun2os4) + basic_machine=m68000-sun + os=-sunos4 + ;; + sun3os3) + basic_machine=m68k-sun + os=-sunos3 + ;; + sun3os4) + basic_machine=m68k-sun + os=-sunos4 + ;; + sun4os3) + basic_machine=sparc-sun + os=-sunos3 + ;; + sun4os4) + basic_machine=sparc-sun + os=-sunos4 + ;; + sun4sol2) + basic_machine=sparc-sun + os=-solaris2 + ;; + sun3 | sun3-*) + basic_machine=m68k-sun + ;; + sun4) + basic_machine=sparc-sun + ;; + sun386 | sun386i | roadrunner) + basic_machine=i386-sun + ;; + sv1) + basic_machine=sv1-cray + os=-unicos + ;; + symmetry) + basic_machine=i386-sequent + os=-dynix + ;; + t3e) + basic_machine=alphaev5-cray + os=-unicos + ;; + t90) + basic_machine=t90-cray + os=-unicos + ;; + tile*) + basic_machine=$basic_machine-unknown + os=-linux-gnu + ;; + tx39) + basic_machine=mipstx39-unknown + ;; + tx39el) + basic_machine=mipstx39el-unknown + ;; + toad1) + basic_machine=pdp10-xkl + os=-tops20 + ;; + tower | tower-32) + basic_machine=m68k-ncr + ;; + tpf) + basic_machine=s390x-ibm + os=-tpf + ;; + udi29k) + basic_machine=a29k-amd + os=-udi + ;; + ultra3) + basic_machine=a29k-nyu + os=-sym1 + ;; + v810 | necv810) + basic_machine=v810-nec + os=-none + ;; + vaxv) + basic_machine=vax-dec + os=-sysv + ;; + vms) + basic_machine=vax-dec + os=-vms + ;; + vpp*|vx|vx-*) + basic_machine=f301-fujitsu + ;; + vxworks960) + basic_machine=i960-wrs + os=-vxworks + ;; + vxworks68) + basic_machine=m68k-wrs + os=-vxworks + ;; + vxworks29k) + basic_machine=a29k-wrs + os=-vxworks + ;; + w65*) + basic_machine=w65-wdc + os=-none + ;; + w89k-*) + basic_machine=hppa1.1-winbond + os=-proelf + ;; + xbox) + basic_machine=i686-pc + os=-mingw32 + ;; + xps | xps100) + basic_machine=xps100-honeywell + ;; + xscale-* | xscalee[bl]-*) + basic_machine=`echo $basic_machine | sed 's/^xscale/arm/'` + ;; + ymp) + basic_machine=ymp-cray + os=-unicos + ;; + z8k-*-coff) + basic_machine=z8k-unknown + os=-sim + ;; + z80-*-coff) + basic_machine=z80-unknown + os=-sim + ;; + none) + basic_machine=none-none + os=-none + ;; + +# Here we handle the default manufacturer of certain CPU types. It is in +# some cases the only manufacturer, in others, it is the most popular. + w89k) + basic_machine=hppa1.1-winbond + ;; + op50n) + basic_machine=hppa1.1-oki + ;; + op60c) + basic_machine=hppa1.1-oki + ;; + romp) + basic_machine=romp-ibm + ;; + mmix) + basic_machine=mmix-knuth + ;; + rs6000) + basic_machine=rs6000-ibm + ;; + vax) + basic_machine=vax-dec + ;; + pdp10) + # there are many clones, so DEC is not a safe bet + basic_machine=pdp10-unknown + ;; + pdp11) + basic_machine=pdp11-dec + ;; + we32k) + basic_machine=we32k-att + ;; + sh[1234] | sh[24]a | sh[24]aeb | sh[34]eb | sh[1234]le | sh[23]ele) + basic_machine=sh-unknown + ;; + sparc | sparcv8 | sparcv9 | sparcv9b | sparcv9v) + basic_machine=sparc-sun + ;; + cydra) + basic_machine=cydra-cydrome + ;; + orion) + basic_machine=orion-highlevel + ;; + orion105) + basic_machine=clipper-highlevel + ;; + mac | mpw | mac-mpw) + basic_machine=m68k-apple + ;; + pmac | pmac-mpw) + basic_machine=powerpc-apple + ;; + *-unknown) + # Make sure to match an already-canonicalized machine name. + ;; + *) + echo Invalid configuration \`$1\': machine \`$basic_machine\' not recognized 1>&2 + exit 1 + ;; +esac + +# Here we canonicalize certain aliases for manufacturers. +case $basic_machine in + *-digital*) + basic_machine=`echo $basic_machine | sed 's/digital.*/dec/'` + ;; + *-commodore*) + basic_machine=`echo $basic_machine | sed 's/commodore.*/cbm/'` + ;; + *) + ;; +esac + +# Decode manufacturer-specific aliases for certain operating systems. + +if [ x"$os" != x"" ] +then +case $os in + # First match some system type aliases + # that might get confused with valid system types. + # -solaris* is a basic system type, with this one exception. + -auroraux) + os=-auroraux + ;; + -solaris1 | -solaris1.*) + os=`echo $os | sed -e 's|solaris1|sunos4|'` + ;; + -solaris) + os=-solaris2 + ;; + -svr4*) + os=-sysv4 + ;; + -unixware*) + os=-sysv4.2uw + ;; + -gnu/linux*) + os=`echo $os | sed -e 's|gnu/linux|linux-gnu|'` + ;; + # First accept the basic system types. + # The portable systems comes first. + # Each alternative MUST END IN A *, to match a version number. + # -sysv* is not here because it comes later, after sysvr4. + -gnu* | -bsd* | -mach* | -minix* | -genix* | -ultrix* | -irix* \ + | -*vms* | -sco* | -esix* | -isc* | -aix* | -cnk* | -sunos | -sunos[34]*\ + | -hpux* | -unos* | -osf* | -luna* | -dgux* | -auroraux* | -solaris* \ + | -sym* | -kopensolaris* | -plan9* \ + | -amigaos* | -amigados* | -msdos* | -newsos* | -unicos* | -aof* \ + | -aos* | -aros* \ + | -nindy* | -vxsim* | -vxworks* | -ebmon* | -hms* | -mvs* \ + | -clix* | -riscos* | -uniplus* | -iris* | -rtu* | -xenix* \ + | -hiux* | -386bsd* | -knetbsd* | -mirbsd* | -netbsd* \ + | -bitrig* | -openbsd* | -solidbsd* \ + | -ekkobsd* | -kfreebsd* | -freebsd* | -riscix* | -lynxos* \ + | -bosx* | -nextstep* | -cxux* | -aout* | -elf* | -oabi* \ + | -ptx* | -coff* | -ecoff* | -winnt* | -domain* | -vsta* \ + | -udi* | -eabi* | -lites* | -ieee* | -go32* | -aux* \ + | -chorusos* | -chorusrdb* | -cegcc* \ + | -cygwin* | -msys* | -pe* | -psos* | -moss* | -proelf* | -rtems* \ + | -mingw32* | -mingw64* | -linux-gnu* | -linux-android* \ + | -linux-newlib* | -linux-musl* | -linux-uclibc* \ + | -uxpv* | -beos* | -mpeix* | -udk* \ + | -interix* | -uwin* | -mks* | -rhapsody* | -darwin* | -opened* \ + | -openstep* | -oskit* | -conix* | -pw32* | -nonstopux* \ + | -storm-chaos* | -tops10* | -tenex* | -tops20* | -its* \ + | -os2* | -vos* | -palmos* | -uclinux* | -nucleus* \ + | -morphos* | -superux* | -rtmk* | -rtmk-nova* | -windiss* \ + | -powermax* | -dnix* | -nx6 | -nx7 | -sei* | -dragonfly* \ + | -skyos* | -haiku* | -rdos* | -toppers* | -drops* | -es* | -tirtos*) + # Remember, each alternative MUST END IN *, to match a version number. + ;; + -qnx*) + case $basic_machine in + x86-* | i*86-*) + ;; + *) + os=-nto$os + ;; + esac + ;; + -nto-qnx*) + ;; + -nto*) + os=`echo $os | sed -e 's|nto|nto-qnx|'` + ;; + -sim | -es1800* | -hms* | -xray | -os68k* | -none* | -v88r* \ + | -windows* | -osx | -abug | -netware* | -os9* | -beos* | -haiku* \ + | -macos* | -mpw* | -magic* | -mmixware* | -mon960* | -lnews*) + ;; + -mac*) + os=`echo $os | sed -e 's|mac|macos|'` + ;; + -linux-dietlibc) + os=-linux-dietlibc + ;; + -linux*) + os=`echo $os | sed -e 's|linux|linux-gnu|'` + ;; + -sunos5*) + os=`echo $os | sed -e 's|sunos5|solaris2|'` + ;; + -sunos6*) + os=`echo $os | sed -e 's|sunos6|solaris3|'` + ;; + -opened*) + os=-openedition + ;; + -os400*) + os=-os400 + ;; + -wince*) + os=-wince + ;; + -osfrose*) + os=-osfrose + ;; + -osf*) + os=-osf + ;; + -utek*) + os=-bsd + ;; + -dynix*) + os=-bsd + ;; + -acis*) + os=-aos + ;; + -atheos*) + os=-atheos + ;; + -syllable*) + os=-syllable + ;; + -386bsd) + os=-bsd + ;; + -ctix* | -uts*) + os=-sysv + ;; + -nova*) + os=-rtmk-nova + ;; + -ns2 ) + os=-nextstep2 + ;; + -nsk*) + os=-nsk + ;; + # Preserve the version number of sinix5. + -sinix5.*) + os=`echo $os | sed -e 's|sinix|sysv|'` + ;; + -sinix*) + os=-sysv4 + ;; + -tpf*) + os=-tpf + ;; + -triton*) + os=-sysv3 + ;; + -oss*) + os=-sysv3 + ;; + -svr4) + os=-sysv4 + ;; + -svr3) + os=-sysv3 + ;; + -sysvr4) + os=-sysv4 + ;; + # This must come after -sysvr4. + -sysv*) + ;; + -ose*) + os=-ose + ;; + -es1800*) + os=-ose + ;; + -xenix) + os=-xenix + ;; + -*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*) + os=-mint + ;; + -aros*) + os=-aros + ;; + -zvmoe) + os=-zvmoe + ;; + -dicos*) + os=-dicos + ;; + -nacl*) + ;; + -none) + ;; + *) + # Get rid of the `-' at the beginning of $os. + os=`echo $os | sed 's/[^-]*-//'` + echo Invalid configuration \`$1\': system \`$os\' not recognized 1>&2 + exit 1 + ;; +esac +else + +# Here we handle the default operating systems that come with various machines. +# The value should be what the vendor currently ships out the door with their +# machine or put another way, the most popular os provided with the machine. + +# Note that if you're going to try to match "-MANUFACTURER" here (say, +# "-sun"), then you have to tell the case statement up towards the top +# that MANUFACTURER isn't an operating system. Otherwise, code above +# will signal an error saying that MANUFACTURER isn't an operating +# system, and we'll never get to this point. + +case $basic_machine in + score-*) + os=-elf + ;; + spu-*) + os=-elf + ;; + *-acorn) + os=-riscix1.2 + ;; + arm*-rebel) + os=-linux + ;; + arm*-semi) + os=-aout + ;; + c4x-* | tic4x-*) + os=-coff + ;; + c8051-*) + os=-elf + ;; + hexagon-*) + os=-elf + ;; + tic54x-*) + os=-coff + ;; + tic55x-*) + os=-coff + ;; + tic6x-*) + os=-coff + ;; + # This must come before the *-dec entry. + pdp10-*) + os=-tops20 + ;; + pdp11-*) + os=-none + ;; + *-dec | vax-*) + os=-ultrix4.2 + ;; + m68*-apollo) + os=-domain + ;; + i386-sun) + os=-sunos4.0.2 + ;; + m68000-sun) + os=-sunos3 + ;; + m68*-cisco) + os=-aout + ;; + mep-*) + os=-elf + ;; + mips*-cisco) + os=-elf + ;; + mips*-*) + os=-elf + ;; + or32-*) + os=-coff + ;; + *-tti) # must be before sparc entry or we get the wrong os. + os=-sysv3 + ;; + sparc-* | *-sun) + os=-sunos4.1.1 + ;; + *-be) + os=-beos + ;; + *-haiku) + os=-haiku + ;; + *-ibm) + os=-aix + ;; + *-knuth) + os=-mmixware + ;; + *-wec) + os=-proelf + ;; + *-winbond) + os=-proelf + ;; + *-oki) + os=-proelf + ;; + *-hp) + os=-hpux + ;; + *-hitachi) + os=-hiux + ;; + i860-* | *-att | *-ncr | *-altos | *-motorola | *-convergent) + os=-sysv + ;; + *-cbm) + os=-amigaos + ;; + *-dg) + os=-dgux + ;; + *-dolphin) + os=-sysv3 + ;; + m68k-ccur) + os=-rtu + ;; + m88k-omron*) + os=-luna + ;; + *-next ) + os=-nextstep + ;; + *-sequent) + os=-ptx + ;; + *-crds) + os=-unos + ;; + *-ns) + os=-genix + ;; + i370-*) + os=-mvs + ;; + *-next) + os=-nextstep3 + ;; + *-gould) + os=-sysv + ;; + *-highlevel) + os=-bsd + ;; + *-encore) + os=-bsd + ;; + *-sgi) + os=-irix + ;; + *-siemens) + os=-sysv4 + ;; + *-masscomp) + os=-rtu + ;; + f30[01]-fujitsu | f700-fujitsu) + os=-uxpv + ;; + *-rom68k) + os=-coff + ;; + *-*bug) + os=-coff + ;; + *-apple) + os=-macos + ;; + *-atari*) + os=-mint + ;; + *) + os=-none + ;; +esac +fi + +# Here we handle the case where we know the os, and the CPU type, but not the +# manufacturer. We pick the logical manufacturer. +vendor=unknown +case $basic_machine in + *-unknown) + case $os in + -riscix*) + vendor=acorn + ;; + -sunos*) + vendor=sun + ;; + -cnk*|-aix*) + vendor=ibm + ;; + -beos*) + vendor=be + ;; + -hpux*) + vendor=hp + ;; + -mpeix*) + vendor=hp + ;; + -hiux*) + vendor=hitachi + ;; + -unos*) + vendor=crds + ;; + -dgux*) + vendor=dg + ;; + -luna*) + vendor=omron + ;; + -genix*) + vendor=ns + ;; + -mvs* | -opened*) + vendor=ibm + ;; + -os400*) + vendor=ibm + ;; + -ptx*) + vendor=sequent + ;; + -tpf*) + vendor=ibm + ;; + -vxsim* | -vxworks* | -windiss*) + vendor=wrs + ;; + -aux*) + vendor=apple + ;; + -hms*) + vendor=hitachi + ;; + -mpw* | -macos*) + vendor=apple + ;; + -*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*) + vendor=atari + ;; + -vos*) + vendor=stratus + ;; + esac + basic_machine=`echo $basic_machine | sed "s/unknown/$vendor/"` + ;; +esac + +echo $basic_machine$os +exit + +# Local variables: +# eval: (add-hook 'write-file-hooks 'time-stamp) +# time-stamp-start: "timestamp='" +# time-stamp-format: "%:y-%02m-%02d" +# time-stamp-end: "'" +# End: diff --git a/depends/funcs.mk b/depends/funcs.mk new file mode 100644 index 00000000..33763425 --- /dev/null +++ b/depends/funcs.mk @@ -0,0 +1,234 @@ +define int_vars +#Set defaults for vars which may be overridden per-package +$(1)_cc=$($($(1)_type)_CC) +$(1)_cxx=$($($(1)_type)_CXX) +$(1)_objc=$($($(1)_type)_OBJC) +$(1)_objcxx=$($($(1)_type)_OBJCXX) +$(1)_ar=$($($(1)_type)_AR) +$(1)_ranlib=$($($(1)_type)_RANLIB) +$(1)_libtool=$($($(1)_type)_LIBTOOL) +$(1)_nm=$($($(1)_type)_NM) +$(1)_cflags=$($($(1)_type)_CFLAGS) $($($(1)_type)_$(release_type)_CFLAGS) +$(1)_cxxflags=$($($(1)_type)_CXXFLAGS) $($($(1)_type)_$(release_type)_CXXFLAGS) +$(1)_ldflags=$($($(1)_type)_LDFLAGS) $($($(1)_type)_$(release_type)_LDFLAGS) -L$($($(1)_type)_prefix)/lib +$(1)_cppflags=$($($(1)_type)_CPPFLAGS) $($($(1)_type)_$(release_type)_CPPFLAGS) -I$($($(1)_type)_prefix)/include +$(1)_recipe_hash:= +endef + +define int_get_all_dependencies +$(sort $(foreach dep,$(2),$(2) $(call int_get_all_dependencies,$(1),$($(dep)_dependencies)))) +endef + +define fetch_file +(test -f $$($(1)_source_dir)/$(4) || \ + ( mkdir -p $$($(1)_download_dir) && echo Fetching $(1)... && \ + ( $(build_DOWNLOAD) "$$($(1)_download_dir)/$(4).temp" "$(2)/$(3)" || \ + $(build_DOWNLOAD) "$$($(1)_download_dir)/$(4).temp" "$(FALLBACK_DOWNLOAD_PATH)/$(3)" ) && \ + echo "$(5) $$($(1)_download_dir)/$(4).temp" > $$($(1)_download_dir)/.$(4).hash && \ + $(build_SHA256SUM) -c $$($(1)_download_dir)/.$(4).hash && \ + mv $$($(1)_download_dir)/$(4).temp $$($(1)_source_dir)/$(4) && \ + rm -rf $$($(1)_download_dir) )) +endef + +define int_get_build_recipe_hash +$(eval $(1)_all_file_checksums:=$(shell $(build_SHA256SUM) $(meta_depends) packages/$(1).mk $(addprefix $(PATCHES_PATH)/$(1)/,$($(1)_patches)) | cut -d" " -f1)) +$(eval $(1)_recipe_hash:=$(shell echo -n "$($(1)_all_file_checksums)" | $(build_SHA256SUM) | cut -d" " -f1)) +endef + +define int_get_build_id +$(eval $(1)_dependencies += $($(1)_$(host_arch)_$(host_os)_dependencies) $($(1)_$(host_os)_dependencies)) +$(eval $(1)_all_dependencies:=$(call int_get_all_dependencies,$(1),$($($(1)_type)_native_toolchain) $($(1)_dependencies))) +$(foreach dep,$($(1)_all_dependencies),$(eval $(1)_build_id_deps+=$(dep)-$($(dep)_version)-$($(dep)_recipe_hash))) +$(eval $(1)_build_id_long:=$(1)-$($(1)_version)-$($(1)_recipe_hash)-$(release_type) $($(1)_build_id_deps)) +$(eval $(1)_build_id:=$(shell echo -n "$($(1)_build_id_long)" | $(build_SHA256SUM) | cut -c-$(HASH_LENGTH))) +final_build_id_long+=$($(package)_build_id_long) + +#compute package-specific paths +$(1)_build_subdir?=. +$(1)_download_file?=$($(1)_file_name) +$(1)_source_dir:=$(SOURCES_PATH) +$(1)_source:=$$($(1)_source_dir)/$($(1)_file_name) +$(1)_staging_dir=$(base_staging_dir)/$(host)/$(1)/$($(1)_version)-$($(1)_build_id) +$(1)_staging_prefix_dir:=$$($(1)_staging_dir)$($($(1)_type)_prefix) +$(1)_extract_dir:=$(base_build_dir)/$(host)/$(1)/$($(1)_version)-$($(1)_build_id) +$(1)_download_dir:=$(base_download_dir)/$(1)-$($(1)_version) +$(1)_build_dir:=$$($(1)_extract_dir)/$$($(1)_build_subdir) +$(1)_patch_dir:=$(base_build_dir)/$(host)/$(1)/$($(1)_version)-$($(1)_build_id)/.patches-$($(1)_build_id) +$(1)_prefixbin:=$($($(1)_type)_prefix)/bin/ +$(1)_cached:=$(BASE_CACHE)/$(host)/$(1)/$(1)-$($(1)_version)-$($(1)_build_id).tar.gz + +#stamps +$(1)_fetched=$$($(1)_source_dir)/download-stamps/.stamp_fetched-$(1)-$($(1)_file_name) +$(1)_extracted=$$($(1)_extract_dir)/.stamp_extracted +$(1)_preprocessed=$$($(1)_extract_dir)/.stamp_preprocessed +$(1)_cleaned=$$($(1)_extract_dir)/.stamp_cleaned +$(1)_built=$$($(1)_build_dir)/.stamp_built +$(1)_configured=$$($(1)_build_dir)/.stamp_configured +$(1)_staged=$$($(1)_staging_dir)/.stamp_staged +$(1)_postprocessed=$$($(1)_staging_prefix_dir)/.stamp_postprocessed +$(1)_download_path_fixed=$(subst :,\:,$$($(1)_download_path)) + + +#default commands +$(1)_fetch_cmds ?= $(call fetch_file,$(1),$(subst \:,:,$$($(1)_download_path_fixed)),$$($(1)_download_file),$($(1)_file_name),$($(1)_sha256_hash)) +$(1)_extract_cmds ?= mkdir -p $$($(1)_extract_dir) && echo "$$($(1)_sha256_hash) $$($(1)_source)" > $$($(1)_extract_dir)/.$$($(1)_file_name).hash && $(build_SHA256SUM) -c $$($(1)_extract_dir)/.$$($(1)_file_name).hash && tar --strip-components=1 -xf $$($(1)_source) +$(1)_preprocess_cmds ?= +$(1)_build_cmds ?= +$(1)_config_cmds ?= +$(1)_stage_cmds ?= +$(1)_set_vars ?= + + +all_sources+=$$($(1)_fetched) +endef +#$(foreach dep_target,$($(1)_all_dependencies),$(eval $(1)_dependency_targets=$($(dep_target)_cached))) + + +define int_config_attach_build_config +$(eval $(call $(1)_set_vars,$(1))) +$(1)_cflags+=$($(1)_cflags_$(release_type)) +$(1)_cflags+=$($(1)_cflags_$(host_arch)) $($(1)_cflags_$(host_arch)_$(release_type)) +$(1)_cflags+=$($(1)_cflags_$(host_os)) $($(1)_cflags_$(host_os)_$(release_type)) +$(1)_cflags+=$($(1)_cflags_$(host_arch)_$(host_os)) $($(1)_cflags_$(host_arch)_$(host_os)_$(release_type)) + +$(1)_cxxflags+=$($(1)_cxxflags_$(release_type)) +$(1)_cxxflags+=$($(1)_cxxflags_$(host_arch)) $($(1)_cxxflags_$(host_arch)_$(release_type)) +$(1)_cxxflags+=$($(1)_cxxflags_$(host_os)) $($(1)_cxxflags_$(host_os)_$(release_type)) +$(1)_cxxflags+=$($(1)_cxxflags_$(host_arch)_$(host_os)) $($(1)_cxxflags_$(host_arch)_$(host_os)_$(release_type)) + +$(1)_cppflags+=$($(1)_cppflags_$(release_type)) +$(1)_cppflags+=$($(1)_cppflags_$(host_arch)) $($(1)_cppflags_$(host_arch)_$(release_type)) +$(1)_cppflags+=$($(1)_cppflags_$(host_os)) $($(1)_cppflags_$(host_os)_$(release_type)) +$(1)_cppflags+=$($(1)_cppflags_$(host_arch)_$(host_os)) $($(1)_cppflags_$(host_arch)_$(host_os)_$(release_type)) + +$(1)_ldflags+=$($(1)_ldflags_$(release_type)) +$(1)_ldflags+=$($(1)_ldflags_$(host_arch)) $($(1)_ldflags_$(host_arch)_$(release_type)) +$(1)_ldflags+=$($(1)_ldflags_$(host_os)) $($(1)_ldflags_$(host_os)_$(release_type)) +$(1)_ldflags+=$($(1)_ldflags_$(host_arch)_$(host_os)) $($(1)_ldflags_$(host_arch)_$(host_os)_$(release_type)) + +$(1)_build_opts+=$$($(1)_build_opts_$(release_type)) +$(1)_build_opts+=$$($(1)_build_opts_$(host_arch)) $$($(1)_build_opts_$(host_arch)_$(release_type)) +$(1)_build_opts+=$$($(1)_build_opts_$(host_os)) $$($(1)_build_opts_$(host_os)_$(release_type)) +$(1)_build_opts+=$$($(1)_build_opts_$(host_arch)_$(host_os)) $$($(1)_build_opts_$(host_arch)_$(host_os)_$(release_type)) + +$(1)_config_opts+=$$($(1)_config_opts_$(release_type)) +$(1)_config_opts+=$$($(1)_config_opts_$(host_arch)) $$($(1)_config_opts_$(host_arch)_$(release_type)) +$(1)_config_opts+=$$($(1)_config_opts_$(host_os)) $$($(1)_config_opts_$(host_os)_$(release_type)) +$(1)_config_opts+=$$($(1)_config_opts_$(host_arch)_$(host_os)) $$($(1)_config_opts_$(host_arch)_$(host_os)_$(release_type)) + +$(1)_config_env+=$$($(1)_config_env_$(release_type)) +$(1)_config_env+=$($(1)_config_env_$(host_arch)) $($(1)_config_env_$(host_arch)_$(release_type)) +$(1)_config_env+=$($(1)_config_env_$(host_os)) $($(1)_config_env_$(host_os)_$(release_type)) +$(1)_config_env+=$($(1)_config_env_$(host_arch)_$(host_os)) $($(1)_config_env_$(host_arch)_$(host_os)_$(release_type)) + +$(1)_config_env+=PKG_CONFIG_LIBDIR=$($($(1)_type)_prefix)/lib/pkgconfig +$(1)_config_env+=PKG_CONFIG_PATH=$($($(1)_type)_prefix)/share/pkgconfig +$(1)_config_env+=PATH=$(build_prefix)/bin:$(PATH) +$(1)_build_env+=PATH=$(build_prefix)/bin:$(PATH) +$(1)_stage_env+=PATH=$(build_prefix)/bin:$(PATH) +$(1)_autoconf=./configure --host=$($($(1)_type)_host) --disable-dependency-tracking --prefix=$($($(1)_type)_prefix) $$($(1)_config_opts) CC="$$($(1)_cc)" CXX="$$($(1)_cxx)" + +ifneq ($($(1)_nm),) +$(1)_autoconf += NM="$$($(1)_nm)" +endif +ifneq ($($(1)_ranlib),) +$(1)_autoconf += RANLIB="$$($(1)_ranlib)" +endif +ifneq ($($(1)_ar),) +$(1)_autoconf += AR="$$($(1)_ar)" +endif +ifneq ($($(1)_cflags),) +$(1)_autoconf += CFLAGS="$$($(1)_cflags)" +endif +ifneq ($($(1)_cxxflags),) +$(1)_autoconf += CXXFLAGS="$$($(1)_cxxflags)" +endif +ifneq ($($(1)_cppflags),) +$(1)_autoconf += CPPFLAGS="$$($(1)_cppflags)" +endif +ifneq ($($(1)_ldflags),) +$(1)_autoconf += LDFLAGS="$$($(1)_ldflags)" +endif +endef + +define int_add_cmds +$($(1)_fetched): + $(AT)mkdir -p $$(@D) $(SOURCES_PATH) + $(AT)cd $$(@D); $(call $(1)_fetch_cmds,$(1)) + $(AT)touch $$@ +$($(1)_extracted): | $($(1)_fetched) + $(AT)echo Extracting $(1)... + $(AT)mkdir -p $$(@D) + $(AT)cd $$(@D); $(call $(1)_extract_cmds,$(1)) + $(AT)touch $$@ +$($(1)_preprocessed): | $($(1)_dependencies) $($(1)_extracted) + $(AT)echo Preprocessing $(1)... + $(AT)mkdir -p $$(@D) $($(1)_patch_dir) + $(AT)$(foreach patch,$($(1)_patches),cd $(PATCHES_PATH)/$(1); cp $(patch) $($(1)_patch_dir) ;) + $(AT)cd $$(@D); $(call $(1)_preprocess_cmds, $(1)) + $(AT)touch $$@ +$($(1)_configured): | $($(1)_preprocessed) + $(AT)echo Configuring $(1)... + $(AT)rm -rf $(host_prefix); mkdir -p $(host_prefix)/lib; cd $(host_prefix); $(foreach package,$($(1)_all_dependencies), tar xf $($(package)_cached); ) + $(AT)mkdir -p $$(@D) + $(AT)+cd $$(@D); $($(1)_config_env) $(call $(1)_config_cmds, $(1)) + $(AT)touch $$@ +$($(1)_built): | $($(1)_configured) + $(AT)echo Building $(1)... + $(AT)mkdir -p $$(@D) + $(AT)+cd $$(@D); $($(1)_build_env) $(call $(1)_build_cmds, $(1)) + $(AT)touch $$@ +$($(1)_staged): | $($(1)_built) + $(AT)echo Staging $(1)... + $(AT)mkdir -p $($(1)_staging_dir)/$(host_prefix) + $(AT)cd $($(1)_build_dir); $($(1)_stage_env) $(call $(1)_stage_cmds, $(1)) + $(AT)rm -rf $($(1)_extract_dir) + $(AT)touch $$@ +$($(1)_postprocessed): | $($(1)_staged) + $(AT)echo Postprocessing $(1)... + $(AT)cd $($(1)_staging_prefix_dir); $(call $(1)_postprocess_cmds) + $(AT)touch $$@ +$($(1)_cached): | $($(1)_dependencies) $($(1)_postprocessed) + $(AT)echo Caching $(1)... + $(AT)cd $$($(1)_staging_dir)/$(host_prefix); find . | sort | tar --no-recursion -czf $$($(1)_staging_dir)/$$(@F) -T - + $(AT)mkdir -p $$(@D) + $(AT)rm -rf $$(@D) && mkdir -p $$(@D) + $(AT)mv $$($(1)_staging_dir)/$$(@F) $$(@) + $(AT)rm -rf $($(1)_staging_dir) + +.PHONY: $(1) +$(1): | $($(1)_cached) +.SECONDARY: $($(1)_postprocessed) $($(1)_staged) $($(1)_built) $($(1)_configured) $($(1)_preprocessed) $($(1)_extracted) $($(1)_fetched) + +endef + +# These functions create the build targets for each package. They must be +# broken down into small steps so that each part is done for all packages +# before moving on to the next step. Otherwise, a package's info +# (build-id for example) would only be avilable to another package if it +# happened to be computed already. + +#set the type for host/build packages. +$(foreach native_package,$(native_packages),$(eval $(native_package)_type=build)) +$(foreach package,$(packages),$(eval $(package)_type=$(host_arch)_$(host_os))) + +#set overridable defaults +$(foreach package,$(all_packages),$(eval $(call int_vars,$(package)))) + +#include package files +$(foreach package,$(all_packages),$(eval include packages/$(package).mk)) + +#compute a hash of all files that comprise this package's build recipe +$(foreach package,$(all_packages),$(eval $(call int_get_build_recipe_hash,$(package)))) + +#generate a unique id for this package, incorporating its dependencies as well +$(foreach package,$(all_packages),$(eval $(call int_get_build_id,$(package)))) + +#compute final vars after reading package vars +$(foreach package,$(all_packages),$(eval $(call int_config_attach_build_config,$(package)))) + +#create build targets +$(foreach package,$(all_packages),$(eval $(call int_add_cmds,$(package)))) + +#special exception: if a toolchain package exists, all non-native packages depend on it +$(foreach package,$(packages),$(eval $($(package)_unpacked): |$($($(host_arch)_$(host_os)_native_toolchain)_cached) )) diff --git a/depends/hosts/darwin.mk b/depends/hosts/darwin.mk new file mode 100644 index 00000000..8d718eba --- /dev/null +++ b/depends/hosts/darwin.mk @@ -0,0 +1,16 @@ +OSX_MIN_VERSION=10.6 +OSX_SDK_VERSION=10.7 +OSX_SDK=$(SDK_PATH)/MacOSX$(OSX_SDK_VERSION).sdk +darwin_CC=clang -target $(host) -mmacosx-version-min=$(OSX_MIN_VERSION) --sysroot $(OSX_SDK) +darwin_CXX=clang++ -target $(host) -mmacosx-version-min=$(OSX_MIN_VERSION) --sysroot $(OSX_SDK) + +darwin_CFLAGS=-pipe +darwin_CXXFLAGS=$(darwin_CFLAGS) + +darwin_release_CFLAGS=-O2 +darwin_release_CXXFLAGS=$(darwin_release_CFLAGS) + +darwin_debug_CFLAGS=-O1 +darwin_debug_CXXFLAGS=$(darwin_debug_CFLAGS) + +darwin_native_toolchain=native_cctools diff --git a/depends/hosts/default.mk b/depends/hosts/default.mk new file mode 100644 index 00000000..6f60d6b3 --- /dev/null +++ b/depends/hosts/default.mk @@ -0,0 +1,26 @@ +default_host_CC = $(host_toolchain)gcc +default_host_CXX = $(host_toolchain)g++ +default_host_AR = $(host_toolchain)ar +default_host_RANLIB = $(host_toolchain)ranlib +default_host_STRIP = $(host_toolchain)strip +default_host_LIBTOOL = $(host_toolchain)libtool +default_host_INSTALL_NAME_TOOL = $(host_toolchain)install_name_tool +default_host_OTOOL = $(host_toolchain)otool +default_host_NM = $(host_toolchain)nm + +define add_host_tool_func +$(host_os)_$1?=$$(default_host_$1) +$(host_arch)_$(host_os)_$1?=$$($(host_os)_$1) +$(host_arch)_$(host_os)_$(release_type)_$1?=$$($(host_os)_$1) +host_$1=$$($(host_arch)_$(host_os)_$1) +endef + +define add_host_flags_func +$(host_arch)_$(host_os)_$1 += $($(host_os)_$1) +$(host_arch)_$(host_os)_$(release_type)_$1 += $($(host_os)_$(release_type)_$1) +host_$1 = $$($(host_arch)_$(host_os)_$1) +host_$(release_type)_$1 = $$($(host_arch)_$(host_os)_$(release_type)_$1) +endef + +$(foreach tool,CC CXX AR RANLIB STRIP NM LIBTOOL OTOOL INSTALL_NAME_TOOL,$(eval $(call add_host_tool_func,$(tool)))) +$(foreach flags,CFLAGS CXXFLAGS CPPFLAGS LDFLAGS, $(eval $(call add_host_flags_func,$(flags)))) diff --git a/depends/hosts/linux.mk b/depends/hosts/linux.mk new file mode 100644 index 00000000..b13a0f1a --- /dev/null +++ b/depends/hosts/linux.mk @@ -0,0 +1,31 @@ +linux_CFLAGS=-pipe +linux_CXXFLAGS=$(linux_CFLAGS) + +linux_release_CFLAGS=-O2 +linux_release_CXXFLAGS=$(linux_release_CFLAGS) + +linux_debug_CFLAGS=-O1 +linux_debug_CXXFLAGS=$(linux_debug_CFLAGS) + +linux_debug_CPPFLAGS=-D_GLIBCXX_DEBUG -D_GLIBCXX_DEBUG_PEDANTIC + +ifeq (86,$(findstring 86,$(build_arch))) +i686_linux_CC=gcc -m32 +i686_linux_CXX=g++ -m32 +i686_linux_AR=ar +i686_linux_RANLIB=ranlib +i686_linux_NM=nm +i686_linux_STRIP=strip + +x86_64_linux_CC=gcc -m64 +x86_64_linux_CXX=g++ -m64 +x86_64_linux_AR=ar +x86_64_linux_RANLIB=ranlib +x86_64_linux_NM=nm +x86_64_linux_STRIP=strip +else +i686_linux_CC=$(default_host_CC) -m32 +i686_linux_CXX=$(default_host_CXX) -m32 +x86_64_linux_CC=$(default_host_CC) -m64 +x86_64_linux_CXX=$(default_host_CXX) -m64 +endif diff --git a/depends/hosts/mingw32.mk b/depends/hosts/mingw32.mk new file mode 100644 index 00000000..dbfb62fd --- /dev/null +++ b/depends/hosts/mingw32.mk @@ -0,0 +1,10 @@ +mingw32_CFLAGS=-pipe +mingw32_CXXFLAGS=$(mingw32_CFLAGS) + +mingw32_release_CFLAGS=-O2 +mingw32_release_CXXFLAGS=$(mingw32_release_CFLAGS) + +mingw32_debug_CFLAGS=-O1 +mingw32_debug_CXXFLAGS=$(mingw32_debug_CFLAGS) + +mingw32_debug_CPPFLAGS=-D_GLIBCXX_DEBUG -D_GLIBCXX_DEBUG_PEDANTIC diff --git a/depends/packages/bdb.mk b/depends/packages/bdb.mk new file mode 100644 index 00000000..68841afd --- /dev/null +++ b/depends/packages/bdb.mk @@ -0,0 +1,28 @@ +package=bdb +$(package)_version=4.8.30 +$(package)_download_path=http://download.oracle.com/berkeley-db +$(package)_file_name=db-$($(package)_version).NC.tar.gz +$(package)_sha256_hash=12edc0df75bf9abd7f82f821795bcee50f42cb2e5f76a6a281b85732798364ef +$(package)_build_subdir=build_unix + +define $(package)_set_vars +$(package)_config_opts=--disable-shared --enable-cxx --disable-replication +$(package)_config_opts_mingw32=--enable-mingw +$(package)_config_opts_linux=--with-pic +endef + +define $(package)_preprocess_cmds + sed -i.old 's/__atomic_compare_exchange/__atomic_compare_exchange_db/' dbinc/atomic.h +endef + +define $(package)_config_cmds + ../dist/$($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) libdb_cxx-4.8.a libdb-4.8.a +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install_lib install_include +endef diff --git a/depends/packages/boost.mk b/depends/packages/boost.mk new file mode 100644 index 00000000..f50828c5 --- /dev/null +++ b/depends/packages/boost.mk @@ -0,0 +1,44 @@ +package=boost +$(package)_version=1_55_0 +$(package)_download_path=http://sourceforge.net/projects/boost/files/boost/1.55.0 +$(package)_file_name=$(package)_$($(package)_version).tar.bz2 +$(package)_sha256_hash=fff00023dd79486d444c8e29922f4072e1d451fc5a4d2b6075852ead7f2b7b52 +$(package)_patches=darwin_boost_atomic-1.patch darwin_boost_atomic-2.patch + +define $(package)_set_vars +$(package)_config_opts_release=variant=release +$(package)_config_opts_debug=variant=debug +$(package)_config_opts=--layout=tagged --build-type=complete --user-config=user-config.jam +$(package)_config_opts+=threading=multi link=static -sNO_BZIP2=1 -sNO_ZLIB=1 +$(package)_config_opts_linux=threadapi=pthread runtime-link=shared +$(package)_config_opts_darwin=--toolset=darwin-4.2.1 runtime-link=shared +$(package)_config_opts_mingw32=binary-format=pe target-os=windows threadapi=win32 runtime-link=static +$(package)_config_opts_x86_64_mingw32=address-model=64 +$(package)_config_opts_i686_mingw32=address-model=32 +$(package)_config_opts_i686_linux=address-model=32 architecture=x86 +$(package)_toolset_$(host_os)=gcc +$(package)_archiver_$(host_os)=$($(package)_ar) +$(package)_toolset_darwin=darwin +$(package)_archiver_darwin=$($(package)_libtool) +$(package)_config_libraries=chrono,filesystem,program_options,system,thread,test +$(package)_cxxflags=-fvisibility=hidden +$(package)_cxxflags_linux=-fPIC +endef + +define $(package)_preprocess_cmds + patch -p2 < $($(package)_patch_dir)/darwin_boost_atomic-1.patch && \ + patch -p2 < $($(package)_patch_dir)/darwin_boost_atomic-2.patch && \ + echo "using $(boost_toolset_$(host_os)) : : $($(package)_cxx) : \"$($(package)_cxxflags) $($(package)_cppflags)\" \"$($(package)_ldflags)\" \"$(boost_archiver_$(host_os))\" \"$(host_STRIP)\" \"$(host_RANLIB)\" \"$(host_WINDRES)\" : ;" > user-config.jam +endef + +define $(package)_config_cmds + ./bootstrap.sh --without-icu --with-libraries=$(boost_config_libraries) +endef + +define $(package)_build_cmds + ./b2 -d2 -j2 -d1 --prefix=$($(package)_staging_prefix_dir) $($(package)_config_opts) stage +endef + +define $(package)_stage_cmds + ./b2 -d0 -j4 --prefix=$($(package)_staging_prefix_dir) $($(package)_config_opts) install +endef diff --git a/depends/packages/dbus.mk b/depends/packages/dbus.mk new file mode 100644 index 00000000..8ac9ab74 --- /dev/null +++ b/depends/packages/dbus.mk @@ -0,0 +1,23 @@ +package=dbus +$(package)_version=1.8.6 +$(package)_download_path=http://dbus.freedesktop.org/releases/dbus +$(package)_file_name=$(package)-$($(package)_version).tar.gz +$(package)_sha256_hash=eded83ca007b719f32761e60fd8b9ffd0f5796a4caf455b01b5a5ef740ebd23f +$(package)_dependencies=expat + +define $(package)_set_vars + $(package)_config_opts=--disable-tests --disable-doxygen-docs --disable-xml-docs --disable-static --without-x +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) -C dbus libdbus-1.la +endef + +define $(package)_stage_cmds + $(MAKE) -C dbus DESTDIR=$($(package)_staging_dir) install-libLTLIBRARIES install-dbusincludeHEADERS install-nodist_dbusarchincludeHEADERS && \ + $(MAKE) DESTDIR=$($(package)_staging_dir) install-pkgconfigDATA +endef diff --git a/depends/packages/expat.mk b/depends/packages/expat.mk new file mode 100644 index 00000000..1ac44353 --- /dev/null +++ b/depends/packages/expat.mk @@ -0,0 +1,21 @@ +package=expat +$(package)_version=2.1.0 +$(package)_download_path=http://sourceforge.net/projects/expat/files/expat/$($(package)_version) +$(package)_file_name=$(package)-$($(package)_version).tar.gz +$(package)_sha256_hash=823705472f816df21c8f6aa026dd162b280806838bb55b3432b0fb1fcca7eb86 + +define $(package)_set_vars +$(package)_config_opts=--disable-static +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/packages/fontconfig.mk b/depends/packages/fontconfig.mk new file mode 100644 index 00000000..2cf553ed --- /dev/null +++ b/depends/packages/fontconfig.mk @@ -0,0 +1,22 @@ +package=fontconfig +$(package)_version=2.11.1 +$(package)_download_path=http://www.freedesktop.org/software/fontconfig/release/ +$(package)_file_name=$(package)-$($(package)_version).tar.bz2 +$(package)_sha256_hash=dc62447533bca844463a3c3fd4083b57c90f18a70506e7a9f4936b5a1e516a99 +$(package)_dependencies=freetype expat + +define $(package)_set_vars + $(package)_config_opts=--disable-docs --disable-static +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/packages/freetype.mk b/depends/packages/freetype.mk new file mode 100644 index 00000000..f7d6e0f9 --- /dev/null +++ b/depends/packages/freetype.mk @@ -0,0 +1,22 @@ +package=freetype +$(package)_version=2.5.3 +$(package)_download_path=http://downloads.sourceforge.net/$(package) +$(package)_file_name=$(package)-$($(package)_version).tar.bz2 +$(package)_sha256_hash=c0848b29d52ef3ca27ad92e08351f023c5e24ce8cea7d8fe69fc96358e65f75e + +define $(package)_set_vars + $(package)_config_opts=--without-zlib --without-png --disable-static + $(package)_config_opts_linux=--with-pic +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/packages/libICE.mk b/depends/packages/libICE.mk new file mode 100644 index 00000000..fc60323b --- /dev/null +++ b/depends/packages/libICE.mk @@ -0,0 +1,23 @@ +package=libICE +$(package)_version=1.0.9 +$(package)_download_path=http://xorg.freedesktop.org/releases/individual/lib/ +$(package)_file_name=$(package)-$($(package)_version).tar.bz2 +$(package)_sha256_hash=8f7032f2c1c64352b5423f6b48a8ebdc339cc63064af34d66a6c9aa79759e202 +$(package)_dependencies=xtrans xproto + +define $(package)_set_vars + $(package)_config_opts=--disable-static --disable-docs --disable-specs --without-xsltproc + $(package)_config_opts_linux=--with-pic +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/packages/libSM.mk b/depends/packages/libSM.mk new file mode 100644 index 00000000..0f9307ca --- /dev/null +++ b/depends/packages/libSM.mk @@ -0,0 +1,23 @@ +package=libSM +$(package)_version=1.2.2 +$(package)_download_path=http://xorg.freedesktop.org/releases/individual/lib/ +$(package)_file_name=$(package)-$($(package)_version).tar.bz2 +$(package)_sha256_hash=0baca8c9f5d934450a70896c4ad38d06475521255ca63b717a6510fdb6e287bd +$(package)_dependencies=xtrans xproto libICE + +define $(package)_set_vars + $(package)_config_opts=--without-libuuid --without-xsltproc --disable-docs --disable-static + $(package)_config_opts_linux=--with-pic +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/packages/libX11.mk b/depends/packages/libX11.mk new file mode 100644 index 00000000..178d592e --- /dev/null +++ b/depends/packages/libX11.mk @@ -0,0 +1,23 @@ +package=libX11 +$(package)_version=1.6.2 +$(package)_download_path=http://xorg.freedesktop.org/releases/individual/lib/ +$(package)_file_name=$(package)-$($(package)_version).tar.bz2 +$(package)_sha256_hash=2aa027e837231d2eeea90f3a4afe19948a6eb4c8b2bec0241eba7dbc8106bd16 +$(package)_dependencies=libxcb xtrans xextproto xproto + +define $(package)_set_vars +$(package)_config_opts=--disable-xkb --disable-static +$(package)_config_opts_linux=--with-pic +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/packages/libXau.mk b/depends/packages/libXau.mk new file mode 100644 index 00000000..e87df2e4 --- /dev/null +++ b/depends/packages/libXau.mk @@ -0,0 +1,23 @@ +package=libXau +$(package)_version=1.0.8 +$(package)_download_path=http://xorg.freedesktop.org/releases/individual/lib/ +$(package)_file_name=$(package)-$($(package)_version).tar.bz2 +$(package)_sha256_hash=fdd477320aeb5cdd67272838722d6b7d544887dfe7de46e1e7cc0c27c2bea4f2 +$(package)_dependencies=xproto + +define $(package)_set_vars + $(package)_config_opts=--disable-shared + $(package)_config_opts_linux=--with-pic +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/packages/libXext.mk b/depends/packages/libXext.mk new file mode 100644 index 00000000..4db83606 --- /dev/null +++ b/depends/packages/libXext.mk @@ -0,0 +1,22 @@ +package=libXext +$(package)_version=1.3.2 +$(package)_download_path=http://xorg.freedesktop.org/releases/individual/lib/ +$(package)_file_name=$(package)-$($(package)_version).tar.bz2 +$(package)_sha256_hash=f829075bc646cdc085fa25d98d5885d83b1759ceb355933127c257e8e50432e0 +$(package)_dependencies=xproto xextproto libX11 libXau + +define $(package)_set_vars + $(package)_config_opts=--disable-static +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/packages/libxcb.mk b/depends/packages/libxcb.mk new file mode 100644 index 00000000..f29b577f --- /dev/null +++ b/depends/packages/libxcb.mk @@ -0,0 +1,30 @@ +package=libxcb +$(package)_version=1.10 +$(package)_download_path=http://xcb.freedesktop.org/dist +$(package)_file_name=$(package)-$($(package)_version).tar.bz2 +$(package)_sha256_hash=98d9ab05b636dd088603b64229dd1ab2d2cc02ab807892e107d674f9c3f2d5b5 +$(package)_dependencies=xcb_proto libXau xproto + +define $(package)_set_vars +$(package)_config_opts=--disable-static +endef + +define $(package)_preprocess_cmds + sed "s/pthread-stubs//" -i configure +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef + +define $(package)_postprocess_cmds + rm -rf share/man share/doc +endef diff --git a/depends/packages/miniupnpc.mk b/depends/packages/miniupnpc.mk new file mode 100644 index 00000000..00101f1b --- /dev/null +++ b/depends/packages/miniupnpc.mk @@ -0,0 +1,28 @@ +package=miniupnpc +$(package)_version=1.9.20140701 +$(package)_download_path=http://miniupnp.free.fr/files +$(package)_file_name=$(package)-$($(package)_version).tar.gz +$(package)_sha256_hash=26f3985bad7768b8483b793448ae49414cdc4451d0ec83e7c1944367e15f9f07 + +define $(package)_set_vars +$(package)_build_opts=CC="$($(package)_cc)" +$(package)_build_opts_darwin=OS=Darwin +$(package)_build_opts_mingw32=-f Makefile.mingw +$(package)_build_env+=CFLAGS="$($(package)_cflags) $($(package)_cppflags)" AR="$($(package)_ar)" +endef + +define $(package)_preprocess_cmds + mkdir dll && \ + sed -e 's|MINIUPNPC_VERSION_STRING \"version\"|MINIUPNPC_VERSION_STRING \"$($(package)_version)\"|' -e 's|OS/version|$(host)|' miniupnpcstrings.h.in > miniupnpcstrings.h && \ + sed -i.old "s|miniupnpcstrings.h: miniupnpcstrings.h.in wingenminiupnpcstrings|miniupnpcstrings.h: miniupnpcstrings.h.in|" Makefile.mingw +endef + +define $(package)_build_cmds + $(MAKE) libminiupnpc.a $($(package)_build_opts) +endef + +define $(package)_stage_cmds + mkdir -p $($(package)_staging_prefix_dir)/include/miniupnpc $($(package)_staging_prefix_dir)/lib &&\ + install *.h $($(package)_staging_prefix_dir)/include/miniupnpc &&\ + install libminiupnpc.a $($(package)_staging_prefix_dir)/lib +endef diff --git a/depends/packages/native_ccache.mk b/depends/packages/native_ccache.mk new file mode 100644 index 00000000..3226e89a --- /dev/null +++ b/depends/packages/native_ccache.mk @@ -0,0 +1,25 @@ +package=native_ccache +$(package)_version=3.1.9 +$(package)_download_path=http://samba.org/ftp/ccache +$(package)_file_name=ccache-$($(package)_version).tar.bz2 +$(package)_sha256_hash=04d3e2e438ac8d4cc4b110b68cdd61bd59226c6588739a4a386869467f5ced7c + +define $(package)_set_vars +$(package)_config_opts= +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef + +define $(package)_postprocess_cmds + rm -rf lib include +endef diff --git a/depends/packages/native_cctools.mk b/depends/packages/native_cctools.mk new file mode 100644 index 00000000..1675afe7 --- /dev/null +++ b/depends/packages/native_cctools.mk @@ -0,0 +1,80 @@ +package=native_cctools +$(package)_version=809 +$(package)_download_path=http://www.opensource.apple.com/tarballs/cctools +$(package)_file_name=cctools-$($(package)_version).tar.gz +$(package)_sha256_hash=03ba62749b843b131c7304a044a98c6ffacd65b1399b921d69add0375f79d8ad +$(package)_build_subdir=cctools2odcctools/odcctools-$($(package)_version) +$(package)_dependencies=native_libuuid native_openssl +$(package)_ld64_download_file=ld64-127.2.tar.gz +$(package)_ld64_download_path=http://www.opensource.apple.com/tarballs/ld64 +$(package)_ld64_file_name=$($(package)_ld64_download_file) +$(package)_ld64_sha256_hash=97b75547b2bd761306ab3e15ae297f01e7ab9760b922bc657f4ef72e4e052142 +$(package)_dyld_download_file=dyld-195.5.tar.gz +$(package)_dyld_download_path=http://www.opensource.apple.com/tarballs/dyld +$(package)_dyld_file_name=$($(package)_dyld_download_file) +$(package)_dyld_sha256_hash=2cf0484c87cf79b606b351a7055a247dae84093ae92c747a74e0cde2c8c8f83c +$(package)_toolchain4_download_file=10cc648683617cca8bcbeae507888099b41b530c.tar.gz +$(package)_toolchain4_download_path=https://github.com/mingwandroid/toolchain4/archive +$(package)_toolchain4_file_name=toolchain4-1.tar.gz +$(package)_toolchain4_sha256_hash=18406961fd4a1ec5c7ea35c91d6a80a2f8bb797a2bd243a610bd75e13eff9aca +$(package)_clang_download_file=clang+llvm-3.2-x86-linux-ubuntu-12.04.tar.gz +$(package)_clang_download_path=http://llvm.org/releases/3.2 +$(package)_clang_file_name=clang-llvm-3.2-x86-linux-ubuntu-12.04.tar.gz +$(package)_clang_sha256_hash=b9d57a88f9514fa1f327a1a703756d0c1c960f4c58494a5bd80313245d13ffff + +define $(package)_fetch_cmds +$(call fetch_file,$(package),$($(package)_download_path),$($(package)_download_file),$($(package)_file_name),$($(package)_sha256_hash)) && \ +$(call fetch_file,$(package),$($(package)_ld64_download_path),$($(package)_ld64_download_file),$($(package)_ld64_file_name),$($(package)_ld64_sha256_hash)) && \ +$(call fetch_file,$(package),$($(package)_dyld_download_path),$($(package)_dyld_download_file),$($(package)_dyld_file_name),$($(package)_dyld_sha256_hash)) && \ +$(call fetch_file,$(package),$($(package)_clang_download_path),$($(package)_clang_download_file),$($(package)_clang_file_name),$($(package)_clang_sha256_hash)) && \ +$(call fetch_file,$(package),$($(package)_toolchain4_download_path),$($(package)_toolchain4_download_file),$($(package)_toolchain4_file_name),$($(package)_toolchain4_sha256_hash)) +endef + +define $(package)_set_vars +$(package)_config_opts=--target=$(host) --with-sysroot=$(OSX_SDK) +$(package)_cflags+=-m32 +$(package)_cxxflags+=-m32 +$(package)_cppflags+=-D__DARWIN_UNIX03 -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS +$(package)_ldflags+=-m32 -Wl,-rpath=\\$$$$$$$$\$$$$$$$$ORIGIN/../lib +$(package)_ldflags+=-L$$(native_cctools_extract_dir)/clang+llvm-3.2-x86-linux-ubuntu-12.04/lib +endef +define $(package)_extract_cmds + tar --strip-components=1 -xf $($(package)_source_dir)/$($(package)_toolchain4_file_name) && \ + ln -sf $($(package)_source) cctools2odcctools/$($(package)_file_name) && \ + ln -sf $($(package)_source_dir)/$($(package)_ld64_file_name) cctools2odcctools/$($(package)_ld64_file_name) && \ + ln -sf $($(package)_source_dir)/$($(package)_dyld_file_name) cctools2odcctools/$($(package)_dyld_file_name) && \ + tar xf $($(package)_source_dir)/$($(package)_clang_file_name) && \ + mkdir -p $(SDK_PATH) sdks &&\ + cd sdks; ln -sf $(OSX_SDK) MacOSX$(OSX_SDK_VERSION).sdk +endef + +define $(package)_preprocess_cmds + sed -i "s|GCC_DIR|LLVM_CLANG_DIR|g" cctools2odcctools/extract.sh && \ + sed -i "s|llvmgcc42-2336.1|clang+llvm-3.2-x86-linux-ubuntu-12.04|g" cctools2odcctools/extract.sh && \ + sed -i "s|/llvmCore/include/llvm-c|/include/llvm-c \$$$${LLVM_CLANG_DIR}/include/llvm |" cctools2odcctools/extract.sh && \ + sed -i "s|fAC_INIT|AC_INIT|" cctools2odcctools/files/configure.ac && \ + sed -i 's/\# Dynamically linked LTO/\t ;\&\n\t linux*)\n# Dynamically linked LTO/' cctools2odcctools/files/configure.ac && \ + cd cctools2odcctools; ./extract.sh --osxver $(OSX_SDK_VERSION) && \ + sed -i "s|define\tPC|define\tPC_|" odcctools-809/include/architecture/sparc/reg.h +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install && \ + cd ../../clang+llvm-3.2-x86-linux-ubuntu-12.04 && \ + mkdir -p $($(package)_staging_prefix_dir)/lib/clang/3.2/include && \ + mkdir -p $($(package)_staging_prefix_dir)/bin && \ + cp -P bin/clang bin/clang++ $($(package)_staging_prefix_dir)/bin/ &&\ + cp lib/libLTO.so $($(package)_staging_prefix_dir)/lib/ && \ + cp lib/clang/3.2/include/* $($(package)_staging_prefix_dir)/lib/clang/3.2/include/ && \ + echo "#!/bin/sh" > $($(package)_staging_prefix_dir)/bin/$(host)-dsymutil && \ + echo "exit 0" >> $($(package)_staging_prefix_dir)/bin/$(host)-dsymutil && \ + chmod +x $($(package)_staging_prefix_dir)/bin/$(host)-dsymutil +endef diff --git a/depends/packages/native_cdrkit.mk b/depends/packages/native_cdrkit.mk new file mode 100644 index 00000000..cf694edb --- /dev/null +++ b/depends/packages/native_cdrkit.mk @@ -0,0 +1,26 @@ +package=native_cdrkit +$(package)_version=1.1.11 +$(package)_download_path=http://distro.ibiblio.org/fatdog/source/600/c +$(package)_file_name=cdrkit-$($(package)_version).tar.bz2 +$(package)_sha256_hash=b50d64c214a65b1a79afe3a964c691931a4233e2ba605d793eb85d0ac3652564 +$(package)_patches=cdrkit-deterministic.patch + +define $(package)_preprocess_cmds + patch -p1 < $($(package)_patch_dir)/cdrkit-deterministic.patch +endef + +define $(package)_config_cmds + cmake -DCMAKE_INSTALL_PREFIX=$(build_prefix) +endef + +define $(package)_build_cmds + $(MAKE) genisoimage +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) -C genisoimage install +endef + +define $(package)_postprocess_cmds + rm bin/isovfy bin/isoinfo bin/isodump bin/isodebug bin/devdump +endef diff --git a/depends/packages/native_comparisontool.mk b/depends/packages/native_comparisontool.mk new file mode 100644 index 00000000..d1b86dc2 --- /dev/null +++ b/depends/packages/native_comparisontool.mk @@ -0,0 +1,21 @@ +package=native_comparisontool +$(package)_version=0f7b5d8 +$(package)_download_path=https://github.com/TheBlueMatt/test-scripts/raw/38b490a2599d422b12d5ce8f165792f63fd8f54f +$(package)_file_name=pull-tests-$($(package)_version).jar +$(package)_sha256_hash=ecd43b988a8b673b483e4f69f931596360a5e90fc415c75c4c259faa690df198 +$(package)_install_dirname=BitcoindComparisonTool_jar +$(package)_install_filename=BitcoindComparisonTool.jar + +define $(package)_extract_cmds +endef + +define $(package)_configure_cmds +endef + +define $(package)_build_cmds +endef + +define $(package)_stage_cmds + mkdir -p $($(package)_staging_prefix_dir)/share/$($(package)_install_dirname) && \ + cp $($(package)_source) $($(package)_staging_prefix_dir)/share/$($(package)_install_dirname)/$($(package)_install_filename) +endef diff --git a/depends/packages/native_libdmg-hfsplus.mk b/depends/packages/native_libdmg-hfsplus.mk new file mode 100644 index 00000000..a4ffb604 --- /dev/null +++ b/depends/packages/native_libdmg-hfsplus.mk @@ -0,0 +1,22 @@ +package=native_libdmg-hfsplus +$(package)_version=0.1 +$(package)_download_path=https://github.com/theuni/libdmg-hfsplus/archive +$(package)_file_name=libdmg-hfsplus-v$($(package)_version).tar.gz +$(package)_sha256_hash=6569a02eb31c2827080d7d59001869ea14484c281efab0ae7f2b86af5c3120b3 +$(package)_build_subdir=build + +define $(package)_preprocess_cmds + mkdir build +endef + +define $(package)_config_cmds + cmake -DCMAKE_INSTALL_PREFIX:PATH=$(build_prefix)/bin .. +endef + +define $(package)_build_cmds + $(MAKE) -C dmg +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) -C dmg install +endef diff --git a/depends/packages/native_libuuid.mk b/depends/packages/native_libuuid.mk new file mode 100644 index 00000000..b25540f8 --- /dev/null +++ b/depends/packages/native_libuuid.mk @@ -0,0 +1,24 @@ +package:=native_libuuid +$(package)_version=1.41.14 +$(package)_download_path=http://downloads.sourceforge.net/e2fsprogs +$(package)_file_name=e2fsprogs-libs-$($(package)_version).tar.gz +$(package)_sha256_hash=dbc7a138a3218d9b80a0626b5b692d76934d6746d8cbb762751be33785d8d9f5 + +define $(package)_set_vars +$(package)_config_opts=--disable-elf-shlibs --disable-uuidd +$(package)_cflags+=-m32 +$(package)_ldflags+=-m32 +$(package)_cxxflags+=-m32 +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) -C lib/uuid +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) -C lib/uuid install +endef diff --git a/depends/packages/native_openssl.mk b/depends/packages/native_openssl.mk new file mode 100644 index 00000000..1f25d6af --- /dev/null +++ b/depends/packages/native_openssl.mk @@ -0,0 +1,21 @@ +package=native_openssl +$(package)_version=1.0.1h +$(package)_download_path=https://www.openssl.org/source +$(package)_file_name=openssl-$($(package)_version).tar.gz +$(package)_sha256_hash=9d1c8a9836aa63e2c6adb684186cbd4371c9e9dcc01d6e3bb447abf2d4d3d093 +define $(package)_set_vars +$(package)_build_config_opts= --prefix=$(build_prefix) no-zlib no-shared no-krb5C linux-generic32 -m32 +endef + +define $(package)_config_cmds + ./Configure $($(package)_build_config_opts) &&\ + sed -i "s|engines apps test|engines|" Makefile +endef + +define $(package)_build_cmds + $(MAKE) -j1 +endef + +define $(package)_stage_cmds + $(MAKE) INSTALL_PREFIX=$($(package)_staging_dir) -j1 install_sw +endef diff --git a/depends/packages/native_protobuf.mk b/depends/packages/native_protobuf.mk new file mode 100644 index 00000000..f36998bc --- /dev/null +++ b/depends/packages/native_protobuf.mk @@ -0,0 +1,26 @@ +package=native_protobuf +$(package)_version=2.5.0 +#$(package)_download_path=https://protobuf.googlecode.com/files +$(package)_download_path=https://github.com/google/protobuf/releases/download/v2.5.0 +$(package)_file_name=protobuf-$($(package)_version).tar.bz2 +$(package)_sha256_hash=13bfc5ae543cf3aa180ac2485c0bc89495e3ae711fc6fab4f8ffe90dfb4bb677 + +define $(package)_set_vars +$(package)_config_opts=--disable-shared +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) -C src protoc +endef + +define $(package)_stage_cmds + $(MAKE) -C src DESTDIR=$($(package)_staging_dir) install-strip +endef + +define $(package)_postprocess_cmds + rm -rf lib include +endef diff --git a/depends/packages/openssl.mk b/depends/packages/openssl.mk new file mode 100644 index 00000000..eb477941 --- /dev/null +++ b/depends/packages/openssl.mk @@ -0,0 +1,41 @@ +package=openssl +$(package)_version=1.0.1k +$(package)_download_path=https://www.openssl.org/source +$(package)_file_name=$(package)-$($(package)_version).tar.gz +$(package)_sha256_hash=8f9faeaebad088e772f4ef5e38252d472be4d878c6b3a2718c10a4fcebe7a41c + +define $(package)_set_vars +$(package)_config_env=AR="$($(package)_ar)" RANLIB="$($(package)_ranlib)" CC="$($(package)_cc)" +$(package)_config_opts=--prefix=$(host_prefix) --openssldir=$(host_prefix)/etc/openssl no-zlib no-shared no-dso +$(package)_config_opts+=no-krb5 no-camellia no-capieng no-cast no-cms no-dtls1 no-gost no-gmp no-heartbeats no-idea no-jpake no-md2 +$(package)_config_opts+=no-mdc2 no-rc5 no-rdrand no-rfc3779 no-rsax no-sctp no-seed no-sha0 no-static_engine no-whirlpool no-rc2 no-rc4 no-ssl2 no-ssl3 +$(package)_config_opts+=$($(package)_cflags) $($(package)_cppflags) +$(package)_config_opts_linux=-fPIC +$(package)_config_opts_x86_64_linux=linux-x86_64 +$(package)_config_opts_i686_linux=linux-generic32 +$(package)_config_opts_arm_linux=linux-generic32 +$(package)_config_opts_x86_64_darwin=darwin64-x86_64-cc +$(package)_config_opts_x86_64_mingw32=mingw64 +$(package)_config_opts_i686_mingw32=mingw +endef + +define $(package)_preprocess_cmds + sed -i.old "/define DATE/d" util/mkbuildinf.pl && \ + sed -i.old "s|engines apps test|engines|" Makefile.org +endef + +define $(package)_config_cmds + ./Configure $($(package)_config_opts) +endef + +define $(package)_build_cmds + $(MAKE) -j1 build_libs libcrypto.pc libssl.pc openssl.pc +endef + +define $(package)_stage_cmds + $(MAKE) INSTALL_PREFIX=$($(package)_staging_dir) -j1 install_sw +endef + +define $(package)_postprocess_cmds + rm -rf share bin etc +endef diff --git a/depends/packages/packages.mk b/depends/packages/packages.mk new file mode 100644 index 00000000..bbf53cc2 --- /dev/null +++ b/depends/packages/packages.mk @@ -0,0 +1,23 @@ +packages:=boost openssl +native_packages := native_ccache native_comparisontool + +qt_native_packages = native_protobuf +qt_packages = qrencode protobuf + +qt46_linux_packages = qt46 expat dbus libxcb xcb_proto libXau xproto freetype libX11 xextproto libXext xtrans libICE libSM +qt5_linux_packages= qt expat dbus libxcb xcb_proto libXau xproto freetype fontconfig libX11 xextproto libXext xtrans + +qt_darwin_packages=qt +qt_mingw32_packages=qt + +qt_linux_$(USE_LINUX_STATIC_QT5):=$(qt5_linux_packages) +qt_linux_:=$(qt46_linux_packages) +qt_linux_packages:=$(qt_linux_$(USE_LINUX_STATIC_QT5)) + +wallet_packages=bdb + +upnp_packages=miniupnpc + +ifneq ($(build_os),darwin) +darwin_native_packages=native_libuuid native_openssl native_cctools native_cdrkit native_libdmg-hfsplus +endif diff --git a/depends/packages/protobuf.mk b/depends/packages/protobuf.mk new file mode 100644 index 00000000..5affad28 --- /dev/null +++ b/depends/packages/protobuf.mk @@ -0,0 +1,28 @@ +package=protobuf +$(package)_version=$(native_$(package)_version) +$(package)_download_path=$(native_$(package)_download_path) +$(package)_file_name=$(native_$(package)_file_name) +$(package)_sha256_hash=$(native_$(package)_sha256_hash) +$(package)_dependencies=native_$(package) + +define $(package)_set_vars + $(package)_config_opts=--disable-shared --with-protoc=$(build_prefix)/bin/protoc + $(package)_config_opts_linux=--with-pic +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) -C src libprotobuf.la +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) -C src install-libLTLIBRARIES install-nobase_includeHEADERS &&\ + $(MAKE) DESTDIR=$($(package)_staging_dir) install-pkgconfigDATA +endef + +define $(package)_postprocess_cmds + rm lib/libprotoc.a +endef diff --git a/depends/packages/qrencode.mk b/depends/packages/qrencode.mk new file mode 100644 index 00000000..1ad329e9 --- /dev/null +++ b/depends/packages/qrencode.mk @@ -0,0 +1,22 @@ +package=qrencode +$(package)_version=3.4.3 +$(package)_download_path=https://fukuchi.org/works/qrencode/ +$(package)_file_name=qrencode-$(qrencode_version).tar.bz2 +$(package)_sha256_hash=dfd71487513c871bad485806bfd1fdb304dedc84d2b01a8fb8e0940b50597a98 + +define $(package)_set_vars +$(package)_config_opts=--disable-shared -without-tools --disable-sdltest +$(package)_config_opts_linux=--with-pic +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/packages/qt.mk b/depends/packages/qt.mk new file mode 100644 index 00000000..6a8e714a --- /dev/null +++ b/depends/packages/qt.mk @@ -0,0 +1,98 @@ +PACKAGE=qt +$(package)_version=5.2.1 +$(package)_download_path=http://download.qt-project.org/official_releases/qt/5.2/$($(package)_version)/single +$(package)_file_name=$(package)-everywhere-opensource-src-$($(package)_version).tar.gz +$(package)_sha256_hash=84e924181d4ad6db00239d87250cc89868484a14841f77fb85ab1f1dbdcd7da1 +$(package)_dependencies=openssl +$(package)_linux_dependencies=freetype fontconfig dbus libxcb libX11 xproto libXext +$(package)_build_subdir=qtbase +$(package)_qt_libs=corelib network widgets gui plugins testlib +$(package)_patches=mac-qmake.conf fix-xcb-include-order.patch qt5-tablet-osx.patch + +define $(package)_set_vars +$(package)_config_opts_release = -release +$(package)_config_opts_debug = -debug +$(package)_config_opts += -opensource -confirm-license -no-audio-backend -no-sql-tds -no-glib -no-icu +$(package)_config_opts += -no-cups -no-iconv -no-gif -no-audio-backend -no-freetype +$(package)_config_opts += -no-sql-sqlite -no-nis -no-cups -no-iconv -no-pch +$(package)_config_opts += -no-gif -no-feature-style-plastique +$(package)_config_opts += -no-qml-debug -no-pch -no-nis -nomake examples -nomake tests +$(package)_config_opts += -no-feature-style-cde -no-feature-style-s60 -no-feature-style-motif +$(package)_config_opts += -no-feature-style-windowsmobile -no-feature-style-windowsce +$(package)_config_opts += -no-feature-style-cleanlooks +$(package)_config_opts += -no-sql-db2 -no-sql-ibase -no-sql-oci -no-sql-tds -no-sql-mysql +$(package)_config_opts += -no-sql-odbc -no-sql-psql -no-sql-sqlite -no-sql-sqlite2 +$(package)_config_opts += -skip qtsvg -skip qtwebkit -skip qtwebkit-examples -skip qtserialport +$(package)_config_opts += -skip qtdeclarative -skip qtmultimedia -skip qtimageformats -skip qtx11extras +$(package)_config_opts += -skip qtlocation -skip qtsensors -skip qtquick1 -skip qtxmlpatterns +$(package)_config_opts += -skip qtquickcontrols -skip qtactiveqt -skip qtconnectivity -skip qtmacextras +$(package)_config_opts += -skip qtwinextras -skip qtxmlpatterns -skip qtscript -skip qtdoc + +$(package)_config_opts += -prefix $(host_prefix) -bindir $(build_prefix)/bin +$(package)_config_opts += -no-c++11 -openssl-linked -v -static -silent -pkg-config +$(package)_config_opts += -qt-libpng -qt-libjpeg -qt-zlib -qt-pcre + +ifneq ($(build_os),darwin) +$(package)_config_opts_darwin = -xplatform macx-clang-linux -device-option MAC_SDK_PATH=$(OSX_SDK) -device-option CROSS_COMPILE="$(host)-" +$(package)_config_opts_darwin += -device-option MAC_MIN_VERSION=$(OSX_MIN_VERSION) -device-option MAC_TARGET=$(host) +endif + +$(package)_config_opts_linux = -qt-xkbcommon -qt-xcb -no-eglfs -no-linuxfb -system-freetype -no-sm -fontconfig -no-xinput2 -no-libudev -no-egl -no-opengl +$(package)_config_opts_arm_linux = -platform linux-g++ -xplatform $(host) +$(package)_config_opts_i686_linux = -xplatform linux-g++-32 +$(package)_config_opts_mingw32 = -no-opengl -xplatform win32-g++ -device-option CROSS_COMPILE="$(host)-" +$(package)_build_env = QT_RCC_TEST=1 +endef + +define $(package)_preprocess_cmds + sed -i.old "s|updateqm.commands = \$$$$\$$$$LRELEASE|updateqm.commands = $($(package)_extract_dir)/qttools/bin/lrelease|" qttranslations/translations/translations.pro && \ + sed -i.old "s/src_plugins.depends = src_sql src_xml src_network/src_plugins.depends = src_xml src_network/" qtbase/src/src.pro && \ + sed -i.old "/XIproto.h/d" qtbase/src/plugins/platforms/xcb/qxcbxsettings.cpp && \ + sed -i.old 's/if \[ "$$$$XPLATFORM_MAC" = "yes" \]; then xspecvals=$$$$(macSDKify/if \[ "$$$$BUILD_ON_MAC" = "yes" \]; then xspecvals=$$$$(macSDKify/' qtbase/configure && \ + mkdir -p qtbase/mkspecs/macx-clang-linux &&\ + cp -f qtbase/mkspecs/macx-clang/Info.plist.lib qtbase/mkspecs/macx-clang-linux/ &&\ + cp -f qtbase/mkspecs/macx-clang/Info.plist.app qtbase/mkspecs/macx-clang-linux/ &&\ + cp -f qtbase/mkspecs/macx-clang/qplatformdefs.h qtbase/mkspecs/macx-clang-linux/ &&\ + cp -f $($(package)_patch_dir)/mac-qmake.conf qtbase/mkspecs/macx-clang-linux/qmake.conf && \ + patch -p1 < $($(package)_patch_dir)/fix-xcb-include-order.patch && \ + patch -p1 < $($(package)_patch_dir)/qt5-tablet-osx.patch && \ + echo "QMAKE_CFLAGS += $($(package)_cflags) $($(package)_cppflags)" >> qtbase/mkspecs/common/gcc-base.conf && \ + echo "QMAKE_CXXFLAGS += $($(package)_cxxflags) $($(package)_cppflags)" >> qtbase/mkspecs/common/gcc-base.conf && \ + echo "QMAKE_LFLAGS += $($(package)_ldflags)" >> qtbase/mkspecs/common/gcc-base.conf && \ + sed -i.old "s|QMAKE_CFLAGS = |QMAKE_CFLAGS = $($(package)_cflags) $($(package)_cppflags) |" qtbase/mkspecs/win32-g++/qmake.conf && \ + sed -i.old "s|QMAKE_LFLAGS = |QMAKE_LFLAGS = $($(package)_ldflags) |" qtbase/mkspecs/win32-g++/qmake.conf && \ + sed -i.old "s|QMAKE_CXXFLAGS = |QMAKE_CXXFLAGS = $($(package)_cxxflags) $($(package)_cppflags) |" qtbase/mkspecs/win32-g++/qmake.conf +endef + +define $(package)_config_cmds + export PKG_CONFIG_SYSROOT_DIR=/ && \ + export PKG_CONFIG_LIBDIR=$(host_prefix)/lib/pkgconfig && \ + export PKG_CONFIG_PATH=$(host_prefix)/share/pkgconfig && \ + export CPATH=$(host_prefix)/include && \ + ./configure $($(package)_config_opts) && \ + $(MAKE) sub-src-clean && \ + cd ../qttranslations && ../qtbase/bin/qmake qttranslations.pro -o Makefile && \ + cd translations && ../../qtbase/bin/qmake translations.pro -o Makefile && cd ../.. &&\ + cd qttools/src/linguist/lrelease/ && ../../../../qtbase/bin/qmake lrelease.pro -o Makefile +endef + +define $(package)_build_cmds + export CPATH=$(host_prefix)/include && \ + $(MAKE) -C src $(addprefix sub-,$($(package)_qt_libs)) && \ + $(MAKE) -C ../qttools/src/linguist/lrelease && \ + $(MAKE) -C ../qttranslations +endef + +define $(package)_stage_cmds + $(MAKE) -C src INSTALL_ROOT=$($(package)_staging_dir) $(addsuffix -install_subtargets,$(addprefix sub-,$($(package)_qt_libs))) && cd .. &&\ + $(MAKE) -C qttools/src/linguist/lrelease INSTALL_ROOT=$($(package)_staging_dir) install_target && \ + $(MAKE) -C qttranslations INSTALL_ROOT=$($(package)_staging_dir) install_subtargets && \ + if `test -f qtbase/src/plugins/platforms/xcb/xcb-static/libxcb-static.a`; then \ + cp qtbase/src/plugins/platforms/xcb/xcb-static/libxcb-static.a $($(package)_staging_prefix_dir)/lib; \ + fi +endef + +define $(package)_postprocess_cmds + rm -rf mkspecs/ lib/cmake/ && \ + rm lib/libQt5Bootstrap.a lib/lib*.la lib/*.prl plugins/*/*.prl +endef diff --git a/depends/packages/qt46.mk b/depends/packages/qt46.mk new file mode 100644 index 00000000..8fb30a5c --- /dev/null +++ b/depends/packages/qt46.mk @@ -0,0 +1,66 @@ +PACKAGE=qt46 +$(package)_version=4.6.4 +$(package)_download_path=http://download.qt-project.org/archive/qt/4.6/ +$(package)_file_name=qt-everywhere-opensource-src-$($(package)_version).tar.gz +$(package)_sha256_hash=9ad4d46c721b53a429ed5a2eecfd3c239a9ab566562f183f99d3125f1a234250 +$(package)_dependencies=openssl freetype dbus libX11 xproto libXext libICE libSM +$(package)_patches=stlfix.patch + +define $(package)_set_vars +$(package)_config_opts = -prefix $(host_prefix) -headerdir $(host_prefix)/include/qt4 -bindir $(build_prefix)/bin +$(package)_config_opts += -release -no-separate-debug-info -opensource -confirm-license +$(package)_config_opts += -stl -qt-zlib + +$(package)_config_opts += -nomake examples -nomake tests -nomake tools -nomake translations -nomake demos -nomake docs +$(package)_config_opts += -no-audio-backend -no-glib -no-nis -no-cups -no-iconv -no-gif -no-pch +$(package)_config_opts += -no-xkb -no-xrender -no-xrandr -no-xfixes -no-xcursor -no-xinerama -no-xsync -no-xinput -no-mitshm -no-xshape +$(package)_config_opts += -no-libtiff -no-fontconfig -openssl-linked +$(package)_config_opts += -no-sql-db2 -no-sql-ibase -no-sql-oci -no-sql-tds -no-sql-mysql +$(package)_config_opts += -no-sql-odbc -no-sql-psql -no-sql-sqlite -no-sql-sqlite2 +$(package)_config_opts += -no-xmlpatterns -no-multimedia -no-phonon -no-scripttools -no-declarative +$(package)_config_opts += -no-phonon-backend -no-webkit -no-javascript-jit -no-script +$(package)_config_opts += -no-svg -no-libjpeg -no-libtiff -no-libpng -no-libmng -no-qt3support -no-opengl + +$(package)_config_opts_x86_64_linux += -platform linux-g++-64 +$(package)_config_opts_i686_linux = -platform linux-g++-32 +$(package)_build_env = QT_RCC_TEST=1 +endef + +define $(package)_preprocess_cmds + sed -i.old "s|/include /usr/include||" config.tests/unix/freetype/freetype.pri && \ + sed -i.old "s|src_plugins.depends = src_gui src_sql src_svg|src_plugins.depends = src_gui src_sql|" src/src.pro && \ + sed -i.old "s|\.lower(|\.toLower(|g" src/network/ssl/qsslsocket_openssl.cpp && \ + sed -i.old "s|Key_BackSpace|Key_Backspace|" src/gui/itemviews/qabstractitemview.cpp && \ + sed -i.old "s|/usr/X11R6/lib64|$(host_prefix)/lib|" mkspecs/*/*.conf && \ + sed -i.old "s|/usr/X11R6/lib|$(host_prefix)/lib|" mkspecs/*/*.conf && \ + sed -i.old "s|/usr/X11R6/include|$(host_prefix)/include|" mkspecs/*/*.conf && \ + sed -i.old "s|QMAKE_LFLAGS_SHLIB\t+= -shared|QMAKE_LFLAGS_SHLIB\t+= -shared -Wl,--exclude-libs,ALL|" mkspecs/common/g++.conf && \ + sed -i.old "/SSLv2_client_method/d" src/network/ssl/qsslsocket_openssl.cpp src/network/ssl/qsslsocket_openssl_symbols.cpp && \ + sed -i.old "/SSLv2_server_method/d" src/network/ssl/qsslsocket_openssl.cpp src/network/ssl/qsslsocket_openssl_symbols.cpp && \ + patch -p1 < $($(package)_patch_dir)/stlfix.patch +endef + +define $(package)_config_cmds + export PKG_CONFIG_SYSROOT_DIR=/ && \ + export PKG_CONFIG_LIBDIR=$(host_prefix)/lib/pkgconfig && \ + export PKG_CONFIG_PATH=$(host_prefix)/share/pkgconfig && \ + export CPATH=$(host_prefix)/include && \ + OPENSSL_LIBS='-L$(host_prefix)/lib -lssl -lcrypto' ./configure $($(package)_config_opts) && \ + cd tools/linguist/lrelease; ../../../bin/qmake -o Makefile lrelease.pro +endef + +define $(package)_build_cmds + export CPATH=$(host_prefix)/include && \ + $(MAKE) -C src && \ + $(MAKE) -C tools/linguist/lrelease +endef + +define $(package)_stage_cmds + $(MAKE) -C src INSTALL_ROOT=$($(package)_staging_dir) install && \ + $(MAKE) -C tools/linguist/lrelease INSTALL_ROOT=$($(package)_staging_dir) install +endef + +define $(package)_postprocess_cmds + rm -rf mkspecs/ lib/cmake/ lib/*.prl lib/*.la && \ + find native/bin -type f -exec mv {} {}-qt4 \; +endef diff --git a/depends/packages/xcb_proto.mk b/depends/packages/xcb_proto.mk new file mode 100644 index 00000000..0c7c958d --- /dev/null +++ b/depends/packages/xcb_proto.mk @@ -0,0 +1,27 @@ +package=xcb_proto +$(package)_version=1.10 +$(package)_download_path=http://xcb.freedesktop.org/dist +$(package)_file_name=xcb-proto-$($(package)_version).tar.bz2 +$(package)_sha256_hash=7ef40ddd855b750bc597d2a435da21e55e502a0fefa85b274f2c922800baaf05 + +define $(package)_set_vars + $(package)_config_opts=--disable-shared + $(package)_config_opts_linux=--with-pic +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef + +define $(package)_postprocess_cmds + find -name "*.pyc" -delete && \ + find -name "*.pyo" -delete +endef diff --git a/depends/packages/xextproto.mk b/depends/packages/xextproto.mk new file mode 100644 index 00000000..98a11eb4 --- /dev/null +++ b/depends/packages/xextproto.mk @@ -0,0 +1,21 @@ +package=xextproto +$(package)_version=7.3.0 +$(package)_download_path=http://xorg.freedesktop.org/releases/individual/proto +$(package)_file_name=$(package)-$($(package)_version).tar.bz2 +$(package)_sha256_hash=f3f4b23ac8db9c3a9e0d8edb591713f3d70ef9c3b175970dd8823dfc92aa5bb0 + +define $(package)_set_vars +$(package)_config_opts=--disable-shared +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/packages/xproto.mk b/depends/packages/xproto.mk new file mode 100644 index 00000000..50a90b26 --- /dev/null +++ b/depends/packages/xproto.mk @@ -0,0 +1,21 @@ +package=xproto +$(package)_version=7.0.26 +$(package)_download_path=http://xorg.freedesktop.org/releases/individual/proto +$(package)_file_name=$(package)-$($(package)_version).tar.bz2 +$(package)_sha256_hash=636162c1759805a5a0114a369dffdeccb8af8c859ef6e1445f26a4e6e046514f + +define $(package)_set_vars +$(package)_config_opts=--disable-shared +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/packages/xtrans.mk b/depends/packages/xtrans.mk new file mode 100644 index 00000000..99eefa6d --- /dev/null +++ b/depends/packages/xtrans.mk @@ -0,0 +1,22 @@ +package=xtrans +$(package)_version=1.3.4 +$(package)_download_path=http://xorg.freedesktop.org/releases/individual/lib/ +$(package)_file_name=$(package)-$($(package)_version).tar.bz2 +$(package)_sha256_hash=054d4ee3efd52508c753e9f7bc655ef185a29bd2850dd9e2fc2ccc33544f583a +$(package)_dependencies= + +define $(package)_set_vars +$(package)_config_opts_linux=--with-pic --disable-static +endef + +define $(package)_config_cmds + $($(package)_autoconf) +endef + +define $(package)_build_cmds + $(MAKE) +endef + +define $(package)_stage_cmds + $(MAKE) DESTDIR=$($(package)_staging_dir) install +endef diff --git a/depends/patches/boost/darwin_boost_atomic-1.patch b/depends/patches/boost/darwin_boost_atomic-1.patch new file mode 100644 index 00000000..97f59cb7 --- /dev/null +++ b/depends/patches/boost/darwin_boost_atomic-1.patch @@ -0,0 +1,35 @@ +diff --git a/include/boost/atomic/detail/cas128strong.hpp b/include/boost/atomic/detail/cas128strong.hpp +index 906c13e..dcb4d7d 100644 +--- a/include/boost/atomic/detail/cas128strong.hpp ++++ b/include/boost/atomic/detail/cas128strong.hpp +@@ -196,15 +196,17 @@ class base_atomic + + public: + BOOST_DEFAULTED_FUNCTION(base_atomic(void), {}) +- explicit base_atomic(value_type const& v) BOOST_NOEXCEPT : v_(0) ++ explicit base_atomic(value_type const& v) BOOST_NOEXCEPT + { ++ memset(&v_, 0, sizeof(v_)); + memcpy(&v_, &v, sizeof(value_type)); + } + + void + store(value_type const& value, memory_order order = memory_order_seq_cst) volatile BOOST_NOEXCEPT + { +- storage_type value_s = 0; ++ storage_type value_s; ++ memset(&value_s, 0, sizeof(value_s)); + memcpy(&value_s, &value, sizeof(value_type)); + platform_fence_before_store(order); + platform_store128(value_s, &v_); +@@ -247,7 +249,9 @@ class base_atomic + memory_order success_order, + memory_order failure_order) volatile BOOST_NOEXCEPT + { +- storage_type expected_s = 0, desired_s = 0; ++ storage_type expected_s, desired_s; ++ memset(&expected_s, 0, sizeof(expected_s)); ++ memset(&desired_s, 0, sizeof(desired_s)); + memcpy(&expected_s, &expected, sizeof(value_type)); + memcpy(&desired_s, &desired, sizeof(value_type)); + diff --git a/depends/patches/boost/darwin_boost_atomic-2.patch b/depends/patches/boost/darwin_boost_atomic-2.patch new file mode 100644 index 00000000..ca507652 --- /dev/null +++ b/depends/patches/boost/darwin_boost_atomic-2.patch @@ -0,0 +1,55 @@ +diff --git a/include/boost/atomic/detail/gcc-atomic.hpp b/include/boost/atomic/detail/gcc-atomic.hpp +index a130590..4af99a1 100644 +--- a/include/boost/atomic/detail/gcc-atomic.hpp ++++ b/include/boost/atomic/detail/gcc-atomic.hpp +@@ -958,14 +958,16 @@ class base_atomic + + public: + BOOST_DEFAULTED_FUNCTION(base_atomic(void), {}) +- explicit base_atomic(value_type const& v) BOOST_NOEXCEPT : v_(0) ++ explicit base_atomic(value_type const& v) BOOST_NOEXCEPT + { ++ memset(&v_, 0, sizeof(v_)); + memcpy(&v_, &v, sizeof(value_type)); + } + + void store(value_type const& v, memory_order order = memory_order_seq_cst) volatile BOOST_NOEXCEPT + { +- storage_type tmp = 0; ++ storage_type tmp; ++ memset(&tmp, 0, sizeof(tmp)); + memcpy(&tmp, &v, sizeof(value_type)); + __atomic_store_n(&v_, tmp, atomics::detail::convert_memory_order_to_gcc(order)); + } +@@ -980,7 +982,8 @@ class base_atomic + + value_type exchange(value_type const& v, memory_order order = memory_order_seq_cst) volatile BOOST_NOEXCEPT + { +- storage_type tmp = 0; ++ storage_type tmp; ++ memset(&tmp, 0, sizeof(tmp)); + memcpy(&tmp, &v, sizeof(value_type)); + tmp = __atomic_exchange_n(&v_, tmp, atomics::detail::convert_memory_order_to_gcc(order)); + value_type res; +@@ -994,7 +997,9 @@ class base_atomic + memory_order success_order, + memory_order failure_order) volatile BOOST_NOEXCEPT + { +- storage_type expected_s = 0, desired_s = 0; ++ storage_type expected_s, desired_s; ++ memset(&expected_s, 0, sizeof(expected_s)); ++ memset(&desired_s, 0, sizeof(desired_s)); + memcpy(&expected_s, &expected, sizeof(value_type)); + memcpy(&desired_s, &desired, sizeof(value_type)); + const bool success = __atomic_compare_exchange_n(&v_, &expected_s, desired_s, false, +@@ -1010,7 +1015,9 @@ class base_atomic + memory_order success_order, + memory_order failure_order) volatile BOOST_NOEXCEPT + { +- storage_type expected_s = 0, desired_s = 0; ++ storage_type expected_s, desired_s; ++ memset(&expected_s, 0, sizeof(expected_s)); ++ memset(&desired_s, 0, sizeof(desired_s)); + memcpy(&expected_s, &expected, sizeof(value_type)); + memcpy(&desired_s, &desired, sizeof(value_type)); + const bool success = __atomic_compare_exchange_n(&v_, &expected_s, desired_s, true, diff --git a/depends/patches/native_cdrkit/cdrkit-deterministic.patch b/depends/patches/native_cdrkit/cdrkit-deterministic.patch new file mode 100644 index 00000000..8ab0993d --- /dev/null +++ b/depends/patches/native_cdrkit/cdrkit-deterministic.patch @@ -0,0 +1,86 @@ +--- cdrkit-1.1.11.old/genisoimage/tree.c 2008-10-21 19:57:47.000000000 -0400 ++++ cdrkit-1.1.11/genisoimage/tree.c 2013-12-06 00:23:18.489622668 -0500 +@@ -1139,8 +1139,9 @@ + scan_directory_tree(struct directory *this_dir, char *path, + struct directory_entry *de) + { +- DIR *current_dir; ++ int current_file; + char whole_path[PATH_MAX]; ++ struct dirent **d_list; + struct dirent *d_entry; + struct directory *parent; + int dflag; +@@ -1164,7 +1165,8 @@ + this_dir->dir_flags |= DIR_WAS_SCANNED; + + errno = 0; /* Paranoia */ +- current_dir = opendir(path); ++ //current_dir = opendir(path); ++ current_file = scandir(path, &d_list, NULL, alphasort); + d_entry = NULL; + + /* +@@ -1173,12 +1175,12 @@ + */ + old_path = path; + +- if (current_dir) { ++ if (current_file >= 0) { + errno = 0; +- d_entry = readdir(current_dir); ++ d_entry = d_list[0]; + } + +- if (!current_dir || !d_entry) { ++ if (current_file < 0 || !d_entry) { + int ret = 1; + + #ifdef USE_LIBSCHILY +@@ -1191,8 +1193,8 @@ + de->isorec.flags[0] &= ~ISO_DIRECTORY; + ret = 0; + } +- if (current_dir) +- closedir(current_dir); ++ if(d_list) ++ free(d_list); + return (ret); + } + #ifdef ABORT_DEEP_ISO_ONLY +@@ -1208,7 +1210,7 @@ + errmsgno(EX_BAD, "use Rock Ridge extensions via -R or -r,\n"); + errmsgno(EX_BAD, "or allow deep ISO9660 directory nesting via -D.\n"); + } +- closedir(current_dir); ++ free(d_list); + return (1); + } + #endif +@@ -1250,13 +1252,13 @@ + * The first time through, skip this, since we already asked + * for the first entry when we opened the directory. + */ +- if (dflag) +- d_entry = readdir(current_dir); ++ if (dflag && current_file >= 0) ++ d_entry = d_list[current_file]; + dflag++; + +- if (!d_entry) ++ if (current_file < 0) + break; +- ++ current_file--; + /* OK, got a valid entry */ + + /* If we do not want all files, then pitch the backups. */ +@@ -1348,7 +1350,7 @@ + insert_file_entry(this_dir, whole_path, d_entry->d_name); + #endif /* APPLE_HYB */ + } +- closedir(current_dir); ++ free(d_list); + + #ifdef APPLE_HYB + /* diff --git a/depends/patches/qt/fix-xcb-include-order.patch b/depends/patches/qt/fix-xcb-include-order.patch new file mode 100644 index 00000000..bf6c6dca --- /dev/null +++ b/depends/patches/qt/fix-xcb-include-order.patch @@ -0,0 +1,21 @@ +--- old/qtbase/src/plugins/platforms/xcb/xcb-plugin.pro 2014-07-30 18:17:27.384458441 -0400 ++++ new/qtbase/src/plugins/platforms/xcb/xcb-plugin.pro 2014-07-30 18:18:28.620459303 -0400 +@@ -101,10 +101,6 @@ + } + } + +-DEFINES += $$QMAKE_DEFINES_XCB +-LIBS += $$QMAKE_LIBS_XCB +-QMAKE_CXXFLAGS += $$QMAKE_CFLAGS_XCB +- + CONFIG += qpa/genericunixfontdatabase + + contains(QT_CONFIG, dbus) { +@@ -141,3 +137,7 @@ + INCLUDEPATH += ../../../3rdparty/xkbcommon/xkbcommon/ + } + } ++ ++DEFINES += $$QMAKE_DEFINES_XCB ++LIBS += $$QMAKE_LIBS_XCB ++INCLUDEPATH += $$QMAKE_CFLAGS_XCB diff --git a/depends/patches/qt/mac-qmake.conf b/depends/patches/qt/mac-qmake.conf new file mode 100644 index 00000000..f7302265 --- /dev/null +++ b/depends/patches/qt/mac-qmake.conf @@ -0,0 +1,23 @@ +MAKEFILE_GENERATOR = UNIX +CONFIG += app_bundle incremental global_init_link_order lib_version_first plugin_no_soname absolute_library_soname +QMAKE_INCREMENTAL_STYLE = sublib +include(../common/macx.conf) +include(../common/gcc-base-mac.conf) +include(../common/clang.conf) +include(../common/clang-mac.conf) +QMAKE_MAC_SDK_PATH=$${MAC_SDK_PATH} +QMAKE_XCODE_VERSION=4.3 +QMAKE_XCODE_DEVELOPER_PATH=/Developer +QMAKE_MACOSX_DEPLOYMENT_TARGET = $${MAC_MIN_VERSION} +QMAKE_MAC_SDK=macosx +QMAKE_MAC_SDK.macosx.path = $$QMAKE_MAC_SDK_PATH +QMAKE_MAC_SDK.macosx.platform_name = macosx +QMAKE_CFLAGS += -target $${MAC_TARGET} +QMAKE_OBJECTIVE_CFLAGS += $$QMAKE_CFLAGS +QMAKE_CXXFLAGS += $$QMAKE_CFLAGS +QMAKE_LFLAGS += -target $${MAC_TARGET} +QMAKE_AR = $${CROSS_COMPILE}ar cq +QMAKE_RANLIB=$${CROSS_COMPILE}ranlib +QMAKE_LIBTOOL=$${CROSS_COMPILE}libtool +QMAKE_INSTALL_NAME_TOOL=$${CROSS_COMPILE}install_name_tool +load(qt_config) diff --git a/depends/patches/qt/qt5-tablet-osx.patch b/depends/patches/qt/qt5-tablet-osx.patch new file mode 100644 index 00000000..7deabf8d --- /dev/null +++ b/depends/patches/qt/qt5-tablet-osx.patch @@ -0,0 +1,20 @@ +--- old/qtbase/src/widgets/kernel/qwidgetwindow.cpp 2014-09-05 20:45:18.717570370 -0400 ++++ new/qtbase/src/widgets/kernel/qwidgetwindow.cpp 2014-09-05 20:52:38.653576561 -0400 +@@ -57,7 +57,7 @@ + Q_WIDGETS_EXPORT extern bool qt_tab_all_widgets(); + + QWidget *qt_button_down = 0; // widget got last button-down +-static QWidget *qt_tablet_target = 0; ++static QPointer qt_tablet_target = 0; + + // popup control + QWidget *qt_popup_down = 0; // popup that contains the pressed widget +@@ -96,8 +96,6 @@ + + QWidgetWindow::~QWidgetWindow() + { +- if (m_widget == qt_tablet_target) +- qt_tablet_target = 0; + } + + #ifndef QT_NO_ACCESSIBILITY diff --git a/depends/patches/qt46/stlfix.patch b/depends/patches/qt46/stlfix.patch new file mode 100644 index 00000000..f8f6fb04 --- /dev/null +++ b/depends/patches/qt46/stlfix.patch @@ -0,0 +1,10 @@ +--- old/config.tests/unix/stl/stltest.cpp 2011-06-23 03:45:23.000000000 -0400 ++++ new/config.tests/unix/stl/stltest.cpp 2014-08-28 00:54:04.154837604 -0400 +@@ -49,6 +49,7 @@ + #include + #include + #include ++#include + + // something mean to see if the compiler and C++ standard lib are good enough + template diff --git a/pkg.m4 b/pkg.m4 new file mode 100644 index 00000000..c5b26b52 --- /dev/null +++ b/pkg.m4 @@ -0,0 +1,214 @@ +# pkg.m4 - Macros to locate and utilise pkg-config. -*- Autoconf -*- +# serial 1 (pkg-config-0.24) +# +# Copyright © 2004 Scott James Remnant . +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +# +# As a special exception to the GNU General Public License, if you +# distribute this file as part of a program that contains a +# configuration script generated by Autoconf, you may include it under +# the same distribution terms that you use for the rest of that program. + +# PKG_PROG_PKG_CONFIG([MIN-VERSION]) +# ---------------------------------- +AC_DEFUN([PKG_PROG_PKG_CONFIG], +[m4_pattern_forbid([^_?PKG_[A-Z_]+$]) +m4_pattern_allow([^PKG_CONFIG(_(PATH|LIBDIR|SYSROOT_DIR|ALLOW_SYSTEM_(CFLAGS|LIBS)))?$]) +m4_pattern_allow([^PKG_CONFIG_(DISABLE_UNINSTALLED|TOP_BUILD_DIR|DEBUG_SPEW)$]) +AC_ARG_VAR([PKG_CONFIG], [path to pkg-config utility]) +AC_ARG_VAR([PKG_CONFIG_PATH], [directories to add to pkg-config's search path]) +AC_ARG_VAR([PKG_CONFIG_LIBDIR], [path overriding pkg-config's built-in search path]) + +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then + AC_PATH_TOOL([PKG_CONFIG], [pkg-config]) +fi +if test -n "$PKG_CONFIG"; then + _pkg_min_version=m4_default([$1], [0.9.0]) + AC_MSG_CHECKING([pkg-config is at least version $_pkg_min_version]) + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then + AC_MSG_RESULT([yes]) + else + AC_MSG_RESULT([no]) + PKG_CONFIG="" + fi +fi[]dnl +])# PKG_PROG_PKG_CONFIG + +# PKG_CHECK_EXISTS(MODULES, [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND]) +# +# Check to see whether a particular set of modules exists. Similar +# to PKG_CHECK_MODULES(), but does not set variables or print errors. +# +# Please remember that m4 expands AC_REQUIRE([PKG_PROG_PKG_CONFIG]) +# only at the first occurence in configure.ac, so if the first place +# it's called might be skipped (such as if it is within an "if", you +# have to call PKG_CHECK_EXISTS manually +# -------------------------------------------------------------- +AC_DEFUN([PKG_CHECK_EXISTS], +[AC_REQUIRE([PKG_PROG_PKG_CONFIG])dnl +if test -n "$PKG_CONFIG" && \ + AC_RUN_LOG([$PKG_CONFIG --exists --print-errors "$1"]); then + m4_default([$2], [:]) +m4_ifvaln([$3], [else + $3])dnl +fi]) + +# _PKG_CONFIG([VARIABLE], [COMMAND], [MODULES]) +# --------------------------------------------- +m4_define([_PKG_CONFIG], +[if test -n "$$1"; then + pkg_cv_[]$1="$$1" + elif test -n "$PKG_CONFIG"; then + PKG_CHECK_EXISTS([$3], + [pkg_cv_[]$1=`$PKG_CONFIG --[]$2 "$3" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes ], + [pkg_failed=yes]) + else + pkg_failed=untried +fi[]dnl +])# _PKG_CONFIG + +# _PKG_SHORT_ERRORS_SUPPORTED +# ----------------------------- +AC_DEFUN([_PKG_SHORT_ERRORS_SUPPORTED], +[AC_REQUIRE([PKG_PROG_PKG_CONFIG]) +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi[]dnl +])# _PKG_SHORT_ERRORS_SUPPORTED + + +# PKG_CHECK_MODULES(VARIABLE-PREFIX, MODULES, [ACTION-IF-FOUND], +# [ACTION-IF-NOT-FOUND]) +# +# +# Note that if there is a possibility the first call to +# PKG_CHECK_MODULES might not happen, you should be sure to include an +# explicit call to PKG_PROG_PKG_CONFIG in your configure.ac +# +# +# -------------------------------------------------------------- +AC_DEFUN([PKG_CHECK_MODULES], +[AC_REQUIRE([PKG_PROG_PKG_CONFIG])dnl +AC_ARG_VAR([$1][_CFLAGS], [C compiler flags for $1, overriding pkg-config])dnl +AC_ARG_VAR([$1][_LIBS], [linker flags for $1, overriding pkg-config])dnl + +pkg_failed=no +AC_MSG_CHECKING([for $1]) + +_PKG_CONFIG([$1][_CFLAGS], [cflags], [$2]) +_PKG_CONFIG([$1][_LIBS], [libs], [$2]) + +m4_define([_PKG_TEXT], [Alternatively, you may set the environment variables $1[]_CFLAGS +and $1[]_LIBS to avoid the need to call pkg-config. +See the pkg-config man page for more details.]) + +if test $pkg_failed = yes; then + AC_MSG_RESULT([no]) + _PKG_SHORT_ERRORS_SUPPORTED + if test $_pkg_short_errors_supported = yes; then + $1[]_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "$2" 2>&1` + else + $1[]_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "$2" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$$1[]_PKG_ERRORS" >&AS_MESSAGE_LOG_FD + + m4_default([$4], [AC_MSG_ERROR( +[Package requirements ($2) were not met: + +$$1_PKG_ERRORS + +Consider adjusting the PKG_CONFIG_PATH environment variable if you +installed software in a non-standard prefix. + +_PKG_TEXT])[]dnl + ]) +elif test $pkg_failed = untried; then + AC_MSG_RESULT([no]) + m4_default([$4], [AC_MSG_FAILURE( +[The pkg-config script could not be found or is too old. Make sure it +is in your PATH or set the PKG_CONFIG environment variable to the full +path to pkg-config. + +_PKG_TEXT + +To get pkg-config, see .])[]dnl + ]) +else + $1[]_CFLAGS=$pkg_cv_[]$1[]_CFLAGS + $1[]_LIBS=$pkg_cv_[]$1[]_LIBS + AC_MSG_RESULT([yes]) + $3 +fi[]dnl +])# PKG_CHECK_MODULES + + +# PKG_INSTALLDIR(DIRECTORY) +# ------------------------- +# Substitutes the variable pkgconfigdir as the location where a module +# should install pkg-config .pc files. By default the directory is +# $libdir/pkgconfig, but the default can be changed by passing +# DIRECTORY. The user can override through the --with-pkgconfigdir +# parameter. +AC_DEFUN([PKG_INSTALLDIR], +[m4_pushdef([pkg_default], [m4_default([$1], ['${libdir}/pkgconfig'])]) +m4_pushdef([pkg_description], + [pkg-config installation directory @<:@]pkg_default[@:>@]) +AC_ARG_WITH([pkgconfigdir], + [AS_HELP_STRING([--with-pkgconfigdir], pkg_description)],, + [with_pkgconfigdir=]pkg_default) +AC_SUBST([pkgconfigdir], [$with_pkgconfigdir]) +m4_popdef([pkg_default]) +m4_popdef([pkg_description]) +]) dnl PKG_INSTALLDIR + + +# PKG_NOARCH_INSTALLDIR(DIRECTORY) +# ------------------------- +# Substitutes the variable noarch_pkgconfigdir as the location where a +# module should install arch-independent pkg-config .pc files. By +# default the directory is $datadir/pkgconfig, but the default can be +# changed by passing DIRECTORY. The user can override through the +# --with-noarch-pkgconfigdir parameter. +AC_DEFUN([PKG_NOARCH_INSTALLDIR], +[m4_pushdef([pkg_default], [m4_default([$1], ['${datadir}/pkgconfig'])]) +m4_pushdef([pkg_description], + [pkg-config arch-independent installation directory @<:@]pkg_default[@:>@]) +AC_ARG_WITH([noarch-pkgconfigdir], + [AS_HELP_STRING([--with-noarch-pkgconfigdir], pkg_description)],, + [with_noarch_pkgconfigdir=]pkg_default) +AC_SUBST([noarch_pkgconfigdir], [$with_noarch_pkgconfigdir]) +m4_popdef([pkg_default]) +m4_popdef([pkg_description]) +]) dnl PKG_NOARCH_INSTALLDIR + + +# PKG_CHECK_VAR(VARIABLE, MODULE, CONFIG-VARIABLE, +# [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND]) +# ------------------------------------------- +# Retrieves the value of the pkg-config variable for the given module. +AC_DEFUN([PKG_CHECK_VAR], +[AC_REQUIRE([PKG_PROG_PKG_CONFIG])dnl +AC_ARG_VAR([$1], [value of $3 for $2, overriding pkg-config])dnl + +_PKG_CONFIG([$1], [variable="][$3]["], [$2]) +AS_VAR_COPY([$1], [pkg_cv_][$1]) + +AS_VAR_IF([$1], [""], [$5], [$4])dnl +])# PKG_CHECK_VAR diff --git a/src/.clang-format b/src/.clang-format new file mode 100644 index 00000000..226a15d1 --- /dev/null +++ b/src/.clang-format @@ -0,0 +1,51 @@ +AccessModifierOffset: -4 +AlignEscapedNewlinesLeft: true +AlignTrailingComments: true +AllowAllParametersOfDeclarationOnNextLine: false +AllowShortBlocksOnASingleLine: false +AllowShortFunctionsOnASingleLine: All +AllowShortIfStatementsOnASingleLine: false +AllowShortLoopsOnASingleLine: false +AlwaysBreakBeforeMultilineStrings: false +AlwaysBreakTemplateDeclarations: true +BinPackParameters: false +BreakBeforeBinaryOperators: false +BreakBeforeBraces: Linux +BreakBeforeTernaryOperators: false +BreakConstructorInitializersBeforeComma: false +ColumnLimit: 0 +CommentPragmas: '^ IWYU pragma:' +ConstructorInitializerAllOnOneLineOrOnePerLine: false +ConstructorInitializerIndentWidth: 4 +ContinuationIndentWidth: 4 +Cpp11BracedListStyle: true +DerivePointerAlignment: false +DisableFormat: false +ForEachMacros: [ foreach, Q_FOREACH, BOOST_FOREACH, BOOST_REVERSE_FOREACH ] +IndentCaseLabels: false +IndentFunctionDeclarationAfterType: false +IndentWidth: 4 +KeepEmptyLinesAtTheStartOfBlocks: false +Language: Cpp +MaxEmptyLinesToKeep: 2 +NamespaceIndentation: None +ObjCSpaceAfterProperty: false +ObjCSpaceBeforeProtocolList: false +PenaltyBreakBeforeFirstCallParameter: 1 +PenaltyBreakComment: 300 +PenaltyBreakFirstLessLess: 120 +PenaltyBreakString: 1000 +PenaltyExcessCharacter: 1000000 +PenaltyReturnTypeOnItsOwnLine: 200 +PointerAlignment: Left +SpaceBeforeAssignmentOperators: true +SpaceBeforeParens: ControlStatements +SpaceInEmptyParentheses: false +SpacesBeforeTrailingComments: 1 +SpacesInAngles: false +SpacesInContainerLiterals: true +SpacesInCStyleCastParentheses: false +SpacesInParentheses: false +Standard: Cpp03 +TabWidth: 8 +UseTab: Never diff --git a/src/Makefile.am b/src/Makefile.am new file mode 100644 index 00000000..3a7e2238 --- /dev/null +++ b/src/Makefile.am @@ -0,0 +1,543 @@ +walDIST_SUBDIRS = secp256k1 +AM_LDFLAGS = $(PTHREAD_CFLAGS) $(LIBTOOL_LDFLAGS) + + +if EMBEDDED_LEVELDB +LEVELDB_CPPFLAGS += -I$(srcdir)/leveldb/include +LEVELDB_CPPFLAGS += -I$(srcdir)/leveldb/helpers/memenv +LIBLEVELDB += $(builddir)/leveldb/libleveldb.a +LIBMEMENV += $(builddir)/leveldb/libmemenv.a + +# NOTE: This dependency is not strictly necessary, but without it make may try to build both in parallel, which breaks the LevelDB build system in a race +$(LIBLEVELDB): $(LIBMEMENV) + +$(LIBLEVELDB) $(LIBMEMENV): + @echo "Building LevelDB ..." && $(MAKE) -C $(@D) $(@F) CXX="$(CXX)" \ + CC="$(CC)" PLATFORM=$(TARGET_OS) AR="$(AR)" $(LEVELDB_TARGET_FLAGS) \ + OPT="$(CXXFLAGS) $(CPPFLAGS)" +endif + +BITCOIN_CONFIG_INCLUDES=-I$(builddir)/config +BITCOIN_INCLUDES=-I$(builddir) -I$(builddir)/obj $(BOOST_CPPFLAGS) $(LEVELDB_CPPFLAGS) $(CRYPTO_CFLAGS) $(SSL_CFLAGS) + +BITCOIN_INCLUDES += -I$(srcdir)/secp256k1/include + +LIBBITCOIN_SERVER=libbitcoin_server.a +LIBBITCOIN_WALLET=libbitcoin_wallet.a +LIBBITCOIN_COMMON=libbitcoin_common.a +LIBBITCOIN_CLI=libbitcoin_cli.a +LIBBITCOIN_UTIL=libbitcoin_util.a +LIBBITCOIN_CRYPTO=crypto/libbitcoin_crypto.a +LIBBITCOIN_MULTICHAIN=multichain/libbitcoin_multichain.a +LIBBITCOIN_UNIVALUE=univalue/libbitcoin_univalue.a +LIBBITCOINQT=qt/libbitcoinqt.a +LIBSECP256K1=secp256k1/libsecp256k1.la + +$(LIBSECP256K1): $(wildcard secp256k1/src/*) $(wildcard secp256k1/include/*) + $(AM_V_at)$(MAKE) $(AM_MAKEFLAGS) -C $(@D) $(@F) + +# Make is not made aware of per-object dependencies to avoid limiting building parallelization +# But to build the less dependent modules first, we manually select their order here: +EXTRA_LIBRARIES = \ + crypto/libbitcoin_crypto.a \ + multichain/libbitcoin_multichain.a \ + libbitcoin_util.a \ + libbitcoin_common.a \ + univalue/libbitcoin_univalue.a \ + libbitcoin_server.a \ + libbitcoin_cli.a +if ENABLE_WALLET +BITCOIN_INCLUDES += $(BDB_CPPFLAGS) +EXTRA_LIBRARIES += libbitcoin_wallet.a +endif + +if BUILD_BITCOIN_LIBS +lib_LTLIBRARIES = libbitcoinconsensus.la +LIBBITCOIN_CONSENSUS=libbitcoinconsensus.la +else +LIBBITCOIN_CONSENSUS= +endif + +bin_PROGRAMS = +TESTS = + +if BUILD_BITCOIND +# bin_PROGRAMS += bitcoind multichaind # MCHN + bin_PROGRAMS += multichaind # MCHN +endif + +if BUILD_BITCOIN_UTILS +# bin_PROGRAMS += multichain-util multichain-cli bitcoin-cli bitcoin-tx # MCHN + bin_PROGRAMS += multichain-util multichain-cli # MCHN +endif + +.PHONY: FORCE +# bitcoin core # +BITCOIN_CORE_H = \ + storage/addrman.h \ + structs/alert.h \ + utils/allocators.h \ + structs/amount.h \ + structs/base58.h \ + structs/bloom.h \ + chain/chain.h \ + chainparams/chainparams.h \ + chainparams/chainparamsbase.h \ + chainparams/chainparamsseeds.h \ + chain/checkpoints.h \ + checkqueue.h \ + version/clientversion.h \ + coincontrol.h \ + storage/coins.h \ + utils/compat.h \ + utils/compressor.h \ + primitives/block.h \ + primitives/transaction.h \ + utils/core_io.h \ + wallet/crypter.h \ + wallet/db.h \ + structs/hash.h \ + core/init.h \ + keys/key.h \ + wallet/keystore.h \ + storage/leveldbwrapper.h \ + structs/limitedmap.h \ + core/main.h \ + chain/merkleblock.h \ + miner/miner.h \ + utils/mruset.h \ + net/netbase.h \ + net/net.h \ + ui/noui.h \ + chain/pow.h \ + protocol/netprotocol.h \ + keys/pubkey.h \ + utils/random.h \ + utils/utilparse.h \ + rpc/rpcclient.h \ + rpc/rpcprotocol.h \ + rpc/rpcutils.h \ + rpc/rpcwallet.h \ + rpc/rpcserver.h \ + script/interpreter.h \ + script/interpreter.h \ + script/script.h \ + script/sigcache.h \ + script/sign.h \ + script/standard.h \ + script/script_error.h \ + utils/serialize.h \ + utils/streams.h \ + utils/sync.h \ + utils/threadsafety.h \ + utils/timedata.h \ + utils/tinyformat.h \ + storage/txdb.h \ + chain/txmempool.h \ + ui/ui_interface.h \ + structs/uint256.h \ + chain/undo.h \ + utils/util.h \ + utils/utilstrencodings.h \ + utils/utilmoneystr.h \ + utils/utiltime.h \ + version/bcversion.h \ + wallet/wallet.h \ + wallet/wallettxs.h \ + wallet/wallet_ismine.h \ + wallet/walletdb.h \ + compat/sanity.h + +JSON_H = \ + json/json_spirit.h \ + json/json_spirit_error_position.h \ + json/json_spirit_reader.h \ + json/json_spirit_reader_template.h \ + json/json_spirit_stream_reader.h \ + json/json_spirit_utils.h \ + json/json_spirit_value.h \ + json/json_spirit_writer.h \ + json/json_spirit_writer_template.h + +obj/build.h: FORCE + @$(MKDIR_P) $(builddir)/obj + @$(top_srcdir)/share/genbuild.sh $(abs_top_builddir)/src/obj/build.h \ + $(abs_top_srcdir) +libbitcoin_util_a-clientversion.$(OBJEXT): obj/build.h + +# server: shared between bitcoind and bitcoin-qt +libbitcoin_server_a_CPPFLAGS = $(BITCOIN_INCLUDES) $(MINIUPNPC_CPPFLAGS) +libbitcoin_server_a_SOURCES = \ + storage/addrman.cpp \ + structs/alert.cpp \ + structs/bloom.cpp \ + chain/chain.cpp \ + chain/checkpoints.cpp \ + core/init.cpp \ + storage/leveldbwrapper.cpp \ + core/main.cpp \ + protocol/multichaintx.cpp \ + protocol/multichainblock.cpp \ + protocol/handshake.cpp \ + chain/merkleblock.cpp \ + miner/miner.cpp \ + net/net.cpp \ + ui/noui.cpp \ + chain/pow.cpp \ + net/rest.cpp \ + utils/utilparse.cpp \ + rpc/rpcutils.cpp \ + rpc/rpchelp.cpp \ + rpc/rpcblockchain.cpp \ + rpc/rpcmining.cpp \ + rpc/rpcmisc.cpp \ + rpc/rpcnet.cpp \ + rpc/rpcrawtransaction.cpp \ + rpc/rpcserver.cpp \ + script/sigcache.cpp \ + utils/timedata.cpp \ + storage/txdb.cpp \ + chain/txmempool.cpp \ + $(JSON_H) \ + $(BITCOIN_CORE_H) + +# wallet: shared between bitcoind and bitcoin-qt, but only linked +# when wallet enabled +libbitcoin_wallet_a_CPPFLAGS = $(BITCOIN_INCLUDES) +libbitcoin_wallet_a_SOURCES = \ + wallet/db.cpp \ + wallet/crypter.cpp \ + rpc/rpcdump.cpp \ + rpc/rpcwallet.cpp \ + rpc/rpcwalletutils.cpp \ + rpc/rpcwallettxs.cpp \ + rpc/rpcexchange.cpp \ + rpc/rpcwalletsend.cpp \ + rpc/rpcpermissions.cpp \ + rpc/rpcassets.cpp \ + rpc/rpcstreams.cpp \ + wallet/wallet.cpp \ + wallet/walletcoins.cpp \ + wallet/wallettxs.cpp \ + wallet/wallet_ismine.cpp \ + wallet/walletdb.cpp \ + $(BITCOIN_CORE_H) + +# crypto primitives library +crypto_libbitcoin_crypto_a_CPPFLAGS = $(BITCOIN_CONFIG_INCLUDES) +crypto_libbitcoin_crypto_a_SOURCES = \ + crypto/sha1.cpp \ + crypto/sha256.cpp \ + crypto/sha512.cpp \ + crypto/hmac_sha256.cpp \ + crypto/hmac_sha512.cpp \ + crypto/ripemd160.cpp \ + crypto/common.h \ + crypto/sha256.h \ + crypto/sha512.h \ + crypto/hmac_sha256.h \ + crypto/hmac_sha512.h \ + crypto/sha1.h \ + crypto/ripemd160.h + +# multichain library +multichain_libbitcoin_multichain_a_CPPFLAGS = $(BITCOIN_CONFIG_INCLUDES) +multichain_libbitcoin_multichain_a_SOURCES = \ + utils/utility.cpp \ + utils/systemdependent.cpp \ + utils/tools.cpp \ + utils/utilwrapper.cpp \ + version/version.cpp \ + chainparams/params.cpp \ + protocol/multichainscript.cpp \ + utils/dbwrapper.cpp \ + wallet/wallettxdb.cpp \ + permissions/permission.cpp \ + entities/asset.cpp + +# univalue JSON library +univalue_libbitcoin_univalue_a_SOURCES = \ + univalue/univalue.cpp \ + univalue/univalue_read.cpp \ + univalue/univalue_write.cpp \ + univalue/univalue_escapes.h \ + univalue/univalue.h + +# common: shared between bitcoind, and bitcoin-qt and non-server tools +libbitcoin_common_a_CPPFLAGS = $(BITCOIN_INCLUDES) +libbitcoin_common_a_SOURCES = \ + utils/allocators.cpp \ + structs/amount.cpp \ + structs/base58.cpp \ + chainparams/chainparams.cpp \ + storage/coins.cpp \ + utils/compressor.cpp \ + primitives/block.cpp \ + primitives/transaction.cpp \ + utils/core_read.cpp \ + utils/core_write.cpp \ + structs/hash.cpp \ + keys/key.cpp \ + wallet/keystore.cpp \ + net/netbase.cpp \ + protocol/netprotocol.cpp \ + keys/pubkey.cpp \ + script/interpreter.cpp \ + script/script.cpp \ + script/sign.cpp \ + script/standard.cpp \ + script/script_error.cpp \ + $(BITCOIN_CORE_H) + +# util: shared between all executables. +# This library *must* be included to make sure that the glibc +# backward-compatibility objects and their sanity checks are linked. +libbitcoin_util_a_CPPFLAGS = $(BITCOIN_INCLUDES) +libbitcoin_util_a_SOURCES = \ + compat/strnlen.cpp \ + compat/glibc_sanity.cpp \ + compat/glibcxx_sanity.cpp \ + chainparams/chainparamsbase.cpp \ + version/clientversion.cpp \ + utils/random.cpp \ + rpc/rpcprotocol.cpp \ + utils/sync.cpp \ + structs/uint256.cpp \ + utils/util.cpp \ + utils/utilstrencodings.cpp \ + utils/utilmoneystr.cpp \ + utils/utiltime.cpp \ + $(BITCOIN_CORE_H) + +if GLIBC_BACK_COMPAT +libbitcoin_util_a_SOURCES += compat/glibc_compat.cpp +libbitcoin_util_a_SOURCES += compat/glibcxx_compat.cpp +endif + +# cli: shared between bitcoin-cli and bitcoin-qt +libbitcoin_cli_a_CPPFLAGS = $(BITCOIN_INCLUDES) +libbitcoin_cli_a_SOURCES = \ + rpc/rpcclient.cpp \ + $(BITCOIN_CORE_H) + +nodist_libbitcoin_util_a_SOURCES = $(srcdir)/obj/build.h +# + +# bitcoind binary # +#bitcoind_LDADD = \ + $(LIBBITCOIN_SERVER) \ + $(LIBBITCOIN_COMMON) \ + $(LIBBITCOIN_UNIVALUE) \ + $(LIBBITCOIN_WALLET) \ + $(LIBBITCOIN_MULTICHAIN) \ + $(LIBBITCOIN_UTIL) \ + $(LIBBITCOIN_CRYPTO) \ + $(LIBLEVELDB) \ + $(LIBMEMENV) \ + $(LIBSECP256K1) + +#if ENABLE_WALLET +#bitcoind_LDADD += libbitcoin_wallet.a +#endif +#bitcoind_LDADD += multichain/libbitcoin_multichain.a +#bitcoind_SOURCES = bitcoind.cpp chainparams/buildgenesis.cpp +# + +if TARGET_WINDOWS +#bitcoind_SOURCES += bitcoind-res.rc +endif + +#bitcoind_LDADD += $(BOOST_LIBS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(MINIUPNPC_LIBS) +#bitcoind_CPPFLAGS = $(BITCOIN_INCLUDES) +#bitcoind_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) + +# bitcoin-cli binary # +#bitcoin_cli_LDADD = \ + $(LIBBITCOIN_CLI) \ + $(LIBBITCOIN_UTIL) \ + $(LIBBITCOIN_MULTICHAIN) \ + $(LIBBITCOIN_CRYPTO) \ + $(LIBLEVELDB) \ + $(BOOST_LIBS) \ + $(SSL_LIBS) \ + $(CRYPTO_LIBS) + +#bitcoin_cli_SOURCES = \ + bitcoin-cli.cpp + +#bitcoin_cli_CPPFLAGS = $(BITCOIN_INCLUDES) +# + +# bitcoin-tx binary # +#bitcoin_tx_LDADD = \ + $(LIBBITCOIN_COMMON) \ + $(LIBBITCOIN_UNIVALUE) \ + $(LIBBITCOIN_MULTICHAIN) \ + $(LIBBITCOIN_UTIL) \ + $(LIBBITCOIN_CRYPTO) \ + $(LIBLEVELDB) \ + $(LIBSECP256K1) \ + $(BOOST_LIBS) \ + $(CRYPTO_LIBS) + +#bitcoin_tx_SOURCES = bitcoin-tx.cpp +#bitcoin_tx_CPPFLAGS = $(BITCOIN_INCLUDES) +# +#bitcoin_tx_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) + +if TARGET_WINDOWS +#bitcoin_cli_SOURCES += bitcoin-cli-res.rc +endif +#bitcoin_cli_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) + +# MCHN START + +# multichaind binary # +multichaind_LDADD = \ + $(LIBBITCOIN_SERVER) \ + $(LIBBITCOIN_COMMON) \ + $(LIBBITCOIN_UNIVALUE) \ + $(LIBBITCOIN_WALLET) \ + $(LIBBITCOIN_MULTICHAIN) \ + $(LIBBITCOIN_UTIL) \ + $(LIBBITCOIN_CRYPTO) \ + $(LIBLEVELDB) \ + $(LIBMEMENV) \ + $(LIBSECP256K1) + +#if ENABLE_WALLET +#multichaind_LDADD += libbitcoin_wallet.a +#endif +multichaind_SOURCES = multichain/multichaind.cpp chainparams/buildgenesis.cpp +# + +if TARGET_WINDOWS +multichaind_SOURCES += multichaind-res.rc +endif + +multichaind_LDADD += $(BOOST_LIBS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(MINIUPNPC_LIBS) +multichaind_CPPFLAGS = $(BITCOIN_INCLUDES) +multichaind_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) + +# multichain-cli binary # +multichain_cli_LDADD = \ + $(LIBBITCOIN_CLI) \ + $(LIBBITCOIN_UTIL) \ + $(LIBBITCOIN_MULTICHAIN) \ + $(LIBBITCOIN_CRYPTO) \ + $(LIBLEVELDB) \ + $(BOOST_LIBS) \ + $(SSL_LIBS) \ + $(CRYPTO_LIBS) + +multichain_cli_SOURCES = multichain/multichain-cli.cpp + +multichain_cli_CPPFLAGS = $(BITCOIN_INCLUDES) +# + +if TARGET_WINDOWS +multichain_cli_SOURCES += multichain-cli-res.rc +endif +multichain_cli_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) + +# multichain-util binary # +multichain_util_LDADD = \ + $(LIBBITCOIN_UNIVALUE) \ + $(LIBBITCOIN_MULTICHAIN) \ + $(LIBBITCOIN_COMMON) \ + $(LIBBITCOIN_UTIL) \ + $(LIBBITCOIN_CRYPTO) \ + $(LIBLEVELDB) \ + $(LIBSECP256K1) \ + $(BOOST_LIBS) \ + $(CRYPTO_LIBS) + +multichain_util_SOURCES = multichain/multichain-util.cpp chainparams/buildgenesis.cpp + +multichain_util_CPPFLAGS = $(BITCOIN_INCLUDES) +# +if TARGET_WINDOWS +multichain_util_SOURCES += multichain-util-res.rc +endif +multichain_util_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) + + +# MCHN END + + + + +if BUILD_BITCOIN_LIBS +include_HEADERS = script/bitcoinconsensus.h +libbitcoinconsensus_la_SOURCES = \ + primitives/transaction.cpp \ + crypto/hmac_sha512.cpp \ + crypto/sha1.cpp \ + crypto/sha256.cpp \ + crypto/sha512.cpp \ + crypto/ripemd160.cpp \ + utils/utility.cpp \ + utils/systemdependent.cpp \ + utils/tools.cpp \ + utils/utilwrapper.cpp \ + chainparams/buildgenesis.cpp \ + version/version.cpp \ + chainparams/chainparams.cpp \ + protocol/multichainscript.cpp \ + utils/dbwrapper.cpp \ + wallet/wallettxdb.cpp \ + permissions/permission.cpp \ + entities/asset.cpp \ + structs/hash.cpp \ + keys/pubkey.cpp \ + script/script.cpp \ + script/interpreter.cpp \ + script/bitcoinconsensus.cpp \ + structs/uint256.cpp \ + utils/utilstrencodings.cpp + +if GLIBC_BACK_COMPAT + libbitcoinconsensus_la_SOURCES += compat/glibc_compat.cpp + libbitcoinconsensus_la_SOURCES += compat/glibcxx_compat.cpp +endif + +libbitcoinconsensus_la_LDFLAGS = -no-undefined $(RELDFLAGS) +libbitcoinconsensus_la_LIBADD = $(LIBSECP256K1) +libbitcoinconsensus_la_CPPFLAGS = -I$(builddir)/obj -I$(srcdir)/secp256k1/include -DBUILD_BITCOIN_INTERNAL +endif + +CLEANFILES = leveldb/libleveldb.a leveldb/libmemenv.a *.gcda *.gcno + +DISTCLEANFILES = obj/build.h + +EXTRA_DIST = leveldb + +clean-local: + -$(MAKE) -C leveldb clean + -$(MAKE) -C secp256k1 clean + rm -f leveldb/*/*.gcno leveldb/helpers/memenv/*.gcno + -rm -f config.h + +.rc.o: + @test -f $(WINDRES) + $(AM_V_GEN) $(WINDRES) -DWINDRES_PREPROC -i $< -o $@ + +.mm.o: + $(AM_V_CXX) $(OBJCXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \ + $(CPPFLAGS) $(AM_CXXFLAGS) $(QT_INCLUDES) $(CXXFLAGS) -c -o $@ $< + +%.pb.cc %.pb.h: %.proto + @test -f $(PROTOC) + $(AM_V_GEN) $(PROTOC) --cpp_out=$(@D) --proto_path=$(abspath $( forms/ui_foo.h +QT_FORMS_H=$(join $(dir $(QT_FORMS_UI)),$(addprefix ui_, $(notdir $(QT_FORMS_UI:.ui=.h)))) + +# Most files will depend on the forms and moc files as includes. Generate them +# before anything else. +$(QT_MOC): $(QT_FORMS_H) +$(qt_libbitcoinqt_a_OBJECTS) $(qt_bitcoin_qt_OBJECTS) : | $(QT_MOC) + +#Generating these with a half-written protobuf header leads to wacky results. +#This makes sure it's done. +$(QT_MOC): $(PROTOBUF_H) +$(QT_MOC_CPP): $(PROTOBUF_H) + +# bitcoin-qt binary # +qt_bitcoin_qt_CPPFLAGS = $(BITCOIN_INCLUDES) $(BITCOIN_QT_INCLUDES) \ + $(QT_INCLUDES) $(PROTOBUF_CFLAGS) $(QR_CFLAGS) + +qt_bitcoin_qt_SOURCES = qt/bitcoin.cpp +if TARGET_DARWIN + qt_bitcoin_qt_SOURCES += $(BITCOIN_MM) +endif +if TARGET_WINDOWS + qt_bitcoin_qt_SOURCES += $(BITCOIN_RC) +endif +qt_bitcoin_qt_LDADD = qt/libbitcoinqt.a $(LIBBITCOIN_SERVER) +if ENABLE_WALLET +qt_bitcoin_qt_LDADD += $(LIBBITCOIN_WALLET) +endif +qt_bitcoin_qt_LDADD += $(LIBBITCOIN_CLI) $(LIBBITCOIN_COMMON) $(LIBBITCOIN_UTIL) $(LIBBITCOIN_CRYPTO) $(LIBBITCOIN_UNIVALUE) $(LIBLEVELDB) $(LIBMEMENV) \ + $(BOOST_LIBS) $(QT_LIBS) $(QT_DBUS_LIBS) $(QR_LIBS) $(PROTOBUF_LIBS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(MINIUPNPC_LIBS) $(LIBSECP256K1) +qt_bitcoin_qt_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(QT_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +qt_bitcoin_qt_LIBTOOLFLAGS = --tag CXX + +#locale/foo.ts -> locale/foo.qm +QT_QM=$(QT_TS:.ts=.qm) + +.SECONDARY: $(QT_QM) + +qt/bitcoinstrings.cpp: $(libbitcoin_server_a_SOURCES) $(libbitcoin_wallet_a_SOURCES) + @test -n $(XGETTEXT) || echo "xgettext is required for updating translations" + $(AM_V_GEN) cd $(srcdir); XGETTEXT=$(XGETTEXT) ../share/qt/extract_strings_qt.py $^ + +translate: qt/bitcoinstrings.cpp $(QT_FORMS_UI) $(QT_FORMS_UI) $(BITCOIN_QT_CPP) $(BITCOIN_QT_H) $(BITCOIN_MM) + @test -n $(LUPDATE) || echo "lupdate is required for updating translations" + $(AM_V_GEN) QT_SELECT=$(QT_SELECT) $(LUPDATE) $^ -locations relative -no-obsolete -ts qt/locale/bitcoin_en.ts + +$(QT_QRC_LOCALE_CPP): $(QT_QRC_LOCALE) $(QT_QM) + @test -f $(RCC) + @test -f $(@D)/$( $@ + +$(QT_QRC_CPP): $(QT_QRC) $(QT_FORMS_H) $(RES_ICONS) $(RES_IMAGES) $(RES_MOVIES) $(PROTOBUF_H) + @test -f $(RCC) + $(AM_V_GEN) QT_SELECT=$(QT_SELECT) $(RCC) -name bitcoin $< | \ + $(SED) -e '/^\*\*.*Created:/d' -e '/^\*\*.*by:/d' > $@ + +CLEAN_QT = $(nodist_qt_libbitcoinqt_a_SOURCES) $(QT_QM) $(QT_FORMS_H) qt/*.gcda qt/*.gcno + +CLEANFILES += $(CLEAN_QT) + +bitcoin_qt_clean: FORCE + rm -f $(CLEAN_QT) $(qt_libbitcoinqt_a_OBJECTS) $(qt_bitcoin_qt_OBJECTS) qt/bitcoin-qt$(EXEEXT) $(LIBBITCOINQT) + +bitcoin_qt : qt/bitcoin-qt$(EXEEXT) + +ui_%.h: %.ui + @test -f $(UIC) + @$(MKDIR_P) $(@D) + $(AM_V_GEN) QT_SELECT=$(QT_SELECT) $(UIC) -o $@ $< || (echo "Error creating $@"; false) + +%.moc: %.cpp + $(AM_V_GEN) QT_SELECT=$(QT_SELECT) $(MOC) $(QT_INCLUDES) $(MOC_DEFS) $< | \ + $(SED) -e '/^\*\*.*Created:/d' -e '/^\*\*.*by:/d' > $@ + +moc_%.cpp: %.h + $(AM_V_GEN) QT_SELECT=$(QT_SELECT) $(MOC) $(QT_INCLUDES) $(MOC_DEFS) $< | \ + $(SED) -e '/^\*\*.*Created:/d' -e '/^\*\*.*by:/d' > $@ + +%.qm: %.ts + @test -f $(LRELEASE) + @$(MKDIR_P) $(@D) + $(AM_V_GEN) QT_SELECT=$(QT_SELECT) $(LRELEASE) -silent $< -qm $@ diff --git a/src/Makefile.qttest.include b/src/Makefile.qttest.include new file mode 100644 index 00000000..c5392cf3 --- /dev/null +++ b/src/Makefile.qttest.include @@ -0,0 +1,48 @@ +bin_PROGRAMS += qt/test/test_bitcoin-qt +TESTS += qt/test/test_bitcoin-qt + +TEST_QT_MOC_CPP = qt/test/moc_uritests.cpp + +if ENABLE_WALLET +TEST_QT_MOC_CPP += qt/test/moc_paymentservertests.cpp +endif + +TEST_QT_H = \ + qt/test/uritests.h \ + qt/test/paymentrequestdata.h \ + qt/test/paymentservertests.h + +qt_test_test_bitcoin_qt_CPPFLAGS = $(BITCOIN_INCLUDES) $(BITCOIN_QT_INCLUDES) \ + $(QT_INCLUDES) $(QT_TEST_INCLUDES) $(PROTOBUF_CFLAGS) + +qt_test_test_bitcoin_qt_SOURCES = \ + qt/test/test_main.cpp \ + qt/test/uritests.cpp \ + $(TEST_QT_H) +if ENABLE_WALLET +qt_test_test_bitcoin_qt_SOURCES += \ + qt/test/paymentservertests.cpp +endif + +nodist_qt_test_test_bitcoin_qt_SOURCES = $(TEST_QT_MOC_CPP) + +qt_test_test_bitcoin_qt_LDADD = $(LIBBITCOINQT) $(LIBBITCOIN_SERVER) +if ENABLE_WALLET +qt_test_test_bitcoin_qt_LDADD += $(LIBBITCOIN_WALLET) +endif +qt_test_test_bitcoin_qt_LDADD += $(LIBBITCOIN_CLI) $(LIBBITCOIN_COMMON) $(LIBBITCOIN_UTIL) $(LIBBITCOIN_CRYPTO) $(LIBBITCOIN_UNIVALUE) $(LIBLEVELDB) \ + $(LIBMEMENV) $(BOOST_LIBS) $(QT_DBUS_LIBS) $(QT_TEST_LIBS) $(QT_LIBS) \ + $(QR_LIBS) $(PROTOBUF_LIBS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(MINIUPNPC_LIBS) $(LIBSECP256K1) +qt_test_test_bitcoin_qt_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(QT_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) + +CLEAN_BITCOIN_QT_TEST = $(TEST_QT_MOC_CPP) qt/test/*.gcda qt/test/*.gcno + +CLEANFILES += $(CLEAN_BITCOIN_QT_TEST) + +test_bitcoin_qt : qt/test/test_bitcoin-qt$(EXEEXT) + +test_bitcoin_qt_check : qt/test/test_bitcoin-qt$(EXEEXT) FORCE + $(MAKE) check-TESTS TESTS=$^ + +test_bitcoin_qt_clean: FORCE + rm -f $(CLEAN_BITCOIN_QT_TEST) $(qt_test_test_bitcoin_qt_OBJECTS) diff --git a/src/Makefile.test.include b/src/Makefile.test.include new file mode 100644 index 00000000..6a921ca4 --- /dev/null +++ b/src/Makefile.test.include @@ -0,0 +1,126 @@ +TESTS += test/test_bitcoin test/bitcoin-util-test.py +bin_PROGRAMS += test/test_bitcoin +TEST_SRCDIR = test +TEST_BINARY=test/test_bitcoin$(EXEEXT) + + +EXTRA_DIST += \ + test/bctest.py \ + test/bitcoin-util-test.py \ + test/data/bitcoin-util-test.json \ + test/data/blanktx.hex \ + test/data/tt-delin1-out.hex \ + test/data/tt-delout1-out.hex \ + test/data/tt-locktime317000-out.hex \ + test/data/tx394b54bb.hex \ + test/data/txcreate1.hex \ + test/data/txcreate2.hex \ + test/data/txcreatesign.hex + +JSON_TEST_FILES = \ + test/data/script_valid.json \ + test/data/base58_keys_valid.json \ + test/data/sig_canonical.json \ + test/data/sig_noncanonical.json \ + test/data/base58_encode_decode.json \ + test/data/base58_keys_invalid.json \ + test/data/script_invalid.json \ + test/data/tx_invalid.json \ + test/data/tx_valid.json \ + test/data/sighash.json + +RAW_TEST_FILES = test/data/alertTests.raw + +GENERATED_TEST_FILES = $(JSON_TEST_FILES:.json=.json.h) $(RAW_TEST_FILES:.raw=.raw.h) + +BITCOIN_TESTS =\ + test/bignum.h \ + test/alert_tests.cpp \ + test/allocator_tests.cpp \ + test/base32_tests.cpp \ + test/base58_tests.cpp \ + test/base64_tests.cpp \ + test/bloom_tests.cpp \ + test/checkblock_tests.cpp \ + test/Checkpoints_tests.cpp \ + test/coins_tests.cpp \ + test/compress_tests.cpp \ + test/crypto_tests.cpp \ + test/DoS_tests.cpp \ + test/getarg_tests.cpp \ + test/hash_tests.cpp \ + test/key_tests.cpp \ + test/main_tests.cpp \ + test/miner_tests.cpp \ + test/mruset_tests.cpp \ + test/multisig_tests.cpp \ + test/netbase_tests.cpp \ + test/pmt_tests.cpp \ + test/rpc_tests.cpp \ + test/sanity_tests.cpp \ + test/script_P2SH_tests.cpp \ + test/script_tests.cpp \ + test/scriptnum_tests.cpp \ + test/serialize_tests.cpp \ + test/sighash_tests.cpp \ + test/sigopcount_tests.cpp \ + test/skiplist_tests.cpp \ + test/test_bitcoin.cpp \ + test/timedata_tests.cpp \ + test/transaction_tests.cpp \ + test/uint256_tests.cpp \ + test/univalue_tests.cpp \ + test/util_tests.cpp + +if ENABLE_WALLET +BITCOIN_TESTS += \ + test/accounting_tests.cpp \ + test/wallet_tests.cpp \ + test/rpc_wallet_tests.cpp +endif + +test_test_bitcoin_SOURCES = $(BITCOIN_TESTS) $(JSON_TEST_FILES) $(RAW_TEST_FILES) +test_test_bitcoin_CPPFLAGS = $(BITCOIN_INCLUDES) -I$(builddir)/test/ $(TESTDEFS) +test_test_bitcoin_LDADD = $(LIBBITCOIN_SERVER) $(LIBBITCOIN_CLI) $(LIBBITCOIN_COMMON) $(LIBBITCOIN_UTIL) $(LIBBITCOIN_CRYPTO) $(LIBBITCOIN_UNIVALUE) $(LIBBITCOIN_WALLET) $(LIBBITCOIN_MULTICHAIN) $(LIBLEVELDB) $(LIBMEMENV) \ + $(BOOST_LIBS) $(BOOST_UNIT_TEST_FRAMEWORK_LIB) $(LIBSECP256K1) +#if ENABLE_WALLET +#test_test_bitcoin_LDADD += $(LIBBITCOIN_WALLET) +#endif + +test_test_bitcoin_LDADD += $(LIBBITCOIN_CONSENSUS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(MINIUPNPC_LIBS) +test_test_bitcoin_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) -static + +nodist_test_test_bitcoin_SOURCES = $(GENERATED_TEST_FILES) + +$(BITCOIN_TESTS): $(GENERATED_TEST_FILES) + +CLEAN_BITCOIN_TEST = test/*.gcda test/*.gcno $(GENERATED_TEST_FILES) + +CLEANFILES += $(CLEAN_BITCOIN_TEST) + +bitcoin_test: $(TEST_BINARY) + +bitcoin_test_check: $(TEST_BINARY) FORCE + $(MAKE) check-TESTS TESTS=$^ + +bitcoin_test_clean : FORCE + rm -f $(CLEAN_BITCOIN_TEST) $(test_test_bitcoin_OBJECTS) $(TEST_BINARY) + +check-local: + $(AM_V_at)$(MAKE) $(AM_MAKEFLAGS) -C secp256k1 check + +%.json.h: %.json + @$(MKDIR_P) $(@D) + @echo "namespace json_tests{" > $@ + @echo "static unsigned const char $(*F)[] = {" >> $@ + @$(HEXDUMP) -v -e '8/1 "0x%02x, "' -e '"\n"' $< | $(SED) -e 's/0x ,//g' >> $@ + @echo "};};" >> $@ + @echo "Generated $@" + +%.raw.h: %.raw + @$(MKDIR_P) $(@D) + @echo "namespace alert_tests{" > $@ + @echo "static unsigned const char $(*F)[] = {" >> $@ + @$(HEXDUMP) -v -e '8/1 "0x%02x, "' -e '"\n"' $< | $(SED) -e 's/0x ,//g' >> $@ + @echo "};};" >> $@ + @echo "Generated $@" diff --git a/src/bitcoin-cli-res.rc b/src/bitcoin-cli-res.rc new file mode 100644 index 00000000..b1aa1b0e --- /dev/null +++ b/src/bitcoin-cli-res.rc @@ -0,0 +1,35 @@ +#include // needed for VERSIONINFO +#include "clientversion.h" // holds the needed client version information + +#define VER_PRODUCTVERSION CLIENT_VERSION_MAJOR,CLIENT_VERSION_MINOR,CLIENT_VERSION_REVISION,CLIENT_VERSION_BUILD +#define VER_PRODUCTVERSION_STR STRINGIZE(CLIENT_VERSION_MAJOR) "." STRINGIZE(CLIENT_VERSION_MINOR) "." STRINGIZE(CLIENT_VERSION_REVISION) "." STRINGIZE(CLIENT_VERSION_BUILD) +#define VER_FILEVERSION VER_PRODUCTVERSION +#define VER_FILEVERSION_STR VER_PRODUCTVERSION_STR + +VS_VERSION_INFO VERSIONINFO +FILEVERSION VER_FILEVERSION +PRODUCTVERSION VER_PRODUCTVERSION +FILEOS VOS_NT_WINDOWS32 +FILETYPE VFT_APP +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "040904E4" // U.S. English - multilingual (hex) + BEGIN + VALUE "CompanyName", "Bitcoin" + VALUE "FileDescription", "Bitcoin-cli (OSS RPC client for Bitcoin)" + VALUE "FileVersion", VER_FILEVERSION_STR + VALUE "InternalName", "bitcoin-cli" + VALUE "LegalCopyright", COPYRIGHT_STR + VALUE "LegalTrademarks1", "Distributed under the MIT/X11 software license, see the accompanying file COPYING or http://www.opensource.org/licenses/mit-license.php." + VALUE "OriginalFilename", "bitcoin-cli.exe" + VALUE "ProductName", "Bitcoin-cli" + VALUE "ProductVersion", VER_PRODUCTVERSION_STR + END + END + + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x0, 1252 // language neutral - multilingual (decimal) + END +END diff --git a/src/bitcoind-res.rc b/src/bitcoind-res.rc new file mode 100644 index 00000000..2e6d7544 --- /dev/null +++ b/src/bitcoind-res.rc @@ -0,0 +1,35 @@ +#include // needed for VERSIONINFO +#include "clientversion.h" // holds the needed client version information + +#define VER_PRODUCTVERSION CLIENT_VERSION_MAJOR,CLIENT_VERSION_MINOR,CLIENT_VERSION_REVISION,CLIENT_VERSION_BUILD +#define VER_PRODUCTVERSION_STR STRINGIZE(CLIENT_VERSION_MAJOR) "." STRINGIZE(CLIENT_VERSION_MINOR) "." STRINGIZE(CLIENT_VERSION_REVISION) "." STRINGIZE(CLIENT_VERSION_BUILD) +#define VER_FILEVERSION VER_PRODUCTVERSION +#define VER_FILEVERSION_STR VER_PRODUCTVERSION_STR + +VS_VERSION_INFO VERSIONINFO +FILEVERSION VER_FILEVERSION +PRODUCTVERSION VER_PRODUCTVERSION +FILEOS VOS_NT_WINDOWS32 +FILETYPE VFT_APP +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "040904E4" // U.S. English - multilingual (hex) + BEGIN + VALUE "CompanyName", "Bitcoin" + VALUE "FileDescription", "Bitcoind (OSS daemon/client for Bitcoin)" + VALUE "FileVersion", VER_FILEVERSION_STR + VALUE "InternalName", "bitcoind" + VALUE "LegalCopyright", COPYRIGHT_STR + VALUE "LegalTrademarks1", "Distributed under the MIT/X11 software license, see the accompanying file COPYING or http://www.opensource.org/licenses/mit-license.php." + VALUE "OriginalFilename", "bitcoind.exe" + VALUE "ProductName", "Bitcoind" + VALUE "ProductVersion", VER_PRODUCTVERSION_STR + END + END + + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x0, 1252 // language neutral - multilingual (decimal) + END +END diff --git a/src/chain/chain.cpp b/src/chain/chain.cpp new file mode 100644 index 00000000..2277bfd2 --- /dev/null +++ b/src/chain/chain.cpp @@ -0,0 +1,60 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "chain/chain.h" + +using namespace std; + +/** + * CChain implementation + */ +void CChain::SetTip(CBlockIndex *pindex) { + if (pindex == NULL) { + vChain.clear(); + return; + } + vChain.resize(pindex->nHeight + 1); + while (pindex && vChain[pindex->nHeight] != pindex) { + vChain[pindex->nHeight] = pindex; + pindex = pindex->pprev; + } +} + +CBlockLocator CChain::GetLocator(const CBlockIndex *pindex) const { + int nStep = 1; + std::vector vHave; + vHave.reserve(32); + + if (!pindex) + pindex = Tip(); + while (pindex) { + vHave.push_back(pindex->GetBlockHash()); + // Stop when we have added the genesis block. + if (pindex->nHeight == 0) + break; + // Exponentially larger steps back, plus the genesis block. + int nHeight = std::max(pindex->nHeight - nStep, 0); + if (Contains(pindex)) { + // Use O(1) CChain index if possible. + pindex = (*this)[nHeight]; + } else { + // Otherwise, use O(log n) skiplist. + pindex = pindex->GetAncestor(nHeight); + } + if (vHave.size() > 10) + nStep *= 2; + } + + return CBlockLocator(vHave); +} + +const CBlockIndex *CChain::FindFork(const CBlockIndex *pindex) const { + if (pindex->nHeight > Height()) + pindex = pindex->GetAncestor(Height()); + while (pindex && !Contains(pindex)) + pindex = pindex->pprev; + return pindex; +} diff --git a/src/chain/chain.h b/src/chain/chain.h new file mode 100644 index 00000000..a373ea21 --- /dev/null +++ b/src/chain/chain.h @@ -0,0 +1,424 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_CHAIN_H +#define BITCOIN_CHAIN_H + +#include "primitives/block.h" +#include "chain/pow.h" +#include "utils/tinyformat.h" +#include "structs/uint256.h" + +#include + +#include + +struct CDiskBlockPos +{ + int nFile; + unsigned int nPos; + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(VARINT(nFile)); + READWRITE(VARINT(nPos)); + } + + CDiskBlockPos() { + SetNull(); + } + + CDiskBlockPos(int nFileIn, unsigned int nPosIn) { + nFile = nFileIn; + nPos = nPosIn; + } + + friend bool operator==(const CDiskBlockPos &a, const CDiskBlockPos &b) { + return (a.nFile == b.nFile && a.nPos == b.nPos); + } + + friend bool operator!=(const CDiskBlockPos &a, const CDiskBlockPos &b) { + return !(a == b); + } + + void SetNull() { nFile = -1; nPos = 0; } + bool IsNull() const { return (nFile == -1); } +}; + +enum BlockStatus { + //! Unused. + BLOCK_VALID_UNKNOWN = 0, + + //! Parsed, version ok, hash satisfies claimed PoW, 1 <= vtx count <= max, timestamp not in future + BLOCK_VALID_HEADER = 1, + + //! All parent headers found, difficulty matches, timestamp >= median previous, checkpoint. Implies all parents + //! are also at least TREE. + BLOCK_VALID_TREE = 2, + + /** + * Only first tx is coinbase, 2 <= coinbase input script length <= 100, transactions valid, no duplicate txids, + * sigops, size, merkle root. Implies all parents are at least TREE but not necessarily TRANSACTIONS. When all + * parent blocks also have TRANSACTIONS, CBlockIndex::nChainTx will be set. + */ + BLOCK_VALID_TRANSACTIONS = 3, + + //! Outputs do not overspend inputs, no double spends, coinbase output ok, immature coinbase spends, BIP30. + //! Implies all parents are also at least CHAIN. + BLOCK_VALID_CHAIN = 4, + + //! Scripts & signatures ok. Implies all parents are also at least SCRIPTS. + BLOCK_VALID_SCRIPTS = 5, + + //! All validity bits. + BLOCK_VALID_MASK = BLOCK_VALID_HEADER | BLOCK_VALID_TREE | BLOCK_VALID_TRANSACTIONS | + BLOCK_VALID_CHAIN | BLOCK_VALID_SCRIPTS, + + BLOCK_HAVE_DATA = 8, //! full block available in blk*.dat + BLOCK_HAVE_UNDO = 16, //! undo data available in rev*.dat + BLOCK_HAVE_MASK = BLOCK_HAVE_DATA | BLOCK_HAVE_UNDO, + + BLOCK_FAILED_VALID = 32, //! stage after last reached validness failed + BLOCK_FAILED_CHILD = 64, //! descends from failed block + BLOCK_FAILED_MASK = BLOCK_FAILED_VALID | BLOCK_FAILED_CHILD, +}; + +/** The block chain is a tree shaped structure starting with the + * genesis block at the root, with each block potentially having multiple + * candidates to be the next block. A blockindex may have multiple pprev pointing + * to it, but at most one of them can be part of the currently active branch. + */ +class CBlockIndex +{ +public: + //! pointer to the hash of the block, if any. memory is owned by this CBlockIndex + const uint256* phashBlock; + + //! pointer to the index of the predecessor of this block + CBlockIndex* pprev; + + //! pointer to the index of some further predecessor of this block + CBlockIndex* pskip; + + //! height of the entry in the chain. The genesis block has height 0 + int nHeight; + + //! Which # file this block is stored in (blk?????.dat) + int nFile; + + //! Byte offset within blk?????.dat where this block's data is stored + unsigned int nDataPos; + + //! Byte offset within rev?????.dat where this block's undo data is stored + unsigned int nUndoPos; + + //! (memory only) Total amount of work (expected number of hashes) in the chain up to and including this block + uint256 nChainWork; + + //! Number of transactions in this block. + //! Note: in a potential headers-first mode, this number cannot be relied upon + unsigned int nTx; + + //! (memory only) Number of transactions in the chain up to and including this block. + //! This value will be non-zero only if and only if transactions for this block and all its parents are available. + //! Change to 64-bit type when necessary; won't happen before 2030 + unsigned int nChainTx; + + //! Verification status of this block. See enum BlockStatus + unsigned int nStatus; + + //! block header + int nVersion; + uint256 hashMerkleRoot; + unsigned int nTime; + unsigned int nBits; + unsigned int nNonce; + + //! (memory only) Sequential id assigned to distinguish order in which blocks are received. + uint32_t nSequenceId; + +/* MCHN START */ + int nHeightMinedByMe; + uint32_t nCanMine; +/* MCHN END */ + + void SetNull() + { + phashBlock = NULL; + pprev = NULL; + pskip = NULL; + nHeight = 0; + nFile = 0; + nDataPos = 0; + nUndoPos = 0; + nChainWork = 0; + nTx = 0; + nChainTx = 0; + nStatus = 0; + nSequenceId = 0; + + nVersion = 0; + hashMerkleRoot = 0; + nTime = 0; + nBits = 0; + nNonce = 0; + +/* MCHN START */ + nHeightMinedByMe=0; + nCanMine=0; +/* MCHN END */ + } + + CBlockIndex() + { + SetNull(); + } + + CBlockIndex(const CBlockHeader& block) + { + SetNull(); + + nVersion = block.nVersion; + hashMerkleRoot = block.hashMerkleRoot; + nTime = block.nTime; + nBits = block.nBits; + nNonce = block.nNonce; + } + + CDiskBlockPos GetBlockPos() const { + CDiskBlockPos ret; + if (nStatus & BLOCK_HAVE_DATA) { + ret.nFile = nFile; + ret.nPos = nDataPos; + } + return ret; + } + + CDiskBlockPos GetUndoPos() const { + CDiskBlockPos ret; + if (nStatus & BLOCK_HAVE_UNDO) { + ret.nFile = nFile; + ret.nPos = nUndoPos; + } + return ret; + } + + CBlockHeader GetBlockHeader() const + { + CBlockHeader block; + block.nVersion = nVersion; + if (pprev) + block.hashPrevBlock = pprev->GetBlockHash(); + block.hashMerkleRoot = hashMerkleRoot; + block.nTime = nTime; + block.nBits = nBits; + block.nNonce = nNonce; + return block; + } + + uint256 GetBlockHash() const + { + return *phashBlock; + } + + int64_t GetBlockTime() const + { + return (int64_t)nTime; + } + + enum { nMedianTimeSpan=11 }; + + int64_t GetMedianTimePast() const + { + int64_t pmedian[nMedianTimeSpan]; + int64_t* pbegin = &pmedian[nMedianTimeSpan]; + int64_t* pend = &pmedian[nMedianTimeSpan]; + + const CBlockIndex* pindex = this; + for (int i = 0; i < nMedianTimeSpan && pindex; i++, pindex = pindex->pprev) + *(--pbegin) = pindex->GetBlockTime(); + + std::sort(pbegin, pend); + return pbegin[(pend - pbegin)/2]; + } + + /** + * Returns true if there are nRequired or more blocks of minVersion or above + * in the last Params().ToCheckBlockUpgradeMajority() blocks, starting at pstart + * and going backwards. + */ + static bool IsSuperMajority(int minVersion, const CBlockIndex* pstart, + unsigned int nRequired); + + std::string ToString() const + { + return strprintf("CBlockIndex(pprev=%p, nHeight=%d, merkle=%s, hashBlock=%s)", + pprev, nHeight, + hashMerkleRoot.ToString(), + GetBlockHash().ToString()); + } + + //! Check whether this block index entry is valid up to the passed validity level. + bool IsValid(enum BlockStatus nUpTo = BLOCK_VALID_TRANSACTIONS) const + { + assert(!(nUpTo & ~BLOCK_VALID_MASK)); // Only validity flags allowed. + if (nStatus & BLOCK_FAILED_MASK) + return false; +/* MCHN START */ +// return ((nStatus & BLOCK_VALID_MASK) >= nUpTo); + return ((nStatus & BLOCK_VALID_MASK) >= (unsigned int)nUpTo); +/* MCHN END */ + } + + //! Raise the validity level of this block index entry. + //! Returns true if the validity was changed. + bool RaiseValidity(enum BlockStatus nUpTo) + { + assert(!(nUpTo & ~BLOCK_VALID_MASK)); // Only validity flags allowed. + if (nStatus & BLOCK_FAILED_MASK) + return false; +/* MCHN START */ +// if ((nStatus & BLOCK_VALID_MASK) < nUpTo) { + if ((nStatus & BLOCK_VALID_MASK) < (unsigned int)nUpTo) { +/* MCHN END */ + nStatus = (nStatus & ~BLOCK_VALID_MASK) | nUpTo; + return true; + } + return false; + } + + //! Build the skiplist pointer for this entry. + void BuildSkip(); + + //! Efficiently find an ancestor of this block. + CBlockIndex* GetAncestor(int height); + const CBlockIndex* GetAncestor(int height) const; +}; + +/** Used to marshal pointers into hashes for db storage. */ +class CDiskBlockIndex : public CBlockIndex +{ +public: + uint256 hashPrev; + + CDiskBlockIndex() { + hashPrev = 0; + } + + explicit CDiskBlockIndex(CBlockIndex* pindex) : CBlockIndex(*pindex) { + hashPrev = (pprev ? pprev->GetBlockHash() : 0); + } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + if (!(nType & SER_GETHASH)) + READWRITE(VARINT(nVersion)); + + READWRITE(VARINT(nHeight)); + READWRITE(VARINT(nStatus)); + READWRITE(VARINT(nTx)); + if (nStatus & (BLOCK_HAVE_DATA | BLOCK_HAVE_UNDO)) + READWRITE(VARINT(nFile)); + if (nStatus & BLOCK_HAVE_DATA) + READWRITE(VARINT(nDataPos)); + if (nStatus & BLOCK_HAVE_UNDO) + READWRITE(VARINT(nUndoPos)); + + // block header + READWRITE(this->nVersion); + READWRITE(hashPrev); + READWRITE(hashMerkleRoot); + READWRITE(nTime); + READWRITE(nBits); + READWRITE(nNonce); + } + + uint256 GetBlockHash() const + { + CBlockHeader block; + block.nVersion = nVersion; + block.hashPrevBlock = hashPrev; + block.hashMerkleRoot = hashMerkleRoot; + block.nTime = nTime; + block.nBits = nBits; + block.nNonce = nNonce; + return block.GetHash(); + } + + + std::string ToString() const + { + std::string str = "CDiskBlockIndex("; + str += CBlockIndex::ToString(); + str += strprintf("\n hashBlock=%s, hashPrev=%s)", + GetBlockHash().ToString(), + hashPrev.ToString()); + return str; + } +}; + +/** An in-memory indexed chain of blocks. */ +class CChain { +private: + std::vector vChain; + +public: + /** Returns the index entry for the genesis block of this chain, or NULL if none. */ + CBlockIndex *Genesis() const { + return vChain.size() > 0 ? vChain[0] : NULL; + } + + /** Returns the index entry for the tip of this chain, or NULL if none. */ + CBlockIndex *Tip() const { + return vChain.size() > 0 ? vChain[vChain.size() - 1] : NULL; + } + + /** Returns the index entry at a particular height in this chain, or NULL if no such height exists. */ + CBlockIndex *operator[](int nHeight) const { + if (nHeight < 0 || nHeight >= (int)vChain.size()) + return NULL; + return vChain[nHeight]; + } + + /** Compare two chains efficiently. */ + friend bool operator==(const CChain &a, const CChain &b) { + return a.vChain.size() == b.vChain.size() && + a.vChain[a.vChain.size() - 1] == b.vChain[b.vChain.size() - 1]; + } + + /** Efficiently check whether a block is present in this chain. */ + bool Contains(const CBlockIndex *pindex) const { + return (*this)[pindex->nHeight] == pindex; + } + + /** Find the successor of a block in this chain, or NULL if the given index is not found or is the tip. */ + CBlockIndex *Next(const CBlockIndex *pindex) const { + if (Contains(pindex)) + return (*this)[pindex->nHeight + 1]; + else + return NULL; + } + + /** Return the maximal height in the chain. Is equal to chain.Tip() ? chain.Tip()->nHeight : -1. */ + int Height() const { + return vChain.size() - 1; + } + + /** Set/initialize a chain with a given tip. */ + void SetTip(CBlockIndex *pindex); + + /** Return a CBlockLocator that refers to a block in this chain (by default the tip). */ + CBlockLocator GetLocator(const CBlockIndex *pindex = NULL) const; + + /** Find the last common block between this chain and a block index entry. */ + const CBlockIndex *FindFork(const CBlockIndex *pindex) const; +}; + +#endif // BITCOIN_CHAIN_H diff --git a/src/chain/checkpoints.cpp b/src/chain/checkpoints.cpp new file mode 100644 index 00000000..84d6f94b --- /dev/null +++ b/src/chain/checkpoints.cpp @@ -0,0 +1,100 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "chain/checkpoints.h" + +#include "chainparams/chainparams.h" +#include "core/main.h" +#include "structs/uint256.h" + +#include + +#include + +namespace Checkpoints { + + /** + * How many times we expect transactions after the last checkpoint to + * be slower. This number is a compromise, as it can't be accurate for + * every system. When reindexing from a fast disk with a slow CPU, it + * can be up to 20, while when downloading from a slow network with a + * fast multicore CPU, it won't be much higher than 1. + */ + static const double SIGCHECK_VERIFICATION_FACTOR = 5.0; + + bool fEnabled = true; + + bool CheckBlock(int nHeight, const uint256& hash) + { + if (!fEnabled) + return true; + + const MapCheckpoints& checkpoints = *Params().Checkpoints().mapCheckpoints; + + MapCheckpoints::const_iterator i = checkpoints.find(nHeight); + if (i == checkpoints.end()) return true; + return hash == i->second; + } + + //! Guess how far we are in the verification process at the given block index + double GuessVerificationProgress(CBlockIndex *pindex, bool fSigchecks) { + if (pindex==NULL) + return 0.0; + + int64_t nNow = time(NULL); + + double fSigcheckVerificationFactor = fSigchecks ? SIGCHECK_VERIFICATION_FACTOR : 1.0; + double fWorkBefore = 0.0; // Amount of work done before pindex + double fWorkAfter = 0.0; // Amount of work left after pindex (estimated) + // Work is defined as: 1.0 per transaction before the last checkpoint, and + // fSigcheckVerificationFactor per transaction after. + + const CCheckpointData &data = Params().Checkpoints(); + + if (pindex->nChainTx <= data.nTransactionsLastCheckpoint) { + double nCheapBefore = pindex->nChainTx; + double nCheapAfter = data.nTransactionsLastCheckpoint - pindex->nChainTx; + double nExpensiveAfter = (nNow - data.nTimeLastCheckpoint)/86400.0*data.fTransactionsPerDay; + fWorkBefore = nCheapBefore; + fWorkAfter = nCheapAfter + nExpensiveAfter*fSigcheckVerificationFactor; + } else { + double nCheapBefore = data.nTransactionsLastCheckpoint; + double nExpensiveBefore = pindex->nChainTx - data.nTransactionsLastCheckpoint; + double nExpensiveAfter = (nNow - pindex->GetBlockTime())/86400.0*data.fTransactionsPerDay; + fWorkBefore = nCheapBefore + nExpensiveBefore*fSigcheckVerificationFactor; + fWorkAfter = nExpensiveAfter*fSigcheckVerificationFactor; + } + + return fWorkBefore / (fWorkBefore + fWorkAfter); + } + + int GetTotalBlocksEstimate() + { + if (!fEnabled) + return 0; + + const MapCheckpoints& checkpoints = *Params().Checkpoints().mapCheckpoints; + + return checkpoints.rbegin()->first; + } + + CBlockIndex* GetLastCheckpoint() + { + if (!fEnabled) + return NULL; + + const MapCheckpoints& checkpoints = *Params().Checkpoints().mapCheckpoints; + + BOOST_REVERSE_FOREACH(const MapCheckpoints::value_type& i, checkpoints) + { + const uint256& hash = i.second; + BlockMap::const_iterator t = mapBlockIndex.find(hash); + if (t != mapBlockIndex.end()) + return t->second; + } + return NULL; + } + +} // namespace Checkpoints diff --git a/src/chain/checkpoints.h b/src/chain/checkpoints.h new file mode 100644 index 00000000..1e38c28c --- /dev/null +++ b/src/chain/checkpoints.h @@ -0,0 +1,45 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_CHECKPOINTS_H +#define BITCOIN_CHECKPOINTS_H + +#include "structs/uint256.h" + +#include + +class CBlockIndex; + +/** + * Block-chain checkpoints are compiled-in sanity checks. + * They are updated every release or three. + */ +namespace Checkpoints +{ +typedef std::map MapCheckpoints; + +struct CCheckpointData { + const MapCheckpoints *mapCheckpoints; + int64_t nTimeLastCheckpoint; + int64_t nTransactionsLastCheckpoint; + double fTransactionsPerDay; +}; + +//! Returns true if block passes checkpoint checks +bool CheckBlock(int nHeight, const uint256& hash); + +//! Return conservative estimate of total number of blocks, 0 if unknown +int GetTotalBlocksEstimate(); + +//! Returns last CBlockIndex* in mapBlockIndex that is a checkpoint +CBlockIndex* GetLastCheckpoint(); + +double GuessVerificationProgress(CBlockIndex* pindex, bool fSigchecks = true); + +extern bool fEnabled; + +} //namespace Checkpoints + +#endif // BITCOIN_CHECKPOINTS_H diff --git a/src/chain/merkleblock.cpp b/src/chain/merkleblock.cpp new file mode 100644 index 00000000..464cd0b8 --- /dev/null +++ b/src/chain/merkleblock.cpp @@ -0,0 +1,153 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "chain/merkleblock.h" + +#include "structs/hash.h" +#include "primitives/block.h" // for MAX_BLOCK_SIZE +#include "utils/utilstrencodings.h" + +using namespace std; + +CMerkleBlock::CMerkleBlock(const CBlock& block, CBloomFilter& filter) +{ + header = block.GetBlockHeader(); + + vector vMatch; + vector vHashes; + + vMatch.reserve(block.vtx.size()); + vHashes.reserve(block.vtx.size()); + + for (unsigned int i = 0; i < block.vtx.size(); i++) + { + const uint256& hash = block.vtx[i].GetHash(); + if (filter.IsRelevantAndUpdate(block.vtx[i])) + { + vMatch.push_back(true); + vMatchedTxn.push_back(make_pair(i, hash)); + } + else + vMatch.push_back(false); + vHashes.push_back(hash); + } + + txn = CPartialMerkleTree(vHashes, vMatch); +} + +uint256 CPartialMerkleTree::CalcHash(int height, unsigned int pos, const std::vector &vTxid) { + if (height == 0) { + // hash at height 0 is the txids themself + return vTxid[pos]; + } else { + // calculate left hash + uint256 left = CalcHash(height-1, pos*2, vTxid), right; + // calculate right hash if not beyond the end of the array - copy left hash otherwise1 + if (pos*2+1 < CalcTreeWidth(height-1)) + right = CalcHash(height-1, pos*2+1, vTxid); + else + right = left; + // combine subhashes + return Hash(BEGIN(left), END(left), BEGIN(right), END(right)); + } +} + +void CPartialMerkleTree::TraverseAndBuild(int height, unsigned int pos, const std::vector &vTxid, const std::vector &vMatch) { + // determine whether this node is the parent of at least one matched txid + bool fParentOfMatch = false; + for (unsigned int p = pos << height; p < (pos+1) << height && p < nTransactions; p++) + fParentOfMatch |= vMatch[p]; + // store as flag bit + vBits.push_back(fParentOfMatch); + if (height==0 || !fParentOfMatch) { + // if at height 0, or nothing interesting below, store hash and stop + vHash.push_back(CalcHash(height, pos, vTxid)); + } else { + // otherwise, don't store any hash, but descend into the subtrees + TraverseAndBuild(height-1, pos*2, vTxid, vMatch); + if (pos*2+1 < CalcTreeWidth(height-1)) + TraverseAndBuild(height-1, pos*2+1, vTxid, vMatch); + } +} + +uint256 CPartialMerkleTree::TraverseAndExtract(int height, unsigned int pos, unsigned int &nBitsUsed, unsigned int &nHashUsed, std::vector &vMatch) { + if (nBitsUsed >= vBits.size()) { + // overflowed the bits array - failure + fBad = true; + return 0; + } + bool fParentOfMatch = vBits[nBitsUsed++]; + if (height==0 || !fParentOfMatch) { + // if at height 0, or nothing interesting below, use stored hash and do not descend + if (nHashUsed >= vHash.size()) { + // overflowed the hash array - failure + fBad = true; + return 0; + } + const uint256 &hash = vHash[nHashUsed++]; + if (height==0 && fParentOfMatch) // in case of height 0, we have a matched txid + vMatch.push_back(hash); + return hash; + } else { + // otherwise, descend into the subtrees to extract matched txids and hashes + uint256 left = TraverseAndExtract(height-1, pos*2, nBitsUsed, nHashUsed, vMatch), right; + if (pos*2+1 < CalcTreeWidth(height-1)) + right = TraverseAndExtract(height-1, pos*2+1, nBitsUsed, nHashUsed, vMatch); + else + right = left; + // and combine them before returning + return Hash(BEGIN(left), END(left), BEGIN(right), END(right)); + } +} + +CPartialMerkleTree::CPartialMerkleTree(const std::vector &vTxid, const std::vector &vMatch) : nTransactions(vTxid.size()), fBad(false) { + // reset state + vBits.clear(); + vHash.clear(); + + // calculate height of tree + int nHeight = 0; + while (CalcTreeWidth(nHeight) > 1) + nHeight++; + + // traverse the partial tree + TraverseAndBuild(nHeight, 0, vTxid, vMatch); +} + +CPartialMerkleTree::CPartialMerkleTree() : nTransactions(0), fBad(true) {} + +uint256 CPartialMerkleTree::ExtractMatches(std::vector &vMatch) { + vMatch.clear(); + // An empty set will not work + if (nTransactions == 0) + return 0; + // check for excessively high numbers of transactions + if (nTransactions > MAX_BLOCK_SIZE / 60) // 60 is the lower bound for the size of a serialized CTransaction + return 0; + // there can never be more hashes provided than one for every txid + if (vHash.size() > nTransactions) + return 0; + // there must be at least one bit per node in the partial tree, and at least one node per hash + if (vBits.size() < vHash.size()) + return 0; + // calculate height of tree + int nHeight = 0; + while (CalcTreeWidth(nHeight) > 1) + nHeight++; + // traverse the partial tree + unsigned int nBitsUsed = 0, nHashUsed = 0; + uint256 hashMerkleRoot = TraverseAndExtract(nHeight, 0, nBitsUsed, nHashUsed, vMatch); + // verify that no problems occured during the tree traversal + if (fBad) + return 0; + // verify that all bits were consumed (except for the padding caused by serializing it as a byte sequence) + if ((nBitsUsed+7)/8 != (vBits.size()+7)/8) + return 0; + // verify that all hashes were consumed + if (nHashUsed != vHash.size()) + return 0; + return hashMerkleRoot; +} diff --git a/src/chain/merkleblock.h b/src/chain/merkleblock.h new file mode 100644 index 00000000..545c746f --- /dev/null +++ b/src/chain/merkleblock.h @@ -0,0 +1,152 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_MERKLEBLOCK_H +#define BITCOIN_MERKLEBLOCK_H + +#include "utils/serialize.h" +#include "structs/uint256.h" +#include "primitives/block.h" +#include "structs/bloom.h" + +#include + +/** Data structure that represents a partial merkle tree. + * + * It represents a subset of the txid's of a known block, in a way that + * allows recovery of the list of txid's and the merkle root, in an + * authenticated way. + * + * The encoding works as follows: we traverse the tree in depth-first order, + * storing a bit for each traversed node, signifying whether the node is the + * parent of at least one matched leaf txid (or a matched txid itself). In + * case we are at the leaf level, or this bit is 0, its merkle node hash is + * stored, and its children are not explorer further. Otherwise, no hash is + * stored, but we recurse into both (or the only) child branch. During + * decoding, the same depth-first traversal is performed, consuming bits and + * hashes as they written during encoding. + * + * The serialization is fixed and provides a hard guarantee about the + * encoded size: + * + * SIZE <= 10 + ceil(32.25*N) + * + * Where N represents the number of leaf nodes of the partial tree. N itself + * is bounded by: + * + * N <= total_transactions + * N <= 1 + matched_transactions*tree_height + * + * The serialization format: + * - uint32 total_transactions (4 bytes) + * - varint number of hashes (1-3 bytes) + * - uint256[] hashes in depth-first order (<= 32*N bytes) + * - varint number of bytes of flag bits (1-3 bytes) + * - byte[] flag bits, packed per 8 in a byte, least significant bit first (<= 2*N-1 bits) + * The size constraints follow from this. + */ +class CPartialMerkleTree +{ +protected: + /** the total number of transactions in the block */ + unsigned int nTransactions; + + /** node-is-parent-of-matched-txid bits */ + std::vector vBits; + + /** txids and internal hashes */ + std::vector vHash; + + /** flag set when encountering invalid data */ + bool fBad; + + /** helper function to efficiently calculate the number of nodes at given height in the merkle tree */ + unsigned int CalcTreeWidth(int height) { + return (nTransactions+(1 << height)-1) >> height; + } + + /** calculate the hash of a node in the merkle tree (at leaf level: the txid's themselves) */ + uint256 CalcHash(int height, unsigned int pos, const std::vector &vTxid); + + /** recursive function that traverses tree nodes, storing the data as bits and hashes */ + void TraverseAndBuild(int height, unsigned int pos, const std::vector &vTxid, const std::vector &vMatch); + + /** + * recursive function that traverses tree nodes, consuming the bits and hashes produced by TraverseAndBuild. + * it returns the hash of the respective node. + */ + uint256 TraverseAndExtract(int height, unsigned int pos, unsigned int &nBitsUsed, unsigned int &nHashUsed, std::vector &vMatch); + +public: + + /** serialization implementation */ + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(nTransactions); + READWRITE(vHash); + std::vector vBytes; + if (ser_action.ForRead()) { + READWRITE(vBytes); + CPartialMerkleTree &us = *(const_cast(this)); + us.vBits.resize(vBytes.size() * 8); + for (unsigned int p = 0; p < us.vBits.size(); p++) + us.vBits[p] = (vBytes[p / 8] & (1 << (p % 8))) != 0; + us.fBad = false; + } else { + vBytes.resize((vBits.size()+7)/8); + for (unsigned int p = 0; p < vBits.size(); p++) + vBytes[p / 8] |= vBits[p] << (p % 8); + READWRITE(vBytes); + } + } + + /** Construct a partial merkle tree from a list of transaction id's, and a mask that selects a subset of them */ + CPartialMerkleTree(const std::vector &vTxid, const std::vector &vMatch); + + CPartialMerkleTree(); + + /** + * extract the matching txid's represented by this partial merkle tree. + * returns the merkle root, or 0 in case of failure + */ + uint256 ExtractMatches(std::vector &vMatch); +}; + + +/** + * Used to relay blocks as header + vector + * to filtered nodes. + */ +class CMerkleBlock +{ +public: + /** Public only for unit testing */ + CBlockHeader header; + CPartialMerkleTree txn; + +public: + /** Public only for unit testing and relay testing (not relayed) */ + std::vector > vMatchedTxn; + + /** + * Create from a CBlock, filtering transactions according to filter + * Note that this will call IsRelevantAndUpdate on the filter for each transaction, + * thus the filter will likely be modified. + */ + CMerkleBlock(const CBlock& block, CBloomFilter& filter); + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(header); + READWRITE(txn); + } +}; + +#endif // BITCOIN_MERKLEBLOCK_H diff --git a/src/chain/pow.cpp b/src/chain/pow.cpp new file mode 100644 index 00000000..4a18f6ba --- /dev/null +++ b/src/chain/pow.cpp @@ -0,0 +1,120 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "chain/pow.h" + +#include "chain/chain.h" +#include "chainparams/chainparams.h" +#include "primitives/block.h" +#include "structs/uint256.h" +#include "utils/util.h" + +unsigned int GetNextWorkRequired(const CBlockIndex* pindexLast, const CBlockHeader *pblock) +{ + unsigned int nProofOfWorkLimit = Params().ProofOfWorkLimit().GetCompact(); + + // Genesis block + if (pindexLast == NULL) + return nProofOfWorkLimit; + +/* MCHN START */ + if(Params().Interval() <= 0) + { + return nProofOfWorkLimit; + } +/* MCHN END */ + // Only change once per interval + if ((pindexLast->nHeight+1) % Params().Interval() != 0) + { + if (Params().AllowMinDifficultyBlocks()) + { + // Special difficulty rule for testnet: + // If the new block's timestamp is more than 2* 10 minutes + // then allow mining of a min-difficulty block. + if (pblock->GetBlockTime() > pindexLast->GetBlockTime() + Params().TargetSpacing()*2) + return nProofOfWorkLimit; + else + { + // Return the last non-special-min-difficulty-rules-block + const CBlockIndex* pindex = pindexLast; + while (pindex->pprev && pindex->nHeight % Params().Interval() != 0 && pindex->nBits == nProofOfWorkLimit) + pindex = pindex->pprev; + return pindex->nBits; + } + } + return pindexLast->nBits; + } + + // Go back by what we want to be 14 days worth of blocks + const CBlockIndex* pindexFirst = pindexLast; + for (int i = 0; pindexFirst && i < Params().Interval()-1; i++) + pindexFirst = pindexFirst->pprev; + assert(pindexFirst); + + // Limit adjustment step + int64_t nActualTimespan = pindexLast->GetBlockTime() - pindexFirst->GetBlockTime(); + LogPrintf(" nActualTimespan = %d before bounds\n", nActualTimespan); + if (nActualTimespan < Params().TargetTimespan()/4) + nActualTimespan = Params().TargetTimespan()/4; + if (nActualTimespan > Params().TargetTimespan()*4) + nActualTimespan = Params().TargetTimespan()*4; + + // Retarget + uint256 bnNew; + uint256 bnOld; + bnNew.SetCompact(pindexLast->nBits); + bnOld = bnNew; + bnNew *= nActualTimespan; + bnNew /= Params().TargetTimespan(); + + if (bnNew > Params().ProofOfWorkLimit()) + bnNew = Params().ProofOfWorkLimit(); + + /// debug print + LogPrintf("GetNextWorkRequired RETARGET\n"); + LogPrintf("Params().TargetTimespan() = %d nActualTimespan = %d\n", Params().TargetTimespan(), nActualTimespan); + LogPrintf("Before: %08x %s\n", pindexLast->nBits, bnOld.ToString()); + LogPrintf("After: %08x %s\n", bnNew.GetCompact(), bnNew.ToString()); + + return bnNew.GetCompact(); +} + +bool CheckProofOfWork(uint256 hash, unsigned int nBits) +{ + bool fNegative; + bool fOverflow; + uint256 bnTarget; + + if (Params().SkipProofOfWorkCheck()) + return true; + + bnTarget.SetCompact(nBits, &fNegative, &fOverflow); + + // Check range + if (fNegative || bnTarget == 0 || fOverflow || bnTarget > Params().ProofOfWorkLimit()) + return error("CheckProofOfWork() : nBits below minimum work"); + + // Check proof of work matches claimed amount + if (hash > bnTarget) + return error("CheckProofOfWork() : hash doesn't match nBits"); + + return true; +} + +uint256 GetBlockProof(const CBlockIndex& block) +{ + uint256 bnTarget; + bool fNegative; + bool fOverflow; + bnTarget.SetCompact(block.nBits, &fNegative, &fOverflow); + if (fNegative || fOverflow || bnTarget == 0) + return 0; + // We need to compute 2**256 / (bnTarget+1), but we can't represent 2**256 + // as it's too large for a uint256. However, as 2**256 is at least as large + // as bnTarget+1, it is equal to ((2**256 - bnTarget - 1) / (bnTarget+1)) + 1, + // or ~bnTarget / (nTarget+1) + 1. + return (~bnTarget / (bnTarget + 1)) + 1; +} diff --git a/src/chain/pow.h b/src/chain/pow.h new file mode 100644 index 00000000..14a4f4a8 --- /dev/null +++ b/src/chain/pow.h @@ -0,0 +1,22 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_POW_H +#define BITCOIN_POW_H + +#include + +class CBlockHeader; +class CBlockIndex; +class uint256; + +unsigned int GetNextWorkRequired(const CBlockIndex* pindexLast, const CBlockHeader *pblock); + +/** Check whether a block hash satisfies the proof-of-work requirement specified by nBits */ +bool CheckProofOfWork(uint256 hash, unsigned int nBits); +uint256 GetBlockProof(const CBlockIndex& block); + +#endif // BITCOIN_POW_H diff --git a/src/chain/txmempool.cpp b/src/chain/txmempool.cpp new file mode 100644 index 00000000..e6bf64b5 --- /dev/null +++ b/src/chain/txmempool.cpp @@ -0,0 +1,846 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "chain/txmempool.h" + +#include "version/clientversion.h" +#include "core/main.h" +#include "utils/streams.h" +#include "utils/util.h" +#include "utils/utilmoneystr.h" +#include "version/bcversion.h" + +#include + +#include "wallet/wallettxs.h" +#include "core/init.h" + +void InvalidWTx(const uint256& wtxid, const char * reason); + +using namespace std; + +void InvalidMempoolWTx(const uint256& wtxid, const char * reason) +{ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + if(pwalletTxsMain->SaveTxFlag((unsigned char*)&wtxid,MC_TFL_INVALID,1) != MC_ERR_NOT_FOUND) + { + LogPrintf("wtxs: Tx %s was removed from mempool: %s, setting INVALID flag\n", wtxid.ToString(),reason); + } + } +} +//void InvalidWTx(const uint256& wtxid, const char * reason); + +CTxMemPoolEntry::CTxMemPoolEntry(): + nFee(0), nTxSize(0), nModSize(0), nTime(0), dPriority(0.0) +{ + nHeight = MEMPOOL_HEIGHT; +} + +CTxMemPoolEntry::CTxMemPoolEntry(const CTransaction& _tx, const CAmount& _nFee, + int64_t _nTime, double _dPriority, + unsigned int _nHeight): + tx(_tx), nFee(_nFee), nTime(_nTime), dPriority(_dPriority), nHeight(_nHeight) +{ + nTxSize = ::GetSerializeSize(tx, SER_NETWORK, PROTOCOL_VERSION); + + nModSize = tx.CalculateModifiedSize(nTxSize); +} + +CTxMemPoolEntry::CTxMemPoolEntry(const CTxMemPoolEntry& other) +{ + *this = other; +} + +double +CTxMemPoolEntry::GetPriority(unsigned int currentHeight) const +{ + CAmount nValueIn = tx.GetValueOut()+nFee; + double deltaPriority = ((double)(currentHeight-nHeight)*nValueIn)/nModSize; + double dResult = dPriority + deltaPriority; + return dResult; +} + +/** + * Keep track of fee/priority for transactions confirmed within N blocks + */ +class CBlockAverage +{ +private: + boost::circular_buffer feeSamples; + boost::circular_buffer prioritySamples; + + template std::vector buf2vec(boost::circular_buffer buf) const + { + std::vector vec(buf.begin(), buf.end()); + return vec; + } + +public: + CBlockAverage() : feeSamples(100), prioritySamples(100) { } + + void RecordFee(const CFeeRate& feeRate) { + feeSamples.push_back(feeRate); + } + + void RecordPriority(double priority) { + prioritySamples.push_back(priority); + } + + size_t FeeSamples() const { return feeSamples.size(); } + size_t GetFeeSamples(std::vector& insertInto) const + { + BOOST_FOREACH(const CFeeRate& f, feeSamples) + insertInto.push_back(f); + return feeSamples.size(); + } + size_t PrioritySamples() const { return prioritySamples.size(); } + size_t GetPrioritySamples(std::vector& insertInto) const + { + BOOST_FOREACH(double d, prioritySamples) + insertInto.push_back(d); + return prioritySamples.size(); + } + + /** + * Used as belt-and-suspenders check when reading to detect + * file corruption + */ + static bool AreSane(const CFeeRate fee, const CFeeRate& minRelayFee) + { + if (fee < CFeeRate(0)) + return false; + if (fee.GetFeePerK() > minRelayFee.GetFeePerK() * 10000) + return false; + return true; + } + static bool AreSane(const std::vector& vecFee, const CFeeRate& minRelayFee) + { + BOOST_FOREACH(CFeeRate fee, vecFee) + { + if (!AreSane(fee, minRelayFee)) + return false; + } + return true; + } + static bool AreSane(const double priority) + { + return priority >= 0; + } + static bool AreSane(const std::vector vecPriority) + { + BOOST_FOREACH(double priority, vecPriority) + { + if (!AreSane(priority)) + return false; + } + return true; + } + + void Write(CAutoFile& fileout) const + { + std::vector vecFee = buf2vec(feeSamples); + fileout << vecFee; + std::vector vecPriority = buf2vec(prioritySamples); + fileout << vecPriority; + } + + void Read(CAutoFile& filein, const CFeeRate& minRelayFee) { + std::vector vecFee; + filein >> vecFee; + if (AreSane(vecFee, minRelayFee)) + feeSamples.insert(feeSamples.end(), vecFee.begin(), vecFee.end()); + else + throw runtime_error("Corrupt fee value in estimates file."); + std::vector vecPriority; + filein >> vecPriority; + if (AreSane(vecPriority)) + prioritySamples.insert(prioritySamples.end(), vecPriority.begin(), vecPriority.end()); + else + throw runtime_error("Corrupt priority value in estimates file."); + if (feeSamples.size() + prioritySamples.size() > 0) + LogPrint("estimatefee", "Read %d fee samples and %d priority samples\n", + feeSamples.size(), prioritySamples.size()); + } +}; + +class CMinerPolicyEstimator +{ +private: + /** + * Records observed averages transactions that confirmed within one block, two blocks, + * three blocks etc. + */ + std::vector history; + std::vector sortedFeeSamples; + std::vector sortedPrioritySamples; + + int nBestSeenHeight; + + /** + * nBlocksAgo is 0 based, i.e. transactions that confirmed in the highest seen block are + * nBlocksAgo == 0, transactions in the block before that are nBlocksAgo == 1 etc. + */ + void seenTxConfirm(const CFeeRate& feeRate, const CFeeRate& minRelayFee, double dPriority, int nBlocksAgo) + { + // Last entry records "everything else". + int nBlocksTruncated = min(nBlocksAgo, (int) history.size() - 1); + assert(nBlocksTruncated >= 0); + + // We need to guess why the transaction was included in a block-- either + // because it is high-priority or because it has sufficient fees. + bool sufficientFee = (feeRate > minRelayFee); + bool sufficientPriority = AllowFree(dPriority); + const char* assignedTo = "unassigned"; + if (sufficientFee && !sufficientPriority && CBlockAverage::AreSane(feeRate, minRelayFee)) + { + history[nBlocksTruncated].RecordFee(feeRate); + assignedTo = "fee"; + } + else if (sufficientPriority && !sufficientFee && CBlockAverage::AreSane(dPriority)) + { + history[nBlocksTruncated].RecordPriority(dPriority); + assignedTo = "priority"; + } + else + { + // Neither or both fee and priority sufficient to get confirmed: + // don't know why they got confirmed. + } + LogPrint("estimatefee", "Seen TX confirm: %s : %s fee/%g priority, took %d blocks\n", + assignedTo, feeRate.ToString(), dPriority, nBlocksAgo); + } + +public: + CMinerPolicyEstimator(int nEntries) : nBestSeenHeight(0) + { + history.resize(nEntries); + } + + void seenBlock(const std::vector& entries, int nBlockHeight, const CFeeRate minRelayFee) + { + if (nBlockHeight <= nBestSeenHeight) + { + // Ignore side chains and re-orgs; assuming they are random + // they don't affect the estimate. + // And if an attacker can re-org the chain at will, then + // you've got much bigger problems than "attacker can influence + // transaction fees." + return; + } + nBestSeenHeight = nBlockHeight; + + // Fill up the history buckets based on how long transactions took + // to confirm. + std::vector > entriesByConfirmations; + entriesByConfirmations.resize(history.size()); + BOOST_FOREACH(const CTxMemPoolEntry& entry, entries) + { + // How many blocks did it take for miners to include this transaction? + int delta = nBlockHeight - entry.GetHeight(); + if (delta <= 0) + { + // Re-org made us lose height, this should only happen if we happen + // to re-org on a difficulty transition point: very rare! + continue; + } + if ((delta-1) >= (int)history.size()) + delta = history.size(); // Last bucket is catch-all + entriesByConfirmations.at(delta-1).push_back(&entry); + } + for (size_t i = 0; i < entriesByConfirmations.size(); i++) + { + std::vector &e = entriesByConfirmations.at(i); + // Insert at most 10 random entries per bucket, otherwise a single block + // can dominate an estimate: + if (e.size() > 10) { + std::random_shuffle(e.begin(), e.end()); + e.resize(10); + } + BOOST_FOREACH(const CTxMemPoolEntry* entry, e) + { + // Fees are stored and reported as BTC-per-kb: + CFeeRate feeRate(entry->GetFee(), entry->GetTxSize()); + double dPriority = entry->GetPriority(entry->GetHeight()); // Want priority when it went IN + seenTxConfirm(feeRate, minRelayFee, dPriority, i); + } + } + + //After new samples are added, we have to clear the sorted lists, + //so they'll be resorted the next time someone asks for an estimate + sortedFeeSamples.clear(); + sortedPrioritySamples.clear(); + + for (size_t i = 0; i < history.size(); i++) { + if (history[i].FeeSamples() + history[i].PrioritySamples() > 0) + LogPrint("estimatefee", "estimates: for confirming within %d blocks based on %d/%d samples, fee=%s, prio=%g\n", + i, + history[i].FeeSamples(), history[i].PrioritySamples(), + estimateFee(i+1).ToString(), estimatePriority(i+1)); + } + } + + /** + * Can return CFeeRate(0) if we don't have any data for that many blocks back. nBlocksToConfirm is 1 based. + */ + CFeeRate estimateFee(int nBlocksToConfirm) + { + nBlocksToConfirm--; + + if (nBlocksToConfirm < 0 || nBlocksToConfirm >= (int)history.size()) + return CFeeRate(0); + + if (sortedFeeSamples.size() == 0) + { + for (size_t i = 0; i < history.size(); i++) + history.at(i).GetFeeSamples(sortedFeeSamples); + std::sort(sortedFeeSamples.begin(), sortedFeeSamples.end(), + std::greater()); + } + if (sortedFeeSamples.size() < 11) + { + // Eleven is Gavin's Favorite Number + // ... but we also take a maximum of 10 samples per block so eleven means + // we're getting samples from at least two different blocks + return CFeeRate(0); + } + + int nBucketSize = history.at(nBlocksToConfirm).FeeSamples(); + + // Estimates should not increase as number of confirmations goes up, + // but the estimates are noisy because confirmations happen discretely + // in blocks. To smooth out the estimates, use all samples in the history + // and use the nth highest where n is (number of samples in previous bucket + + // half the samples in nBlocksToConfirm bucket): + size_t nPrevSize = 0; + for (int i = 0; i < nBlocksToConfirm; i++) + nPrevSize += history.at(i).FeeSamples(); + size_t index = min(nPrevSize + nBucketSize/2, sortedFeeSamples.size()-1); + return sortedFeeSamples[index]; + } + double estimatePriority(int nBlocksToConfirm) + { + nBlocksToConfirm--; + + if (nBlocksToConfirm < 0 || nBlocksToConfirm >= (int)history.size()) + return -1; + + if (sortedPrioritySamples.size() == 0) + { + for (size_t i = 0; i < history.size(); i++) + history.at(i).GetPrioritySamples(sortedPrioritySamples); + std::sort(sortedPrioritySamples.begin(), sortedPrioritySamples.end(), + std::greater()); + } + if (sortedPrioritySamples.size() < 11) + return -1.0; + + int nBucketSize = history.at(nBlocksToConfirm).PrioritySamples(); + + // Estimates should not increase as number of confirmations needed goes up, + // but the estimates are noisy because confirmations happen discretely + // in blocks. To smooth out the estimates, use all samples in the history + // and use the nth highest where n is (number of samples in previous buckets + + // half the samples in nBlocksToConfirm bucket). + size_t nPrevSize = 0; + for (int i = 0; i < nBlocksToConfirm; i++) + nPrevSize += history.at(i).PrioritySamples(); + size_t index = min(nPrevSize + nBucketSize/2, sortedPrioritySamples.size()-1); + return sortedPrioritySamples[index]; + } + + void Write(CAutoFile& fileout) const + { + fileout << nBestSeenHeight; + fileout << history.size(); + BOOST_FOREACH(const CBlockAverage& entry, history) + { + entry.Write(fileout); + } + } + + void Read(CAutoFile& filein, const CFeeRate& minRelayFee) + { + int nFileBestSeenHeight; + filein >> nFileBestSeenHeight; + size_t numEntries; + filein >> numEntries; + if (numEntries <= 0 || numEntries > 10000) + throw runtime_error("Corrupt estimates file. Must have between 1 and 10k entries."); + + std::vector fileHistory; + + for (size_t i = 0; i < numEntries; i++) + { + CBlockAverage entry; + entry.Read(filein, minRelayFee); + fileHistory.push_back(entry); + } + + // Now that we've processed the entire fee estimate data file and not + // thrown any errors, we can copy it to our history + nBestSeenHeight = nFileBestSeenHeight; + history = fileHistory; + assert(history.size() > 0); + } +}; + + +CTxMemPool::CTxMemPool(const CFeeRate& _minRelayFee) : + nTransactionsUpdated(0), + minRelayFee(_minRelayFee) +{ + // Sanity checks off by default for performance, because otherwise + // accepting transactions becomes O(N^2) where N is the number + // of transactions in the pool + fSanityCheck = false; + + // 25 blocks is a compromise between using a lot of disk/memory and + // trying to give accurate estimates to people who might be willing + // to wait a day or two to save a fraction of a penny in fees. + // Confirmation times for very-low-fee transactions that take more + // than an hour or three to confirm are highly variable. + minerPolicyEstimator = new CMinerPolicyEstimator(25); + +/* MCHN START */ + hashList=new mc_Buffer; + hashList->Initialize(sizeof(uint256),sizeof(uint256),0); + hashListPos=0; +/* MCHN END */ +} + +CTxMemPool::~CTxMemPool() +{ +/* MCHN START */ + delete hashList; +/* MCHN END */ + delete minerPolicyEstimator; +} + +void CTxMemPool::pruneSpent(const uint256 &hashTx, CCoins &coins) +{ + LOCK(cs); + + std::map::iterator it = mapNextTx.lower_bound(COutPoint(hashTx, 0)); + + // iterate over all COutPoints in mapNextTx whose hash equals the provided hashTx + while (it != mapNextTx.end() && it->first.hash == hashTx) { + coins.Spend(it->first.n); // and remove those outputs from coins + it++; + } +} + +unsigned int CTxMemPool::GetTransactionsUpdated() const +{ + LOCK(cs); + return nTransactionsUpdated; +} + +void CTxMemPool::AddTransactionsUpdated(unsigned int n) +{ + LOCK(cs); + nTransactionsUpdated += n; +} + + +bool CTxMemPool::addUnchecked(const uint256& hash, const CTxMemPoolEntry &entry) +{ + // Add to memory pool without checking anything. + // Used by main.cpp AcceptToMemoryPool(), which DOES do + // all the appropriate checks. + LOCK(cs); + { + mapTx[hash] = entry; + const CTransaction& tx = mapTx[hash].GetTx(); + for (unsigned int i = 0; i < tx.vin.size(); i++) + mapNextTx[tx.vin[i].prevout] = CInPoint(&tx, i); + nTransactionsUpdated++; + totalTxSize += entry.GetTxSize(); + +/* MCHN START */ + if(hashListPosm_Count) + { + hashList->PutRow(hashListPos,&hash,NULL); + hashListPos++; + } + else + { + hashList->Add(&hash,NULL); + hashListPos=hashList->m_Count; + } +/* MCHN END */ + } + return true; +} + +/* MCHN START */ + +bool CTxMemPool::defragmentHashList() +{ + int posIn,posOut; + uint256 hash; + + posIn=0; + posOut=0; + + while(posInm_Count) + { + hash=*(uint256*)hashList->GetRow(posIn); + if(exists(hash)) + { + if(posIn != posOut) + { + hashList->PutRow(posOut,&hash,NULL); + } + posOut++; + } + posIn++; + } + + if(posOutm_Count) + { + memset(hashList->GetRow(posOut),0,sizeof(uint256)*(hashList->m_Count-posOut)); + } + hashList->SetCount(posOut); + hashListPos=posOut; + + return true; +} + +bool CTxMemPool::shiftHashList(uint32_t rows) +{ + unsigned char *ptr; + int oldrows=hashList->m_Count; + + hashList->SetCount(hashList->m_Count+rows); + + if(oldrows) + { + ptr=hashList->GetRow(0); + memmove(ptr+rows*hashList->m_RowSize,ptr,oldrows*hashList->m_RowSize); + memset(ptr,0,rows*hashList->m_RowSize); + } + + hashListPos=0; + + return true; +} + +/* MCHN END */ + + +void CTxMemPool::remove(const CTransaction &origTx, std::list& removed, bool fRecursive, string wtx_reason) +{ + // Remove transaction from memory pool + { + LOCK(cs); + std::deque txToRemove; + txToRemove.push_back(origTx.GetHash()); + while (!txToRemove.empty()) + { + uint256 hash = txToRemove.front(); + txToRemove.pop_front(); + if (!mapTx.count(hash)) + continue; + const CTransaction& tx = mapTx[hash].GetTx(); + if (fRecursive) { + for (unsigned int i = 0; i < tx.vout.size(); i++) { + std::map::iterator it = mapNextTx.find(COutPoint(hash, i)); + if (it == mapNextTx.end()) + continue; + txToRemove.push_back(it->second.ptx->GetHash()); + } + } + BOOST_FOREACH(const CTxIn& txin, tx.vin) + mapNextTx.erase(txin.prevout); + + removed.push_back(tx); + totalTxSize -= mapTx[hash].GetTxSize(); + mapTx.erase(hash); + nTransactionsUpdated++; + if(wtx_reason.size()) + { + if(hash == origTx.GetHash()) + { + InvalidMempoolWTx(hash,wtx_reason.c_str()); + } + else + { + InvalidMempoolWTx(hash,"orphan"); + } + } + } + } + +} + +void CTxMemPool::removeCoinbaseSpends(const CCoinsViewCache *pcoins, unsigned int nMemPoolHeight) +{ + // Remove transactions spending a coinbase which are now immature + LOCK(cs); + list transactionsToRemove; + for (std::map::const_iterator it = mapTx.begin(); it != mapTx.end(); it++) { + const CTransaction& tx = it->second.GetTx(); + BOOST_FOREACH(const CTxIn& txin, tx.vin) { + std::map::const_iterator it2 = mapTx.find(txin.prevout.hash); + if (it2 != mapTx.end()) + continue; + const CCoins *coins = pcoins->AccessCoins(txin.prevout.hash); + if (fSanityCheck) assert(coins); +/* MCHN START */ +// if (!coins || (coins->IsCoinBase() && nMemPoolHeight - coins->nHeight < COINBASE_MATURITY)) { + if (!coins || (coins->IsCoinBase() && nMemPoolHeight - coins->nHeight < (unsigned int)COINBASE_MATURITY)) { +/* MCHN END */ + transactionsToRemove.push_back(tx); + break; + } + } + } + BOOST_FOREACH(const CTransaction& tx, transactionsToRemove) { + list removed; + remove(tx, removed, true, "coinbase spend"); + } +} + +void CTxMemPool::removeConflicts(const CTransaction &tx, std::list& removed) +{ + // Remove transactions which depend on inputs of tx, recursively + list result; + LOCK(cs); + BOOST_FOREACH(const CTxIn &txin, tx.vin) { + std::map::iterator it = mapNextTx.find(txin.prevout); + if (it != mapNextTx.end()) { + const CTransaction &txConflict = *it->second.ptx; + if (txConflict != tx) + { + remove(txConflict, removed, true, "conflict"); + } + } + } +} + +/** + * Called when a block is connected. Removes from mempool and updates the miner fee estimator. + */ +void CTxMemPool::removeForBlock(const std::vector& vtx, unsigned int nBlockHeight, + std::list& conflicts) +{ + LOCK(cs); + std::vector entries; + BOOST_FOREACH(const CTransaction& tx, vtx) + { + uint256 hash = tx.GetHash(); + if (mapTx.count(hash)) + entries.push_back(mapTx[hash]); + } + minerPolicyEstimator->seenBlock(entries, nBlockHeight, minRelayFee); + BOOST_FOREACH(const CTransaction& tx, vtx) + { + std::list dummy; + remove(tx, dummy, false); + removeConflicts(tx, conflicts); + ClearPrioritisation(tx.GetHash()); + } +} + + +void CTxMemPool::clear() +{ + LOCK(cs); + mapTx.clear(); + mapNextTx.clear(); +/* MCHN START */ + hashList->Clear(); + hashListPos=0; +/* MCHN END */ + totalTxSize = 0; + ++nTransactionsUpdated; +} + +void CTxMemPool::check(const CCoinsViewCache *pcoins) const +{ + if (!fSanityCheck) + return; + + LogPrint("mempool", "Checking mempool with %u transactions and %u inputs\n", (unsigned int)mapTx.size(), (unsigned int)mapNextTx.size()); + + uint64_t checkTotal = 0; + + CCoinsViewCache mempoolDuplicate(const_cast(pcoins)); + + LOCK(cs); + list waitingOnDependants; + for (std::map::const_iterator it = mapTx.begin(); it != mapTx.end(); it++) { + unsigned int i = 0; + checkTotal += it->second.GetTxSize(); + const CTransaction& tx = it->second.GetTx(); + bool fDependsWait = false; + BOOST_FOREACH(const CTxIn &txin, tx.vin) { + // Check that every mempool transaction's inputs refer to available coins, or other mempool tx's. + std::map::const_iterator it2 = mapTx.find(txin.prevout.hash); + if (it2 != mapTx.end()) { + const CTransaction& tx2 = it2->second.GetTx(); + assert(tx2.vout.size() > txin.prevout.n && !tx2.vout[txin.prevout.n].IsNull()); + fDependsWait = true; + } else { + const CCoins* coins = pcoins->AccessCoins(txin.prevout.hash); + assert(coins && coins->IsAvailable(txin.prevout.n)); + } + // Check whether its inputs are marked in mapNextTx. + std::map::const_iterator it3 = mapNextTx.find(txin.prevout); + assert(it3 != mapNextTx.end()); + assert(it3->second.ptx == &tx); + assert(it3->second.n == i); + i++; + } + if (fDependsWait) + waitingOnDependants.push_back(&it->second); + else { + CValidationState state; CTxUndo undo; + assert(CheckInputs(tx, state, mempoolDuplicate, false, 0, false, NULL)); + UpdateCoins(tx, state, mempoolDuplicate, undo, 1000000); + } + } + unsigned int stepsSinceLastRemove = 0; + while (!waitingOnDependants.empty()) { + const CTxMemPoolEntry* entry = waitingOnDependants.front(); + waitingOnDependants.pop_front(); + CValidationState state; + if (!mempoolDuplicate.HaveInputs(entry->GetTx())) { + waitingOnDependants.push_back(entry); + stepsSinceLastRemove++; + assert(stepsSinceLastRemove < waitingOnDependants.size()); + } else { + assert(CheckInputs(entry->GetTx(), state, mempoolDuplicate, false, 0, false, NULL)); + CTxUndo undo; + UpdateCoins(entry->GetTx(), state, mempoolDuplicate, undo, 1000000); + stepsSinceLastRemove = 0; + } + } + for (std::map::const_iterator it = mapNextTx.begin(); it != mapNextTx.end(); it++) { + uint256 hash = it->second.ptx->GetHash(); + map::const_iterator it2 = mapTx.find(hash); + const CTransaction& tx = it2->second.GetTx(); + assert(it2 != mapTx.end()); + assert(&tx == it->second.ptx); + assert(tx.vin.size() > it->second.n); + assert(it->first == it->second.ptx->vin[it->second.n].prevout); + } + + assert(totalTxSize == checkTotal); +} + +void CTxMemPool::queryHashes(vector& vtxid) +{ + vtxid.clear(); + + LOCK(cs); + vtxid.reserve(mapTx.size()); + for (map::iterator mi = mapTx.begin(); mi != mapTx.end(); ++mi) + vtxid.push_back((*mi).first); +} + +bool CTxMemPool::lookup(uint256 hash, CTransaction& result) const +{ + LOCK(cs); + map::const_iterator i = mapTx.find(hash); + if (i == mapTx.end()) return false; + result = i->second.GetTx(); + return true; +} + +CFeeRate CTxMemPool::estimateFee(int nBlocks) const +{ + LOCK(cs); + return minerPolicyEstimator->estimateFee(nBlocks); +} +double CTxMemPool::estimatePriority(int nBlocks) const +{ + LOCK(cs); + return minerPolicyEstimator->estimatePriority(nBlocks); +} + +bool +CTxMemPool::WriteFeeEstimates(CAutoFile& fileout) const +{ + try { + LOCK(cs); + fileout << 99900; // version required to read: 0.9.99 or later + fileout << CLIENT_VERSION; // version that wrote the file + minerPolicyEstimator->Write(fileout); + } + catch (const std::exception &) { + LogPrintf("CTxMemPool::WriteFeeEstimates() : unable to write policy estimator data (non-fatal)"); + return false; + } + return true; +} + +bool +CTxMemPool::ReadFeeEstimates(CAutoFile& filein) +{ + try { + int nVersionRequired, nVersionThatWrote; + filein >> nVersionRequired >> nVersionThatWrote; + if (nVersionRequired > CLIENT_VERSION) + return error("CTxMemPool::ReadFeeEstimates() : up-version (%d) fee estimate file", nVersionRequired); + + LOCK(cs); + minerPolicyEstimator->Read(filein, minRelayFee); + } + catch (const std::exception &) { + LogPrintf("CTxMemPool::ReadFeeEstimates() : unable to read policy estimator data (non-fatal)"); + return false; + } + return true; +} + +void CTxMemPool::PrioritiseTransaction(const uint256 hash, const string strHash, double dPriorityDelta, const CAmount& nFeeDelta) +{ + { + LOCK(cs); + std::pair &deltas = mapDeltas[hash]; + deltas.first += dPriorityDelta; + deltas.second += nFeeDelta; + } + LogPrintf("PrioritiseTransaction: %s priority += %f, fee += %d\n", strHash, dPriorityDelta, FormatMoney(nFeeDelta)); +} + +void CTxMemPool::ApplyDeltas(const uint256 hash, double &dPriorityDelta, CAmount &nFeeDelta) +{ + LOCK(cs); + std::map >::iterator pos = mapDeltas.find(hash); + if (pos == mapDeltas.end()) + return; + const std::pair &deltas = pos->second; + dPriorityDelta += deltas.first; + nFeeDelta += deltas.second; +} + +void CTxMemPool::ClearPrioritisation(const uint256 hash) +{ + LOCK(cs); + mapDeltas.erase(hash); +} + + +CCoinsViewMemPool::CCoinsViewMemPool(CCoinsView *baseIn, CTxMemPool &mempoolIn) : CCoinsViewBacked(baseIn), mempool(mempoolIn) { } + +bool CCoinsViewMemPool::GetCoins(const uint256 &txid, CCoins &coins) const { + // If an entry in the mempool exists, always return that one, as it's guaranteed to never + // conflict with the underlying cache, and it cannot have pruned entries (as it contains full) + // transactions. First checking the underlying cache risks returning a pruned entry instead. + CTransaction tx; + if (mempool.lookup(txid, tx)) { + coins = CCoins(tx, MEMPOOL_HEIGHT); + return true; + } + return (base->GetCoins(txid, coins) && !coins.IsPruned()); +} + +bool CCoinsViewMemPool::HaveCoins(const uint256 &txid) const { + return mempool.exists(txid) || base->HaveCoins(txid); +} diff --git a/src/chain/txmempool.h b/src/chain/txmempool.h new file mode 100644 index 00000000..be0b58e0 --- /dev/null +++ b/src/chain/txmempool.h @@ -0,0 +1,189 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_TXMEMPOOL_H +#define BITCOIN_TXMEMPOOL_H + +#include + +#include "structs/amount.h" +#include "storage/coins.h" +#include "primitives/transaction.h" +#include "utils/sync.h" + +#include "multichain/multichain.h" + +class CAutoFile; + +inline double AllowFreeThreshold() +{ + return COIN * 144 / 250; +} + +inline bool AllowFree(double dPriority) +{ + // Large (in bytes) low-priority (new, small-coin) transactions + // need a fee. + return dPriority > AllowFreeThreshold(); +} + +/** Fake height value used in CCoins to signify they are only in the memory pool (since 0.8) */ +static const unsigned int MEMPOOL_HEIGHT = 0x7FFFFFFF; + +/** + * CTxMemPool stores these: + */ +class CTxMemPoolEntry +{ +private: + CTransaction tx; + CAmount nFee; //! Cached to avoid expensive parent-transaction lookups + size_t nTxSize; //! ... and avoid recomputing tx size + size_t nModSize; //! ... and modified size for priority + int64_t nTime; //! Local time when entering the mempool + double dPriority; //! Priority when entering the mempool + unsigned int nHeight; //! Chain height when entering the mempool + +public: + CTxMemPoolEntry(const CTransaction& _tx, const CAmount& _nFee, + int64_t _nTime, double _dPriority, unsigned int _nHeight); + CTxMemPoolEntry(); + CTxMemPoolEntry(const CTxMemPoolEntry& other); + + const CTransaction& GetTx() const { return this->tx; } + double GetPriority(unsigned int currentHeight) const; + CAmount GetFee() const { return nFee; } + size_t GetTxSize() const { return nTxSize; } + int64_t GetTime() const { return nTime; } + unsigned int GetHeight() const { return nHeight; } +}; + +class CMinerPolicyEstimator; + +/** An inpoint - a combination of a transaction and an index n into its vin */ +class CInPoint +{ +public: + const CTransaction* ptx; + uint32_t n; + + CInPoint() { SetNull(); } + CInPoint(const CTransaction* ptxIn, uint32_t nIn) { ptx = ptxIn; n = nIn; } + void SetNull() { ptx = NULL; n = (uint32_t) -1; } + bool IsNull() const { return (ptx == NULL && n == (uint32_t) -1); } +}; + +/** + * CTxMemPool stores valid-according-to-the-current-best-chain + * transactions that may be included in the next block. + * + * Transactions are added when they are seen on the network + * (or created by the local node), but not all transactions seen + * are added to the pool: if a new transaction double-spends + * an input of a transaction in the pool, it is dropped, + * as are non-standard transactions. + */ +class CTxMemPool +{ +private: + bool fSanityCheck; //! Normally false, true if -checkmempool or -regtest + unsigned int nTransactionsUpdated; + CMinerPolicyEstimator* minerPolicyEstimator; + + CFeeRate minRelayFee; //! Passed to constructor to avoid dependency on main + uint64_t totalTxSize; //! sum of all mempool tx' byte sizes + +public: + mutable CCriticalSection cs; + std::map mapTx; + std::map mapNextTx; + std::map > mapDeltas; + + CTxMemPool(const CFeeRate& _minRelayFee); + ~CTxMemPool(); + + /** + * If sanity-checking is turned on, check makes sure the pool is + * consistent (does not contain two transactions that spend the same inputs, + * all inputs are in the mapNextTx array). If sanity-checking is turned off, + * check does nothing. + */ + void check(const CCoinsViewCache *pcoins) const; + void setSanityCheck(bool _fSanityCheck) { fSanityCheck = _fSanityCheck; } + + bool addUnchecked(const uint256& hash, const CTxMemPoolEntry &entry); + void remove(const CTransaction &tx, std::list& removed, bool fRecursive = false, std::string wtx_reason=""); + void removeCoinbaseSpends(const CCoinsViewCache *pcoins, unsigned int nMemPoolHeight); + void removeConflicts(const CTransaction &tx, std::list& removed); + void removeForBlock(const std::vector& vtx, unsigned int nBlockHeight, + std::list& conflicts); + void clear(); + void queryHashes(std::vector& vtxid); + void pruneSpent(const uint256& hash, CCoins &coins); + unsigned int GetTransactionsUpdated() const; + void AddTransactionsUpdated(unsigned int n); + + /** Affect CreateNewBlock prioritisation of transactions */ + void PrioritiseTransaction(const uint256 hash, const std::string strHash, double dPriorityDelta, const CAmount& nFeeDelta); + void ApplyDeltas(const uint256 hash, double &dPriorityDelta, CAmount &nFeeDelta); + void ClearPrioritisation(const uint256 hash); + + unsigned long size() + { + LOCK(cs); + return mapTx.size(); + } + uint64_t GetTotalTxSize() + { + LOCK(cs); + return totalTxSize; + } + + bool exists(uint256 hash) + { + LOCK(cs); + return (mapTx.count(hash) != 0); + } + + bool lookup(uint256 hash, CTransaction& result) const; + + /** Estimate fee rate needed to get into the next nBlocks */ + CFeeRate estimateFee(int nBlocks) const; + + /** Estimate priority needed to get into the next nBlocks */ + double estimatePriority(int nBlocks) const; + + /** Write/Read estimates to disk */ + bool WriteFeeEstimates(CAutoFile& fileout) const; + bool ReadFeeEstimates(CAutoFile& filein); + +/* MCHN START */ + + mc_Buffer *hashList; + int hashListPos; + + bool defragmentHashList(); + bool shiftHashList(uint32_t rows); + +/* MCHN END */ +}; + +/** + * CCoinsView that brings transactions from a memorypool into view. + * It does not check for spendings by memory pool transactions. + */ +class CCoinsViewMemPool : public CCoinsViewBacked +{ +protected: + CTxMemPool &mempool; + +public: + CCoinsViewMemPool(CCoinsView *baseIn, CTxMemPool &mempoolIn); + bool GetCoins(const uint256 &txid, CCoins &coins) const; + bool HaveCoins(const uint256 &txid) const; +}; + +#endif // BITCOIN_TXMEMPOOL_H diff --git a/src/chain/undo.h b/src/chain/undo.h new file mode 100644 index 00000000..b30c30c1 --- /dev/null +++ b/src/chain/undo.h @@ -0,0 +1,72 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2013 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_UNDO_H +#define BITCOIN_UNDO_H + +#include "utils/compressor.h" +#include "primitives/transaction.h" +#include "utils/serialize.h" + +/** Undo information for a CTxIn + * + * Contains the prevout's CTxOut being spent, and if this was the + * last output of the affected transaction, its metadata as well + * (coinbase or not, height, transaction version) + */ +class CTxInUndo +{ +public: + CTxOut txout; // the txout data before being spent + bool fCoinBase; // if the outpoint was the last unspent: whether it belonged to a coinbase + unsigned int nHeight; // if the outpoint was the last unspent: its height + int nVersion; // if the outpoint was the last unspent: its version + + CTxInUndo() : txout(), fCoinBase(false), nHeight(0), nVersion(0) {} + CTxInUndo(const CTxOut &txoutIn, bool fCoinBaseIn = false, unsigned int nHeightIn = 0, int nVersionIn = 0) : txout(txoutIn), fCoinBase(fCoinBaseIn), nHeight(nHeightIn), nVersion(nVersionIn) { } + + unsigned int GetSerializeSize(int nType, int nVersion) const { + return ::GetSerializeSize(VARINT(nHeight*2+(fCoinBase ? 1 : 0)), nType, nVersion) + + (nHeight > 0 ? ::GetSerializeSize(VARINT(this->nVersion), nType, nVersion) : 0) + + ::GetSerializeSize(CTxOutCompressor(REF(txout)), nType, nVersion); + } + + template + void Serialize(Stream &s, int nType, int nVersion) const { + ::Serialize(s, VARINT(nHeight*2+(fCoinBase ? 1 : 0)), nType, nVersion); + if (nHeight > 0) + ::Serialize(s, VARINT(this->nVersion), nType, nVersion); + ::Serialize(s, CTxOutCompressor(REF(txout)), nType, nVersion); + } + + template + void Unserialize(Stream &s, int nType, int nVersion) { + unsigned int nCode = 0; + ::Unserialize(s, VARINT(nCode), nType, nVersion); + nHeight = nCode / 2; + fCoinBase = nCode & 1; + if (nHeight > 0) + ::Unserialize(s, VARINT(this->nVersion), nType, nVersion); + ::Unserialize(s, REF(CTxOutCompressor(REF(txout))), nType, nVersion); + } +}; + +/** Undo information for a CTransaction */ +class CTxUndo +{ +public: + // undo information for all txins + std::vector vprevout; + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(vprevout); + } +}; + +#endif // BITCOIN_UNDO_H diff --git a/src/chainparams/buildgenesis.cpp b/src/chainparams/buildgenesis.cpp new file mode 100644 index 00000000..f4b337b1 --- /dev/null +++ b/src/chainparams/buildgenesis.cpp @@ -0,0 +1,355 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" + +#include "chainparams/chainparams.h" +#include "utils/util.h" +#include "utils/utilstrencodings.h" +#include "structs/hash.h" + +#include "utils/random.h" +#include "structs/base58.h" + +using namespace std; + +void mc_GetCompoundHash160(void *result,const void *hash1,const void *hash2) +{ + unsigned char conc_subkey[2*20]; + + memcpy(conc_subkey,hash1,20); + memcpy(conc_subkey+20,hash2,20); + *(uint160*)result= Hash160(conc_subkey,conc_subkey+40); +} + +int mc_RandomEncodedBase58String(char * dest,int size) +{ + GetRandBytes((unsigned char*)dest, size); + string str=EncodeBase58((unsigned char*)(&dest[0]),(unsigned char*)(&dest[0])+size); + strcpy(dest,str.c_str()); + return MC_ERR_NOERROR; +} + + +int mc_GenerateConfFiles(const char *network_name) +{ + FILE *fileHan; + char rpcpwd[64]; + + fileHan=mc_OpenFile(NULL,"multichain",".conf","r",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR); + if(fileHan == NULL) + { + fileHan=mc_OpenFile(NULL,"multichain",".conf","w",MC_FOM_RELATIVE_TO_DATADIR); + } + if(fileHan) + { + mc_CloseFile(fileHan); + } + + fileHan=mc_OpenFile(network_name,"multichain",".conf","r",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR); + if(fileHan == NULL) + { + fileHan=mc_OpenFile(network_name,"multichain",".conf","w",MC_FOM_RELATIVE_TO_DATADIR); + if(fileHan) + { + fprintf(fileHan,"rpcuser=multichainrpc\n"); + mc_RandomEncodedBase58String(rpcpwd,32); + fprintf(fileHan,"rpcpassword=%s\n",rpcpwd); +// mc_CloseFile(fileHan); + } + } + if(fileHan) + { + mc_CloseFile(fileHan); + } + + return MC_ERR_NOERROR; +} + + + +int mc_MultichainParams::Build(const unsigned char* pubkey, int pubkey_size) +{ + if(m_Status == MC_PRM_STATUS_VALID) + { + return MC_ERR_NOERROR; + } + + if(m_Status != MC_PRM_STATUS_GENERATED) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + mc_RandomSeed(mc_TimeNowAsUInt()); + + unsigned char hash[32]; + + int err,size; + CBlock genesis; + CMutableTransaction txNew; + int look_for_genesis=1; + int intBits=GetInt64Param("powminimumbits"); + uint32_t nBits,timestamp; + int i; + const unsigned char *ptr; + const unsigned char *pubkey_hash=(unsigned char *)Hash160(pubkey,pubkey+pubkey_size).begin(); + size_t elem_size; + const unsigned char *elem; + int root_stream_name_size; + const unsigned char *root_stream_name; + mc_Script *lpDetails; + mc_Script *lpDetailsScript; + + nBits = 0x7fffff; + for(i=0;i<(intBits-1)%8;i++) + { + nBits /= 2; + } + nBits += 0x20000000; + for(i=0;i<(intBits-1)/8;i++) + { + nBits -= 0x01000000; + } + + + err=SetParam("genesispubkey",(const char*)pubkey,pubkey_size); + if(err) + { + return err; + } + + uint256 bnProofOfWorkLimit=~uint256(0) >> intBits; + uint256 bnProofOfWork=~uint256(0); + + root_stream_name_size=0; + root_stream_name=NULL; + + if(mc_gState->m_Features->Streams()) + { + root_stream_name=(unsigned char *)GetParam("rootstreamname",&root_stream_name_size); + if(IsProtocolMultichain() == 0) + { + root_stream_name_size=0; + } + } + + while(look_for_genesis) + { + genesis.nBits=nBits; + timestamp=mc_TimeNowAsUInt(); + + txNew.vin.resize(1); + + if(root_stream_name_size) + { + txNew.vout.resize(2); + } + else + { + txNew.vout.resize(1); + } + + ptr=(unsigned char *)GetParam("chaindescription",&size); + txNew.vin[0].scriptSig = CScript() << nBits << CScriptNum(4) << vector(ptr, ptr + size - 1); + + txNew.vout[0].nValue = GetInt64Param("initialblockreward");// * COIN; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + txNew.vout[0].scriptPubKey = CScript() << OP_DUP << OP_HASH160 << vector(pubkey_hash, pubkey_hash + 20) << OP_EQUALVERIFY << OP_CHECKSIG; + } + else + { + txNew.vout[0].scriptPubKey = CScript() << vector(pubkey, pubkey + pubkey_size) << OP_CHECKSIG; + } + +// txNew.vout[0].scriptPubKey = CScript() << OP_DUP << OP_HASH160 << vector(pubkey_hash, pubkey_hash + 20) << OP_EQUALVERIFY << OP_CHECKSIG; + +// if(GetInt64Param("anyonecanmine") == 0) + if(IsProtocolMultichain()) + { + mc_Script *lpScript; + + lpScript=new mc_Script; + + if(mc_gState->m_Features->Streams()) + { + lpScript->SetPermission(MC_PTP_GLOBAL_ALL,0,0xffffffff,timestamp); + } + else + { + lpScript->SetPermission(MC_PTP_ALL,0,0xffffffff,timestamp); + } + + elem = lpScript->GetData(0,&elem_size); + txNew.vout[0].scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + + delete lpScript; + } + + if(root_stream_name_size) + { + txNew.vout[1].nValue=0; + lpDetails=new mc_Script; + lpDetails->AddElement(); + if(GetInt64Param("rootstreamopen")) + { + unsigned char b=1; + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_ANYONE_CAN_WRITE,&b,1); + } + + if(mc_gState->m_Features->FixedIn10007()) + { + if( (root_stream_name_size > 1) && (root_stream_name[root_stream_name_size - 1] == 0x00) ) + { + root_stream_name_size--; + } + } + + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_NAME,root_stream_name,root_stream_name_size); + + size_t bytes; + const unsigned char *script; + script=lpDetails->GetData(0,&bytes); + + lpDetailsScript=new mc_Script; + + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_STREAM,0,script,bytes); + + elem = lpDetailsScript->GetData(0,&elem_size); + txNew.vout[1].scriptPubKey=CScript(); + txNew.vout[1].scriptPubKey << vector(elem, elem + elem_size) << OP_DROP << OP_RETURN; + } + else + { + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_STREAM); + + lpDetailsScript->SetGeneralDetails(script,bytes); + txNew.vout[1].scriptPubKey=CScript(); + + for(int e=0;eGetNumElements();e++) + { + elem = lpDetailsScript->GetData(e,&elem_size); + if(e == (lpDetailsScript->GetNumElements() - 1) ) + { + if(elem_size > 0) + { + txNew.vout[1].scriptPubKey << OP_RETURN << vector(elem, elem + elem_size); + } + else + { + txNew.vout[1].scriptPubKey << OP_RETURN; + } + } + else + { + if(elem_size > 0) + { + txNew.vout[1].scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + } + } + } + } + + delete lpDetails; + delete lpDetailsScript; + } + + + genesis.vtx.push_back(txNew); + genesis.hashPrevBlock=uint256(0); + genesis.hashMerkleRoot = genesis.BuildMerkleTree(); + genesis.nVersion = 1; + genesis.nTime = timestamp; + genesis.nNonce=0; + while(look_for_genesis) + { + genesis.nNonce++; + bnProofOfWork=genesis.GetHash(); + if((genesis.nNonce % 0x1000000) == 0) + { + printf("%02x x 2^24 nonces checked\n",genesis.nNonce >> 24); + } + if(bnProofOfWork.CompareTo(bnProofOfWorkLimit) < 0) + { + look_for_genesis=0; + } + if(genesis.nNonce>=0xffffffff) + { + look_for_genesis=0; + } + } + if(genesis.nNonce>=0xffffffff) + { + look_for_genesis=1; + } + } + + err=SetParam("genesisversion",genesis.nVersion); + if(err) + { + return err; + } + err=SetParam("genesistimestamp",genesis.nTime); + if(err) + { + return err; + } + err=SetParam("genesisnbits",genesis.nBits); + if(err) + { + return err; + } + err=SetParam("genesisnonce",genesis.nNonce); + if(err) + { + return err; + } + err=SetParam("genesispubkeyhash",(const char*)pubkey_hash,20); + if(err) + { + return err; + } + if(mc_gState->m_Features->Streams() == 0) + { + err=SetParam("genesisopreturnscript","[not set]",9); // Some value required to make parameter set valid, but valid value should start from OP_RETURN + if(err) + { + return err; + } + } + + mc_HexToBin(hash,genesis.GetHash().ToString().c_str(),32); + err=SetParam("genesishash",(const char*)hash,32); + if(err) + { + return err; + } + + + CalculateHash(hash); + + err=SetParam("chainparamshash",(const char*)hash,32); + if(err) + { + return err; + } + + err=Validate(); + if(err) + { + return err; + } + + err=Write(1); + if(err) + { + return err; + } + + return MC_ERR_NOERROR; +} + + diff --git a/src/chainparams/chainparams.cpp b/src/chainparams/chainparams.cpp new file mode 100644 index 00000000..44cab9af --- /dev/null +++ b/src/chainparams/chainparams.cpp @@ -0,0 +1,738 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "chainparams/chainparams.h" + +#include "utils/random.h" +#include "utils/util.h" +#include "utils/utilstrencodings.h" + +#include + +#include "multichain/multichain.h" + +#include + +using namespace std; +using namespace boost::assign; + +struct SeedSpec6 { + uint8_t addr[16]; + uint16_t port; +}; + +#include "chainparams/chainparamsseeds.h" + +/** + * Main network + */ + +//! Convert the pnSeeds6 array into usable address objects. +static void convertSeed6(std::vector &vSeedsOut, const SeedSpec6 *data, unsigned int count) +{ + // It'll only connect to one or two seed nodes because once it connects, + // it'll get a pile of addresses with newer timestamps. + // Seed nodes are given a random 'last seen time' of between one and two + // weeks ago. + const int64_t nOneWeek = 7*24*60*60; + for (unsigned int i = 0; i < count; i++) + { + struct in6_addr ip; + memcpy(&ip, data[i].addr, sizeof(ip)); + CAddress addr(CService(ip, data[i].port)); + addr.nTime = GetTime() - GetRand(nOneWeek) - nOneWeek; + vSeedsOut.push_back(addr); + } +} + +/** + * What makes a good checkpoint block? + * + Is surrounded by blocks with reasonable timestamps + * (no blocks before with a timestamp after, none after with + * timestamp before) + * + Contains no strange transactions + */ +static Checkpoints::MapCheckpoints mapCheckpoints = + boost::assign::map_list_of + ( 11111, uint256("0x0000000069e244f73d78e8fd29ba2fd2ed618bd6fa2ee92559f542fdb26e7c1d")) + ( 33333, uint256("0x000000002dd5588a74784eaa7ab0507a18ad16a236e7b1ce69f00d7ddfb5d0a6")) + ( 74000, uint256("0x0000000000573993a3c9e41ce34471c079dcf5f52a0e824a81e7f953b8661a20")) + (105000, uint256("0x00000000000291ce28027faea320c8d2b054b2e0fe44a773f3eefb151d6bdc97")) + (134444, uint256("0x00000000000005b12ffd4cd315cd34ffd4a594f430ac814c91184a0d42d2b0fe")) + (168000, uint256("0x000000000000099e61ea72015e79632f216fe6cb33d7899acb35b75c8303b763")) + (193000, uint256("0x000000000000059f452a5f7340de6682a977387c17010ff6e6c3bd83ca8b1317")) + (210000, uint256("0x000000000000048b95347e83192f69cf0366076336c639f9b7228e9ba171342e")) + (216116, uint256("0x00000000000001b4f4b433e81ee46494af945cf96014816a4e2370f11b23df4e")) + (225430, uint256("0x00000000000001c108384350f74090433e7fcf79a606b8e797f065b130575932")) + (250000, uint256("0x000000000000003887df1f29024b06fc2200b55f8af8f35453d7be294df2d214")) + (279000, uint256("0x0000000000000001ae8c72a0b0c301f67e3afca10e819efa9041e458e9bd7e40")) + (295000, uint256("0x00000000000000004d9b4ef50f0f9d686fd69db2e03af35a100370c64632a983")) + ; +static const Checkpoints::CCheckpointData data = { + &mapCheckpoints, + 1397080064, // * UNIX timestamp of last checkpoint block + 36544669, // * total number of transactions between genesis and last checkpoint + // (the tx=... number in the SetBestChain debug.log lines) + 60000.0 // * estimated number of transactions per day after checkpoint + }; + +static Checkpoints::MapCheckpoints mapCheckpointsTestnet = + boost::assign::map_list_of + ( 546, uint256("000000002a936ca763904c3c35fce2f3556c559c0214345d31b1bcebf76acb70")) + ; +static const Checkpoints::CCheckpointData dataTestnet = { + &mapCheckpointsTestnet, + 1337966069, + 1488, + 300 + }; + +/* MCHN START */ +static Checkpoints::MapCheckpoints mapCheckpointsMultichain = + boost::assign::map_list_of + ( -1, uint256("0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206")) + ; +static const Checkpoints::CCheckpointData dataMultichain = { + &mapCheckpointsMultichain, + 0, + 0, + 0 + }; +/* MCHN END */ +static Checkpoints::MapCheckpoints mapCheckpointsRegtest = + boost::assign::map_list_of + ( 0, uint256("0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206")) + ; +static const Checkpoints::CCheckpointData dataRegtest = { + &mapCheckpointsRegtest, + 0, + 0, + 0 + }; + +class CMainParams : public CChainParams { +public: + CMainParams() { + networkID = CBaseChainParams::MAIN; + strNetworkID = "main"; + /** + * The message start string is designed to be unlikely to occur in normal data. + * The characters are rarely used upper ASCII, not valid as UTF-8, and produce + * a large 4-byte int at any alignment. + */ + pchMessageStart[0] = 0xf9; + pchMessageStart[1] = 0xbe; + pchMessageStart[2] = 0xb4; + pchMessageStart[3] = 0xd9; + vAlertPubKey = ParseHex("04fc9702847840aaf195de8442ebecedf5b095cdbb9bc716bda9110971b28a49e0ead8564ff0db22209e0374782c093bb899692d524e9d6a6956e7c5ecbcd68284"); + nDefaultPort = 8333; + bnProofOfWorkLimit = ~uint256(0) >> 32; + nSubsidyHalvingInterval = 210000; + nEnforceBlockUpgradeMajority = 750; + nRejectBlockOutdatedMajority = 950; + nToCheckBlockUpgradeMajority = 1000; + nMinerThreads = 0; + nTargetTimespan = 14 * 24 * 60 * 60; // two weeks + nTargetSpacing = 10 * 60; + + /** + * Build the genesis block. Note that the output of the genesis coinbase cannot + * be spent as it did not originally exist in the database. + * + * CBlock(hash=000000000019d6, ver=1, hashPrevBlock=00000000000000, hashMerkleRoot=4a5e1e, nTime=1231006505, nBits=1d00ffff, nNonce=2083236893, vtx=1) + * CTransaction(hash=4a5e1e, ver=1, vin.size=1, vout.size=1, nLockTime=0) + * CTxIn(COutPoint(000000, -1), coinbase 04ffff001d0104455468652054696d65732030332f4a616e2f32303039204368616e63656c6c6f72206f6e206272696e6b206f66207365636f6e64206261696c6f757420666f722062616e6b73) + * CTxOut(nValue=50.00000000, scriptPubKey=0x5F1DF16B2B704C8A578D0B) + * vMerkleTree: 4a5e1e + */ + const char* pszTimestamp = "The Times 03/Jan/2009 Chancellor on brink of second bailout for banks"; + CMutableTransaction txNew; + txNew.vin.resize(1); + txNew.vout.resize(1); + txNew.vin[0].scriptSig = CScript() << 486604799 << CScriptNum(4) << vector((const unsigned char*)pszTimestamp, (const unsigned char*)pszTimestamp + strlen(pszTimestamp)); + txNew.vout[0].nValue = 50 * COIN; + txNew.vout[0].scriptPubKey = CScript() << ParseHex("04678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5f") << OP_CHECKSIG; + genesis.vtx.push_back(txNew); + genesis.hashPrevBlock = 0; + genesis.hashMerkleRoot = genesis.BuildMerkleTree(); + genesis.nVersion = 1; + genesis.nTime = 1231006505; + genesis.nBits = 0x1d00ffff; + genesis.nNonce = 2083236893; + + hashGenesisBlock = genesis.GetHash(); + assert(hashGenesisBlock == uint256("0x000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f")); + assert(genesis.hashMerkleRoot == uint256("0x4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b")); + + vSeeds.push_back(CDNSSeedData("bitcoin.sipa.be", "seed.bitcoin.sipa.be")); + vSeeds.push_back(CDNSSeedData("bluematt.me", "dnsseed.bluematt.me")); + vSeeds.push_back(CDNSSeedData("dashjr.org", "dnsseed.bitcoin.dashjr.org")); + vSeeds.push_back(CDNSSeedData("bitcoinstats.com", "seed.bitcoinstats.com")); + vSeeds.push_back(CDNSSeedData("xf2.org", "bitseed.xf2.org")); + + base58Prefixes[PUBKEY_ADDRESS] = list_of(0); + base58Prefixes[SCRIPT_ADDRESS] = list_of(5); + base58Prefixes[SECRET_KEY] = list_of(128); + base58Prefixes[EXT_PUBLIC_KEY] = list_of(0x04)(0x88)(0xB2)(0x1E); + base58Prefixes[EXT_SECRET_KEY] = list_of(0x04)(0x88)(0xAD)(0xE4); + + convertSeed6(vFixedSeeds, pnSeed6_main, ARRAYLEN(pnSeed6_main)); + + fRequireRPCPassword = true; + fMiningRequiresPeers = true; + fDefaultCheckMemPool = false; + fAllowMinDifficultyBlocks = false; + fRequireStandard = true; + fMineBlocksOnDemand = false; + fSkipProofOfWorkCheck = false; + fTestnetToBeDeprecatedFieldRPC = false; + } + + const Checkpoints::CCheckpointData& Checkpoints() const + { + return data; + } +}; +static CMainParams mainParams; + +/** + * Testnet (v3) + */ +class CTestNetParams : public CMainParams { +public: + CTestNetParams() { + networkID = CBaseChainParams::TESTNET; + strNetworkID = "test"; + pchMessageStart[0] = 0x0b; + pchMessageStart[1] = 0x11; + pchMessageStart[2] = 0x09; + pchMessageStart[3] = 0x07; + vAlertPubKey = ParseHex("04302390343f91cc401d56d68b123028bf52e5fca1939df127f63c6467cdf9c8e2c14b61104cf817d0b780da337893ecc4aaff1309e536162dabbdb45200ca2b0a"); + nDefaultPort = 18333; + nEnforceBlockUpgradeMajority = 51; + nRejectBlockOutdatedMajority = 75; + nToCheckBlockUpgradeMajority = 100; + nMinerThreads = 0; + nTargetTimespan = 14 * 24 * 60 * 60; //! two weeks + nTargetSpacing = 10 * 60; + + //! Modify the testnet genesis block so the timestamp is valid for a later start. + genesis.nTime = 1296688602; + genesis.nNonce = 414098458; + hashGenesisBlock = genesis.GetHash(); + assert(hashGenesisBlock == uint256("0x000000000933ea01ad0ee984209779baaec3ced90fa3f408719526f8d77f4943")); + + vFixedSeeds.clear(); + vSeeds.clear(); + vSeeds.push_back(CDNSSeedData("alexykot.me", "testnet-seed.alexykot.me")); + vSeeds.push_back(CDNSSeedData("bitcoin.petertodd.org", "testnet-seed.bitcoin.petertodd.org")); + vSeeds.push_back(CDNSSeedData("bluematt.me", "testnet-seed.bluematt.me")); + vSeeds.push_back(CDNSSeedData("bitcoin.schildbach.de", "testnet-seed.bitcoin.schildbach.de")); + + base58Prefixes[PUBKEY_ADDRESS] = list_of(111); + base58Prefixes[SCRIPT_ADDRESS] = list_of(196); + base58Prefixes[SECRET_KEY] = list_of(239); + base58Prefixes[EXT_PUBLIC_KEY] = list_of(0x04)(0x35)(0x87)(0xCF); + base58Prefixes[EXT_SECRET_KEY] = list_of(0x04)(0x35)(0x83)(0x94); + + convertSeed6(vFixedSeeds, pnSeed6_test, ARRAYLEN(pnSeed6_test)); + + fRequireRPCPassword = true; + fMiningRequiresPeers = true; + fDefaultCheckMemPool = false; + fAllowMinDifficultyBlocks = true; + fRequireStandard = false; + fMineBlocksOnDemand = false; + fTestnetToBeDeprecatedFieldRPC = true; + } + const Checkpoints::CCheckpointData& Checkpoints() const + { + return dataTestnet; + } +}; +static CTestNetParams testNetParams; + +/** + * Regression test + */ +class CRegTestParams : public CTestNetParams { +public: + CRegTestParams() { + networkID = CBaseChainParams::REGTEST; + strNetworkID = "regtest"; + pchMessageStart[0] = 0xfa; + pchMessageStart[1] = 0xbf; + pchMessageStart[2] = 0xb5; + pchMessageStart[3] = 0xda; + nSubsidyHalvingInterval = 150; + nEnforceBlockUpgradeMajority = 750; + nRejectBlockOutdatedMajority = 950; + nToCheckBlockUpgradeMajority = 1000; + nMinerThreads = 1; + nTargetTimespan = 14 * 24 * 60 * 60; //! two weeks + nTargetSpacing = 10 * 60; + bnProofOfWorkLimit = ~uint256(0) >> 1; + genesis.nTime = 1296688602; + genesis.nBits = 0x207fffff; + genesis.nNonce = 2; + hashGenesisBlock = genesis.GetHash(); + nDefaultPort = 18444; + assert(hashGenesisBlock == uint256("0x0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206")); + + vFixedSeeds.clear(); //! Regtest mode doesn't have any fixed seeds. + vSeeds.clear(); //! Regtest mode doesn't have any DNS seeds. + + fRequireRPCPassword = false; + fMiningRequiresPeers = false; + fDefaultCheckMemPool = true; + fAllowMinDifficultyBlocks = true; + fRequireStandard = false; + fMineBlocksOnDemand = true; + fTestnetToBeDeprecatedFieldRPC = false; + } + const Checkpoints::CCheckpointData& Checkpoints() const + { + return dataRegtest; + } +}; +static CRegTestParams regTestParams; + +/** + * Unit test + */ +class CUnitTestParams : public CMainParams, public CModifiableParams { +public: + CUnitTestParams() { + networkID = CBaseChainParams::UNITTEST; + strNetworkID = "unittest"; + nDefaultPort = 18445; + vFixedSeeds.clear(); //! Unit test mode doesn't have any fixed seeds. + vSeeds.clear(); //! Unit test mode doesn't have any DNS seeds. + + fRequireRPCPassword = false; + fMiningRequiresPeers = false; + fDefaultCheckMemPool = true; + fAllowMinDifficultyBlocks = false; + fMineBlocksOnDemand = true; + } + + const Checkpoints::CCheckpointData& Checkpoints() const + { + // UnitTest share the same checkpoints as MAIN + return data; + } + + //! Published setters to allow changing values in unit test cases + virtual void setSubsidyHalvingInterval(int anSubsidyHalvingInterval) { nSubsidyHalvingInterval=anSubsidyHalvingInterval; } + virtual void setEnforceBlockUpgradeMajority(int anEnforceBlockUpgradeMajority) { nEnforceBlockUpgradeMajority=anEnforceBlockUpgradeMajority; } + virtual void setRejectBlockOutdatedMajority(int anRejectBlockOutdatedMajority) { nRejectBlockOutdatedMajority=anRejectBlockOutdatedMajority; } + virtual void setToCheckBlockUpgradeMajority(int anToCheckBlockUpgradeMajority) { nToCheckBlockUpgradeMajority=anToCheckBlockUpgradeMajority; } + virtual void setDefaultCheckMemPool(bool afDefaultCheckMemPool) { fDefaultCheckMemPool=afDefaultCheckMemPool; } + virtual void setAllowMinDifficultyBlocks(bool afAllowMinDifficultyBlocks) { fAllowMinDifficultyBlocks=afAllowMinDifficultyBlocks; } + virtual void setSkipProofOfWorkCheck(bool afSkipProofOfWorkCheck) { fSkipProofOfWorkCheck = afSkipProofOfWorkCheck; } +}; +static CUnitTestParams unitTestParams; + + +static CChainParams *pCurrentParams = 0; + +CModifiableParams *ModifiableParams() +{ + assert(pCurrentParams); + assert(pCurrentParams==&unitTestParams); + return (CModifiableParams*)&unitTestParams; +} + +const CChainParams &Params() { + assert(pCurrentParams); + return *pCurrentParams; +} + +CChainParams &Params(CBaseChainParams::Network network) { + switch (network) { + case CBaseChainParams::MAIN: + return mainParams; + case CBaseChainParams::TESTNET: + return testNetParams; + case CBaseChainParams::REGTEST: + return regTestParams; + case CBaseChainParams::UNITTEST: + return unitTestParams; + default: + assert(false && "Unimplemented network"); + return mainParams; + } +} + +void SelectParams(CBaseChainParams::Network network) { + SelectBaseParams(network); + pCurrentParams = &Params(network); +} + +bool SelectParamsFromCommandLine() +{ + CBaseChainParams::Network network = NetworkIdFromCommandLine(); + if (network == CBaseChainParams::MAX_NETWORK_TYPES) + return false; + + SelectParams(network); + return true; +} + +/* MCHN START */ + +class CMultiChainParams : public CMainParams { +public: + CMultiChainParams() { }; + + void SetFixedParams(const char *NetworkName) + { + networkID = CBaseChainParams::MULTICHAIN; + strNetworkID=NetworkName; + nMinerThreads = 0; + fDefaultCheckMemPool = false; + fRequireRPCPassword = false; + + const unsigned char *ucPtr; + + nDefaultPort=MC_DEFAULT_NETWORK_PORT; + + ucPtr=mc_gState->m_NetworkParams->DefaultMessageStart(); + + pchMessageStart[0] = ucPtr[0]; + pchMessageStart[1] = ucPtr[1]; + pchMessageStart[2] = ucPtr[2]; + pchMessageStart[3] = ucPtr[3]; + + vFixedSeeds.clear(); + vSeeds.clear(); + } + + void Initialize(){ + + switch(mc_gState->m_NetworkParams->m_Status) + { + case MC_PRM_STATUS_GENERATED: + case MC_PRM_STATUS_MINIMAL: + case MC_PRM_STATUS_VALID: + break; + default: + return; + } + + const unsigned char *ucPtr; + int size; + + networkID = CBaseChainParams::MULTICHAIN; + + strNetworkID = string(mc_gState->m_NetworkParams->Name()); + + ucPtr=mc_gState->m_NetworkParams->MessageStart(); + pchMessageStart[0] = ucPtr[0]; + pchMessageStart[1] = ucPtr[1]; + pchMessageStart[2] = ucPtr[2]; + pchMessageStart[3] = ucPtr[3]; + + +// vAlertPubKey = ParseHex("04302390343f91cc401d56d68b123028bf52e5fca1939df127f63c6467cdf9c8e2c14b61104cf817d0b780da337893ecc4aaff1309e536162dabbdb45200ca2b0a"); + + nDefaultPort = (int)mc_gState->m_NetworkParams->GetInt64Param("defaultnetworkport"); + bnProofOfWorkLimit = ~uint256(0) >> (int)mc_gState->m_NetworkParams->GetInt64Param("powminimumbits"); + nSubsidyHalvingInterval = (int)(mc_gState->m_NetworkParams->GetInt64Param("rewardhalvinginterval")); + + +/* // MCHN-TODO currently copied from main, decide what to do with it later. + nEnforceBlockUpgradeMajority = 51; + nRejectBlockOutdatedMajority = 75; + nToCheckBlockUpgradeMajority = 100; + */ + + + + nTargetTimespan = (int)mc_gState->m_NetworkParams->GetInt64Param("targetadjustfreq"); + nTargetSpacing = (int)mc_gState->m_NetworkParams->GetInt64Param("targetblocktime"); + + //! Modify the testnet genesis block so the timestamp is valid for a later start. +/* + genesis.nTime = 1296688602; + genesis.nNonce = 414098458; + hashGenesisBlock = genesis.GetHash(); + assert(hashGenesisBlock == uint256("0x000000000933ea01ad0ee984209779baaec3ced90fa3f408719526f8d77f4943")); +*/ + + vFixedSeeds.clear(); + vSeeds.clear(); + + + ucPtr=(const unsigned char*)mc_gState->m_NetworkParams->GetParam("addresspubkeyhashversion",&size); + if(ucPtr) + { + base58Prefixes[PUBKEY_ADDRESS] = list_of(ucPtr[0])(ucPtr[1])(ucPtr[2])(ucPtr[3]); + } + base58Prefixes[PUBKEY_ADDRESS].resize(size); + + ucPtr=(const unsigned char*)mc_gState->m_NetworkParams->GetParam("addressscripthashversion",&size); + if(ucPtr) + { + base58Prefixes[SCRIPT_ADDRESS] = list_of(ucPtr[0])(ucPtr[1])(ucPtr[2])(ucPtr[3]); + } + base58Prefixes[SCRIPT_ADDRESS].resize(size); + + ucPtr=(const unsigned char*)mc_gState->m_NetworkParams->GetParam("privatekeyversion",&size); + if(ucPtr) + { + base58Prefixes[SECRET_KEY] = list_of(ucPtr[0])(ucPtr[1])(ucPtr[2])(ucPtr[3]); + } + base58Prefixes[SECRET_KEY].resize(size); + +/* // MCHN-TODO currently copied from main, decide what to do with it later. + base58Prefixes[EXT_PUBLIC_KEY] = list_of(0x04)(0x35)(0x87)(0xCF); + base58Prefixes[EXT_SECRET_KEY] = list_of(0x04)(0x35)(0x83)(0x94); +*/ + +// convertSeed6(vFixedSeeds, pnSeed6_test, ARRAYLEN(pnSeed6_test)); + + + fMiningRequiresPeers = (mc_gState->m_NetworkParams->GetInt64Param("miningrequirespeers") != 0); + fMiningRequiresPeers=GetBoolArg("-miningrequirespeers", fMiningRequiresPeers); + fAllowMinDifficultyBlocks = (mc_gState->m_NetworkParams->GetInt64Param("allowmindifficultyblocks") != 0); + fRequireStandard = (mc_gState->m_NetworkParams->GetInt64Param("onlyacceptstdtxs") != 0); + fRequireStandard=GetBoolArg("-requirestandard", fRequireStandard); + fMineBlocksOnDemand = GetBoolArg("-mineblocksondemand", false); + fTestnetToBeDeprecatedFieldRPC = (mc_gState->m_NetworkParams->GetInt64Param("chainistestnet") != 0); + + } + + void SetGenesis() + { + int size; + const unsigned char *ptrSigScript; + const unsigned char *ptrPubKeyHash; +// const unsigned char *ptrOpReturnScript; + size_t elem_size; + const unsigned char *elem; + int root_stream_name_size; + const unsigned char *root_stream_name; + mc_Script *lpDetails; + mc_Script *lpDetailsScript; + + switch(mc_gState->m_NetworkParams->m_Status) + { + case MC_PRM_STATUS_VALID: + break; + default: + return; + } + + genesis.vtx.clear(); + + CMutableTransaction txNew; + txNew.vin.resize(1); + +/* + ptrOpReturnScript=NULL; + + if(mc_gState->m_Features->Streams() == 0) + { + ptrOpReturnScript=(unsigned char*)mc_gState->m_NetworkParams->GetParam("genesisopreturnscript",&sizeOpReturn); + } +*/ + root_stream_name_size=0; + root_stream_name=NULL; + if(mc_gState->m_Features->Streams()) + { + root_stream_name=(unsigned char *)mc_gState->m_NetworkParams->GetParam("rootstreamname",&root_stream_name_size); + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + root_stream_name_size=0; + } + } +// if((sizeOpReturn > 0) && (ptrOpReturnScript[0] == OP_RETURN)) + if(root_stream_name_size) + { + txNew.vout.resize(2); + } + else + { + txNew.vout.resize(1); + } + + genesis.nBits = (uint32_t)mc_gState->m_NetworkParams->GetInt64Param("genesisnbits"); + + ptrSigScript=(unsigned char*)mc_gState->m_NetworkParams->GetParam("chaindescription",&size); + txNew.vin[0].scriptSig = CScript() << genesis.nBits << CScriptNum(4) << vector(ptrSigScript, ptrSigScript + size - 1); + + txNew.vout[0].nValue = mc_gState->m_NetworkParams->GetInt64Param("initialblockreward");// * COIN; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + ptrPubKeyHash=(unsigned char*)mc_gState->m_NetworkParams->GetParam("genesispubkeyhash",&size); + txNew.vout[0].scriptPubKey = CScript() << OP_DUP << OP_HASH160 << vector(ptrPubKeyHash, ptrPubKeyHash + size) << OP_EQUALVERIFY << OP_CHECKSIG; + } + else + { + ptrPubKeyHash=(unsigned char*)mc_gState->m_NetworkParams->GetParam("genesispubkey",&size); + txNew.vout[0].scriptPubKey = CScript() << vector(ptrPubKeyHash, ptrPubKeyHash + size) << OP_CHECKSIG; + } + +// if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanmine") == 0) + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + mc_Script *lpScript; + + lpScript=new mc_Script; + + if(mc_gState->m_Features->Streams()) + { + lpScript->SetPermission(MC_PTP_GLOBAL_ALL,0,0xffffffff,(uint32_t)mc_gState->m_NetworkParams->GetInt64Param("genesistimestamp")); + } + else + { + lpScript->SetPermission(MC_PTP_ALL,0,0xffffffff,(uint32_t)mc_gState->m_NetworkParams->GetInt64Param("genesistimestamp")); + } + + elem = lpScript->GetData(0,&elem_size); + txNew.vout[0].scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + + delete lpScript; + } + + if(root_stream_name_size) + { + txNew.vout[1].nValue=0; + lpDetails=new mc_Script; + lpDetails->AddElement(); + if(mc_gState->m_NetworkParams->GetInt64Param("rootstreamopen")) + { + unsigned char b=1; + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_ANYONE_CAN_WRITE,&b,1); + } + + + if(mc_gState->m_Features->FixedIn10007()) + { + if( (root_stream_name_size > 1) && (root_stream_name[root_stream_name_size - 1] == 0x00) ) + { + root_stream_name_size--; + } + } + + + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_NAME,root_stream_name,root_stream_name_size); + + size_t bytes; + const unsigned char *script; + script=lpDetails->GetData(0,&bytes); + + lpDetailsScript=new mc_Script; + + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_STREAM,0,script,bytes); + + elem = lpDetailsScript->GetData(0,&elem_size); + txNew.vout[1].scriptPubKey=CScript(); + txNew.vout[1].scriptPubKey << vector(elem, elem + elem_size) << OP_DROP << OP_RETURN; + } + else + { + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_STREAM); + + lpDetailsScript->SetGeneralDetails(script,bytes); + txNew.vout[1].scriptPubKey=CScript(); + + for(int e=0;eGetNumElements();e++) + { + elem = lpDetailsScript->GetData(e,&elem_size); + if(e == (lpDetailsScript->GetNumElements() - 1) ) + { + if(elem_size > 0) + { + txNew.vout[1].scriptPubKey << OP_RETURN << vector(elem, elem + elem_size); + } + else + { + txNew.vout[1].scriptPubKey << OP_RETURN; + } + } + else + { + if(elem_size > 0) + { + txNew.vout[1].scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + } + } + } + } + + delete lpDetails; + delete lpDetailsScript; + } + +/* + if(ptrOpReturnScript[0] == OP_RETURN) + { + txNew.vout[1].nValue = 0; + txNew.vout[1].scriptPubKey = CScript() << vector(ptrOpReturnScript+1, ptrOpReturnScript + sizeOpReturn - 1); + } +*/ + + genesis.vtx.push_back(txNew); + genesis.hashPrevBlock = 0; + genesis.hashMerkleRoot = genesis.BuildMerkleTree(); + genesis.nVersion = (uint32_t)mc_gState->m_NetworkParams->GetInt64Param("genesisversion"); + genesis.nTime = (uint32_t)mc_gState->m_NetworkParams->GetInt64Param("genesistimestamp"); + genesis.nNonce = (uint32_t)mc_gState->m_NetworkParams->GetInt64Param("genesisnonce"); + + hashGenesisBlock = genesis.GetHash(); + + char storedHash[65]; + mc_BinToHex(storedHash,mc_gState->m_NetworkParams->GetParam("genesishash",NULL),32); + + assert(strcmp(storedHash,hashGenesisBlock.GetHex().c_str()) == 0); + + mapCheckpointsMultichain = + boost::assign::map_list_of + ( 0, uint256(storedHash)) + ; +// assert(hashGenesisBlock == uint256("0x0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206")); + + } + + const Checkpoints::CCheckpointData& Checkpoints() const + { + return dataMultichain; + } +}; +static CMultiChainParams multiChainParams; + + +bool SelectMultiChainParams(const char *NetworkName) +{ + SelectMultiChainBaseParams(NetworkName,(int)mc_gState->m_NetworkParams->GetInt64Param("defaultrpcport")); + + multiChainParams.SetFixedParams(NetworkName); + + multiChainParams.Initialize(); + + pCurrentParams = &multiChainParams; + + return true; +} + +bool InitializeMultiChainParams() +{ + + multiChainParams.Initialize(); + + multiChainParams.SetGenesis(); + + return true; +} + +/* MCHN END */ + + diff --git a/src/chainparams/chainparams.h b/src/chainparams/chainparams.h new file mode 100644 index 00000000..28df6175 --- /dev/null +++ b/src/chainparams/chainparams.h @@ -0,0 +1,163 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_CHAINPARAMS_H +#define BITCOIN_CHAINPARAMS_H + +#include "chainparams/chainparamsbase.h" +#include "chain/checkpoints.h" +#include "primitives/block.h" +#include "protocol/netprotocol.h" +#include "structs/uint256.h" + +#include + +typedef unsigned char MessageStartChars[MESSAGE_START_SIZE]; + +struct CDNSSeedData { + std::string name, host; + CDNSSeedData(const std::string &strName, const std::string &strHost) : name(strName), host(strHost) {} +}; + +/** + * CChainParams defines various tweakable parameters of a given instance of the + * Bitcoin system. There are three: the main network on which people trade goods + * and services, the public test network which gets reset from time to time and + * a regression test mode which is intended for private networks only. It has + * minimal difficulty to ensure that blocks can be found instantly. + */ +class CChainParams +{ +public: + enum Base58Type { + PUBKEY_ADDRESS, + SCRIPT_ADDRESS, + SECRET_KEY, + EXT_PUBLIC_KEY, + EXT_SECRET_KEY, + + MAX_BASE58_TYPES + }; + + const uint256& HashGenesisBlock() const { return hashGenesisBlock; } + const MessageStartChars& MessageStart() const { return pchMessageStart; } + const std::vector& AlertKey() const { return vAlertPubKey; } + int GetDefaultPort() const { return nDefaultPort; } + const uint256& ProofOfWorkLimit() const { return bnProofOfWorkLimit; } + int SubsidyHalvingInterval() const { return nSubsidyHalvingInterval; } + /** Used to check majorities for block version upgrade */ + int EnforceBlockUpgradeMajority() const { return nEnforceBlockUpgradeMajority; } + int RejectBlockOutdatedMajority() const { return nRejectBlockOutdatedMajority; } + int ToCheckBlockUpgradeMajority() const { return nToCheckBlockUpgradeMajority; } + + /** Used if GenerateBitcoins is called with a negative number of threads */ + int DefaultMinerThreads() const { return nMinerThreads; } + const CBlock& GenesisBlock() const { return genesis; } + bool RequireRPCPassword() const { return fRequireRPCPassword; } + /** Make miner wait to have peers to avoid wasting work */ + bool MiningRequiresPeers() const { return fMiningRequiresPeers; } + /** Default value for -checkmempool argument */ + bool DefaultCheckMemPool() const { return fDefaultCheckMemPool; } + /** Allow mining of a min-difficulty block */ + bool AllowMinDifficultyBlocks() const { return fAllowMinDifficultyBlocks; } + /** Skip proof-of-work check: allow mining of any difficulty block */ + bool SkipProofOfWorkCheck() const { return fSkipProofOfWorkCheck; } + /** Make standard checks */ + bool RequireStandard() const { return fRequireStandard; } + int64_t TargetTimespan() const { return nTargetTimespan; } + int64_t TargetSpacing() const { return nTargetSpacing; } + int64_t Interval() const { return nTargetTimespan / nTargetSpacing; } + /** Make miner stop after a block is found. In RPC, don't return until nGenProcLimit blocks are generated */ + bool MineBlocksOnDemand() const { return fMineBlocksOnDemand; } + /** In the future use NetworkIDString() for RPC fields */ + bool TestnetToBeDeprecatedFieldRPC() const { return fTestnetToBeDeprecatedFieldRPC; } + /** Return the BIP70 network string (main, test or regtest) */ + std::string NetworkIDString() const { return strNetworkID; } + const std::vector& DNSSeeds() const { return vSeeds; } + const std::vector& Base58Prefix(Base58Type type) const { return base58Prefixes[type]; } + const std::vector& FixedSeeds() const { return vFixedSeeds; } + virtual const Checkpoints::CCheckpointData& Checkpoints() const = 0; +protected: + CChainParams() {} + + uint256 hashGenesisBlock; + MessageStartChars pchMessageStart; + //! Raw pub key bytes for the broadcast alert signing key. + std::vector vAlertPubKey; + int nDefaultPort; + uint256 bnProofOfWorkLimit; + int nSubsidyHalvingInterval; + int nEnforceBlockUpgradeMajority; + int nRejectBlockOutdatedMajority; + int nToCheckBlockUpgradeMajority; + int64_t nTargetTimespan; + int64_t nTargetSpacing; + int nMinerThreads; + std::vector vSeeds; + std::vector base58Prefixes[MAX_BASE58_TYPES]; + CBaseChainParams::Network networkID; + std::string strNetworkID; + CBlock genesis; + std::vector vFixedSeeds; + bool fRequireRPCPassword; + bool fMiningRequiresPeers; + bool fDefaultCheckMemPool; + bool fAllowMinDifficultyBlocks; + bool fRequireStandard; + bool fMineBlocksOnDemand; + bool fSkipProofOfWorkCheck; + bool fTestnetToBeDeprecatedFieldRPC; +}; + +/** + * Modifiable parameters interface is used by test cases to adapt the parameters in order + * to test specific features more easily. Test cases should always restore the previous + * values after finalization. + */ + +class CModifiableParams { +public: + //! Published setters to allow changing values in unit test cases + virtual void setSubsidyHalvingInterval(int anSubsidyHalvingInterval) =0; + virtual void setEnforceBlockUpgradeMajority(int anEnforceBlockUpgradeMajority)=0; + virtual void setRejectBlockOutdatedMajority(int anRejectBlockOutdatedMajority)=0; + virtual void setToCheckBlockUpgradeMajority(int anToCheckBlockUpgradeMajority)=0; + virtual void setDefaultCheckMemPool(bool aDefaultCheckMemPool)=0; + virtual void setAllowMinDifficultyBlocks(bool aAllowMinDifficultyBlocks)=0; + virtual void setSkipProofOfWorkCheck(bool aSkipProofOfWorkCheck)=0; +}; + + +/** + * Return the currently selected parameters. This won't change after app startup + * outside of the unit tests. + */ +const CChainParams &Params(); + +/** Return parameters for the given network. */ +CChainParams &Params(CBaseChainParams::Network network); + +/** Get modifiable network parameters (UNITTEST only) */ +CModifiableParams *ModifiableParams(); + +/** Sets the params returned by Params() to those for the given network. */ +void SelectParams(CBaseChainParams::Network network); + +/** + * Looks for -regtest or -testnet and then calls SelectParams as appropriate. + * Returns false if an invalid combination is given. + */ +bool SelectParamsFromCommandLine(); + +/* MCHN START */ + +bool SelectMultiChainParams(const char *NetworkName); +bool InitializeMultiChainParams(); + +/* MCHN END */ + + +#endif // BITCOIN_CHAINPARAMS_H diff --git a/src/chainparams/chainparamsbase.cpp b/src/chainparams/chainparamsbase.cpp new file mode 100644 index 00000000..14f2bb6c --- /dev/null +++ b/src/chainparams/chainparamsbase.cpp @@ -0,0 +1,167 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "chainparams/chainparamsbase.h" + +#include "utils/util.h" + +#include + +#include + +using namespace boost::assign; + +/** + * Main network + */ +class CBaseMainParams : public CBaseChainParams +{ +public: + CBaseMainParams() + { + networkID = CBaseChainParams::MAIN; + nRPCPort = 8332; + } +}; +static CBaseMainParams mainParams; + +/** + * Testnet (v3) + */ +class CBaseTestNetParams : public CBaseMainParams +{ +public: + CBaseTestNetParams() + { + networkID = CBaseChainParams::TESTNET; + nRPCPort = 18332; + strDataDir = "testnet3"; + } +}; +static CBaseTestNetParams testNetParams; + +/* + * Regression test + */ +class CBaseRegTestParams : public CBaseTestNetParams +{ +public: + CBaseRegTestParams() + { + networkID = CBaseChainParams::REGTEST; + strDataDir = "regtest"; + } +}; +static CBaseRegTestParams regTestParams; + +/* MCHN START */ + +class CBaseMultiChainParams : public CBaseTestNetParams +{ +public: + CBaseMultiChainParams() + { + networkID = CBaseChainParams::MULTICHAIN; + strDataDir = "multichain"; + } + void setDataDir(std::string NetworkName) + { + strDataDir=NetworkName; + } + void setRPCPort(int port) + { + nRPCPort=port; + } +}; +static CBaseMultiChainParams multiChainParams; +/* MCHN END */ + + + +/* + * Unit test + */ +class CBaseUnitTestParams : public CBaseMainParams +{ +public: + CBaseUnitTestParams() + { + networkID = CBaseChainParams::UNITTEST; + strDataDir = "unittest"; + } +}; +static CBaseUnitTestParams unitTestParams; + +static CBaseChainParams* pCurrentBaseParams = 0; + +const CBaseChainParams& BaseParams() +{ + assert(pCurrentBaseParams); + return *pCurrentBaseParams; +} + +void SelectBaseParams(CBaseChainParams::Network network) +{ + switch (network) { + case CBaseChainParams::MAIN: + pCurrentBaseParams = &mainParams; + break; + case CBaseChainParams::TESTNET: + pCurrentBaseParams = &testNetParams; + break; + case CBaseChainParams::REGTEST: + pCurrentBaseParams = ®TestParams; + break; + case CBaseChainParams::UNITTEST: + pCurrentBaseParams = &unitTestParams; + break; + default: + assert(false && "Unimplemented network"); + return; + } +} + +CBaseChainParams::Network NetworkIdFromCommandLine() +{ + bool fRegTest = GetBoolArg("-regtest", false); + bool fTestNet = GetBoolArg("-testnet", false); + + if (fTestNet && fRegTest) + return CBaseChainParams::MAX_NETWORK_TYPES; + if (fRegTest) + return CBaseChainParams::REGTEST; + if (fTestNet) + return CBaseChainParams::TESTNET; + return CBaseChainParams::MAIN; +} + +bool SelectBaseParamsFromCommandLine() +{ + CBaseChainParams::Network network = NetworkIdFromCommandLine(); + if (network == CBaseChainParams::MAX_NETWORK_TYPES) + return false; + + SelectBaseParams(network); + return true; +} + +/* MCHN START */ + +bool SelectMultiChainBaseParams(const char *NetworkName,int RPCPort) +{ + multiChainParams.setDataDir(std::string(NetworkName)); + multiChainParams.setRPCPort(RPCPort); + pCurrentBaseParams = &multiChainParams; + + return true; +} + +/* MCHN END */ + +bool AreBaseParamsConfigured() +{ + return pCurrentBaseParams != NULL; +} diff --git a/src/chainparams/chainparamsbase.h b/src/chainparams/chainparamsbase.h new file mode 100644 index 00000000..28c26629 --- /dev/null +++ b/src/chainparams/chainparamsbase.h @@ -0,0 +1,75 @@ +// Copyright (c) 2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_CHAINPARAMSBASE_H +#define BITCOIN_CHAINPARAMSBASE_H + +#include +#include + +/** + * CBaseChainParams defines the base parameters (shared between bitcoin-cli and bitcoind) + * of a given instance of the Bitcoin system. + */ +class CBaseChainParams +{ +public: + enum Network { + MAIN, + TESTNET, + REGTEST, + UNITTEST, +/* MCHN START */ + MULTICHAIN, +/* MCHN END */ + + MAX_NETWORK_TYPES + }; + + const std::string& DataDir() const { return strDataDir; } + int RPCPort() const { return nRPCPort; } + +protected: + CBaseChainParams() {} + + int nRPCPort; + std::string strDataDir; + Network networkID; +}; + +/** + * Return the currently selected parameters. This won't change after app startup + * outside of the unit tests. + */ +const CBaseChainParams& BaseParams(); + +/** Sets the params returned by Params() to those for the given network. */ +void SelectBaseParams(CBaseChainParams::Network network); + +/** + * Looks for -regtest or -testnet and returns the appropriate Network ID. + * Returns MAX_NETWORK_TYPES if an invalid combination is given. + */ +CBaseChainParams::Network NetworkIdFromCommandLine(); + +/** + * Calls NetworkIdFromCommandLine() and then calls SelectParams as appropriate. + * Returns false if an invalid combination is given. + */ +bool SelectBaseParamsFromCommandLine(); + +/** + * Return true if SelectBaseParamsFromCommandLine() has been called to select + * a network. + */ +bool AreBaseParamsConfigured(); + +/* MCHN START */ + +bool SelectMultiChainBaseParams(const char *NetworkName,int RPCPort); + +/* MCHN END */ + +#endif // BITCOIN_CHAINPARAMSBASE_H diff --git a/src/chainparams/chainparamsseeds.h b/src/chainparams/chainparamsseeds.h new file mode 100644 index 00000000..6b6e5103 --- /dev/null +++ b/src/chainparams/chainparamsseeds.h @@ -0,0 +1,552 @@ +#ifndef BITCOIN_CHAINPARAMSSEEDS_H +#define BITCOIN_CHAINPARAMSSEEDS_H +/** + * List of fixed seed nodes for the bitcoin network + * AUTOGENERATED by share/seeds/generate-seeds.py + * + * Each line contains a 16-byte IPv6 address and a port. + * IPv4 as well as onion addresses are wrapped inside a IPv6 address accordingly. + */ +static SeedSpec6 pnSeed6_main[] = { + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x01,0x21,0xc5,0x6e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x01,0x22,0xb4,0xf5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x01,0xca,0x80,0xda}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x02,0x23,0xc3,0x19}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x05,0x64,0x7b,0x13}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x05,0xaf,0x91,0xa9}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x05,0xc7,0x85,0xc1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x05,0xc7,0x97,0x0a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x05,0xe4,0x01,0xe6}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x0e,0xc8,0xc8,0x91}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x12,0xe4,0x00,0xbc}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x12,0xe4,0x00,0xc8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x17,0x1e,0xf3,0x99}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x17,0x58,0xe8,0x31}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x17,0x63,0x69,0x09}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x17,0xe2,0x89,0xd0}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x17,0xe3,0xb1,0xa1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x17,0xe3,0xbf,0x32}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x17,0xe5,0x2d,0x20}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x17,0xec,0x90,0x45}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x17,0xfd,0x94,0x71}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x17,0xfd,0xf1,0x16}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x17,0xff,0xe3,0xe7}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x18,0x14,0xcd,0xde}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x18,0x17,0x78,0xfc}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x18,0x5e,0x62,0x60}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x18,0x62,0x5f,0xc9}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x18,0x6f,0x5a,0x37}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x18,0x77,0x77,0x69}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x18,0x8a,0x19,0x95}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x1f,0x03,0xd6,0x2d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x1f,0xba,0x57,0x2e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x1f,0xba,0x65,0x62}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x1f,0xba,0xfa,0xba}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x1f,0xcc,0x99,0x6b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x25,0x2c,0x10,0xe7}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x25,0x2c,0x2c,0x0b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x25,0x78,0xa8,0xcc}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x25,0x8f,0x56,0x1a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x25,0xbb,0x4b,0x18}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x25,0xbc,0x44,0xa9}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x25,0xc0,0x5f,0x96}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x25,0xc9,0xf6,0x74}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x25,0xcd,0x0a,0x8c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2e,0x0a,0xd2,0x11}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2e,0x13,0x8a,0x9a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2e,0x1c,0xcc,0x7b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2e,0x1c,0xcd,0x43}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2e,0x26,0xeb,0xe5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2e,0xa3,0x4c,0xe6}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2e,0xa6,0xa2,0x5b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2e,0xad,0xbe,0x32}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2e,0xe3,0x42,0x84}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2e,0xe5,0xee,0xbb}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2e,0xec,0x74,0xd1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x2f,0x37,0x0e,0x41}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0x07,0xfc,0xe5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0x2e,0x9f,0x5b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0x4e,0x31,0xb5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0x4e,0xe7,0x39}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0x4f,0x99,0x41}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0x74,0x22,0x2c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0x7e,0x56,0xfd}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0x8e,0x29,0x17}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0xc7,0x71,0xc1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0xc8,0x4e,0x6b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0xce,0x8a,0xb1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x32,0xfc,0x34,0x31}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x36,0xa5,0x19,0x4b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x36,0xa9,0x6b,0x28}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x36,0xb3,0xbe,0x38}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x36,0xbb,0x52,0x79}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x36,0xf6,0x55,0xf6}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3a,0x4a,0x07,0xcd}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3a,0x60,0xb7,0x79}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3d,0x3e,0x3a,0x26}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3d,0x3f,0x5b,0x48}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3d,0x3f,0x5b,0x70}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3d,0x48,0xd3,0xe4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3e,0x2b,0x28,0x9a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3e,0x2b,0x82,0xb2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3e,0x50,0xb9,0xd5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3e,0x6d,0x31,0x1a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3e,0xad,0x8b,0x3a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3e,0xb5,0xee,0xba}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3e,0xd2,0x72,0x7f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3f,0x8d,0xe4,0x8a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3f,0x99,0xd5,0x4e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3f,0xdf,0x54,0x91}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x3f,0xfb,0x58,0x70}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x40,0x1f,0x6e,0x32}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x40,0x22,0x79,0x2d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x40,0x72,0x06,0x2a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x40,0x8c,0x7d,0x62}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x40,0x9c,0xc1,0x64}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x41,0x1e,0x2f,0x74}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x41,0x23,0x84,0xb1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x41,0x60,0xc1,0xa5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x41,0x6f,0xbd,0x1a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x42,0x44,0x0a,0x1e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x42,0x72,0x21,0xfa}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x42,0x82,0x2e,0x3f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x42,0xaf,0xd7,0x87}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x42,0xbe,0xfd,0xa5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x42,0xc2,0x26,0xfe}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x42,0xf4,0x62,0x6f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x43,0xa2,0xee,0x1e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x43,0xa9,0xff,0x11}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x43,0xb7,0xad,0x19}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x43,0xdb,0xe9,0x8c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x43,0xe3,0xf0,0x73}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x43,0xf7,0xde,0x47}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x44,0x2b,0x72,0x42}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x44,0x34,0x21,0x24}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x44,0xc6,0xf5,0xf1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x0c,0xe2,0xa5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x0d,0xc6,0xbc}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x0f,0xb3,0x3e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x27,0xef,0x2f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x2f,0x2d,0x57}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x3e,0xd9,0xce}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x40,0x2a,0x1f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x40,0x51,0x3d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x43,0xdb,0xc8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x5a,0x84,0x9d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x5e,0x1e,0xb1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x45,0x88,0xaf,0xf1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x46,0x3d,0x61,0xe4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x46,0x7b,0x76,0x84}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x47,0x3b,0x98,0xb6}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x47,0xc6,0xf8,0x97}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x47,0xc8,0xf2,0x59}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x47,0xe1,0xb3,0x9d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x48,0x0e,0xbb,0x33}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x48,0x26,0x22,0xb4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x48,0x34,0x48,0xbb}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x48,0x5b,0x90,0xb6}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x48,0xa7,0x31,0xd9}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x48,0xc9,0xf3,0x37}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x48,0xdf,0x3c,0xf9}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x48,0xe4,0x99,0x66}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x49,0x1a,0x65,0xe4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x49,0x32,0x9e,0xc8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x49,0xb5,0xcc,0xaa}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4a,0x39,0xc7,0xb4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4a,0x3f,0xde,0xe2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4a,0x51,0xe7,0x15}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4a,0xc1,0x7e,0x52}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4a,0xcf,0xeb,0xa4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4b,0x53,0xc5,0x72}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4b,0x90,0x72,0x09}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4c,0x70,0x05,0xf7}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4c,0xae,0x14,0xf7}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4d,0x25,0xf0,0x8e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4d,0x39,0xca,0x6b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4d,0xac,0x7b,0x35}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4d,0xdd,0x5b,0xfd}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4d,0xeb,0x30,0x30}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4d,0xf5,0x4e,0x02}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4e,0x08,0x3a,0xf9}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4e,0x1b,0xbf,0xb6}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4e,0x81,0xec,0x8d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4e,0x83,0x58,0x2f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4e,0x9d,0xcd,0x06}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4f,0x84,0xe6,0x90}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4f,0x8f,0xbc,0x9b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4f,0xa0,0xdd,0x8c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x4f,0xa1,0x6f,0x72}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x50,0x64,0xbd,0x03}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x50,0x93,0x8c,0x79}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x50,0xcb,0x4b,0x85}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x50,0xdc,0x63,0xe3}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x50,0xde,0x14,0xa9}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x50,0xf1,0x01,0x07}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x51,0x17,0xbf,0xf3}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x51,0x26,0x0b,0xca}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x51,0x50,0x09,0x47}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x51,0x6e,0xd5,0xa5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x51,0x85,0x9b,0xed}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x51,0xab,0x22,0x25}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x51,0xb5,0x9b,0xb4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0x27,0x9c,0x89}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0x49,0xa1,0x5f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0x82,0x2d,0x28}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0xa5,0x99,0x2f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0xa8,0x80,0x85}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0xb3,0xe1,0x76}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0xc2,0xf5,0x9e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0xc7,0x66,0x0a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0xd3,0x1e,0xf3}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0xd9,0x85,0x91}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0xdd,0x80,0x23}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0xdd,0x83,0xb1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x52,0xe9,0xe1,0xcd}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0x00,0xf9,0x92}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0x59,0x1f,0xf9}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0x80,0x1d,0xe7}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0x80,0xfd,0x8e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0x8f,0x82,0x38}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0x96,0x02,0x63}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0x96,0x09,0xc4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0xa1,0x40,0x2d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0xd4,0x67,0xd4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0xd4,0x6f,0x72}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0xf6,0x4b,0x08}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0xfe,0x51,0x1f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x53,0xfe,0x96,0x36}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x54,0x02,0x22,0x68}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x54,0x0f,0x3d,0x3c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x54,0x11,0x19,0x87}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x54,0x2a,0x90,0x13}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x54,0xd4,0xd2,0x87}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x54,0xd7,0xa5,0xe7}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x54,0xee,0x8c,0xb0}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x54,0xf0,0x1f,0xb8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x55,0x19,0xd6,0x89}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x55,0x8b,0xa3,0x84}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x55,0xc7,0x04,0xe4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x55,0xd6,0x3d,0xd1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x55,0xd6,0x6c,0x4d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x56,0x7b,0x10,0x11}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x57,0x30,0x2a,0xc7}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x57,0x68,0xa8,0x68}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x57,0xe5,0x49,0xab}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x57,0xec,0xc4,0x4d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x58,0x61,0x38,0x62}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x58,0x86,0xb2,0x59}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x58,0x96,0xe9,0x13}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x58,0xa8,0x85,0x03}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x58,0xd0,0x12,0xf6}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x58,0xd0,0x21,0xca}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x59,0x12,0x1c,0x15}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x59,0x55,0xdc,0x54}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x59,0xa3,0xe3,0x1c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x59,0xb8,0x53,0x3c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x59,0xe7,0x60,0x53}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x59,0xec,0x31,0x75}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0x5a,0x42,0xd1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0x6a,0xc2,0x61}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0x86,0x4b,0x73}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0x98,0xc1,0x24}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0x98,0xdb,0x23}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0xc5,0x0a,0xea}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0xd1,0x4d,0x65}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0xd2,0x6a,0x93}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0xd6,0xc8,0xcd}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0xdf,0x73,0x26}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0xea,0x30,0xe8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5b,0xfa,0x56,0x12}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5c,0x1b,0x07,0xd1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5c,0xff,0xcf,0x49}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5d,0x4a,0xa3,0xea}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5d,0x54,0x72,0x6a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5d,0x98,0xa6,0x1d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5d,0xab,0xd8,0xdd}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5d,0xb9,0xb1,0x47}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0x13,0x0c,0xf4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0x2a,0x73,0x32}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0x4f,0xb1,0xce}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0x88,0x93,0x77}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0x8f,0xf5,0x05}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0xbc,0x32,0x27}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0xbe,0xe3,0x70}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0xc6,0x87,0x1d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0xe2,0x6b,0x56}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0xf2,0xdb,0x5a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0xf2,0xe5,0xa8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5e,0xf4,0xa0,0x54}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5f,0x1f,0x0a,0xd1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5f,0x55,0x19,0x29}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5f,0x69,0xa1,0x88}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5f,0x9a,0xa5,0x2d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5f,0x9a,0xc8,0xd8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5f,0xa7,0x6d,0x7d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5f,0xd3,0x7d,0xe7}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x5f,0xd3,0xd8,0xeb}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x60,0x21,0x19,0x11}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x60,0x2b,0x82,0xb2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x61,0x76,0x08,0xec}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x62,0x66,0x06,0x7d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x62,0xca,0x14,0x2d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x62,0xd9,0x7d,0xe1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x62,0xea,0xd2,0x6f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x62,0xed,0x14,0x7b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x62,0xff,0x90,0xb0}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x63,0x71,0x40,0x2b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x63,0xe5,0x16,0x08}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x67,0x01,0xd4,0x13}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x67,0x1e,0x2a,0xbd}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x67,0xe0,0xa5,0x30}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x67,0xf3,0x5e,0x8c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x68,0x83,0x6b,0x6b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x68,0x83,0x74,0xb8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x68,0x8f,0x00,0x9c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x68,0xdb,0xb8,0x09}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6a,0xb9,0x26,0xae}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6b,0x06,0x04,0x91}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6b,0x96,0x08,0x1b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6b,0x96,0x21,0x14}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6b,0xaa,0xe4,0x81}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6b,0xaa,0xf0,0xad}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6c,0x33,0x14,0x56}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6c,0x3d,0x95,0xde}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6c,0x3d,0x97,0xac}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6c,0xa1,0x81,0xf7}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6c,0xaa,0x8c,0x15}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0x3c,0xd3,0xd8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0x49,0x2a,0x24}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0x49,0xac,0x8a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0xa3,0xeb,0xef}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0xbe,0xc4,0xdc}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0xc9,0x87,0xd8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0xe4,0x98,0x02}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0xe4,0x9a,0x51}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0xe6,0xdc,0x7d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0xea,0x9c,0xda}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0xeb,0x31,0x1b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x6d,0xeb,0x45,0x54}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x70,0x7c,0x47,0x00}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x71,0x92,0x44,0xfb}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x73,0x1d,0x11,0x52}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x73,0x46,0xb0,0x11}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x75,0x29,0xa2,0xb8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x76,0x1b,0x08,0xaa}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x77,0xe6,0x07,0xd3}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x77,0xf6,0x47,0x34}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x79,0xac,0x08,0x64}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x7a,0x80,0x6d,0x94}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x7b,0xe7,0xe0,0x3f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x80,0xaf,0xc3,0x1f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x80,0xc7,0xa4,0x60}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x80,0xc7,0xfe,0xf4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x81,0x61,0x45,0x4c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x81,0x7b,0x07,0x07}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x81,0x7b,0x07,0x27}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x81,0xba,0x11,0x11}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x83,0xf7,0xa9,0xbe}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x85,0xf2,0xd1,0x3f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x86,0x66,0x5e,0x26}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x86,0x77,0x11,0x91}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x89,0x74,0xa0,0xb0}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x89,0xe2,0x22,0x2a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x8a,0xd2,0xd9,0xaa}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x8d,0xff,0xa6,0xc2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x8f,0xd7,0x81,0x7e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x90,0x4c,0xf4,0x13}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x92,0x94,0x34,0xa2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x92,0x94,0x50,0x39}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x92,0xb9,0x13,0x1e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x92,0xb9,0x8e,0x56}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x92,0xb9,0xfd,0x33}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x94,0xfb,0x06,0xd6}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x95,0x9a,0x9b,0xeb}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x95,0xd2,0x85,0xf4}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x97,0xe0,0xf8,0xfc}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x99,0x79,0x4b,0xe5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x99,0x7f,0xfb,0x43}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x9a,0x14,0x02,0x8b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x9d,0x0d,0x3d,0x05}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x9e,0x3a,0xad,0x30}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0x9f,0xfd,0x17,0x84}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xd1,0x6e,0xda}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xd5,0xfe,0xcd}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xef,0xfe,0x64}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xf2,0x96,0x27}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xf3,0x51,0x8a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xf3,0xeb,0x38}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xf4,0x4f,0x10}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xf5,0xd9,0x77}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xf8,0x66,0x75}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xfb,0x6c,0x35}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xfe,0x95,0x8b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa2,0xff,0x74,0x4e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa6,0x46,0x5e,0x6a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa7,0x58,0x2d,0x7c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xa7,0x58,0x78,0xd2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xad,0x1a,0x31,0x2b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xad,0x1e,0x0e,0x06}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xad,0x50,0x72,0xc5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xad,0xa7,0xd6,0xf3}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xad,0xd0,0xdb,0x6c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xad,0xdc,0x43,0x9c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xad,0xec,0x65,0x22}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xad,0xf6,0x6b,0x22}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xad,0xff,0xed,0xf1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xae,0x02,0xd5,0xd1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xae,0x33,0x17,0xe0}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xae,0x33,0x7b,0x9f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xae,0x39,0xd4,0x79}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xae,0x6d,0x21,0x1c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xaf,0x7e,0x7c,0x5b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xaf,0x7e,0x7c,0x5c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb0,0x0a,0x74,0xf2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb0,0x24,0x23,0x7e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb0,0x24,0x63,0xde}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb0,0x7c,0x6e,0x2f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb0,0xc2,0x21,0x2c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb0,0xdf,0xc9,0xc6}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb2,0x3e,0x1a,0x53}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb2,0x3e,0x24,0x30}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb2,0x3e,0xd4,0x8d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb2,0x3e,0xfe,0x3b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb2,0x4e,0xfa,0x03}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb2,0x9b,0x56,0xe2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb2,0xaf,0x86,0x23}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb2,0xf8,0x6f,0x04}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb2,0xfe,0x01,0xaa}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb2,0xfe,0x22,0xa1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb3,0x2b,0x72,0x0e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb6,0xd5,0xd0,0x1c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb8,0x44,0x02,0x2e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb8,0x48,0xee,0x2a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb8,0x5e,0xe2,0x22}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb8,0x5e,0xe3,0x3a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb8,0x6b,0x8b,0x3a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb8,0x6b,0xce,0x2d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb9,0x0a,0x30,0x75}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb9,0x15,0xd8,0x9c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb9,0x26,0x2f,0xe0}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb9,0x2d,0xc0,0x81}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb9,0x35,0x81,0xe6}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb9,0x35,0x83,0x72}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb9,0x37,0x35,0x3d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb9,0x37,0x35,0x3f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb9,0x3d,0x77,0x02}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xb9,0x3d,0x94,0xcb}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xba,0x02,0xa7,0x17}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xbc,0x5c,0x4b,0xb2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xbc,0x7a,0x5c,0x86}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xbc,0x8a,0x09,0xd0}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xbc,0xa5,0xd1,0x94}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xbc,0xe2,0xce,0xef}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xbe,0x0a,0x08,0x7c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xbe,0x0a,0x0a,0x93}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc0,0x00,0x82,0x8e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc0,0x03,0x59,0x9f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc0,0x49,0xea,0x8a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc0,0x4b,0x5f,0x6b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc0,0x5f,0x64,0x66}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc0,0x9b,0x54,0xb5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc0,0xa9,0xe9,0xce}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc0,0xc6,0x5d,0x56}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc0,0xe3,0x87,0xd8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc1,0x00,0x6d,0x03}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc1,0x4d,0x32,0xd0}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc1,0x6d,0x44,0x3e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc1,0x96,0x79,0x25}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc1,0xe0,0x45,0x62}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc2,0x4f,0x08,0x25}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc2,0x8d,0x56,0x0a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc3,0x0c,0xb4,0x5e}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc3,0x38,0x3f,0x0a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc3,0x74,0x5d,0x5d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc3,0x9a,0xae,0xe2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc3,0x9f,0x6f,0x62}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc3,0xa9,0x8a,0x02}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc3,0xbd,0x7e,0x23}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc3,0xc5,0xaf,0xbe}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc5,0xf2,0x5d,0x52}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc6,0x0b,0xd6,0x93}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc6,0x31,0x29,0x15}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc7,0x21,0x7c,0xba}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc7,0xcc,0xba,0x92}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc7,0xe9,0xee,0x73}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xc7,0xf1,0xbd,0x42}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xca,0x3c,0x44,0xf2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xca,0x3c,0x45,0xe8}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xcb,0xb7,0x97,0x27}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xcb,0xdb,0x0e,0xcc}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xcc,0x2c,0x7b,0x6d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xcc,0x2c,0x7b,0xa2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xcc,0x2d,0x78,0xb2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xce,0xbe,0x86,0x2c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xce,0xf8,0xb8,0x7f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xcf,0xf4,0x49,0x08}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd0,0x42,0x1e,0x1b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd1,0x51,0x09,0xdf}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd1,0x69,0xf3,0xe5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd1,0x7e,0x46,0x9f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd1,0x8c,0x1e,0xa9}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd1,0xa5,0x80,0xeb}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd1,0xbe,0x02,0xf2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd2,0x42,0xfe,0xec}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd2,0x49,0x1b,0x21}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd3,0x48,0x42,0xe5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd4,0x19,0x25,0x7c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd4,0x47,0xeb,0x72}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd4,0x47,0xfc,0x6d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd4,0x72,0x30,0x1f}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd4,0xae,0x97,0x76}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd5,0x42,0xcd,0xc2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd5,0x81,0xf8,0x8b}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd5,0x88,0x57,0x22}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd5,0xa5,0x52,0x85}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd5,0xa7,0x11,0x06}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd5,0xb3,0x9e,0xfd}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd5,0xbd,0x35,0x7d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd5,0xde,0xd0,0x5d}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd8,0x31,0x9e,0xa1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd8,0x37,0x8f,0x9a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd8,0x83,0x5b,0x64}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd8,0xf5,0xce,0xb5}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd8,0xfa,0x8a,0xe6}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd9,0x0b,0xe1,0xbd}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd9,0x17,0x06,0x85}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd9,0x4b,0x58,0xb2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd9,0xac,0x8f,0x8c}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd9,0xc3,0xa9,0xd1}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xd9,0xc4,0xf8,0x6a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xdb,0x8a,0xa1,0xa2}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xde,0xa7,0xf8,0x5a}, 8333}, + {{0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xff,0xff,0xdf,0x12,0xfe,0x37}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x0a,0x26,0x27,0x21,0xae,0x94,0xd5,0xc2,0x72,0x24}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0xea,0xb9,0x5b,0x63,0x1d,0x94,0xe2,0xed,0xec,0xa1}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0xce,0x36,0xa1,0xc1,0xd6,0x64,0x43,0xfb,0xb3,0xe7}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x26,0xe6,0xdf,0xeb,0xe5,0xc5,0x9a,0x87,0x5e,0x22}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x52,0x71,0xa2,0x43,0x2a,0xe6,0x6c,0x8e,0xe4,0x7b}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x7c,0xf4,0x0b,0x4c,0x52,0xd5,0x16,0xcf,0xf5,0x06}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x98,0xff,0x33,0x38,0xbb,0x43,0x08,0x8d,0x95,0x9e}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x16,0x64,0x1b,0x1f,0x8f,0x87,0x18,0x7d,0xa3,0x2b}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0xb8,0xda,0x83,0x67,0x90,0x6f,0x46,0x10,0xdb,0x53}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0xc3,0x1b,0x22,0x8c,0x89,0x60,0xbf,0xca,0x88,0xa1}, 7033}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x4d,0xe3,0x5b,0x75,0x10,0x46,0x5e,0xf0,0x99,0x8b}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x0a,0xba,0x44,0x94,0x9d,0xf5,0xc0,0xaa,0xcd,0x4a}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x96,0x64,0xce,0x6d,0xd4,0xfb,0xa7,0x6b,0x60,0xb5}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0xe9,0x8f,0x0b,0x72,0xc9,0xf1,0xde,0x62,0xd4,0x66}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x25,0x5c,0xbc,0x34,0xe8,0x9f,0xe4,0x7c,0x90,0x93}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0xe0,0xa2,0x72,0xef,0xfa,0x7b,0x88,0x95,0x8b,0x9c}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x99,0x69,0xc5,0x40,0xa7,0x95,0xbb,0x25,0xc1,0xfa}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x46,0xa3,0xd9,0x84,0x08,0xc8,0x7f,0xd3,0xeb,0xc5}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x0c,0xc4,0xd2,0x4f,0x74,0x99,0xb3,0x8c,0xe8,0x25}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0xac,0x9d,0xb8,0xf8,0x4c,0x4b,0x9c,0xc3,0x9c,0xc6}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x7c,0x1d,0x28,0x9f,0xd6,0x28,0x28,0x22,0x4f,0x7a}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0xce,0x36,0xa1,0xc1,0xd6,0x64,0x43,0xfb,0xb3,0xe7}, 8333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x8f,0x06,0x4e,0x64,0xbc,0x5e,0x1a,0x8a,0x71,0x97}, 8444} +}; + +static SeedSpec6 pnSeed6_test[] = { + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x99,0xcb,0x26,0x31,0xba,0x48,0x51,0x31,0x39,0x0d}, 18333}, + {{0xfd,0x87,0xd8,0x7e,0xeb,0x43,0x44,0xf4,0xf4,0xf0,0xbf,0xf7,0x7e,0x6d,0xc4,0xe8}, 18333} +}; +#endif // BITCOIN_CHAINPARAMSSEEDS_H diff --git a/src/chainparams/globals.h b/src/chainparams/globals.h new file mode 100644 index 00000000..411dcada --- /dev/null +++ b/src/chainparams/globals.h @@ -0,0 +1,21 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef GLOBALS_H +#define GLOBALS_H + +mc_State* mc_gState; +unsigned int MIN_RELAY_TX_FEE = 1000; // new +unsigned int MAX_OP_RETURN_RELAY = 40; // standard.h +unsigned int MAX_BLOCK_SIZE = 1000000; // block.h +unsigned int DEFAULT_BLOCK_MAX_SIZE = 750000; // main.h +unsigned int MAX_STANDARD_TX_SIZE = 100000; // main.h +int COINBASE_MATURITY = 100; // main.h +int64_t COIN = 100000000; // amount.h +int64_t CENT = 1000000; // amount.h +int64_t MAX_MONEY = 21000000 * COIN; // amount.h +unsigned int MAX_SCRIPT_ELEMENT_SIZE=520; // script.h +int MAX_OP_RETURN_SHOWN=16384; + +#endif /* GLOBALS_H */ + diff --git a/src/chainparams/paramlist.h b/src/chainparams/paramlist.h new file mode 100644 index 00000000..f8e12243 --- /dev/null +++ b/src/chainparams/paramlist.h @@ -0,0 +1,286 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAINPARAMLIST_H +#define MULTICHAINPARAMLIST_H + +static const mc_OneMultichainParam MultichainParamArray[] = +{ + { "chainprotocol" , "chain-protocol" , + MC_PRM_STRING | MC_PRM_USER | MC_PRM_CLONE | MC_PRM_SPECIAL , 16, 0, 0, 0, 0.0, 10001, 0, "-mc-chainprotocol", + "chaindescription","Basic chain parameters", + "Chain protocol: multichain (permissions, native assets) or bitcoin"}, + { "chaindescription" , "chain-description" , + MC_PRM_STRING | MC_PRM_USER | MC_PRM_CLONE | MC_PRM_SPECIAL ,256, 0, 0, 0, 0.0, 10001, 0, "-mc-chaindescription", + "rootstreamname","", + "Chain description, embedded in genesis block coinbase, max 256 chars."}, + { "rootstreamname" , "root-stream-name" , + MC_PRM_STRING | MC_PRM_USER | MC_PRM_CLONE | MC_PRM_SPECIAL,256, 0, 0, 0, 0.0, 10006, 0, "-mc-rootstreamname", + "rootstreamopen","", + "Root stream name, blank means no root stream."}, + { "rootstreamopen" , "root-stream-open" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1 , 1, 0, 0, 0.0, 10006, 0, "-mc-rootstreamopen", + "chainistestnet","", + "Allow anyone to publish in root stream"}, + { "chainistestnet" , "chain-is-testnet" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0, 0, 0.0, 10001, 0, "-mc-chainistestnet", + "targetblocktime","", + "Content of the 'testnet' field of API responses, for compatibility."}, + { "targetblocktime" , "target-block-time" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE , -1, 15, 5, 86400, 0.0, 10001, 0, "-mc-targetblocktime", + "maximumblocksize","", + "Target time between blocks (transaction confirmation delay), seconds."}, + { "maximumblocksize" , "maximum-block-size" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE , -1, 8388608, 1000,1000000000, 0.0, 10001, 0, "-mc-maximumblocksize", + "defaultnetworkport","", + "Maximum block size in bytes."}, + { "defaultnetworkport" , "default-network-port" , + MC_PRM_UINT32 | MC_PRM_GENERATED | MC_PRM_CLONE , -1, MC_DEFAULT_NETWORK_PORT, 1024, 65535, 0.0, 10001, 0, "-mc-defaultnetworkport", + "defaultrpcport","", + "Default TCP/IP port for peer-to-peer connection with other nodes."}, + { "defaultrpcport" , "default-rpc-port" , + MC_PRM_UINT32 | MC_PRM_GENERATED | MC_PRM_CLONE | MC_PRM_MINIMAL , -1, MC_DEFAULT_RPC_PORT, 1024, 65535, 0.0, 10001, 0, "-mc-defaultrpcport", + "anyonecanconnect","", + "Default TCP/IP port for incoming JSON-RPC API requests."}, + + { "anyonecanconnect" , "anyone-can-connect" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0, 0, 0.0, 10001, 0, "-mc-anyonecanconnect", + "anyonecansend","Global permissions", + "Anyone can connect, i.e. a publicly readable blockchain."}, + { "anyonecansend" , "anyone-can-send" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0, 0, 0.0, 10001, 0, "-mc-anyonecansend", + "anyonecanreceive","", + "Anyone can send, i.e. transaction signing not restricted by address."}, + { "anyonecanreceive" , "anyone-can-receive" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0, 0, 0.0, 10001, 0, "-mc-anyonecanreceive", + "anyonecanreceiveempty","", + "Anyone can receive, i.e. transaction outputs not restricted by address."}, + { "anyonecanreceiveempty" , "anyone-can-receive-empty" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 1, 0, 0, 0.0, 10007, 0, "-mc-anyonecanreceiveempty", + "anyonecancreate","", + "Anyone can receive empty output, i.e. without permission grants, asset transafrs and zero native currency."}, + { "anyonecancreate" , "anyone-can-create" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0, 0, 0.0, 10006, 0, "-mc-anyonecancreate", + "anyonecanissue","", + "Anyone can create new streams."}, + { "anyonecanissue" , "anyone-can-issue" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0, 0, 0.0, 10001, 0, "-mc-anyonecanissue", + "anyonecanmine","", + "Anyone can issue new native assets."}, + { "anyonecanmine" , "anyone-can-mine" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0, 0, 0.0, 10001, 0, "-mc-anyonecanmine", + "anyonecanactivate","", + "Anyone can mine blocks (confirm transactions)."}, + { "anyonecanactivate" , "anyone-can-activate" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0, 0, 0.0, 10003, 0, "-mc-anyonecanactivate", + "anyonecanadmin","", + "Anyone can grant or revoke connect, send and receive permissions."}, + { "anyonecanadmin" , "anyone-can-admin" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0, 0, 0.0, 10001, 0, "-mc-anyonecanadmin", + "supportminerprecheck","", + "Anyone can grant or revoke all permissions."}, + { "supportminerprecheck" , "support-miner-precheck" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 1, 0, 0, 0.0, 10007, 0, "-mc-supportminerprecheck", + "allowp2shoutputs","", + "Require special metadata output with cached scriptPubKey for input, to support advanced miner checks."}, + { "allowp2shoutputs" , "allow-p2sh-outputs" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 1, 0, 0, 0.0, 10001, 0, "-mc-allowp2shoutputs", + "allowmultisigoutputs","", + "Allow pay-to-scripthash (P2SH) scripts, often used for multisig."}, + { "allowmultisigoutputs" , "allow-multisig-outputs" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 1, 0, 0, 0.0, 10001, 0, "-mc-allowmultisigoutputs", + "setupfirstblocks","", + "Allow bare multisignature scripts, rarely used but still supported."}, + + { "setupfirstblocks" , "setup-first-blocks" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE , -1, 60, 1, 31536000, 0.0, 10001, 0, "-mc-setupfirstblocks", + "miningdiversity","Consensus requirements", + "Length of initial setup phase in blocks, in which mining-diversity,\nadmin-consensus-* and mining-requires-peers are not applied."}, + { "miningdiversity" , "mining-diversity" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE | MC_PRM_DECIMAL , -1, 300000, 0, 1000000, 0.0, 10001, 0, "-mc-miningdiversity", + "adminconsensusadmin","", + "Miners must wait * between blocks."}, + { "adminconsensusadmin" , "admin-consensus-admin" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE | MC_PRM_DECIMAL , -1, 500000, 0, 1000000, 0.0, 10001, 0, "-mc-adminconsensusadmin", + "adminconsensusactivate","", + "* needed to change admin perms."}, + { "adminconsensusactivate" , "admin-consensus-activate" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE | MC_PRM_DECIMAL , -1, 500000, 0, 1000000, 0.0, 10003, 0, "-mc-adminconsensusactivate", + "adminconsensusmine","", + "* to change activate perms."}, + { "adminconsensusmine" , "admin-consensus-mine" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE | MC_PRM_DECIMAL , -1, 500000, 0, 1000000, 0.0, 10001, 0, "-mc-adminconsensusmine", + "adminconsensuscreate","", + "* to change mining permissions."}, + { "adminconsensuscreate" , "admin-consensus-create" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE | MC_PRM_DECIMAL , -1, 0, 0, 1000000, 0.0, 10006, 0, "-mc-adminconsensuscreate", + "adminconsensusissue","", + "* to change create permissions."}, + { "adminconsensusissue" , "admin-consensus-issue" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE | MC_PRM_DECIMAL , -1, 0, 0, 1000000, 0.0, 10003, 0, "-mc-adminconsensusissue", + "miningrequirespeers","", + "* to change issue permissions."}, + { "miningrequirespeers" , "mining-requires-peers" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE | MC_PRM_NOHASH , -1, 1, 0, 0, 0.0, 10001, 0, "-mc-miningrequirespeers", + "initialblockreward","", + "Default for whether nodes only mine blocks if connected to other nodes."}, +/* + { "minimumblocktxs" , "minimum-block-txs" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE | MC_PRM_NOHASH , -1, 0, 0, 1000, 0.0, 10007, 0, "-mc-minimumblocktxs", + "initialblockreward","", + "Recommended minimal number of transactions in block."}, +*/ + { "firstblockreward" , "first-block-reward" , + MC_PRM_INT64 | MC_PRM_USER | MC_PRM_CLONE , -1, -1, -1,1000000000000000000, 0.0, 10002, 0, "-mc-genesisblockreward", + "rewardhalvinginterval","", + "Different mining reward for first block only, ignored if negative."}, + { "initialblockreward" , "initial-block-reward" , + MC_PRM_INT64 | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0,1000000000000000000, 0.0, 10001, 0, "-mc-initialblockreward", + "firstblockreward","Native blockchain currency (likely not required)", + "Initial block mining reward in raw native currency units."}, + { "rewardhalvinginterval" , "reward-halving-interval" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE , -1, 52560000, 60,4294967295U, 0.0, 10001, 0, "-mc-rewardhalvinginterval", + "rewardspendabledelay","", + "Interval for halving of mining rewards, in blocks."}, + { "rewardspendabledelay" , "reward-spendable-delay" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE , -1, 1, 1, 100000, 0.0, 10001, 0, "-mc-rewardspendabledelay", + "minimumperoutput","", + "Delay before mining reward can be spent, in blocks."}, + { "minimumperoutput" , "minimum-per-output" , + MC_PRM_INT64 | MC_PRM_USER | MC_PRM_CLONE , -1, 0, -1,1000000000, 0.0, 10001, 0, "-mc-minimumperoutput", + "maximumperoutput","", + "Minimum native currency per output (anti-dust), in raw units.\nIf set to -1, this is calculated from minimum-relay-fee."}, + { "maximumperoutput" , "maximum-per-output" , + MC_PRM_INT64 | MC_PRM_USER | MC_PRM_CLONE , -1, 100000000000000, 0,1000000000000000000, 0.0, 10001, 0, "-mc-maximumperoutput", + "minimumrelayfee","", + "Maximum native currency per output, in raw units."}, + { "minimumrelayfee" , "minimum-relay-fee" , + MC_PRM_INT64 | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0,1000000000, 0.0, 10001, 0, "-mc-minimumrelayfee", + "nativecurrencymultiple","", + "Minimum transaction fee, in raw units of native currency."}, + { "nativecurrencymultiple" , "native-currency-multiple" , + MC_PRM_INT64 | MC_PRM_USER | MC_PRM_CLONE , -1, 100000000, 0,1000000000, 0.0, 10001, 0, "-mc-nativecurrencymultiple", + "skippowcheck","", + "Number of raw units of native currency per display unit."}, + + + { "skippowcheck" , "skip-pow-check" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0, 0, 0.0, 10001, 0, "-mc-skippowcheck", + "powminimumbits","Advanced mining parameters", + "Skip checking whether block hashes demonstrate proof of work."}, + { "powminimumbits" , "pow-minimum-bits" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE , -1, 16, 1, 32, 0.0, 10001, 0, "-mc-powminimumbits", + "targetadjustfreq","", + "Initial and minimum proof of work difficulty, in leading zero bits."}, + { "targetadjustfreq" , "target-adjust-freq" , + MC_PRM_INT32 | MC_PRM_USER | MC_PRM_CLONE , -1, -1, -1,4294967295U, 0.0, 10001, 0, "-mc-targetadjustfreq", + "allowmindifficultyblocks","", + "Interval between proof of work difficulty adjustments, in seconds, if negative - never adjusted."}, + { "allowmindifficultyblocks" , "allow-min-difficulty-blocks" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 0, 0, 0, 0.0, 10001, 0, "-mc-allowmindifficultyblocks", + "onlyacceptstdtxs","", + "Allow lower difficulty blocks if none after 2*."}, + + { "onlyacceptstdtxs" , "only-accept-std-txs" , + MC_PRM_BOOLEAN | MC_PRM_USER | MC_PRM_CLONE , -1, 1, 0, 0, 0.0, 10001, 0, "-mc-onlyacceptstdtxs", + "maxstdtxsize","Standard transaction definitions", + "Only accept and relay transactions which qualify as 'standard'."}, + { "maxstdtxsize" , "max-std-tx-size" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE , -1, 4194304, 1024, 10000000, 0.0, 10001, 0, "-mc-maxstdtxsize", + "maxstdopreturnscount","", + "Maximum size of standard transactions, in bytes."}, + { "maxstdopreturnscount" , "max-std-op-returns-count" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE , -1, 10, 0, 1024, 0.0, 10006, 0, "-mc-maxstdopreturnscount", + "maxstdopreturnsize","", + "Maximum number of OP_RETURN metadata outputs in standard transactions."}, + { "maxstdopreturnsize" , "max-std-op-return-size" , + MC_PRM_UINT32 | MC_PRM_USER | MC_PRM_CLONE , -1, 2097152, 0, 8388608, 0.0, 10001, 0, "-mc-maxstdopreturnsize", + "maxstdopdropscount","", + "Maximum size of OP_RETURN metadata in standard transactions, in bytes."}, + { "maxstdopdropscount" , "max-std-op-drops-count" , + MC_PRM_INT64 | MC_PRM_USER | MC_PRM_CLONE , -1, 5, 0, 100, 0.0, 10001, 0, "-mc-maxstdopdropscount", + "maxstdelementsize","", + "Maximum number of OP_DROPs per output in standard transactions."}, + { "maxstdelementsize" , "max-std-element-size" , + MC_PRM_INT64 | MC_PRM_USER | MC_PRM_CLONE , -1, 8192, 128, 32768, 0.0, 10003, 0, "-mc-maxscriptelementsize", + "maxstdopdropsize","", + "Maximum size of data elements in standard transactions, in bytes."}, + + { "maxstdopdropsize" , "max-std-op-drop-size" , + MC_PRM_INT64 | MC_PRM_USER | MC_PRM_CLONE , -1, 64, 0, 32768, 0.0, 10001, 10003, "-mc-maxstdopdropsize", + "chainname","", + "Obsolete. Maximum size of OP_DROP metadata in standard transactions, in bytes."}, + + { "chainname" , "chain-name" , + MC_PRM_STRING | MC_PRM_GENERATED | MC_PRM_MINIMAL , MC_PRM_NETWORK_NAME_MAX_SIZE, 0, 0, 0, 0.0, 10001, 0, "", + "protocolversion","", + "Chain name, used as first argument for multichaind and multichain-cli."}, + { "protocolversion" , "protocol-version" , + MC_PRM_UINT32 | MC_PRM_GENERATED , -1, 0, 0, 0, 0.0, 10001, 0, "", + "networkmessagestart","", + "Protocol version at the moment of blockchain genesis."}, + { "networkmessagestart" , "network-message-start" , + MC_PRM_BINARY | MC_PRM_GENERATED | MC_PRM_MINIMAL , 4, 0, 0, 0, 0.0, 10001, 0, "", + "addresspubkeyhashversion","", + "Magic value sent as the first 4 bytes of every peer-to-peer message."}, + { "addresspubkeyhashversion" , "address-pubkeyhash-version" , + MC_PRM_BINARY | MC_PRM_GENERATED | MC_PRM_MINIMAL , 4, 0, 0, 0, 0.0, 10001, 0, "", + "addressscripthashversion","", + "Version bytes used for pay-to-pubkeyhash addresses."}, + { "addressscripthashversion" , "address-scripthash-version" , + MC_PRM_BINARY | MC_PRM_GENERATED , 4, 0, 0, 0, 0.0, 10001, 0, "", + "privatekeyversion","", + "Version bytes used for pay-to-scripthash addresses."}, + { "privatekeyversion" , "private-key-version" , + MC_PRM_BINARY | MC_PRM_GENERATED , 4, 0, 0, 0, 0.0, 10001, 0, "", + "addresschecksumvalue","", + "Version bytes used for exporting private keys."}, + { "addresschecksumvalue" , "address-checksum-value" , + MC_PRM_BINARY | MC_PRM_GENERATED | MC_PRM_MINIMAL , 4, 0, 0, 0, 0.0, 10001, 0, "", + "genesispubkey","", + "Bytes used for XOR in address checksum calculation."}, + + + { "genesispubkey" , "genesis-pubkey" , + MC_PRM_BINARY | MC_PRM_CALCULATED , 65, 0, 0, 0, 0.0, 10001, 0, "", + "genesisversion","", + "Genesis block coinbase output public key."}, + { "genesisversion" , "genesis-version" , + MC_PRM_UINT32 | MC_PRM_CALCULATED , -1, 1, 0, -1, 0.0, 10001, 0, "", + "genesistimestamp","", + "Genesis block version."}, + { "genesistimestamp" , "genesis-timestamp" , + MC_PRM_UINT32 | MC_PRM_CALCULATED , -1, 0, 0, 0, 0.0, 10001, 0, "", + "genesisnbits","", + "Genesis block timestamp."}, + { "genesisnbits" , "genesis-nbits" , + MC_PRM_UINT32 | MC_PRM_CALCULATED , -1, 0, 0, 0, 0.0, 10001, 0, "", + "genesisnonce","", + "Genesis block difficulty (nBits)."}, + { "genesisnonce" , "genesis-nonce" , + MC_PRM_UINT32 | MC_PRM_CALCULATED , -1, 0, 0, 0, 0.0, 10001, 0, "", + "genesispubkeyhash","", + "Genesis block nonce."}, + { "genesispubkeyhash" , "genesis-pubkey-hash" , + MC_PRM_BINARY | MC_PRM_CALCULATED , 20, 0, 0, 0, 0.0, 10001, 0, "", + "genesisopreturnscript","", + "Genesis block coinbase output public key hash."}, + { "genesisopreturnscript" , "genesis-op-return-script" , + MC_PRM_BINARY | MC_PRM_CALCULATED ,300, 0, 0, 0, 0.0, 10001, 10006, "", + "genesishash","", + "Genesis block coinbase OP_RETURN script."}, + { "genesishash" , "genesis-hash" , + MC_PRM_BINARY | MC_PRM_CALCULATED , 32, 0, 0, 0, 0.0, 10001, 0, "", + "chainparamshash","", + "Genesis block hash."}, + + + { "chainparamshash" , "chain-params-hash" , + MC_PRM_BINARY | MC_PRM_CALCULATED | MC_PRM_NOHASH , 32, 0, 0, 0, 0.0, 10001, 0, "", + "","", + "Hash of blockchain parameters, to prevent accidental changes."}, +}; + + +#endif /* MULTICHAINPARAMLIST_H */ + diff --git a/src/chainparams/params.cpp b/src/chainparams/params.cpp new file mode 100644 index 00000000..7fa1600b --- /dev/null +++ b/src/chainparams/params.cpp @@ -0,0 +1,1765 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" + + +#define MC_PRM_DAT_FILE_LINE_SIZE 39 + +const uint32_t FreePortRangesOver50[]={2644,2744,2870,4244,4324,4374,4754,5744,6264,6446, + 6716,6790,7172,7314,7404,7718,8338,9218,9538,9696}; + +const unsigned char c_DefaultMessageStart[4]={0xfb,0xb4,0xc7,0xde}; + +#include "chainparams/paramlist.h" + + +int mc_OneMultichainParam::IsRelevant(int version) +{ + int ret=1; + + if(m_ProtocolVersion > version) + { + ret=0; + } + + if(m_Removed > 0) + { + if(m_Removed <= version) + { + ret=0; + } + } + + return ret; +} + +void mc_MultichainParams::Zero() +{ + m_lpData = NULL; + m_lpParams = NULL; + m_lpIndex=NULL; + m_lpCoord=NULL; + m_Status=MC_PRM_STATUS_EMPTY; + m_Size=0; + m_IsProtocolMultiChain=1; + m_ProtocolVersion=0; + + m_AssetRefSize=MC_AST_SHORT_TXID_SIZE; +} + +void mc_MultichainParams::Destroy() +{ + if(m_lpData) + { + mc_Delete(m_lpData); + m_lpData = NULL; + } + if(m_lpParams) + { + mc_Delete(m_lpParams); + m_lpParams = NULL; + } + if(m_lpCoord) + { + mc_Delete(m_lpCoord); + m_lpCoord = NULL; + } + if(m_lpIndex) + { + delete m_lpIndex; + m_lpIndex=NULL; + } +} + + +int64_t mc_MultichainParams::GetInt64Param(const char *param) +{ + int size; + void* ptr=GetParam(param,&size); + if(ptr == NULL) + { + if(m_lpIndex == NULL) + { + return -1; + } + int index=m_lpIndex->Get(param); + if(index<0) + { + printf("Parameter not found: %s\n",param); + return -1; + } + + return (m_lpParams+index)->m_DefaultIntegerValue; + } + + return mc_GetLE(ptr,size); +} + + +void* mc_MultichainParams::GetParam(const char *param,int* size) +{ + if(m_lpIndex == NULL) + { + return NULL; + } + int index=m_lpIndex->Get(param); + if(index<0) + { + return NULL; + } + int offset=m_lpCoord[2 * index + 0]; + if(offset<0) + { + return NULL; + } + if(size) + { + *size=m_lpCoord[2 * index + 1]; + } + + return m_lpData+offset; +} + + +int mc_MultichainParams::SetParam(const char *param,const char* value,int size) +{ + int offset; + if(m_lpIndex == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + int index=m_lpIndex->Get(param); + if(index<0) + { + return MC_ERR_INTERNAL_ERROR; + } + + switch((m_lpParams+index)->m_Type & MC_PRM_SOURCE_MASK) + { + case MC_PRM_COMMENT: + case MC_PRM_CALCULATED: + break; + default: + return MC_ERR_INTERNAL_ERROR; + } + + if(m_lpCoord[2 * index + 0] >= 0) + { + return MC_ERR_INTERNAL_ERROR; + } + + offset=m_Size; + + strcpy(m_lpData+offset,param); + offset+=strlen(param)+1; + + if(size) + { + memcpy(m_lpData+offset+MC_PRM_PARAM_SIZE_BYTES,value,size); + } + + mc_PutLE(m_lpData+offset,&size,MC_PRM_PARAM_SIZE_BYTES); + offset+=MC_PRM_PARAM_SIZE_BYTES; + m_lpCoord[2 * index + 0]=offset; + m_lpCoord[2 * index + 1]=size; + offset+=size; + + m_Size=offset; + + return MC_ERR_NOERROR; +} + +int mc_MultichainParams::SetParam(const char *param,int64_t value) +{ + int size; + char buf[8]; + if(m_lpIndex == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + int index=m_lpIndex->Get(param); + if(index<0) + { + return MC_ERR_INTERNAL_ERROR; + } + + size=0; + switch((m_lpParams+index)->m_Type & MC_PRM_DATA_TYPE_MASK) + { + case MC_PRM_BOOLEAN: + size=1; + break; + case MC_PRM_INT32: + case MC_PRM_UINT32: + size=4; + break; + case MC_PRM_INT64: + size=8; + break; + default: + return MC_ERR_INTERNAL_ERROR; + } + + mc_PutLE(buf,&value,size); + return SetParam(param,buf,size); +} + + +void mc_MultichainParams::Init() +{ + int size,max_size,i; + + Destroy(); + + m_lpIndex=new mc_MapStringIndex; + + m_Count=sizeof(MultichainParamArray)/sizeof(mc_OneMultichainParam); + max_size=0; + + for(i=0;im_Type & MC_PRM_DATA_TYPE_MASK) + { + case MC_PRM_BINARY : size=(MultichainParamArray+i)->m_MaxStringSize; break; + case MC_PRM_STRING : size=(MultichainParamArray+i)->m_MaxStringSize+1; break; + case MC_PRM_BOOLEAN : size=1; break; + case MC_PRM_INT32 : size=4; break; + case MC_PRM_INT64 : size=8; break; + case MC_PRM_DOUBLE : size=sizeof(double); break; + case MC_PRM_UINT32 : size=4; break; + } + + max_size+=MC_PRM_MAX_PARAM_NAME_SIZE+1+MC_PRM_PARAM_SIZE_BYTES+size; + } + + m_lpParams=(mc_OneMultichainParam*)mc_New(sizeof(MultichainParamArray)); + m_lpData=(char*)mc_New(max_size); + m_lpCoord=(int*)mc_New(2*m_Count*sizeof(int)); + + memcpy(m_lpParams,MultichainParamArray,sizeof(MultichainParamArray)); + for(i=0;iAdd((m_lpParams+i)->m_Name,i); + m_lpCoord[2 * i + 0]=-1; + } + m_Size=0; + +} + +int mc_MultichainParams::Create(const char* name,int version) +{ + int size,offset,i,set; + mc_OneMultichainParam *param; + char *ptrData; + int num_sets; + uint32_t network_port=MC_DEFAULT_NETWORK_PORT; + uint32_t rpc_port=MC_DEFAULT_RPC_PORT; + + int param_sets[]={MC_PRM_COMMENT, MC_PRM_USER, MC_PRM_GENERATED}; + num_sets=sizeof(param_sets)/sizeof(int); + + Init(); + + mc_RandomSeed(mc_TimeNowAsUInt()); + + offset=0; + + for(set=0;setm_Type & MC_PRM_SOURCE_MASK) == param_sets[set]) + { + strcpy(m_lpData+offset,param->m_Name); + offset+=strlen(param->m_Name)+1; + size=0; + + ptrData=m_lpData+offset+MC_PRM_PARAM_SIZE_BYTES; + + switch(param->m_Type & MC_PRM_SOURCE_MASK) + { + case MC_PRM_COMMENT: + case MC_PRM_USER: + switch((MultichainParamArray+i)->m_Type & MC_PRM_DATA_TYPE_MASK) + { + case MC_PRM_BINARY : + size=0; + break; + case MC_PRM_STRING : + size=1; + if((MultichainParamArray+i)->m_Type & MC_PRM_SPECIAL) + { + if(strcmp(param->m_Name,"chaindescription") == 0) + { + if(strlen(name)+19<=(size_t)(param->m_MaxStringSize)) + { + sprintf(ptrData,"MultiChain %s",name); + } + size=strlen(ptrData)+1; + } + if(strcmp(param->m_Name,"rootstreamname") == 0) + { + sprintf(ptrData,"root"); + size=strlen(ptrData)+1; + } + if(strcmp(param->m_Name,"seednode") == 0) + { + size=1; + } + if(strcmp(param->m_Name,"chainprotocol") == 0) + { + sprintf(ptrData,"multichain"); + size=strlen(ptrData)+1; + } + } + break; + case MC_PRM_BOOLEAN: + size=1; + ptrData[0]=0; + if(param->m_DefaultIntegerValue) + { + ptrData[0]=1; + } + break; + case MC_PRM_INT32: + case MC_PRM_UINT32: + size=4; + mc_PutLE(ptrData,&(param->m_DefaultIntegerValue),4); + break; + case MC_PRM_INT64: + size=8; + mc_PutLE(ptrData,&(param->m_DefaultIntegerValue),8); + break; + case MC_PRM_DOUBLE: + size=8; + *((double*)ptrData)=param->m_DefaultDoubleValue; + break; + } + break; + case MC_PRM_GENERATED: + if(strcmp(param->m_Name,"defaultnetworkport") == 0) + { + size=4; + network_port=mc_RandomInRange(0,sizeof(FreePortRangesOver50)/sizeof(uint32_t)-1); + network_port=FreePortRangesOver50[network_port]; + network_port+=1+2*mc_RandomInRange(0,24); + mc_PutLE(ptrData,&network_port,4); + } + if(strcmp(param->m_Name,"defaultrpcport") == 0) + { + size=4; + rpc_port=network_port-1; + mc_PutLE(ptrData,&rpc_port,4); + } + if(strcmp(param->m_Name,"protocolversion") == 0) + { + size=4; + mc_PutLE(ptrData,&version,4); + } + if(strcmp(param->m_Name,"chainname") == 0) + { + if(strlen(name)>(size_t)(param->m_MaxStringSize)) + { + mc_print("Invalid network name - too long"); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + size=strlen(name)+1; + strcpy(ptrData,name); + } + if(strcmp(param->m_Name,"networkmessagestart") == 0) + { + size=4; + memcpy(ptrData,c_DefaultMessageStart,4); + while(memcmp(ptrData,c_DefaultMessageStart,4) == 0) + { + *((unsigned char*)ptrData+0)=mc_RandomInRange(0xf0,0xff); + *((unsigned char*)ptrData+1)=mc_RandomInRange(0xc0,0xff); + *((unsigned char*)ptrData+2)=mc_RandomInRange(0xc0,0xff); + *((unsigned char*)ptrData+3)=mc_RandomInRange(0xe0,0xff); + } + } + if(strcmp(param->m_Name,"addresspubkeyhashversion") == 0) + { + size=4; + *((unsigned char*)ptrData+0)=0x00; + *((unsigned char*)ptrData+1)=mc_RandomInRange(0x00,0xff); + *((unsigned char*)ptrData+2)=mc_RandomInRange(0x00,0xff); + *((unsigned char*)ptrData+3)=mc_RandomInRange(0x00,0xff); + } + if(strcmp(param->m_Name,"addressscripthashversion") == 0) + { + size=4; + *((unsigned char*)ptrData+0)=0x05; + *((unsigned char*)ptrData+1)=mc_RandomInRange(0x00,0xff); + *((unsigned char*)ptrData+2)=mc_RandomInRange(0x00,0xff); + *((unsigned char*)ptrData+3)=mc_RandomInRange(0x00,0xff); + } + if(strcmp(param->m_Name,"privatekeyversion") == 0) + { + size=4; + *((unsigned char*)ptrData+0)=0x80; + *((unsigned char*)ptrData+1)=mc_RandomInRange(0x00,0xff); + *((unsigned char*)ptrData+2)=mc_RandomInRange(0x00,0xff); + *((unsigned char*)ptrData+3)=mc_RandomInRange(0x00,0xff); + } + if(strcmp(param->m_Name,"addresschecksumvalue") == 0) + { + size=4; + *((unsigned char*)ptrData+0)=mc_RandomInRange(0x00,0xff); + *((unsigned char*)ptrData+1)=mc_RandomInRange(0x00,0xff); + *((unsigned char*)ptrData+2)=mc_RandomInRange(0x00,0xff); + *((unsigned char*)ptrData+3)=mc_RandomInRange(0x00,0xff); + } + break; + } + + mc_PutLE(m_lpData+offset,&size,MC_PRM_PARAM_SIZE_BYTES); + offset+=MC_PRM_PARAM_SIZE_BYTES; + m_lpCoord[2 * i + 0]=offset; + m_lpCoord[2 * i + 1]=size; + offset+=size; + } + param++; + } + + } + + m_Size=offset; + + return MC_ERR_NOERROR; +} + +int mc_MultichainParams::Read(const char* name) +{ + return Read(name,0,NULL,0); +} + +int mc_MultichainParams::Read(const char* name,int argc, char* argv[],int create_version) +{ + mc_MapStringString *mapConfig; + int err; + int size,offset,i,version,len0,len1,len2; + mc_OneMultichainParam *param; + char *ptrData; + const char *ptr; + + if(name == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + + err=MC_ERR_NOERROR; + offset=0; + + mc_MultichainParams *lpDefaultParams; + + lpDefaultParams=NULL; + mapConfig=new mc_MapStringString; + if(argc) + { + err=mc_ReadParamArgs(mapConfig,argc,argv,""); + } + else + { + err=mc_ReadGeneralConfigFile(mapConfig,name,"params",".dat"); + } + + if(err) + { + goto exitlbl; + } + + Init(); + + version=0; + if(mapConfig->Get("protocolversion") != NULL) + { + version=atoi(mapConfig->Get("protocolversion")); + } + if(create_version) + { + version=create_version; + } + + if(version == 0) + { + version=mc_gState->GetProtocolVersion(); + } + + if(argc == 0) + { + if(mapConfig->Get("chainname") == NULL) + { + err=MC_ERR_NOERROR; + goto exitlbl; + } + + + if(strcmp(name,mapConfig->Get("chainname")) != 0) + { + printf("chain-name parameter (%s) doesn't match (%s)\n",mapConfig->Get("chainname"),name); + err=MC_ERR_INVALID_PARAMETER_VALUE; + goto exitlbl; + } + } + + lpDefaultParams = new mc_MultichainParams; + + err=lpDefaultParams->Create(name,version); + + if(err) + { + goto exitlbl; + } + + + param=m_lpParams; + for(i=0;iGet(param->m_Name); + if(ptr) + { + if(strcmp(ptr,"[null]") == 0) + { + ptr=NULL;; + } + } + + + if(ptr) + { + strcpy(m_lpData+offset,param->m_Name); + offset+=strlen(param->m_Name)+1; + size=0; + + ptrData=m_lpData+offset+MC_PRM_PARAM_SIZE_BYTES; + + switch((MultichainParamArray+i)->m_Type & MC_PRM_DATA_TYPE_MASK) + { + case MC_PRM_BINARY : + if(strlen(ptr) % 2) + { + printf("Invalid parameter value for %s - odd length: %d\n",param->m_DisplayName,(int)strlen(ptr)); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + size=strlen(ptr) / 2; + if(size > param->m_MaxStringSize) + { + printf("Invalid parameter value for %s - too long: %d\n",param->m_DisplayName,(int)strlen(ptr)); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + if(mc_HexToBin(ptrData,ptr,size) != size) + { + printf("Invalid parameter value for %s - cannot parse hex string: %s\n",param->m_DisplayName,ptr); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + break; + case MC_PRM_STRING : + size=strlen(ptr); + if(size > param->m_MaxStringSize) + { + printf("Invalid parameter value for %s - too long: %d\n",param->m_DisplayName,(int)strlen(ptr)); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + strcpy(ptrData,ptr); + size++; + break; + case MC_PRM_BOOLEAN: + ptrData[0]=0; + if(strcasecmp(ptr,"true") == 0) + { + ptrData[0]=1; + } + else + { + ptrData[0]=atoi(ptr); + } + size=1; + break; + case MC_PRM_INT32: + size=4; + if(((MultichainParamArray+i)->m_Type & MC_PRM_SOURCE_MASK) == MC_PRM_USER) + { + if(atoll(ptr) > (MultichainParamArray+i)->m_MaxIntegerValue) + { + printf("Invalid parameter value for %s - too high: %s\n",param->m_DisplayName,ptr); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + if(atoll(ptr) < (MultichainParamArray+i)->m_MinIntegerValue) + { + printf("Invalid parameter value for %s - too low: %s\n",param->m_DisplayName,ptr); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + } + if((MultichainParamArray+i)->m_Type & MC_PRM_DECIMAL) + { + *(int32_t*)ptrData=(int32_t)(atof(ptr)*MC_PRM_DECIMAL_GRANULARITY+0.5); + } + else + { + *(int32_t*)ptrData=(int32_t)atol(ptr); + } + break; + case MC_PRM_UINT32: + size=4; + if(((MultichainParamArray+i)->m_Type & MC_PRM_SOURCE_MASK) == MC_PRM_USER) + { + if(atoll(ptr) > (MultichainParamArray+i)->m_MaxIntegerValue) + { + printf("Invalid parameter value for %s - too high: %s\n",param->m_DisplayName,ptr); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + if(atoll(ptr) < (MultichainParamArray+i)->m_MinIntegerValue) + { + printf("Invalid parameter value for %s - too low: %s\n",param->m_DisplayName,ptr); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + } + if((MultichainParamArray+i)->m_Type & MC_PRM_DECIMAL) + { + *(int32_t*)ptrData=(int32_t)(atof(ptr)*MC_PRM_DECIMAL_GRANULARITY+0.5); + } + else + { + *(int32_t*)ptrData=(int32_t)atol(ptr); + } + if(ptr[0]=='-') + { + printf("Invalid parameter value for %s - should be non-negative\n",param->m_DisplayName); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + break; + case MC_PRM_INT64: + if(((MultichainParamArray+i)->m_Type & MC_PRM_SOURCE_MASK) == MC_PRM_USER) + { + if(atoll(ptr) > (MultichainParamArray+i)->m_MaxIntegerValue) + { + printf("Invalid parameter value for %s - too high: %s\n",param->m_DisplayName,ptr); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + if(atoll(ptr) < (MultichainParamArray+i)->m_MinIntegerValue) + { + printf("Invalid parameter value for %s - too low: %s\n",param->m_DisplayName,ptr); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + } + size=8; + *(int64_t*)ptrData=(int64_t)atoll(ptr); + break; + case MC_PRM_DOUBLE: + size=8; + *((double*)ptrData)=atof(ptr); + break; + } + + mc_PutLE(m_lpData+offset,&size,MC_PRM_PARAM_SIZE_BYTES); + offset+=MC_PRM_PARAM_SIZE_BYTES; + m_lpCoord[2 * i + 0]=offset; + m_lpCoord[2 * i + 1]=size; + offset+=size; + } + else + { + if( ((((MultichainParamArray+i)->m_Type & MC_PRM_SOURCE_MASK) != MC_PRM_CALCULATED) && + (((MultichainParamArray+i)->m_Type & MC_PRM_SOURCE_MASK) != MC_PRM_GENERATED)) || + (argc > 0) ) + { + strcpy(m_lpData+offset,param->m_Name); + offset+=strlen(param->m_Name)+1; + size=0; + + ptrData=m_lpData+offset+MC_PRM_PARAM_SIZE_BYTES; + + ptr=(char*)lpDefaultParams->GetParam(param->m_Name,&size); + if(size) + { + memcpy(ptrData,ptr,size); + } + mc_PutLE(m_lpData+offset,&size,MC_PRM_PARAM_SIZE_BYTES); + offset+=MC_PRM_PARAM_SIZE_BYTES; + m_lpCoord[2 * i + 0]=offset; + m_lpCoord[2 * i + 1]=size; + offset+=size; + } + } + + param++; + } + +exitlbl: + + m_Size=offset; + + delete mapConfig; + if(lpDefaultParams) + { + delete lpDefaultParams; + } + if(err == MC_ERR_NOERROR) + { + len0=0; + GetParam("addresspubkeyhashversion",&len0); + len1=0; + GetParam("addressscripthashversion",&len1); + len2=0; + GetParam("privatekeyversion",&len2); + if(len1) // If params.dat is complete + { + if( (len0 != len1) || (len0 != len2) ) + { + printf("address-pubkeyhash-version, address-scripthash-version and private-key-version should have identical length \n"); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + } + } + + return err; +} + +int mc_MultichainParams::Set(const char *name,const char *source,int source_size) +{ + int size,offset,i,j,n; + char *ptrData; + + Init(); + offset=0; + + j=0; + while((j < source_size) && (*(source+j)!=0x00)) + { + n=j; + i=m_lpIndex->Get(source+j); + j+=strlen(source+j)+1; + if(j+MC_PRM_PARAM_SIZE_BYTES <= source_size) + { + size=mc_GetLE((void*)(source+j),MC_PRM_PARAM_SIZE_BYTES); + } + j+=MC_PRM_PARAM_SIZE_BYTES; + if(j+size <= source_size) + { + if(i >= 0) + { + if(m_lpCoord[2 * i + 0] < 0) + { + strcpy(m_lpData+offset,source+n); + offset+=strlen(source+n)+1; + + ptrData=m_lpData+offset+MC_PRM_PARAM_SIZE_BYTES; + if(size>0) + { + memcpy(ptrData,source+j,size); + } + mc_PutLE(m_lpData+offset,&size,MC_PRM_PARAM_SIZE_BYTES); + offset+=MC_PRM_PARAM_SIZE_BYTES; + m_lpCoord[2 * i + 0]=offset; + m_lpCoord[2 * i + 1]=size; + offset+=size; + } + } + j+=size; + } + } + + m_Size=offset; + return MC_ERR_NOERROR; +} + +int mc_MultichainParams::Clone(const char* name, mc_MultichainParams* source) +{ + int err; + int size,offset,i,version; + mc_OneMultichainParam *param; + char *ptrData; + void *ptr; + + version=source->ProtocolVersion(); + if(version == 0) + { + version=mc_gState->GetProtocolVersion(); + } + + mc_MultichainParams *lpDefaultParams; + lpDefaultParams = new mc_MultichainParams; + + err=lpDefaultParams->Create(name,version); + + if(err) + { + delete lpDefaultParams; + return err; + } + + Init(); + offset=0; + + param=m_lpParams; + for(i=0;im_Type & MC_PRM_CLONE) + { + ptr=source->GetParam(param->m_Name,&size); + } + if(ptr == NULL) + { + ptr=lpDefaultParams->GetParam(param->m_Name,&size); + } + if(ptr) + { +// m_lpIndex->Add(param->m_Name,i); + + strcpy(m_lpData+offset,param->m_Name); + offset+=strlen(param->m_Name)+1; + + ptrData=m_lpData+offset+MC_PRM_PARAM_SIZE_BYTES; + if(size) + { + memcpy(ptrData,ptr,size); + } + mc_PutLE(m_lpData+offset,&size,MC_PRM_PARAM_SIZE_BYTES); + offset+=MC_PRM_PARAM_SIZE_BYTES; + m_lpCoord[2 * i + 0]=offset; + m_lpCoord[2 * i + 1]=size; + offset+=size; + } + param++; + } + + m_Size=offset; + delete lpDefaultParams; + + return MC_ERR_NOERROR; +} + +int mc_MultichainParams::CalculateHash(unsigned char *hash) +{ + int i; + int take_it; + mc_SHA256 *hasher; + + hasher=new mc_SHA256; + for(i=0;im_Type & MC_PRM_SOURCE_MASK) == MC_PRM_COMMENT) + { + take_it=0; + } + if((m_lpParams+i)->m_Type & MC_PRM_NOHASH) + { + take_it=0; + } + if((m_lpParams+i)->m_Removed > 0) + { + if((m_lpParams+i)->m_Removed <= (int)GetInt64Param("protocolversion")) + { + take_it=0; + } + } + + if((m_lpParams+i)->IsRelevant((int)GetInt64Param("protocolversion")) == 0) + { + take_it=0; + switch((m_lpParams+i)->m_Type & MC_PRM_DATA_TYPE_MASK) + { + case MC_PRM_BOOLEAN: + case MC_PRM_INT32: + case MC_PRM_UINT32: + case MC_PRM_INT64: + if(GetInt64Param((m_lpParams+i)->m_Name) != (m_lpParams+i)->m_DefaultIntegerValue) + { + take_it=1; + } + break; + case MC_PRM_STRING: + take_it=1; + + if(strcmp((m_lpParams+i)->m_Name,"rootstreamname") == 0) + { + if(strcmp((char*)GetParam((m_lpParams+i)->m_Name,NULL),"root") == 0) + { + take_it=0; + } + } + break; + case MC_PRM_BINARY: + take_it=1; + + if(strcmp((m_lpParams+i)->m_Name,"genesisopreturnscript") == 0) + { + take_it=0; + } + break; + default: + take_it=1; // Not supported + break; + } + } + if(take_it) + { + if(m_lpCoord[2* i + 1] > 0) + { + hasher->Write(m_lpData+m_lpCoord[2* i + 0],m_lpCoord[2* i + 1]); + } + } + } + + hasher->GetHash(hash); + hasher->Reset(); + hasher->Write(hash,32); + hasher->GetHash(hash); + delete hasher; + + return MC_ERR_NOERROR; +} + + +int mc_MultichainParams::Validate() +{ + int i,size,offset; + int isGenerated; + int isMinimal; + int isValid; + unsigned char hash[32]; + char *ptrData; + void *stored_hash; + + m_Status=MC_PRM_STATUS_EMPTY; + + if((m_lpParams == NULL) || (m_Size == 0)) + { + return MC_ERR_NOERROR; + } + + isGenerated=1; + isMinimal=1; + isValid=1; + + offset=m_Size; + + for(i=0;im_Type & MC_PRM_SOURCE_MASK) + { + case MC_PRM_COMMENT: break; + case MC_PRM_USER: + if((m_lpParams+i)->IsRelevant((int)GetInt64Param("protocolversion")) == 0) + { + switch((m_lpParams+i)->m_Type & MC_PRM_DATA_TYPE_MASK) + { + case MC_PRM_BOOLEAN: + case MC_PRM_INT32: + case MC_PRM_UINT32: + case MC_PRM_INT64: + case MC_PRM_DOUBLE: + case MC_PRM_STRING: + + strcpy(m_lpData+offset,(m_lpParams+i)->m_Name); + offset+=strlen((m_lpParams+i)->m_Name)+1; + size=0; + + ptrData=m_lpData+offset+MC_PRM_PARAM_SIZE_BYTES; + + switch((m_lpParams+i)->m_Type & MC_PRM_DATA_TYPE_MASK) + { + case MC_PRM_STRING: + if(strcmp((m_lpParams+i)->m_Name,"rootstreamname") == 0) + { + sprintf(ptrData,"root"); + size=strlen(ptrData)+1; + } + break; + case MC_PRM_BOOLEAN: + size=1; + if((m_lpParams+i)->m_DefaultIntegerValue) + { + ptrData[0]=1; + } + else + { + ptrData[0]=0; + } + break; + case MC_PRM_INT64: + size=8; + mc_PutLE(ptrData,&((m_lpParams+i)->m_DefaultIntegerValue),8); + break; + case MC_PRM_DOUBLE: + size=8; + *((double*)ptrData)=(m_lpParams+i)->m_DefaultDoubleValue; + break; + default: + size=4; + mc_PutLE(ptrData,&((m_lpParams+i)->m_DefaultIntegerValue),4); + break; + } + + mc_PutLE(m_lpData+offset,&size,MC_PRM_PARAM_SIZE_BYTES); + offset+=MC_PRM_PARAM_SIZE_BYTES; + m_lpCoord[2 * i + 0]=offset; + m_lpCoord[2 * i + 1]=size; + offset+=size; + + break; + default: // Not supported + isGenerated=0; + isValid=0; + break; + } + } + else + { + isGenerated=0; + isValid=0; + } + break; + case MC_PRM_GENERATED: + isGenerated=0; + isValid=0; + break; + case MC_PRM_CALCULATED: + if((m_lpParams+i)->IsRelevant((int)GetInt64Param("protocolversion"))) + { + isValid=0; + } + break; + } + if((m_lpParams+i)->m_Type & MC_PRM_MINIMAL) + { + isMinimal=0; + } + } + } + + m_Size=offset; + + if(isValid) + { + m_Status=MC_PRM_STATUS_VALID; + + CalculateHash(hash); + + stored_hash=GetParam("chainparamshash",&size); + if((stored_hash == NULL) || (size != 32)) + { + m_Status=MC_PRM_STATUS_INVALID; + } + else + { + if(IsProtocolMultichain()) + { + if(memcmp(hash,stored_hash,32)) + { + m_Status=MC_PRM_STATUS_INVALID; + } + } + } + } + else + { + if(isGenerated) + { + m_Status=MC_PRM_STATUS_GENERATED; + } + else + { + if(isMinimal) + { + m_Status=MC_PRM_STATUS_MINIMAL; + } + else + { + m_Status=MC_PRM_STATUS_ERROR; + } + } + } + + return MC_ERR_NOERROR; +} + +int mc_MultichainParams::Print(FILE* fileHan) +{ + int i,c,size; + int version; + int header_printed; + int set,chars_remaining; + void *ptr; + char line[MC_PRM_DAT_FILE_LINE_SIZE+1+100]; + char *cptr; + int num_sets; + double d,d1,d2; + + int param_sets[]={MC_PRM_COMMENT, MC_PRM_USER, MC_PRM_GENERATED, MC_PRM_CALCULATED}; + num_sets=sizeof(param_sets)/sizeof(int); + + fprintf(fileHan,"# ==== MultiChain configuration file ====\n\n"); + fprintf(fileHan,"# Created by multichain-util \n"); + + version=ProtocolVersion(); + if(version) + { + fprintf(fileHan,"# Protocol version: %d \n\n",version); + } + else + { + version=mc_gState->m_Features->LastVersionNotSendingProtocolVersionInHandShake(); + } + + switch(m_Status) + { + case MC_PRM_STATUS_EMPTY: + fprintf(fileHan,"# Parameter set is EMPTY \n"); + fprintf(fileHan,"# To join network please run \"multichaind %s@[:]\".\n",Name()); + return MC_ERR_NOERROR; + case MC_PRM_STATUS_ERROR: + fprintf(fileHan,"# This parameter set cannot be used for generating network. \n"); + fprintf(fileHan,"# One of the parameters is invalid. \n"); + fprintf(fileHan,"# Please fix it and rerun multichain-util. \n"); + break; + case MC_PRM_STATUS_MINIMAL: + fprintf(fileHan,"# This parameter set contains MINIMAL number of parameters required for connection to existing network. \n"); + fprintf(fileHan,"# To join network please run \"multichaind %s@[:]\".\n",Name()); + break; + case MC_PRM_STATUS_GENERATED: + fprintf(fileHan,"# This parameter set is properly GENERATED. \n"); + fprintf(fileHan,"# To generate network please run \"multichaind %s\".\n",Name()); + break; + case MC_PRM_STATUS_VALID: + fprintf(fileHan,"# This parameter set is VALID. \n"); + fprintf(fileHan,"# To join network please run \"multichaind %s\".\n",Name()); + break; + } + + for(set=0;set=0) + { + if( (((m_lpParams+i)->m_Type & MC_PRM_SOURCE_MASK) == param_sets[set]) && + ((m_lpParams+i)->IsRelevant(version) > 0)) + { + if(header_printed == 0) + { + fprintf(fileHan,"\n"); + header_printed=1; + switch(param_sets[set]) + { + case MC_PRM_COMMENT: + fprintf(fileHan,"# The following parameters don't influence multichain network configuration. \n"); + fprintf(fileHan,"# They may be edited at any moment. \n"); + break; + case MC_PRM_USER: + if(m_Status == MC_PRM_STATUS_ERROR) + { + fprintf(fileHan,"# The following parameters can be edited to fix errors. \n"); + if(Name()) + { + fprintf(fileHan,"# Please rerun \"multichain-util clone %s \". \n",Name()); + } + } + else + { + if(m_Status == MC_PRM_STATUS_GENERATED) + { + fprintf(fileHan,"# The following parameters can be edited before running multichaind for this chain. \n"); + } + else + { + fprintf(fileHan,"# The following parameters can only be edited if this file is a prototype of another configuration file. \n"); + fprintf(fileHan,"# Please run \"multichain-util clone %s \" to generate new network. \n",Name()); + } + } + break; + case MC_PRM_GENERATED: + fprintf(fileHan,"# The following parameters were generated by multichain-util.\n"); + fprintf(fileHan,"# They SHOULD ONLY BE EDITED IF YOU KNOW WHAT YOU ARE DOING. \n"); + break; + case MC_PRM_CALCULATED: + fprintf(fileHan,"# The following parameters were generated by multichaind.\n"); + fprintf(fileHan,"# They SHOULD NOT BE EDITED. \n"); + break; + } + fprintf(fileHan,"\n"); + } + + if(strlen((m_lpParams+i)->m_Group)) + { + fprintf(fileHan,"\n# %s\n\n",(m_lpParams+i)->m_Group); + } + + chars_remaining=0; + + sprintf(line,"%s = ",(m_lpParams+i)->m_DisplayName); + ptr=GetParam((m_lpParams+i)->m_Name,&size); + if(size == 0) + { + ptr=NULL; + } + else + { + if(((m_lpParams+i)->m_Type & MC_PRM_DATA_TYPE_MASK) == MC_PRM_STRING) + { + if(size == 1) + { + ptr=NULL; + } + } + } + if(ptr) + { + switch((m_lpParams+i)->m_Type & MC_PRM_DATA_TYPE_MASK) + { + case MC_PRM_BINARY: + if(2*size+strlen(line)>MC_PRM_DAT_FILE_LINE_SIZE) + { + fprintf(fileHan,"%s",line); + for(c=0;cMC_PRM_DAT_FILE_LINE_SIZE) + { + fprintf(fileHan,"%s",line); + fprintf(fileHan,"%s",(char*)ptr); + chars_remaining=1; + } + else + { + sprintf(line+strlen(line),"%s",(char*)ptr); + } + break; + case MC_PRM_BOOLEAN: + if(*(char*)ptr) + { + sprintf(line+strlen(line),"true"); + } + else + { + sprintf(line+strlen(line),"false"); + } + break; + case MC_PRM_INT32: + if((m_lpParams+i)->m_Type & MC_PRM_DECIMAL) + { + if(mc_GetLE(ptr,4)) + { + d=((double)mc_GetLE(ptr,4)+0.1)/MC_PRM_DECIMAL_GRANULARITY; + sprintf(line+strlen(line),"%0.6g",d); + } + else + { + d=0; + sprintf(line+strlen(line),"0.0"); + } + } + else + { + sprintf(line+strlen(line),"%d",(int)mc_GetLE(ptr,4)); + } + break; + case MC_PRM_UINT32: + if((m_lpParams+i)->m_Type & MC_PRM_DECIMAL) + { + if(mc_GetLE(ptr,4)) + { + d=((double)mc_GetLE(ptr,4)+0.1)/MC_PRM_DECIMAL_GRANULARITY; + sprintf(line+strlen(line),"%0.6g",d); + } + else + { + d=0; + sprintf(line+strlen(line),"0.0"); + } + } + else + { + sprintf(line+strlen(line),"%ld",mc_GetLE(ptr,4)); + } + break; + case MC_PRM_INT64: + sprintf(line+strlen(line),"%lld",(long long int)mc_GetLE(ptr,8)); + break; + case MC_PRM_DOUBLE: + sprintf(line+strlen(line),"%f",*(double*)ptr); + break; + } + } + else + { + sprintf(line+strlen(line),"[null]"); + } + if(chars_remaining == 0) + { + fprintf(fileHan,"%s",line); + chars_remaining=MC_PRM_DAT_FILE_LINE_SIZE-strlen(line)+1; + } + for(c=0;cm_Description; + while(*cptr) + { + c=0; + + while((c<(int)strlen(cptr)) && (cptr[c]!='\n')) + { + c++; + } + + if(c<(int)strlen(cptr)) + { + cptr[c]=0x00; + fprintf(fileHan,"# %s",cptr); + memset(line,0x20,MC_PRM_DAT_FILE_LINE_SIZE); + line[MC_PRM_DAT_FILE_LINE_SIZE]=0x00; + fprintf(fileHan,"\n%s ",line); + cptr+=c+1; + } + else + { + fprintf(fileHan,"# %s",cptr); + cptr+=c; + } + } + + switch((m_lpParams+i)->m_Type & MC_PRM_DATA_TYPE_MASK) + { + case MC_PRM_INT32: + case MC_PRM_INT64: + case MC_PRM_UINT32: + switch(param_sets[set]) + { + case MC_PRM_COMMENT: + case MC_PRM_USER: + if((m_lpParams+i)->m_MinIntegerValue <= (m_lpParams+i)->m_MaxIntegerValue) + { + if((m_lpParams+i)->m_Type & MC_PRM_DECIMAL) + { + d1=0; + if((m_lpParams+i)->m_MinIntegerValue) + { + d1=((double)((m_lpParams+i)->m_MinIntegerValue)+0.1)/MC_PRM_DECIMAL_GRANULARITY; + } + d2=0; + if((m_lpParams+i)->m_MaxIntegerValue) + { + d2=((double)((m_lpParams+i)->m_MaxIntegerValue)+0.1)/MC_PRM_DECIMAL_GRANULARITY; + } + fprintf(fileHan," (%0.6g - %0.6g)",d1,d2); + } + else + { + fprintf(fileHan," (%ld - %ld)",(m_lpParams+i)->m_MinIntegerValue,(m_lpParams+i)->m_MaxIntegerValue); + } + } + break; + } + break; + } + fprintf(fileHan,"\n"); + } + + if(strlen((m_lpParams+i)->m_Next)) + { + i=m_lpIndex->Get((m_lpParams+i)->m_Next); + } + else + { + i=-1; + } + } + + } + + fprintf(fileHan,"\n"); + return MC_ERR_NOERROR; + +} + +int mc_MultichainParams::Write(int overwrite) +{ + FILE *fileHan; + int create; + char fileName[MC_DCT_DB_MAX_PATH]; + + if(Name() == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + + fileHan=mc_OpenFile(Name(),"params",".dat","r",MC_FOM_RELATIVE_TO_DATADIR); + if(fileHan) + { + mc_CloseFile(fileHan); + if(overwrite) + { + mc_BackupFile(Name(),"params",".dat",MC_FOM_RELATIVE_TO_DATADIR); + } + else + { + mc_GetFullFileName(mc_gState->m_Params->m_Arguments[1],"params", ".dat",MC_FOM_RELATIVE_TO_DATADIR,fileName); + printf("Cannot create chain parameter set, file %s already exists\n",fileName); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + } + + create=MC_FOM_CREATE_DIR; + if(overwrite) + { + create=0; + } + + fileHan=mc_OpenFile(Name(),"params",".dat","w",MC_FOM_RELATIVE_TO_DATADIR | create); + if(fileHan == NULL) + { + mc_GetFullFileName(mc_gState->m_Params->m_Arguments[1],"params", ".dat",MC_FOM_RELATIVE_TO_DATADIR,fileName); + printf("Cannot create chain parameter set, cannot open file %s for writing\n",fileName); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(Print(fileHan)) + { + mc_GetFullFileName(mc_gState->m_Params->m_Arguments[1],"params", ".dat",MC_FOM_RELATIVE_TO_DATADIR,fileName); + printf("Cannot create chain parameter set, write error to file %s\n",fileName); + mc_CloseFile(fileHan); + if(overwrite) + { + mc_RecoverFile(Name(),"params",".dat",MC_FOM_RELATIVE_TO_DATADIR); + } + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + mc_CloseFile(fileHan); + + + return MC_ERR_NOERROR; +} + + + + +const char* mc_MultichainParams::Name() +{ + return (char*)GetParam("chainname",NULL); +} + +const unsigned char* mc_MultichainParams::MessageStart() +{ + return (unsigned char*)GetParam("networkmessagestart",NULL); +} + +const unsigned char* mc_MultichainParams::DefaultMessageStart() +{ + return c_DefaultMessageStart; +} + + +const unsigned char* mc_MultichainParams::AddressVersion() +{ + return (unsigned char*)GetParam("addresspubkeyhashversion",NULL); +} + +const unsigned char* mc_MultichainParams::AddressScriptVersion() +{ + return (unsigned char*)GetParam("addressscripthashversion",NULL); +} + +const unsigned char* mc_MultichainParams::AddressCheckumValue() +{ + return (unsigned char*)GetParam("addresschecksumvalue",NULL); +} + + +int mc_MultichainParams::ProtocolVersion() +{ + if(m_ProtocolVersion) + { + return m_ProtocolVersion; + } + void *ptr=GetParam("protocolversion",NULL); + if(ptr) + { + return mc_GetLE(ptr,4); + } + return 0; +} + +int mc_MultichainParams::IsProtocolMultichain() +{ + return m_IsProtocolMultiChain; +} + + +int mc_Features::ActivatePermission() +{ + int ret=0; + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(protocol) + { + if(protocol >= 10003) + { + ret=1; + } + } + + return ret; +} + +int mc_Features::LastVersionNotSendingProtocolVersionInHandShake() +{ + return 10002; +} + +int mc_Features::VerifySizeOfOpDropElements() +{ + + int ret=0; + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return 1; + } + + if(protocol) + { + if(protocol >= 10003) + { + ret=1; + } + } + + return ret; +} + +int mc_Features::PerEntityPermissions() +{ + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return 0; + } + int ret=0; + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(protocol) + { + if(protocol >= 10004) + { + ret=1; + } + } + + return ret; +} + +int mc_Features::FollowOnIssues() +{ + int ret=0; + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return 0; + } + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + if(protocol) + { + if(protocol >= 10004) + { + ret=1; + } + } + + return ret; +} + +int mc_Features::SpecialParamsInDetailsScript() +{ + int ret=0; + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(protocol) + { + if(protocol >= 10004) + { + ret=1; + } + } + + return ret; +} + +int mc_Features::FixedGrantsInTheSameTx() +{ + int ret=0; + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(protocol) + { + if(protocol >= 10004) + { + ret=1; + } + } + + return ret; +} + +int mc_Features::UnconfirmedMinersCannotMine() +{ + int ret=0; + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(protocol) + { + if(protocol >= 10004) + { + ret=1; + } + } + + return ret; +} + +int mc_Features::Streams() +{ + int ret=0; + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return 0; + } + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(protocol) + { + if(protocol >= 10006) + { + ret=1; + } + } + + return ret; +} + +int mc_Features::OpDropDetailsScripts() +{ + int ret=0; + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return 0; + } + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(protocol) + { + if(protocol >= 10007) + { + ret=1; + } + } + + return ret; +} + +int mc_Features::ShortTxIDAsAssetRef() +{ + int ret=0; + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return 0; + } + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(protocol) + { + if(protocol >= 10007) + { + ret=1; + } + } + + return ret; +} + +int mc_Features::CachedInputScript() +{ + int ret=0; + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return 0; + } + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(protocol) + { + if(protocol >= 10007) + { + ret=1; + } + } + + return ret; +} + +int mc_Features::AnyoneCanReceiveEmpty() +{ + int ret=0; + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return 0; + } + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(protocol) + { + if(protocol >= 10007) + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanreceiveempty")) + { + ret=1; + } + } + } + + return ret; +} + +int mc_Features::FixedIn10007() +{ + int ret=0; + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return 0; + } + int protocol=mc_gState->m_NetworkParams->ProtocolVersion(); + + if(protocol) + { + if(protocol >= 10007) + { + ret=1; + } + } + + return ret; +} + + diff --git a/src/chainparams/params.h b/src/chainparams/params.h new file mode 100644 index 00000000..6b481814 --- /dev/null +++ b/src/chainparams/params.h @@ -0,0 +1,134 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAINPARAMS_H +#define MULTICHAINPARAMS_H + +#include "utils/declare.h" + +#define MC_DEFAULT_NETWORK_PORT 8571 +#define MC_DEFAULT_RPC_PORT 8570 + + +#define MC_PRM_MAX_PARAM_NAME_SIZE 31 +#define MC_PRM_MAX_ARG_NAME_SIZE 31 +#define MC_PRM_PARAM_SIZE_BYTES 2 +#define MC_PRM_MAX_DESCRIPTION_SIZE 255 +#define MC_PRM_DECIMAL_GRANULARITY 1000000 + +#define MC_PRM_UNKNOWN 0x00000000 +#define MC_PRM_BINARY 0x00000001 +#define MC_PRM_STRING 0x00000002 +#define MC_PRM_BOOLEAN 0x00000003 +#define MC_PRM_INT32 0x00000004 +#define MC_PRM_INT64 0x00000005 +#define MC_PRM_DOUBLE 0x00000006 +#define MC_PRM_UINT32 0x00000007 +#define MC_PRM_DATA_TYPE_MASK 0x0000000F + +#define MC_PRM_COMMENT 0x00000010 +#define MC_PRM_USER 0x00000020 +#define MC_PRM_GENERATED 0x00000030 +#define MC_PRM_CALCULATED 0x00000040 +#define MC_PRM_SOURCE_MASK 0x000000F0 + +#define MC_PRM_CLONE 0x00010000 +#define MC_PRM_SPECIAL 0x00020000 +#define MC_PRM_NOHASH 0x00040000 +#define MC_PRM_MINIMAL 0x00080000 +#define MC_PRM_DECIMAL 0x00100000 + + +#define MC_PRM_STATUS_EMPTY 0 +#define MC_PRM_STATUS_MINIMAL 1 +#define MC_PRM_STATUS_ERROR 2 +#define MC_PRM_STATUS_GENERATED 3 +#define MC_PRM_STATUS_INVALID 4 +#define MC_PRM_STATUS_VALID 5 + + +typedef struct mc_OneMultichainParam +{ + char m_Name[MC_PRM_MAX_ARG_NAME_SIZE+1]; + char m_DisplayName[MC_PRM_MAX_ARG_NAME_SIZE+1]; + int m_Type; + int m_MaxStringSize; + int64_t m_DefaultIntegerValue; + int64_t m_MinIntegerValue; + int64_t m_MaxIntegerValue; + double m_DefaultDoubleValue; + int m_ProtocolVersion; + int m_Removed; + char m_ArgName[MC_PRM_MAX_PARAM_NAME_SIZE+1]; + char m_Next[MC_PRM_MAX_ARG_NAME_SIZE+1]; + char m_Group[MC_PRM_MAX_DESCRIPTION_SIZE+1]; + char m_Description[MC_PRM_MAX_DESCRIPTION_SIZE+1]; + + int IsRelevant(int version); +} mc_OneMultichainParam; + +typedef struct mc_MultichainParams +{ + char *m_lpData; + mc_MapStringIndex *m_lpIndex; + mc_OneMultichainParam *m_lpParams; + int *m_lpCoord; + int m_Status; + int m_Size; + int m_Count; + int m_IsProtocolMultiChain; + int m_ProtocolVersion; + + int m_AssetRefSize; + + mc_MultichainParams() + { + Zero(); + } + + ~mc_MultichainParams() + { + Destroy(); + } + + void Zero(); // Initializes parameters set object + void Init(); // Initializes parameters set object + void Destroy(); // Destroys parameters set object + + int Create(const char *name,int version); + int Read(const char *name); + int Read(const char* name,int argc, char* argv[],int create_version); + int Clone(const char *name,mc_MultichainParams *source); + int Build(const unsigned char* pubkey,int pubkey_size); + int Validate(); + int CalculateHash(unsigned char *hash); + int Write(int overwrite); + int Print(FILE *); + int SetGlobals(); + int Import(const char *name,const char *source_address); + int Set(const char *name,const char *source,int source_size); + + int FindParam(const char *param); + void* GetParam(const char *param,int* size); + int64_t GetInt64Param(const char *param); + + int SetParam(const char *param,const char* value,int size); + int SetParam(const char *param,int64_t value); + + + const char* Name(); + const unsigned char* DefaultMessageStart(); + const unsigned char* MessageStart(); + const unsigned char* AddressVersion(); + const unsigned char* AddressCheckumValue(); + const unsigned char* AddressScriptVersion(); + int ProtocolVersion(); + int IsProtocolMultichain(); + + + +} mc_MultichainParams; + + +#endif /* MULTICHAINPARAMS_H */ + diff --git a/src/chainparams/state.h b/src/chainparams/state.h new file mode 100644 index 00000000..daedc7a0 --- /dev/null +++ b/src/chainparams/state.h @@ -0,0 +1,222 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAIN_STATE_H +#define MULTICHAIN_STATE_H + +#include "utils/define.h" +#include "chainparams/params.h" +#include "protocol/multichainscript.h" +#include "permissions/permission.h" +#include "entities/asset.h" + +#define MC_FAT_UNKNOWN 1 +#define MC_FAT_COMMAND 2 +#define MC_FAT_NETWORK 3 +#define MC_FAT_NETWORKSEED 4 + +#define MC_NTS_UNCONNECTED 0 +#define MC_NTS_WAITING_FOR_SEED 1 +#define MC_NTS_SEED_READY 2 +#define MC_NTS_NOT_READY 3 +#define MC_NTS_SEED_NO_PARAMS 4 + +#define MC_NPS_NONE 0x00000000 +#define MC_NPS_NETWORK 0x00000001 +#define MC_NPS_INCOMING 0x00000002 +#define MC_NPS_MINING 0x00000004 +#define MC_NPS_ALL 0xFFFFFFFF + +#define MC_WMD_NONE 0x00000000 +#define MC_WMD_TXS 0x00000001 +#define MC_WMD_ADDRESS_TXS 0x00000002 +#define MC_WMD_MAP_TXS 0x00010000 +#define MC_WMD_MODE_MASK 0x00FFFFFF +#define MC_WMD_DEBUG 0x01000000 +#define MC_WMD_AUTOSUBSCRIBE_STREAMS 0x02000000 +#define MC_WMD_AUTOSUBSCRIBE_ASSETS 0x04000000 +#define MC_WMD_AUTO 0x10000000 + + +#ifdef __cplusplus +extern "C" { +#endif + + +typedef struct mc_Params +{ + int m_NumArguments; + char** m_Arguments; + int m_FirstArgumentType; + char m_DataDirNetSpecific[MC_DCT_DB_MAX_PATH]; + char m_DataDir[MC_DCT_DB_MAX_PATH]; + + mc_Params() + { + InitDefaults(); + } + + ~mc_Params() + { + Destroy(); + } + + void InitDefaults() + { + m_NumArguments=0; + m_Arguments=NULL; + m_FirstArgumentType=MC_FAT_UNKNOWN; + m_DataDir[0]=0; + m_DataDirNetSpecific[0]=0; + } + + void Destroy() + { + if(m_Arguments) + { + if(m_Arguments[0]) + { + mc_Delete(m_Arguments[0]); + } + mc_Delete(m_Arguments); + } + } + + void Parse(int argc, const char* const argv[]); + int ReadConfig(const char *network_name); + const char* GetOption(const char* strArg,const char* strDefault); + int64_t GetOption(const char* strArg,int64_t nDefault); + int64_t HasOption(const char* strArg); + + const char *NetworkName(); + const char *SeedNode(); + const char *Command(); + const char *DataDir(); + const char *DataDir(int network_specific,int create); + +} mc_Params; + +typedef struct mc_Features +{ + int ActivatePermission(); + int LastVersionNotSendingProtocolVersionInHandShake(); + int VerifySizeOfOpDropElements(); + int PerEntityPermissions(); + int FollowOnIssues(); + int SpecialParamsInDetailsScript(); + int FixedGrantsInTheSameTx(); + int UnconfirmedMinersCannotMine(); + int Streams(); + int OpDropDetailsScripts(); + int ShortTxIDAsAssetRef(); + int CachedInputScript(); + int AnyoneCanReceiveEmpty(); + int FixedIn10007(); +} mc_Features; + +typedef struct mc_State +{ + mc_State() + { + InitDefaults(); + } + + ~mc_State() + { + Destroy(); + } + + mc_Params *m_Params; + mc_MultichainParams *m_NetworkParams; + mc_Permissions *m_Permissions; + mc_AssetDB *m_Assets; + mc_Features *m_Features; + int m_NetworkState; + uint32_t m_NodePausedState; + uint32_t m_IPv4Address; + uint32_t m_WalletMode; + void *m_pSeedNode; + + mc_Script *m_TmpScript; + mc_Script *m_TmpScript1; + mc_Buffer *m_TmpAssetsOut; + mc_Buffer *m_TmpAssetsIn; + + + void InitDefaults() + { + m_Params=new mc_Params; + m_Features=new mc_Features; + m_NetworkParams=new mc_MultichainParams; + m_Permissions=NULL; + m_Assets=NULL; + m_TmpScript=new mc_Script; + m_TmpScript1=new mc_Script; + m_NetworkState=MC_NTS_UNCONNECTED; + m_NodePausedState=MC_NPS_NONE; + m_IPv4Address=0; + m_WalletMode=0; + m_TmpAssetsOut=new mc_Buffer; + mc_InitABufferMap(m_TmpAssetsOut); + m_TmpAssetsIn=new mc_Buffer; + mc_InitABufferMap(m_TmpAssetsIn); + m_pSeedNode=NULL; + } + + void Destroy() + { + if(m_Params) + { + delete m_Params; + } + if(m_Features) + { + delete m_Features; + } + if(m_Permissions) + { + delete m_Permissions; + } + if(m_Assets) + { + delete m_Assets; + } + if(m_NetworkParams) + { + delete m_NetworkParams; + } + if(m_TmpScript) + { + delete m_TmpScript; + } + if(m_TmpScript1) + { + delete m_TmpScript1; + } + if(m_TmpAssetsOut) + { + delete m_TmpAssetsOut; + } + if(m_TmpAssetsIn) + { + delete m_TmpAssetsIn; + } + } + + const char* GetVersion(); + const char* GetFullVersion(); + int GetProtocolVersion(); + const char* GetSeedNode(); + +} cs_State; + + +#ifdef __cplusplus +} +#endif + + +extern mc_State* mc_gState; + +#endif /* MULTICHAIN_STATE_H */ + diff --git a/src/checkqueue.h b/src/checkqueue.h new file mode 100644 index 00000000..faa7164c --- /dev/null +++ b/src/checkqueue.h @@ -0,0 +1,212 @@ +// Copyright (c) 2012-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_CHECKQUEUE_H +#define BITCOIN_CHECKQUEUE_H + +#include +#include + +#include +#include +#include +#include + +template +class CCheckQueueControl; + +/** + * Queue for verifications that have to be performed. + * The verifications are represented by a type T, which must provide an + * operator(), returning a bool. + * + * One thread (the master) is assumed to push batches of verifications + * onto the queue, where they are processed by N-1 worker threads. When + * the master is done adding work, it temporarily joins the worker pool + * as an N'th worker, until all jobs are done. + */ +template +class CCheckQueue +{ +private: + //! Mutex to protect the inner state + boost::mutex mutex; + + //! Worker threads block on this when out of work + boost::condition_variable condWorker; + + //! Master thread blocks on this when out of work + boost::condition_variable condMaster; + + //! The queue of elements to be processed. + //! As the order of booleans doesn't matter, it is used as a LIFO (stack) + std::vector queue; + + //! The number of workers (including the master) that are idle. + int nIdle; + + //! The total number of workers (including the master). + int nTotal; + + //! The temporary evaluation result. + bool fAllOk; + + /** + * Number of verifications that haven't completed yet. + * This includes elements that are not anymore in queue, but still in + * worker's own batches. + */ + unsigned int nTodo; + + //! Whether we're shutting down. + bool fQuit; + + //! The maximum number of elements to be processed in one batch + unsigned int nBatchSize; + + /** Internal function that does bulk of the verification work. */ + bool Loop(bool fMaster = false) + { + boost::condition_variable& cond = fMaster ? condMaster : condWorker; + std::vector vChecks; + vChecks.reserve(nBatchSize); + unsigned int nNow = 0; + bool fOk = true; + do { + { + boost::unique_lock lock(mutex); + // first do the clean-up of the previous loop run (allowing us to do it in the same critsect) + if (nNow) { + fAllOk &= fOk; + nTodo -= nNow; + if (nTodo == 0 && !fMaster) + // We processed the last element; inform the master he can exit and return the result + condMaster.notify_one(); + } else { + // first iteration + nTotal++; + } + // logically, the do loop starts here + while (queue.empty()) { + if ((fMaster || fQuit) && nTodo == 0) { + nTotal--; + bool fRet = fAllOk; + // reset the status for new work later + if (fMaster) + fAllOk = true; + // return the current status + return fRet; + } + nIdle++; + cond.wait(lock); // wait + nIdle--; + } + // Decide how many work units to process now. + // * Do not try to do everything at once, but aim for increasingly smaller batches so + // all workers finish approximately simultaneously. + // * Try to account for idle jobs which will instantly start helping. + // * Don't do batches smaller than 1 (duh), or larger than nBatchSize. + nNow = std::max(1U, std::min(nBatchSize, (unsigned int)queue.size() / (nTotal + nIdle + 1))); + vChecks.resize(nNow); + for (unsigned int i = 0; i < nNow; i++) { + // We want the lock on the mutex to be as short as possible, so swap jobs from the global + // queue to the local batch vector instead of copying. + vChecks[i].swap(queue.back()); + queue.pop_back(); + } + // Check whether we need to do work at all + fOk = fAllOk; + } + // execute work + BOOST_FOREACH (T& check, vChecks) + if (fOk) + fOk = check(); + vChecks.clear(); + } while (true); + } + +public: + //! Create a new check queue + CCheckQueue(unsigned int nBatchSizeIn) : nIdle(0), nTotal(0), fAllOk(true), nTodo(0), fQuit(false), nBatchSize(nBatchSizeIn) {} + + //! Worker thread + void Thread() + { + Loop(); + } + + //! Wait until execution finishes, and return whether all evaluations where successful. + bool Wait() + { + return Loop(true); + } + + //! Add a batch of checks to the queue + void Add(std::vector& vChecks) + { + boost::unique_lock lock(mutex); + BOOST_FOREACH (T& check, vChecks) { + queue.push_back(T()); + check.swap(queue.back()); + } + nTodo += vChecks.size(); + if (vChecks.size() == 1) + condWorker.notify_one(); + else if (vChecks.size() > 1) + condWorker.notify_all(); + } + + ~CCheckQueue() + { + } + + friend class CCheckQueueControl; +}; + +/** + * RAII-style controller object for a CCheckQueue that guarantees the passed + * queue is finished before continuing. + */ +template +class CCheckQueueControl +{ +private: + CCheckQueue* pqueue; + bool fDone; + +public: + CCheckQueueControl(CCheckQueue* pqueueIn) : pqueue(pqueueIn), fDone(false) + { + // passed queue is supposed to be unused, or NULL + if (pqueue != NULL) { + assert(pqueue->nTotal == pqueue->nIdle); + assert(pqueue->nTodo == 0); + assert(pqueue->fAllOk == true); + } + } + + bool Wait() + { + if (pqueue == NULL) + return true; + bool fRet = pqueue->Wait(); + fDone = true; + return fRet; + } + + void Add(std::vector& vChecks) + { + if (pqueue != NULL) + pqueue->Add(vChecks); + } + + ~CCheckQueueControl() + { + if (!fDone) + Wait(); + } +}; + +#endif // BITCOIN_CHECKQUEUE_H diff --git a/src/compat/glibc_compat.cpp b/src/compat/glibc_compat.cpp new file mode 100644 index 00000000..f149a08c --- /dev/null +++ b/src/compat/glibc_compat.cpp @@ -0,0 +1,29 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#if defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" +#endif + +#include + +#if defined(HAVE_SYS_SELECT_H) +#include +#endif + +// Prior to GLIBC_2.14, memcpy was aliased to memmove. +extern "C" void* memmove(void* a, const void* b, size_t c); +extern "C" void* memcpy(void* a, const void* b, size_t c) +{ + return memmove(a, b, c); +} + +extern "C" void __chk_fail(void) __attribute__((__noreturn__)); +extern "C" FDELT_TYPE __fdelt_warn(FDELT_TYPE a) +{ + if (a >= FD_SETSIZE) + __chk_fail(); + return a / __NFDBITS; +} +extern "C" FDELT_TYPE __fdelt_chk(FDELT_TYPE) __attribute__((weak, alias("__fdelt_warn"))); diff --git a/src/compat/glibc_sanity.cpp b/src/compat/glibc_sanity.cpp new file mode 100644 index 00000000..607e23b5 --- /dev/null +++ b/src/compat/glibc_sanity.cpp @@ -0,0 +1,68 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#if defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" +#endif + +#include + +#if defined(HAVE_SYS_SELECT_H) +#include +#endif + +extern "C" void* memcpy(void* a, const void* b, size_t c); +void* memcpy_int(void* a, const void* b, size_t c) +{ + return memcpy(a, b, c); +} + +namespace +{ +// trigger: Use the memcpy_int wrapper which calls our internal memcpy. +// A direct call to memcpy may be optimized away by the compiler. +// test: Fill an array with a sequence of integers. memcpy to a new empty array. +// Verify that the arrays are equal. Use an odd size to decrease the odds of +// the call being optimized away. +template +bool sanity_test_memcpy() +{ + unsigned int memcpy_test[T]; + unsigned int memcpy_verify[T] = {}; + for (unsigned int i = 0; i != T; ++i) + memcpy_test[i] = i; + + memcpy_int(memcpy_verify, memcpy_test, sizeof(memcpy_test)); + + for (unsigned int i = 0; i != T; ++i) { + if (memcpy_verify[i] != i) + return false; + } + return true; +} + +#if defined(HAVE_SYS_SELECT_H) +// trigger: Call FD_SET to trigger __fdelt_chk. FORTIFY_SOURCE must be defined +// as >0 and optimizations must be set to at least -O2. +// test: Add a file descriptor to an empty fd_set. Verify that it has been +// correctly added. +bool sanity_test_fdelt() +{ + fd_set fds; + FD_ZERO(&fds); + FD_SET(0, &fds); + return FD_ISSET(0, &fds); +} +#endif + +} // anon namespace + +bool glibc_sanity_test() +{ +#if defined(HAVE_SYS_SELECT_H) + if (!sanity_test_fdelt()) + return false; +#endif + return sanity_test_memcpy<1025>(); +} diff --git a/src/compat/glibcxx_compat.cpp b/src/compat/glibcxx_compat.cpp new file mode 100644 index 00000000..e0b4ac51 --- /dev/null +++ b/src/compat/glibcxx_compat.cpp @@ -0,0 +1,94 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include +#include +#include +#include + +#ifndef _GLIBCXX_USE_NOEXCEPT +#define _GLIBCXX_USE_NOEXCEPT throw() +#endif + +namespace std +{ +const char* bad_exception::what() const throw() +{ + return "std::bad_exception"; +} + +const char* bad_cast::what() const throw() +{ + return "std::bad_cast"; +} + +const char* bad_alloc::what() const throw() +{ + return "std::bad_alloc"; +} + +namespace __detail +{ +struct _List_node_base { + void _M_hook(std::__detail::_List_node_base* const __position) throw() __attribute__((used)) + { + _M_next = __position; + _M_prev = __position->_M_prev; + __position->_M_prev->_M_next = this; + __position->_M_prev = this; + } + + void _M_unhook() __attribute__((used)) + { + _List_node_base* const __next_node = _M_next; + _List_node_base* const __prev_node = _M_prev; + __prev_node->_M_next = __next_node; + __next_node->_M_prev = __prev_node; + } + + _List_node_base* _M_next; + _List_node_base* _M_prev; +}; +} // namespace detail + +template ostream& ostream::_M_insert(bool); +template ostream& ostream::_M_insert(long); +template ostream& ostream::_M_insert(double); +template ostream& ostream::_M_insert(unsigned long); +template ostream& ostream::_M_insert(const void*); +template ostream& __ostream_insert(ostream&, const char*, streamsize); +template istream& istream::_M_extract(long&); +template istream& istream::_M_extract(unsigned short&); + +out_of_range::~out_of_range() _GLIBCXX_USE_NOEXCEPT {} + +length_error::~length_error() _GLIBCXX_USE_NOEXCEPT {} + +// Used with permission. +// See: https://github.com/madlib/madlib/commit/c3db418c0d34d6813608f2137fef1012ce03043d + +void ctype::_M_widen_init() const +{ + char __tmp[sizeof(_M_widen)]; + for (unsigned __i = 0; __i < sizeof(_M_widen); ++__i) + __tmp[__i] = __i; + do_widen(__tmp, __tmp + sizeof(__tmp), _M_widen); + + _M_widen_ok = 1; + // Set _M_widen_ok to 2 if memcpy can't be used. + for (unsigned __i = 0; __i < sizeof(_M_widen); ++__i) + if (__tmp[__i] != _M_widen[__i]) { + _M_widen_ok = 2; + break; + } +} + +void __throw_out_of_range_fmt(const char*, ...) __attribute__((__noreturn__)); +void __throw_out_of_range_fmt(const char* err, ...) +{ + // Safe and over-simplified version. Ignore the format and print it as-is. + __throw_out_of_range(err); +} + +} // namespace std diff --git a/src/compat/glibcxx_sanity.cpp b/src/compat/glibcxx_sanity.cpp new file mode 100644 index 00000000..aafa4a6a --- /dev/null +++ b/src/compat/glibcxx_sanity.cpp @@ -0,0 +1,61 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include +#include +#include + +namespace +{ +// trigger: use ctype::widen to trigger ctype::_M_widen_init(). +// test: convert a char from narrow to wide and back. Verify that the result +// matches the original. +bool sanity_test_widen(char testchar) +{ + const std::ctype& test(std::use_facet >(std::locale())); + return test.narrow(test.widen(testchar), 'b') == testchar; +} + +// trigger: use list::push_back and list::pop_back to trigger _M_hook and +// _M_unhook. +// test: Push a sequence of integers into a list. Pop them off and verify that +// they match the original sequence. +bool sanity_test_list(unsigned int size) +{ + std::list test; + for (unsigned int i = 0; i != size; ++i) + test.push_back(i + 1); + + if (test.size() != size) + return false; + + while (!test.empty()) { + if (test.back() != test.size()) + return false; + test.pop_back(); + } + return true; +} + +} // anon namespace + +// trigger: string::at(x) on an empty string to trigger __throw_out_of_range_fmt. +// test: force std::string to throw an out_of_range exception. Verify that +// it's caught correctly. +bool sanity_test_range_fmt() +{ + std::string test; + try { + test.at(1); + } catch (const std::out_of_range&) { + return true; + } catch (...) { + } + return false; +} + +bool glibcxx_sanity_test() +{ + return sanity_test_widen('a') && sanity_test_list(100) && sanity_test_range_fmt(); +} diff --git a/src/compat/sanity.h b/src/compat/sanity.h new file mode 100644 index 00000000..7f5bc1a4 --- /dev/null +++ b/src/compat/sanity.h @@ -0,0 +1,11 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#ifndef BITCOIN_COMPAT_SANITY_H +#define BITCOIN_COMPAT_SANITY_H + +bool glibc_sanity_test(); +bool glibcxx_sanity_test(); + +#endif // BITCOIN_COMPAT_SANITY_H diff --git a/src/compat/strnlen.cpp b/src/compat/strnlen.cpp new file mode 100644 index 00000000..7f3e1598 --- /dev/null +++ b/src/compat/strnlen.cpp @@ -0,0 +1,18 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#if defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" +#endif + +#include + +#if HAVE_DECL_STRNLEN == 0 +size_t strnlen( const char *start, size_t max_len) +{ + const char *end = (const char *)memchr(start, '\0', max_len); + + return end ? (size_t)(end - start) : max_len; +} +#endif // HAVE_DECL_STRNLEN diff --git a/src/config/.empty b/src/config/.empty new file mode 100644 index 00000000..e69de29b diff --git a/src/core/init.cpp b/src/core/init.cpp new file mode 100644 index 00000000..3b96181e --- /dev/null +++ b/src/core/init.cpp @@ -0,0 +1,2176 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2016 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#if defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" +#endif + +#include "core/init.h" + +#include "storage/addrman.h" +#include "structs/amount.h" +#include "chain/checkpoints.h" +#include "compat/sanity.h" +#include "keys/key.h" +#include "core/main.h" +#include "miner/miner.h" +#include "net/net.h" +#include "rpc/rpcserver.h" +#include "script/standard.h" +#include "storage/txdb.h" +#include "ui/ui_interface.h" +#include "utils/util.h" +#include "utils/utilmoneystr.h" +#ifdef ENABLE_WALLET +#include "wallet/db.h" +#include "wallet/wallet.h" +#include "wallet/walletdb.h" +#endif + + +/* MCHN START */ + +#include "structs/base58.h" +#include "multichain/multichain.h" +#include "wallet/wallettxs.h" +std::string BurnAddress(const std::vector& vchVersion); + +/* MCHN END */ + +#include +#include + +#ifndef WIN32 +#include +#endif + +#include +#include +#include +#include +#include +#include + +using namespace boost; +using namespace std; + +#ifdef ENABLE_WALLET +CWallet* pwalletMain = NULL; +mc_WalletTxs* pwalletTxsMain = NULL; +#endif +bool fFeeEstimatesInitialized = false; + +#ifdef WIN32 +// Win32 LevelDB doesn't use filedescriptors, and the ones used for +// accessing block files, don't count towards to fd_set size limit +// anyway. +#define MIN_CORE_FILEDESCRIPTORS 0 +#else +#define MIN_CORE_FILEDESCRIPTORS 150 +#endif + +/** Used to pass flags to the Bind() function */ +enum BindFlags { + BF_NONE = 0, + BF_EXPLICIT = (1U << 0), + BF_REPORT_ERROR = (1U << 1), + BF_WHITELIST = (1U << 2), +}; + +static const char* FEE_ESTIMATES_FILENAME="fee_estimates.dat"; +CClientUIInterface uiInterface; + +////////////////////////////////////////////////////////////////////////////// +// +// Shutdown +// + +// +// Thread management and startup/shutdown: +// +// The network-processing threads are all part of a thread group +// created by AppInit() or the Qt main() function. +// +// A clean exit happens when StartShutdown() or the SIGTERM +// signal handler sets fRequestShutdown, which triggers +// the DetectShutdownThread(), which interrupts the main thread group. +// DetectShutdownThread() then exits, which causes AppInit() to +// continue (it .joins the shutdown thread). +// Shutdown() is then +// called to clean up database connections, and stop other +// threads that should only be stopped after the main network-processing +// threads have exited. +// +// Note that if running -daemon the parent process returns from AppInit2 +// before adding any threads to the threadGroup, so .join_all() returns +// immediately and the parent exits from main(). +// +// Shutdown for Qt is very similar, only it uses a QTimer to detect +// fRequestShutdown getting set, and then does the normal Qt +// shutdown thing. +// + +volatile bool fRequestShutdown = false; +volatile bool fShutdownCompleted = false; + +void StartShutdown() +{ + fRequestShutdown = true; +} +bool ShutdownRequested() +{ + return fRequestShutdown; +} + +class CCoinsViewErrorCatcher : public CCoinsViewBacked +{ +public: + CCoinsViewErrorCatcher(CCoinsView* view) : CCoinsViewBacked(view) {} + bool GetCoins(const uint256 &txid, CCoins &coins) const { + try { + return CCoinsViewBacked::GetCoins(txid, coins); + } catch(const std::runtime_error& e) { + uiInterface.ThreadSafeMessageBox(_("Error reading from database, shutting down."), "", CClientUIInterface::MSG_ERROR); + LogPrintf("Error reading from database: %s\n", e.what()); + // Starting the shutdown sequence and returning false to the caller would be + // interpreted as 'entry not found' (as opposed to unable to read data), and + // could lead to invalid interpration. Just exit immediately, as we can't + // continue anyway, and all writes should be atomic. + abort(); + } + } + // Writes do not need similar protection, as failure to write is handled by the caller. +}; + +static CCoinsViewDB *pcoinsdbview = NULL; +static CCoinsViewErrorCatcher *pcoinscatcher = NULL; +static boost::scoped_ptr globalVerifyHandle; + +void Shutdown() +{ + LogPrintf("%s: In progress...\n", __func__); + static CCriticalSection cs_Shutdown; + TRY_LOCK(cs_Shutdown, lockShutdown); + if (!lockShutdown) + return; + + /// Note: Shutdown() must be able to handle cases in which AppInit2() failed part of the way, + /// for example if the data directory was found to be locked. + /// Be sure that anything that writes files or flushes caches only does this if the respective + /// module was initialized. + RenameThread("bitcoin-shutoff"); + mempool.AddTransactionsUpdated(1); + StopRPCThreads(); +#ifdef ENABLE_WALLET + if (pwalletMain) + bitdb.Flush(false); + GenerateBitcoins(false, NULL, 0); +#endif + StopNode(); + UnregisterNodeSignals(GetNodeSignals()); + + if (fFeeEstimatesInitialized) + { + boost::filesystem::path est_path = GetDataDir() / FEE_ESTIMATES_FILENAME; + CAutoFile est_fileout(fopen(est_path.string().c_str(), "wb"), SER_DISK, CLIENT_VERSION); + if (!est_fileout.IsNull()) + mempool.WriteFeeEstimates(est_fileout); + else + LogPrintf("%s: Failed to write fee estimates to %s\n", __func__, est_path.string()); + fFeeEstimatesInitialized = false; + } + + { + LOCK(cs_main); + if (pcoinsTip != NULL) { + FlushStateToDisk(); + } + delete pcoinsTip; + pcoinsTip = NULL; + delete pcoinscatcher; + pcoinscatcher = NULL; + delete pcoinsdbview; + pcoinsdbview = NULL; + delete pblocktree; + pblocktree = NULL; + } +#ifdef ENABLE_WALLET + if (pwalletMain) + bitdb.Flush(true); +#endif +#ifndef WIN32 + boost::filesystem::remove(GetPidFile()); +#endif + UnregisterAllValidationInterfaces(); +#ifdef ENABLE_WALLET + delete pwalletMain; + pwalletMain = NULL; +/* MCHN START */ + if(pwalletTxsMain) + { + delete pwalletTxsMain; + pwalletTxsMain=NULL; + } +/* MCHN END */ +#endif + globalVerifyHandle.reset(); + ECC_Stop(); + LogPrintf("%s: done\n", __func__); + fShutdownCompleted = true; +} + +/** + * Signal handlers are very limited in what they are allowed to do, so: + */ +void HandleSIGTERM(int) +{ + fRequestShutdown = true; + fShutdownCompleted = false; +#ifdef WIN32 + while(!fShutdownCompleted) + { + MilliSleep(100); + } +#endif +} + +void HandleSIGHUP(int) +{ + fReopenDebugLog = true; +} + +bool static InitError(const std::string &str) +{ + uiInterface.ThreadSafeMessageBox(str, "", CClientUIInterface::MSG_ERROR); + return false; +} + +bool static InitWarning(const std::string &str) +{ + uiInterface.ThreadSafeMessageBox(str, "", CClientUIInterface::MSG_WARNING); + return true; +} + +bool static Bind(const CService &addr, unsigned int flags) { + if (!(flags & BF_EXPLICIT) && IsLimited(addr)) + return false; + std::string strError; + if (!BindListenPort(addr, strError, (flags & BF_WHITELIST) != 0)) { + if (flags & BF_REPORT_ERROR) + return InitError(strError); + return false; + } + return true; +} + +std::string HelpMessage(HelpMessageMode mode) // MCHN +{ + // When adding new options to the categories, please keep and ensure alphabetical ordering. + string strUsage = _("Options:") + "\n"; + strUsage += " -? " + _("This help message") + "\n"; + strUsage += " -alertnotify= " + _("Execute command when a relevant alert is received or we see a really long fork (%s in cmd is replaced by message)") + "\n"; + strUsage += " -blocknotify= " + _("Execute command when the best block changes (%s in cmd is replaced by block hash)") + "\n"; + strUsage += " -checkblocks= " + strprintf(_("How many blocks to check at startup (default: %u, 0 = all)"), 288) + "\n"; + strUsage += " -checklevel= " + strprintf(_("How thorough the block verification of -checkblocks is (0-4, default: %u)"), 3) + "\n"; + strUsage += " -conf= " + strprintf(_("Specify configuration file (default: %s)"), "multichain.conf") + "\n"; + if (mode == HMM_BITCOIND) + { +#if !defined(WIN32) + strUsage += " -daemon " + _("Run in the background as a daemon and accept commands") + "\n"; +#endif + } + strUsage += " -datadir= " + _("Specify data directory") + "\n"; + strUsage += " -dbcache= " + strprintf(_("Set database cache size in megabytes (%d to %d, default: %d)"), nMinDbCache, nMaxDbCache, nDefaultDbCache) + "\n"; + strUsage += " -loadblock= " + _("Imports blocks from external blk000??.dat file") + " " + _("on startup") + "\n"; + strUsage += " -maxorphantx= " + strprintf(_("Keep at most unconnectable transactions in memory (default: %u)"), DEFAULT_MAX_ORPHAN_TRANSACTIONS) + "\n"; + strUsage += " -par= " + strprintf(_("Set the number of script verification threads (%u to %d, 0 = auto, <0 = leave that many cores free, default: %d)"), -(int)boost::thread::hardware_concurrency(), MAX_SCRIPTCHECK_THREADS, DEFAULT_SCRIPTCHECK_THREADS) + "\n"; +#ifndef WIN32 + strUsage += " -pid= " + strprintf(_("Specify pid file (default: %s)"), "multichain.pid") + "\n"; +#endif + strUsage += " -reindex " + _("Rebuild block chain index from current blk000??.dat files") + " " + _("on startup") + "\n"; +#if !defined(WIN32) + strUsage += " -sysperms " + _("Create new files with system default permissions, instead of umask 077 (only effective with disabled wallet functionality)") + "\n"; + strUsage += " -shortoutput " + _("Returns connection string if this node can start or default multichain address otherwise") + "\n"; +#endif +/* MCHN START */ +/* Default was 0 */ + strUsage += " -txindex " + strprintf(_("Maintain a full transaction index, used by the getrawtransaction rpc call (default: %u)"), 1) + "\n"; +/* MCHN END */ + + strUsage += "\n" + _("Connection options:") + "\n"; + strUsage += " -addnode= " + _("Add a node to connect to and attempt to keep the connection open") + "\n"; + strUsage += " -banscore= " + strprintf(_("Threshold for disconnecting misbehaving peers (default: %u)"), 100) + "\n"; + strUsage += " -bantime= " + strprintf(_("Number of seconds to keep misbehaving peers from reconnecting (default: %u)"), 86400) + "\n"; + strUsage += " -bind= " + _("Bind to given address and always listen on it. Use [host]:port notation for IPv6") + "\n"; + strUsage += " -connect= " + _("Connect only to the specified node(s)") + "\n"; + strUsage += " -discover " + _("Discover own IP address (default: 1 when listening and no -externalip)") + "\n"; + strUsage += " -dns " + _("Allow DNS lookups for -addnode, -seednode and -connect") + " " + _("(default: 1)") + "\n"; + strUsage += " -dnsseed " + _("Query for peer addresses via DNS lookup, if low on addresses (default: 1 unless -connect)") + "\n"; + strUsage += " -externalip= " + _("Specify your own public address") + "\n"; + strUsage += " -forcednsseed " + strprintf(_("Always query for peer addresses via DNS lookup (default: %u)"), 0) + "\n"; + strUsage += " -listen " + _("Accept connections from outside (default: 1 if no -proxy or -connect)") + "\n"; + strUsage += " -maxconnections= " + strprintf(_("Maintain at most connections to peers (default: %u)"), 125) + "\n"; + strUsage += " -maxreceivebuffer= " + strprintf(_("Maximum per-connection receive buffer, *1000 bytes (default: %u)"), 5000) + "\n"; + strUsage += " -maxsendbuffer= " + strprintf(_("Maximum per-connection send buffer, *1000 bytes (default: %u)"), 1000) + "\n"; + strUsage += " -onion= " + strprintf(_("Use separate SOCKS5 proxy to reach peers via Tor hidden services (default: %s)"), "-proxy") + "\n"; + strUsage += " -onlynet= " + _("Only connect to nodes in network (ipv4, ipv6 or onion)") + "\n"; + strUsage += " -permitbaremultisig " + strprintf(_("Relay non-P2SH multisig (default: %u)"), 1) + "\n"; + strUsage += " -port= " + strprintf(_("Listen for connections on (default: %u or testnet: %u)"), 8333, 18333) + "\n"; + strUsage += " -proxy= " + _("Connect through SOCKS5 proxy") + "\n"; + strUsage += " -seednode= " + _("Connect to a node to retrieve peer addresses, and disconnect") + "\n"; + strUsage += " -timeout= " + strprintf(_("Specify connection timeout in milliseconds (minimum: 1, default: %d)"), DEFAULT_CONNECT_TIMEOUT) + "\n"; +#ifdef USE_UPNP +#if USE_UPNP + strUsage += " -upnp " + _("Use UPnP to map the listening port (default: 1 when listening)") + "\n"; +#else + strUsage += " -upnp " + strprintf(_("Use UPnP to map the listening port (default: %u)"), 0) + "\n"; +#endif +#endif + strUsage += " -whitebind= " + _("Bind to given address and whitelist peers connecting to it. Use [host]:port notation for IPv6") + "\n"; + strUsage += " -whitelist= " + _("Whitelist peers connecting from the given netmask or IP address. Can be specified multiple times.") + "\n"; + strUsage += " " + _("Whitelisted peers cannot be DoS banned and their transactions are always relayed, even if they are already in the mempool, useful e.g. for a gateway") + "\n"; + +#ifdef ENABLE_WALLET + strUsage += "\n" + _("Wallet options:") + "\n"; + strUsage += " -disablewallet " + _("Do not load the wallet and disable wallet RPC calls") + "\n"; + strUsage += " -keypool= " + strprintf(_("Set key pool size to (default: %u)"), 100) + "\n"; + if (GetBoolArg("-help-debug", false)) + strUsage += " -mintxfee= " + strprintf(_("Fees (in BTC/Kb) smaller than this are considered zero fee for transaction creation (default: %s)"), FormatMoney(CWallet::minTxFee.GetFeePerK())) + "\n"; + strUsage += " -paytxfee= " + strprintf(_("Fee (in BTC/kB) to add to transactions you send (default: %s)"), FormatMoney(payTxFee.GetFeePerK())) + "\n"; + strUsage += " -rescan " + _("Rescan the block chain for missing wallet transactions") + " " + _("on startup") + "\n"; + strUsage += " -salvagewallet " + _("Attempt to recover private keys from a corrupt wallet.dat") + " " + _("on startup") + "\n"; + strUsage += " -sendfreetransactions " + strprintf(_("Send transactions as zero-fee transactions if possible (default: %u)"), 0) + "\n"; + strUsage += " -spendzeroconfchange " + strprintf(_("Spend unconfirmed change when sending transactions (default: %u)"), 1) + "\n"; + strUsage += " -txconfirmtarget= " + strprintf(_("If paytxfee is not set, include enough fee so transactions begin confirmation on average within n blocks (default: %u)"), 1) + "\n"; + strUsage += " -maxtxfee= " + strprintf(_("Maximum total fees to use in a single wallet transaction, setting too low may abort large transactions (default: %s)"), FormatMoney(maxTxFee)) + "\n"; + strUsage += " -upgradewallet " + _("Upgrade wallet to latest format") + " " + _("on startup") + "\n"; + strUsage += " -wallet= " + _("Specify wallet file (within data directory)") + " " + strprintf(_("(default: %s)"), "wallet.dat") + "\n"; + strUsage += " -walletnotify= " + _("Execute command when a wallet transaction changes (%s in cmd is replaced by TxID)") + "\n"; +/* MCHN START */ + strUsage += " -walletdbversion=1|2 " + _("Specify wallet version, 1 - not scalable, 2 (default) - scalable") + "\n"; + strUsage += " -autosubscribe=streams|assets|\"streams,assets\"|\"assets,streams\" " + _("Automatically subscribe to new streams and/or assets") + "\n"; + strUsage += " -maxshowndata= " + strprintf(_("Any piece of data in an OP_RETURN longer than this is curtailed in API outputs and converted into an object (default: %u)"), MAX_OP_RETURN_SHOWN) + "\n"; +/* MCHN END */ + strUsage += " -zapwallettxes= " + _("Delete all wallet transactions and only recover those parts of the blockchain through -rescan on startup") + "\n"; + strUsage += " " + _("(1 = keep tx meta data e.g. account owner and payment request information, 2 = drop tx meta data)") + "\n"; +#endif + + strUsage += "\n" + _("Debugging/Testing options:") + "\n"; + if (GetBoolArg("-help-debug", false)) + { + strUsage += " -checkpoints " + strprintf(_("Only accept block chain matching built-in checkpoints (default: %u)"), 1) + "\n"; + strUsage += " -dblogsize= " + strprintf(_("Flush database activity from memory pool to disk log every megabytes (default: %u)"), 100) + "\n"; + strUsage += " -disablesafemode " + strprintf(_("Disable safemode, override a real safe mode event (default: %u)"), 0) + "\n"; + strUsage += " -testsafemode " + strprintf(_("Force safe mode (default: %u)"), 0) + "\n"; + strUsage += " -dropmessagestest= " + _("Randomly drop 1 of every network messages") + "\n"; + strUsage += " -fuzzmessagestest= " + _("Randomly fuzz 1 of every network messages") + "\n"; + strUsage += " -flushwallet " + strprintf(_("Run a thread to flush wallet periodically (default: %u)"), 1) + "\n"; + strUsage += " -stopafterblockimport " + strprintf(_("Stop running after importing blocks from disk (default: %u)"), 0) + "\n"; + } + strUsage += " -debug= " + strprintf(_("Output debugging information (default: %u, supplying is optional)"), 0) + "\n"; + strUsage += " " + _("If is not supplied, output all debugging information.") + "\n"; + strUsage += " " + _(" can be:"); + strUsage += " addrman, alert, bench, coindb, db, lock, rand, rpc, selectcoins, mempool, net"; // Don't translate these and qt below + if (mode == HMM_BITCOIN_QT) + strUsage += ", qt"; + strUsage += ".\n"; +#ifdef ENABLE_WALLET + strUsage += " -gen " + strprintf(_("Generate coins (default: %u)"), 1) + "\n"; + strUsage += " -genproclimit= " + strprintf(_("Set the number of threads for coin generation if enabled (-1 = all cores, default: %d)"), 1) + "\n"; +#endif + strUsage += " -help-debug " + _("Show all debugging options (usage: --help -help-debug)") + "\n"; + strUsage += " -logips " + strprintf(_("Include IP addresses in debug output (default: %u)"), 0) + "\n"; + strUsage += " -logtimestamps " + strprintf(_("Prepend debug output with timestamp (default: %u)"), 1) + "\n"; + if (GetBoolArg("-help-debug", false)) + { + strUsage += " -limitfreerelay= " + strprintf(_("Continuously rate-limit free transactions to *1000 bytes per minute (default:%u)"), 15) + "\n"; + strUsage += " -relaypriority " + strprintf(_("Require high priority for relaying free or low-fee transactions (default:%u)"), 1) + "\n"; + strUsage += " -maxsigcachesize= " + strprintf(_("Limit size of signature cache to entries (default: %u)"), 50000) + "\n"; + } + strUsage += " -minrelaytxfee= " + strprintf(_("Fees (in BTC/Kb) smaller than this are considered zero fee for relaying (default: %s)"), FormatMoney(::minRelayTxFee.GetFeePerK())) + "\n"; + strUsage += " -printtoconsole " + _("Send trace/debug info to console instead of debug.log file") + "\n"; + if (GetBoolArg("-help-debug", false)) + { + strUsage += " -printpriority " + strprintf(_("Log transaction priority and fee per kB when mining blocks (default: %u)"), 0) + "\n"; + strUsage += " -privdb " + strprintf(_("Sets the DB_PRIVATE flag in the wallet db environment (default: %u)"), 1) + "\n"; +/* MCHN START */ +/* + strUsage += " -regtest " + _("Enter regression test mode, which uses a special chain in which blocks can be solved instantly.") + "\n"; + */ +/* MCHN END */ + strUsage += " " + _("This is intended for regression testing tools and app development.") + "\n"; + strUsage += " " + _("In this mode -genproclimit controls how many blocks are generated immediately.") + "\n"; + } + strUsage += " -shrinkdebugfile " + _("Shrink debug.log file on client startup (default: 1 when no -debug)") + "\n"; +/* MCHN START */ +/* + strUsage += " -testnet " + _("Use the test network") + "\n"; + */ +/* MCHN END */ + + strUsage += "\n" + _("Node relay options:") + "\n"; + strUsage += " -datacarrier " + strprintf(_("Relay and mine data carrier transactions (default: %u)"), 1) + "\n"; + strUsage += " -datacarriersize " + strprintf(_("Maximum size of data in data carrier transactions we relay and mine (default: %u)"), MAX_OP_RETURN_RELAY) + "\n"; + + strUsage += "\n" + _("Block creation options:") + "\n"; + strUsage += " -blockminsize= " + strprintf(_("Set minimum block size in bytes (default: %u)"), 0) + "\n"; + strUsage += " -blockmaxsize= " + strprintf(_("Set maximum block size in bytes (default: %d)"), DEFAULT_BLOCK_MAX_SIZE) + "\n"; + strUsage += " -blockprioritysize= " + strprintf(_("Set maximum size of high-priority/low-fee transactions in bytes (default: %d)"), DEFAULT_BLOCK_PRIORITY_SIZE) + "\n"; + + strUsage += "\n" + _("RPC server options:") + "\n"; + strUsage += " -server " + _("Accept command line and JSON-RPC commands") + "\n"; + strUsage += " -rest " + strprintf(_("Accept public REST requests (default: %u)"), 0) + "\n"; + strUsage += " -rpcbind= " + _("Bind to given address to listen for JSON-RPC connections. Use [host]:port notation for IPv6. This option can be specified multiple times (default: bind to all interfaces)") + "\n"; + strUsage += " -rpcuser= " + _("Username for JSON-RPC connections") + "\n"; + strUsage += " -rpcpassword= " + _("Password for JSON-RPC connections") + "\n"; + strUsage += " -rpcport= " + strprintf(_("Listen for JSON-RPC connections on (default: %u or testnet: %u)"), 8332, 18332) + "\n"; + strUsage += " -rpcallowip= " + _("Allow JSON-RPC connections from specified source. Valid for are a single IP (e.g. 1.2.3.4), a network/netmask (e.g. 1.2.3.4/255.255.255.0) or a network/CIDR (e.g. 1.2.3.4/24). This option can be specified multiple times") + "\n"; + strUsage += " -rpcthreads= " + strprintf(_("Set the number of threads to service RPC calls (default: %d)"), 4) + "\n"; + strUsage += " -rpckeepalive " + strprintf(_("RPC support for HTTP persistent connections (default: %d)"), 1) + "\n"; + + strUsage += "\n" + _("RPC SSL options") + "\n"; + strUsage += " -rpcssl " + _("Use OpenSSL (https) for JSON-RPC connections") + "\n"; + strUsage += " -rpcsslcertificatechainfile= " + strprintf(_("Server certificate file (default: %s)"), "server.cert") + "\n"; + strUsage += " -rpcsslprivatekeyfile= " + strprintf(_("Server private key (default: %s)"), "server.pem") + "\n"; + strUsage += " -rpcsslciphers= " + strprintf(_("Acceptable ciphers (default: %s)"), "TLSv1.2+HIGH:TLSv1+HIGH:!SSLv2:!aNULL:!eNULL:!3DES:@STRENGTH") + "\n"; + + strUsage += "\n" + _("MultiChain runtime parameters") + "\n"; + strUsage += " -handshakelocal=
" + _("Manually override the wallet address which is used for handshaking with other peers in a MultiChain blockchain.") + "\n"; + strUsage += " -hideknownopdrops= " + strprintf(_("Remove recognized MultiChain OP_DROP metadata from the responses to JSON_RPC calls (default: %u)"), 0) + "\n"; + strUsage += " -miningrequirespeers= " + _("If set overrides mining-reqires-peers blockhain setting, values 0/1.") + "\n"; + strUsage += " -shrinkdebugfilesize= " + _("If set debug.log is shrinked to size in range - 5.") + "\n"; + + strUsage += "\n" + _("Wallet optimization options:") + "\n"; + strUsage += " -autocombineminconf " + _("Minimum confirmations for automatically combined outputs, default 1") + "\n"; + strUsage += " -autocombinemininputs " + _("Minimum inputs in automatically created combine transaction, default 50") + "\n"; + strUsage += " -autocombinemaxinputs " + _("Maximum inputs in automatically created combine transaction, default 100") + "\n"; + strUsage += " -autocombinedelay " + _("Minimium delay between two auto-combine transactions, in seconds, default 1") + "\n"; + strUsage += " -autocombinesuspend " + _("Auto-combine transaction delay after listunspent API call, in seconds, default 15") + "\n"; + + + + return strUsage; +} + +std::string LicenseInfo() +{ + return FormatParagraph(_("Copyright (c) Coin Sciences Ltd - www.multichain.com")) + "\n" + + "\n" + + FormatParagraph(_("You are granted a non-exclusive license to use this software for any legal purpose, and to redistribute it unmodified.")) + "\n" + + "\n" + + FormatParagraph(_("The software product under this license is provided free of charge. ")) + "\n" + + "\n" + + FormatParagraph(_("Full terms are shown at: http://www.multichain.com/terms-of-service/")) + + "\n"; + +/* + return FormatParagraph(strprintf(_("Copyright (C) 2009-%i The Bitcoin Core Developers"), COPYRIGHT_YEAR)) + "\n" + + "\n" + + FormatParagraph(_("This is experimental software.")) + "\n" + + "\n" + + FormatParagraph(_("Distributed under the MIT software license, see the accompanying file COPYING or .")) + "\n" + + "\n" + + FormatParagraph(_("This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit and cryptographic software written by Eric Young and UPnP software written by Thomas Bernard.")) + + "\n"; + */ + + +} + +static void BlockNotifyCallback(const uint256& hashNewTip) +{ + std::string strCmd = GetArg("-blocknotify", ""); + + boost::replace_all(strCmd, "%s", hashNewTip.GetHex()); + boost::thread t(runCommand, strCmd); // thread runs free +} + +struct CImportingNow +{ + CImportingNow() { + assert(fImporting == false); + fImporting = true; + } + + ~CImportingNow() { + assert(fImporting == true); + fImporting = false; + } +}; + +void ThreadImport(std::vector vImportFiles) +{ + RenameThread("bitcoin-loadblk"); + + // -reindex + if (fReindex) { + CImportingNow imp; + int nFile = 0; + while (true) { + CDiskBlockPos pos(nFile, 0); + if (!boost::filesystem::exists(GetBlockPosFilename(pos, "blk"))) + break; // No block files left to reindex + FILE *file = OpenBlockFile(pos, true); + if (!file) + break; // This error is logged in OpenBlockFile + LogPrintf("Reindexing block file blk%05u.dat...\n", (unsigned int)nFile); + LoadExternalBlockFile(file, &pos); + nFile++; + } + pblocktree->WriteReindexing(false); + fReindex = false; + LogPrintf("Reindexing finished\n"); + // To avoid ending up in a situation without genesis block, re-try initializing (no-op if reindexing worked): + InitBlockIndex(); + } + + // hardcoded $DATADIR/bootstrap.dat + filesystem::path pathBootstrap = GetDataDir() / "bootstrap.dat"; + if (filesystem::exists(pathBootstrap)) { + FILE *file = fopen(pathBootstrap.string().c_str(), "rb"); + if (file) { + CImportingNow imp; + filesystem::path pathBootstrapOld = GetDataDir() / "bootstrap.dat.old"; + LogPrintf("Importing bootstrap.dat...\n"); + LoadExternalBlockFile(file); + RenameOver(pathBootstrap, pathBootstrapOld); + } else { + LogPrintf("Warning: Could not open bootstrap file %s\n", pathBootstrap.string()); + } + } + + // -loadblock= + BOOST_FOREACH(boost::filesystem::path &path, vImportFiles) { + FILE *file = fopen(path.string().c_str(), "rb"); + if (file) { + CImportingNow imp; + LogPrintf("Importing blocks file %s...\n", path.string()); + LoadExternalBlockFile(file); + } else { + LogPrintf("Warning: Could not open blocks file %s\n", path.string()); + } + } + + if (GetBoolArg("-stopafterblockimport", false)) { + LogPrintf("Stopping after block import\n"); + StartShutdown(); + } +} + +/** Sanity checks + * Ensure that Bitcoin is running in a usable environment with all + * necessary library support. + */ +bool InitSanityCheck(void) +{ + if(!ECC_InitSanityCheck()) { + InitError("Elliptic curve cryptography sanity check failure. Aborting."); + return false; + } + if (!glibc_sanity_test() || !glibcxx_sanity_test()) + return false; + + return true; +} + +/** Initialize bitcoin. + * @pre Parameters should be parsed and config file should be read. + */ +bool AppInit2(boost::thread_group& threadGroup,int OutputPipe) +{ +/* MCHN START */ + char bufOutput[4096]; + size_t bytes_written; +/* MCHN END */ + // ********************************************************* Step 1: setup +#ifdef _MSC_VER + // Turn off Microsoft heap dump noise + _CrtSetReportMode(_CRT_WARN, _CRTDBG_MODE_FILE); + _CrtSetReportFile(_CRT_WARN, CreateFileA("NUL", GENERIC_WRITE, 0, NULL, OPEN_EXISTING, 0, 0)); +#endif +#if _MSC_VER >= 1400 + // Disable confusing "helpful" text message on abort, Ctrl-C + _set_abort_behavior(0, _WRITE_ABORT_MSG | _CALL_REPORTFAULT); +#endif +#ifdef WIN32 + // Enable Data Execution Prevention (DEP) + // Minimum supported OS versions: WinXP SP3, WinVista >= SP1, Win Server 2008 + // A failure is non-critical and needs no further attention! +#ifndef PROCESS_DEP_ENABLE + // We define this here, because GCCs winbase.h limits this to _WIN32_WINNT >= 0x0601 (Windows 7), + // which is not correct. Can be removed, when GCCs winbase.h is fixed! +#define PROCESS_DEP_ENABLE 0x00000001 +#endif + typedef BOOL (WINAPI *PSETPROCDEPPOL)(DWORD); + PSETPROCDEPPOL setProcDEPPol = (PSETPROCDEPPOL)GetProcAddress(GetModuleHandleA("Kernel32.dll"), "SetProcessDEPPolicy"); + if (setProcDEPPol != NULL) setProcDEPPol(PROCESS_DEP_ENABLE); + + // Initialize Windows Sockets + WSADATA wsadata; + int ret = WSAStartup(MAKEWORD(2,2), &wsadata); + if (ret != NO_ERROR || LOBYTE(wsadata.wVersion ) != 2 || HIBYTE(wsadata.wVersion) != 2) + { + return InitError(strprintf("Error: Winsock library failed to start (WSAStartup returned error %d)", ret)); + } +#endif +#ifndef WIN32 + + if (GetBoolArg("-sysperms", false)) { +#ifdef ENABLE_WALLET + if (!GetBoolArg("-disablewallet", false)) + return InitError("Error: -sysperms is not allowed in combination with enabled wallet functionality"); +#endif + } else { + umask(077); + } + + // Clean shutdown on SIGTERM + struct sigaction sa; + sa.sa_handler = HandleSIGTERM; + sigemptyset(&sa.sa_mask); + sa.sa_flags = 0; + sigaction(SIGTERM, &sa, NULL); + sigaction(SIGINT, &sa, NULL); + + // Reopen debug.log on SIGHUP + struct sigaction sa_hup; + sa_hup.sa_handler = HandleSIGHUP; + sigemptyset(&sa_hup.sa_mask); + sa_hup.sa_flags = 0; + sigaction(SIGHUP, &sa_hup, NULL); + +#if defined (__SVR4) && defined (__sun) + // ignore SIGPIPE on Solaris + signal(SIGPIPE, SIG_IGN); +#endif + +#else + + SetConsoleCtrlHandler( (PHANDLER_ROUTINE) HandleSIGTERM, TRUE ); + +#endif + + // ********************************************************* Step 2: parameter interactions + // Set this early so that parameter interactions go to console + fPrintToConsole = GetBoolArg("-printtoconsole", false); + fLogTimestamps = GetBoolArg("-logtimestamps", true); + fLogIPs = GetBoolArg("-logips", false); +/* MCHN START */ + fLogTimeMillis = GetBoolArg("-logtimemillis", false); +/* MCHN END */ + + if (mapArgs.count("-bind") || mapArgs.count("-whitebind")) { + // when specifying an explicit binding address, you want to listen on it + // even when -connect or -proxy is specified + if (SoftSetBoolArg("-listen", true)) + LogPrintf("AppInit2 : parameter interaction: -bind or -whitebind set -> setting -listen=1\n"); + } + + if (mapArgs.count("-connect") && mapMultiArgs["-connect"].size() > 0) { + // when only connecting to trusted nodes, do not seed via DNS, or listen by default + if (SoftSetBoolArg("-dnsseed", false)) + LogPrintf("AppInit2 : parameter interaction: -connect set -> setting -dnsseed=0\n"); + if (SoftSetBoolArg("-listen", false)) + LogPrintf("AppInit2 : parameter interaction: -connect set -> setting -listen=0\n"); + } + + if (mapArgs.count("-proxy")) { + // to protect privacy, do not listen by default if a default proxy server is specified + if (SoftSetBoolArg("-listen", false)) + LogPrintf("AppInit2 : parameter interaction: -proxy set -> setting -listen=0\n"); + // to protect privacy, do not discover addresses by default + if (SoftSetBoolArg("-discover", false)) + LogPrintf("AppInit2 : parameter interaction: -proxy set -> setting -discover=0\n"); + } + + if (!GetBoolArg("-listen", true)) { + // do not map ports or try to retrieve public IP when not listening (pointless) + if (SoftSetBoolArg("-upnp", false)) + LogPrintf("AppInit2 : parameter interaction: -listen=0 -> setting -upnp=0\n"); + if (SoftSetBoolArg("-discover", false)) + LogPrintf("AppInit2 : parameter interaction: -listen=0 -> setting -discover=0\n"); + } + + if (mapArgs.count("-externalip")) { + // if an explicit public IP is specified, do not try to find others + if (SoftSetBoolArg("-discover", false)) + LogPrintf("AppInit2 : parameter interaction: -externalip set -> setting -discover=0\n"); + } + + if (GetBoolArg("-salvagewallet", false)) { + // Rewrite just private keys: rescan to find transactions + if (SoftSetBoolArg("-rescan", true)) + LogPrintf("AppInit2 : parameter interaction: -salvagewallet=1 -> setting -rescan=1\n"); + } + + // -zapwallettx implies a rescan + if (GetBoolArg("-zapwallettxes", false)) { + if (SoftSetBoolArg("-rescan", true)) + LogPrintf("AppInit2 : parameter interaction: -zapwallettxes= -> setting -rescan=1\n"); + } + + // Make sure enough file descriptors are available + int nBind = std::max((int)mapArgs.count("-bind") + (int)mapArgs.count("-whitebind"), 1); + nMaxConnections = GetArg("-maxconnections", 125); + nMaxConnections = std::max(std::min(nMaxConnections, (int)(FD_SETSIZE - nBind - MIN_CORE_FILEDESCRIPTORS)), 0); + int nFD = RaiseFileDescriptorLimit(nMaxConnections + MIN_CORE_FILEDESCRIPTORS); + if (nFD < MIN_CORE_FILEDESCRIPTORS) + return InitError(_("Not enough file descriptors available.")); + if (nFD - MIN_CORE_FILEDESCRIPTORS < nMaxConnections) + nMaxConnections = nFD - MIN_CORE_FILEDESCRIPTORS; + + // ********************************************************* Step 3: parameter-to-internal-flags + + fDebug = !mapMultiArgs["-debug"].empty(); + // Special-case: if -debug=0/-nodebug is set, turn off debugging messages + const vector& categories = mapMultiArgs["-debug"]; + if (GetBoolArg("-nodebug", false) || find(categories.begin(), categories.end(), string("0")) != categories.end()) + fDebug = false; + + // Check for -debugnet + if (GetBoolArg("-debugnet", false)) + InitWarning(_("Warning: Unsupported argument -debugnet ignored, use -debug=net.")); + // Check for -socks - as this is a privacy risk to continue, exit here + if (mapArgs.count("-socks")) + return InitError(_("Error: Unsupported argument -socks found. Setting SOCKS version isn't possible anymore, only SOCKS5 proxies are supported.")); + // Check for -tor - as this is a privacy risk to continue, exit here + if (GetBoolArg("-tor", false)) + return InitError(_("Error: Unsupported argument -tor found, use -onion.")); + + if (GetBoolArg("-benchmark", false)) + InitWarning(_("Warning: Unsupported argument -benchmark ignored, use -debug=bench.")); + + // Checkmempool defaults to true in regtest mode + mempool.setSanityCheck(GetBoolArg("-checkmempool", Params().DefaultCheckMemPool())); + Checkpoints::fEnabled = GetBoolArg("-checkpoints", true); + + // -par=0 means autodetect, but nScriptCheckThreads==0 means no concurrency + nScriptCheckThreads = GetArg("-par", DEFAULT_SCRIPTCHECK_THREADS); + if (nScriptCheckThreads <= 0) + nScriptCheckThreads += boost::thread::hardware_concurrency(); + if (nScriptCheckThreads <= 1) + nScriptCheckThreads = 0; + else if (nScriptCheckThreads > MAX_SCRIPTCHECK_THREADS) + nScriptCheckThreads = MAX_SCRIPTCHECK_THREADS; + + fServer = GetBoolArg("-server", false); +#ifdef ENABLE_WALLET + bool fDisableWallet = GetBoolArg("-disablewallet", false); +#endif + + nConnectTimeout = GetArg("-timeout", DEFAULT_CONNECT_TIMEOUT); + if (nConnectTimeout <= 0) + nConnectTimeout = DEFAULT_CONNECT_TIMEOUT; + + // Continue to put "/P2SH/" in the coinbase to monitor + // BIP16 support. + // This can be removed eventually... + const char* pszP2SH = "/P2SH/"; + COINBASE_FLAGS << std::vector(pszP2SH, pszP2SH+strlen(pszP2SH)); + + // Fee-per-kilobyte amount considered the same as "free" + // If you are mining, be careful setting this: + // if you set it to zero then + // a transaction spammer can cheaply fill blocks using + // 1-satoshi-fee transactions. It should be set above the real + // cost to you of processing a transaction. + +/* MCHN START */ + ::minRelayTxFee = CFeeRate(MIN_RELAY_TX_FEE); + +/* MCHN END */ + if (mapArgs.count("-minrelaytxfee")) + { + CAmount n = 0; + if (ParseMoney(mapArgs["-minrelaytxfee"], n) && n > 0) + ::minRelayTxFee = CFeeRate(n); + else + return InitError(strprintf(_("Invalid amount for -minrelaytxfee=: '%s'"), mapArgs["-minrelaytxfee"])); + } + +#ifdef ENABLE_WALLET + if (mapArgs.count("-mintxfee")) + { + CAmount n = 0; + if (ParseMoney(mapArgs["-mintxfee"], n) && n > 0) + CWallet::minTxFee = CFeeRate(n); + else + return InitError(strprintf(_("Invalid amount for -mintxfee=: '%s'"), mapArgs["-mintxfee"])); + } + if (mapArgs.count("-paytxfee")) + { + CAmount nFeePerK = 0; + if (!ParseMoney(mapArgs["-paytxfee"], nFeePerK)) + return InitError(strprintf(_("Invalid amount for -paytxfee=: '%s'"), mapArgs["-paytxfee"])); + if (nFeePerK > nHighTransactionFeeWarning) + InitWarning(_("Warning: -paytxfee is set very high! This is the transaction fee you will pay if you send a transaction.")); + payTxFee = CFeeRate(nFeePerK, 1000); + if (payTxFee < ::minRelayTxFee) + { + return InitError(strprintf(_("Invalid amount for -paytxfee=: '%s' (must be at least %s)"), + mapArgs["-paytxfee"], ::minRelayTxFee.ToString())); + } + } + if (mapArgs.count("-maxtxfee")) + { + CAmount nMaxFee = 0; + if (!ParseMoney(mapArgs["-maxtxfee"], nMaxFee)) + return InitError(strprintf(_("Invalid amount for -maxtxfee=: '%s'"), mapArgs["-maptxfee"])); + if (nMaxFee > nHighTransactionMaxFeeWarning) + InitWarning(_("Warning: -maxtxfee is set very high! Fees this large could be paid on a single transaction.")); + maxTxFee = nMaxFee; + if (CFeeRate(maxTxFee, 1000) < ::minRelayTxFee) + { + return InitError(strprintf(_("Invalid amount for -maxtxfee=: '%s' (must be at least the minrelay fee of %s to prevent stuck transactions)"), + mapArgs["-maxtxfee"], ::minRelayTxFee.ToString())); + } + } + nTxConfirmTarget = GetArg("-txconfirmtarget", 1); + bSpendZeroConfChange = GetArg("-spendzeroconfchange", true); + fSendFreeTransactions = GetArg("-sendfreetransactions", false); + + std::string strWalletFile = GetArg("-wallet", "wallet.dat"); +#endif // ENABLE_WALLET + + fIsBareMultisigStd = GetArg("-permitbaremultisig", true) != 0; + nMaxDatacarrierBytes = GetArg("-datacarriersize", nMaxDatacarrierBytes); + + // ********************************************************* Step 4: application initialization: dir lock, daemonize, pidfile, debug log + + // Initialize elliptic curve code + ECC_Start(); + globalVerifyHandle.reset(new ECCVerifyHandle()); + + // Sanity check + if (!InitSanityCheck()) + return InitError(_("Initialization sanity check failed. MultiChain Core is shutting down.")); + + std::string strDataDir = GetDataDir().string(); + LogPrint("mchn","mchn: Data directory: %s\n",strDataDir.c_str()); +#ifdef ENABLE_WALLET + // Wallet file must be a plain filename without a directory + if (strWalletFile != boost::filesystem::basename(strWalletFile) + boost::filesystem::extension(strWalletFile)) + return InitError(strprintf(_("Wallet %s resides outside data directory %s"), strWalletFile, strDataDir)); +#endif + // Make sure only a single Bitcoin process is using the data directory. + boost::filesystem::path pathLockFile = GetDataDir() / ".lock"; + FILE* file = fopen(pathLockFile.string().c_str(), "a"); // empty lock file; created if it doesn't exist. + if (file) fclose(file); + +/* MCHN START ifndef */ +// In windows .lock file cannot be removed by remove_all if needed +// Anyway, we don't need this lock as we have permission db lock +#ifndef WIN32 + static boost::interprocess::file_lock lock(pathLockFile.string().c_str()); + if (!lock.try_lock()) + return InitError(strprintf(_("Cannot obtain a lock on data directory %s. MultiChain Core is probably already running."), strDataDir)); +#endif +/* MCHN END */ + +#ifndef WIN32 + CreatePidFile(GetPidFile(), getpid()); +#endif + if (GetBoolArg("-shrinkdebugfile", !fDebug)) + ShrinkDebugFile(); + LogPrintf("\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n"); +// LogPrintf("Bitcoin version %s (%s)\n", FormatFullVersion(), CLIENT_DATE); +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + LogPrintf("MultiChain version %s (%s)\n", mc_gState->GetFullVersion(), CLIENT_DATE); + } + +/* MCHN END */ + LogPrintf("Using OpenSSL version %s\n", SSLeay_version(SSLEAY_VERSION)); +#ifdef ENABLE_WALLET + LogPrintf("Using BerkeleyDB version %s\n", DbEnv::version(0, 0, 0)); +#endif + if (!fLogTimestamps) + LogPrintf("Startup time: %s\n", DateTimeStrFormat("%Y-%m-%d %H:%M:%S", GetTime())); + LogPrintf("Default data directory %s\n", GetDefaultDataDir().string()); + LogPrintf("Using data directory %s\n", strDataDir); + LogPrintf("Using config file %s\n", GetConfigFile().string()); + LogPrintf("Using at most %i connections (%i file descriptors available)\n", nMaxConnections, nFD); + std::ostringstream strErrors; + + LogPrintf("Using %u threads for script verification\n", nScriptCheckThreads); + if (nScriptCheckThreads) { + for (int i=0; im_NetworkParams->m_Status != MC_PRM_STATUS_VALID) + { + pwalletMain = new CWallet(strWalletFile); + DBErrors nLoadWalletRetForBuild = pwalletMain->LoadWallet(fFirstRunForBuild); + + if (nLoadWalletRetForBuild != DB_LOAD_OK) // MCHN-TODO wallet recovery + { + return InitError(_("wallet.dat corrupted. Please remove it and restart.")); + } + + if(!pwalletMain->vchDefaultKey.IsValid()) + { + LogPrintf("mchn: Default key is not found - creating new... \n"); + // Create new keyUser and set as default key + RandAddSeedPerfmon(); + + pwalletMain->SetMinVersion(FEATURE_LATEST); // permanently upgrade the wallet immediately + CPubKey newDefaultKey; + if (pwalletMain->GetKeyFromPool(newDefaultKey)) { + pwalletMain->SetDefaultKey(newDefaultKey); + } + } + } + + fNameLookup = GetBoolArg("-dns", true); + + string seed_error=""; + const char *seed_node; + bool zap_wallet_txs=false; + bool new_wallet_txs=false; + seed_node=mc_gState->GetSeedNode(); + + if((mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_EMPTY) + || (mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_MINIMAL)) + { + if(mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_MINIMAL) + { + InitializeMultiChainParams(); + } + + if(seed_node) + { + if(!GetBoolArg("-shortoutput", false)) + { + sprintf(bufOutput,"Retrieving blockchain parameters from the seed node %s ...\n",seed_node); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + } + + LogPrintf("mchn: Parameter set is not complete - starting paramset discovery thread...\n"); + boost::thread_group seedThreadGroup; + + mc_gState->m_NetworkState=MC_NTS_WAITING_FOR_SEED; + + string seed_ip=seed_node; + string seed_port=""; + stringstream ss(seed_ip); + string tok; + int size; + + if(getline(ss, tok, ':')) + { + seed_ip=tok; + if(getline(ss, tok, ':')) + { + seed_port=tok; + } + } + + if(mc_QuerySeed(seedThreadGroup,seed_node)) + { + if((mc_gState->m_NetworkState == MC_NTS_SEED_READY) || (mc_gState->m_NetworkState == MC_NTS_SEED_NO_PARAMS) ) + { + seed_error="Couldn't disconnect from the seed node, please restart multichaind"; +// return InitError(_("Couldn't disconnect from the seed node, please restart multichaind")); + } + + +// seed_error="Couldn't connect to the seed node"; + if(seed_port.size() == 0) + { + seed_error=strprintf("Couldn't connect to the seed node %s - please specify port number explicitly.",seed_node); + } + else + { + seed_error=strprintf("Couldn't connect to the seed node %s on port %s - please check multichaind is running at that address and that your firewall settings allow incoming connections.", + seed_ip.c_str(),seed_port.c_str()); + } +// return InitError(_("Couldn't connect to the seed node")); + } + + if( (mc_gState->m_NetworkParams->GetParam("protocolversion",&size) != NULL) && + (mc_gState->GetProtocolVersion() < (int)mc_gState->m_NetworkParams->GetInt64Param("protocolversion")) ) + { + seed_error=strprintf("Couldn't connect to the seed node %s on port %s.\n" + "Blockchain was created by multichaind with newer protocol version (%d)\n" + "Please upgrade to the latest version of MultiChain or connect only to blockchains using protocol version %d or earlier.\n", + seed_ip.c_str(),seed_port.c_str(),(int)mc_gState->m_NetworkParams->GetInt64Param("protocolversion"), mc_gState->GetProtocolVersion()); + } + else + { + if(mc_gState->m_NetworkState == MC_NTS_SEED_NO_PARAMS) + { + char fileName[MC_DCT_DB_MAX_PATH]; + mc_GetFullFileName(mc_gState->m_Params->NetworkName(),"params", ".dat",MC_FOM_RELATIVE_TO_DATADIR,fileName); + seed_error=strprintf("Couldn't retrieve blockchain parameters from the seed node %s on port %s.\n" + "For bitcoin protocol blockchains, the file %s must be copied manually from an existing node.", + seed_ip.c_str(),seed_port.c_str(),fileName); + + } + } + LogPrintf("mchn: Exited from paramset discovery thread\n"); + + if(mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_VALID) + { + SelectMultiChainParams(mc_gState->m_Params->NetworkName()); + delete mc_gState->m_Permissions; + mc_gState->m_Permissions= new mc_Permissions; +/* + mc_gState->m_Assets->RemoveFiles(); + delete mc_gState->m_Assets; + mc_gState->m_Assets= new mc_AssetDB(); + */ + if(mc_gState->m_Permissions->Initialize(mc_gState->m_Params->NetworkName(),0)) + { + seed_error="Couldn't initialize permission database with retrieved parameters\n"; + } + } + + } + else + { + if(mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_GENERATED) + { + const unsigned char *pubKey=pwalletMain->vchDefaultKey.begin(); + int pubKeySize=pwalletMain->vchDefaultKey.size(); + + LogPrintf("mchn: Parameter set is new, THIS IS GENESIS NODE - looking for genesis block...\n"); + if(!GetBoolArg("-shortoutput", false)) + { + sprintf(bufOutput,"Looking for genesis block...\n"); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + mc_gState->m_NetworkParams->SetGlobals(); // Needed to update IsProtocolMultichain flag in case of bitcoin + if(mc_gState->m_NetworkParams->Build(pubKey,pubKeySize)) + { +/* + delete pwalletMain; + pwalletMain=NULL; + */ + return InitError(_("Cannot build new blockchain")); + } + LogPrintf("mchn: Genesis block found\n"); + if(!GetBoolArg("-shortoutput", false)) + { + sprintf(bufOutput,"Genesis block found\n\n"); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + + mc_gState->m_NetworkParams->Validate(); + + if(mc_gState->m_NetworkParams->m_Status != MC_PRM_STATUS_VALID) + { +/* + delete pwalletMain; + pwalletMain=NULL; + */ + return InitError(_("Invalid parameter set")); + } + } + else + { + if(seed_node) + { + if(!GetBoolArg("-shortoutput", false)) + { + sprintf(bufOutput,"Chain %s already exists, adding %s to list of peers\n\n",mc_gState->m_NetworkParams->Name(),seed_node); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + } + } + } + + if(mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_VALID) + { + LogPrintf("mchn: Parameter set is valid - initializing blockchain parameters...\n"); + mc_gState->m_NetworkParams->SetGlobals(); + InitializeMultiChainParams(); + + if(GetBoolArg("-reindex", false)) + { + if(mc_gState->m_Features->FollowOnIssues()) + { + mc_RemoveDir(mc_gState->m_Params->NetworkName(),"entities.db"); + mc_RemoveFile(mc_gState->m_Params->NetworkName(),"entities",".dat",MC_FOM_RELATIVE_TO_DATADIR); + } + else + { + mc_RemoveDir(mc_gState->m_Params->NetworkName(),"assets.db"); + mc_RemoveFile(mc_gState->m_Params->NetworkName(),"assets",".dat",MC_FOM_RELATIVE_TO_DATADIR); + } + } + + mc_gState->m_Assets= new mc_AssetDB; + if(mc_gState->m_Assets->Initialize(mc_gState->m_Params->NetworkName(),0)) + { + seed_error=strprintf("ERROR: Couldn't initialize asset database for blockchain %s. Exiting...\n",mc_gState->m_Params->NetworkName()); + return InitError(_(seed_error.c_str())); + } + + int64_t wallet_mode=GetArg("-walletdbversion",0); + bool wallet_mode_valid=false; + if(wallet_mode == 0) + { + mc_gState->m_WalletMode=MC_WMD_AUTO; + wallet_mode_valid=true; + } + if(wallet_mode == MC_TDB_WALLET_VERSION) + { + mc_gState->m_WalletMode=MC_WMD_TXS | MC_WMD_ADDRESS_TXS; + wallet_mode_valid=true; + } + if(wallet_mode == 1) + { + mc_gState->m_WalletMode=MC_WMD_NONE; + wallet_mode_valid=true; + zap_wallet_txs=false; + } + if(wallet_mode == -1) + { + mc_gState->m_WalletMode=MC_WMD_TXS | MC_WMD_ADDRESS_TXS | MC_WMD_MAP_TXS; + wallet_mode_valid=true; + zap_wallet_txs=false; + } + + if(!wallet_mode_valid) + { + return InitError(_("Invalid wallet version, possible values 1 or 2.\n")); + } + + vector vSubscribedEntities; + if(GetBoolArg("-reindex", false) || GetBoolArg("-rescan", false)) + { + if(mc_gState->m_Features->Streams()) + { + pwalletTxsMain=new mc_WalletTxs; + if(pwalletTxsMain->Initialize(mc_gState->m_NetworkParams->Name(),MC_WMD_TXS | MC_WMD_ADDRESS_TXS) == MC_ERR_NOERROR) + { + mc_Buffer *entity_list; + entity_list=pwalletTxsMain->GetEntityList(); + for(int e=0;eGetCount();e++) + { + mc_TxEntityStat *stat; + stat=(mc_TxEntityStat *)entity_list->GetRow(e); + switch(stat->m_Entity.m_EntityType & MC_TET_TYPE_MASK) + { + case MC_TET_PUBKEY_ADDRESS: + case MC_TET_SCRIPT_ADDRESS: + case MC_TET_STREAM: + case MC_TET_STREAM_KEY: + case MC_TET_STREAM_PUBLISHER: + vSubscribedEntities.push_back(stat->m_Entity); + break; + } + } + __US_Sleep(1000); + } + pwalletTxsMain->Destroy(); + delete pwalletTxsMain; + } + mc_RemoveDir(mc_gState->m_Params->NetworkName(),"wallet"); + zap_wallet_txs=true; + } + + + + pwalletTxsMain=new mc_WalletTxs; + mc_TxEntity entity; + boost::filesystem::path pathWallet=GetDataDir() / "wallet"; + + if(mc_gState->m_WalletMode == MC_WMD_NONE) + { + if(boost::filesystem::exists(pathWallet)) + { + return InitError(strprintf("Wallet was created in version 2. To switch to version 1, with worse performance and scalability, run: \nmultichaind %s -walletdbversion=1 -rescan\n",mc_gState->m_NetworkParams->Name())); + } + } + else + { + if(!boost::filesystem::exists(pathWallet)) + { + if((mc_gState->m_Permissions->m_Block >= 0) && !GetBoolArg("-rescan", false)) + { + if(mc_gState->m_WalletMode == MC_WMD_AUTO) + { + mc_gState->m_WalletMode=MC_WMD_NONE; + } + if(mc_gState->m_WalletMode != MC_WMD_NONE) + { + return InitError(strprintf("Wallet was created in version 1. To switch to version 2, with better performance and scalability, run: \nmultichaind %s -walletdbversion=2 -rescan\n",mc_gState->m_NetworkParams->Name())); + } + } + else + { + new_wallet_txs=true; + if(mc_gState->m_WalletMode == MC_WMD_AUTO) + { + mc_gState->m_WalletMode = MC_WMD_TXS | MC_WMD_ADDRESS_TXS; + wallet_mode=MC_TDB_WALLET_VERSION; + } + } + } + + if(mc_gState->m_WalletMode != MC_WMD_NONE) + { + boost::filesystem::create_directories(pathWallet); + if(LogAcceptCategory("walletdump")) + { + mc_gState->m_WalletMode |= MC_WMD_DEBUG; + } + string autosubscribe=GetArg("-autosubscribe","none"); + + if(autosubscribe=="streams") + { + mc_gState->m_WalletMode |= MC_WMD_AUTOSUBSCRIBE_STREAMS; + } + if(autosubscribe=="assets") + { + mc_gState->m_WalletMode |= MC_WMD_AUTOSUBSCRIBE_ASSETS; + } + if( (autosubscribe=="assets,streams") || (autosubscribe=="streams,assets")) + { + mc_gState->m_WalletMode |= MC_WMD_AUTOSUBSCRIBE_STREAMS; + mc_gState->m_WalletMode |= MC_WMD_AUTOSUBSCRIBE_ASSETS; + } + + pwalletTxsMain->Initialize(mc_gState->m_NetworkParams->Name(),mc_gState->m_WalletMode); + + if(mc_gState->m_WalletMode & MC_WMD_AUTO) + { + mc_gState->m_WalletMode=pwalletTxsMain->m_Database->m_DBStat.m_InitMode; + wallet_mode=pwalletTxsMain->m_Database->m_DBStat.m_WalletVersion; + } + if(wallet_mode == -1) + { + wallet_mode=pwalletTxsMain->m_Database->m_DBStat.m_WalletVersion; + } + + if((pwalletTxsMain->m_Database->m_DBStat.m_WalletVersion) != wallet_mode) + { + return InitError(strprintf("Wallet tx database was created with different wallet version (%d). Please restart multichaind with reindex=1 \n",pwalletTxsMain->m_Database->m_DBStat.m_WalletVersion)); + } + + if((pwalletTxsMain->m_Database->m_DBStat.m_InitMode & MC_WMD_MODE_MASK) != (mc_gState->m_WalletMode & MC_WMD_MODE_MASK)) + { + return InitError(strprintf("Wallet tx database was created in different mode (%08X). Please restart multichaind with reindex=1 \n",pwalletTxsMain->m_Database->m_DBStat.m_InitMode)); + } + } + } + LogPrintf("Wallet mode: %08X\n",mc_gState->m_WalletMode); + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + if(GetBoolArg("-zapwallettxes", false) && !GetBoolArg("-reindex", false)) + { + return InitError(_("-zapwallettxes is not supported with scalable wallet.\n")); + } + } +// int addr_status=mc_FindIPv4ServerAddress(); + +/* + if(!GetBoolArg("-shortoutput", false)) + { + printf("New users can connect to this node using\n"); + printf("multichaind %s:%d\n\n",MultichainServerAddress().c_str(),GetListenPort()); + } + else + { + printf("%s:%d\n",MultichainServerAddress().c_str(),GetListenPort()); + } + + int version=mc_gState->m_NetworkParams->GetInt64Param("protocolversion"); + if(version != mc_gState->GetProtocolVersion()) + { + if(!GetBoolArg("-shortoutput", false)) + { + printf("Protocol version %d\n\n",version); + } + } + */ + + if(pwalletMain == NULL) // Opening wallet only after multichain parameters were initizalized + { + pwalletMain = new CWallet(strWalletFile); + DBErrors nLoadWalletRetForBuild = pwalletMain->LoadWallet(fFirstRunForBuild); + + if (nLoadWalletRetForBuild != DB_LOAD_OK) // MCHN-TODO wallet recovery + { + return InitError(_("wallet.dat corrupted. Please remove it and restart.")); + } + + if(!pwalletMain->vchDefaultKey.IsValid()) + { + LogPrintf("mchn: Default key is not found - creating new... \n"); + // Create new keyUser and set as default key + RandAddSeedPerfmon(); + + pwalletMain->SetMinVersion(FEATURE_LATEST); // permanently upgrade the wallet immediately + CPubKey newDefaultKey; + if (pwalletMain->GetKeyFromPool(newDefaultKey)) { + pwalletMain->SetDefaultKey(newDefaultKey); + } + } + } + + if(pwalletMain) + { + if(fFirstRunForBuild || (pwalletMain->mapAddressBook.size() == 0)) + { + if (!pwalletMain->SetAddressBook(pwalletMain->vchDefaultKey.GetID(), "", "receive")) + strErrors << _("Cannot write default address") << "\n"; + } + + if(new_wallet_txs) + { + entity.Zero(); + entity.m_EntityType=MC_TET_WALLET_ALL | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,0); + entity.m_EntityType=MC_TET_WALLET_ALL | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,0); + entity.m_EntityType=MC_TET_WALLET_SPENDABLE | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,0); + entity.m_EntityType=MC_TET_WALLET_SPENDABLE | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,0); + + for(int e=0;e<(int)vSubscribedEntities.size();e++) + { + pwalletTxsMain->AddEntity(&(vSubscribedEntities[e]),MC_EFL_NOT_IN_SYNC); + } + + mc_TxEntityStat entstat; + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, pwalletMain->mapAddressBook) + { + const CBitcoinAddress& address = item.first; + CTxDestination addressRet=address.Get(); + const CKeyID *lpKeyID=boost::get (&addressRet); + const CScriptID *lpScriptID=boost::get (&addressRet); + + entstat.Zero(); + if(lpKeyID) + { + memcpy(entstat.m_Entity.m_EntityID,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + entstat.m_Entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + if(!pwalletTxsMain->FindEntity(&entstat)) + { + pwalletTxsMain->AddEntity(&(entstat.m_Entity),0); + entstat.m_Entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&(entstat.m_Entity),0); + } + } + + if(lpScriptID) + { + memcpy(entstat.m_Entity.m_EntityID,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + entstat.m_Entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + if(!pwalletTxsMain->FindEntity(&entstat)) + { + pwalletTxsMain->AddEntity(&(entstat.m_Entity),0); + entstat.m_Entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&(entstat.m_Entity),0); + } + } + } + + } + + CPubKey pkey; + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_CONNECT)) + { + LogPrint("mchn","mchn: Default connect address: %s\n",CBitcoinAddress(pkey.GetID()).ToString().c_str()); + } + if(pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_SEND)) + { + LogPrint("mchn","mchn: Default send address: %s\n",CBitcoinAddress(pkey.GetID()).ToString().c_str()); + } + if(pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_RECEIVE)) + { + LogPrint("mchn","mchn: Default receive address: %s\n",CBitcoinAddress(pkey.GetID()).ToString().c_str()); + } + if(pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_CREATE)) + { + LogPrint("mchn","mchn: Default create address: %s\n",CBitcoinAddress(pkey.GetID()).ToString().c_str()); + } + if(pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_ISSUE)) + { + LogPrint("mchn","mchn: Default issue address: %s\n",CBitcoinAddress(pkey.GetID()).ToString().c_str()); + } + if(pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_MINE)) + { + LogPrint("mchn","mchn: Default mine address: %s\n",CBitcoinAddress(pkey.GetID()).ToString().c_str()); + } + if(pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_ADMIN)) + { + LogPrint("mchn","mchn: Default admin address: %s\n",CBitcoinAddress(pkey.GetID()).ToString().c_str()); + } + if(mc_gState->m_Features->ActivatePermission()) + { + if(pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_ACTIVATE)) + { + LogPrint("mchn","mchn: Default activate address: %s\n",CBitcoinAddress(pkey.GetID()).ToString().c_str()); + } + } + } + else + { + if(pwalletMain->GetKeyFromAddressBook(pkey,0)) + { + LogPrint("mchn","mchn: Default address: %s\n",CBitcoinAddress(pkey.GetID()).ToString().c_str()); + } + } + delete pwalletMain; + pwalletMain=NULL; + } + + if(GetBoolArg("-reindex", false)) + { + mc_gState->m_Assets->RollBack(-1); + mc_gState->m_Permissions->RollBack(-1); + } + } + else + { + if(mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_MINIMAL) + { + InitializeMultiChainParams(); + + if(seed_node) + { + FILE *fileHan; + + fileHan=mc_OpenFile(mc_gState->m_NetworkParams->Name(),"seed",".dat","w",MC_FOM_RELATIVE_TO_DATADIR); + if(fileHan) + { + fprintf(fileHan,"seed=%s\n",seed_node); + mc_CloseFile(fileHan); + } + +// mc_gState->m_NetworkParams->SetParam("seednode",seed_node,strlen(seed_node)+1); +// mc_gState->m_NetworkParams->Write(1); + } + if(pwalletMain) + { + if(pwalletMain->vchDefaultKey.IsValid()) + { + LogPrintf("mchn: Minimal blockchain parameter set is created, default address: %s\n", + CBitcoinAddress(pwalletMain->vchDefaultKey.GetID()).ToString().c_str()); + if(fFirstRunForBuild) + { + if (!pwalletMain->SetAddressBook(pwalletMain->vchDefaultKey.GetID(), "", "receive")) + strErrors << _("Cannot write default address") << "\n"; + } + + + if(seed_error.size()) + { + sprintf(bufOutput,"\nError: %s\n\n",seed_error.c_str()); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + else + { + if(!GetBoolArg("-shortoutput", false)) + { + sprintf(bufOutput,"Blockchain successfully initialized.\n\n"); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + if(mc_gState->m_Features->ActivatePermission()) + { + sprintf(bufOutput,"Please ask blockchain admin or user having activate permission to let you connect and/or transact:\n"); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + else + { + sprintf(bufOutput,"Please ask blockchain admin to let you connect and/or transact:\n"); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + + sprintf(bufOutput,"multichain-cli %s grant %s connect\n",mc_gState->m_NetworkParams->Name(), + CBitcoinAddress(pwalletMain->vchDefaultKey.GetID()).ToString().c_str()); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + sprintf(bufOutput,"multichain-cli %s grant %s connect,send,receive\n\n",mc_gState->m_NetworkParams->Name(), + CBitcoinAddress(pwalletMain->vchDefaultKey.GetID()).ToString().c_str()); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + else + { + sprintf(bufOutput,"%s\n",CBitcoinAddress(pwalletMain->vchDefaultKey.GetID()).ToString().c_str()); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + } + +/* + delete pwalletMain; + pwalletMain=NULL; + */ + return false; + } + } + } +/* + if(pwalletMain) + { + delete pwalletMain; + pwalletMain=NULL; + } +*/ + if(seed_error.size()) + { + return InitError(_(seed_error.c_str())); + } + return InitError(_("Invalid parameter set")); + } + + mc_gState->m_NodePausedState=GetArg("-paused",0); + LogPrintf("Node paused state is set to %08X\n",mc_gState->m_NodePausedState); + + pwalletMain=NULL; + + if (fServer) + { + uiInterface.InitMessage.connect(SetRPCWarmupStatus); + StartRPCThreads(); + } +/* MCHN END*/ + + +#endif // ENABLE_WALLET + // ********************************************************* Step 6: network initialization + +// RegisterNodeSignals(GetNodeSignals()); + + if (mapArgs.count("-onlynet")) { + std::set nets; + BOOST_FOREACH(std::string snet, mapMultiArgs["-onlynet"]) { + enum Network net = ParseNetwork(snet); + if (net == NET_UNROUTABLE) + return InitError(strprintf(_("Unknown network specified in -onlynet: '%s'"), snet)); + nets.insert(net); + } + for (int n = 0; n < NET_MAX; n++) { + enum Network net = (enum Network)n; + if (!nets.count(net)) + SetLimited(net); + } + } + + if (mapArgs.count("-whitelist")) { + BOOST_FOREACH(const std::string& net, mapMultiArgs["-whitelist"]) { + CSubNet subnet(net); + if (!subnet.IsValid()) + return InitError(strprintf(_("Invalid netmask specified in -whitelist: '%s'"), net)); + CNode::AddWhitelistedRange(subnet); + } + } + + CService addrProxy; + bool fProxy = false; + if (mapArgs.count("-proxy")) { + addrProxy = CService(mapArgs["-proxy"], 9050); + if (!addrProxy.IsValid()) + return InitError(strprintf(_("Invalid -proxy address: '%s'"), mapArgs["-proxy"])); + + SetProxy(NET_IPV4, addrProxy); + SetProxy(NET_IPV6, addrProxy); + SetNameProxy(addrProxy); + fProxy = true; + } + + // -onion can override normal proxy, -noonion disables tor entirely + if (!(mapArgs.count("-onion") && mapArgs["-onion"] == "0") && + (fProxy || mapArgs.count("-onion"))) { + CService addrOnion; + if (!mapArgs.count("-onion")) + addrOnion = addrProxy; + else + addrOnion = CService(mapArgs["-onion"], 9050); + if (!addrOnion.IsValid()) + return InitError(strprintf(_("Invalid -onion address: '%s'"), mapArgs["-onion"])); + SetProxy(NET_TOR, addrOnion); + SetReachable(NET_TOR); + } + + // see Step 2: parameter interactions for more information about these + fListen = GetBoolArg("-listen", DEFAULT_LISTEN); + fDiscover = GetBoolArg("-discover", true); + fNameLookup = GetBoolArg("-dns", true); + + bool fBound = false; + bool fThisBound; + mc_gState->m_IPv4Address=0; + if (fListen) { + if (mapArgs.count("-bind") || mapArgs.count("-whitebind")) { + BOOST_FOREACH(std::string strBind, mapMultiArgs["-bind"]) { + CService addrBind; + if (!Lookup(strBind.c_str(), addrBind, GetListenPort(), false)) + return InitError(strprintf(_("Cannot resolve -bind address: '%s'"), strBind)); +/* MCHN START */ + fThisBound = Bind(addrBind, (BF_EXPLICIT | BF_REPORT_ERROR)); + fBound |= fThisBound; + if(fThisBound) + { + if(mc_gState->m_IPv4Address == 0) + { + mc_SetIPv4ServerAddress(strBind.c_str()); + } + } +/* MCHN END */ + } + BOOST_FOREACH(std::string strBind, mapMultiArgs["-whitebind"]) { + CService addrBind; + if (!Lookup(strBind.c_str(), addrBind, 0, false)) + return InitError(strprintf(_("Cannot resolve -whitebind address: '%s'"), strBind)); + if (addrBind.GetPort() == 0) + return InitError(strprintf(_("Need to specify a port with -whitebind: '%s'"), strBind)); + fBound |= Bind(addrBind, (BF_EXPLICIT | BF_REPORT_ERROR | BF_WHITELIST)); + } + } + else { + struct in_addr inaddr_any; + inaddr_any.s_addr = INADDR_ANY; + fBound |= Bind(CService(in6addr_any, GetListenPort()), BF_NONE); + fBound |= Bind(CService(inaddr_any, GetListenPort()), !fBound ? BF_REPORT_ERROR : BF_NONE); + } + if (!fBound) + return InitError(_("Failed to listen on any port. Use -listen=0 if you want this.")); + } +/* MCHN START */ + int max_ips=64; + uint32_t all_ips[64]; + int found_ips=1; + if(mc_gState->m_IPv4Address == 0) + { + found_ips=mc_FindIPv4ServerAddress(all_ips,max_ips); + } + if(!GetBoolArg("-shortoutput", false)) + { + sprintf(bufOutput,"Other nodes can connect to this node using:\n"); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + sprintf(bufOutput,"multichaind %s:%d\n\n",MultichainServerAddress().c_str(),GetListenPort()); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + if(found_ips > 1) + { + sprintf(bufOutput,"\nThis host has multiple IP addresses, so from some networks:\n"); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + for(int i_ips=0;i_ipsm_IPv4Address) + { + unsigned char *ptr; + ptr=(unsigned char *)(all_ips+i_ips); + sprintf(bufOutput,"multichaind %s@%u.%u.%u.%u:%d\n\n",mc_gState->m_NetworkParams->Name(),ptr[3],ptr[2],ptr[1],ptr[0],GetListenPort()); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + if(bytes_written != strlen(bufOutput)) + { + found_ips=0; + } + } + } + } + } + else + { + sprintf(bufOutput,"%s:%d\n",MultichainServerAddress().c_str(),GetListenPort()); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + + int version=mc_gState->m_NetworkParams->GetInt64Param("protocolversion"); + if(version != mc_gState->GetProtocolVersion()) + { + if(!GetBoolArg("-shortoutput", false)) + { + sprintf(bufOutput,"Protocol version %d\n\n",version); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + } + +/* MCHN END */ + + if (mapArgs.count("-externalip")) { + BOOST_FOREACH(string strAddr, mapMultiArgs["-externalip"]) { + CService addrLocal(strAddr, GetListenPort(), fNameLookup); + if (!addrLocal.IsValid()) + return InitError(strprintf(_("Cannot resolve -externalip address: '%s'"), strAddr)); + AddLocal(CService(strAddr, GetListenPort(), fNameLookup), LOCAL_MANUAL); + } + } + + BOOST_FOREACH(string strDest, mapMultiArgs["-seednode"]) + AddOneShot(strDest); + + // ********************************************************* Step 7: load block chain + + fReindex = GetBoolArg("-reindex", false); + + // Upgrading to 0.8; hard-link the old blknnnn.dat files into /blocks/ + filesystem::path blocksDir = GetDataDir() / "blocks"; + if (!filesystem::exists(blocksDir)) + { + filesystem::create_directories(blocksDir); + bool linked = false; + for (unsigned int i = 1; i < 10000; i++) { + filesystem::path source = GetDataDir() / strprintf("blk%04u.dat", i); + if (!filesystem::exists(source)) break; + filesystem::path dest = blocksDir / strprintf("blk%05u.dat", i-1); + try { + filesystem::create_hard_link(source, dest); + LogPrintf("Hardlinked %s -> %s\n", source.string(), dest.string()); + linked = true; + } catch (filesystem::filesystem_error & e) { + // Note: hardlink creation failing is not a disaster, it just means + // blocks will get re-downloaded from peers. + LogPrintf("Error hardlinking blk%04u.dat : %s\n", i, e.what()); + break; + } + } + if (linked) + { + fReindex = true; + } + } + + // cache size calculations + size_t nTotalCache = (GetArg("-dbcache", nDefaultDbCache) << 20); + if (nTotalCache < (nMinDbCache << 20)) + nTotalCache = (nMinDbCache << 20); // total cache cannot be less than nMinDbCache + else if (nTotalCache > (nMaxDbCache << 20)) + nTotalCache = (nMaxDbCache << 20); // total cache cannot be greater than nMaxDbCache + size_t nBlockTreeDBCache = nTotalCache / 8; +/* MCHN START */ +/* Default was false */ + if (nBlockTreeDBCache > (1 << 21) && !GetBoolArg("-txindex", true)) + nBlockTreeDBCache = (1 << 21); // block tree db cache shouldn't be larger than 2 MiB +/* MCHN END */ + nTotalCache -= nBlockTreeDBCache; + size_t nCoinDBCache = nTotalCache / 2; // use half of the remaining cache for coindb cache + nTotalCache -= nCoinDBCache; + nCoinCacheSize = nTotalCache / 300; // coins in memory require around 300 bytes + + bool fLoaded = false; + while (!fLoaded) { + bool fReset = fReindex; + std::string strLoadError; + + uiInterface.InitMessage(_("Loading block index...")); + + nStart = GetTimeMillis(); + do { + try { + UnloadBlockIndex(); + delete pcoinsTip; + delete pcoinsdbview; + delete pcoinscatcher; + delete pblocktree; + + pblocktree = new CBlockTreeDB(nBlockTreeDBCache, false, fReindex); + pcoinsdbview = new CCoinsViewDB(nCoinDBCache, false, fReindex); + pcoinscatcher = new CCoinsViewErrorCatcher(pcoinsdbview); + pcoinsTip = new CCoinsViewCache(pcoinscatcher); + + if (fReindex) + pblocktree->WriteReindexing(true); + + if (!LoadBlockIndex()) { + strLoadError = _("Error loading block database"); + break; + } + + // If the loaded chain has a wrong genesis, bail out immediately + // (we're likely using a testnet datadir, or the other way around). + if (!mapBlockIndex.empty() && mapBlockIndex.count(Params().HashGenesisBlock()) == 0) + return InitError(_("Incorrect or no genesis block found. Wrong datadir for network?")); + + // Initialize the block index (no-op if non-empty database was already loaded) + if (!InitBlockIndex()) { + strLoadError = _("Error initializing block database"); + break; + } + + // Check for changed -txindex state +/* MCHN START */ +/* Default was false */ + if (fTxIndex != GetBoolArg("-txindex", true)) { + strLoadError = _("You need to rebuild the database using -reindex to change -txindex"); + break; + } +/* MCHN END */ + + uiInterface.InitMessage(_("Verifying blocks...")); + if (!CVerifyDB().VerifyDB(pcoinsdbview, GetArg("-checklevel", 3), + GetArg("-checkblocks", 288))) { + strLoadError = _("Corrupted block database detected"); + break; + } + } catch(std::exception &e) { + if (fDebug) LogPrintf("%s\n", e.what()); + strLoadError = _("Error opening block database"); + break; + } + + fLoaded = true; + } while(false); + + if (!fLoaded) { + // first suggest a reindex + if (!fReset) { +/* MCHN START */ + bool fRet = uiInterface.ThreadSafeMessageBox( + strLoadError + ".\n\n" + _("Please restart multichaind with reindex=1."), + "", CClientUIInterface::BTN_ABORT); + +/* + bool fRet = uiInterface.ThreadSafeMessageBox( + strLoadError + ".\n\n" + _("Do you want to rebuild the block database now?"), + "", CClientUIInterface::MSG_ERROR | CClientUIInterface::BTN_ABORT); + */ +/* MCHN END */ + if (fRet) { + fReindex = true; + fRequestShutdown = false; + } else { + LogPrintf("Aborted block database rebuild. Exiting.\n"); + return false; + } + } else { + return InitError(strLoadError); + } + } + } + + // As LoadBlockIndex can take several minutes, it's possible the user + // requested to kill the GUI during the last operation. If so, exit. + // As the program has not fully started yet, Shutdown() is possibly overkill. + if (fRequestShutdown) + { + LogPrintf("Shutdown requested. Exiting.\n"); + return false; + } + LogPrintf(" block index %15dms\n", GetTimeMillis() - nStart); + + boost::filesystem::path est_path = GetDataDir() / FEE_ESTIMATES_FILENAME; + CAutoFile est_filein(fopen(est_path.string().c_str(), "rb"), SER_DISK, CLIENT_VERSION); + // Allowed to fail as this file IS missing on first startup. + if (!est_filein.IsNull()) + mempool.ReadFeeEstimates(est_filein); + fFeeEstimatesInitialized = true; + + // ********************************************************* Step 8: load wallet +#ifdef ENABLE_WALLET + if (fDisableWallet) { + pwalletMain = NULL; + LogPrintf("Wallet disabled!\n"); + } else { + + // needed to restore wallet transaction meta data after -zapwallettxes + std::vector vWtx; + + if (GetBoolArg("-zapwallettxes", false) || zap_wallet_txs) { + uiInterface.InitMessage(_("Zapping all transactions from wallet...")); + + pwalletMain = new CWallet(strWalletFile); + DBErrors nZapWalletRet = pwalletMain->ZapWalletTx(vWtx); + if (nZapWalletRet != DB_LOAD_OK) { + uiInterface.InitMessage(_("Error loading wallet.dat: Wallet corrupted")); + return false; + } + + delete pwalletMain; + pwalletMain = NULL; + } + + uiInterface.InitMessage(_("Loading wallet...")); + + nStart = GetTimeMillis(); + bool fFirstRun = true; + pwalletMain = new CWallet(strWalletFile); + DBErrors nLoadWalletRet = pwalletMain->LoadWallet(fFirstRun); + if (nLoadWalletRet != DB_LOAD_OK) + { + if (nLoadWalletRet == DB_CORRUPT) + strErrors << _("Error loading wallet.dat: Wallet corrupted") << "\n"; + else if (nLoadWalletRet == DB_NONCRITICAL_ERROR) + { + string msg(_("Warning: error reading wallet.dat! All keys read correctly, but transaction data" + " or address book entries might be missing or incorrect.")); + InitWarning(msg); + } + else if (nLoadWalletRet == DB_TOO_NEW) // MCHN + strErrors << _("Error loading wallet.dat: Wallet requires newer version of MultiChain Core") << "\n"; + else if (nLoadWalletRet == DB_NEED_REWRITE) // MCHN + { + strErrors << _("Wallet needed to be rewritten: restart MultiChain Core to complete") << "\n"; + LogPrintf("%s", strErrors.str()); + return InitError(strErrors.str()); + } + else + strErrors << _("Error loading wallet.dat") << "\n"; + } + + if (GetBoolArg("-upgradewallet", fFirstRun)) + { + int nMaxVersion = GetArg("-upgradewallet", 0); + if (nMaxVersion == 0) // the -upgradewallet without argument case + { + LogPrintf("Performing wallet upgrade to %i\n", FEATURE_LATEST); + nMaxVersion = CLIENT_VERSION; + pwalletMain->SetMinVersion(FEATURE_LATEST); // permanently upgrade the wallet immediately + } + else + LogPrintf("Allowing wallet upgrade up to %i\n", nMaxVersion); + if (nMaxVersion < pwalletMain->GetVersion()) + strErrors << _("Cannot downgrade wallet") << "\n"; + pwalletMain->SetMaxVersion(nMaxVersion); + } + + if (fFirstRun) + { + // Create new keyUser and set as default key + RandAddSeedPerfmon(); + + CPubKey newDefaultKey; + if (pwalletMain->GetKeyFromPool(newDefaultKey)) { + pwalletMain->SetDefaultKey(newDefaultKey); +// printf("Writing default address on open \n"); + if (!pwalletMain->SetAddressBook(pwalletMain->vchDefaultKey.GetID(), "", "receive")) + strErrors << _("Cannot write default address") << "\n"; + } + + pwalletMain->SetBestChain(chainActive.GetLocator()); + } + + LogPrintf("%s", strErrors.str()); + LogPrintf(" wallet %15dms\n", GetTimeMillis() - nStart); + + RegisterValidationInterface(pwalletMain); + + CBlockIndex *pindexRescan = chainActive.Tip(); + if (GetBoolArg("-rescan", false)) + pindexRescan = chainActive.Genesis(); + else + { + if( (mc_gState->m_WalletMode & MC_WMD_TXS) == 0 ) + { + CWalletDB walletdb(strWalletFile); + CBlockLocator locator; + if (walletdb.ReadBestBlock(locator)) + pindexRescan = FindForkInGlobalIndex(chainActive, locator); + else + pindexRescan = chainActive.Genesis(); + } + } +/* MCHN START */ + pwalletTxsMain->BindWallet(pwalletMain); +/* MCHN END */ + if (chainActive.Tip() && chainActive.Tip() != pindexRescan) + { + uiInterface.InitMessage(_("Rescanning...")); + LogPrintf("Rescanning last %i blocks (from block %i)...\n", chainActive.Height() - pindexRescan->nHeight, pindexRescan->nHeight); + nStart = GetTimeMillis(); + pwalletMain->ScanForWalletTransactions(pindexRescan, true, false); + LogPrintf(" rescan %15dms\n", GetTimeMillis() - nStart); + pwalletMain->SetBestChain(chainActive.GetLocator()); + nWalletDBUpdated++; + + // Restore wallet transaction metadata after -zapwallettxes=1 + if (GetBoolArg("-zapwallettxes", false) && GetArg("-zapwallettxes", "1") != "2") + { + BOOST_FOREACH(const CWalletTx& wtxOld, vWtx) + { + uint256 hash = wtxOld.GetHash(); + std::map::iterator mi = pwalletMain->mapWallet.find(hash); + if (mi != pwalletMain->mapWallet.end()) + { + const CWalletTx* copyFrom = &wtxOld; + CWalletTx* copyTo = &mi->second; + copyTo->mapValue = copyFrom->mapValue; + copyTo->vOrderForm = copyFrom->vOrderForm; + copyTo->nTimeReceived = copyFrom->nTimeReceived; + copyTo->nTimeSmart = copyFrom->nTimeSmart; + copyTo->fFromMe = copyFrom->fFromMe; + copyTo->strFromAccount = copyFrom->strFromAccount; + copyTo->nOrderPos = copyFrom->nOrderPos; + copyTo->WriteToDisk(); + } + } + } + } +/* MCHN START */ + pwalletMain->lpWalletTxs=pwalletTxsMain; + pwalletMain->InitializeUnspentList(); +/* MCHN END */ + } // (!fDisableWallet) +#else // ENABLE_WALLET + LogPrintf("No wallet compiled in!\n"); +#endif // !ENABLE_WALLET + // ********************************************************* Step 9: import blocks + + if (mapArgs.count("-blocknotify")) + uiInterface.NotifyBlockTip.connect(BlockNotifyCallback); + + // scan for better chains in the block chain database, that are not yet connected in the active best chain + CValidationState state; + if (!ActivateBestChain(state)) + strErrors << "Failed to connect best block"; + + std::vector vImportFiles; + if (mapArgs.count("-loadblock")) + { + BOOST_FOREACH(string strFile, mapMultiArgs["-loadblock"]) + vImportFiles.push_back(strFile); + } + threadGroup.create_thread(boost::bind(&ThreadImport, vImportFiles)); + if (chainActive.Tip() == NULL) { + LogPrintf("Waiting for genesis block to be imported...\n"); + while (!fRequestShutdown && chainActive.Tip() == NULL) + MilliSleep(10); + } + + // ********************************************************* Step 10: start node + + if (!CheckDiskSpace()) + return false; + + if (!strErrors.str().empty()) + return InitError(strErrors.str()); + + RandAddSeedPerfmon(); + + //// debug print + LogPrintf("mapBlockIndex.size() = %u\n", mapBlockIndex.size()); + LogPrintf("nBestHeight = %d\n", chainActive.Height()); +#ifdef ENABLE_WALLET + LogPrintf("setKeyPool.size() = %u\n", pwalletMain ? pwalletMain->setKeyPool.size() : 0); + if(mc_gState->m_WalletMode & MC_WMD_TXS) + { + LogPrintf("oldWallet.size() = %u\n", pwalletMain ? pwalletMain->mapWallet.size() : 0); + LogPrintf("mapWallet.size() = %u\n", pwalletTxsMain->m_Database->m_DBStat.m_Count); + } + else + { + LogPrintf("mapWallet.size() = %u\n", pwalletMain ? pwalletMain->mapWallet.size() : 0); +/* + BOOST_FOREACH(PAIRTYPE(const uint256, CWalletTx)& item, pwalletMain->mapWallet) + { + printf("%s\n",item.first.ToString().c_str()); + printf("%s\n",item.second.ToString().c_str()); + } + */ + } + LogPrintf("mapAddressBook.size() = %u\n", pwalletMain ? pwalletMain->mapAddressBook.size() : 0); +#endif + + StartNode(threadGroup); + +#ifdef ENABLE_WALLET + // Generate coins in the background + if (pwalletMain) +/* MCHN START */ +// GenerateBitcoins(GetBoolArg("-gen", false), pwalletMain, GetArg("-genproclimit", 1)); + GenerateBitcoins(GetBoolArg("-gen", true), pwalletMain, GetArg("-genproclimit", 1)); +/* MCHN END */ +#endif + + // ********************************************************* Step 11: finished + + SetRPCWarmupFinished(); + uiInterface.InitMessage(_("Done loading")); + +#ifdef ENABLE_WALLET + if (pwalletMain) { + // Add wallet transactions that aren't already in a block to mapTransactions + pwalletMain->ReacceptWalletTransactions(); + + // Run a thread to flush wallet periodically + threadGroup.create_thread(boost::bind(&ThreadFlushWalletDB, boost::ref(pwalletMain->strWalletFile))); + } +#endif + + +/* MCHN START */ + if(!GetBoolArg("-shortoutput", false)) + { + sprintf(bufOutput,"Node started\n"); + bytes_written=write(OutputPipe,bufOutput,strlen(bufOutput)); + } + mc_InitRPCHelpMap(); + + LogPrintf("Node started\n"); +/* MCHN END */ + return !fRequestShutdown; +} diff --git a/src/core/init.h b/src/core/init.h new file mode 100644 index 00000000..0a0cfde9 --- /dev/null +++ b/src/core/init.h @@ -0,0 +1,51 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_INIT_H +#define BITCOIN_INIT_H + +#include + +class CWallet; +struct mc_WalletTxs; + +namespace boost +{ +class thread_group; +} // namespace boost + +extern CWallet* pwalletMain; +extern mc_WalletTxs* pwalletTxsMain; + + +void StartShutdown(); +bool ShutdownRequested(); +void Shutdown(); + +/* MCHN START */ +#ifndef STDOUT_FILENO +#define STDIN_FILENO 0 +#define STDOUT_FILENO 1 +#define STDERR_FILENO 2 +#endif +/* MCHN END */ + + + +bool AppInit2(boost::thread_group& threadGroup,int OutputPipe=STDOUT_FILENO); + +/** The help message mode determines what help message to show */ +enum HelpMessageMode { + HMM_BITCOIND, + HMM_BITCOIN_QT +}; + +/** Help for options shared between UI and daemon (for -help) */ +std::string HelpMessage(HelpMessageMode mode); +/** Returns licensing information (for -version) */ +std::string LicenseInfo(); + +#endif // BITCOIN_INIT_H diff --git a/src/core/main.cpp b/src/core/main.cpp new file mode 100644 index 00000000..f21086fa --- /dev/null +++ b/src/core/main.cpp @@ -0,0 +1,6315 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2016 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "core/main.h" + +#include "storage/addrman.h" +#include "structs/alert.h" +#include "chainparams/chainparams.h" +#include "chain/checkpoints.h" +#include "checkqueue.h" +#include "core/init.h" +#include "chain/merkleblock.h" +#include "net/net.h" +#include "chain/pow.h" +#include "storage/txdb.h" +#include "chain/txmempool.h" +#include "ui/ui_interface.h" +#include "utils/util.h" +#include "utils/utilmoneystr.h" + +/* MCHN START */ + +#include "structs/base58.h" +#include "keys/pubkey.h" +#include "keys/key.h" +#include "wallet/wallet.h" +#include "multichain/multichain.h" +#include "wallet/wallettxs.h" +#include "script/script.h" + + +extern mc_WalletTxs* pwalletTxsMain; + + +/* MCHN END */ + +#include + +#include +#include +#include +#include + +using namespace boost; +using namespace std; + +bool AcceptMultiChainTransaction(const CTransaction& tx, + const CCoinsViewCache &inputs, + int offset, + bool accept, + string& reason); +bool AcceptAssetTransfers(const CTransaction& tx, const CCoinsViewCache &inputs, string& reason); +bool AcceptAssetGenesis(const CTransaction &tx,int offset,bool accept,string& reason); +bool AcceptPermissionsAndCheckForDust(const CTransaction &tx,bool accept,string& reason); +bool ReplayMemPool(CTxMemPool& pool, int from,bool accept); +bool VerifyBlockSignature(CBlock *block,bool force); +bool CheckBlockPermissions(const CBlock& block,CBlockIndex* prev_block,unsigned char *lpMinerAddress); +bool ProcessMultichainVerack(CNode* pfrom, CDataStream& vRecv,bool fIsVerackack,bool *disconnect_flag); +bool PushMultiChainVerack(CNode* pfrom, bool fIsVerackack); +bool MultichainNode_CanConnect(CNode *pnode); +bool MultichainNode_DisconnectRemote(CNode *pnode); +bool MultichainNode_DisconnectLocal(CNode *pnode); +bool MultichainNode_RespondToGetData(CNode *pnode); +bool MultichainNode_SendInv(CNode *pnode); +bool MultichainNode_AcceptData(CNode *pnode); +bool MultichainNode_IgnoreIncoming(CNode *pnode); +bool MultichainNode_IsLocal(CNode *pnode); + + + + +#if defined(NDEBUG) +# error "Bitcoin cannot be compiled without assertions." +#endif + +/** + * Global state + */ + +CCriticalSection cs_main; + +BlockMap mapBlockIndex; +CChain chainActive; +CBlockIndex *pindexBestHeader = NULL; +int64_t nTimeBestReceived = 0; +CWaitableCriticalSection csBestBlock; +CConditionVariable cvBlockChange; +int nScriptCheckThreads = 0; +bool fImporting = false; +bool fReindex = false; +bool fTxIndex = false; +bool fIsBareMultisigStd = true; +unsigned int nCoinCacheSize = 5000; + + +/** Fees smaller than this (in satoshi) are considered zero fee (for relaying and mining) */ +/* MCHN START */ +//CFeeRate minRelayTxFee = CFeeRate(1000); +CFeeRate minRelayTxFee = CFeeRate(MIN_RELAY_TX_FEE); +/* MCHN END */ + +CTxMemPool mempool(::minRelayTxFee); + +struct COrphanTx { + CTransaction tx; + NodeId fromPeer; +}; +map mapOrphanTransactions; +map > mapOrphanTransactionsByPrev; +void EraseOrphansFor(NodeId peer); + +/* MCHN START */ +#define MC_TXSET_BLOCKS 50 +set setBlockTransactions[MC_TXSET_BLOCKS]; +/* MCHN END */ + +/** Constant stuff for coinbase transactions we create: */ +CScript COINBASE_FLAGS; + +const string strMessageMagic = "MultiChain Signed Message:\n"; + +// Internal stuff +namespace { + + struct CBlockIndexWorkComparator + { + bool operator()(CBlockIndex *pa, CBlockIndex *pb) { + // First sort by most total work, ... + if (pa->nChainWork > pb->nChainWork) return false; + if (pa->nChainWork < pb->nChainWork) return true; + + +/* MCHN START */ + // Prefer chains we mined long time ago + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if((pa->nCanMine > 0) && (pb->nCanMine == 0)) return false; + if((pa->nCanMine == 0) && (pb->nCanMine > 0)) return true; + if((pa->nCanMine == 0) && (pb->nCanMine == 0)) + { + if (pa->nHeightMinedByMe < pb->nHeightMinedByMe) return false; + if (pa->nHeightMinedByMe > pb->nHeightMinedByMe) return true; + } + } +/* MCHN END */ + + // ... then by earliest time received, ... + if (pa->nSequenceId < pb->nSequenceId) return false; + if (pa->nSequenceId > pb->nSequenceId) return true; + + // Use pointer address as tie breaker (should only happen with blocks + // loaded from disk, as those all have id 0). + if (pa < pb) return false; + if (pa > pb) return true; + + // Identical blocks. + return false; + } + }; + + CBlockIndex *pindexBestInvalid; + + /** + * The set of all CBlockIndex entries with BLOCK_VALID_TRANSACTIONS or better that are at least + * as good as our current tip. Entries may be failed, though. + */ + set setBlockIndexCandidates; + /** Number of nodes with fSyncStarted. */ + int nSyncStarted = 0; + /** All pairs A->B, where A (or one if its ancestors) misses transactions, but B has transactions. */ + multimap mapBlocksUnlinked; + + CCriticalSection cs_LastBlockFile; + std::vector vinfoBlockFile; + int nLastBlockFile = 0; + + /** + * Every received block is assigned a unique and increasing identifier, so we + * know which one to give priority in case of a fork. + */ + CCriticalSection cs_nBlockSequenceId; + /** Blocks loaded from disk are assigned id 0, so start the counter at 1. */ + uint32_t nBlockSequenceId = 1; + + /** + * Sources of received blocks, to be able to send them reject messages or ban + * them, if processing happens afterwards. Protected by cs_main. + */ + map mapBlockSource; + + /** Blocks that are in flight, and that are in the queue to be downloaded. Protected by cs_main. */ + struct QueuedBlock { + uint256 hash; + CBlockIndex *pindex; //! Optional. + int64_t nTime; //! Time of "getdata" request in microseconds. + int nValidatedQueuedBefore; //! Number of blocks queued with validated headers (globally) at the time this one is requested. + bool fValidatedHeaders; //! Whether this block has validated headers at the time of request. + }; + map::iterator> > mapBlocksInFlight; + + /** Number of blocks in flight with validated headers. */ + int nQueuedValidatedHeaders = 0; + + /** Number of preferable block download peers. */ + int nPreferredDownload = 0; + + /** Dirty block index entries. */ + set setDirtyBlockIndex; + + /** Dirty block file entries. */ + set setDirtyFileInfo; +} // anon namespace + +////////////////////////////////////////////////////////////////////////////// +// +// dispatching functions +// + +// These functions dispatch to one or all registered wallets + +namespace { + +struct CMainSignals { + /** Notifies listeners of updated transaction data (transaction, and optionally the block it is found in. */ + boost::signals2::signal SyncTransaction; + /** Notifies listeners of an erased transaction (currently disabled, requires transaction replacement). */ + boost::signals2::signal EraseTransaction; + /** Notifies listeners of an updated transaction without new data (for now: a coinbase potentially becoming visible). */ + boost::signals2::signal UpdatedTransaction; + /** Notifies listeners of a new active block chain. */ + boost::signals2::signal SetBestChain; + /** Notifies listeners about an inventory item being seen on the network. */ + boost::signals2::signal Inventory; + /** Tells listeners to broadcast their data. */ + boost::signals2::signal Broadcast; + /** Notifies listeners of a block validation result */ + boost::signals2::signal BlockChecked; +} g_signals; + +} // anon namespace + +void RegisterValidationInterface(CValidationInterface* pwalletIn) { + g_signals.SyncTransaction.connect(boost::bind(&CValidationInterface::SyncTransaction, pwalletIn, _1, _2)); + g_signals.EraseTransaction.connect(boost::bind(&CValidationInterface::EraseFromWallet, pwalletIn, _1)); + g_signals.UpdatedTransaction.connect(boost::bind(&CValidationInterface::UpdatedTransaction, pwalletIn, _1)); + g_signals.SetBestChain.connect(boost::bind(&CValidationInterface::SetBestChain, pwalletIn, _1)); + g_signals.Inventory.connect(boost::bind(&CValidationInterface::Inventory, pwalletIn, _1)); + g_signals.Broadcast.connect(boost::bind(&CValidationInterface::ResendWalletTransactions, pwalletIn)); + g_signals.BlockChecked.connect(boost::bind(&CValidationInterface::BlockChecked, pwalletIn, _1, _2)); +} + +void UnregisterValidationInterface(CValidationInterface* pwalletIn) { + g_signals.BlockChecked.disconnect(boost::bind(&CValidationInterface::BlockChecked, pwalletIn, _1, _2)); + g_signals.Broadcast.disconnect(boost::bind(&CValidationInterface::ResendWalletTransactions, pwalletIn)); + g_signals.Inventory.disconnect(boost::bind(&CValidationInterface::Inventory, pwalletIn, _1)); + g_signals.SetBestChain.disconnect(boost::bind(&CValidationInterface::SetBestChain, pwalletIn, _1)); + g_signals.UpdatedTransaction.disconnect(boost::bind(&CValidationInterface::UpdatedTransaction, pwalletIn, _1)); + g_signals.EraseTransaction.disconnect(boost::bind(&CValidationInterface::EraseFromWallet, pwalletIn, _1)); + g_signals.SyncTransaction.disconnect(boost::bind(&CValidationInterface::SyncTransaction, pwalletIn, _1, _2)); +} + +void UnregisterAllValidationInterfaces() { + g_signals.BlockChecked.disconnect_all_slots(); + g_signals.Broadcast.disconnect_all_slots(); + g_signals.Inventory.disconnect_all_slots(); + g_signals.SetBestChain.disconnect_all_slots(); + g_signals.UpdatedTransaction.disconnect_all_slots(); + g_signals.EraseTransaction.disconnect_all_slots(); + g_signals.SyncTransaction.disconnect_all_slots(); +} + +void SyncWithWallets(const CTransaction &tx, const CBlock *pblock) { + g_signals.SyncTransaction(tx, pblock); +} + +////////////////////////////////////////////////////////////////////////////// +// +// Registration of network node signals. +// + +namespace { + +struct CBlockReject { + unsigned char chRejectCode; + string strRejectReason; + uint256 hashBlock; +}; + +/** + * Maintain validation-specific state about nodes, protected by cs_main, instead + * by CNode's own locks. This simplifies asynchronous operation, where + * processing of incoming data is done after the ProcessMessage call returns, + * and we're no longer holding the node's locks. + */ +struct CNodeState { + //! Accumulated misbehaviour score for this peer. + int nMisbehavior; + //! Whether this peer should be disconnected and banned (unless whitelisted). + bool fShouldBan; + //! String name of this peer (debugging/logging purposes). + std::string name; + //! List of asynchronously-determined block rejections to notify this peer about. + std::vector rejects; + //! The best known block we know this peer has announced. + CBlockIndex *pindexBestKnownBlock; + //! The hash of the last unknown block this peer has announced. + uint256 hashLastUnknownBlock; + //! The last full block we both have. + CBlockIndex *pindexLastCommonBlock; + //! Whether we've started headers synchronization with this peer. + bool fSyncStarted; + //! Since when we're stalling block download progress (in microseconds), or 0. + int64_t nStallingSince; + list vBlocksInFlight; + int nBlocksInFlight; + //! Whether we consider this a preferred download peer. + bool fPreferredDownload; + + CNodeState() { + nMisbehavior = 0; + fShouldBan = false; + pindexBestKnownBlock = NULL; + hashLastUnknownBlock = uint256(0); + pindexLastCommonBlock = NULL; + fSyncStarted = false; + nStallingSince = 0; + nBlocksInFlight = 0; + fPreferredDownload = false; + } +}; + +/** Map maintaining per-node state. Requires cs_main. */ +map mapNodeState; + +/* MCHN START */ + +bool MultichainNode_IsBlockChainSynced(CNode *pnode) +{ + if(pnode->fSyncedOnce) + { + return true; + } + + int this_height=(int)chainActive.Height(); + map::iterator it = mapNodeState.find(pnode->id); + if (it == mapNodeState.end()) + return false; + CNodeState *state = &it->second; + int sync_height = state->pindexBestKnownBlock ? state->pindexBestKnownBlock->nHeight : -1; + int common_height = state->pindexLastCommonBlock ? state->pindexLastCommonBlock->nHeight : -1; + + if((this_height > 0) && (common_height>0)) + { + if(this_height == common_height) + { + if(sync_height == common_height) + { + pnode->fSyncedOnce=true; + } + } + if((this_height > common_height) && (this_height > sync_height)) + { + pnode->fSyncedOnce=true; + } + } + + return pnode->fSyncedOnce; +} + + + +/* MCHN END */ + + +// Requires cs_main. +CNodeState *State(NodeId pnode) { + map::iterator it = mapNodeState.find(pnode); + if (it == mapNodeState.end()) + return NULL; + return &it->second; +} + +int GetHeight() +{ + LOCK(cs_main); + return chainActive.Height(); +} + +void UpdatePreferredDownload(CNode* node, CNodeState* state) +{ + nPreferredDownload -= state->fPreferredDownload; + + // Whether this node should be marked as a preferred download node. + state->fPreferredDownload = (!node->fInbound || node->fWhitelisted) && !node->fOneShot && !node->fClient; + + nPreferredDownload += state->fPreferredDownload; +} + +void InitializeNode(NodeId nodeid, const CNode *pnode) { + LOCK(cs_main); + CNodeState &state = mapNodeState.insert(std::make_pair(nodeid, CNodeState())).first->second; + state.name = pnode->addrName; +} + +void FinalizeNode(NodeId nodeid) { + LOCK(cs_main); + CNodeState *state = State(nodeid); + + if (state->fSyncStarted) + nSyncStarted--; + + BOOST_FOREACH(const QueuedBlock& entry, state->vBlocksInFlight) + mapBlocksInFlight.erase(entry.hash); + EraseOrphansFor(nodeid); + nPreferredDownload -= state->fPreferredDownload; + + mapNodeState.erase(nodeid); +} + +// Requires cs_main. +void MarkBlockAsReceived(const uint256& hash) { + map::iterator> >::iterator itInFlight = mapBlocksInFlight.find(hash); + if (itInFlight != mapBlocksInFlight.end()) { + CNodeState *state = State(itInFlight->second.first); + nQueuedValidatedHeaders -= itInFlight->second.second->fValidatedHeaders; + state->vBlocksInFlight.erase(itInFlight->second.second); + state->nBlocksInFlight--; + state->nStallingSince = 0; + mapBlocksInFlight.erase(itInFlight); + } +} + +// Requires cs_main. +void MarkBlockAsInFlight(NodeId nodeid, const uint256& hash, CBlockIndex *pindex = NULL) { + CNodeState *state = State(nodeid); + assert(state != NULL); + + // Make sure it's not listed somewhere already. + MarkBlockAsReceived(hash); + + QueuedBlock newentry = {hash, pindex, GetTimeMicros(), nQueuedValidatedHeaders, pindex != NULL}; + nQueuedValidatedHeaders += newentry.fValidatedHeaders; + list::iterator it = state->vBlocksInFlight.insert(state->vBlocksInFlight.end(), newentry); + state->nBlocksInFlight++; + mapBlocksInFlight[hash] = std::make_pair(nodeid, it); +} + +/** Check whether the last unknown block a peer advertized is not yet known. */ +void ProcessBlockAvailability(NodeId nodeid) { + CNodeState *state = State(nodeid); + assert(state != NULL); + + if (state->hashLastUnknownBlock != 0) { + BlockMap::iterator itOld = mapBlockIndex.find(state->hashLastUnknownBlock); + if (itOld != mapBlockIndex.end() && itOld->second->nChainWork > 0) { + if (state->pindexBestKnownBlock == NULL || itOld->second->nChainWork >= state->pindexBestKnownBlock->nChainWork) + state->pindexBestKnownBlock = itOld->second; + state->hashLastUnknownBlock = uint256(0); + } + } +} + +/** Update tracking information about which blocks a peer is assumed to have. */ +void UpdateBlockAvailability(NodeId nodeid, const uint256 &hash) { + CNodeState *state = State(nodeid); + assert(state != NULL); + + ProcessBlockAvailability(nodeid); + + BlockMap::iterator it = mapBlockIndex.find(hash); + if (it != mapBlockIndex.end() && it->second->nChainWork > 0) { + // An actually better block was announced. + if (state->pindexBestKnownBlock == NULL || it->second->nChainWork >= state->pindexBestKnownBlock->nChainWork) + state->pindexBestKnownBlock = it->second; + } else { + // An unknown block was announced; just assume that the latest one is the best one. + state->hashLastUnknownBlock = hash; + } +} + +/** Find the last common ancestor two blocks have. + * Both pa and pb must be non-NULL. */ +CBlockIndex* LastCommonAncestor(CBlockIndex* pa, CBlockIndex* pb) { + if (pa->nHeight > pb->nHeight) { + pa = pa->GetAncestor(pb->nHeight); + } else if (pb->nHeight > pa->nHeight) { + pb = pb->GetAncestor(pa->nHeight); + } + + while (pa != pb && pa && pb) { + pa = pa->pprev; + pb = pb->pprev; + } + + // Eventually all chain branches meet at the genesis block. + assert(pa == pb); + return pa; +} + +/** Update pindexLastCommonBlock and add not-in-flight missing successors to vBlocks, until it has + * at most count entries. */ +void FindNextBlocksToDownload(NodeId nodeid, unsigned int count, std::vector& vBlocks, NodeId& nodeStaller) { + if (count == 0) + return; + + vBlocks.reserve(vBlocks.size() + count); + CNodeState *state = State(nodeid); + assert(state != NULL); + + // Make sure pindexBestKnownBlock is up to date, we'll need it. + ProcessBlockAvailability(nodeid); + + if (state->pindexBestKnownBlock == NULL || state->pindexBestKnownBlock->nChainWork < chainActive.Tip()->nChainWork) { + // This peer has nothing interesting. + return; + } + + if (state->pindexLastCommonBlock == NULL) { + // Bootstrap quickly by guessing a parent of our best tip is the forking point. + // Guessing wrong in either direction is not a problem. + state->pindexLastCommonBlock = chainActive[std::min(state->pindexBestKnownBlock->nHeight, chainActive.Height())]; + } + + // If the peer reorganized, our previous pindexLastCommonBlock may not be an ancestor + // of their current tip anymore. Go back enough to fix that. + state->pindexLastCommonBlock = LastCommonAncestor(state->pindexLastCommonBlock, state->pindexBestKnownBlock); + if (state->pindexLastCommonBlock == state->pindexBestKnownBlock) + return; + + std::vector vToFetch; + CBlockIndex *pindexWalk = state->pindexLastCommonBlock; + // Never fetch further than the best block we know the peer has, or more than BLOCK_DOWNLOAD_WINDOW + 1 beyond the last + // linked block we have in common with this peer. The +1 is so we can detect stalling, namely if we would be able to + // download that next block if the window were 1 larger. + int nWindowEnd = state->pindexLastCommonBlock->nHeight + BLOCK_DOWNLOAD_WINDOW; + int nMaxHeight = std::min(state->pindexBestKnownBlock->nHeight, nWindowEnd + 1); + NodeId waitingfor = -1; + while (pindexWalk->nHeight < nMaxHeight) { + // Read up to 128 (or more, if more blocks than that are needed) successors of pindexWalk (towards + // pindexBestKnownBlock) into vToFetch. We fetch 128, because CBlockIndex::GetAncestor may be as expensive + // as iterating over ~100 CBlockIndex* entries anyway. + int nToFetch = std::min(nMaxHeight - pindexWalk->nHeight, std::max(count - vBlocks.size(), 128)); + vToFetch.resize(nToFetch); + pindexWalk = state->pindexBestKnownBlock->GetAncestor(pindexWalk->nHeight + nToFetch); + vToFetch[nToFetch - 1] = pindexWalk; + for (unsigned int i = nToFetch - 1; i > 0; i--) { + vToFetch[i - 1] = vToFetch[i]->pprev; + } + + // Iterate over those blocks in vToFetch (in forward direction), adding the ones that + // are not yet downloaded and not in flight to vBlocks. In the mean time, update + // pindexLastCommonBlock as long as all ancestors are already downloaded. + BOOST_FOREACH(CBlockIndex* pindex, vToFetch) { + if (!pindex->IsValid(BLOCK_VALID_TREE)) { + // We consider the chain that this peer is on invalid. + return; + } + if (pindex->nStatus & BLOCK_HAVE_DATA) { + if (pindex->nChainTx) + state->pindexLastCommonBlock = pindex; + } else if (mapBlocksInFlight.count(pindex->GetBlockHash()) == 0) { + // The block is not already downloaded, and not yet in flight. + if (pindex->nHeight > nWindowEnd) { + // We reached the end of the window. + if (vBlocks.size() == 0 && waitingfor != nodeid) { + // We aren't able to fetch anything, but we would be if the download window was one larger. + nodeStaller = waitingfor; + } + return; + } + vBlocks.push_back(pindex); + if (vBlocks.size() == count) { + return; + } + } else if (waitingfor == -1) { + // This is the first already-in-flight block. + waitingfor = mapBlocksInFlight[pindex->GetBlockHash()].first; + } + } + } +} + +} // anon namespace + +bool GetNodeStateStats(NodeId nodeid, CNodeStateStats &stats) { + LOCK(cs_main); + CNodeState *state = State(nodeid); + if (state == NULL) + return false; + stats.nMisbehavior = state->nMisbehavior; + stats.nSyncHeight = state->pindexBestKnownBlock ? state->pindexBestKnownBlock->nHeight : -1; + stats.nCommonHeight = state->pindexLastCommonBlock ? state->pindexLastCommonBlock->nHeight : -1; + BOOST_FOREACH(const QueuedBlock& queue, state->vBlocksInFlight) { + if (queue.pindex) + stats.vHeightInFlight.push_back(queue.pindex->nHeight); + } + return true; +} + +void RegisterNodeSignals(CNodeSignals& nodeSignals) +{ + nodeSignals.GetHeight.connect(&GetHeight); + nodeSignals.ProcessMessages.connect(&ProcessMessages); + nodeSignals.SendMessages.connect(&SendMessages); + nodeSignals.InitializeNode.connect(&InitializeNode); + nodeSignals.FinalizeNode.connect(&FinalizeNode); +} + +void UnregisterNodeSignals(CNodeSignals& nodeSignals) +{ + nodeSignals.GetHeight.disconnect(&GetHeight); + nodeSignals.ProcessMessages.disconnect(&ProcessMessages); + nodeSignals.SendMessages.disconnect(&SendMessages); + nodeSignals.InitializeNode.disconnect(&InitializeNode); + nodeSignals.FinalizeNode.disconnect(&FinalizeNode); +} + +CBlockIndex* FindForkInGlobalIndex(const CChain& chain, const CBlockLocator& locator) +{ + // Find the first block the caller has in the main chain + BOOST_FOREACH(const uint256& hash, locator.vHave) { + BlockMap::iterator mi = mapBlockIndex.find(hash); + if (mi != mapBlockIndex.end()) + { + CBlockIndex* pindex = (*mi).second; + if (chain.Contains(pindex)) + return pindex; + } + } + return chain.Genesis(); +} + +CCoinsViewCache *pcoinsTip = NULL; +CBlockTreeDB *pblocktree = NULL; + +////////////////////////////////////////////////////////////////////////////// +// +// mapOrphanTransactions +// + +bool AddOrphanTx(const CTransaction& tx, NodeId peer) +{ + uint256 hash = tx.GetHash(); + if (mapOrphanTransactions.count(hash)) + return false; + + // Ignore big transactions, to avoid a + // send-big-orphans memory exhaustion attack. If a peer has a legitimate + // large transaction with a missing parent then we assume + // it will rebroadcast it later, after the parent transaction(s) + // have been mined or received. + // 10,000 orphans, each of which is at most 5,000 bytes big is + // at most 500 megabytes of orphans: + unsigned int sz = tx.GetSerializeSize(SER_NETWORK, CTransaction::CURRENT_VERSION); + if (sz > 5000) + { + LogPrint("mempool", "ignoring large orphan tx (size: %u, hash: %s)\n", sz, hash.ToString()); + return false; + } + + mapOrphanTransactions[hash].tx = tx; + mapOrphanTransactions[hash].fromPeer = peer; + BOOST_FOREACH(const CTxIn& txin, tx.vin) + mapOrphanTransactionsByPrev[txin.prevout.hash].insert(hash); + + LogPrint("mempool", "stored orphan tx %s (mapsz %u prevsz %u)\n", hash.ToString(), + mapOrphanTransactions.size(), mapOrphanTransactionsByPrev.size()); + return true; +} + +void static EraseOrphanTx(uint256 hash) +{ + map::iterator it = mapOrphanTransactions.find(hash); + if (it == mapOrphanTransactions.end()) + return; + BOOST_FOREACH(const CTxIn& txin, it->second.tx.vin) + { + map >::iterator itPrev = mapOrphanTransactionsByPrev.find(txin.prevout.hash); + if (itPrev == mapOrphanTransactionsByPrev.end()) + continue; + itPrev->second.erase(hash); + if (itPrev->second.empty()) + mapOrphanTransactionsByPrev.erase(itPrev); + } + mapOrphanTransactions.erase(it); +} + +void EraseOrphansFor(NodeId peer) +{ + int nErased = 0; + map::iterator iter = mapOrphanTransactions.begin(); + while (iter != mapOrphanTransactions.end()) + { + map::iterator maybeErase = iter++; // increment to avoid iterator becoming invalid + if (maybeErase->second.fromPeer == peer) + { + EraseOrphanTx(maybeErase->second.tx.GetHash()); + ++nErased; + } + } + if (nErased > 0) LogPrint("mempool", "Erased %d orphan tx from peer %d\n", nErased, peer); +} + + +unsigned int LimitOrphanTxSize(unsigned int nMaxOrphans) +{ + unsigned int nEvicted = 0; + while (mapOrphanTransactions.size() > nMaxOrphans) + { + // Evict a random orphan: + uint256 randomhash = GetRandHash(); + map::iterator it = mapOrphanTransactions.lower_bound(randomhash); + if (it == mapOrphanTransactions.end()) + it = mapOrphanTransactions.begin(); + EraseOrphanTx(it->first); + ++nEvicted; + } + return nEvicted; +} + + +/* MCHN START */ +bool IsStandardTx(const CTransaction& tx, string& reason,bool check_for_dust) +/* MCHN END */ +{ + AssertLockHeld(cs_main); + if (tx.nVersion > CTransaction::CURRENT_VERSION || tx.nVersion < 1) { + reason = "version"; + return false; + } + + // Treat non-final transactions as non-standard to prevent a specific type + // of double-spend attack, as well as DoS attacks. (if the transaction + // can't be mined, the attacker isn't expending resources broadcasting it) + // Basically we don't want to propagate transactions that can't be included in + // the next block. + // + // However, IsFinalTx() is confusing... Without arguments, it uses + // chainActive.Height() to evaluate nLockTime; when a block is accepted, chainActive.Height() + // is set to the value of nHeight in the block. However, when IsFinalTx() + // is called within CBlock::AcceptBlock(), the height of the block *being* + // evaluated is what is used. Thus if we want to know if a transaction can + // be part of the *next* block, we need to call IsFinalTx() with one more + // than chainActive.Height(). + // + // Timestamps on the other hand don't get any special treatment, because we + // can't know what timestamp the next block will have, and there aren't + // timestamp applications where it matters. + if (!IsFinalTx(tx, chainActive.Height() + 1)) { + reason = "non-final"; + return false; + } + + // Extremely large transactions with lots of inputs can cost the network + // almost as much to process as they cost the sender in fees, because + // computing signature hashes is O(ninputs*txsize). Limiting transactions + // to MAX_STANDARD_TX_SIZE mitigates CPU exhaustion attacks. + unsigned int sz = tx.GetSerializeSize(SER_NETWORK, CTransaction::CURRENT_VERSION); + if (sz >= MAX_STANDARD_TX_SIZE) { + reason = "tx-size"; + return false; + } + +/* MCHN START */ + // This transaction will not fit any block + if (sz > MAX_BLOCK_SIZE-81) { + reason = "tx-size"; + return false; + } +/* MCHN END */ + + + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + // Biggest 'standard' txin is a 15-of-15 P2SH multisig with compressed + // keys. (remember the 520 byte limit on redeemScript size) That works + // out to a (15*(33+1))+3=513 byte redeemScript, 513+1+15*(73+1)+3=1627 + // bytes of scriptSig, which we round off to 1650 bytes for some minor + // future-proofing. That's also enough to spend a 20-of-20 + // CHECKMULTISIG scriptPubKey, though such a scriptPubKey is not + // considered standard) + if (txin.scriptSig.size() > 1650) { + reason = "scriptsig-size"; + return false; + } + if (!txin.scriptSig.IsPushOnly()) { + reason = "scriptsig-not-pushonly"; + return false; + } + } + + unsigned int nDataOut = 0; + txnouttype whichType; + BOOST_FOREACH(const CTxOut& txout, tx.vout) { + if (!::IsStandard(txout.scriptPubKey, whichType)) { + reason = "scriptpubkey"; + return false; + } + + if (whichType == TX_NULL_DATA) + nDataOut++; + else if ((whichType == TX_MULTISIG) && (!fIsBareMultisigStd)) { + reason = "bare-multisig"; + return false; + } else if (txout.IsDust(::minRelayTxFee)) { +/* MCHN START */ + if(check_for_dust) + { +/* MCHN END */ + reason = "dust"; + return false; +/* MCHN START */ + } +/* MCHN END */ + } + } + + // only one OP_RETURN txout is permitted +/* MCHN START */ + int max_op_returns=1; + if(mc_gState->m_Features->Streams()) + { + max_op_returns=(int)mc_gState->m_NetworkParams->GetInt64Param("maxstdopreturnscount"); + } +/* MCHN END */ + + if ((int)nDataOut > max_op_returns) { + reason = "multi-op-return"; + return false; + } + + return true; +} + +/* MCHN START */ +bool IsStandardTx(const CTransaction& tx, string& reason) +{ + return IsStandardTx(tx,reason,true); +} +/* MCHN START */ + +bool IsFinalTx(const CTransaction &tx, int nBlockHeight, int64_t nBlockTime) +{ + AssertLockHeld(cs_main); + // Time based nLockTime implemented in 0.1.6 + if (tx.nLockTime == 0) + return true; + if (nBlockHeight == 0) + nBlockHeight = chainActive.Height(); + if (nBlockTime == 0) + nBlockTime = GetAdjustedTime(); + if ((int64_t)tx.nLockTime < ((int64_t)tx.nLockTime < LOCKTIME_THRESHOLD ? (int64_t)nBlockHeight : nBlockTime)) + return true; + BOOST_FOREACH(const CTxIn& txin, tx.vin) + if (!txin.IsFinal()) + return false; + return true; +} + +/** + * Check transaction inputs to mitigate two + * potential denial-of-service attacks: + * + * 1. scriptSigs with extra data stuffed into them, + * not consumed by scriptPubKey (or P2SH script) + * 2. P2SH scripts with a crazy number of expensive + * CHECKSIG/CHECKMULTISIG operations + */ +bool AreInputsStandard(const CTransaction& tx, const CCoinsViewCache& mapInputs) +{ + if (tx.IsCoinBase()) + return true; // Coinbases don't use vin normally + + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + const CTxOut& prev = mapInputs.GetOutputFor(tx.vin[i]); + + vector > vSolutions; + txnouttype whichType; + // get the scriptPubKey corresponding to this input: + const CScript& prevScript = prev.scriptPubKey; + if (!Solver(prevScript, whichType, vSolutions)) + return false; + int nArgsExpected = ScriptSigArgsExpected(whichType, vSolutions); + if (nArgsExpected < 0) + return false; + + // Transactions with extra stuff in their scriptSigs are + // non-standard. Note that this EvalScript() call will + // be quick, because if there are any operations + // beside "push data" in the scriptSig + // IsStandard() will have already returned false + // and this method isn't called. + vector > stack; + if (!EvalScript(stack, tx.vin[i].scriptSig, false, BaseSignatureChecker())) + return false; + + if (whichType == TX_SCRIPTHASH) + { + if (stack.empty()) + return false; + CScript subscript(stack.back().begin(), stack.back().end()); + vector > vSolutions2; + txnouttype whichType2; + if (Solver(subscript, whichType2, vSolutions2)) + { + int tmpExpected = ScriptSigArgsExpected(whichType2, vSolutions2); + if (tmpExpected < 0) + return false; + nArgsExpected += tmpExpected; + } + else + { + // Any other Script with less than 15 sigops OK: + unsigned int sigops = subscript.GetSigOpCount(true); + // ... extra data left on the stack after execution is OK, too: + return (sigops <= MAX_P2SH_SIGOPS); + } + } + + if (stack.size() != (unsigned int)nArgsExpected) + return false; + } + + return true; +} + +unsigned int GetLegacySigOpCount(const CTransaction& tx) +{ + unsigned int nSigOps = 0; + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + nSigOps += txin.scriptSig.GetSigOpCount(false); + } + BOOST_FOREACH(const CTxOut& txout, tx.vout) + { + nSigOps += txout.scriptPubKey.GetSigOpCount(false); + } + return nSigOps; +} + +unsigned int GetP2SHSigOpCount(const CTransaction& tx, const CCoinsViewCache& inputs) +{ + if (tx.IsCoinBase()) + return 0; + + unsigned int nSigOps = 0; + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + const CTxOut &prevout = inputs.GetOutputFor(tx.vin[i]); + if (prevout.scriptPubKey.IsPayToScriptHash()) + nSigOps += prevout.scriptPubKey.GetSigOpCount(tx.vin[i].scriptSig); + } + return nSigOps; +} + + + + + + + + +bool CheckTransaction(const CTransaction& tx, CValidationState &state) +{ + // Basic checks that don't depend on any context + if (tx.vin.empty()) + return state.DoS(10, error("CheckTransaction() : vin empty"), + REJECT_INVALID, "bad-txns-vin-empty"); + if (tx.vout.empty()) + return state.DoS(10, error("CheckTransaction() : vout empty"), + REJECT_INVALID, "bad-txns-vout-empty"); + // Size limits + if (::GetSerializeSize(tx, SER_NETWORK, PROTOCOL_VERSION) > MAX_BLOCK_SIZE) + return state.DoS(100, error("CheckTransaction() : size limits failed"), + REJECT_INVALID, "bad-txns-oversize"); + + // Check for negative or overflow output values + CAmount nValueOut = 0; + BOOST_FOREACH(const CTxOut& txout, tx.vout) + { + if (txout.nValue < 0) + return state.DoS(100, error("CheckTransaction() : txout.nValue negative"), + REJECT_INVALID, "bad-txns-vout-negative"); + if (txout.nValue > MAX_MONEY) + return state.DoS(100, error("CheckTransaction() : txout.nValue too high"), + REJECT_INVALID, "bad-txns-vout-toolarge"); + nValueOut += txout.nValue; + if (!MoneyRange(nValueOut)) + return state.DoS(100, error("CheckTransaction() : txout total out of range"), + REJECT_INVALID, "bad-txns-txouttotal-toolarge"); + } + + // Check for duplicate inputs + set vInOutPoints; + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + if (vInOutPoints.count(txin.prevout)) + return state.DoS(100, error("CheckTransaction() : duplicate inputs"), + REJECT_INVALID, "bad-txns-inputs-duplicate"); + vInOutPoints.insert(txin.prevout); + } + + if (tx.IsCoinBase()) + { +/* MCHN START */ +/* + string reason; + if(!AcceptPermissionsAndCheckForDust(tx,false,reason)) + { + return state.DoS(100, error("CheckTransaction() : coinbase permission script - %s",reason), + REJECT_INVALID, "bad-perm-script"); + } + */ +/* MCHN END */ + if (tx.vin[0].scriptSig.size() < 2 || tx.vin[0].scriptSig.size() > 100) + return state.DoS(100, error("CheckTransaction() : coinbase script size"), + REJECT_INVALID, "bad-cb-length"); + } + else + { + BOOST_FOREACH(const CTxIn& txin, tx.vin) + if (txin.prevout.IsNull()) + return state.DoS(10, error("CheckTransaction() : prevout is null"), + REJECT_INVALID, "bad-txns-prevout-null"); + } + + return true; +} + +CAmount GetMinRelayFee(const CTransaction& tx, unsigned int nBytes, bool fAllowFree) +{ + { + LOCK(mempool.cs); + uint256 hash = tx.GetHash(); + double dPriorityDelta = 0; + CAmount nFeeDelta = 0; + mempool.ApplyDeltas(hash, dPriorityDelta, nFeeDelta); + if (dPriorityDelta > 0 || nFeeDelta > 0) + return 0; + } + + CAmount nMinFee = ::minRelayTxFee.GetFee(nBytes); + + if (fAllowFree) + { + // There is a free transaction area in blocks created by most miners, + // * If we are relaying we allow transactions up to DEFAULT_BLOCK_PRIORITY_SIZE - 1000 + // to be considered to fall into this category. We don't want to encourage sending + // multiple transactions instead of one big transaction to avoid fees. + if (nBytes < (DEFAULT_BLOCK_PRIORITY_SIZE - 1000)) + nMinFee = 0; + } + + if (!MoneyRange(nMinFee)) + nMinFee = MAX_MONEY; + return nMinFee; +} + +bool AcceptToMemoryPool(CTxMemPool& pool, CValidationState &state, const CTransaction &tx, bool fLimitFree, + bool* pfMissingInputs, bool fRejectInsaneFee,bool fAddToWallet) +{ + AssertLockHeld(cs_main); + if (pfMissingInputs) + *pfMissingInputs = false; + if (!CheckTransaction(tx, state)) + return error("AcceptToMemoryPool: : CheckTransaction failed"); + + // Coinbase is only valid in a block, not as a loose transaction + if (tx.IsCoinBase()) + return state.DoS(100, error("AcceptToMemoryPool: : coinbase as individual tx"), + REJECT_INVALID, "coinbase"); + +/* MCHN START */ + bool check_for_dust=true; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + check_for_dust=false; + } + +/* MCHN END */ + + // Rather not work on nonstandard transactions (unless -testnet/-regtest) + string reason; + if (Params().RequireStandard() && !IsStandardTx(tx, reason, check_for_dust))//MCHN + return state.DoS(0, + error("AcceptToMemoryPool : nonstandard transaction: %s", reason), + REJECT_NONSTANDARD, reason); + + // is it already in the memory pool? + uint256 hash = tx.GetHash(); + if (pool.exists(hash)) + return false; + +/* MCHN START */ + if(mc_gState->m_Features->Streams() == 0) + { + if(!AcceptPermissionsAndCheckForDust(tx,false,reason)) + { + return state.DoS(0, + error("AcceptToMemoryPool: : AcceptPermissionsAndCheckForDust failed %s : %s", hash.ToString(),reason), + REJECT_NONSTANDARD, reason); + } + if(!AcceptAssetGenesis(tx,-1,false,reason)) + { + return state.DoS(0, + error("AcceptToMemoryPool: : AcceptAssetGenesis failed %s : %s", hash.ToString(),reason), + REJECT_INVALID, reason); + } + } +/* MCHN END */ + + // Check for conflicts with in-memory transactions + { + LOCK(pool.cs); // protect pool.mapNextTx + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + COutPoint outpoint = tx.vin[i].prevout; + if (pool.mapNextTx.count(outpoint)) + { + LogPrint("mchn","Conflicting with in-memory %s\n",tx.vin[i].ToString().c_str()); + // Disable replacement feature for now + return false; + } + } + } + + { + CCoinsView dummy; + CCoinsViewCache view(&dummy); + + CAmount nValueIn = 0; + { + LOCK(pool.cs); + CCoinsViewMemPool viewMemPool(pcoinsTip, pool); + view.SetBackend(viewMemPool); + + // do we already have it? + if (view.HaveCoins(hash)) + return false; + + // do all inputs exist? + // Note that this does not check for the presence of actual outputs (see the next check for that), + // only helps filling in pfMissingInputs (to determine missing vs spent). + BOOST_FOREACH(const CTxIn txin, tx.vin) { + if (!view.HaveCoins(txin.prevout.hash)) { + LogPrint("mchn","Missing tx (%s)\n",txin.prevout.hash.ToString().c_str()); + if (pfMissingInputs) + *pfMissingInputs = true; +/* MCHN START */ + return false; +/* + return state.Invalid(error("AcceptToMemoryPool : inputs already spent"), + REJECT_DUPLICATE, "bad-txns-inputs-spent"); + */ +/* MCHN END */ + } + } + + // are the actual inputs available? + if (!view.HaveInputs(tx)) +/* MCHN START */ + { + if (!tx.IsCoinBase()) { + for (unsigned int i = 0; i < tx.vin.size(); i++) { + const COutPoint &prevout = tx.vin[i].prevout; + const CCoins* coins = view.AccessCoins(prevout.hash); + if (!coins || !coins->IsAvailable(prevout.n)) { + LogPrint("mchn","Missing coin (%s,%d)\n",prevout.hash.ToString().c_str(),prevout.n); + return state.Invalid(error("AcceptToMemoryPool : inputs already spent"), + REJECT_DUPLICATE, "bad-txns-inputs-spent"); + } + } + } +/* MCHN END */ + return state.Invalid(error("AcceptToMemoryPool : inputs already spent"), + REJECT_DUPLICATE, "bad-txns-inputs-spent"); +/* MCHN START */ + } +/* MCHN END */ + + // Bring the best block into scope + view.GetBestBlock(); + + nValueIn = view.GetValueIn(tx); + + // we have all inputs cached now, so switch back to dummy, so we don't need to keep lock on mempool + view.SetBackend(dummy); + } + + // Check for non-standard pay-to-script-hash in inputs + if (Params().RequireStandard() && !AreInputsStandard(tx, view)) +/* MCHN START */ +// return error("AcceptToMemoryPool: : nonstandard transaction input"); + return state.DoS(0,error("AcceptToMemoryPool: : nonstandard transaction input"),REJECT_NONSTANDARD,"Nonstandard transaction input"); +/* MCHN END */ + + // Check that the transaction doesn't have an excessive number of + // sigops, making it impossible to mine. Since the coinbase transaction + // itself can contain sigops MAX_TX_SIGOPS is less than + // MAX_BLOCK_SIGOPS; we still consider this an invalid rather than + // merely non-standard transaction. + unsigned int nSigOps = GetLegacySigOpCount(tx); + nSigOps += GetP2SHSigOpCount(tx, view); + if (nSigOps > MAX_TX_SIGOPS) + return state.DoS(0, + error("AcceptToMemoryPool : too many sigops %s, %d > %d", + hash.ToString(), nSigOps, MAX_TX_SIGOPS), + REJECT_NONSTANDARD, "bad-txns-too-many-sigops"); + + CAmount nValueOut = tx.GetValueOut(); + CAmount nFees = nValueIn-nValueOut; + double dPriority = view.GetPriority(tx, chainActive.Height()); + + CTxMemPoolEntry entry(tx, nFees, GetTime(), dPriority, chainActive.Height()); + unsigned int nSize = entry.GetTxSize(); + + // Don't accept it if it can't get into a block + CAmount txMinFee = GetMinRelayFee(tx, nSize, true); + if (fLimitFree && nFees < txMinFee) + return state.DoS(0, error("AcceptToMemoryPool : not enough fees %s, %d < %d", + hash.ToString(), nFees, txMinFee), + REJECT_INSUFFICIENTFEE, "insufficient fee"); + + ::minRelayTxFee = CFeeRate(MIN_RELAY_TX_FEE); + + // Require that free transactions have sufficient priority to be mined in the next block. + if (GetBoolArg("-relaypriority", true) && nFees < ::minRelayTxFee.GetFee(nSize) && !AllowFree(view.GetPriority(tx, chainActive.Height() + 1))) { + return state.DoS(0, false, REJECT_INSUFFICIENTFEE, "insufficient priority"); + } + + // Continuously rate-limit free (really, very-low-fee) transactions + // This mitigates 'penny-flooding' -- sending thousands of free transactions just to + // be annoying or make others' transactions take longer to confirm. + if (fLimitFree && nFees < ::minRelayTxFee.GetFee(nSize)) + { + static CCriticalSection csFreeLimiter; + static double dFreeCount; + static int64_t nLastTime; + int64_t nNow = GetTime(); + + LOCK(csFreeLimiter); + + // Use an exponentially decaying ~10-minute window: + dFreeCount *= pow(1.0 - 1.0/600.0, (double)(nNow - nLastTime)); + nLastTime = nNow; + // -limitfreerelay unit is thousand-bytes-per-minute + // At default rate it would take over a month to fill 1GB + if (dFreeCount >= GetArg("-limitfreerelay", 15)*10*1000) + return state.DoS(0, error("AcceptToMemoryPool : free transaction rejected by rate limiter"), + REJECT_INSUFFICIENTFEE, "rate limited free transaction"); + LogPrint("mempool", "Rate limit dFreeCount: %g => %g\n", dFreeCount, dFreeCount+nSize); + dFreeCount += nSize; + } + +/* MCHN START */ + if(MIN_RELAY_TX_FEE != 0) + { +/* MCHN END */ + if (fRejectInsaneFee && nFees > ::minRelayTxFee.GetFee(nSize) * 10000) +/* MCHN START */ +/* + return error("AcceptToMemoryPool: : insane fees %s, %d > %d", + hash.ToString(), + nFees, ::minRelayTxFee.GetFee(nSize) * 10000); + */ + return state.DoS(0, + error("AcceptToMemoryPool: : insane fees %s, %d > %d", + hash.ToString(), + nFees, ::minRelayTxFee.GetFee(nSize) * 10000), + REJECT_INVALID,"Insane fees"); + +/* MCHN START */ + } +/* MCHN END */ + + if(mc_gState->m_Features->Streams()) + { + if(!AcceptMultiChainTransaction(tx,view,-1,false,reason)) + { + return state.DoS(0, + error("AcceptToMemoryPool: : AcceptMultiChainTransaction failed %s : %s", hash.ToString(),reason), + REJECT_NONSTANDARD, reason); + } + } + +/* MCHN END */ + + // Check against previous transactions + // This is done last to help prevent CPU exhaustion denial-of-service attacks. + if (!CheckInputs(tx, state, view, true, STANDARD_SCRIPT_VERIFY_FLAGS, true)) + { +/* MCHN START */ +// return error("AcceptToMemoryPool: : ConnectInputs failed %s", hash.ToString()); + return state.DoS(0,error("AcceptToMemoryPool: : ConnectInputs failed %s", hash.ToString()),REJECT_INVALID,"ConnectInputs failed"); +/* MCHN END */ + } + + + // Check again against just the consensus-critical mandatory script + // verification flags, in case of bugs in the standard flags that cause + // transactions to pass as valid when they're actually invalid. For + // instance the STRICTENC flag was incorrectly allowing certain + // CHECKSIG NOT scripts to pass, even though they were invalid. + // + // There is a similar check in CreateNewBlock() to prevent creating + // invalid blocks, however allowing such transactions into the mempool + // can be exploited as a DoS attack. + if (!CheckInputs(tx, state, view, true, MANDATORY_SCRIPT_VERIFY_FLAGS, true)) + { +/* MCHN START */ +// return error("AcceptToMemoryPool: : BUG! PLEASE REPORT THIS! ConnectInputs failed against MANDATORY but not STANDARD flags %s", hash.ToString()); + return state.DoS(0,error("AcceptToMemoryPool: : BUG! PLEASE REPORT THIS! ConnectInputs failed against MANDATORY but not STANDARD flags %s", hash.ToString()),REJECT_INVALID,"ConnectInputs failed"); +/* MCHN END */ + } + +/* MCHN START */ + + if(mc_gState->m_Features->Streams()) + { + if(!AcceptMultiChainTransaction(tx,view,-1,true,reason)) + { + return state.DoS(0, + error("AcceptToMemoryPool: : AcceptMultiChainTransaction failed %s : %s", hash.ToString(),reason), + REJECT_NONSTANDARD, reason); + } + } + else + { + if(!AcceptPermissionsAndCheckForDust(tx,true,reason)) + { + return state.DoS(0, + error("AcceptToMemoryPool: : AcceptPermissionChanges failed when adding to permission db %s - %s", hash.ToString(),reason), + REJECT_INVALID, reason); + } + if(!AcceptAssetGenesis(tx,-1,true,reason)) + { + return state.DoS(0, + error("AcceptToMemoryPool: : AcceptAssetGenesis failed when adding to asset db %s : %s", hash.ToString(),reason), + REJECT_INVALID, reason); + } + if(!AcceptAssetTransfers(tx, view, reason)) + { + return state.DoS(0, + error("AcceptToMemoryPool: : AcceptAssetTransfers failed %s : %s", hash.ToString(),reason), + REJECT_INVALID, reason); + } + } + +/* MCHN END */ + // Store transaction in memory + pool.addUnchecked(hash, entry); + } + +/* MCHN START */ + if(fAddToWallet) + { + pwalletTxsMain->AddTx(NULL,tx,-1,NULL,-1); + } +/* MCHN END */ + if(fAddToWallet) + { + SyncWithWallets(tx, NULL); + } + + return true; +} + +/** Return transaction in tx, and if it was found inside a block, its hash is placed in hashBlock */ +bool GetTransaction(const uint256 &hash, CTransaction &txOut, uint256 &hashBlock, bool fAllowSlow) +{ + CBlockIndex *pindexSlow = NULL; + { + LOCK(cs_main); + { + if (mempool.lookup(hash, txOut)) + { + return true; + } + } + + if (fTxIndex) { + CDiskTxPos postx; + if (pblocktree->ReadTxIndex(hash, postx)) { + CAutoFile file(OpenBlockFile(postx, true), SER_DISK, CLIENT_VERSION); + if (file.IsNull()) + return error("%s: OpenBlockFile failed", __func__); + CBlockHeader header; + try { + file >> header; + fseek(file.Get(), postx.nTxOffset, SEEK_CUR); + file >> txOut; + } catch (std::exception &e) { + return error("%s : Deserialize or I/O error - %s", __func__, e.what()); + } + hashBlock = header.GetHash(); + if (txOut.GetHash() != hash) + return error("%s : txid mismatch", __func__); + return true; + } + } + + if (fAllowSlow) { // use coin database to locate block that contains transaction, and scan it + int nHeight = -1; + { + CCoinsViewCache &view = *pcoinsTip; + const CCoins* coins = view.AccessCoins(hash); + if (coins) + nHeight = coins->nHeight; + } + if (nHeight > 0) + pindexSlow = chainActive[nHeight]; + } + } + + if (pindexSlow) { + CBlock block; + if (ReadBlockFromDisk(block, pindexSlow)) { + BOOST_FOREACH(const CTransaction &tx, block.vtx) { + if (tx.GetHash() == hash) { + txOut = tx; + hashBlock = pindexSlow->GetBlockHash(); + return true; + } + } + } + } + + return false; +} + + + + + + +////////////////////////////////////////////////////////////////////////////// +// +// CBlock and CBlockIndex +// + +bool WriteBlockToDisk(CBlock& block, CDiskBlockPos& pos) +{ + // Open history file to append + CAutoFile fileout(OpenBlockFile(pos), SER_DISK, CLIENT_VERSION); + if (fileout.IsNull()) + return error("WriteBlockToDisk : OpenBlockFile failed"); + + // Write index header + unsigned int nSize = fileout.GetSerializeSize(block); + fileout << FLATDATA(Params().MessageStart()) << nSize; + + // Write block + long fileOutPos = ftell(fileout.Get()); + if (fileOutPos < 0) + return error("WriteBlockToDisk : ftell failed"); + pos.nPos = (unsigned int)fileOutPos; + fileout << block; + + return true; +} + +bool ReadBlockFromDisk(CBlock& block, const CDiskBlockPos& pos) +{ + block.SetNull(); + + // Open history file to read + CAutoFile filein(OpenBlockFile(pos, true), SER_DISK, CLIENT_VERSION); + if (filein.IsNull()) + return error("ReadBlockFromDisk : OpenBlockFile failed"); + + // Read block + try { + filein >> block; + } + catch (std::exception &e) { + return error("%s : Deserialize or I/O error - %s", __func__, e.what()); + } + + // Check the header + if (!CheckProofOfWork(block.GetHash(), block.nBits)) + return error("ReadBlockFromDisk : Errors in block header"); + + return true; +} + +bool ReadBlockFromDisk(CBlock& block, const CBlockIndex* pindex) +{ + if (!ReadBlockFromDisk(block, pindex->GetBlockPos())) + return false; + if (block.GetHash() != pindex->GetBlockHash()) + return error("ReadBlockFromDisk(CBlock&, CBlockIndex*) : GetHash() doesn't match index"); +/* MCHN START */ + VerifyBlockSignature(&block,true); +/* MCHN END */ + return true; +} + +CAmount GetBlockValue(int nHeight, const CAmount& nFees) +{ +/* MCHN START */ +// CAmount nSubsidy = 50 * COIN; + CAmount nSubsidy = mc_gState->m_NetworkParams->GetInt64Param("initialblockreward");// * COIN; + if(nHeight == 1) + { + if(mc_gState->m_NetworkParams->GetInt64Param("firstblockreward") >= 0) + { + nSubsidy = mc_gState->m_NetworkParams->GetInt64Param("firstblockreward");// * COIN; + } + } +/* MCHN END */ + int halvings = nHeight / Params().SubsidyHalvingInterval(); + + // Force block reward to zero when right shift is undefined. + if (halvings >= 64) + return nFees; + + // Subsidy is cut in half every 210,000 blocks which will occur approximately every 4 years. + nSubsidy >>= halvings; + + return nSubsidy + nFees; +} + +bool IsInitialBlockDownload() +{ + LOCK(cs_main); + if (fImporting || fReindex || chainActive.Height() < Checkpoints::GetTotalBlocksEstimate()) + return true; + static bool lockIBDState = false; + if (lockIBDState) + return false; +/* MCHN START */ +/* + bool state = (chainActive.Height() < pindexBestHeader->nHeight - 24 * 6 || + pindexBestHeader->GetBlockTime() < GetTime() - 24 * 60 * 60); + */ + bool state = (chainActive.Height() < pindexBestHeader->nHeight - 86400 / mc_gState->m_NetworkParams->GetInt64Param("targetblocktime") || + pindexBestHeader->GetBlockTime() < GetTime() - 24 * 60 * 60); +/* MCHN END */ + if (!state) + lockIBDState = true; + return state; +} + +bool fLargeWorkForkFound = false; +bool fLargeWorkInvalidChainFound = false; +CBlockIndex *pindexBestForkTip = NULL, *pindexBestForkBase = NULL; + +void CheckForkWarningConditions() +{ + AssertLockHeld(cs_main); + // Before we get past initial download, we cannot reliably alert about forks + // (we assume we don't get stuck on a fork before the last checkpoint) + if (IsInitialBlockDownload()) + return; + + // If our best fork is no longer within 72 blocks (+/- 12 hours if no one mines it) + // of our head, drop it +/* MCHN START */ +// if (pindexBestForkTip && chainActive.Height() - pindexBestForkTip->nHeight >= 72) + if (pindexBestForkTip && chainActive.Height() - pindexBestForkTip->nHeight >= 43200 / mc_gState->m_NetworkParams->GetInt64Param("targetblocktime")) + pindexBestForkTip = NULL; +/* MCHN END */ + + if (pindexBestForkTip || (pindexBestInvalid && pindexBestInvalid->nChainWork > chainActive.Tip()->nChainWork + (GetBlockProof(*chainActive.Tip()) * 6))) + { + if (!fLargeWorkForkFound && pindexBestForkBase) + { + std::string warning = std::string("'Warning: Large-work fork detected, forking after block ") + + pindexBestForkBase->phashBlock->ToString() + std::string("'"); + CAlert::Notify(warning, true); + } + if (pindexBestForkTip && pindexBestForkBase) + { + LogPrintf("CheckForkWarningConditions: Warning: Large valid fork found\n forking the chain at height %d (%s)\n lasting to height %d (%s).\nChain state database corruption likely.\n", + pindexBestForkBase->nHeight, pindexBestForkBase->phashBlock->ToString(), + pindexBestForkTip->nHeight, pindexBestForkTip->phashBlock->ToString()); + fLargeWorkForkFound = true; + } + else + { + LogPrintf("CheckForkWarningConditions: Warning: Found invalid chain at least ~6 blocks longer than our best chain.\nChain state database corruption likely.\n"); + fLargeWorkInvalidChainFound = true; + } + } + else + { + fLargeWorkForkFound = false; + fLargeWorkInvalidChainFound = false; + } +} + +void CheckForkWarningConditionsOnNewFork(CBlockIndex* pindexNewForkTip) +{ + AssertLockHeld(cs_main); + // If we are on a fork that is sufficiently large, set a warning flag + CBlockIndex* pfork = pindexNewForkTip; + CBlockIndex* plonger = chainActive.Tip(); + while (pfork && pfork != plonger) + { + while (plonger && plonger->nHeight > pfork->nHeight) + plonger = plonger->pprev; + if (pfork == plonger) + break; + pfork = pfork->pprev; + } + + // We define a condition which we should warn the user about as a fork of at least 7 blocks + // who's tip is within 72 blocks (+/- 12 hours if no one mines it) of ours + // We use 7 blocks rather arbitrarily as it represents just under 10% of sustained network + // hash rate operating on the fork. + // or a chain that is entirely longer than ours and invalid (note that this should be detected by both) + // We define it this way because it allows us to only store the highest fork tip (+ base) which meets + // the 7-block condition and from this always have the most-likely-to-cause-warning fork + if (pfork && (!pindexBestForkTip || (pindexBestForkTip && pindexNewForkTip->nHeight > pindexBestForkTip->nHeight)) && + pindexNewForkTip->nChainWork - pfork->nChainWork > (GetBlockProof(*pfork) * 7) && +/* MCHN START */ +// chainActive.Height() - pindexNewForkTip->nHeight < 72) + chainActive.Height() - pindexNewForkTip->nHeight < 43200 / mc_gState->m_NetworkParams->GetInt64Param("targetblocktime")) +/* MCHN END */ + { + pindexBestForkTip = pindexNewForkTip; + pindexBestForkBase = pfork; + } + + CheckForkWarningConditions(); +} + +// Requires cs_main. +void Misbehaving(NodeId pnode, int howmuch) +{ + if (howmuch == 0) + return; + + CNodeState *state = State(pnode); + if (state == NULL) + return; + + state->nMisbehavior += howmuch; + int banscore = GetArg("-banscore", 100); + if (state->nMisbehavior >= banscore && state->nMisbehavior - howmuch < banscore) + { + LogPrintf("Misbehaving: %s (%d -> %d) BAN THRESHOLD EXCEEDED\n", state->name, state->nMisbehavior-howmuch, state->nMisbehavior); + state->fShouldBan = true; + } else + LogPrintf("Misbehaving: %s (%d -> %d)\n", state->name, state->nMisbehavior-howmuch, state->nMisbehavior); +} + +void static InvalidChainFound(CBlockIndex* pindexNew) +{ + if (!pindexBestInvalid || pindexNew->nChainWork > pindexBestInvalid->nChainWork) + pindexBestInvalid = pindexNew; + LogPrintf("InvalidChainFound: invalid block=%s height=%d log2_work=%.8g date=%s\n", + pindexNew->GetBlockHash().ToString(), pindexNew->nHeight, + log(pindexNew->nChainWork.getdouble())/log(2.0), DateTimeStrFormat("%Y-%m-%d %H:%M:%S", + pindexNew->GetBlockTime())); +/* MCHNS START */ + if(chainActive.Height() >= 0) // Crashes if genesis is invalid + { +/* MCHNS END */ + LogPrintf("InvalidChainFound: current best=%s height=%d log2_work=%.8g date=%s\n", + chainActive.Tip()->GetBlockHash().ToString(), chainActive.Height(), log(chainActive.Tip()->nChainWork.getdouble())/log(2.0), + DateTimeStrFormat("%Y-%m-%d %H:%M:%S", chainActive.Tip()->GetBlockTime())); +/* MCHNS START */ + } +/* MCHNS END */ + CheckForkWarningConditions(); +} + +void static InvalidBlockFound(CBlockIndex *pindex, const CValidationState &state) { + int nDoS = 0; + if (state.IsInvalid(nDoS)) { + std::map::iterator it = mapBlockSource.find(pindex->GetBlockHash()); + if (it != mapBlockSource.end() && State(it->second)) { + CBlockReject reject = {state.GetRejectCode(), state.GetRejectReason().substr(0, MAX_REJECT_MESSAGE_LENGTH), pindex->GetBlockHash()}; + State(it->second)->rejects.push_back(reject); + if (nDoS > 0) + Misbehaving(it->second, nDoS); + } + } + if (!state.CorruptionPossible()) { + pindex->nStatus |= BLOCK_FAILED_VALID; + setDirtyBlockIndex.insert(pindex); + setBlockIndexCandidates.erase(pindex); + InvalidChainFound(pindex); + } +} + +void UpdateCoins(const CTransaction& tx, CValidationState &state, CCoinsViewCache &inputs, CTxUndo &txundo, int nHeight) +{ + // mark inputs spent + if (!tx.IsCoinBase()) { + txundo.vprevout.reserve(tx.vin.size()); + BOOST_FOREACH(const CTxIn &txin, tx.vin) { + txundo.vprevout.push_back(CTxInUndo()); + bool ret = inputs.ModifyCoins(txin.prevout.hash)->Spend(txin.prevout, txundo.vprevout.back()); + assert(ret); + } + } + + // add outputs + inputs.ModifyCoins(tx.GetHash())->FromTx(tx, nHeight); +} + +bool CScriptCheck::operator()() { + const CScript &scriptSig = ptxTo->vin[nIn].scriptSig; +/* + printf("VerifyScript %d %d\n",scriptSig.end()-scriptSig.begin(),scriptPubKey.end()-scriptPubKey.begin()); +// mc_DumpSize("sigScript",(unsigned char*)(&scriptSig.begin()[0]),scriptSig.end()-scriptSig.begin(),scriptSig.end()-scriptSig.begin()); +// mc_DumpSize("scriptPubKey",(unsigned char*)(&scriptPubKey.begin()[0]),scriptPubKey.end()-scriptPubKey.begin(),scriptPubKey.end()-scriptPubKey.begin()); + */ + if (!VerifyScript(scriptSig, scriptPubKey, nFlags, CachingTransactionSignatureChecker(ptxTo, nIn, cacheStore), &error)) { + return ::error("CScriptCheck(): %s:%d VerifySignature failed: %s", ptxTo->GetHash().ToString(), nIn, ScriptErrorString(error)); + } + return true; +} + +bool CheckInputs(const CTransaction& tx, CValidationState &state, const CCoinsViewCache &inputs, bool fScriptChecks, unsigned int flags, bool cacheStore, std::vector *pvChecks) +{ + if (!tx.IsCoinBase()) + { + if (pvChecks) + pvChecks->reserve(tx.vin.size()); + + // This doesn't trigger the DoS code on purpose; if it did, it would make it easier + // for an attacker to attempt to split the network. + if (!inputs.HaveInputs(tx)) + return state.Invalid(error("CheckInputs() : %s inputs unavailable", tx.GetHash().ToString())); + + // While checking, GetBestBlock() refers to the parent block. + // This is also true for mempool checks. + CBlockIndex *pindexPrev = mapBlockIndex.find(inputs.GetBestBlock())->second; + int nSpendHeight = pindexPrev->nHeight + 1; + CAmount nValueIn = 0; + CAmount nFees = 0; + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + const COutPoint &prevout = tx.vin[i].prevout; + const CCoins *coins = inputs.AccessCoins(prevout.hash); + assert(coins); + + // If prev is coinbase, check that it's matured + if (coins->IsCoinBase()) { + if (nSpendHeight - coins->nHeight < COINBASE_MATURITY) + return state.Invalid( + error("CheckInputs() : tried to spend coinbase at depth %d", nSpendHeight - coins->nHeight), + REJECT_INVALID, "bad-txns-premature-spend-of-coinbase"); + } + + // Check for negative or overflow input values + nValueIn += coins->vout[prevout.n].nValue; + if (!MoneyRange(coins->vout[prevout.n].nValue) || !MoneyRange(nValueIn)) + return state.DoS(100, error("CheckInputs() : txin values out of range"), + REJECT_INVALID, "bad-txns-inputvalues-outofrange"); + + } + + if (nValueIn < tx.GetValueOut()) + return state.DoS(100, error("CheckInputs() : %s value in (%s) < value out (%s)", + tx.GetHash().ToString(), FormatMoney(nValueIn), FormatMoney(tx.GetValueOut())), + REJECT_INVALID, "bad-txns-in-belowout"); + + // Tally transaction fees + CAmount nTxFee = nValueIn - tx.GetValueOut(); + if (nTxFee < 0) + return state.DoS(100, error("CheckInputs() : %s nTxFee < 0", tx.GetHash().ToString()), + REJECT_INVALID, "bad-txns-fee-negative"); + nFees += nTxFee; + if (!MoneyRange(nFees)) + return state.DoS(100, error("CheckInputs() : nFees out of range"), + REJECT_INVALID, "bad-txns-fee-outofrange"); + + // The first loop above does all the inexpensive checks. + // Only if ALL inputs pass do we perform expensive ECDSA signature checks. + // Helps prevent CPU exhaustion attacks. + + // Skip ECDSA signature verification when connecting blocks + // before the last block chain checkpoint. This is safe because block merkle hashes are + // still computed and checked, and any change will be caught at the next checkpoint. + if (fScriptChecks) { + for (unsigned int i = 0; i < tx.vin.size(); i++) { + const COutPoint &prevout = tx.vin[i].prevout; + const CCoins* coins = inputs.AccessCoins(prevout.hash); + assert(coins); + + // Verify signature + CScriptCheck check(*coins, tx, i, flags, cacheStore); +/* + CScript script2=(*coins).vout[tx.vin[i].prevout.n].scriptPubKey; + CScript::const_iterator pc = script2.begin(); + CScript::const_iterator pend = script2.end(); + opcodetype opcode; + vector vchPushValue; + int count=0; + while(pcpush_back(CScriptCheck()); + check.swap(pvChecks->back()); + } else if (!check()) { + if (flags & STANDARD_NOT_MANDATORY_VERIFY_FLAGS) { + // Check whether the failure was caused by a + // non-mandatory script verification check, such as + // non-standard DER encodings or non-null dummy + // arguments; if so, don't trigger DoS protection to + // avoid splitting the network between upgraded and + // non-upgraded nodes. + CScriptCheck check(*coins, tx, i, + flags & ~STANDARD_NOT_MANDATORY_VERIFY_FLAGS, cacheStore); + if (check()) + return state.Invalid(false, REJECT_NONSTANDARD, strprintf("non-mandatory-script-verify-flag (%s)", ScriptErrorString(check.GetScriptError()))); + } + // Failures of other flags indicate a transaction that is + // invalid in new blocks, e.g. a invalid P2SH. We DoS ban + // such nodes as they are not following the protocol. That + // said during an upgrade careful thought should be taken + // as to the correct behavior - we may want to continue + // peering with non-upgraded nodes even after a soft-fork + // super-majority vote has passed. +/* MCHN START */ +// return state.DoS(100,false, REJECT_INVALID, strprintf("mandatory-script-verify-flag-failed (%s)", ScriptErrorString(check.GetScriptError()))); + return state.DoS(1,false, REJECT_INVALID, strprintf("mandatory-script-verify-flag-failed (%s)", ScriptErrorString(check.GetScriptError()))); +/* MCHN END */ + } + } + } + } + + return true; +} + + + +bool DisconnectBlock(CBlock& block, CValidationState& state, CBlockIndex* pindex, CCoinsViewCache& view, bool* pfClean) +{ + assert(pindex->GetBlockHash() == view.GetBestBlock()); + + if (pfClean) + *pfClean = false; + + bool fClean = true; + + CBlockUndo blockUndo; + CDiskBlockPos pos = pindex->GetUndoPos(); + if (pos.IsNull()) + return error("DisconnectBlock() : no undo data available"); + if (!blockUndo.ReadFromDisk(pos, pindex->pprev->GetBlockHash())) + return error("DisconnectBlock() : failure reading undo data"); + + if (blockUndo.vtxundo.size() + 1 != block.vtx.size()) + return error("DisconnectBlock() : block and undo data inconsistent"); + + // undo transactions in reverse order + for (int i = block.vtx.size() - 1; i >= 0; i--) { + const CTransaction &tx = block.vtx[i]; + uint256 hash = tx.GetHash(); + + // Check that all outputs are available and match the outputs in the block itself + // exactly. Note that transactions with only provably unspendable outputs won't + // have outputs available even in the block itself, so we handle that case + // specially with outsEmpty. + { + CCoins outsEmpty; + CCoinsModifier outs = view.ModifyCoins(hash); + outs->ClearUnspendable(); + + CCoins outsBlock(tx, pindex->nHeight); + // The CCoins serialization does not serialize negative numbers. + // No network rules currently depend on the version here, so an inconsistency is harmless + // but it must be corrected before txout nversion ever influences a network rule. + if (outsBlock.nVersion < 0) + outs->nVersion = outsBlock.nVersion; + if (*outs != outsBlock) + fClean = fClean && error("DisconnectBlock() : added transaction mismatch? database corrupted"); + + // remove outputs + outs->Clear(); + } + + // restore inputs + if (i > 0) { // not coinbases + const CTxUndo &txundo = blockUndo.vtxundo[i-1]; + if (txundo.vprevout.size() != tx.vin.size()) + return error("DisconnectBlock() : transaction and undo data inconsistent"); + for (unsigned int j = tx.vin.size(); j-- > 0;) { + const COutPoint &out = tx.vin[j].prevout; + const CTxInUndo &undo = txundo.vprevout[j]; + CCoinsModifier coins = view.ModifyCoins(out.hash); + if (undo.nHeight != 0) { + // undo data contains height: this is the last output of the prevout tx being spent + if (!coins->IsPruned()) + fClean = fClean && error("DisconnectBlock() : undo data overwriting existing transaction"); + coins->Clear(); + coins->fCoinBase = undo.fCoinBase; + coins->nHeight = undo.nHeight; + coins->nVersion = undo.nVersion; + } else { + if (coins->IsPruned()) + fClean = fClean && error("DisconnectBlock() : undo data adding output to missing transaction"); + } + if (coins->IsAvailable(out.n)) + fClean = fClean && error("DisconnectBlock() : undo data overwriting existing output"); + if (coins->vout.size() < out.n+1) + coins->vout.resize(out.n+1); + coins->vout[out.n] = undo.txout; + } + } + } + + // move best block pointer to prevout block + view.SetBestBlock(pindex->pprev->GetBlockHash()); + + if (pfClean) { + *pfClean = fClean; + return true; + } else { + return fClean; + } +} + +void static FlushBlockFile(bool fFinalize = false) +{ + LOCK(cs_LastBlockFile); + + CDiskBlockPos posOld(nLastBlockFile, 0); + + FILE *fileOld = OpenBlockFile(posOld); + if (fileOld) { + if (fFinalize) + TruncateFile(fileOld, vinfoBlockFile[nLastBlockFile].nSize); + FileCommit(fileOld); + fclose(fileOld); + } + + fileOld = OpenUndoFile(posOld); + if (fileOld) { + if (fFinalize) + TruncateFile(fileOld, vinfoBlockFile[nLastBlockFile].nUndoSize); + FileCommit(fileOld); + fclose(fileOld); + } +} + +bool FindUndoPos(CValidationState &state, int nFile, CDiskBlockPos &pos, unsigned int nAddSize); + +static CCheckQueue scriptcheckqueue(128); + +void ThreadScriptCheck() { + RenameThread("bitcoin-scriptch"); + scriptcheckqueue.Thread(); +} + +static int64_t nTimeVerify = 0; +static int64_t nTimeConnect = 0; +static int64_t nTimeIndex = 0; +static int64_t nTimeCallbacks = 0; +static int64_t nTimeTotal = 0; + +bool ConnectBlock(const CBlock& block, CValidationState& state, CBlockIndex* pindex, CCoinsViewCache& view, bool fJustCheck) +{ + AssertLockHeld(cs_main); + // Check it again in case a previous version let a bad block in + if (!CheckBlock(block, state, !fJustCheck, !fJustCheck)) + return false; + +/* MCHN START */ + uint256 block_hash; + unsigned char miner_address[20]; +/* MCHN START */ + int offset = 80 + 1; + if(block.vtx.size() >= 0xfd) + { + offset+=2; + } + if(block.vtx.size() > 0xffff) + { + offset+=2; + } + int coinbase_offset=offset; +/* MCHN END */ + + block_hash=block.GetHash(); + if(!fJustCheck) + { + if(pindex->pprev) + { + LogPrint("mchn","mchn: Connecting block %s (height %d) ...\n",block.GetHash().ToString().c_str(),pindex->pprev->nHeight+1); + } + else + { + LogPrint("mchn","mchn: Connecting genesis block...\n"); + } + if(!CheckBlockPermissions(block,pindex->pprev,miner_address)) + { + return state.DoS(100, error("ConnectBlock() : invalid permission changes or miner has no permission"), + REJECT_INVALID, "bad-perm-chngs"); + } + } +/* MCHN END */ + + + // verify that the view's current state corresponds to the previous block + uint256 hashPrevBlock = pindex->pprev == NULL ? uint256(0) : pindex->pprev->GetBlockHash(); + assert(hashPrevBlock == view.GetBestBlock()); + +/* MCHN START */ + if(!fJustCheck) + { + mc_gState->m_Permissions->ClearMemPool(); + mc_gState->m_Assets->ClearMemPool(); + } +/* MCHN END */ + + // Special case for the genesis block, skipping connection of its transactions + // (its coinbase is unspendable) +/* MCHN START */ + if (block.GetHash() == Params().HashGenesisBlock()){ + LogPrint("mchn","mchn: Checking block permission transactions for genesis block...\n"); + for (unsigned int i = 0; i < block.vtx.size(); i++) + { + const CTransaction &tx = block.vtx[i]; + string reason; + if(mc_gState->m_Features->Streams()) + { + if(!AcceptMultiChainTransaction(tx,view,offset,true,reason)) + { + return state.DoS(100, error(reason.c_str()), + REJECT_INVALID, "bad-transaction"); + } +// unsigned char *root_stream_name; + int root_stream_name_size; + mc_gState->m_NetworkParams->GetParam("rootstreamname",&root_stream_name_size); + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + root_stream_name_size=0; + } + if(root_stream_name_size) + { + if(pwalletTxsMain) + { + if(mc_gState->m_WalletMode & MC_WMD_TXS) + { + mc_TxEntity entity; + uint256 genesis_hash=block.vtx[0].GetHash(); + entity.Zero(); + + memcpy(entity.m_EntityID,(unsigned char*)&genesis_hash+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + entity.m_EntityType=MC_TET_STREAM | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,0); + entity.m_EntityType=MC_TET_STREAM | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,0); + entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,0); + entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,0); + entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,0); + entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,0); + } + } + } + } + else + { + if(!AcceptPermissionsAndCheckForDust(tx,true,reason)) + { + return state.DoS(100, error(reason.c_str()), + REJECT_INVALID, "bad-transaction"); + return false; + } + } + offset+=tx.GetSerializeSize(SER_NETWORK,tx.nVersion); + } + if(mc_gState->m_Permissions->Commit(miner_address,&block_hash) != 0) + { + return state.DoS(100, error("ConnectBlock() : error on permission commit for the genesis block"), + REJECT_INVALID, "bad-prm-commit"); + } + if(mc_gState->m_Assets->Commit() != 0) + { + return state.DoS(100, error("ConnectBlock() : error on asset commit for the genesis block"), + REJECT_INVALID, "bad-prm-commit"); + } +/* MCHN END */ + view.SetBestBlock(pindex->GetBlockHash()); + return true; + } + + bool fScriptChecks = pindex->nHeight >= Checkpoints::GetTotalBlocksEstimate(); + +/* MCHN START */ + + fScriptChecks &= !fJustCheck; // When miner checks the blocks before submission + // signature verification can fail because of lost send permission + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { +/* MCHN END */ + + // Do not allow blocks that contain transactions which 'overwrite' older transactions, + // unless those are already completely spent. + // If such overwrites are allowed, coinbases and transactions depending upon those + // can be duplicated to remove the ability to spend the first instance -- even after + // being sent to another address. + // See BIP30 and http://r6.ca/blog/20120206T005236Z.html for more information. + // This logic is not necessary for memory pool transactions, as AcceptToMemoryPool + // already refuses previously-known transaction ids entirely. + // This rule was originally applied all blocks whose timestamp was after March 15, 2012, 0:00 UTC. + // Now that the whole chain is irreversibly beyond that time it is applied to all blocks except the + // two in the chain that violate it. This prevents exploiting the issue against nodes in their + // initial block download. + bool fEnforceBIP30 = (!pindex->phashBlock) || // Enforce on CreateNewBlock invocations which don't have a hash. + !((pindex->nHeight==91842 && pindex->GetBlockHash() == uint256("0x00000000000a4d0a398161ffc163c503763b1f4360639393e0e4c8e300e0caec")) || + (pindex->nHeight==91880 && pindex->GetBlockHash() == uint256("0x00000000000743f190a18c5577a3c2d2a1f610ae9601ac046a38084ccb7cd721"))); + if (fEnforceBIP30) { + BOOST_FOREACH(const CTransaction& tx, block.vtx) { + const CCoins* coins = view.AccessCoins(tx.GetHash()); + if (coins && !coins->IsPruned()) + return state.DoS(100, error("ConnectBlock() : tried to overwrite transaction"), + REJECT_INVALID, "bad-txns-BIP30"); + } + } + +/* MCHN START */ + } +/* MCHN END */ + + // BIP16 didn't become active until Apr 1 2012 + int64_t nBIP16SwitchTime = 1333238400; + bool fStrictPayToScriptHash = (pindex->GetBlockTime() >= nBIP16SwitchTime); + + unsigned int flags = fStrictPayToScriptHash ? SCRIPT_VERIFY_P2SH : SCRIPT_VERIFY_NONE; + + // Start enforcing the DERSIG (BIP66) rules, for block.nVersion=3 blocks, when 75% of the network has upgraded: + if (block.nVersion >= 3 && CBlockIndex::IsSuperMajority(3, pindex->pprev, Params().EnforceBlockUpgradeMajority())) { + flags |= SCRIPT_VERIFY_DERSIG; + } + + CBlockUndo blockundo; + + CCheckQueueControl control(fScriptChecks && nScriptCheckThreads ? &scriptcheckqueue : NULL); + + int64_t nTimeStart = GetTimeMicros(); + CAmount nFees = 0; + int nInputs = 0; + unsigned int nSigOps = 0; + + CDiskTxPos pos(pindex->GetBlockPos(), GetSizeOfCompactSize(block.vtx.size())); + std::vector > vPos; + vPos.reserve(block.vtx.size()); + blockundo.vtxundo.reserve(block.vtx.size() - 1); + for (unsigned int i = 0; i < block.vtx.size(); i++) + { + const CTransaction &tx = block.vtx[i]; + + nInputs += tx.vin.size(); + nSigOps += GetLegacySigOpCount(tx); + if (nSigOps > MAX_BLOCK_SIGOPS) + return state.DoS(100, error("ConnectBlock() : too many sigops"), + REJECT_INVALID, "bad-blk-sigops"); + +/* MCHN START */ + LogPrint("mccoin", "COIN: NW Write %s\n", tx.GetHash().ToString().c_str()); +/* MCHN END */ + if (!tx.IsCoinBase()) + { + if (!view.HaveInputs(tx)) + return state.DoS(100, error("ConnectBlock() : inputs missing/spent"), + REJECT_INVALID, "bad-txns-inputs-missingorspent"); + + if (fStrictPayToScriptHash) + { + // Add in sigops done by pay-to-script-hash inputs; + // this is to prevent a "rogue miner" from creating + // an incredibly-expensive-to-validate block. + nSigOps += GetP2SHSigOpCount(tx, view); + if (nSigOps > MAX_BLOCK_SIGOPS) + return state.DoS(100, error("ConnectBlock() : too many sigops"), + REJECT_INVALID, "bad-blk-sigops"); + } + + nFees += view.GetValueIn(tx)-tx.GetValueOut(); + + std::vector vChecks; + if (!CheckInputs(tx, state, view, fScriptChecks, flags, false, nScriptCheckThreads ? &vChecks : NULL)) + return false; + control.Add(vChecks); + +/* MCHN START */ + string reason; + if(!fJustCheck) + { + if(mc_gState->m_Features->Streams()) + { + if(!AcceptMultiChainTransaction(tx,view,offset,true,reason)) + { + return state.DoS(0, + error("AcceptToMemoryPool: : AcceptMultiChainTransaction failed %s : %s", tx.GetHash().ToString(),reason), + REJECT_NONSTANDARD, reason); + } + } + else + { + if(!AcceptPermissionsAndCheckForDust(tx,true,reason)) + { + return state.DoS(0, + error("ConnectBlock: AcceptPermissionChanges failed when adding to permission db %s - %s", tx.GetHash().ToString(),reason), + REJECT_INVALID, reason); + } + if(!AcceptAssetGenesis(tx,offset,true,reason)) + { + return state.DoS(0, + error("ConnectBlock: AcceptAssetGenesis failed when adding to asset db %s : %s", tx.GetHash().ToString(),reason), + REJECT_INVALID, reason); + } + if(!AcceptAssetTransfers(tx, view, reason)) + { + return state.DoS(0, + error("ConnectBlock: AcceptAssetTransfers failed %s : %s", tx.GetHash().ToString(),reason), + REJECT_INVALID, reason); + } + } +/* + if(!AcceptPermissionsAndCheckForDust(tx,true,reason)) + { + return false; + } + if(!AcceptAssetGenesis(tx,offset,true,reason)) + { + return false; + } + if(!AcceptAssetTransfers(tx, view, reason)) + { + return false; + } +*/ + } +/* MCHN END */ + } + + offset+=tx.GetSerializeSize(SER_NETWORK,tx.nVersion); + + CTxUndo undoDummy; + if (i > 0) { + blockundo.vtxundo.push_back(CTxUndo()); + } + UpdateCoins(tx, state, view, i == 0 ? undoDummy : blockundo.vtxundo.back(), pindex->nHeight); + + vPos.push_back(std::make_pair(tx.GetHash(), pos)); + pos.nTxOffset += ::GetSerializeSize(tx, SER_DISK, CLIENT_VERSION); + } + +/* MCHN START */ + for (unsigned int i = 0; i < block.vtx.size(); i++) + { + const CTransaction &tx = block.vtx[i]; + if (tx.IsCoinBase()) + { + string reason; + if(!fJustCheck) + { + if(mc_gState->m_Features->Streams()) + { + if(!AcceptMultiChainTransaction(tx,view,coinbase_offset,true,reason)) + { + return false; + } + } + else + { + if(!AcceptPermissionsAndCheckForDust(tx,true,reason)) + { + return false; + } + } + } + } + } +/* MCHN END */ + + + int64_t nTime1 = GetTimeMicros(); nTimeConnect += nTime1 - nTimeStart; + LogPrint("bench", " - Connect %u transactions: %.2fms (%.3fms/tx, %.3fms/txin) [%.2fs]\n", (unsigned)block.vtx.size(), 0.001 * (nTime1 - nTimeStart), 0.001 * (nTime1 - nTimeStart) / block.vtx.size(), nInputs <= 1 ? 0 : 0.001 * (nTime1 - nTimeStart) / (nInputs-1), nTimeConnect * 0.000001); + + if (block.vtx[0].GetValueOut() > GetBlockValue(pindex->nHeight, nFees)) + return state.DoS(100, + error("ConnectBlock() : coinbase pays too much (actual=%d vs limit=%d)", + block.vtx[0].GetValueOut(), GetBlockValue(pindex->nHeight, nFees)), + REJECT_INVALID, "bad-cb-amount"); + + if (!control.Wait()) + return state.DoS(100, false); + int64_t nTime2 = GetTimeMicros(); nTimeVerify += nTime2 - nTimeStart; + LogPrint("bench", " - Verify %u txins: %.2fms (%.3fms/txin) [%.2fs]\n", nInputs - 1, 0.001 * (nTime2 - nTimeStart), nInputs <= 1 ? 0 : 0.001 * (nTime2 - nTimeStart) / (nInputs-1), nTimeVerify * 0.000001); + + if (fJustCheck) + return true; + + // Write undo information to disk + if (pindex->GetUndoPos().IsNull() || !pindex->IsValid(BLOCK_VALID_SCRIPTS)) + { + if (pindex->GetUndoPos().IsNull()) { + CDiskBlockPos pos; + if (!FindUndoPos(state, pindex->nFile, pos, ::GetSerializeSize(blockundo, SER_DISK, CLIENT_VERSION) + 40)) + return error("ConnectBlock() : FindUndoPos failed"); + if (!blockundo.WriteToDisk(pos, pindex->pprev->GetBlockHash())) + return state.Abort("Failed to write undo data"); + + // update nUndoPos in block index + pindex->nUndoPos = pos.nPos; + pindex->nStatus |= BLOCK_HAVE_UNDO; + } + + pindex->RaiseValidity(BLOCK_VALID_SCRIPTS); + setDirtyBlockIndex.insert(pindex); + } + + if (fTxIndex) + if (!pblocktree->WriteTxIndex(vPos)) + return state.Abort("Failed to write transaction index"); + +/* MCHN START */ + LogPrint("mchn","mchn: Committing permission changes for block %d...\n",mc_gState->m_Permissions->m_Block+1); + if(mc_gState->m_Permissions->Commit(miner_address,&block_hash) != 0) + { + return state.DoS(100, error("ConnectBlock() : error on permission commit"), + REJECT_INVALID, "bad-prm-commit"); + } + if(mc_gState->m_Assets->Commit() != 0) + { + return state.DoS(100, error("ConnectBlock() : error on asset commit"), + REJECT_INVALID, "bad-prm-commit"); + } + + setBlockTransactions[mc_gState->m_Permissions->m_Block%MC_TXSET_BLOCKS].clear(); + for (unsigned int i = 0; i < block.vtx.size(); i++) + { + setBlockTransactions[mc_gState->m_Permissions->m_Block%MC_TXSET_BLOCKS].insert(block.vtx[i].GetHash()); + } + +/* MCHN END */ + + // add this block to the view's block chain + view.SetBestBlock(pindex->GetBlockHash()); + + int64_t nTime3 = GetTimeMicros(); nTimeIndex += nTime3 - nTime2; + LogPrint("bench", " - Index writing: %.2fms [%.2fs]\n", 0.001 * (nTime3 - nTime2), nTimeIndex * 0.000001); + + // Watch for changes to the previous coinbase transaction. + static uint256 hashPrevBestCoinBase; + g_signals.UpdatedTransaction(hashPrevBestCoinBase); + hashPrevBestCoinBase = block.vtx[0].GetHash(); + + int64_t nTime4 = GetTimeMicros(); nTimeCallbacks += nTime4 - nTime3; + LogPrint("bench", " - Callbacks: %.2fms [%.2fs]\n", 0.001 * (nTime4 - nTime3), nTimeCallbacks * 0.000001); + + return true; +} + +enum FlushStateMode { + FLUSH_STATE_IF_NEEDED, + FLUSH_STATE_PERIODIC, + FLUSH_STATE_ALWAYS +}; + +/** + * Update the on-disk chain state. + * The caches and indexes are flushed if either they're too large, forceWrite is set, or + * fast is not set and it's been a while since the last write. + */ +bool static FlushStateToDisk(CValidationState &state, FlushStateMode mode) { + LOCK(cs_main); + static int64_t nLastWrite = 0; + try { + if ((mode == FLUSH_STATE_ALWAYS) || + ((mode == FLUSH_STATE_PERIODIC || mode == FLUSH_STATE_IF_NEEDED) && pcoinsTip->GetCacheSize() > nCoinCacheSize) || + (mode == FLUSH_STATE_PERIODIC && GetTimeMicros() > nLastWrite + DATABASE_WRITE_INTERVAL * 1000000)) { + // Typical CCoins structures on disk are around 100 bytes in size. + // Pushing a new one to the database can cause it to be written + // twice (once in the log, and once in the tables). This is already + // an overestimation, as most will delete an existing entry or + // overwrite one. Still, use a conservative safety factor of 2. + if (!CheckDiskSpace(100 * 2 * 2 * pcoinsTip->GetCacheSize())) + return state.Error("out of disk space"); + // First make sure all block and undo data is flushed to disk. + FlushBlockFile(); + // Then update all block file information (which may refer to block and undo files). + bool fileschanged = false; + for (set::iterator it = setDirtyFileInfo.begin(); it != setDirtyFileInfo.end(); ) { + if (!pblocktree->WriteBlockFileInfo(*it, vinfoBlockFile[*it])) { + return state.Abort("Failed to write to block index"); + } + fileschanged = true; + setDirtyFileInfo.erase(it++); + } + if (fileschanged && !pblocktree->WriteLastBlockFile(nLastBlockFile)) { + return state.Abort("Failed to write to block index"); + } + for (set::iterator it = setDirtyBlockIndex.begin(); it != setDirtyBlockIndex.end(); ) { + if (!pblocktree->WriteBlockIndex(CDiskBlockIndex(*it))) { + return state.Abort("Failed to write to block index"); + } + setDirtyBlockIndex.erase(it++); + } + pblocktree->Sync(); + // Finally flush the chainstate (which may refer to block index entries). + if (!pcoinsTip->Flush()) + return state.Abort("Failed to write to coin database"); + // Update best block in wallet (so we can detect restored wallets). + if (mode != FLUSH_STATE_IF_NEEDED) { + g_signals.SetBestChain(chainActive.GetLocator()); + } + nLastWrite = GetTimeMicros(); + } + } catch (const std::runtime_error& e) { + return state.Abort(std::string("System error while flushing: ") + e.what()); + } + return true; +} + +void FlushStateToDisk() { + CValidationState state; + FlushStateToDisk(state, FLUSH_STATE_ALWAYS); +} + +/** Update chainActive and related internal data structures. */ +void static UpdateTip(CBlockIndex *pindexNew) { + chainActive.SetTip(pindexNew); + + // New best block + nTimeBestReceived = GetTime(); + mempool.AddTransactionsUpdated(1); + + LogPrintf("UpdateTip: new best=%s height=%d log2_work=%.8g tx=%lu date=%s progress=%f cache=%u\n", + chainActive.Tip()->GetBlockHash().ToString(), chainActive.Height(), log(chainActive.Tip()->nChainWork.getdouble())/log(2.0), (unsigned long)chainActive.Tip()->nChainTx, + DateTimeStrFormat("%Y-%m-%d %H:%M:%S", chainActive.Tip()->GetBlockTime()), + Checkpoints::GuessVerificationProgress(chainActive.Tip()), (unsigned int)pcoinsTip->GetCacheSize()); + + cvBlockChange.notify_all(); + + // Check the version of the last 100 blocks to see if we need to upgrade: + static bool fWarned = false; + if (!IsInitialBlockDownload() && !fWarned) + { + int nUpgraded = 0; + const CBlockIndex* pindex = chainActive.Tip(); + for (int i = 0; i < 100 && pindex != NULL; i++) + { + if (pindex->nVersion > CBlock::CURRENT_VERSION) + ++nUpgraded; + pindex = pindex->pprev; + } + if (nUpgraded > 0) + LogPrintf("SetBestChain: %d of last 100 blocks above version %d\n", nUpgraded, (int)CBlock::CURRENT_VERSION); + if (nUpgraded > 100/2) + { + // strMiscWarning is read by GetWarnings(), called by Qt and the JSON-RPC code to warn the user: + strMiscWarning = _("Warning: This version is obsolete, upgrade required!"); + CAlert::Notify(strMiscWarning, true); + fWarned = true; + } + } +} + +/** Disconnect chainActive's tip. */ +bool static DisconnectTip(CValidationState &state) { + CBlockIndex *pindexDelete = chainActive.Tip(); + assert(pindexDelete); + mempool.check(pcoinsTip); + // Read block from disk. + CBlock block; + if (!ReadBlockFromDisk(block, pindexDelete)) + return state.Abort("Failed to read block"); + // Apply the block atomically to the chain state. + int64_t nStart = GetTimeMicros(); + { + CCoinsViewCache view(pcoinsTip); + if (!DisconnectBlock(block, state, pindexDelete, view)) + return error("DisconnectTip() : DisconnectBlock %s failed", pindexDelete->GetBlockHash().ToString()); + assert(view.Flush()); + } + LogPrint("bench", "- Disconnect block: %.2fms\n", (GetTimeMicros() - nStart) * 0.001); + // Write the chain state to disk, if necessary. + if (!FlushStateToDisk(state, FLUSH_STATE_ALWAYS))// MCHN was FLUSH_STATE_IF_NEEDED + return false; + +/* MCHN START */ + int old_height=pindexDelete->nHeight; + mempool.defragmentHashList(); + int new_txs=mempool.hashList->m_Count; + LogPrint("mchn","mchn: Disconnecting block %s (height %d) from permission DB (%d transactions in mempool)\n",pindexDelete->GetBlockHash().ToString(),old_height,new_txs); + mempool.shiftHashList(block.vtx.size()); + setBlockTransactions[old_height%MC_TXSET_BLOCKS].clear(); + mc_gState->m_Permissions->RollBack(old_height-1); + mc_gState->m_Assets->RollBack(old_height-1); + if(mc_gState->m_WalletMode & MC_WMD_TXS) + { + pwalletTxsMain->RollBack(NULL,old_height-1); + } +/* MCHN END */ + + // Resurrect mempool transactions from the disconnected block. + BOOST_FOREACH(const CTransaction &tx, block.vtx) { + // ignore validation errors in resurrected transactions + list removed; + CValidationState stateDummy; + if (tx.IsCoinBase() || !AcceptToMemoryPool(mempool, stateDummy, tx, false, NULL)) + { + mempool.remove(tx, removed, true, "resurrection"); + } + } +/* MCHN START */ + int new_shift=mempool.hashList->m_Count-new_txs; +/* MCHN END */ + mempool.removeCoinbaseSpends(pcoinsTip, pindexDelete->nHeight); +/* MCHN START */ +// mempool.defragmentHashList(); + ReplayMemPool(mempool,new_shift,true); + mempool.defragmentHashList(); +/* MCHN END */ + + mempool.check(pcoinsTip); + // Update chainActive and related variables. + UpdateTip(pindexDelete->pprev); + // Let wallets know transactions went from 1-confirmed to + // 0-confirmed or conflicted: + BOOST_FOREACH(const CTransaction &tx, block.vtx) { + SyncWithWallets(tx, NULL); + } + return true; +} + +static int64_t nTimeReadFromDisk = 0; +static int64_t nTimeConnectTotal = 0; +static int64_t nTimeFlush = 0; +static int64_t nTimeChainState = 0; +static int64_t nTimePostConnect = 0; + +/** + * Connect a new block to chainActive. pblock is either NULL or a pointer to a CBlock + * corresponding to pindexNew, to bypass loading it again from disk. + */ +bool static ConnectTip(CValidationState &state, CBlockIndex *pindexNew, CBlock *pblock) { + assert(pindexNew->pprev == chainActive.Tip()); + mempool.check(pcoinsTip); + // Read block from disk. + int64_t nTime1 = GetTimeMicros(); + CBlock block; + if (!pblock) { + if (!ReadBlockFromDisk(block, pindexNew)) + return state.Abort("Failed to read block"); + pblock = █ + } + // Apply the block atomically to the chain state. + int64_t nTime2 = GetTimeMicros(); nTimeReadFromDisk += nTime2 - nTime1; + int64_t nTime3; + LogPrint("bench", " - Load block from disk: %.2fms [%.2fs]\n", (nTime2 - nTime1) * 0.001, nTimeReadFromDisk * 0.000001); + { + CCoinsViewCache view(pcoinsTip); + CInv inv(MSG_BLOCK, pindexNew->GetBlockHash()); + bool rv = ConnectBlock(*pblock, state, pindexNew, view); + g_signals.BlockChecked(*pblock, state); + if (!rv) { + if (state.IsInvalid()) + InvalidBlockFound(pindexNew, state); + return error("ConnectTip() : ConnectBlock %s failed", pindexNew->GetBlockHash().ToString()); + } + mapBlockSource.erase(inv.hash); + nTime3 = GetTimeMicros(); nTimeConnectTotal += nTime3 - nTime2; + LogPrint("bench", " - Connect total: %.2fms [%.2fs]\n", (nTime3 - nTime2) * 0.001, nTimeConnectTotal * 0.000001); + assert(view.Flush()); + } + int64_t nTime4 = GetTimeMicros(); nTimeFlush += nTime4 - nTime3; + LogPrint("bench", " - Flush: %.2fms [%.2fs]\n", (nTime4 - nTime3) * 0.001, nTimeFlush * 0.000001); + // Write the chain state to disk, if necessary. + if (!FlushStateToDisk(state, FLUSH_STATE_ALWAYS))// MCHN was FLUSH_STATE_IF_NEEDED + return false; + int64_t nTime5 = GetTimeMicros(); nTimeChainState += nTime5 - nTime4; + LogPrint("bench", " - Writing chainstate: %.2fms [%.2fs]\n", (nTime5 - nTime4) * 0.001, nTimeChainState * 0.000001); + // Remove conflicting transactions from the mempool. + list txConflicted; + mempool.removeForBlock(pblock->vtx, pindexNew->nHeight, txConflicted); + mempool.check(pcoinsTip); +/* MCHN START */ + BOOST_FOREACH(const CTransaction &tx, pblock->vtx) { + EraseOrphanTx(tx.GetHash()); + } + + LogPrint("wallet","wtxs: Committing block %d\n",pindexNew->nHeight); + + int err=MC_ERR_NOERROR; + err=pwalletTxsMain->BeforeCommit(NULL); + if(err) + { + return error("ConnectTip() : ConnectBlock %s failed, Wtxs BeforeCommit, error: %d", pindexNew->GetBlockHash().ToString(),err); + } + CDiskTxPos pos(pindexNew->GetBlockPos(), GetSizeOfCompactSize(pblock->vtx.size())); + for (unsigned int i = 0; i < pblock->vtx.size(); i++) + { + const CTransaction &tx = pblock->vtx[i]; + err=pwalletTxsMain->AddTx(NULL,tx,pindexNew->nHeight,&pos,i); + if(err) + { + return error("ConnectTip() : ConnectBlock %s failed, Wtxs AddTx %s, error: %d", pindexNew->GetBlockHash().ToString(),tx.GetHash().ToString(),err); + } + pos.nTxOffset += ::GetSerializeSize(tx, SER_DISK, CLIENT_VERSION); + } + err=pwalletTxsMain->Commit(NULL); + if(err) + { + return error("ConnectTip() : ConnectBlock %s failed, Wtxs Commit, error: %d", pindexNew->GetBlockHash().ToString(),err); + } + err=pwalletTxsMain->CleanUpAfterBlock(NULL,pindexNew->nHeight,pindexNew->nHeight-1); + if(err) + { + return error("ConnectTip() : ConnectBlock %s failed, Wtxs CleanUpAfterBlock, error: %d", pindexNew->GetBlockHash().ToString(),err); + } + +/* MCHN END */ + // Update chainActive & related variables. + UpdateTip(pindexNew); + // Tell wallet about transactions that went from mempool + // to conflicted: + BOOST_FOREACH(const CTransaction &tx, txConflicted) { + SyncWithWallets(tx, NULL); + } + // ... and about transactions that got confirmed: +/* MCHN START */ + VerifyBlockSignature(pblock,false); +/* MCHN END */ + + BOOST_FOREACH(const CTransaction &tx, pblock->vtx) { + SyncWithWallets(tx, pblock); + } + +/* MCHN START */ +// CTransaction emptyTx; // Triggering wallet optimization +// SyncWithWallets(emptyTx, pblock); +/* MCHN END */ + + int64_t nTime6 = GetTimeMicros(); nTimePostConnect += nTime6 - nTime5; nTimeTotal += nTime6 - nTime1; + LogPrint("bench", " - Connect postprocess: %.2fms [%.2fs]\n", (nTime6 - nTime5) * 0.001, nTimePostConnect * 0.000001); + LogPrint("bench", "- Connect block: %.2fms [%.2fs]\n", (nTime6 - nTime1) * 0.001, nTimeTotal * 0.000001); + return true; +} + +/** + * Return the tip of the chain with the most work in it, that isn't + * known to be invalid (it's however far from certain to be valid). + */ +static CBlockIndex* FindMostWorkChain() { + do { + CBlockIndex *pindexNew = NULL; + + // Find the best candidate header. + { +/* MCHN START */ + bool take_it=false; + uint32_t max_work,work; + int max_count; + if(setBlockIndexCandidates.size() > 1) + { + max_work=0; + std::set::iterator it; + for (it = setBlockIndexCandidates.begin(); it != setBlockIndexCandidates.end(); ++it) + { + CBlockIndex* pindex=*it; + work=(uint32_t)mc_GetLE(&(pindex->nChainWork),32); + if(work > max_work) + { + max_work=work; + } + } + max_count=0; + for (it = setBlockIndexCandidates.begin(); it != setBlockIndexCandidates.end(); ++it) + { + CBlockIndex* pindex=*it; + work=(uint32_t)mc_GetLE(&(pindex->nChainWork),32); + if(work == max_work) + { + max_count++; + } + } + if(max_count>1) + { + take_it=true; + } + + if(take_it) + { + LogPrint("mchn","mchn: Choosing chain from %d candidates, current height: %d\n",(int)setBlockIndexCandidates.size(),chainActive.Tip()->nHeight); + for (it = setBlockIndexCandidates.begin(); it != setBlockIndexCandidates.end(); ++it) + { + CBlockIndex* pindex=*it; + work=(uint32_t)mc_GetLE(&(pindex->nChainWork),32); + LogPrint("mchn","mchn: Forked block index: %s, work: %d, height: %d, mined-by-me: %d, can-mine: %d\n",pindex->GetBlockHash().ToString().c_str(), + work, pindex->nHeight,pindex->nHeightMinedByMe,pindex->nCanMine); + } + } + } + + set setTempBlockIndexCandidates; + std::set::iterator fit; + for (fit = setBlockIndexCandidates.begin(); fit != setBlockIndexCandidates.end(); ++fit) + { + setTempBlockIndexCandidates.insert(*fit); + } + +/* MCHN END */ +/* + std::set::reverse_iterator it = setBlockIndexCandidates.rbegin(); + if (it == setBlockIndexCandidates.rend()) + return NULL; + */ + std::set::reverse_iterator it = setTempBlockIndexCandidates.rbegin(); + if (it == setTempBlockIndexCandidates.rend()) + return NULL; + pindexNew = *it; +/* MCHN START */ + if(take_it) + { + CBlockIndex* pindex=*it; + work=(uint32_t)mc_GetLE(&(pindex->nChainWork),32); + LogPrint("mchn","mchn: Selected forked block index: %s, Active chain tip: %s\n",pindex->GetBlockHash().ToString().c_str(), + chainActive.Tip()->GetBlockHash().ToString().c_str()); + + } +/* MCHN END */ + } + + // Check whether all blocks on the path between the currently active chain and the candidate are valid. + // Just going until the active chain is an optimization, as we know all blocks in it are valid already. + CBlockIndex *pindexTest = pindexNew; + bool fInvalidAncestor = false; + while (pindexTest && !chainActive.Contains(pindexTest)) { + assert(pindexTest->nStatus & BLOCK_HAVE_DATA); + assert(pindexTest->nChainTx || pindexTest->nHeight == 0); + if (pindexTest->nStatus & BLOCK_FAILED_MASK) { + // Candidate has an invalid ancestor, remove entire chain from the set. + if (pindexBestInvalid == NULL || pindexNew->nChainWork > pindexBestInvalid->nChainWork) + pindexBestInvalid = pindexNew; + CBlockIndex *pindexFailed = pindexNew; + while (pindexTest != pindexFailed) { + pindexFailed->nStatus |= BLOCK_FAILED_CHILD; + setBlockIndexCandidates.erase(pindexFailed); + pindexFailed = pindexFailed->pprev; + } + setBlockIndexCandidates.erase(pindexTest); + fInvalidAncestor = true; + break; + } + pindexTest = pindexTest->pprev; + } + if (!fInvalidAncestor) + return pindexNew; + } while(true); +} + +/** Delete all entries in setBlockIndexCandidates that are worse than the current tip. */ +static void PruneBlockIndexCandidates() { + // Note that we can't delete the current block itself, as we may need to return to it later in case a + // reorganization to a better block fails. + std::set::iterator it = setBlockIndexCandidates.begin(); + while (it != setBlockIndexCandidates.end() && setBlockIndexCandidates.value_comp()(*it, chainActive.Tip())) { + setBlockIndexCandidates.erase(it++); + } + // Either the current tip or a successor of it we're working towards is left in setBlockIndexCandidates. + assert(!setBlockIndexCandidates.empty()); +} + +/* MCHN START */ + +void UpdateChainMiningStatus(const CBlock &block,CBlockIndex *pindexNew) +{ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + std::vector vchPubKey=std::vector (block.vSigner+1, block.vSigner+1+block.vSigner[0]); + CPubKey pubKeyOut(vchPubKey); + CKeyID keyID=pubKeyOut.GetID(); + CKey key; + pindexNew->nCanMine=mc_gState->m_Permissions->CanMine(NULL,keyID.begin()); + if(pwalletMain) + { + if(pwalletMain->GetKey(keyID,key)) + { + pindexNew->nHeightMinedByMe=pindexNew->nHeight; + if(pindexNew->pprev) + { + pindexNew->nCanMine=pindexNew->pprev->nCanMine; + } + else + { + pindexNew->nCanMine=MC_PTP_MINE; + } + if(mc_gState->m_Permissions->GetActiveMinerCount()m_Permissions->m_MinerCount) + { + pindexNew->nCanMine=0; + } + } + else + { + if(pindexNew->pprev) + { + pindexNew->nHeightMinedByMe=pindexNew->pprev->nHeightMinedByMe; + pindexNew->nCanMine=pindexNew->pprev->nCanMine; + if(pindexNew->nHeightMinedByMe+mc_gState->m_Permissions->m_MinerCount-mc_gState->m_Permissions->GetActiveMinerCount() > pindexNew->nHeight) + { + pindexNew->nCanMine=0; + } + } + else + { + pindexNew->nCanMine=0; + } + } + } + if(pindexNew->pprev) + { + LogPrint("mchn","mchn: New block index: %s, prev: %s, height: %d, mined-by-me: %d, can-mine: %d\n",block.GetHash().ToString().c_str(), + pindexNew->pprev->GetBlockHash().ToString().c_str(), + pindexNew->nHeight,pindexNew->nHeightMinedByMe,pindexNew->nCanMine); + } + } +} +/* MCHN END */ + +/** + * Try to make some progress towards making pindexMostWork the active block. + * pblock is either NULL or a pointer to a CBlock corresponding to pindexMostWork. + */ +static bool ActivateBestChainStep(CValidationState &state, CBlockIndex *pindexMostWork, CBlock *pblock) { + AssertLockHeld(cs_main); + bool fInvalidFound = false; + const CBlockIndex *pindexOldTip = chainActive.Tip(); + const CBlockIndex *pindexFork = chainActive.FindFork(pindexMostWork); + + // Disconnect active blocks which are no longer in the best chain. + while (chainActive.Tip() && chainActive.Tip() != pindexFork) { + if (!DisconnectTip(state)) + return false; + } + + // Build list of new blocks to connect. + std::vector vpindexToConnect; + bool fContinue = true; + int nHeight = pindexFork ? pindexFork->nHeight : -1; + while (fContinue && nHeight != pindexMostWork->nHeight) { + // Don't iterate the entire list of potential improvements toward the best tip, as we likely only need + // a few blocks along the way. + int nTargetHeight = std::min(nHeight + 32, pindexMostWork->nHeight); + vpindexToConnect.clear(); + vpindexToConnect.reserve(nTargetHeight - nHeight); + CBlockIndex *pindexIter = pindexMostWork->GetAncestor(nTargetHeight); + while (pindexIter && pindexIter->nHeight != nHeight) { + vpindexToConnect.push_back(pindexIter); + pindexIter = pindexIter->pprev; + } + nHeight = nTargetHeight; + // Connect new blocks. + BOOST_REVERSE_FOREACH(CBlockIndex *pindexConnect, vpindexToConnect) { + if (!ConnectTip(state, pindexConnect, pindexConnect == pindexMostWork ? pblock : NULL)) { + if (state.IsInvalid()) { + // The block violates a consensus rule. + if (!state.CorruptionPossible()) + InvalidChainFound(vpindexToConnect.back()); + state = CValidationState(); + fInvalidFound = true; + fContinue = false; + break; + } else { + // A system error occurred (disk space, database error, ...). + return false; + } + } else { + PruneBlockIndexCandidates(); + if (!pindexOldTip || chainActive.Tip()->nChainWork > pindexOldTip->nChainWork) { + // We're in a better position than we were. Return temporarily to release the lock. + fContinue = false; + break; + } + } + + } + } + +/* MCHN START */ + if (!fInvalidFound) + { + mc_gState->m_Permissions->ClearMemPool(); + mc_gState->m_Assets->ClearMemPool(); + + ReplayMemPool(mempool,0,true); + mempool.defragmentHashList(); + + if(pwalletMain) + { + pwalletMain->ReacceptWalletTransactions(); // Some wallet transactions may become invalid in reorg + // Some may become invalid if not confirmed in time + } + +/* MCHN START */ + if(pwalletMain) + { + CTransaction emptyTx; // Triggering wallet optimization + SyncWithWallets(emptyTx, pblock); + } +/* MCHN END */ + + } +/* MCHN END */ + + // Callbacks/notifications for a new best chain. + if (fInvalidFound) + CheckForkWarningConditionsOnNewFork(vpindexToConnect.back()); + else + CheckForkWarningConditions(); + +/* MCHN START */ + + if (fInvalidFound) + { + if(chainActive.Height() < 0) // Error on genesis block + { + return false; + } + } + +/* MCHN END */ + + return true; +} + +/** + * Make the best chain active, in multiple steps. The result is either failure + * or an activated best chain. pblock is either NULL or a pointer to a block + * that is already loaded (to avoid loading it again from disk). + */ +bool ActivateBestChain(CValidationState &state, CBlock *pblock) { + CBlockIndex *pindexNewTip = NULL; + CBlockIndex *pindexMostWork = NULL; + int attempt=0; + do { + boost::this_thread::interruption_point(); + + uint32_t nCanMine; + + bool fInitialDownload; + { + LOCK(cs_main); + +/* MCHN START */ + if(pwalletMain) + { + CPubKey pubkey; + if(chainActive.Tip()) + { + chainActive.Tip()->nCanMine=pwalletMain->GetKeyFromAddressBook(pubkey,MC_PTP_MINE) ? MC_PTP_MINE : 0; + if(chainActive.Tip()->nHeightMinedByMe == 0) // Can happen if several blocks received in the wrong order + { + if(chainActive.Tip()->pprev) + { + chainActive.Tip()->nHeightMinedByMe=chainActive.Tip()->pprev->nHeightMinedByMe; + } + } + } + } +/* MCHN END */ + + pindexMostWork = FindMostWorkChain(); + // Whether we have anything to do at all. + if (pindexMostWork == NULL || pindexMostWork == chainActive.Tip()) + return true; + + + nCanMine=pindexMostWork->nCanMine; +/* MCHN START */ + if(pindexMostWork->pprev != chainActive.Tip()) + { + if(chainActive.Tip()) + { + LogPrint("mchn","Possible reorg: %d %d->%d\n",attempt,chainActive.Tip()->nHeight,pindexMostWork->nHeight); + if(chainActive.Tip()->nHeight == pindexMostWork->nHeight) + { + LogPrint("mchn","Same-height reorg: %d %d(%d)->%d(%d)\n",attempt,chainActive.Tip()->nCanMine,chainActive.Tip()->nHeight-chainActive.Tip()->nHeightMinedByMe, + pindexMostWork->nCanMine,pindexMostWork->nHeight-pindexMostWork->nHeightMinedByMe); + } + } + attempt++; + } +/* MCHN END */ + + if (!ActivateBestChainStep(state, pindexMostWork, pblock && pblock->GetHash() == pindexMostWork->GetBlockHash() ? pblock : NULL)) + return false; + +/* MCHN START */ + if(pindexMostWork == chainActive.Tip()) + { + if(pwalletMain) + { + CPubKey pubkey; + chainActive.Tip()->nCanMine=pwalletMain->GetKeyFromAddressBook(pubkey,MC_PTP_MINE) ? MC_PTP_MINE : 0; + + LogPrint("mchn","mchn: Chain activated: block: %s (height %d), can-mine: %d\n", + chainActive.Tip()->GetBlockHash().ToString().c_str(), chainActive.Tip()->nHeight,chainActive.Tip()->nCanMine); + + if(nCanMine != chainActive.Tip()->nCanMine) + { +// if(!pwalletMain->GetKeyFromAddressBook(pubkey,MC_PTP_MINE)) + { + LogPrint("mchn","mchn: Wallet mine permission changed on block: %s (height %d), reactivating best chain\n", + chainActive.Tip()->GetBlockHash().ToString().c_str(), chainActive.Tip()->nHeight); + pindexMostWork=NULL; + continue; + } + } + } + } +/* MCHN END */ + + pindexNewTip = chainActive.Tip(); + fInitialDownload = IsInitialBlockDownload(); + } + // When we reach this point, we switched to a new tip (stored in pindexNewTip). + + // Notifications/callbacks that can run without cs_main + if (!fInitialDownload) { + uint256 hashNewTip = pindexNewTip->GetBlockHash(); + // Relay inventory, but don't relay old inventory during initial block download. + int nBlockEstimate = Checkpoints::GetTotalBlocksEstimate(); + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + if (chainActive.Height() > (pnode->nStartingHeight != -1 ? pnode->nStartingHeight - 2000 : nBlockEstimate)) + pnode->PushInventory(CInv(MSG_BLOCK, hashNewTip)); + } + // Notify external listeners about the new tip. + uiInterface.NotifyBlockTip(hashNewTip); + } + } while(pindexMostWork != chainActive.Tip()); + + // Write changes periodically to disk, after relay. + if (!FlushStateToDisk(state, FLUSH_STATE_PERIODIC)) { + return false; + } + + return true; +} + +/* MCHN START */ + +void ClearMemPools() +{ + { + LOCK(cs_main); + mempool.clear(); + mc_gState->m_Permissions->ClearMemPool(); + mc_gState->m_Assets->ClearMemPool(); + LogPrintf("mempool cleared\n"); + } +} + +string SetLastBlock(uint256 hash) +{ + { + LOCK(cs_main); + + CValidationState state; + if(hash == 0) + { + ActivateBestChain(state); + return ""; + } + + if (mapBlockIndex.count(hash) == 0) + { + return "Block not found"; + } + + CBlock block; + CBlockIndex* pblockindex = mapBlockIndex[hash]; + + if(!ReadBlockFromDisk(block, pblockindex)) + { + return "Block not found"; + } + + if(!ActivateBestChainStep(state,pblockindex,&block)) + { + string error=state.GetRejectReason(); + ActivateBestChain(state); + return error; + } + LogPrintf("Set active chain tip: %s\n",hash.GetHex().c_str()); + if(pblockindex->nHeightMinedByMe == pblockindex->nHeight) + { + LogPrint("mchn","mchn: New block %s is mined by me, relay it anyway\n",hash.GetHex().c_str()); + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + { + pnode->PushInventory(CInv(MSG_BLOCK, hash)); + } + } + } + + } + return ""; +} + +/* MCHN END */ + + + +bool InvalidateBlock(CValidationState& state, CBlockIndex *pindex) { + AssertLockHeld(cs_main); + + // Mark the block itself as invalid. + pindex->nStatus |= BLOCK_FAILED_VALID; + setDirtyBlockIndex.insert(pindex); + setBlockIndexCandidates.erase(pindex); + + while (chainActive.Contains(pindex)) { + CBlockIndex *pindexWalk = chainActive.Tip(); + pindexWalk->nStatus |= BLOCK_FAILED_CHILD; + setDirtyBlockIndex.insert(pindexWalk); + setBlockIndexCandidates.erase(pindexWalk); + // ActivateBestChain considers blocks already in chainActive + // unconditionally valid already, so force disconnect away from it. + if (!DisconnectTip(state)) { + return false; + } + } + + // The resulting new best tip may not be in setBlockIndexCandidates anymore, so + // add them again. + BlockMap::iterator it = mapBlockIndex.begin(); + while (it != mapBlockIndex.end()) { + if (it->second->IsValid(BLOCK_VALID_TRANSACTIONS) && it->second->nChainTx && setBlockIndexCandidates.value_comp()(chainActive.Tip(), it->second)) { + setBlockIndexCandidates.insert(pindex); + } + it++; + } + + InvalidChainFound(pindex); + return true; +} + +bool ReconsiderBlock(CValidationState& state, CBlockIndex *pindex) { + AssertLockHeld(cs_main); + + int nHeight = pindex->nHeight; + + // Remove the invalidity flag from this block and all its descendants. + BlockMap::iterator it = mapBlockIndex.begin(); + while (it != mapBlockIndex.end()) { + if (!it->second->IsValid() && it->second->GetAncestor(nHeight) == pindex) { + it->second->nStatus &= ~BLOCK_FAILED_MASK; + setDirtyBlockIndex.insert(it->second); + if (it->second->IsValid(BLOCK_VALID_TRANSACTIONS) && it->second->nChainTx && setBlockIndexCandidates.value_comp()(chainActive.Tip(), it->second)) { + setBlockIndexCandidates.insert(it->second); + } + if (it->second == pindexBestInvalid) { + // Reset invalid block marker if it was pointing to one of those. + pindexBestInvalid = NULL; + } + } + it++; + } + + // Remove the invalidity flag from all ancestors too. + while (pindex != NULL) { + if (pindex->nStatus & BLOCK_FAILED_MASK) { + pindex->nStatus &= ~BLOCK_FAILED_MASK; + setDirtyBlockIndex.insert(pindex); + } + pindex = pindex->pprev; + } + return true; +} + +CBlockIndex* AddToBlockIndex(const CBlockHeader& block) +{ + // Check for duplicate + uint256 hash = block.GetHash(); + BlockMap::iterator it = mapBlockIndex.find(hash); + if (it != mapBlockIndex.end()) + return it->second; + + // Construct new block index object + CBlockIndex* pindexNew = new CBlockIndex(block); + assert(pindexNew); + // We assign the sequence id to blocks only when the full data is available, + // to avoid miners withholding blocks but broadcasting headers, to get a + // competitive advantage. + pindexNew->nSequenceId = 0; + BlockMap::iterator mi = mapBlockIndex.insert(make_pair(hash, pindexNew)).first; + pindexNew->phashBlock = &((*mi).first); + BlockMap::iterator miPrev = mapBlockIndex.find(block.hashPrevBlock); + if (miPrev != mapBlockIndex.end()) + { + pindexNew->pprev = (*miPrev).second; + pindexNew->nHeight = pindexNew->pprev->nHeight + 1; + pindexNew->BuildSkip(); + } + pindexNew->nChainWork = (pindexNew->pprev ? pindexNew->pprev->nChainWork : 0) + GetBlockProof(*pindexNew); + pindexNew->RaiseValidity(BLOCK_VALID_TREE); + if (pindexBestHeader == NULL || pindexBestHeader->nChainWork < pindexNew->nChainWork) + pindexBestHeader = pindexNew; + + setDirtyBlockIndex.insert(pindexNew); + + return pindexNew; +} + +/** Mark a block as having its data received and checked (up to BLOCK_VALID_TRANSACTIONS). */ +bool ReceivedBlockTransactions(const CBlock &block, CValidationState& state, CBlockIndex *pindexNew, const CDiskBlockPos& pos) +{ + pindexNew->nTx = block.vtx.size(); + pindexNew->nChainTx = 0; + pindexNew->nFile = pos.nFile; + pindexNew->nDataPos = pos.nPos; + pindexNew->nUndoPos = 0; + pindexNew->nStatus |= BLOCK_HAVE_DATA; + pindexNew->RaiseValidity(BLOCK_VALID_TRANSACTIONS); + { + LOCK(cs_nBlockSequenceId); + pindexNew->nSequenceId = nBlockSequenceId++; + } + +/* MCHN START */ + UpdateChainMiningStatus(block,pindexNew); + +/* + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + std::vector vchPubKey=std::vector (block.vSigner+1, block.vSigner+1+block.vSigner[0]); + CPubKey pubKeyOut(vchPubKey); + CKeyID keyID=pubKeyOut.GetID(); + CKey key; + pindexNew->nCanMine=mc_gState->m_Permissions->CanMine(NULL,keyID.begin()); + if(pwalletMain) + { + if(pwalletMain->GetKey(keyID,key)) + { + pindexNew->nHeightMinedByMe=pindexNew->nHeight; + if(pindexNew->pprev) + { + pindexNew->nCanMine=pindexNew->pprev->nCanMine; + } + else + { + pindexNew->nCanMine=MC_PTP_MINE; + } + if(mc_gState->m_Permissions->GetActiveMinerCount()m_Permissions->m_MinerCount) + { + pindexNew->nCanMine=0; + } + } + else + { + if(pindexNew->pprev) + { + pindexNew->nHeightMinedByMe=pindexNew->pprev->nHeightMinedByMe; + pindexNew->nCanMine=pindexNew->pprev->nCanMine; + if(pindexNew->nHeightMinedByMe+mc_gState->m_Permissions->m_MinerCount-mc_gState->m_Permissions->GetActiveMinerCount() > pindexNew->nHeight) + { + pindexNew->nCanMine=0; + } + } + else + { + pindexNew->nCanMine=0; + } + } + } + if(pindexNew->pprev) + { + LogPrint("mchn","mchn: New block index: %s, prev: %s, height: %d, mined-by-me: %d, can-mine: %d\n",block.GetHash().ToString().c_str(), + pindexNew->pprev->GetBlockHash().ToString().c_str(), + pindexNew->nHeight,pindexNew->nHeightMinedByMe,pindexNew->nCanMine); + } + } + */ +/* MCHN END */ + + setDirtyBlockIndex.insert(pindexNew); + + if (pindexNew->pprev == NULL || pindexNew->pprev->nChainTx) { + // If pindexNew is the genesis block or all parents are BLOCK_VALID_TRANSACTIONS. + deque queue; + queue.push_back(pindexNew); + + // Recursively process any descendant blocks that now may be eligible to be connected. + while (!queue.empty()) { + CBlockIndex *pindex = queue.front(); + queue.pop_front(); + pindex->nChainTx = (pindex->pprev ? pindex->pprev->nChainTx : 0) + pindex->nTx; + setBlockIndexCandidates.insert(pindex); + std::pair::iterator, std::multimap::iterator> range = mapBlocksUnlinked.equal_range(pindex); + while (range.first != range.second) { + std::multimap::iterator it = range.first; + queue.push_back(it->second); + range.first++; + mapBlocksUnlinked.erase(it); + } + } + } else { + if (pindexNew->pprev && pindexNew->pprev->IsValid(BLOCK_VALID_TREE)) { + mapBlocksUnlinked.insert(std::make_pair(pindexNew->pprev, pindexNew)); + } + } + + return true; +} + +bool FindBlockPos(CValidationState &state, CDiskBlockPos &pos, unsigned int nAddSize, unsigned int nHeight, uint64_t nTime, bool fKnown = false) +{ + LOCK(cs_LastBlockFile); + + unsigned int nFile = fKnown ? pos.nFile : nLastBlockFile; + if (vinfoBlockFile.size() <= nFile) { + vinfoBlockFile.resize(nFile + 1); + } + + if (!fKnown) { + while (vinfoBlockFile[nFile].nSize + nAddSize >= MAX_BLOCKFILE_SIZE) { + LogPrintf("Leaving block file %i: %s\n", nFile, vinfoBlockFile[nFile].ToString()); + FlushBlockFile(true); + nFile++; + if (vinfoBlockFile.size() <= nFile) { + vinfoBlockFile.resize(nFile + 1); + } + } + pos.nFile = nFile; + pos.nPos = vinfoBlockFile[nFile].nSize; + } + + nLastBlockFile = nFile; + vinfoBlockFile[nFile].nSize += nAddSize; + vinfoBlockFile[nFile].AddBlock(nHeight, nTime); + + if (!fKnown) { + unsigned int nOldChunks = (pos.nPos + BLOCKFILE_CHUNK_SIZE - 1) / BLOCKFILE_CHUNK_SIZE; + unsigned int nNewChunks = (vinfoBlockFile[nFile].nSize + BLOCKFILE_CHUNK_SIZE - 1) / BLOCKFILE_CHUNK_SIZE; + if (nNewChunks > nOldChunks) { + if (CheckDiskSpace(nNewChunks * BLOCKFILE_CHUNK_SIZE - pos.nPos)) { + FILE *file = OpenBlockFile(pos); + if (file) { + LogPrintf("Pre-allocating up to position 0x%x in blk%05u.dat\n", nNewChunks * BLOCKFILE_CHUNK_SIZE, pos.nFile); + AllocateFileRange(file, pos.nPos, nNewChunks * BLOCKFILE_CHUNK_SIZE - pos.nPos); + fclose(file); + } + } + else + return state.Error("out of disk space"); + } + } + + setDirtyFileInfo.insert(nFile); + return true; +} + +bool FindUndoPos(CValidationState &state, int nFile, CDiskBlockPos &pos, unsigned int nAddSize) +{ + pos.nFile = nFile; + + LOCK(cs_LastBlockFile); + + unsigned int nNewSize; + pos.nPos = vinfoBlockFile[nFile].nUndoSize; + nNewSize = vinfoBlockFile[nFile].nUndoSize += nAddSize; + setDirtyFileInfo.insert(nFile); + + unsigned int nOldChunks = (pos.nPos + UNDOFILE_CHUNK_SIZE - 1) / UNDOFILE_CHUNK_SIZE; + unsigned int nNewChunks = (nNewSize + UNDOFILE_CHUNK_SIZE - 1) / UNDOFILE_CHUNK_SIZE; + if (nNewChunks > nOldChunks) { + if (CheckDiskSpace(nNewChunks * UNDOFILE_CHUNK_SIZE - pos.nPos)) { + FILE *file = OpenUndoFile(pos); + if (file) { + LogPrintf("Pre-allocating up to position 0x%x in rev%05u.dat\n", nNewChunks * UNDOFILE_CHUNK_SIZE, pos.nFile); + AllocateFileRange(file, pos.nPos, nNewChunks * UNDOFILE_CHUNK_SIZE - pos.nPos); + fclose(file); + } + } + else + return state.Error("out of disk space"); + } + + return true; +} + +bool CheckBlockHeader(const CBlockHeader& block, CValidationState& state, bool fCheckPOW) +{ + // Check proof of work matches claimed amount + if (fCheckPOW && !CheckProofOfWork(block.GetHash(), block.nBits)) + return state.DoS(50, error("CheckBlockHeader() : proof of work failed"), + REJECT_INVALID, "high-hash"); + + // Check timestamp +/* MCHN START */ +// if (block.GetBlockTime() > GetAdjustedTime() + 2 * 60 * 60) + if (block.GetBlockTime() > GetAdjustedTime() + 2 * 6 * Params().TargetSpacing()) +/* MCHN END */ + return state.Invalid(error("CheckBlockHeader() : block timestamp too far in the future"), + REJECT_INVALID, "time-too-new"); + + return true; +} + +/* MCHN START */ +//bool CheckBlock(CBlock& block, CValidationState& state, bool fCheckPOW, bool fCheckMerkleRoot) +bool CheckBlock(const CBlock& block, CValidationState& state, bool fCheckPOW, bool fCheckMerkleRoot) +/* MCHN END */ +{ + // These are checks that are independent of context. + + // Check that the header is valid (particularly PoW). This is mostly + // redundant with the call in AcceptBlockHeader. + if (!CheckBlockHeader(block, state, fCheckPOW)) + return false; + + // Check the merkle root. + if (fCheckMerkleRoot) { + bool mutated; + uint256 hashMerkleRoot2 = block.BuildMerkleTree(&mutated); + if (block.hashMerkleRoot != hashMerkleRoot2) + return state.DoS(100, error("CheckBlock() : hashMerkleRoot mismatch"), + REJECT_INVALID, "bad-txnmrklroot", true); + + // Check for merkle tree malleability (CVE-2012-2459): repeating sequences + // of transactions in a block without affecting the merkle root of a block, + // while still invalidating it. + if (mutated) + return state.DoS(100, error("CheckBlock() : duplicate transaction"), + REJECT_INVALID, "bad-txns-duplicate", true); + } + + // All potential-corruption validation must be done before we do any + // transaction validation, as otherwise we may mark the header as invalid + // because we receive the wrong transactions for it. + + // Size limits + if (block.vtx.empty() || block.vtx.size() > MAX_BLOCK_SIZE || ::GetSerializeSize(block, SER_NETWORK, PROTOCOL_VERSION) > MAX_BLOCK_SIZE) + return state.DoS(100, error("CheckBlock() : size limits failed"), + REJECT_INVALID, "bad-blk-length"); + + // First transaction must be coinbase, the rest must not be + if (block.vtx.empty() || !block.vtx[0].IsCoinBase()) + return state.DoS(100, error("CheckBlock() : first tx is not coinbase"), + REJECT_INVALID, "bad-cb-missing"); + for (unsigned int i = 1; i < block.vtx.size(); i++) + if (block.vtx[i].IsCoinBase()) + return state.DoS(100, error("CheckBlock() : more than one coinbase"), + REJECT_INVALID, "bad-cb-multiple"); + + // Check transactions + BOOST_FOREACH(const CTransaction& tx, block.vtx) + if (!CheckTransaction(tx, state)) + return error("CheckBlock() : CheckTransaction failed"); + + unsigned int nSigOps = 0; + BOOST_FOREACH(const CTransaction& tx, block.vtx) + { + nSigOps += GetLegacySigOpCount(tx); + } + if (nSigOps > MAX_BLOCK_SIGOPS) + return state.DoS(100, error("CheckBlock() : out-of-bounds SigOpCount"), + REJECT_INVALID, "bad-blk-sigops", true); + + return true; +} + +bool ContextualCheckBlockHeader(const CBlockHeader& block, CValidationState& state, CBlockIndex * const pindexPrev) +{ + uint256 hash = block.GetHash(); + if (hash == Params().HashGenesisBlock()) + return true; + + assert(pindexPrev); + + int nHeight = pindexPrev->nHeight+1; + + // Check proof of work + if ((!Params().SkipProofOfWorkCheck()) && + (block.nBits != GetNextWorkRequired(pindexPrev, &block))) + return state.DoS(100, error("%s : incorrect proof of work", __func__), + REJECT_INVALID, "bad-diffbits"); + + // Check timestamp against prev + if (block.GetBlockTime() <= pindexPrev->GetMedianTimePast()) + return state.Invalid(error("%s : block's timestamp is too early", __func__), + REJECT_INVALID, "time-too-old"); + + // Check that the block chain matches the known block chain up to a checkpoint + if (!Checkpoints::CheckBlock(nHeight, hash)) + return state.DoS(100, error("%s : rejected by checkpoint lock-in at %d", __func__, nHeight), + REJECT_CHECKPOINT, "checkpoint mismatch"); + + // Don't accept any forks from the main chain prior to last checkpoint + CBlockIndex* pcheckpoint = Checkpoints::GetLastCheckpoint(); + if (pcheckpoint && nHeight < pcheckpoint->nHeight) + return state.DoS(100, error("%s : forked chain older than last checkpoint (height %d)", __func__, nHeight)); + + // Reject block.nVersion=1 blocks when 95% (75% on testnet) of the network has upgraded: + if (block.nVersion < 2 && + CBlockIndex::IsSuperMajority(2, pindexPrev, Params().RejectBlockOutdatedMajority())) + { + return state.Invalid(error("%s : rejected nVersion=1 block", __func__), + REJECT_OBSOLETE, "bad-version"); + } + + // Reject block.nVersion=2 blocks when 95% (75% on testnet) of the network has upgraded: + if (block.nVersion < 3 && CBlockIndex::IsSuperMajority(3, pindexPrev, Params().RejectBlockOutdatedMajority())) + { + return state.Invalid(error("%s : rejected nVersion=2 block", __func__), + REJECT_OBSOLETE, "bad-version"); + } + + return true; +} + +bool ContextualCheckBlock(const CBlock& block, CValidationState& state, CBlockIndex * const pindexPrev) +{ + const int nHeight = pindexPrev == NULL ? 0 : pindexPrev->nHeight + 1; + + // Check that all transactions are finalized + BOOST_FOREACH(const CTransaction& tx, block.vtx) + if (!IsFinalTx(tx, nHeight, block.GetBlockTime())) { + return state.DoS(10, error("%s : contains a non-final transaction", __func__), REJECT_INVALID, "bad-txns-nonfinal"); + } + + // Enforce block.nVersion=2 rule that the coinbase starts with serialized block height + // if 750 of the last 1,000 blocks are version 2 or greater (51/100 if testnet): + if (block.nVersion >= 2 && + CBlockIndex::IsSuperMajority(2, pindexPrev, Params().EnforceBlockUpgradeMajority())) + { + CScript expect = CScript() << nHeight; + if (block.vtx[0].vin[0].scriptSig.size() < expect.size() || + !std::equal(expect.begin(), expect.end(), block.vtx[0].vin[0].scriptSig.begin())) { + return state.DoS(100, error("%s : block height mismatch in coinbase", __func__), REJECT_INVALID, "bad-cb-height"); + } + } + +/* MCHN START */ +/* + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(nHeight) + { + vector vchPubKey; + + vchPubKey=vector (block.vSigner+1, block.vSigner+1+block.vSigner[0]); + + CPubKey pubKeyOut(vchPubKey); + if (!pubKeyOut.IsValid()) + { + LogPrintf("mchn: ContextualCheckBlock: Invalid pubkey received in block signature\n"); + return state.DoS(100, error("%s : Invalid key in signature", __func__), REJECT_INVALID, "bad-block-signature"); + } + + CKeyID pubKeyHash=pubKeyOut.GetID(); + if(!mc_gState->m_Permissions->CanMineBlock(pubKeyHash.begin(),nHeight)) + { + LogPrintf("mchn: ContextualCheckBlock: Permission denied for miner %s received in block signature for block %d, prev: %s\n", + CBitcoinAddress(pubKeyHash).ToString().c_str(),nHeight,pindexPrev->GetBlockHash().ToString().c_str()); + return false; + } + } + } + */ +/* MCHN END */ + + return true; +} + +bool AcceptBlockHeader(const CBlockHeader& block, CValidationState& state, CBlockIndex** ppindex) +{ + AssertLockHeld(cs_main); + // Check for duplicate + uint256 hash = block.GetHash(); + BlockMap::iterator miSelf = mapBlockIndex.find(hash); + CBlockIndex *pindex = NULL; + if (miSelf != mapBlockIndex.end()) { + // Block header is already known. + pindex = miSelf->second; + if (ppindex) + *ppindex = pindex; + if (pindex->nStatus & BLOCK_FAILED_MASK) + return state.Invalid(error("%s : block is marked invalid", __func__), 0, "duplicate"); + return true; + } + + if (!CheckBlockHeader(block, state)) + return false; + + // Get prev block index + CBlockIndex* pindexPrev = NULL; + if (hash != Params().HashGenesisBlock()) { + BlockMap::iterator mi = mapBlockIndex.find(block.hashPrevBlock); + if (mi == mapBlockIndex.end()) + return state.DoS(10, error("%s : prev block not found", __func__), 0, "bad-prevblk"); + pindexPrev = (*mi).second; + if (pindexPrev->nStatus & BLOCK_FAILED_MASK) + return state.DoS(10, error("%s : prev block invalid", __func__), REJECT_INVALID, "bad-prevblk");// MCHN was 100 before, softened for reorgs due to mining diversity change + } + + if (!ContextualCheckBlockHeader(block, state, pindexPrev)) + return false; + + if (pindex == NULL) + pindex = AddToBlockIndex(block); + + if (ppindex) + *ppindex = pindex; + + return true; +} + +bool AcceptBlock(CBlock& block, CValidationState& state, CBlockIndex** ppindex, CDiskBlockPos* dbp) +{ + AssertLockHeld(cs_main); + + CBlockIndex *&pindex = *ppindex; + + if (!AcceptBlockHeader(block, state, &pindex)) + return false; + + if (pindex->nStatus & BLOCK_HAVE_DATA) { + // TODO: deal better with duplicate blocks. + // return state.DoS(20, error("AcceptBlock() : already have block %d %s", pindex->nHeight, pindex->GetBlockHash().ToString()), REJECT_DUPLICATE, "duplicate"); + return true; + } + +/* MCHN START*/ + if(!VerifyBlockSignature(&block,false)) + { + return false; + } +/* MCHN END*/ + + + if ((!CheckBlock(block, state)) || !ContextualCheckBlock(block, state, pindex->pprev)) { + if (state.IsInvalid() && !state.CorruptionPossible()) { + pindex->nStatus |= BLOCK_FAILED_VALID; + setDirtyBlockIndex.insert(pindex); + } + return false; + } + + int nHeight = pindex->nHeight; + + // Write block to history file + try { + unsigned int nBlockSize = ::GetSerializeSize(block, SER_DISK, CLIENT_VERSION); + CDiskBlockPos blockPos; + if (dbp != NULL) + blockPos = *dbp; + if (!FindBlockPos(state, blockPos, nBlockSize+8, nHeight, block.GetBlockTime(), dbp != NULL)) + return error("AcceptBlock() : FindBlockPos failed"); + if (dbp == NULL) + if (!WriteBlockToDisk(block, blockPos)) + return state.Abort("Failed to write block"); + if (!ReceivedBlockTransactions(block, state, pindex, blockPos)) + return error("AcceptBlock() : ReceivedBlockTransactions failed"); + } catch(std::runtime_error &e) { + return state.Abort(std::string("System error: ") + e.what()); + } + + return true; +} + +bool CBlockIndex::IsSuperMajority(int minVersion, const CBlockIndex* pstart, unsigned int nRequired) +{ + unsigned int nToCheck = Params().ToCheckBlockUpgradeMajority(); + unsigned int nFound = 0; + for (unsigned int i = 0; i < nToCheck && nFound < nRequired && pstart != NULL; i++) + { + if (pstart->nVersion >= minVersion) + ++nFound; + pstart = pstart->pprev; + } + return (nFound >= nRequired); +} + +/** Turn the lowest '1' bit in the binary representation of a number into a '0'. */ +int static inline InvertLowestOne(int n) { return n & (n - 1); } + +/** Compute what height to jump back to with the CBlockIndex::pskip pointer. */ +int static inline GetSkipHeight(int height) { + if (height < 2) + return 0; + + // Determine which height to jump back to. Any number strictly lower than height is acceptable, + // but the following expression seems to perform well in simulations (max 110 steps to go back + // up to 2**18 blocks). + return (height & 1) ? InvertLowestOne(InvertLowestOne(height - 1)) + 1 : InvertLowestOne(height); +} + +CBlockIndex* CBlockIndex::GetAncestor(int height) +{ + if (height > nHeight || height < 0) + return NULL; + + CBlockIndex* pindexWalk = this; + int heightWalk = nHeight; + while (heightWalk > height) { + int heightSkip = GetSkipHeight(heightWalk); + int heightSkipPrev = GetSkipHeight(heightWalk - 1); + if (heightSkip == height || + (heightSkip > height && !(heightSkipPrev < heightSkip - 2 && + heightSkipPrev >= height))) { + // Only follow pskip if pprev->pskip isn't better than pskip->pprev. + pindexWalk = pindexWalk->pskip; + heightWalk = heightSkip; + } else { + pindexWalk = pindexWalk->pprev; + heightWalk--; + } + } + return pindexWalk; +} + +const CBlockIndex* CBlockIndex::GetAncestor(int height) const +{ + return const_cast(this)->GetAncestor(height); +} + +void CBlockIndex::BuildSkip() +{ + if (pprev) + pskip = pprev->GetAncestor(GetSkipHeight(nHeight)); +} + +bool ProcessNewBlock(CValidationState &state, CNode* pfrom, CBlock* pblock, CDiskBlockPos *dbp) +{ +/* MCHN START*/ + if(!VerifyBlockSignature(pblock,true)) + { + return false; + } +/* MCHN END*/ + + // Preliminary checks + bool checked = CheckBlock(*pblock, state); + + { + LOCK(cs_main); + MarkBlockAsReceived(pblock->GetHash()); + if (!checked) { + return error("%s : CheckBlock FAILED", __func__); + } + + // Store to disk + CBlockIndex *pindex = NULL; + bool ret = AcceptBlock(*pblock, state, &pindex, dbp); + if (pindex && pfrom) { + mapBlockSource[pindex->GetBlockHash()] = pfrom->GetId(); + } + if (!ret) + return error("%s : AcceptBlock FAILED", __func__); + } + + if (!ActivateBestChain(state, pblock)) + return error("%s : ActivateBestChain failed", __func__); + +/* MCHN START */ +/* + CBlockIndex* pblockindex = mapBlockIndex[pblock->GetHash()]; + if(pblockindex != chainActive.Tip()) + { + LogPrint("mchn","mchn: New block rejected as chain tip\n"); + if(pblockindex->nHeightMinedByMe == pblockindex->nHeight) + { + LogPrint("mchn","mchn: New block is mined by me, relay it anyway\n"); + if (!IsInitialBlockDownload()) + { + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + { + pnode->PushInventory(CInv(MSG_BLOCK, pblock->GetHash())); + } + } + } + } + } + */ + if(GetBoolArg("-shrinkdebugfilesize",false)) + { + ShrinkDebugFile(); + } +/* MCHN END */ + + return true; +} + +bool TestBlockValidity(CValidationState &state, const CBlock& block, CBlockIndex * const pindexPrev, bool fCheckPOW, bool fCheckMerkleRoot) +{ + AssertLockHeld(cs_main); + assert(pindexPrev == chainActive.Tip()); + + CCoinsViewCache viewNew(pcoinsTip); + CBlockIndex indexDummy(block); + indexDummy.pprev = pindexPrev; + indexDummy.nHeight = pindexPrev->nHeight + 1; + + // NOTE: CheckBlockHeader is called by CheckBlock + if (!ContextualCheckBlockHeader(block, state, pindexPrev)) + return false; + if (!CheckBlock(block, state, fCheckPOW, fCheckMerkleRoot)) + return false; + if (!ContextualCheckBlock(block, state, pindexPrev)) + return false; + if (!ConnectBlock(block, state, &indexDummy, viewNew, true)) + return false; + assert(state.IsValid()); + + return true; +} + + + + + + + + +bool AbortNode(const std::string &strMessage, const std::string &userMessage) { + strMiscWarning = strMessage; + LogPrintf("*** %s\n", strMessage); + uiInterface.ThreadSafeMessageBox( + userMessage.empty() ? _("Error: A fatal internal error occured, see debug.log for details") : userMessage, + "", CClientUIInterface::MSG_ERROR); + StartShutdown(); + return false; +} + +bool CheckDiskSpace(uint64_t nAdditionalBytes) +{ + uint64_t nFreeBytesAvailable = filesystem::space(GetDataDir()).available; + + // Check for nMinDiskSpace bytes (currently 50MB) + if (nFreeBytesAvailable < nMinDiskSpace + nAdditionalBytes) + return AbortNode("Disk space is low!", _("Error: Disk space is low!")); + + return true; +} + +FILE* OpenDiskFile(const CDiskBlockPos &pos, const char *prefix, bool fReadOnly) +{ + if (pos.IsNull()) + return NULL; + boost::filesystem::path path = GetBlockPosFilename(pos, prefix); + boost::filesystem::create_directories(path.parent_path()); + FILE* file = fopen(path.string().c_str(), "rb+"); + if (!file && !fReadOnly) + file = fopen(path.string().c_str(), "wb+"); + if (!file) { + LogPrintf("Unable to open file %s\n", path.string()); + return NULL; + } + if (pos.nPos) { + if (fseek(file, pos.nPos, SEEK_SET)) { + LogPrintf("Unable to seek to position %u of %s\n", pos.nPos, path.string()); + fclose(file); + return NULL; + } + } + return file; +} + +FILE* OpenBlockFile(const CDiskBlockPos &pos, bool fReadOnly) { + return OpenDiskFile(pos, "blk", fReadOnly); +} + +FILE* OpenUndoFile(const CDiskBlockPos &pos, bool fReadOnly) { + return OpenDiskFile(pos, "rev", fReadOnly); +} + +boost::filesystem::path GetBlockPosFilename(const CDiskBlockPos &pos, const char *prefix) +{ + return GetDataDir() / "blocks" / strprintf("%s%05u.dat", prefix, pos.nFile); +} + +CBlockIndex * InsertBlockIndex(uint256 hash) +{ + if (hash == 0) + return NULL; + + // Return existing + BlockMap::iterator mi = mapBlockIndex.find(hash); + if (mi != mapBlockIndex.end()) + return (*mi).second; + + // Create new + CBlockIndex* pindexNew = new CBlockIndex(); + if (!pindexNew) + throw runtime_error("LoadBlockIndex() : new CBlockIndex failed"); + mi = mapBlockIndex.insert(make_pair(hash, pindexNew)).first; + pindexNew->phashBlock = &((*mi).first); + + return pindexNew; +} + +bool static LoadBlockIndexDB() +{ + if (!pblocktree->LoadBlockIndexGuts()) + return false; + + boost::this_thread::interruption_point(); + + // Calculate nChainWork + vector > vSortedByHeight; + vSortedByHeight.reserve(mapBlockIndex.size()); + BOOST_FOREACH(const PAIRTYPE(uint256, CBlockIndex*)& item, mapBlockIndex) + { + CBlockIndex* pindex = item.second; + vSortedByHeight.push_back(make_pair(pindex->nHeight, pindex)); + } + sort(vSortedByHeight.begin(), vSortedByHeight.end()); + BOOST_FOREACH(const PAIRTYPE(int, CBlockIndex*)& item, vSortedByHeight) + { + CBlockIndex* pindex = item.second; + pindex->nChainWork = (pindex->pprev ? pindex->pprev->nChainWork : 0) + GetBlockProof(*pindex); + if (pindex->nStatus & BLOCK_HAVE_DATA) { + if (pindex->pprev) { + if (pindex->pprev->nChainTx) { + pindex->nChainTx = pindex->pprev->nChainTx + pindex->nTx; + } else { + pindex->nChainTx = 0; + mapBlocksUnlinked.insert(std::make_pair(pindex->pprev, pindex)); + } + } else { + pindex->nChainTx = pindex->nTx; + } + } + if (pindex->IsValid(BLOCK_VALID_TRANSACTIONS) && (pindex->nChainTx || pindex->pprev == NULL)) + setBlockIndexCandidates.insert(pindex); + if (pindex->nStatus & BLOCK_FAILED_MASK && (!pindexBestInvalid || pindex->nChainWork > pindexBestInvalid->nChainWork)) + pindexBestInvalid = pindex; + if (pindex->pprev) + pindex->BuildSkip(); + if (pindex->IsValid(BLOCK_VALID_TREE) && (pindexBestHeader == NULL || CBlockIndexWorkComparator()(pindexBestHeader, pindex))) + pindexBestHeader = pindex; + } + + // Load block file info + pblocktree->ReadLastBlockFile(nLastBlockFile); + vinfoBlockFile.resize(nLastBlockFile + 1); + LogPrintf("%s: last block file = %i\n", __func__, nLastBlockFile); + for (int nFile = 0; nFile <= nLastBlockFile; nFile++) { + pblocktree->ReadBlockFileInfo(nFile, vinfoBlockFile[nFile]); + } + LogPrintf("%s: last block file info: %s\n", __func__, vinfoBlockFile[nLastBlockFile].ToString()); + for (int nFile = nLastBlockFile + 1; true; nFile++) { + CBlockFileInfo info; + if (pblocktree->ReadBlockFileInfo(nFile, info)) { + vinfoBlockFile.push_back(info); + } else { + break; + } + } + + // Check presence of blk files + LogPrintf("Checking all blk files are present...\n"); + set setBlkDataFiles; + BOOST_FOREACH(const PAIRTYPE(uint256, CBlockIndex*)& item, mapBlockIndex) + { + CBlockIndex* pindex = item.second; + if (pindex->nStatus & BLOCK_HAVE_DATA) { + setBlkDataFiles.insert(pindex->nFile); + } + } + for (std::set::iterator it = setBlkDataFiles.begin(); it != setBlkDataFiles.end(); it++) + { + CDiskBlockPos pos(*it, 0); + if (CAutoFile(OpenBlockFile(pos, true), SER_DISK, CLIENT_VERSION).IsNull()) { + return false; + } + } + + // Check whether we need to continue reindexing + bool fReindexing = false; + pblocktree->ReadReindexing(fReindexing); + fReindex |= fReindexing; + + // Check whether we have a transaction index + pblocktree->ReadFlag("txindex", fTxIndex); + LogPrintf("LoadBlockIndexDB(): transaction index %s\n", fTxIndex ? "enabled" : "disabled"); + + // Load pointer to end of best chain + BlockMap::iterator it = mapBlockIndex.find(pcoinsTip->GetBestBlock()); + if (it == mapBlockIndex.end()) + return true; + chainActive.SetTip(it->second); + + PruneBlockIndexCandidates(); +/* MCHN START */ + + bool corrupted=false; + if(mc_gState->m_Permissions->m_Block < chainActive.Height()) + { + corrupted=true; + LogPrintf("mchn: Permission DB is behind current chain tip. Permission DB: %d, Chain tip: %d\n",mc_gState->m_Permissions->m_Block,chainActive.Height()); + } + if(mc_gState->m_Assets->m_Block < chainActive.Height()) + { + corrupted=true; + LogPrintf("mchn: Entities DB is behind current chain tip. Entities DB: %d, Chain tip: %d\n",mc_gState->m_Assets->m_Block,chainActive.Height()); + } + if(mc_gState->m_Permissions->m_Block != mc_gState->m_Assets->m_Block) + { + corrupted=true; + LogPrintf("mchn: Permission and Entities DB have different heights. Permission DB: %d, Entities DB: %d, Chain tip: %d\n",mc_gState->m_Permissions->m_Block,mc_gState->m_Assets->m_Block,chainActive.Height()); + } + if(mc_gState->m_WalletMode & MC_WMD_TXS) + { + if(!GetBoolArg("-rescan", false)) + { + if(pwalletTxsMain->GetBlock() < chainActive.Height()) + { + corrupted=true; + LogPrintf("mchn: Wallet Tx DB is behind current chain tip. Wallet Tx DB: %d, Chain tip: %d\n",pwalletTxsMain->GetBlock(),chainActive.Height()); + } + if(mc_gState->m_Permissions->m_Block != pwalletTxsMain->GetBlock()) + { + corrupted=true; + LogPrintf("mchn: Permission and Wallet Tx DB have different heights. Permission DB: %d, Wallet Tx DB: %d, Chain tip: %d\n",mc_gState->m_Permissions->m_Block,pwalletTxsMain->GetBlock(),chainActive.Height()); + } + } + } + if(corrupted) + { + corrupted=false; + int block_to_rollback=mc_gState->m_Permissions->m_Block; + if(mc_gState->m_Assets->m_Block < block_to_rollback) + { + block_to_rollback=mc_gState->m_Assets->m_Block; + } + if(mc_gState->m_WalletMode & MC_WMD_TXS) + { + if(!GetBoolArg("-rescan", false)) + { + if(pwalletTxsMain->GetBlock() < block_to_rollback) + { + block_to_rollback=pwalletTxsMain->GetBlock(); + } + } + } + if(block_to_rollback < chainActive.Height()) + { + LogPrintf("mchn: Permission/Entities/WalletTx DB is behind current chain tip. Shifting chain tip to %d\n",block_to_rollback); + SetLastBlock(chainActive[block_to_rollback]->GetBlockHash()); + SetLastBlock(0); + } +/* + LogPrintf("mchn: Rolling back to height %d\n",block_to_rollback); + mc_gState->m_Permissions->RollBack(block_to_rollback); + mc_gState->m_Assets->RollBack(block_to_rollback); + if(mc_gState->m_Permissions->m_Block != block_to_rollback) + { + corrupted=true; + LogPrintf("mchn: Cannot roll back Permission DB. Height: %d\n",mc_gState->m_Permissions->m_Block); + } + if(mc_gState->m_Assets->m_Block != block_to_rollback) + { + corrupted=true; + LogPrintf("mchn: Cannot roll back Entities DB. Height: %d\n",mc_gState->m_Assets->m_Block); + return false; + } + */ + } +/* + if(mc_gState->m_Permissions->m_Block < chainActive.Height()) + { + LogPrintf("mchn: Permission DB is behind current chain tip. Shifting chain tip to %d\n",mc_gState->m_Permissions->m_Block); + SetLastBlock(chainActive[mc_gState->m_Permissions->m_Block]->GetBlockHash()); + } +*/ + + + LogPrint("mchn","mchn: Rolling back permission DB to height %d\n",chainActive.Height()); + mc_gState->m_Permissions->RollBack(chainActive.Height()); + LogPrint("mchn","mchn: Rolling back asset DB to height %d\n",chainActive.Height()); + mc_gState->m_Assets->RollBack(chainActive.Height()); + if(mc_gState->m_WalletMode & MC_WMD_TXS) + { + LogPrint("mchn","mchn: Rolling back wallet txs DB to height %d\n",chainActive.Height()); + pwalletTxsMain->RollBack(NULL,chainActive.Height()); + } +/* MCHN END */ + LogPrintf("LoadBlockIndexDB(): hashBestChain=%s height=%d date=%s progress=%f\n", + chainActive.Tip()->GetBlockHash().ToString(), chainActive.Height(), + DateTimeStrFormat("%Y-%m-%d %H:%M:%S", chainActive.Tip()->GetBlockTime()), + Checkpoints::GuessVerificationProgress(chainActive.Tip())); + + return true; +} + +CVerifyDB::CVerifyDB() +{ + uiInterface.ShowProgress(_("Verifying blocks..."), 0); +} + +CVerifyDB::~CVerifyDB() +{ + uiInterface.ShowProgress("", 100); +} + +bool CVerifyDB::VerifyDB(CCoinsView *coinsview, int nCheckLevel, int nCheckDepth) +{ + LOCK(cs_main); + if (chainActive.Tip() == NULL || chainActive.Tip()->pprev == NULL) + return true; + + // Verify blocks in the best chain + if (nCheckDepth <= 0) + nCheckDepth = 1000000000; // suffices until the year 19000 + if (nCheckDepth > chainActive.Height()) + nCheckDepth = chainActive.Height(); + nCheckLevel = std::max(0, std::min(4, nCheckLevel)); + LogPrintf("Verifying last %i blocks at level %i\n", nCheckDepth, nCheckLevel); + CCoinsViewCache coins(coinsview); + CBlockIndex* pindexState = chainActive.Tip(); + CBlockIndex* pindexFailure = NULL; + int nGoodTransactions = 0; + CValidationState state; + for (CBlockIndex* pindex = chainActive.Tip(); pindex && pindex->pprev; pindex = pindex->pprev) + { + boost::this_thread::interruption_point(); + uiInterface.ShowProgress(_("Verifying blocks..."), std::max(1, std::min(99, (int)(((double)(chainActive.Height() - pindex->nHeight)) / (double)nCheckDepth * (nCheckLevel >= 4 ? 50 : 100))))); + if (pindex->nHeight < chainActive.Height()-nCheckDepth) + break; + CBlock block; + // check level 0: read from disk + if (!ReadBlockFromDisk(block, pindex)) + return error("VerifyDB() : *** ReadBlockFromDisk failed at %d, hash=%s", pindex->nHeight, pindex->GetBlockHash().ToString()); + // check level 1: verify block validity + if (nCheckLevel >= 1 && !CheckBlock(block, state)) + return error("VerifyDB() : *** found bad block at %d, hash=%s\n", pindex->nHeight, pindex->GetBlockHash().ToString()); + +/* MCHN START */ + if (nCheckLevel >= 1 && (pindex->nHeight <= mc_gState->m_Permissions->m_Block) && (mc_gState->m_Permissions->VerifyBlockHash(pindex->nHeight,pindex->GetBlockHash().begin()) == 0)) + return error("VerifyDB() : *** found bad permission data at %d, hash=%s\n", pindex->nHeight, pindex->GetBlockHash().ToString()); +/* MCHN END */ + // check level 2: verify undo validity + if (nCheckLevel >= 2 && pindex) { + CBlockUndo undo; + CDiskBlockPos pos = pindex->GetUndoPos(); + if (!pos.IsNull()) { + if (!undo.ReadFromDisk(pos, pindex->pprev->GetBlockHash())) + return error("VerifyDB() : *** found bad undo data at %d, hash=%s\n", pindex->nHeight, pindex->GetBlockHash().ToString()); + } + } + // check level 3: check for inconsistencies during memory-only disconnect of tip blocks + if (nCheckLevel >= 3 && pindex == pindexState && (coins.GetCacheSize() + pcoinsTip->GetCacheSize()) <= nCoinCacheSize) { + bool fClean = true; + if (!DisconnectBlock(block, state, pindex, coins, &fClean)) + return error("VerifyDB() : *** irrecoverable inconsistency in block data at %d, hash=%s", pindex->nHeight, pindex->GetBlockHash().ToString()); + pindexState = pindex->pprev; + if (!fClean) { + nGoodTransactions = 0; + pindexFailure = pindex; + } else + nGoodTransactions += block.vtx.size(); + } + if (ShutdownRequested()) + return true; + } + if (pindexFailure) + return error("VerifyDB() : *** coin database inconsistencies found (last %i blocks, %i good transactions before that)\n", chainActive.Height() - pindexFailure->nHeight + 1, nGoodTransactions); + + // check level 4: try reconnecting blocks + if (nCheckLevel >= 4) { + CBlockIndex *pindex = pindexState; + while (pindex != chainActive.Tip()) { + boost::this_thread::interruption_point(); + uiInterface.ShowProgress(_("Verifying blocks..."), std::max(1, std::min(99, 100 - (int)(((double)(chainActive.Height() - pindex->nHeight)) / (double)nCheckDepth * 50)))); + pindex = chainActive.Next(pindex); + CBlock block; + if (!ReadBlockFromDisk(block, pindex)) + return error("VerifyDB() : *** ReadBlockFromDisk failed at %d, hash=%s", pindex->nHeight, pindex->GetBlockHash().ToString()); + if (!ConnectBlock(block, state, pindex, coins)) + return error("VerifyDB() : *** found unconnectable block at %d, hash=%s", pindex->nHeight, pindex->GetBlockHash().ToString()); + } + } + + LogPrintf("No coin database inconsistencies in last %i blocks (%i transactions)\n", chainActive.Height() - pindexState->nHeight, nGoodTransactions); + + return true; +} + +void UnloadBlockIndex() +{ + mapBlockIndex.clear(); + setBlockIndexCandidates.clear(); + chainActive.SetTip(NULL); + pindexBestInvalid = NULL; +} + +bool LoadBlockIndex() +{ + // Load block index from databases + if (!fReindex && !LoadBlockIndexDB()) + return false; + return true; +} + + +bool InitBlockIndex() { + LOCK(cs_main); + // Check whether we're already initialized + if (chainActive.Genesis() != NULL) + return true; + + // Use the provided setting for -txindex in the new database +/* MCHN START */ +/* Default was false */ + fTxIndex = GetBoolArg("-txindex", true); +/* MCHN END */ + pblocktree->WriteFlag("txindex", fTxIndex); + LogPrintf("Initializing databases...\n"); + + // Only add the genesis block if not reindexing (in which case we reuse the one already on disk) + if (!fReindex) { + try { + CBlock &block = const_cast(Params().GenesisBlock()); + // Start new block file + unsigned int nBlockSize = ::GetSerializeSize(block, SER_DISK, CLIENT_VERSION); + CDiskBlockPos blockPos; + CValidationState state; + if (!FindBlockPos(state, blockPos, nBlockSize+8, 0, block.GetBlockTime())) + return error("LoadBlockIndex() : FindBlockPos failed"); + if (!WriteBlockToDisk(block, blockPos)) + return error("LoadBlockIndex() : writing genesis block to disk failed"); + CBlockIndex *pindex = AddToBlockIndex(block); + if (!ReceivedBlockTransactions(block, state, pindex, blockPos)) + return error("LoadBlockIndex() : genesis block not accepted"); + if (!ActivateBestChain(state, &block)) + return error("LoadBlockIndex() : genesis block cannot be activated"); + // Force a chainstate write so that when we VerifyDB in a moment, it doesnt check stale data + return FlushStateToDisk(state, FLUSH_STATE_ALWAYS); + } catch(std::runtime_error &e) { + return error("LoadBlockIndex() : failed to initialize block database: %s", e.what()); + } + } + + return true; +} + + + +bool LoadExternalBlockFile(FILE* fileIn, CDiskBlockPos *dbp) +{ + // Map of disk positions for blocks with unknown parent (only used for reindex) + static std::multimap mapBlocksUnknownParent; + int64_t nStart = GetTimeMillis(); + + int nLoaded = 0; + try { + // This takes over fileIn and calls fclose() on it in the CBufferedFile destructor + CBufferedFile blkdat(fileIn, 2*MAX_BLOCK_SIZE, MAX_BLOCK_SIZE+8, SER_DISK, CLIENT_VERSION); + uint64_t nRewind = blkdat.GetPos(); + while (!blkdat.eof()) { + boost::this_thread::interruption_point(); + + blkdat.SetPos(nRewind); + nRewind++; // start one byte further next time, in case of failure + blkdat.SetLimit(); // remove former limit + unsigned int nSize = 0; + try { + // locate a header + unsigned char buf[MESSAGE_START_SIZE]; + blkdat.FindByte(Params().MessageStart()[0]); + nRewind = blkdat.GetPos()+1; + blkdat >> FLATDATA(buf); + if (memcmp(buf, Params().MessageStart(), MESSAGE_START_SIZE)) + continue; + // read size + blkdat >> nSize; + if (nSize < 80 || nSize > MAX_BLOCK_SIZE) + continue; + } catch (const std::exception &) { + // no valid block header found; don't complain + break; + } + try { + // read block + uint64_t nBlockPos = blkdat.GetPos(); + if (dbp) + dbp->nPos = nBlockPos; + blkdat.SetLimit(nBlockPos + nSize); + blkdat.SetPos(nBlockPos); + CBlock block; + blkdat >> block; + nRewind = blkdat.GetPos(); + + // detect out of order blocks, and store them for later + uint256 hash = block.GetHash(); + if (hash != Params().HashGenesisBlock() && mapBlockIndex.find(block.hashPrevBlock) == mapBlockIndex.end()) { + LogPrint("reindex", "%s: Out of order block %s, parent %s not known\n", __func__, hash.ToString(), + block.hashPrevBlock.ToString()); + if (dbp) + mapBlocksUnknownParent.insert(std::make_pair(block.hashPrevBlock, *dbp)); + continue; + } + + // process in case the block isn't known yet + if (mapBlockIndex.count(hash) == 0 || (mapBlockIndex[hash]->nStatus & BLOCK_HAVE_DATA) == 0) { + CValidationState state; + if (ProcessNewBlock(state, NULL, &block, dbp)) + nLoaded++; + if (state.IsError()) + break; + } else if (hash != Params().HashGenesisBlock() && mapBlockIndex[hash]->nHeight % 1000 == 0) { + LogPrintf("Block Import: already had block %s at height %d\n", hash.ToString(), mapBlockIndex[hash]->nHeight); + } + + // Recursively process earlier encountered successors of this block + deque queue; + queue.push_back(hash); + while (!queue.empty()) { + uint256 head = queue.front(); + queue.pop_front(); + std::pair::iterator, std::multimap::iterator> range = mapBlocksUnknownParent.equal_range(head); + while (range.first != range.second) { + std::multimap::iterator it = range.first; + if (ReadBlockFromDisk(block, it->second)) + { + LogPrintf("%s: Processing out of order child %s of %s\n", __func__, block.GetHash().ToString(), + head.ToString()); + CValidationState dummy; + if (ProcessNewBlock(dummy, NULL, &block, &it->second)) + { + nLoaded++; + queue.push_back(block.GetHash()); + } + } + range.first++; + mapBlocksUnknownParent.erase(it); + } + } + } catch (std::exception &e) { + LogPrintf("%s : Deserialize or I/O error - %s", __func__, e.what()); + } + } + } catch(std::runtime_error &e) { + AbortNode(std::string("System error: ") + e.what()); + } + if (nLoaded > 0) + LogPrintf("Loaded %i blocks from external file in %dms\n", nLoaded, GetTimeMillis() - nStart); + return nLoaded > 0; +} + +////////////////////////////////////////////////////////////////////////////// +// +// CAlert +// + +string GetWarnings(string strFor) +{ + int nPriority = 0; + string strStatusBar; + string strRPC; + + if (!CLIENT_VERSION_IS_RELEASE) + strStatusBar = _("This is a pre-release test build - use at your own risk - do not use for mining or merchant applications"); + + if (GetBoolArg("-testsafemode", false)) + strStatusBar = strRPC = "testsafemode enabled"; + + // Misc warnings like out of disk space and clock is wrong + if (strMiscWarning != "") + { + nPriority = 1000; + strStatusBar = strMiscWarning; + } + + if (fLargeWorkForkFound) + { + nPriority = 2000; + strStatusBar = strRPC = _("Warning: The network does not appear to fully agree! Some miners appear to be experiencing issues."); + } + else if (fLargeWorkInvalidChainFound) + { + nPriority = 2000; + strStatusBar = strRPC = _("Warning: We do not appear to fully agree with our peers! You may need to upgrade, or other nodes may need to upgrade."); + } + + // Alerts + { + LOCK(cs_mapAlerts); + BOOST_FOREACH(PAIRTYPE(const uint256, CAlert)& item, mapAlerts) + { + const CAlert& alert = item.second; + if (alert.AppliesToMe() && alert.nPriority > nPriority) + { + nPriority = alert.nPriority; + strStatusBar = alert.strStatusBar; + } + } + } + + if (strFor == "statusbar") + return strStatusBar; + else if (strFor == "rpc") + return strRPC; + assert(!"GetWarnings() : invalid parameter"); + return "error"; +} + + + + + + + + +////////////////////////////////////////////////////////////////////////////// +// +// Messages +// + + +bool static AlreadyHave(const CInv& inv) +{ + switch (inv.type) + { + case MSG_TX: + { + bool txInMap = false; + txInMap = mempool.exists(inv.hash); + return txInMap || mapOrphanTransactions.count(inv.hash) || + pcoinsTip->HaveCoins(inv.hash); + } + case MSG_BLOCK: + return mapBlockIndex.count(inv.hash); + } + // Don't know what it is, just say we already got one + return true; +} + + +void static ProcessGetData(CNode* pfrom) +{ + std::deque::iterator it = pfrom->vRecvGetData.begin(); + +/* MCHN START */ + if(!MultichainNode_RespondToGetData(pfrom)) + { + pfrom->vRecvGetData.erase(pfrom->vRecvGetData.begin(), pfrom->vRecvGetData.end()); + return; + } +/* MCHN END */ + + vector vNotFound; + + LOCK(cs_main); + + while (it != pfrom->vRecvGetData.end()) { + // Don't bother if send buffer is too full to respond anyway + if (pfrom->nSendSize >= SendBufferSize()) + break; + + const CInv &inv = *it; + { + boost::this_thread::interruption_point(); + it++; + + if (inv.type == MSG_BLOCK || inv.type == MSG_FILTERED_BLOCK) + { + bool send = false; + BlockMap::iterator mi = mapBlockIndex.find(inv.hash); + if (mi != mapBlockIndex.end()) + { + // If the requested block is at a height below our last + // checkpoint, only serve it if it's in the checkpointed chain + int nHeight = mi->second->nHeight; + CBlockIndex* pcheckpoint = Checkpoints::GetLastCheckpoint(); + if (pcheckpoint && nHeight < pcheckpoint->nHeight) { + if (!chainActive.Contains(mi->second)) + { + LogPrintf("ProcessGetData(): ignoring request for old block that isn't in the main chain\n"); + } else { + send = true; + } + } else { + send = true; + } + } + if (send) + { + // Send block from disk + CBlock block; + if (!ReadBlockFromDisk(block, (*mi).second)) + assert(!"cannot load block from disk"); + if (inv.type == MSG_BLOCK) + pfrom->PushMessage("block", block); + else // MSG_FILTERED_BLOCK) + { + LOCK(pfrom->cs_filter); + if (pfrom->pfilter) + { + CMerkleBlock merkleBlock(block, *pfrom->pfilter); + pfrom->PushMessage("merkleblock", merkleBlock); + // CMerkleBlock just contains hashes, so also push any transactions in the block the client did not see + // This avoids hurting performance by pointlessly requiring a round-trip + // Note that there is currently no way for a node to request any single transactions we didnt send here - + // they must either disconnect and retry or request the full block. + // Thus, the protocol spec specified allows for us to provide duplicate txn here, + // however we MUST always provide at least what the remote peer needs + typedef std::pair PairType; + BOOST_FOREACH(PairType& pair, merkleBlock.vMatchedTxn) + if (!pfrom->setInventoryKnown.count(CInv(MSG_TX, pair.second))) + pfrom->PushMessage("tx", block.vtx[pair.first]); + } + // else + // no response + } + + // Trigger them to send a getblocks request for the next batch of inventory + if (inv.hash == pfrom->hashContinue) + { + // Bypass PushInventory, this must send even if redundant, + // and we want it right after the last block so they don't + // wait for other stuff first. + vector vInv; + vInv.push_back(CInv(MSG_BLOCK, chainActive.Tip()->GetBlockHash())); + pfrom->PushMessage("inv", vInv); + pfrom->hashContinue = 0; + } + } + } + else if (inv.IsKnownType()) + { + // Send stream from relay memory + bool pushed = false; + { + LOCK(cs_mapRelay); + map::iterator mi = mapRelay.find(inv); + if (mi != mapRelay.end()) { + pfrom->PushMessage(inv.GetCommand(), (*mi).second); + pushed = true; + } + } + if (!pushed && inv.type == MSG_TX) { + CTransaction tx; + if (mempool.lookup(inv.hash, tx)) { + CDataStream ss(SER_NETWORK, PROTOCOL_VERSION); + ss.reserve(1000); + ss << tx; + pfrom->PushMessage("tx", ss); + pushed = true; + } + } + if (!pushed) { + vNotFound.push_back(inv); + } + } + + // Track requests for our stuff. + g_signals.Inventory(inv.hash); + + if (inv.type == MSG_BLOCK || inv.type == MSG_FILTERED_BLOCK) + break; + } + } + + pfrom->vRecvGetData.erase(pfrom->vRecvGetData.begin(), it); + + if (!vNotFound.empty()) { + // Let the peer know that we didn't find what it asked for, so it doesn't + // have to wait around forever. Currently only SPV clients actually care + // about this message: it's needed when they are recursively walking the + // dependencies of relevant unconfirmed transactions. SPV clients want to + // do that because they want to know about (and store and rebroadcast and + // risk analyze) the dependencies of transactions relevant to them, without + // having to download the entire memory pool. + pfrom->PushMessage("notfound", vNotFound); + } +} + + +/* MCHN START */ + +void CompleteProcessVersion(CNode* pfrom) +{ + if (pfrom->fInbound) + { + if (((CNetAddr)pfrom->addr) == (CNetAddr)pfrom->addrFromVersion) + { + addrman.Add(pfrom->addrFromVersion, pfrom->addrFromVersion); + addrman.Good(pfrom->addrFromVersion); + } + if (addrman.size() < 1000) + { + pfrom->PushMessage("getaddr"); + pfrom->fGetAddr = true; + } + } + + if (!pfrom->fInbound) + { + // Advertise our address + if (fListen && !IsInitialBlockDownload()) + { + CAddress addr = GetLocalAddress(&pfrom->addr); + if (addr.IsRoutable()) + { + pfrom->PushAddress(addr); + } else if (IsPeerAddrLocalGood(pfrom)) { + addr.SetIP(pfrom->addrLocal); + pfrom->PushAddress(addr); + } + } + + // Get recent addresses + if (pfrom->fOneShot || pfrom->nVersion >= CADDR_TIME_VERSION || addrman.size() < 1000) + { + pfrom->PushMessage("getaddr"); + pfrom->fGetAddr = true; + } + addrman.Good(pfrom->addr); + } + // Relay alerts + { + LOCK(cs_mapAlerts); + BOOST_FOREACH(PAIRTYPE(const uint256, CAlert)& item, mapAlerts) + item.second.RelayTo(pfrom); + } + + pfrom->fSuccessfullyConnected = true; +} + +/* MCHN END */ + + +bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, int64_t nTimeReceived) +{ +/* MCHN START */ + static int siBlockCount=0; + static int siBlocksInWindow=10; + static int siPos=-1; + static double sdTxLockTime[10]; + static double sdBlockLockTime[10]; + static double sdStartTime[10]; +/* MCHN END */ + + RandAddSeedPerfmon(); + LogPrint("net", "received: %s (%u bytes) peer=%d\n", strCommand, vRecv.size(), pfrom->id); + if (mapArgs.count("-dropmessagestest") && GetRand(atoi(mapArgs["-dropmessagestest"])) == 0) + { + LogPrintf("dropmessagestest DROPPING RECV MESSAGE\n"); + return true; + } + + + if (strCommand == "version") + { + // Each connection can only send one version message + if (pfrom->nVersion != 0) + { + pfrom->PushMessage("reject", strCommand, REJECT_DUPLICATE, string("Duplicate version message")); + Misbehaving(pfrom->GetId(), 1); + return false; + } + + int64_t nTime; + CAddress addrMe; + CAddress addrFrom; + uint64_t nNonce = 1; + vRecv >> pfrom->nVersion >> pfrom->nServices >> nTime >> addrMe; + if (pfrom->nVersion < MIN_PEER_PROTO_VERSION) + { + // disconnect from peers older than this proto version + LogPrintf("peer=%d using obsolete version %i; disconnecting\n", pfrom->id, pfrom->nVersion); + pfrom->PushMessage("reject", strCommand, REJECT_OBSOLETE, + strprintf("Version must be %d or greater", MIN_PEER_PROTO_VERSION)); + pfrom->fDisconnect = true; + return false; + } + + if (pfrom->nVersion == 10300) + pfrom->nVersion = 300; + if (!vRecv.empty()) + vRecv >> addrFrom >> nNonce; + if (!vRecv.empty()) { + vRecv >> LIMITED_STRING(pfrom->strSubVer, 256); + pfrom->cleanSubVer = SanitizeString(pfrom->strSubVer); + } + if (!vRecv.empty()) + vRecv >> pfrom->nStartingHeight; + if (!vRecv.empty()) + vRecv >> pfrom->fRelayTxes; // set to true after we get the first filter* message + else + pfrom->fRelayTxes = true; + + // Disconnect if we connected to ourself + if (nNonce == nLocalHostNonce && nNonce > 1) + { + LogPrintf("connected to self at %s, disconnecting\n", pfrom->addr.ToString()); + pfrom->fDisconnect = true; + return true; + } + + pfrom->addrLocal = addrMe; + if (pfrom->fInbound && addrMe.IsRoutable()) + { + SeenLocal(addrMe); + } + + // Be shy and don't send version until we hear + if (pfrom->fInbound) + pfrom->PushVersion(); + + pfrom->fClient = !(pfrom->nServices & NODE_NETWORK); + + // Potentially mark this peer as a preferred download peer. + UpdatePreferredDownload(pfrom, State(pfrom->GetId())); + + // Change version +/* MCHN START */ + pfrom->nVersionNonceReceived=nNonce; + pfrom->fVerackackReceived=false; + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(GetBoolArg("-bitcoinstylehandshake", false)) + { + LogPrintf("mchn: bitcoin-style-handshake, sending empty verack to peer %d... \n", pfrom->id); + pfrom->fParameterSetVerified=true; + pfrom->PushMessage("verack"); + } + else + { + PushMultiChainVerack(pfrom,false); + } + } + else + { + pfrom->fParameterSetVerified=true; + pfrom->PushMessage("verack"); + } + + if(mc_gState->m_NetworkParams->m_Status != MC_PRM_STATUS_VALID) + { + return true; + } + +/* MCHN END */ + + pfrom->ssSend.SetVersion(min(pfrom->nVersion, PROTOCOL_VERSION)); + +/* MCHN START */ + pfrom->addrFromVersion=addrFrom; +/* + if (pfrom->fInbound) + { + if (((CNetAddr)pfrom->addr) == (CNetAddr)addrFrom) + { + if(!pfrom->fDefaultMessageStart) + { + addrman.Add(addrFrom, addrFrom); + addrman.Good(addrFrom); + } + } + } +*/ +/* MCHN END */ + + if(pfrom->fParameterSetVerified) + { + CompleteProcessVersion(pfrom); + } + +/* + if (!pfrom->fInbound) + { + // Advertise our address + if (fListen && !IsInitialBlockDownload()) + { + CAddress addr = GetLocalAddress(&pfrom->addr); + if (addr.IsRoutable()) + { + pfrom->PushAddress(addr); + } else if (IsPeerAddrLocalGood(pfrom)) { + addr.SetIP(pfrom->addrLocal); + pfrom->PushAddress(addr); + } + } + + // Get recent addresses + if (pfrom->fOneShot || pfrom->nVersion >= CADDR_TIME_VERSION || addrman.size() < 1000) + { + pfrom->PushMessage("getaddr"); + pfrom->fGetAddr = true; + } + addrman.Good(pfrom->addr); + } else { + if (((CNetAddr)pfrom->addr) == (CNetAddr)addrFrom) + { + addrman.Add(addrFrom, addrFrom); + addrman.Good(addrFrom); + } + } + + // Relay alerts + { + LOCK(cs_mapAlerts); + BOOST_FOREACH(PAIRTYPE(const uint256, CAlert)& item, mapAlerts) + item.second.RelayTo(pfrom); + } + + pfrom->fSuccessfullyConnected = true; + */ + string remoteAddr; + if (fLogIPs) + remoteAddr = ", peeraddr=" + pfrom->addr.ToString(); + + LogPrintf("receive version message: %s: version %d, blocks=%d, us=%s, peer=%d%s\n", + pfrom->cleanSubVer, pfrom->nVersion, + pfrom->nStartingHeight, addrMe.ToString(), pfrom->id, + remoteAddr); + + AddTimeData(pfrom->addr, nTime); + } + + + else if (pfrom->nVersion == 0) + { + // Must have a version message before anything else + Misbehaving(pfrom->GetId(), 1); + return false; + } + + + else if (strCommand == "verack") + { + pfrom->SetRecvVersion(min(pfrom->nVersion, PROTOCOL_VERSION)); +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(vRecv.size() == 0) + { + if(mc_gState->m_NetworkState == MC_NTS_WAITING_FOR_SEED) + { + pfrom->fDisconnect = true; + mc_gState->m_NetworkState = MC_NTS_SEED_NO_PARAMS; + return true; + } + else + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanconnect") != 0) + { + LogPrintf("mchn: bitcoin-style verack received from peer %d, parameter set NOT VERIFIED, connecting... \n", pfrom->id); + pfrom->fParameterSetVerified=true; + CompleteProcessVersion(pfrom); + } + } + } + else + { + if(GetBoolArg("-bitcoinstylehandshake", false)) + { + LogPrintf("mchn: bitcoin-style-handshake, ignoring verack from peer %d \n", pfrom->id); + } + else + { + bool disconnect_flag=false; + if(!ProcessMultichainVerack(pfrom,vRecv,false,&disconnect_flag)) + { + LogPrintf("mchn: Invalid verack message from peer=%d, disconnecting\n", pfrom->id); + pfrom->fDisconnect = true; + mc_gState->m_NetworkState = MC_NTS_SEED_NO_PARAMS; + return true; + } + else + { + if(!pfrom->fDisconnect) + { + pfrom->fDisconnect = !PushMultiChainVerack(pfrom,true); + } + else + { + mc_gState->m_NetworkState = MC_NTS_SEED_READY; + return true; + } + } + if(pfrom->fDisconnect) + { + mc_gState->m_NetworkState = MC_NTS_SEED_READY; + return false; + } + } + } + } + } + + else if (strCommand == "verackack") + { + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + bool disconnect_flag=true; + if(!ProcessMultichainVerack(pfrom,vRecv,true,&disconnect_flag)) + { + pfrom->fDisconnect |= disconnect_flag; + if(pfrom->fDisconnect) + { + LogPrintf("mchn: Invalid verackack message from peer=%d, disconnecting\n", pfrom->id); + } + else + { + LogPrintf("mchn: Parameter set from peer=%d verified\n", pfrom->id); + pfrom->fParameterSetVerified=true; + CompleteProcessVersion(pfrom); + } + } + else + { + pfrom->fDisconnect |= !PushMultiChainVerack(pfrom,false); // MCHN |= to avoid setting fDisconnect to false + } + if(pfrom->fDisconnect) + { + mc_gState->m_NetworkState = MC_NTS_SEED_READY; + return false; + } + } + +/* MCHN END */ + } + + + else if (strCommand == "addr") + { + vector vAddr; + vRecv >> vAddr; + + + // Don't want addr from older versions unless seeding + if (pfrom->nVersion < CADDR_TIME_VERSION && addrman.size() > 1000) + return true; + if (vAddr.size() > 1000) + { + Misbehaving(pfrom->GetId(), 20); + return error("message addr size() = %u", vAddr.size()); + } + + // Store the new addresses + vector vAddrOk; + int64_t nNow = GetAdjustedTime(); + int64_t nSince = nNow - 10 * 60; + LogPrint("mchn","mchn: received addr: %d\n",vAddr.size()); + BOOST_FOREACH(CAddress& addr, vAddr) + { + boost::this_thread::interruption_point(); + + if (addr.nTime <= 100000000 || addr.nTime > nNow + 10 * 60) + addr.nTime = nNow - 5 * 24 * 60 * 60; + pfrom->AddAddressKnown(addr); + bool fReachable = IsReachable(addr); + if (addr.nTime > nSince && !pfrom->fGetAddr && vAddr.size() <= 10 && addr.IsRoutable()) + { + // Relay to a limited number of other nodes + { + LOCK(cs_vNodes); + // Use deterministic randomness to send to the same nodes for 24 hours + // at a time so the setAddrKnowns of the chosen nodes prevent repeats + static uint256 hashSalt; + if (hashSalt == 0) + hashSalt = GetRandHash(); + uint64_t hashAddr = addr.GetHash(); + uint256 hashRand = hashSalt ^ (hashAddr<<32) ^ ((GetTime()+hashAddr)/(24*60*60)); + hashRand = Hash(BEGIN(hashRand), END(hashRand)); + multimap mapMix; + BOOST_FOREACH(CNode* pnode, vNodes) + { + if (pnode->nVersion < CADDR_TIME_VERSION) + continue; + unsigned int nPointer; + memcpy(&nPointer, &pnode, sizeof(nPointer)); + uint256 hashKey = hashRand ^ nPointer; + hashKey = Hash(BEGIN(hashKey), END(hashKey)); + mapMix.insert(make_pair(hashKey, pnode)); + } + int nRelayNodes = fReachable ? 2 : 1; // limited relaying of addresses outside our network(s) + for (multimap::iterator mi = mapMix.begin(); mi != mapMix.end() && nRelayNodes-- > 0; ++mi) + ((*mi).second)->PushAddress(addr); + } + } + // Do not store addresses outside our network + if (fReachable) + { + if(fLogIPs) + LogPrint("mchn","mchn: Got new address %s\n",addr.ToStringIPPort().c_str()); + vAddrOk.push_back(addr); + } + } + addrman.Add(vAddrOk, pfrom->addr, 2 * 60 * 60); + if (vAddr.size() < 1000) + pfrom->fGetAddr = false; + if (pfrom->fOneShot) + pfrom->fDisconnect = true; + } + + + else if (strCommand == "inv") + { + vector vInv; + vRecv >> vInv; + if (vInv.size() > MAX_INV_SZ) + { + Misbehaving(pfrom->GetId(), 20); + return error("message inv size() = %u", vInv.size()); + } + + LOCK(cs_main); + + std::vector vToFetch; + + for (unsigned int nInv = 0; nInv < vInv.size(); nInv++) + { + const CInv &inv = vInv[nInv]; + + boost::this_thread::interruption_point(); + pfrom->AddInventoryKnown(inv); + + bool fAlreadyHave = AlreadyHave(inv); + LogPrint("net", "got inv: %s %s peer=%d\n", inv.ToString(), fAlreadyHave ? "have" : "new", pfrom->id); + + if (!fAlreadyHave && !fImporting && !fReindex && inv.type != MSG_BLOCK) + pfrom->AskFor(inv); + + if (inv.type == MSG_BLOCK) { + UpdateBlockAvailability(pfrom->GetId(), inv.hash); + if (!fAlreadyHave && !fImporting && !fReindex && !mapBlocksInFlight.count(inv.hash)) { + // First request the headers preceeding the announced block. In the normal fully-synced + // case where a new block is announced that succeeds the current tip (no reorganization), + // there are no such headers. + // Secondly, and only when we are close to being synced, we request the announced block directly, + // to avoid an extra round-trip. Note that we must *first* ask for the headers, so by the + // time the block arrives, the header chain leading up to it is already validated. Not + // doing this will result in the received block being rejected as an orphan in case it is + // not a direct successor. + pfrom->PushMessage("getheaders", chainActive.GetLocator(pindexBestHeader), inv.hash); + CNodeState *nodestate = State(pfrom->GetId()); + if (chainActive.Tip()->GetBlockTime() > GetAdjustedTime() - Params().TargetSpacing() * 20 && + nodestate->nBlocksInFlight < MAX_BLOCKS_IN_TRANSIT_PER_PEER) { + vToFetch.push_back(inv); + // Mark block as in flight already, even though the actual "getdata" message only goes out + // later (within the same cs_main lock, though). + MarkBlockAsInFlight(pfrom->GetId(), inv.hash); + } + LogPrint("net", "getheaders (%d) %s to peer=%d\n", pindexBestHeader->nHeight, inv.hash.ToString(), pfrom->id); + } + } + + // Track requests for our stuff + g_signals.Inventory(inv.hash); + + if (pfrom->nSendSize > (SendBufferSize() * 2)) { + Misbehaving(pfrom->GetId(), 50); + return error("send buffer size() = %u", pfrom->nSendSize); + } + } + + if (!vToFetch.empty() && !MultichainNode_IgnoreIncoming(pfrom)) // MCHN + pfrom->PushMessage("getdata", vToFetch); + } + + + else if (strCommand == "getdata") + { + vector vInv; + vRecv >> vInv; + if (vInv.size() > MAX_INV_SZ) + { + Misbehaving(pfrom->GetId(), 20); + return error("message getdata size() = %u", vInv.size()); + } + + if (fDebug || (vInv.size() != 1)) + LogPrint("net", "received getdata (%u invsz) peer=%d\n", vInv.size(), pfrom->id); + + if ((fDebug && vInv.size() > 0) || (vInv.size() == 1)) + LogPrint("net", "received getdata for: %s peer=%d\n", vInv[0].ToString(), pfrom->id); + + pfrom->vRecvGetData.insert(pfrom->vRecvGetData.end(), vInv.begin(), vInv.end()); + ProcessGetData(pfrom); + } + + + else if (strCommand == "getblocks") + { + CBlockLocator locator; + uint256 hashStop; + vRecv >> locator >> hashStop; + + LOCK(cs_main); + + // Find the last block the caller has in the main chain + CBlockIndex* pindex = FindForkInGlobalIndex(chainActive, locator); + + // Send the rest of the chain + if (pindex) + pindex = chainActive.Next(pindex); + int nLimit = 500; + LogPrint("net", "getblocks %d to %s limit %d from peer=%d\n", (pindex ? pindex->nHeight : -1), hashStop==uint256(0) ? "end" : hashStop.ToString(), nLimit, pfrom->id); + for (; pindex; pindex = chainActive.Next(pindex)) + { + if (pindex->GetBlockHash() == hashStop) + { + LogPrint("net", " getblocks stopping at %d %s\n", pindex->nHeight, pindex->GetBlockHash().ToString()); + break; + } + pfrom->PushInventory(CInv(MSG_BLOCK, pindex->GetBlockHash())); + if (--nLimit <= 0) + { + // When this block is requested, we'll send an inv that'll make them + // getblocks the next batch of inventory. + LogPrint("net", " getblocks stopping at limit %d %s\n", pindex->nHeight, pindex->GetBlockHash().ToString()); + pfrom->hashContinue = pindex->GetBlockHash(); + break; + } + } + } + + + else if (strCommand == "getheaders") + { + CBlockLocator locator; + uint256 hashStop; + vRecv >> locator >> hashStop; + + LOCK(cs_main); + + CBlockIndex* pindex = NULL; + if (locator.IsNull()) + { + // If locator is null, return the hashStop block + BlockMap::iterator mi = mapBlockIndex.find(hashStop); + if (mi == mapBlockIndex.end()) + return true; + pindex = (*mi).second; + } + else + { + // Find the last block the caller has in the main chain + pindex = FindForkInGlobalIndex(chainActive, locator); + if (pindex) + pindex = chainActive.Next(pindex); + } + + // we must use CBlocks, as CBlockHeaders won't include the 0x00 nTx count at the end + vector vHeaders; + int nLimit = MAX_HEADERS_RESULTS; + LogPrint("net", "getheaders %d to %s from peer=%d\n", (pindex ? pindex->nHeight : -1), hashStop.ToString(), pfrom->id); + for (; pindex; pindex = chainActive.Next(pindex)) + { + vHeaders.push_back(pindex->GetBlockHeader()); + if (--nLimit <= 0 || pindex->GetBlockHash() == hashStop) + break; + } + pfrom->PushMessage("headers", vHeaders); + } + + + else if (strCommand == "tx" && MultichainNode_AcceptData(pfrom)) // MCHN + { +/* MCHN START */ + if(!MultichainNode_IgnoreIncoming(pfrom)) + { +/* MCHN END */ + vector vWorkQueue; + vector vEraseQueue; + CTransaction tx; + vRecv >> tx; + + CInv inv(MSG_TX, tx.GetHash()); + pfrom->AddInventoryKnown(inv); + + LOCK(cs_main); + + double start_time=mc_TimeNowAsDouble(); + + bool fMissingInputs = false; + CValidationState state; + + mapAlreadyAskedFor.erase(inv); + +/* MCHN START */ + CPubKey pubkey; + uint32_t fCanMine=((pwalletMain != NULL) && pwalletMain->GetKeyFromAddressBook(pubkey,MC_PTP_MINE)) ? MC_PTP_MINE : 0; +/* MCHN END */ + + if (AcceptToMemoryPool(mempool, state, tx, true, &fMissingInputs)) + { + mempool.check(pcoinsTip); + RelayTransaction(tx); + vWorkQueue.push_back(inv.hash); + vEraseQueue.push_back(inv.hash); + + LogPrint("mempool", "AcceptToMemoryPool: peer=%d %s : accepted %s (poolsz %u)\n", + pfrom->id, pfrom->cleanSubVer, + tx.GetHash().ToString(), + mempool.mapTx.size()); + + // Recursively process any orphan transactions that depended on this one + set setMisbehaving; + for (unsigned int i = 0; i < vWorkQueue.size(); i++) + { + map >::iterator itByPrev = mapOrphanTransactionsByPrev.find(vWorkQueue[i]); + if (itByPrev == mapOrphanTransactionsByPrev.end()) + continue; + for (set::iterator mi = itByPrev->second.begin(); + mi != itByPrev->second.end(); + ++mi) + { + const uint256& orphanHash = *mi; + const CTransaction& orphanTx = mapOrphanTransactions[orphanHash].tx; + NodeId fromPeer = mapOrphanTransactions[orphanHash].fromPeer; + bool fMissingInputs2 = false; + // Use a dummy CValidationState so someone can't setup nodes to counter-DoS based on orphan + // resolution (that is, feeding people an invalid transaction based on LegitTxX in order to get + // anyone relaying LegitTxX banned) + CValidationState stateDummy; + + vEraseQueue.push_back(orphanHash); + + if (setMisbehaving.count(fromPeer)) + continue; + if (AcceptToMemoryPool(mempool, stateDummy, orphanTx, true, &fMissingInputs2)) + { + LogPrint("mempool", " accepted orphan tx %s\n", orphanHash.ToString()); + RelayTransaction(orphanTx); + vWorkQueue.push_back(orphanHash); + } + else if (!fMissingInputs2) + { + int nDos = 0; + if (stateDummy.IsInvalid(nDos) && nDos > 0) + { + // Punish peer that gave us an invalid orphan tx + Misbehaving(fromPeer, nDos); + setMisbehaving.insert(fromPeer); + LogPrint("mempool", " invalid orphan tx %s\n", orphanHash.ToString()); + } + // too-little-fee orphan + LogPrint("mempool", " removed orphan tx %s\n", orphanHash.ToString()); + } + mempool.check(pcoinsTip); + } + } + + BOOST_FOREACH(uint256 hash, vEraseQueue) + EraseOrphanTx(hash); + +/* MCHN START */ + if(pwalletMain) + { + if(fCanMine) + { + if(!pwalletMain->GetKeyFromAddressBook(pubkey,MC_PTP_MINE)) + { + LogPrint("mchn","mchn: Wallet lost mine permission on tx: %s (height %d) - message, reactivating best chain\n", + tx.GetHash().ToString().c_str(), chainActive.Tip()->nHeight); + if (!ActivateBestChain(state, NULL)) + return error("%s : ActivateBestChain failed", __func__); + } + } + } +/* MCHN END */ + + + } + else if (fMissingInputs) + { +/* MCHN START */ + bool found_in_oldblocks=false; + for(int d=0;dm_Permissions->m_Block>d) + { + if(setBlockTransactions[(mc_gState->m_Permissions->m_Block-d) % MC_TXSET_BLOCKS].count(tx.GetHash())) + { + found_in_oldblocks=true; + } + } + } + } + + if(!found_in_oldblocks) + { +/* MCHN END */ + AddOrphanTx(tx, pfrom->GetId()); + + // DoS prevention: do not allow mapOrphanTransactions to grow unbounded + unsigned int nMaxOrphanTx = (unsigned int)std::max((int64_t)0, GetArg("-maxorphantx", DEFAULT_MAX_ORPHAN_TRANSACTIONS)); + unsigned int nEvicted = LimitOrphanTxSize(nMaxOrphanTx); + if (nEvicted > 0) + LogPrint("mempool", "mapOrphan overflow, removed %u tx\n", nEvicted); +/* MCHN START */ + } +/* MCHN END */ + } else if (pfrom->fWhitelisted) { + // Always relay transactions received from whitelisted peers, even + // if they are already in the mempool (allowing the node to function + // as a gateway for nodes hidden behind it). + RelayTransaction(tx); + } + int nDoS = 0; + if (state.IsInvalid(nDoS)) + { + LogPrint("mempool", "%s from peer=%d %s was not accepted into the memory pool: %s\n", tx.GetHash().ToString(), + pfrom->id, pfrom->cleanSubVer, + state.GetRejectReason()); + pfrom->PushMessage("reject", strCommand, state.GetRejectCode(), + state.GetRejectReason().substr(0, MAX_REJECT_MESSAGE_LENGTH), inv.hash); + if (nDoS > 0) + Misbehaving(pfrom->GetId(), nDoS); + } + + double end_time=mc_TimeNowAsDouble(); + if(siPos>=0) + { + sdTxLockTime[siPos]+=end_time-start_time; + } +/* MCHN START */ + } +/* MCHN END */ + } + + + else if (strCommand == "headers" && !fImporting && !fReindex) // Ignore headers received while importing + { + std::vector headers; + + // Bypass the normal CBlock deserialization, as we don't want to risk deserializing 2000 full blocks. + unsigned int nCount = ReadCompactSize(vRecv); + if (nCount > MAX_HEADERS_RESULTS) { + Misbehaving(pfrom->GetId(), 20); + return error("headers message size = %u", nCount); + } + headers.resize(nCount); + for (unsigned int n = 0; n < nCount; n++) { + vRecv >> headers[n]; + ReadCompactSize(vRecv); // ignore tx count; assume it is 0. + } + + LOCK(cs_main); + + if (nCount == 0) { + // Nothing interesting. Stop asking this peers for more headers. + return true; + } + + CBlockIndex *pindexLast = NULL; + BOOST_FOREACH(const CBlockHeader& header, headers) { + CValidationState state; + if (pindexLast != NULL && header.hashPrevBlock != pindexLast->GetBlockHash()) { + Misbehaving(pfrom->GetId(), 20); + return error("non-continuous headers sequence"); + } + if (!AcceptBlockHeader(header, state, &pindexLast)) { + int nDoS; + if (state.IsInvalid(nDoS)) { + if (nDoS > 0) + Misbehaving(pfrom->GetId(), nDoS); + return error("invalid header received"); + } + } + } + + if (pindexLast) + UpdateBlockAvailability(pfrom->GetId(), pindexLast->GetBlockHash()); + + if (nCount == MAX_HEADERS_RESULTS && pindexLast) { + // Headers message had its maximum size; the peer may have more headers. + // TODO: optimize: if pindexLast is an ancestor of chainActive.Tip or pindexBestHeader, continue + // from there instead. + LogPrint("net", "more getheaders (%d) to end to peer=%d (startheight:%d)\n", pindexLast->nHeight, pfrom->id, pfrom->nStartingHeight); + pfrom->PushMessage("getheaders", chainActive.GetLocator(pindexLast), uint256(0)); + } + } + // MCHN + else if (strCommand == "block" && !fImporting && !fReindex && MultichainNode_AcceptData(pfrom)) // Ignore blocks received while importing + { + CBlock block; + vRecv >> block; +/* MCHN START */ + CNode* seed_node; + bool seed_could_connect=false; + { + LOCK(cs_main); + seed_node=(CNode*)(mc_gState->m_pSeedNode); + if(seed_node) + { + seed_could_connect=mc_gState->m_Permissions->CanConnect(NULL,seed_node->kAddrRemote.begin()); + } + } +/* MCHN END */ + + CInv inv(MSG_BLOCK, block.GetHash()); + +/* MCHN START */ + if(!MultichainNode_IgnoreIncoming(pfrom)) + { +/* MCHN END */ + + LogPrint("net", "received block %s peer=%d\n", inv.hash.ToString(), pfrom->id); + + pfrom->AddInventoryKnown(inv); + + double start_time=mc_TimeNowAsDouble(); + + CValidationState state; + ProcessNewBlock(state, pfrom, &block); + int nDoS; + if (state.IsInvalid(nDoS)) { + pfrom->PushMessage("reject", strCommand, state.GetRejectCode(), + state.GetRejectReason().substr(0, MAX_REJECT_MESSAGE_LENGTH), inv.hash); + if (nDoS > 0) { + LOCK(cs_main); + Misbehaving(pfrom->GetId(), nDoS); + } + } + +/* MCHN START */ + + { + LOCK(cs_main); + seed_node=(CNode*)(mc_gState->m_pSeedNode); + if(seed_node) + { + LOCK(cs_main); + if(seed_could_connect && !mc_gState->m_Permissions->CanConnect(NULL,seed_node->kAddrRemote.begin())) + { + if(vNodes.size() > 1) + { + LogPrintf("mchn: Seed node lost connect permission on block %d\n",mc_gState->m_Permissions->m_Block); + mc_RemoveFile(mc_gState->m_NetworkParams->Name(),"seed",".dat",MC_FOM_RELATIVE_TO_DATADIR); + mc_gState->m_pSeedNode=NULL; + } + } + } + } + + double end_time=mc_TimeNowAsDouble(); + if(siPos>=0) + { + sdBlockLockTime[siPos]+=end_time-start_time; + } + + if(siPos>=0) + { + int block_count=siBlockCount; + if(block_count>siBlocksInWindow) + { + block_count=siBlocksInWindow; + } + double tx_lock_time=0.; + double block_lock_time=0.; + double block_start_time=0; + for(int i=0;inHeight,siBlockCount,(int)block.vtx.size(), + block_total_time/block_count, + tx_lock_time/block_count,100.*tx_lock_time/block_total_time,block_lock_time/block_count,100.*block_lock_time/block_total_time); + + } + + siPos=(siPos+1)%siBlocksInWindow; + siBlockCount++; + sdTxLockTime[siPos]=0.; + sdBlockLockTime[siPos]=0.; + sdStartTime[siPos]=end_time; +/* MCHN END */ +/* MCHN START */ + } + else + { + LogPrint("net", "ignored block %s peer=%d\n", inv.hash.ToString(), pfrom->id); + pfrom->AskFor(inv); + } +/* MCHN END */ + } + + + else if (strCommand == "getaddr") + { + pfrom->vAddrToSend.clear(); + vector vAddr = addrman.GetAddr(); + BOOST_FOREACH(const CAddress &addr, vAddr) + pfrom->PushAddress(addr); +/* MCHN START */ + LogPrint("mchn","mchn: Sent %d known addresses\n",vAddr.size()); +/* MCHN END */ + } + + + else if (strCommand == "mempool" && MultichainNode_SendInv(pfrom)) // MCHN + { + LOCK2(cs_main, pfrom->cs_filter); + + std::vector vtxid; + mempool.queryHashes(vtxid); + vector vInv; + BOOST_FOREACH(uint256& hash, vtxid) { + CInv inv(MSG_TX, hash); + CTransaction tx; + bool fInMemPool = mempool.lookup(hash, tx); + if (!fInMemPool) continue; // another thread removed since queryHashes, maybe... + if ((pfrom->pfilter && pfrom->pfilter->IsRelevantAndUpdate(tx)) || + (!pfrom->pfilter)) + vInv.push_back(inv); + if (vInv.size() == MAX_INV_SZ) { + pfrom->PushMessage("inv", vInv); + vInv.clear(); + } + } + if (vInv.size() > 0) + pfrom->PushMessage("inv", vInv); + } + + + else if (strCommand == "ping") + { + if (pfrom->nVersion > BIP0031_VERSION) + { + uint64_t nonce = 0; + vRecv >> nonce; + // Echo the message back with the nonce. This allows for two useful features: + // + // 1) A remote node can quickly check if the connection is operational + // 2) Remote nodes can measure the latency of the network thread. If this node + // is overloaded it won't respond to pings quickly and the remote node can + // avoid sending us more work, like chain download requests. + // + // The nonce stops the remote getting confused between different pings: without + // it, if the remote node sends a ping once per second and this node takes 5 + // seconds to respond to each, the 5th ping the remote sends would appear to + // return very quickly. + pfrom->PushMessage("pong", nonce); + } + } + + + else if (strCommand == "pong") + { + int64_t pingUsecEnd = nTimeReceived; + uint64_t nonce = 0; + size_t nAvail = vRecv.in_avail(); + bool bPingFinished = false; + std::string sProblem; + + if (nAvail >= sizeof(nonce)) { + vRecv >> nonce; + + // Only process pong message if there is an outstanding ping (old ping without nonce should never pong) + if (pfrom->nPingNonceSent != 0) { + if (nonce == pfrom->nPingNonceSent) { + // Matching pong received, this ping is no longer outstanding + bPingFinished = true; + int64_t pingUsecTime = pingUsecEnd - pfrom->nPingUsecStart; + if (pingUsecTime > 0) { + // Successful ping time measurement, replace previous + pfrom->nPingUsecTime = pingUsecTime; + } else { + // This should never happen + sProblem = "Timing mishap"; + } + } else { + // Nonce mismatches are normal when pings are overlapping + sProblem = "Nonce mismatch"; + if (nonce == 0) { + // This is most likely a bug in another implementation somewhere, cancel this ping + bPingFinished = true; + sProblem = "Nonce zero"; + } + } + } else { + sProblem = "Unsolicited pong without ping"; + } + } else { + // This is most likely a bug in another implementation somewhere, cancel this ping + bPingFinished = true; + sProblem = "Short payload"; + } + + if (!(sProblem.empty())) { + LogPrint("net", "pong peer=%d %s: %s, %x expected, %x received, %u bytes\n", + pfrom->id, + pfrom->cleanSubVer, + sProblem, + pfrom->nPingNonceSent, + nonce, + nAvail); + } + if (bPingFinished) { + pfrom->nPingNonceSent = 0; + } + } + + + else if (strCommand == "alert") + { + CAlert alert; + vRecv >> alert; + + uint256 alertHash = alert.GetHash(); + if (pfrom->setKnown.count(alertHash) == 0) + { + if (alert.ProcessAlert()) + { + // Relay + pfrom->setKnown.insert(alertHash); + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + alert.RelayTo(pnode); + } + } + else { + // Small DoS penalty so peers that send us lots of + // duplicate/expired/invalid-signature/whatever alerts + // eventually get banned. + // This isn't a Misbehaving(100) (immediate ban) because the + // peer might be an older or different implementation with + // a different signature key, etc. + Misbehaving(pfrom->GetId(), 10); + } + } + } + + + else if (strCommand == "filterload") + { + CBloomFilter filter; + vRecv >> filter; + + if (!filter.IsWithinSizeConstraints()) + // There is no excuse for sending a too-large filter + Misbehaving(pfrom->GetId(), 100); + else + { + LOCK(pfrom->cs_filter); + delete pfrom->pfilter; + pfrom->pfilter = new CBloomFilter(filter); + pfrom->pfilter->UpdateEmptyFull(); + } + pfrom->fRelayTxes = true; + } + + + else if (strCommand == "filteradd") + { + vector vData; + vRecv >> vData; + + // Nodes must NEVER send a data item > 520 bytes (the max size for a script data object, + // and thus, the maximum size any matched object can have) in a filteradd message + if (vData.size() > MAX_SCRIPT_ELEMENT_SIZE) + { + Misbehaving(pfrom->GetId(), 100); + } else { + LOCK(pfrom->cs_filter); + if (pfrom->pfilter) + pfrom->pfilter->insert(vData); + else + Misbehaving(pfrom->GetId(), 100); + } + } + + + else if (strCommand == "filterclear") + { + LOCK(pfrom->cs_filter); + delete pfrom->pfilter; + pfrom->pfilter = new CBloomFilter(); + pfrom->fRelayTxes = true; + } + + + else if (strCommand == "reject") + { + if (fDebug) { + try { + string strMsg; unsigned char ccode; string strReason; + vRecv >> LIMITED_STRING(strMsg, CMessageHeader::COMMAND_SIZE) >> ccode >> LIMITED_STRING(strReason, MAX_REJECT_MESSAGE_LENGTH); + + ostringstream ss; + ss << strMsg << " code " << itostr(ccode) << ": " << strReason; + + if (strMsg == "block" || strMsg == "tx") + { + uint256 hash; + vRecv >> hash; + ss << ": hash " << hash.ToString(); + } + LogPrint("net", "Reject %s\n", SanitizeString(ss.str())); + } catch (std::ios_base::failure& e) { + // Avoid feedback loops by preventing reject messages from triggering a new reject message. + LogPrint("net", "Unparseable reject message received\n"); + } + } + } + + else + { + // Ignore unknown commands for extensibility + LogPrint("net", "Unknown command \"%s\" from peer=%d\n", SanitizeString(strCommand), pfrom->id); + } + + + // Update the last seen time for this node's address + if (pfrom->fNetworkNode) + if (strCommand == "version" || strCommand == "addr" || strCommand == "inv" || strCommand == "getdata" || strCommand == "ping") + AddressCurrentlyConnected(pfrom->addr); + + + return true; +} + +// requires LOCK(cs_vRecvMsg) +bool ProcessMessages(CNode* pfrom) +{ + //if (fDebug) + // LogPrintf("ProcessMessages(%u messages)\n", pfrom->vRecvMsg.size()); + + // + // Message format + // (4) message start + // (12) command + // (4) size + // (4) checksum + // (x) data + // + bool fOk = true; + + if (!pfrom->vRecvGetData.empty()) + ProcessGetData(pfrom); + + // this maintains the order of responses + if (!pfrom->vRecvGetData.empty()) return fOk; + +/* MCHN START */ + { + LOCK(cs_main); + + if(!pfrom->fDisconnect) + { + if(MultichainNode_DisconnectRemote(pfrom)) + { + pfrom->fDisconnect=true; + LogPrintf("mchn: Address %s lost connect permission on peer=%d, diconnecting...\n",CBitcoinAddress(pfrom->kAddrRemote).ToString().c_str(), pfrom->id); + + } + if(!pfrom->fDisconnect) + { + if(pfrom->fCanConnectLocal && MultichainNode_DisconnectLocal(pfrom)) + { + pfrom->fDisconnect=true; + LogPrintf("mchn: Local address %s lost connect permission. disconnecting peer %d...\n",CBitcoinAddress(pfrom->kAddrLocal).ToString().c_str(), pfrom->id); + } + } + } + } + +/* MCHN END */ + + std::deque::iterator it = pfrom->vRecvMsg.begin(); + while (!pfrom->fDisconnect && it != pfrom->vRecvMsg.end()) { + // Don't bother if send buffer is too full to respond anyway + if (pfrom->nSendSize >= SendBufferSize()) + break; + + // get next message + CNetMessage& msg = *it; + + //if (fDebug) + // LogPrintf("ProcessMessages(message %u msgsz, %u bytes, complete:%s)\n", + // msg.hdr.nMessageSize, msg.vRecv.size(), + // msg.complete() ? "Y" : "N"); + + // end, if an incomplete message is found + if (!msg.complete()) + break; + + // at this point, any failure means we can delete the current message + it++; + + // Scan for message start +/* MCHN START */ + + bool fSkipMessageStartCheck=false; + + pfrom->fDefaultMessageStart=false; + + if (memcmp(msg.hdr.pchMessageStart, Params().MessageStart(), MESSAGE_START_SIZE) != 0) { + fSkipMessageStartCheck=true; + if (memcmp(msg.hdr.pchMessageStart, mc_gState->m_NetworkParams->DefaultMessageStart(), MESSAGE_START_SIZE) != 0) { + if(mc_gState->m_NetworkParams->m_Status != MC_PRM_STATUS_EMPTY) + { + LogPrintf("mchn: PROCESSMESSAGE: INVALID MESSAGESTART %s peer=%d\n", msg.hdr.GetCommand(), pfrom->id); + fOk = false; + break; + } + } + else + { + pfrom->fDefaultMessageStart=true; + } +/* MCHN END */ + } + + // Read header + CMessageHeader& hdr = msg.hdr; + if (!hdr.IsValid(fSkipMessageStartCheck)) + { + LogPrintf("mchn: PROCESSMESSAGE: ERRORS IN HEADER %s peer=%d\n", hdr.GetCommand(), pfrom->id); + continue; + } + string strCommand = hdr.GetCommand(); + +/* MCHN START */ + LogPrint("mchnminor","mchn: RECV: %s, peer=%d\n", hdr.GetCommand(), pfrom->id); + + if(mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_EMPTY) + { + if((strCommand != "verack") && (strCommand != "version")) + { + LogPrintf("IGNORED %s, peer=%d\n", hdr.GetCommand(), pfrom->id); + fOk = false; + break; + } + } + + if(!pfrom->fParameterSetVerified) + { + if((strCommand != "verackack") && (strCommand != "verack") && (strCommand != "version")) + { + LogPrintf("mchn: IGNORED %s, peer=%d\n", hdr.GetCommand(), pfrom->id); + fOk = false; + break; + } + } + + +/* MCHN END */ + // Message size + unsigned int nMessageSize = hdr.nMessageSize; + + // Checksum + CDataStream& vRecv = msg.vRecv; + uint256 hash = Hash(vRecv.begin(), vRecv.begin() + nMessageSize); + unsigned int nChecksum = 0; + memcpy(&nChecksum, &hash, sizeof(nChecksum)); + if (nChecksum != hdr.nChecksum) + { + LogPrintf("ProcessMessages(%s, %u bytes) : CHECKSUM ERROR nChecksum=%08x hdr.nChecksum=%08x\n", + strCommand, nMessageSize, nChecksum, hdr.nChecksum); + continue; + } + + // Process message + bool fRet = false; + try + { +/* MCHN START */ + if(pfrom->fDisconnect || !MultichainNode_DisconnectRemote(pfrom)) + { +/* MCHN END */ + fRet = ProcessMessage(pfrom, strCommand, vRecv, msg.nTime); +/* MCHN START */ + } +/* MCHN END */ + boost::this_thread::interruption_point(); + } + catch (std::ios_base::failure& e) + { + pfrom->PushMessage("reject", strCommand, REJECT_MALFORMED, string("error parsing message")); + if (strstr(e.what(), "end of data")) + { + // Allow exceptions from under-length message on vRecv + LogPrintf("ProcessMessages(%s, %u bytes) : Exception '%s' caught, normally caused by a message being shorter than its stated length\n", strCommand, nMessageSize, e.what()); + } + else if (strstr(e.what(), "size too large")) + { + // Allow exceptions from over-long size + LogPrintf("ProcessMessages(%s, %u bytes) : Exception '%s' caught\n", strCommand, nMessageSize, e.what()); + } + else + { + PrintExceptionContinue(&e, "ProcessMessages()"); + } + } + catch (boost::thread_interrupted) { + throw; + } + catch (std::exception& e) { + PrintExceptionContinue(&e, "ProcessMessages()"); + } catch (...) { + PrintExceptionContinue(NULL, "ProcessMessages()"); + } + + if (!fRet) + LogPrintf("ProcessMessage(%s, %u bytes) FAILED peer=%d\n", strCommand, nMessageSize, pfrom->id); + + break; + } + + // In case the connection got shut down, its receive buffer was wiped + if (!pfrom->fDisconnect) + pfrom->vRecvMsg.erase(pfrom->vRecvMsg.begin(), it); + + return fOk; +} + + +bool SendMessages(CNode* pto, bool fSendTrickle) +{ + { + // Don't send anything until we get their version message + if (pto->nVersion == 0) + return true; + + // + // Message: ping + // + bool pingSend = false; + if (pto->fPingQueued) { + // RPC ping request by user + pingSend = true; + } + if (pto->nPingNonceSent == 0 && pto->nPingUsecStart + PING_INTERVAL * 1000000 < GetTimeMicros()) { + // Ping automatically sent as a latency probe & keepalive. + pingSend = true; + } + if (pingSend) { + uint64_t nonce = 0; + while (nonce == 0) { + GetRandBytes((unsigned char*)&nonce, sizeof(nonce)); + } + pto->fPingQueued = false; + pto->nPingUsecStart = GetTimeMicros(); + if (pto->nVersion > BIP0031_VERSION) { + pto->nPingNonceSent = nonce; + pto->PushMessage("ping", nonce); + } else { + // Peer is too old to support ping command with nonce, pong will never arrive. + pto->nPingNonceSent = 0; + pto->PushMessage("ping"); + } + } + + TRY_LOCK(cs_main, lockMain); // Acquire cs_main for IsInitialBlockDownload() and CNodeState() + if (!lockMain) + return true; + + // Address refresh broadcast + static int64_t nLastRebroadcast; + if (!IsInitialBlockDownload() && (GetTime() - nLastRebroadcast > 24 * 60 * 60)) + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + { + // Periodically clear setAddrKnown to allow refresh broadcasts + if (nLastRebroadcast) + pnode->setAddrKnown.clear(); + + // Rebroadcast our address + AdvertizeLocal(pnode); + } + if (!vNodes.empty()) + nLastRebroadcast = GetTime(); + } + + // + // Message: addr + // + if (fSendTrickle) + { + vector vAddr; + vAddr.reserve(pto->vAddrToSend.size()); + BOOST_FOREACH(const CAddress& addr, pto->vAddrToSend) + { + // returns true if wasn't already contained in the set + if (pto->setAddrKnown.insert(addr).second) + { + vAddr.push_back(addr); + // receiver rejects addr messages larger than 1000 + if (vAddr.size() >= 1000) + { + pto->PushMessage("addr", vAddr); + vAddr.clear(); + } + } + } + pto->vAddrToSend.clear(); + if (!vAddr.empty()) + pto->PushMessage("addr", vAddr); + } + + CNodeState &state = *State(pto->GetId()); + if (state.fShouldBan) { + if (pto->fWhitelisted) + LogPrintf("Warning: not punishing whitelisted peer %s!\n", pto->addr.ToString()); + else { + pto->fDisconnect = true; + if (pto->addr.IsLocal()) + LogPrintf("Warning: not banning local peer %s!\n", pto->addr.ToString()); + else + { + CNode::Ban(pto->addr); + } + } + state.fShouldBan = false; + } + + BOOST_FOREACH(const CBlockReject& reject, state.rejects) + pto->PushMessage("reject", (string)"block", reject.chRejectCode, reject.strRejectReason, reject.hashBlock); + state.rejects.clear(); + + // Start block sync + if (pindexBestHeader == NULL) + pindexBestHeader = chainActive.Tip(); + bool fFetch = state.fPreferredDownload || (nPreferredDownload == 0 && !pto->fClient && !pto->fOneShot); // Download if this is a nice peer, or we have no nice peers and this one might do. + if (!state.fSyncStarted && !pto->fClient && fFetch && !fImporting && !fReindex) { + // Only actively request headers from a single peer, unless we're close to today. + if (nSyncStarted == 0 || pindexBestHeader->GetBlockTime() > GetAdjustedTime() - 24 * 60 * 60) { + state.fSyncStarted = true; + nSyncStarted++; + CBlockIndex *pindexStart = pindexBestHeader->pprev ? pindexBestHeader->pprev : pindexBestHeader; + LogPrint("net", "initial getheaders (%d) to peer=%d (startheight:%d)\n", pindexStart->nHeight, pto->id, pto->nStartingHeight); + pto->PushMessage("getheaders", chainActive.GetLocator(pindexStart), uint256(0)); + } + } + + // Resend wallet transactions that haven't gotten in a block yet + // Except during reindex, importing and IBD, when old wallet + // transactions become unconfirmed and spams other nodes. + if (!fReindex && !fImporting && !IsInitialBlockDownload()) + { + g_signals.Broadcast(); + } + + // + // Message: inventory + // + vector vInv; + vector vInvWait; + { + LOCK(pto->cs_inventory); + vInv.reserve(pto->vInventoryToSend.size()); + vInvWait.reserve(pto->vInventoryToSend.size()); + BOOST_FOREACH(const CInv& inv, pto->vInventoryToSend) + { + if (pto->setInventoryKnown.count(inv)) + continue; + + // trickle out tx inv to protect privacy + if (inv.type == MSG_TX && !fSendTrickle) + { + // 1/4 of tx invs blast to all immediately + static uint256 hashSalt; + if (hashSalt == 0) + hashSalt = GetRandHash(); + uint256 hashRand = inv.hash ^ hashSalt; + hashRand = Hash(BEGIN(hashRand), END(hashRand)); + bool fTrickleWait = ((hashRand & 3) != 0); + + if (fTrickleWait) + { + vInvWait.push_back(inv); + continue; + } + } + + // returns true if wasn't already contained in the set + if (pto->setInventoryKnown.insert(inv).second) + { + vInv.push_back(inv); + if (vInv.size() >= 1000) + { + if(MultichainNode_SendInv(pto)) // MCNN + pto->PushMessage("inv", vInv); + vInv.clear(); + } + } + } + pto->vInventoryToSend = vInvWait; + } + if (!vInv.empty()) + if(MultichainNode_SendInv(pto)) // MCNN + pto->PushMessage("inv", vInv); + + // Detect whether we're stalling + int64_t nNow = GetTimeMicros(); + if (!pto->fDisconnect && state.nStallingSince && state.nStallingSince < nNow - 1000000 * BLOCK_STALLING_TIMEOUT) { + // Stalling only triggers when the block download window cannot move. During normal steady state, + // the download window should be much larger than the to-be-downloaded set of blocks, so disconnection + // should only happen during initial block download. + LogPrintf("Peer=%d is stalling block download, disconnecting\n", pto->id); + pto->fDisconnect = true; + } +/* MCHN START */ + bool ignore_incoming=false; + { + LOCK(cs_main); + ignore_incoming=MultichainNode_IgnoreIncoming(pto); + } + if(!ignore_incoming) + { +/* MCHN END */ + // In case there is a block that has been in flight from this peer for (2 + 0.5 * N) times the block interval + // (with N the number of validated blocks that were in flight at the time it was requested), disconnect due to + // timeout. We compensate for in-flight blocks to prevent killing off peers due to our own downstream link + // being saturated. We only count validated in-flight blocks so peers can't advertize nonexisting block hashes + // to unreasonably increase our timeout. + if (!pto->fDisconnect && state.vBlocksInFlight.size() > 0 && state.vBlocksInFlight.front().nTime < nNow - 500000 * Params().TargetSpacing() * (4 + state.vBlocksInFlight.front().nValidatedQueuedBefore)) { + LogPrintf("Timeout downloading block %s from peer=%d, disconnecting\n", state.vBlocksInFlight.front().hash.ToString(), pto->id); + pto->fDisconnect = true; + } +/* MCHN START */ + } +/* MCHN END */ + + // + // Message: getdata (blocks) + // + vector vGetData; +/* MCHN START */ + if(!ignore_incoming) + { +/* MCHN END */ + if (!pto->fDisconnect && !pto->fClient && fFetch && state.nBlocksInFlight < MAX_BLOCKS_IN_TRANSIT_PER_PEER) { + vector vToDownload; + NodeId staller = -1; + FindNextBlocksToDownload(pto->GetId(), MAX_BLOCKS_IN_TRANSIT_PER_PEER - state.nBlocksInFlight, vToDownload, staller); + BOOST_FOREACH(CBlockIndex *pindex, vToDownload) { + vGetData.push_back(CInv(MSG_BLOCK, pindex->GetBlockHash())); + MarkBlockAsInFlight(pto->GetId(), pindex->GetBlockHash(), pindex); + LogPrint("net", "Requesting block %s (%d) peer=%d\n", pindex->GetBlockHash().ToString(), + pindex->nHeight, pto->id); + } + if (state.nBlocksInFlight == 0 && staller != -1) { + if (State(staller)->nStallingSince == 0) { + State(staller)->nStallingSince = nNow; + LogPrint("net", "Stall started peer=%d\n", staller); + } + } + } +/* MCHN START */ + } +/* MCHN END */ + +/* MCHN START */ + { + LOCK(cs_main); + CNode* seed_node; + if(!pto->fSyncedOnce && MultichainNode_IsBlockChainSynced(pto)) + { + LogPrintf("mchn: Synced with node %d on block %d - requesting mempool\n",pto->id,mc_gState->m_Permissions->m_Block); + pto->PushMessage("mempool"); + seed_node=(CNode*)(mc_gState->m_pSeedNode); + if(seed_node == pto) + { + if(!MultichainNode_IsLocal(pto)) + { + LogPrintf("mchn: Synced with seed node on block %d\n",mc_gState->m_Permissions->m_Block); + mc_RemoveFile(mc_gState->m_NetworkParams->Name(),"seed",".dat",MC_FOM_RELATIVE_TO_DATADIR); + mc_gState->m_pSeedNode=NULL; + if(vNodes.size() > 1) + { + LogPrintf("mchn: Disconnecting seed node\n"); + pto->fDisconnect=true; + } + } + } + } + } +/* MCHN END */ + + // + // Message: getdata (non-blocks) + // +/* MCHN START */ + if(!ignore_incoming) + { +/* MCHN END */ + while (!pto->fDisconnect && !pto->mapAskFor.empty() && (*pto->mapAskFor.begin()).first <= nNow) + { + const CInv& inv = (*pto->mapAskFor.begin()).second; + if (!AlreadyHave(inv) || (inv.type == MSG_BLOCK)) // MCHN +ignored blocks + { + if (fDebug) + LogPrint("net", "Requesting %s peer=%d\n", inv.ToString(), pto->id); + vGetData.push_back(inv); + if (vGetData.size() >= 1000) + { + pto->PushMessage("getdata", vGetData); + vGetData.clear(); + } + } + pto->mapAskFor.erase(pto->mapAskFor.begin()); + } + if (!vGetData.empty()) + pto->PushMessage("getdata", vGetData); +/* MCHN START */ + } +/* MCHN END */ + } + return true; +} + + +bool CBlockUndo::WriteToDisk(CDiskBlockPos &pos, const uint256 &hashBlock) +{ + // Open history file to append + CAutoFile fileout(OpenUndoFile(pos), SER_DISK, CLIENT_VERSION); + if (fileout.IsNull()) + return error("CBlockUndo::WriteToDisk : OpenUndoFile failed"); + + // Write index header + unsigned int nSize = fileout.GetSerializeSize(*this); + fileout << FLATDATA(Params().MessageStart()) << nSize; + + // Write undo data + long fileOutPos = ftell(fileout.Get()); + if (fileOutPos < 0) + return error("CBlockUndo::WriteToDisk : ftell failed"); + pos.nPos = (unsigned int)fileOutPos; + fileout << *this; + + // calculate & write checksum + CHashWriter hasher(SER_GETHASH, PROTOCOL_VERSION); + hasher << hashBlock; + hasher << *this; + fileout << hasher.GetHash(); + + return true; +} + +bool CBlockUndo::ReadFromDisk(const CDiskBlockPos &pos, const uint256 &hashBlock) +{ + // Open history file to read + CAutoFile filein(OpenUndoFile(pos, true), SER_DISK, CLIENT_VERSION); + if (filein.IsNull()) + return error("CBlockUndo::ReadFromDisk : OpenBlockFile failed"); + + // Read block + uint256 hashChecksum; + try { + filein >> *this; + filein >> hashChecksum; + } + catch (std::exception &e) { + return error("%s : Deserialize or I/O error - %s", __func__, e.what()); + } + + // Verify checksum + CHashWriter hasher(SER_GETHASH, PROTOCOL_VERSION); + hasher << hashBlock; + hasher << *this; + if (hashChecksum != hasher.GetHash()) + return error("CBlockUndo::ReadFromDisk : Checksum mismatch"); + + return true; +} + + std::string CBlockFileInfo::ToString() const { + return strprintf("CBlockFileInfo(blocks=%u, size=%u, heights=%u...%u, time=%s...%s)", nBlocks, nSize, nHeightFirst, nHeightLast, DateTimeStrFormat("%Y-%m-%d", nTimeFirst), DateTimeStrFormat("%Y-%m-%d", nTimeLast)); + } + + + +class CMainCleanup +{ +public: + CMainCleanup() {} + ~CMainCleanup() { + // block headers + BlockMap::iterator it1 = mapBlockIndex.begin(); + for (; it1 != mapBlockIndex.end(); it1++) + delete (*it1).second; + mapBlockIndex.clear(); + + // orphan transactions + mapOrphanTransactions.clear(); + mapOrphanTransactionsByPrev.clear(); + } +} instance_of_cmaincleanup; diff --git a/src/core/main.h b/src/core/main.h new file mode 100644 index 00000000..9c385b0e --- /dev/null +++ b/src/core/main.h @@ -0,0 +1,578 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_MAIN_H +#define BITCOIN_MAIN_H + +#if defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" +#endif + +#include "structs/amount.h" +#include "chain/chain.h" +#include "chainparams/chainparams.h" +#include "storage/coins.h" +#include "primitives/block.h" +#include "primitives/transaction.h" +#include "net/net.h" +#include "chain/pow.h" +#include "script/script.h" +#include "script/sigcache.h" +#include "script/standard.h" +#include "utils/sync.h" +#include "utils/tinyformat.h" +#include "chain/txmempool.h" +#include "structs/uint256.h" +#include "chain/undo.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +class CBlockIndex; +class CBlockTreeDB; +class CBloomFilter; +class CInv; +class CScriptCheck; +class CValidationInterface; +class CValidationState; + +struct CBlockTemplate; +struct CNodeStateStats; + +/** Default for -blockmaxsize and -blockminsize, which control the range of sizes the mining code will create **/ +extern unsigned int DEFAULT_BLOCK_MAX_SIZE; // MCHN global +static const unsigned int DEFAULT_BLOCK_MIN_SIZE = 0; +/** Default for -blockprioritysize, maximum space for zero/low-fee transactions **/ +static const unsigned int DEFAULT_BLOCK_PRIORITY_SIZE = 50000; +/** The maximum size for transactions we're willing to relay/mine */ +extern unsigned int MAX_STANDARD_TX_SIZE; // MCHN global +/** The maximum allowed number of signature check operations in a block (network rule) */ +static const unsigned int MAX_BLOCK_SIGOPS = MAX_BLOCK_SIZE/50; +/** Maximum number of signature check operations in an IsStandard() P2SH script */ +static const unsigned int MAX_P2SH_SIGOPS = 15; +/** The maximum number of sigops we're willing to relay/mine in a single tx */ +static const unsigned int MAX_TX_SIGOPS = MAX_BLOCK_SIGOPS/5; +/** Default for -maxorphantx, maximum number of orphan transactions kept in memory */ +/* MCHN START */ +//static const unsigned int DEFAULT_MAX_ORPHAN_TRANSACTIONS = 100; +/* MCHN START */ +//static const unsigned int DEFAULT_MAX_ORPHAN_TRANSACTIONS = 1000; +static const unsigned int DEFAULT_MAX_ORPHAN_TRANSACTIONS = 50000; +/* MCHN END */ +extern int MAX_OP_RETURN_SHOWN; +/* MCHN END */ +/** The maximum size of a blk?????.dat file (since 0.8) */ +static const unsigned int MAX_BLOCKFILE_SIZE = 0x8000000; // 128 MiB +/** The pre-allocation chunk size for blk?????.dat files (since 0.8) */ +static const unsigned int BLOCKFILE_CHUNK_SIZE = 0x1000000; // 16 MiB +/** The pre-allocation chunk size for rev?????.dat files (since 0.8) */ +static const unsigned int UNDOFILE_CHUNK_SIZE = 0x100000; // 1 MiB +/** Coinbase transaction outputs can only be spent after this number of new blocks (network rule) */ +extern int COINBASE_MATURITY; // MCHN global +/** Threshold for nLockTime: below this value it is interpreted as block number, otherwise as UNIX timestamp. */ +static const unsigned int LOCKTIME_THRESHOLD = 500000000; // Tue Nov 5 00:53:20 1985 UTC +/** Maximum number of script-checking threads allowed */ +static const int MAX_SCRIPTCHECK_THREADS = 16; +/** -par default (number of script-checking threads, 0 = auto) */ +static const int DEFAULT_SCRIPTCHECK_THREADS = 0; +/** Number of blocks that can be requested at any given time from a single peer. */ +static const int MAX_BLOCKS_IN_TRANSIT_PER_PEER = 16; +/** Timeout in seconds during which a peer must stall block download progress before being disconnected. */ +static const unsigned int BLOCK_STALLING_TIMEOUT = 2; +/** Number of headers sent in one getheaders result. We rely on the assumption that if a peer sends + * less than this number, we reached their tip. Changing this value is a protocol upgrade. */ +static const unsigned int MAX_HEADERS_RESULTS = 2000; +/** Size of the "block download window": how far ahead of our current height do we fetch? + * Larger windows tolerate larger download speed differences between peer, but increase the potential + * degree of disordering of blocks on disk (which make reindexing and in the future perhaps pruning + * harder). We'll probably want to make this a per-peer adaptive value at some point. */ +static const unsigned int BLOCK_DOWNLOAD_WINDOW = 1024; +/** Time to wait (in seconds) between writing blockchain state to disk. */ +static const unsigned int DATABASE_WRITE_INTERVAL = 3600; +/** Maximum length of reject messages. */ +static const unsigned int MAX_REJECT_MESSAGE_LENGTH = 111; + +/** "reject" message codes */ +static const unsigned char REJECT_MALFORMED = 0x01; +static const unsigned char REJECT_INVALID = 0x10; +static const unsigned char REJECT_OBSOLETE = 0x11; +static const unsigned char REJECT_DUPLICATE = 0x12; +static const unsigned char REJECT_NONSTANDARD = 0x40; +static const unsigned char REJECT_DUST = 0x41; +static const unsigned char REJECT_INSUFFICIENTFEE = 0x42; +static const unsigned char REJECT_CHECKPOINT = 0x43; + +struct BlockHasher +{ + size_t operator()(const uint256& hash) const { return hash.GetLow64(); } +}; + +extern CScript COINBASE_FLAGS; +extern CCriticalSection cs_main; +extern CTxMemPool mempool; +typedef boost::unordered_map BlockMap; +extern BlockMap mapBlockIndex; +extern uint64_t nLastBlockTx; +extern uint64_t nLastBlockSize; +extern const std::string strMessageMagic; +extern int64_t nTimeBestReceived; +extern CWaitableCriticalSection csBestBlock; +extern CConditionVariable cvBlockChange; +extern bool fImporting; +extern bool fReindex; +extern int nScriptCheckThreads; +extern bool fTxIndex; +extern bool fIsBareMultisigStd; +extern unsigned int nCoinCacheSize; +extern CFeeRate minRelayTxFee; + +/** Best header we've seen so far (used for getheaders queries' starting points). */ +extern CBlockIndex *pindexBestHeader; + +/** Minimum disk space required - used in CheckDiskSpace() */ +static const uint64_t nMinDiskSpace = 52428800; + + +/* MCHN START */ +std::string MultichainServerAddress(); +void ClearMemPools(); +std::string SetLastBlock(uint256 hash); +//void InvalidWTx(const uint256& wtxid, const std::string& reason); +/* MCHN END */ + + +/** Register a wallet to receive updates from core */ +void RegisterValidationInterface(CValidationInterface* pwalletIn); +/** Unregister a wallet from core */ +void UnregisterValidationInterface(CValidationInterface* pwalletIn); +/** Unregister all wallets from core */ +void UnregisterAllValidationInterfaces(); +/** Push an updated transaction to all registered wallets */ +void SyncWithWallets(const CTransaction& tx, const CBlock* pblock = NULL); + +/** Register with a network node to receive its signals */ +void RegisterNodeSignals(CNodeSignals& nodeSignals); +/** Unregister a network node */ +void UnregisterNodeSignals(CNodeSignals& nodeSignals); + +/** + * Process an incoming block. This only returns after the best known valid + * block is made active. Note that it does not, however, guarantee that the + * specific block passed to it has been checked for validity! + * + * @param[out] state This may be set to an Error state if any error occurred processing it, including during validation/connection/etc of otherwise unrelated blocks during reorganisation; or it may be set to an Invalid state if pblock is itself invalid (but this is not guaranteed even when the block is checked). If you want to *possibly* get feedback on whether pblock is valid, you must also install a CValidationInterface - this will have its BlockChecked method called whenever *any* block completes validation. + * @param[in] pfrom The node which we are receiving the block from; it is added to mapBlockSource and may be penalised if the block is invalid. + * @param[in] pblock The block we want to process. + * @param[out] dbp If pblock is stored to disk (or already there), this will be set to its location. + * @return True if state.IsValid() + */ +bool ProcessNewBlock(CValidationState &state, CNode* pfrom, CBlock* pblock, CDiskBlockPos *dbp = NULL); +/** Check whether enough disk space is available for an incoming block */ +bool CheckDiskSpace(uint64_t nAdditionalBytes = 0); +/** Open a block file (blk?????.dat) */ +FILE* OpenBlockFile(const CDiskBlockPos &pos, bool fReadOnly = false); +/** Open an undo file (rev?????.dat) */ +FILE* OpenUndoFile(const CDiskBlockPos &pos, bool fReadOnly = false); +/** Translation to a filesystem path */ +boost::filesystem::path GetBlockPosFilename(const CDiskBlockPos &pos, const char *prefix); +/** Import blocks from an external file */ +bool LoadExternalBlockFile(FILE* fileIn, CDiskBlockPos *dbp = NULL); +/** Initialize a new block tree database + block data on disk */ +bool InitBlockIndex(); +/** Load the block tree and coins database from disk */ +bool LoadBlockIndex(); +/** Unload database information */ +void UnloadBlockIndex(); +/** Process protocol messages received from a given node */ +bool ProcessMessages(CNode* pfrom); +/** Send queued protocol messages to be sent to a give node */ +bool SendMessages(CNode* pto, bool fSendTrickle); +/** Run an instance of the script checking thread */ +void ThreadScriptCheck(); +/** Check whether we are doing an initial block download (synchronizing from disk or network) */ +bool IsInitialBlockDownload(); +/** Format a string that describes several potential problems detected by the core */ +std::string GetWarnings(std::string strFor); +/** Retrieve a transaction (from memory pool, or from disk, if possible) */ +bool GetTransaction(const uint256 &hash, CTransaction &tx, uint256 &hashBlock, bool fAllowSlow = false); +/** Find the best known block, and make it the tip of the block chain */ +bool ActivateBestChain(CValidationState &state, CBlock *pblock = NULL); +CAmount GetBlockValue(int nHeight, const CAmount& nFees); + +/** Create a new block index entry for a given block hash */ +CBlockIndex * InsertBlockIndex(uint256 hash); +/** Abort with a message */ +bool AbortNode(const std::string &msg, const std::string &userMessage=""); +/** Get statistics from node state */ +bool GetNodeStateStats(NodeId nodeid, CNodeStateStats &stats); +/** Increase a node's misbehavior score. */ +void Misbehaving(NodeId nodeid, int howmuch); +/** Flush all state, indexes and buffers to disk. */ +void FlushStateToDisk(); + + +/** (try to) add transaction to memory pool **/ +bool AcceptToMemoryPool(CTxMemPool& pool, CValidationState &state, const CTransaction &tx, bool fLimitFree, + bool* pfMissingInputs, bool fRejectInsaneFee=false, bool fAddToWallet=true); + + +struct CNodeStateStats { + int nMisbehavior; + int nSyncHeight; + int nCommonHeight; + std::vector vHeightInFlight; +}; + +struct CDiskTxPos : public CDiskBlockPos +{ + unsigned int nTxOffset; // after header + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(*(CDiskBlockPos*)this); + READWRITE(VARINT(nTxOffset)); + } + + CDiskTxPos(const CDiskBlockPos &blockIn, unsigned int nTxOffsetIn) : CDiskBlockPos(blockIn.nFile, blockIn.nPos), nTxOffset(nTxOffsetIn) { + } + + CDiskTxPos() { + SetNull(); + } + + void SetNull() { + CDiskBlockPos::SetNull(); + nTxOffset = 0; + } +}; + + +CAmount GetMinRelayFee(const CTransaction& tx, unsigned int nBytes, bool fAllowFree); + +/** + * Check transaction inputs, and make sure any + * pay-to-script-hash transactions are evaluating IsStandard scripts + * + * Why bother? To avoid denial-of-service attacks; an attacker + * can submit a standard HASH... OP_EQUAL transaction, + * which will get accepted into blocks. The redemption + * script can be anything; an attacker could use a very + * expensive-to-check-upon-redemption script like: + * DUP CHECKSIG DROP ... repeated 100 times... OP_1 + */ + +/** + * Check for standard transaction types + * @param[in] mapInputs Map of previous transactions that have outputs we're spending + * @return True if all inputs (scriptSigs) use only standard transaction forms + */ +bool AreInputsStandard(const CTransaction& tx, const CCoinsViewCache& mapInputs); + +/** + * Count ECDSA signature operations the old-fashioned (pre-0.6) way + * @return number of sigops this transaction's outputs will produce when spent + * @see CTransaction::FetchInputs + */ +unsigned int GetLegacySigOpCount(const CTransaction& tx); + +/** + * Count ECDSA signature operations in pay-to-script-hash inputs. + * + * @param[in] mapInputs Map of previous transactions that have outputs we're spending + * @return maximum number of sigops required to validate this transaction's inputs + * @see CTransaction::FetchInputs + */ +unsigned int GetP2SHSigOpCount(const CTransaction& tx, const CCoinsViewCache& mapInputs); + + +/** + * Check whether all inputs of this transaction are valid (no double spends, scripts & sigs, amounts) + * This does not modify the UTXO set. If pvChecks is not NULL, script checks are pushed onto it + * instead of being performed inline. + */ +bool CheckInputs(const CTransaction& tx, CValidationState &state, const CCoinsViewCache &view, bool fScriptChecks, + unsigned int flags, bool cacheStore, std::vector *pvChecks = NULL); + +/** Apply the effects of this transaction on the UTXO set represented by view */ +void UpdateCoins(const CTransaction& tx, CValidationState &state, CCoinsViewCache &inputs, CTxUndo &txundo, int nHeight); + +/** Context-independent validity checks */ +bool CheckTransaction(const CTransaction& tx, CValidationState& state); + +/** Check for standard transaction types + * @return True if all outputs (scriptPubKeys) use only standard transaction forms + */ +bool IsStandardTx(const CTransaction& tx, std::string& reason); + +bool IsFinalTx(const CTransaction &tx, int nBlockHeight = 0, int64_t nBlockTime = 0); + +/** Undo information for a CBlock */ +class CBlockUndo +{ +public: + std::vector vtxundo; // for all but the coinbase + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(vtxundo); + } + + bool WriteToDisk(CDiskBlockPos &pos, const uint256 &hashBlock); + bool ReadFromDisk(const CDiskBlockPos &pos, const uint256 &hashBlock); +}; + + +/** + * Closure representing one script verification + * Note that this stores references to the spending transaction + */ +class CScriptCheck +{ +private: + CScript scriptPubKey; + const CTransaction *ptxTo; + unsigned int nIn; + unsigned int nFlags; + bool cacheStore; + ScriptError error; + +public: + CScriptCheck(): ptxTo(0), nIn(0), nFlags(0), cacheStore(false), error(SCRIPT_ERR_UNKNOWN_ERROR) {} + CScriptCheck(const CCoins& txFromIn, const CTransaction& txToIn, unsigned int nInIn, unsigned int nFlagsIn, bool cacheIn) : + scriptPubKey(txFromIn.vout[txToIn.vin[nInIn].prevout.n].scriptPubKey), + ptxTo(&txToIn), nIn(nInIn), nFlags(nFlagsIn), cacheStore(cacheIn), error(SCRIPT_ERR_UNKNOWN_ERROR) { } + + bool operator()(); + + void swap(CScriptCheck &check) { + scriptPubKey.swap(check.scriptPubKey); + std::swap(ptxTo, check.ptxTo); + std::swap(nIn, check.nIn); + std::swap(nFlags, check.nFlags); + std::swap(cacheStore, check.cacheStore); + std::swap(error, check.error); + } + + ScriptError GetScriptError() const { return error; } +}; + + +/** Functions for disk access for blocks */ +bool WriteBlockToDisk(CBlock& block, CDiskBlockPos& pos); +bool ReadBlockFromDisk(CBlock& block, const CDiskBlockPos& pos); +bool ReadBlockFromDisk(CBlock& block, const CBlockIndex* pindex); + + +/** Functions for validating blocks and updating the block tree */ + +/** Undo the effects of this block (with given index) on the UTXO set represented by coins. + * In case pfClean is provided, operation will try to be tolerant about errors, and *pfClean + * will be true if no problems were found. Otherwise, the return value will be false in case + * of problems. Note that in any case, coins may be modified. */ +bool DisconnectBlock(CBlock& block, CValidationState& state, CBlockIndex* pindex, CCoinsViewCache& coins, bool* pfClean = NULL); + +/** Apply the effects of this block (with given index) on the UTXO set represented by coins */ +bool ConnectBlock(const CBlock& block, CValidationState& state, CBlockIndex* pindex, CCoinsViewCache& coins, bool fJustCheck = false); + +/** Context-independent validity checks */ +bool CheckBlockHeader(const CBlockHeader& block, CValidationState& state, bool fCheckPOW = true); +bool CheckBlock(const CBlock& block, CValidationState& state, bool fCheckPOW = true, bool fCheckMerkleRoot = true); + +/** Context-dependent validity checks */ +bool ContextualCheckBlockHeader(const CBlockHeader& block, CValidationState& state, CBlockIndex *pindexPrev); +bool ContextualCheckBlock(const CBlock& block, CValidationState& state, CBlockIndex *pindexPrev); + +/** Check a block is completely valid from start to finish (only works on top of our current best block, with cs_main held) */ +bool TestBlockValidity(CValidationState &state, const CBlock& block, CBlockIndex *pindexPrev, bool fCheckPOW = true, bool fCheckMerkleRoot = true); + +/** Store block on disk. If dbp is provided, the file is known to already reside on disk */ +bool AcceptBlock(CBlock& block, CValidationState& state, CBlockIndex **pindex, CDiskBlockPos* dbp = NULL); +bool AcceptBlockHeader(const CBlockHeader& block, CValidationState& state, CBlockIndex **ppindex= NULL); + + + +class CBlockFileInfo +{ +public: + unsigned int nBlocks; //! number of blocks stored in file + unsigned int nSize; //! number of used bytes of block file + unsigned int nUndoSize; //! number of used bytes in the undo file + unsigned int nHeightFirst; //! lowest height of block in file + unsigned int nHeightLast; //! highest height of block in file + uint64_t nTimeFirst; //! earliest time of block in file + uint64_t nTimeLast; //! latest time of block in file + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(VARINT(nBlocks)); + READWRITE(VARINT(nSize)); + READWRITE(VARINT(nUndoSize)); + READWRITE(VARINT(nHeightFirst)); + READWRITE(VARINT(nHeightLast)); + READWRITE(VARINT(nTimeFirst)); + READWRITE(VARINT(nTimeLast)); + } + + void SetNull() { + nBlocks = 0; + nSize = 0; + nUndoSize = 0; + nHeightFirst = 0; + nHeightLast = 0; + nTimeFirst = 0; + nTimeLast = 0; + } + + CBlockFileInfo() { + SetNull(); + } + + std::string ToString() const; + + /** update statistics (does not update nSize) */ + void AddBlock(unsigned int nHeightIn, uint64_t nTimeIn) { + if (nBlocks==0 || nHeightFirst > nHeightIn) + nHeightFirst = nHeightIn; + if (nBlocks==0 || nTimeFirst > nTimeIn) + nTimeFirst = nTimeIn; + nBlocks++; + if (nHeightIn > nHeightLast) + nHeightLast = nHeightIn; + if (nTimeIn > nTimeLast) + nTimeLast = nTimeIn; + } +}; + +/** Capture information about block/transaction validation */ +class CValidationState { +private: + enum mode_state { + MODE_VALID, //! everything ok + MODE_INVALID, //! network rule violation (DoS value may be set) + MODE_ERROR, //! run-time error + } mode; + int nDoS; + std::string strRejectReason; + unsigned char chRejectCode; + bool corruptionPossible; +public: + CValidationState() : mode(MODE_VALID), nDoS(0), chRejectCode(0), corruptionPossible(false) {} + bool DoS(int level, bool ret = false, + unsigned char chRejectCodeIn=0, std::string strRejectReasonIn="", + bool corruptionIn=false) { + chRejectCode = chRejectCodeIn; + strRejectReason = strRejectReasonIn; + corruptionPossible = corruptionIn; + if (mode == MODE_ERROR) + return ret; + nDoS += level; + mode = MODE_INVALID; + return ret; + } + bool Invalid(bool ret = false, + unsigned char _chRejectCode=0, std::string _strRejectReason="") { + return DoS(0, ret, _chRejectCode, _strRejectReason); + } + bool Error(std::string strRejectReasonIn="") { + if (mode == MODE_VALID) + strRejectReason = strRejectReasonIn; + mode = MODE_ERROR; + return false; + } + bool Abort(const std::string &msg) { + AbortNode(msg); + return Error(msg); + } + bool IsValid() const { + return mode == MODE_VALID; + } + bool IsInvalid() const { + return mode == MODE_INVALID; + } + bool IsError() const { + return mode == MODE_ERROR; + } + bool IsInvalid(int &nDoSOut) const { + if (IsInvalid()) { + nDoSOut = nDoS; + return true; + } + return false; + } + bool CorruptionPossible() const { + return corruptionPossible; + } + unsigned char GetRejectCode() const { return chRejectCode; } + std::string GetRejectReason() const { return strRejectReason; } +}; + +/** RAII wrapper for VerifyDB: Verify consistency of the block and coin databases */ +class CVerifyDB { +public: + CVerifyDB(); + ~CVerifyDB(); + bool VerifyDB(CCoinsView *coinsview, int nCheckLevel, int nCheckDepth); +}; + +/** Find the last common block between the parameter chain and a locator. */ +CBlockIndex* FindForkInGlobalIndex(const CChain& chain, const CBlockLocator& locator); + +/** Mark a block as invalid. */ +bool InvalidateBlock(CValidationState& state, CBlockIndex *pindex); + +/** Remove invalidity status from a block and its descendants. */ +bool ReconsiderBlock(CValidationState& state, CBlockIndex *pindex); + +/** The currently-connected chain of blocks. */ +extern CChain chainActive; + +/** Global variable that points to the active CCoinsView (protected by cs_main) */ +extern CCoinsViewCache *pcoinsTip; + +/** Global variable that points to the active block tree (protected by cs_main) */ +extern CBlockTreeDB *pblocktree; + +struct CBlockTemplate +{ + CBlock block; + std::vector vTxFees; + std::vector vTxSigOps; +}; + + + + + + +class CValidationInterface { +protected: + virtual void SyncTransaction(const CTransaction &tx, const CBlock *pblock) {}; + virtual void EraseFromWallet(const uint256 &hash) {}; + virtual void SetBestChain(const CBlockLocator &locator) {}; + virtual void UpdatedTransaction(const uint256 &hash) {}; + virtual void Inventory(const uint256 &hash) {}; + virtual void ResendWalletTransactions() {}; + virtual void BlockChecked(const CBlock&, const CValidationState&) {}; + friend void ::RegisterValidationInterface(CValidationInterface*); + friend void ::UnregisterValidationInterface(CValidationInterface*); + friend void ::UnregisterAllValidationInterfaces(); +}; + +#endif // BITCOIN_MAIN_H diff --git a/src/crypto/common.h b/src/crypto/common.h new file mode 100644 index 00000000..67c30023 --- /dev/null +++ b/src/crypto/common.h @@ -0,0 +1,120 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#ifndef BITCOIN_CRYPTO_COMMON_H +#define BITCOIN_CRYPTO_COMMON_H + +#if defined(HAVE_CONFIG_H) +#include "bitcoin-config.h" +#endif + +#include + +#if defined(HAVE_ENDIAN_H) +#include +#endif + +uint32_t static inline ReadLE32(const unsigned char* ptr) +{ +#if HAVE_DECL_LE32TOH == 1 + return le32toh(*((uint32_t*)ptr)); +#elif !defined(WORDS_BIGENDIAN) + return *((uint32_t*)ptr); +#else + return ((uint32_t)ptr[3] << 24 | (uint32_t)ptr[2] << 16 | (uint32_t)ptr[1] << 8 | (uint32_t)ptr[0]); +#endif +} + +uint64_t static inline ReadLE64(const unsigned char* ptr) +{ +#if HAVE_DECL_LE64TOH == 1 + return le64toh(*((uint64_t*)ptr)); +#elif !defined(WORDS_BIGENDIAN) + return *((uint64_t*)ptr); +#else + return ((uint64_t)ptr[7] << 56 | (uint64_t)ptr[6] << 48 | (uint64_t)ptr[5] << 40 | (uint64_t)ptr[4] << 32 | + (uint64_t)ptr[3] << 24 | (uint64_t)ptr[2] << 16 | (uint64_t)ptr[1] << 8 | (uint64_t)ptr[0]); +#endif +} + +void static inline WriteLE32(unsigned char* ptr, uint32_t x) +{ +#if HAVE_DECL_HTOLE32 == 1 + *((uint32_t*)ptr) = htole32(x); +#elif !defined(WORDS_BIGENDIAN) + *((uint32_t*)ptr) = x; +#else + ptr[3] = x >> 24; + ptr[2] = x >> 16; + ptr[1] = x >> 8; + ptr[0] = x; +#endif +} + +void static inline WriteLE64(unsigned char* ptr, uint64_t x) +{ +#if HAVE_DECL_HTOLE64 == 1 + *((uint64_t*)ptr) = htole64(x); +#elif !defined(WORDS_BIGENDIAN) + *((uint64_t*)ptr) = x; +#else + ptr[7] = x >> 56; + ptr[6] = x >> 48; + ptr[5] = x >> 40; + ptr[4] = x >> 32; + ptr[3] = x >> 24; + ptr[2] = x >> 16; + ptr[1] = x >> 8; + ptr[0] = x; +#endif +} + +uint32_t static inline ReadBE32(const unsigned char* ptr) +{ +#if HAVE_DECL_BE32TOH == 1 + return be32toh(*((uint32_t*)ptr)); +#else + return ((uint32_t)ptr[0] << 24 | (uint32_t)ptr[1] << 16 | (uint32_t)ptr[2] << 8 | (uint32_t)ptr[3]); +#endif +} + +uint64_t static inline ReadBE64(const unsigned char* ptr) +{ +#if HAVE_DECL_BE64TOH == 1 + return be64toh(*((uint64_t*)ptr)); +#else + return ((uint64_t)ptr[0] << 56 | (uint64_t)ptr[1] << 48 | (uint64_t)ptr[2] << 40 | (uint64_t)ptr[3] << 32 | + (uint64_t)ptr[4] << 24 | (uint64_t)ptr[5] << 16 | (uint64_t)ptr[6] << 8 | (uint64_t)ptr[7]); +#endif +} + +void static inline WriteBE32(unsigned char* ptr, uint32_t x) +{ +#if HAVE_DECL_HTOBE32 == 1 + *((uint32_t*)ptr) = htobe32(x); +#else + ptr[0] = x >> 24; + ptr[1] = x >> 16; + ptr[2] = x >> 8; + ptr[3] = x; +#endif +} + +void static inline WriteBE64(unsigned char* ptr, uint64_t x) +{ +#if HAVE_DECL_HTOBE64 == 1 + *((uint64_t*)ptr) = htobe64(x); +#else + ptr[0] = x >> 56; + ptr[1] = x >> 48; + ptr[2] = x >> 40; + ptr[3] = x >> 32; + ptr[4] = x >> 24; + ptr[5] = x >> 16; + ptr[6] = x >> 8; + ptr[7] = x; +#endif +} + +#endif // BITCOIN_CRYPTO_COMMON_H diff --git a/src/crypto/hmac_sha256.cpp b/src/crypto/hmac_sha256.cpp new file mode 100644 index 00000000..43589653 --- /dev/null +++ b/src/crypto/hmac_sha256.cpp @@ -0,0 +1,34 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include "crypto/hmac_sha256.h" + +#include + +CHMAC_SHA256::CHMAC_SHA256(const unsigned char* key, size_t keylen) +{ + unsigned char rkey[64]; + if (keylen <= 64) { + memcpy(rkey, key, keylen); + memset(rkey + keylen, 0, 64 - keylen); + } else { + CSHA256().Write(key, keylen).Finalize(rkey); + memset(rkey + 32, 0, 32); + } + + for (int n = 0; n < 64; n++) + rkey[n] ^= 0x5c; + outer.Write(rkey, 64); + + for (int n = 0; n < 64; n++) + rkey[n] ^= 0x5c ^ 0x36; + inner.Write(rkey, 64); +} + +void CHMAC_SHA256::Finalize(unsigned char hash[OUTPUT_SIZE]) +{ + unsigned char temp[32]; + inner.Finalize(temp); + outer.Write(temp, 32).Finalize(hash); +} diff --git a/src/crypto/hmac_sha256.h b/src/crypto/hmac_sha256.h new file mode 100644 index 00000000..1fdee5a7 --- /dev/null +++ b/src/crypto/hmac_sha256.h @@ -0,0 +1,32 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#ifndef BITCOIN_CRYPTO_HMAC_SHA256_H +#define BITCOIN_CRYPTO_HMAC_SHA256_H + +#include "crypto/sha256.h" + +#include +#include + +/** A hasher class for HMAC-SHA-512. */ +class CHMAC_SHA256 +{ +private: + CSHA256 outer; + CSHA256 inner; + +public: + static const size_t OUTPUT_SIZE = 32; + + CHMAC_SHA256(const unsigned char* key, size_t keylen); + CHMAC_SHA256& Write(const unsigned char* data, size_t len) + { + inner.Write(data, len); + return *this; + } + void Finalize(unsigned char hash[OUTPUT_SIZE]); +}; + +#endif // BITCOIN_CRYPTO_HMAC_SHA256_H diff --git a/src/crypto/hmac_sha512.cpp b/src/crypto/hmac_sha512.cpp new file mode 100644 index 00000000..940a9327 --- /dev/null +++ b/src/crypto/hmac_sha512.cpp @@ -0,0 +1,34 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include "crypto/hmac_sha512.h" + +#include + +CHMAC_SHA512::CHMAC_SHA512(const unsigned char* key, size_t keylen) +{ + unsigned char rkey[128]; + if (keylen <= 128) { + memcpy(rkey, key, keylen); + memset(rkey + keylen, 0, 128 - keylen); + } else { + CSHA512().Write(key, keylen).Finalize(rkey); + memset(rkey + 64, 0, 64); + } + + for (int n = 0; n < 128; n++) + rkey[n] ^= 0x5c; + outer.Write(rkey, 128); + + for (int n = 0; n < 128; n++) + rkey[n] ^= 0x5c ^ 0x36; + inner.Write(rkey, 128); +} + +void CHMAC_SHA512::Finalize(unsigned char hash[OUTPUT_SIZE]) +{ + unsigned char temp[64]; + inner.Finalize(temp); + outer.Write(temp, 64).Finalize(hash); +} diff --git a/src/crypto/hmac_sha512.h b/src/crypto/hmac_sha512.h new file mode 100644 index 00000000..17d75021 --- /dev/null +++ b/src/crypto/hmac_sha512.h @@ -0,0 +1,32 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#ifndef BITCOIN_CRYPTO_HMAC_SHA512_H +#define BITCOIN_CRYPTO_HMAC_SHA512_H + +#include "crypto/sha512.h" + +#include +#include + +/** A hasher class for HMAC-SHA-512. */ +class CHMAC_SHA512 +{ +private: + CSHA512 outer; + CSHA512 inner; + +public: + static const size_t OUTPUT_SIZE = 64; + + CHMAC_SHA512(const unsigned char* key, size_t keylen); + CHMAC_SHA512& Write(const unsigned char* data, size_t len) + { + inner.Write(data, len); + return *this; + } + void Finalize(unsigned char hash[OUTPUT_SIZE]); +}; + +#endif // BITCOIN_CRYPTO_HMAC_SHA512_H diff --git a/src/crypto/rfc6979_hmac_sha256.cpp b/src/crypto/rfc6979_hmac_sha256.cpp new file mode 100644 index 00000000..3f935abf --- /dev/null +++ b/src/crypto/rfc6979_hmac_sha256.cpp @@ -0,0 +1,47 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include "crypto/rfc6979_hmac_sha256.h" + +#include + +#include + +static const unsigned char zero[1] = {0x00}; +static const unsigned char one[1] = {0x01}; + +RFC6979_HMAC_SHA256::RFC6979_HMAC_SHA256(const unsigned char* key, size_t keylen, const unsigned char* msg, size_t msglen) : retry(false) +{ + memset(V, 0x01, sizeof(V)); + memset(K, 0x00, sizeof(K)); + + CHMAC_SHA256(K, sizeof(K)).Write(V, sizeof(V)).Write(zero, sizeof(zero)).Write(key, keylen).Write(msg, msglen).Finalize(K); + CHMAC_SHA256(K, sizeof(K)).Write(V, sizeof(V)).Finalize(V); + CHMAC_SHA256(K, sizeof(K)).Write(V, sizeof(V)).Write(one, sizeof(one)).Write(key, keylen).Write(msg, msglen).Finalize(K); + CHMAC_SHA256(K, sizeof(K)).Write(V, sizeof(V)).Finalize(V); +} + +RFC6979_HMAC_SHA256::~RFC6979_HMAC_SHA256() +{ + memset(V, 0x01, sizeof(V)); + memset(K, 0x00, sizeof(K)); +} + +void RFC6979_HMAC_SHA256::Generate(unsigned char* output, size_t outputlen) +{ + if (retry) { + CHMAC_SHA256(K, sizeof(K)).Write(V, sizeof(V)).Write(zero, sizeof(zero)).Finalize(K); + CHMAC_SHA256(K, sizeof(K)).Write(V, sizeof(V)).Finalize(V); + } + + while (outputlen > 0) { + CHMAC_SHA256(K, sizeof(K)).Write(V, sizeof(V)).Finalize(V); + size_t len = std::min(outputlen, sizeof(V)); + memcpy(output, V, len); + output += len; + outputlen -= len; + } + + retry = true; +} diff --git a/src/crypto/rfc6979_hmac_sha256.h b/src/crypto/rfc6979_hmac_sha256.h new file mode 100644 index 00000000..e67ddcf8 --- /dev/null +++ b/src/crypto/rfc6979_hmac_sha256.h @@ -0,0 +1,36 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#ifndef BITCOIN_RFC6979_HMAC_SHA256_H +#define BITCOIN_RFC6979_HMAC_SHA256_H + +#include "crypto/hmac_sha256.h" + +#include +#include + +/** The RFC 6979 PRNG using HMAC-SHA256. */ +class RFC6979_HMAC_SHA256 +{ +private: + unsigned char V[CHMAC_SHA256::OUTPUT_SIZE]; + unsigned char K[CHMAC_SHA256::OUTPUT_SIZE]; + bool retry; + +public: + /** + * Construct a new RFC6979 PRNG, using the given key and message. + * The message is assumed to be already hashed. + */ + RFC6979_HMAC_SHA256(const unsigned char* key, size_t keylen, const unsigned char* msg, size_t msglen); + + /** + * Generate a byte array. + */ + void Generate(unsigned char* output, size_t outputlen); + + ~RFC6979_HMAC_SHA256(); +}; + +#endif // BITCOIN_RFC6979_HMAC_SHA256_H diff --git a/src/crypto/ripemd160.cpp b/src/crypto/ripemd160.cpp new file mode 100644 index 00000000..cb4a94a4 --- /dev/null +++ b/src/crypto/ripemd160.cpp @@ -0,0 +1,292 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include "crypto/ripemd160.h" + +#include "crypto/common.h" + +#include + +// Internal implementation code. +namespace +{ +/// Internal RIPEMD-160 implementation. +namespace ripemd160 +{ +uint32_t inline f1(uint32_t x, uint32_t y, uint32_t z) { return x ^ y ^ z; } +uint32_t inline f2(uint32_t x, uint32_t y, uint32_t z) { return (x & y) | (~x & z); } +uint32_t inline f3(uint32_t x, uint32_t y, uint32_t z) { return (x | ~y) ^ z; } +uint32_t inline f4(uint32_t x, uint32_t y, uint32_t z) { return (x & z) | (y & ~z); } +uint32_t inline f5(uint32_t x, uint32_t y, uint32_t z) { return x ^ (y | ~z); } + +/** Initialize RIPEMD-160 state. */ +void inline Initialize(uint32_t* s) +{ + s[0] = 0x67452301ul; + s[1] = 0xEFCDAB89ul; + s[2] = 0x98BADCFEul; + s[3] = 0x10325476ul; + s[4] = 0xC3D2E1F0ul; +} + +uint32_t inline rol(uint32_t x, int i) { return (x << i) | (x >> (32 - i)); } + +void inline Round(uint32_t& a, uint32_t b, uint32_t& c, uint32_t d, uint32_t e, uint32_t f, uint32_t x, uint32_t k, int r) +{ + a = rol(a + f + x + k, r) + e; + c = rol(c, 10); +} + +void inline R11(uint32_t& a, uint32_t b, uint32_t& c, uint32_t d, uint32_t e, uint32_t x, int r) { Round(a, b, c, d, e, f1(b, c, d), x, 0, r); } +void inline R21(uint32_t& a, uint32_t b, uint32_t& c, uint32_t d, uint32_t e, uint32_t x, int r) { Round(a, b, c, d, e, f2(b, c, d), x, 0x5A827999ul, r); } +void inline R31(uint32_t& a, uint32_t b, uint32_t& c, uint32_t d, uint32_t e, uint32_t x, int r) { Round(a, b, c, d, e, f3(b, c, d), x, 0x6ED9EBA1ul, r); } +void inline R41(uint32_t& a, uint32_t b, uint32_t& c, uint32_t d, uint32_t e, uint32_t x, int r) { Round(a, b, c, d, e, f4(b, c, d), x, 0x8F1BBCDCul, r); } +void inline R51(uint32_t& a, uint32_t b, uint32_t& c, uint32_t d, uint32_t e, uint32_t x, int r) { Round(a, b, c, d, e, f5(b, c, d), x, 0xA953FD4Eul, r); } + +void inline R12(uint32_t& a, uint32_t b, uint32_t& c, uint32_t d, uint32_t e, uint32_t x, int r) { Round(a, b, c, d, e, f5(b, c, d), x, 0x50A28BE6ul, r); } +void inline R22(uint32_t& a, uint32_t b, uint32_t& c, uint32_t d, uint32_t e, uint32_t x, int r) { Round(a, b, c, d, e, f4(b, c, d), x, 0x5C4DD124ul, r); } +void inline R32(uint32_t& a, uint32_t b, uint32_t& c, uint32_t d, uint32_t e, uint32_t x, int r) { Round(a, b, c, d, e, f3(b, c, d), x, 0x6D703EF3ul, r); } +void inline R42(uint32_t& a, uint32_t b, uint32_t& c, uint32_t d, uint32_t e, uint32_t x, int r) { Round(a, b, c, d, e, f2(b, c, d), x, 0x7A6D76E9ul, r); } +void inline R52(uint32_t& a, uint32_t b, uint32_t& c, uint32_t d, uint32_t e, uint32_t x, int r) { Round(a, b, c, d, e, f1(b, c, d), x, 0, r); } + +/** Perform a RIPEMD-160 transformation, processing a 64-byte chunk. */ +void Transform(uint32_t* s, const unsigned char* chunk) +{ + uint32_t a1 = s[0], b1 = s[1], c1 = s[2], d1 = s[3], e1 = s[4]; + uint32_t a2 = a1, b2 = b1, c2 = c1, d2 = d1, e2 = e1; + uint32_t w0 = ReadLE32(chunk + 0), w1 = ReadLE32(chunk + 4), w2 = ReadLE32(chunk + 8), w3 = ReadLE32(chunk + 12); + uint32_t w4 = ReadLE32(chunk + 16), w5 = ReadLE32(chunk + 20), w6 = ReadLE32(chunk + 24), w7 = ReadLE32(chunk + 28); + uint32_t w8 = ReadLE32(chunk + 32), w9 = ReadLE32(chunk + 36), w10 = ReadLE32(chunk + 40), w11 = ReadLE32(chunk + 44); + uint32_t w12 = ReadLE32(chunk + 48), w13 = ReadLE32(chunk + 52), w14 = ReadLE32(chunk + 56), w15 = ReadLE32(chunk + 60); + + R11(a1, b1, c1, d1, e1, w0, 11); + R12(a2, b2, c2, d2, e2, w5, 8); + R11(e1, a1, b1, c1, d1, w1, 14); + R12(e2, a2, b2, c2, d2, w14, 9); + R11(d1, e1, a1, b1, c1, w2, 15); + R12(d2, e2, a2, b2, c2, w7, 9); + R11(c1, d1, e1, a1, b1, w3, 12); + R12(c2, d2, e2, a2, b2, w0, 11); + R11(b1, c1, d1, e1, a1, w4, 5); + R12(b2, c2, d2, e2, a2, w9, 13); + R11(a1, b1, c1, d1, e1, w5, 8); + R12(a2, b2, c2, d2, e2, w2, 15); + R11(e1, a1, b1, c1, d1, w6, 7); + R12(e2, a2, b2, c2, d2, w11, 15); + R11(d1, e1, a1, b1, c1, w7, 9); + R12(d2, e2, a2, b2, c2, w4, 5); + R11(c1, d1, e1, a1, b1, w8, 11); + R12(c2, d2, e2, a2, b2, w13, 7); + R11(b1, c1, d1, e1, a1, w9, 13); + R12(b2, c2, d2, e2, a2, w6, 7); + R11(a1, b1, c1, d1, e1, w10, 14); + R12(a2, b2, c2, d2, e2, w15, 8); + R11(e1, a1, b1, c1, d1, w11, 15); + R12(e2, a2, b2, c2, d2, w8, 11); + R11(d1, e1, a1, b1, c1, w12, 6); + R12(d2, e2, a2, b2, c2, w1, 14); + R11(c1, d1, e1, a1, b1, w13, 7); + R12(c2, d2, e2, a2, b2, w10, 14); + R11(b1, c1, d1, e1, a1, w14, 9); + R12(b2, c2, d2, e2, a2, w3, 12); + R11(a1, b1, c1, d1, e1, w15, 8); + R12(a2, b2, c2, d2, e2, w12, 6); + + R21(e1, a1, b1, c1, d1, w7, 7); + R22(e2, a2, b2, c2, d2, w6, 9); + R21(d1, e1, a1, b1, c1, w4, 6); + R22(d2, e2, a2, b2, c2, w11, 13); + R21(c1, d1, e1, a1, b1, w13, 8); + R22(c2, d2, e2, a2, b2, w3, 15); + R21(b1, c1, d1, e1, a1, w1, 13); + R22(b2, c2, d2, e2, a2, w7, 7); + R21(a1, b1, c1, d1, e1, w10, 11); + R22(a2, b2, c2, d2, e2, w0, 12); + R21(e1, a1, b1, c1, d1, w6, 9); + R22(e2, a2, b2, c2, d2, w13, 8); + R21(d1, e1, a1, b1, c1, w15, 7); + R22(d2, e2, a2, b2, c2, w5, 9); + R21(c1, d1, e1, a1, b1, w3, 15); + R22(c2, d2, e2, a2, b2, w10, 11); + R21(b1, c1, d1, e1, a1, w12, 7); + R22(b2, c2, d2, e2, a2, w14, 7); + R21(a1, b1, c1, d1, e1, w0, 12); + R22(a2, b2, c2, d2, e2, w15, 7); + R21(e1, a1, b1, c1, d1, w9, 15); + R22(e2, a2, b2, c2, d2, w8, 12); + R21(d1, e1, a1, b1, c1, w5, 9); + R22(d2, e2, a2, b2, c2, w12, 7); + R21(c1, d1, e1, a1, b1, w2, 11); + R22(c2, d2, e2, a2, b2, w4, 6); + R21(b1, c1, d1, e1, a1, w14, 7); + R22(b2, c2, d2, e2, a2, w9, 15); + R21(a1, b1, c1, d1, e1, w11, 13); + R22(a2, b2, c2, d2, e2, w1, 13); + R21(e1, a1, b1, c1, d1, w8, 12); + R22(e2, a2, b2, c2, d2, w2, 11); + + R31(d1, e1, a1, b1, c1, w3, 11); + R32(d2, e2, a2, b2, c2, w15, 9); + R31(c1, d1, e1, a1, b1, w10, 13); + R32(c2, d2, e2, a2, b2, w5, 7); + R31(b1, c1, d1, e1, a1, w14, 6); + R32(b2, c2, d2, e2, a2, w1, 15); + R31(a1, b1, c1, d1, e1, w4, 7); + R32(a2, b2, c2, d2, e2, w3, 11); + R31(e1, a1, b1, c1, d1, w9, 14); + R32(e2, a2, b2, c2, d2, w7, 8); + R31(d1, e1, a1, b1, c1, w15, 9); + R32(d2, e2, a2, b2, c2, w14, 6); + R31(c1, d1, e1, a1, b1, w8, 13); + R32(c2, d2, e2, a2, b2, w6, 6); + R31(b1, c1, d1, e1, a1, w1, 15); + R32(b2, c2, d2, e2, a2, w9, 14); + R31(a1, b1, c1, d1, e1, w2, 14); + R32(a2, b2, c2, d2, e2, w11, 12); + R31(e1, a1, b1, c1, d1, w7, 8); + R32(e2, a2, b2, c2, d2, w8, 13); + R31(d1, e1, a1, b1, c1, w0, 13); + R32(d2, e2, a2, b2, c2, w12, 5); + R31(c1, d1, e1, a1, b1, w6, 6); + R32(c2, d2, e2, a2, b2, w2, 14); + R31(b1, c1, d1, e1, a1, w13, 5); + R32(b2, c2, d2, e2, a2, w10, 13); + R31(a1, b1, c1, d1, e1, w11, 12); + R32(a2, b2, c2, d2, e2, w0, 13); + R31(e1, a1, b1, c1, d1, w5, 7); + R32(e2, a2, b2, c2, d2, w4, 7); + R31(d1, e1, a1, b1, c1, w12, 5); + R32(d2, e2, a2, b2, c2, w13, 5); + + R41(c1, d1, e1, a1, b1, w1, 11); + R42(c2, d2, e2, a2, b2, w8, 15); + R41(b1, c1, d1, e1, a1, w9, 12); + R42(b2, c2, d2, e2, a2, w6, 5); + R41(a1, b1, c1, d1, e1, w11, 14); + R42(a2, b2, c2, d2, e2, w4, 8); + R41(e1, a1, b1, c1, d1, w10, 15); + R42(e2, a2, b2, c2, d2, w1, 11); + R41(d1, e1, a1, b1, c1, w0, 14); + R42(d2, e2, a2, b2, c2, w3, 14); + R41(c1, d1, e1, a1, b1, w8, 15); + R42(c2, d2, e2, a2, b2, w11, 14); + R41(b1, c1, d1, e1, a1, w12, 9); + R42(b2, c2, d2, e2, a2, w15, 6); + R41(a1, b1, c1, d1, e1, w4, 8); + R42(a2, b2, c2, d2, e2, w0, 14); + R41(e1, a1, b1, c1, d1, w13, 9); + R42(e2, a2, b2, c2, d2, w5, 6); + R41(d1, e1, a1, b1, c1, w3, 14); + R42(d2, e2, a2, b2, c2, w12, 9); + R41(c1, d1, e1, a1, b1, w7, 5); + R42(c2, d2, e2, a2, b2, w2, 12); + R41(b1, c1, d1, e1, a1, w15, 6); + R42(b2, c2, d2, e2, a2, w13, 9); + R41(a1, b1, c1, d1, e1, w14, 8); + R42(a2, b2, c2, d2, e2, w9, 12); + R41(e1, a1, b1, c1, d1, w5, 6); + R42(e2, a2, b2, c2, d2, w7, 5); + R41(d1, e1, a1, b1, c1, w6, 5); + R42(d2, e2, a2, b2, c2, w10, 15); + R41(c1, d1, e1, a1, b1, w2, 12); + R42(c2, d2, e2, a2, b2, w14, 8); + + R51(b1, c1, d1, e1, a1, w4, 9); + R52(b2, c2, d2, e2, a2, w12, 8); + R51(a1, b1, c1, d1, e1, w0, 15); + R52(a2, b2, c2, d2, e2, w15, 5); + R51(e1, a1, b1, c1, d1, w5, 5); + R52(e2, a2, b2, c2, d2, w10, 12); + R51(d1, e1, a1, b1, c1, w9, 11); + R52(d2, e2, a2, b2, c2, w4, 9); + R51(c1, d1, e1, a1, b1, w7, 6); + R52(c2, d2, e2, a2, b2, w1, 12); + R51(b1, c1, d1, e1, a1, w12, 8); + R52(b2, c2, d2, e2, a2, w5, 5); + R51(a1, b1, c1, d1, e1, w2, 13); + R52(a2, b2, c2, d2, e2, w8, 14); + R51(e1, a1, b1, c1, d1, w10, 12); + R52(e2, a2, b2, c2, d2, w7, 6); + R51(d1, e1, a1, b1, c1, w14, 5); + R52(d2, e2, a2, b2, c2, w6, 8); + R51(c1, d1, e1, a1, b1, w1, 12); + R52(c2, d2, e2, a2, b2, w2, 13); + R51(b1, c1, d1, e1, a1, w3, 13); + R52(b2, c2, d2, e2, a2, w13, 6); + R51(a1, b1, c1, d1, e1, w8, 14); + R52(a2, b2, c2, d2, e2, w14, 5); + R51(e1, a1, b1, c1, d1, w11, 11); + R52(e2, a2, b2, c2, d2, w0, 15); + R51(d1, e1, a1, b1, c1, w6, 8); + R52(d2, e2, a2, b2, c2, w3, 13); + R51(c1, d1, e1, a1, b1, w15, 5); + R52(c2, d2, e2, a2, b2, w9, 11); + R51(b1, c1, d1, e1, a1, w13, 6); + R52(b2, c2, d2, e2, a2, w11, 11); + + uint32_t t = s[0]; + s[0] = s[1] + c1 + d2; + s[1] = s[2] + d1 + e2; + s[2] = s[3] + e1 + a2; + s[3] = s[4] + a1 + b2; + s[4] = t + b1 + c2; +} + +} // namespace ripemd160 + +} // namespace + +////// RIPEMD160 + +CRIPEMD160::CRIPEMD160() : bytes(0) +{ + ripemd160::Initialize(s); +} + +CRIPEMD160& CRIPEMD160::Write(const unsigned char* data, size_t len) +{ + const unsigned char* end = data + len; + size_t bufsize = bytes % 64; + if (bufsize && bufsize + len >= 64) { + // Fill the buffer, and process it. + memcpy(buf + bufsize, data, 64 - bufsize); + bytes += 64 - bufsize; + data += 64 - bufsize; + ripemd160::Transform(s, buf); + bufsize = 0; + } + while (end >= data + 64) { + // Process full chunks directly from the source. + ripemd160::Transform(s, data); + bytes += 64; + data += 64; + } + if (end > data) { + // Fill the buffer with what remains. + memcpy(buf + bufsize, data, end - data); + bytes += end - data; + } + return *this; +} + +void CRIPEMD160::Finalize(unsigned char hash[OUTPUT_SIZE]) +{ + static const unsigned char pad[64] = {0x80}; + unsigned char sizedesc[8]; + WriteLE64(sizedesc, bytes << 3); + Write(pad, 1 + ((119 - (bytes % 64)) % 64)); + Write(sizedesc, 8); + WriteLE32(hash, s[0]); + WriteLE32(hash + 4, s[1]); + WriteLE32(hash + 8, s[2]); + WriteLE32(hash + 12, s[3]); + WriteLE32(hash + 16, s[4]); +} + +CRIPEMD160& CRIPEMD160::Reset() +{ + bytes = 0; + ripemd160::Initialize(s); + return *this; +} diff --git a/src/crypto/ripemd160.h b/src/crypto/ripemd160.h new file mode 100644 index 00000000..f468ec67 --- /dev/null +++ b/src/crypto/ripemd160.h @@ -0,0 +1,28 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#ifndef BITCOIN_CRYPTO_RIPEMD160_H +#define BITCOIN_CRYPTO_RIPEMD160_H + +#include +#include + +/** A hasher class for RIPEMD-160. */ +class CRIPEMD160 +{ +private: + uint32_t s[5]; + unsigned char buf[64]; + size_t bytes; + +public: + static const size_t OUTPUT_SIZE = 20; + + CRIPEMD160(); + CRIPEMD160& Write(const unsigned char* data, size_t len); + void Finalize(unsigned char hash[OUTPUT_SIZE]); + CRIPEMD160& Reset(); +}; + +#endif // BITCOIN_CRYPTO_RIPEMD160_H diff --git a/src/crypto/sha1.cpp b/src/crypto/sha1.cpp new file mode 100644 index 00000000..7f78fdfc --- /dev/null +++ b/src/crypto/sha1.cpp @@ -0,0 +1,199 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include "crypto/sha1.h" + +#include "crypto/common.h" + +#include + +// Internal implementation code. +namespace +{ +/// Internal SHA-1 implementation. +namespace sha1 +{ +/** One round of SHA-1. */ +void inline Round(uint32_t a, uint32_t& b, uint32_t c, uint32_t d, uint32_t& e, uint32_t f, uint32_t k, uint32_t w) +{ + e += ((a << 5) | (a >> 27)) + f + k + w; + b = (b << 30) | (b >> 2); +} + +uint32_t inline f1(uint32_t b, uint32_t c, uint32_t d) { return d ^ (b & (c ^ d)); } +uint32_t inline f2(uint32_t b, uint32_t c, uint32_t d) { return b ^ c ^ d; } +uint32_t inline f3(uint32_t b, uint32_t c, uint32_t d) { return (b & c) | (d & (b | c)); } + +uint32_t inline left(uint32_t x) { return (x << 1) | (x >> 31); } + +/** Initialize SHA-1 state. */ +void inline Initialize(uint32_t* s) +{ + s[0] = 0x67452301ul; + s[1] = 0xEFCDAB89ul; + s[2] = 0x98BADCFEul; + s[3] = 0x10325476ul; + s[4] = 0xC3D2E1F0ul; +} + +const uint32_t k1 = 0x5A827999ul; +const uint32_t k2 = 0x6ED9EBA1ul; +const uint32_t k3 = 0x8F1BBCDCul; +const uint32_t k4 = 0xCA62C1D6ul; + +/** Perform a SHA-1 transformation, processing a 64-byte chunk. */ +void Transform(uint32_t* s, const unsigned char* chunk) +{ + uint32_t a = s[0], b = s[1], c = s[2], d = s[3], e = s[4]; + uint32_t w0, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, w11, w12, w13, w14, w15; + + Round(a, b, c, d, e, f1(b, c, d), k1, w0 = ReadBE32(chunk + 0)); + Round(e, a, b, c, d, f1(a, b, c), k1, w1 = ReadBE32(chunk + 4)); + Round(d, e, a, b, c, f1(e, a, b), k1, w2 = ReadBE32(chunk + 8)); + Round(c, d, e, a, b, f1(d, e, a), k1, w3 = ReadBE32(chunk + 12)); + Round(b, c, d, e, a, f1(c, d, e), k1, w4 = ReadBE32(chunk + 16)); + Round(a, b, c, d, e, f1(b, c, d), k1, w5 = ReadBE32(chunk + 20)); + Round(e, a, b, c, d, f1(a, b, c), k1, w6 = ReadBE32(chunk + 24)); + Round(d, e, a, b, c, f1(e, a, b), k1, w7 = ReadBE32(chunk + 28)); + Round(c, d, e, a, b, f1(d, e, a), k1, w8 = ReadBE32(chunk + 32)); + Round(b, c, d, e, a, f1(c, d, e), k1, w9 = ReadBE32(chunk + 36)); + Round(a, b, c, d, e, f1(b, c, d), k1, w10 = ReadBE32(chunk + 40)); + Round(e, a, b, c, d, f1(a, b, c), k1, w11 = ReadBE32(chunk + 44)); + Round(d, e, a, b, c, f1(e, a, b), k1, w12 = ReadBE32(chunk + 48)); + Round(c, d, e, a, b, f1(d, e, a), k1, w13 = ReadBE32(chunk + 52)); + Round(b, c, d, e, a, f1(c, d, e), k1, w14 = ReadBE32(chunk + 56)); + Round(a, b, c, d, e, f1(b, c, d), k1, w15 = ReadBE32(chunk + 60)); + + Round(e, a, b, c, d, f1(a, b, c), k1, w0 = left(w0 ^ w13 ^ w8 ^ w2)); + Round(d, e, a, b, c, f1(e, a, b), k1, w1 = left(w1 ^ w14 ^ w9 ^ w3)); + Round(c, d, e, a, b, f1(d, e, a), k1, w2 = left(w2 ^ w15 ^ w10 ^ w4)); + Round(b, c, d, e, a, f1(c, d, e), k1, w3 = left(w3 ^ w0 ^ w11 ^ w5)); + Round(a, b, c, d, e, f2(b, c, d), k2, w4 = left(w4 ^ w1 ^ w12 ^ w6)); + Round(e, a, b, c, d, f2(a, b, c), k2, w5 = left(w5 ^ w2 ^ w13 ^ w7)); + Round(d, e, a, b, c, f2(e, a, b), k2, w6 = left(w6 ^ w3 ^ w14 ^ w8)); + Round(c, d, e, a, b, f2(d, e, a), k2, w7 = left(w7 ^ w4 ^ w15 ^ w9)); + Round(b, c, d, e, a, f2(c, d, e), k2, w8 = left(w8 ^ w5 ^ w0 ^ w10)); + Round(a, b, c, d, e, f2(b, c, d), k2, w9 = left(w9 ^ w6 ^ w1 ^ w11)); + Round(e, a, b, c, d, f2(a, b, c), k2, w10 = left(w10 ^ w7 ^ w2 ^ w12)); + Round(d, e, a, b, c, f2(e, a, b), k2, w11 = left(w11 ^ w8 ^ w3 ^ w13)); + Round(c, d, e, a, b, f2(d, e, a), k2, w12 = left(w12 ^ w9 ^ w4 ^ w14)); + Round(b, c, d, e, a, f2(c, d, e), k2, w13 = left(w13 ^ w10 ^ w5 ^ w15)); + Round(a, b, c, d, e, f2(b, c, d), k2, w14 = left(w14 ^ w11 ^ w6 ^ w0)); + Round(e, a, b, c, d, f2(a, b, c), k2, w15 = left(w15 ^ w12 ^ w7 ^ w1)); + + Round(d, e, a, b, c, f2(e, a, b), k2, w0 = left(w0 ^ w13 ^ w8 ^ w2)); + Round(c, d, e, a, b, f2(d, e, a), k2, w1 = left(w1 ^ w14 ^ w9 ^ w3)); + Round(b, c, d, e, a, f2(c, d, e), k2, w2 = left(w2 ^ w15 ^ w10 ^ w4)); + Round(a, b, c, d, e, f2(b, c, d), k2, w3 = left(w3 ^ w0 ^ w11 ^ w5)); + Round(e, a, b, c, d, f2(a, b, c), k2, w4 = left(w4 ^ w1 ^ w12 ^ w6)); + Round(d, e, a, b, c, f2(e, a, b), k2, w5 = left(w5 ^ w2 ^ w13 ^ w7)); + Round(c, d, e, a, b, f2(d, e, a), k2, w6 = left(w6 ^ w3 ^ w14 ^ w8)); + Round(b, c, d, e, a, f2(c, d, e), k2, w7 = left(w7 ^ w4 ^ w15 ^ w9)); + Round(a, b, c, d, e, f3(b, c, d), k3, w8 = left(w8 ^ w5 ^ w0 ^ w10)); + Round(e, a, b, c, d, f3(a, b, c), k3, w9 = left(w9 ^ w6 ^ w1 ^ w11)); + Round(d, e, a, b, c, f3(e, a, b), k3, w10 = left(w10 ^ w7 ^ w2 ^ w12)); + Round(c, d, e, a, b, f3(d, e, a), k3, w11 = left(w11 ^ w8 ^ w3 ^ w13)); + Round(b, c, d, e, a, f3(c, d, e), k3, w12 = left(w12 ^ w9 ^ w4 ^ w14)); + Round(a, b, c, d, e, f3(b, c, d), k3, w13 = left(w13 ^ w10 ^ w5 ^ w15)); + Round(e, a, b, c, d, f3(a, b, c), k3, w14 = left(w14 ^ w11 ^ w6 ^ w0)); + Round(d, e, a, b, c, f3(e, a, b), k3, w15 = left(w15 ^ w12 ^ w7 ^ w1)); + + Round(c, d, e, a, b, f3(d, e, a), k3, w0 = left(w0 ^ w13 ^ w8 ^ w2)); + Round(b, c, d, e, a, f3(c, d, e), k3, w1 = left(w1 ^ w14 ^ w9 ^ w3)); + Round(a, b, c, d, e, f3(b, c, d), k3, w2 = left(w2 ^ w15 ^ w10 ^ w4)); + Round(e, a, b, c, d, f3(a, b, c), k3, w3 = left(w3 ^ w0 ^ w11 ^ w5)); + Round(d, e, a, b, c, f3(e, a, b), k3, w4 = left(w4 ^ w1 ^ w12 ^ w6)); + Round(c, d, e, a, b, f3(d, e, a), k3, w5 = left(w5 ^ w2 ^ w13 ^ w7)); + Round(b, c, d, e, a, f3(c, d, e), k3, w6 = left(w6 ^ w3 ^ w14 ^ w8)); + Round(a, b, c, d, e, f3(b, c, d), k3, w7 = left(w7 ^ w4 ^ w15 ^ w9)); + Round(e, a, b, c, d, f3(a, b, c), k3, w8 = left(w8 ^ w5 ^ w0 ^ w10)); + Round(d, e, a, b, c, f3(e, a, b), k3, w9 = left(w9 ^ w6 ^ w1 ^ w11)); + Round(c, d, e, a, b, f3(d, e, a), k3, w10 = left(w10 ^ w7 ^ w2 ^ w12)); + Round(b, c, d, e, a, f3(c, d, e), k3, w11 = left(w11 ^ w8 ^ w3 ^ w13)); + Round(a, b, c, d, e, f2(b, c, d), k4, w12 = left(w12 ^ w9 ^ w4 ^ w14)); + Round(e, a, b, c, d, f2(a, b, c), k4, w13 = left(w13 ^ w10 ^ w5 ^ w15)); + Round(d, e, a, b, c, f2(e, a, b), k4, w14 = left(w14 ^ w11 ^ w6 ^ w0)); + Round(c, d, e, a, b, f2(d, e, a), k4, w15 = left(w15 ^ w12 ^ w7 ^ w1)); + + Round(b, c, d, e, a, f2(c, d, e), k4, w0 = left(w0 ^ w13 ^ w8 ^ w2)); + Round(a, b, c, d, e, f2(b, c, d), k4, w1 = left(w1 ^ w14 ^ w9 ^ w3)); + Round(e, a, b, c, d, f2(a, b, c), k4, w2 = left(w2 ^ w15 ^ w10 ^ w4)); + Round(d, e, a, b, c, f2(e, a, b), k4, w3 = left(w3 ^ w0 ^ w11 ^ w5)); + Round(c, d, e, a, b, f2(d, e, a), k4, w4 = left(w4 ^ w1 ^ w12 ^ w6)); + Round(b, c, d, e, a, f2(c, d, e), k4, w5 = left(w5 ^ w2 ^ w13 ^ w7)); + Round(a, b, c, d, e, f2(b, c, d), k4, w6 = left(w6 ^ w3 ^ w14 ^ w8)); + Round(e, a, b, c, d, f2(a, b, c), k4, w7 = left(w7 ^ w4 ^ w15 ^ w9)); + Round(d, e, a, b, c, f2(e, a, b), k4, w8 = left(w8 ^ w5 ^ w0 ^ w10)); + Round(c, d, e, a, b, f2(d, e, a), k4, w9 = left(w9 ^ w6 ^ w1 ^ w11)); + Round(b, c, d, e, a, f2(c, d, e), k4, w10 = left(w10 ^ w7 ^ w2 ^ w12)); + Round(a, b, c, d, e, f2(b, c, d), k4, w11 = left(w11 ^ w8 ^ w3 ^ w13)); + Round(e, a, b, c, d, f2(a, b, c), k4, w12 = left(w12 ^ w9 ^ w4 ^ w14)); + Round(d, e, a, b, c, f2(e, a, b), k4, left(w13 ^ w10 ^ w5 ^ w15)); + Round(c, d, e, a, b, f2(d, e, a), k4, left(w14 ^ w11 ^ w6 ^ w0)); + Round(b, c, d, e, a, f2(c, d, e), k4, left(w15 ^ w12 ^ w7 ^ w1)); + + s[0] += a; + s[1] += b; + s[2] += c; + s[3] += d; + s[4] += e; +} + +} // namespace sha1 + +} // namespace + +////// SHA1 + +CSHA1::CSHA1() : bytes(0) +{ + sha1::Initialize(s); +} + +CSHA1& CSHA1::Write(const unsigned char* data, size_t len) +{ + const unsigned char* end = data + len; + size_t bufsize = bytes % 64; + if (bufsize && bufsize + len >= 64) { + // Fill the buffer, and process it. + memcpy(buf + bufsize, data, 64 - bufsize); + bytes += 64 - bufsize; + data += 64 - bufsize; + sha1::Transform(s, buf); + bufsize = 0; + } + while (end >= data + 64) { + // Process full chunks directly from the source. + sha1::Transform(s, data); + bytes += 64; + data += 64; + } + if (end > data) { + // Fill the buffer with what remains. + memcpy(buf + bufsize, data, end - data); + bytes += end - data; + } + return *this; +} + +void CSHA1::Finalize(unsigned char hash[OUTPUT_SIZE]) +{ + static const unsigned char pad[64] = {0x80}; + unsigned char sizedesc[8]; + WriteBE64(sizedesc, bytes << 3); + Write(pad, 1 + ((119 - (bytes % 64)) % 64)); + Write(sizedesc, 8); + WriteBE32(hash, s[0]); + WriteBE32(hash + 4, s[1]); + WriteBE32(hash + 8, s[2]); + WriteBE32(hash + 12, s[3]); + WriteBE32(hash + 16, s[4]); +} + +CSHA1& CSHA1::Reset() +{ + bytes = 0; + sha1::Initialize(s); + return *this; +} diff --git a/src/crypto/sha1.h b/src/crypto/sha1.h new file mode 100644 index 00000000..e28f98de --- /dev/null +++ b/src/crypto/sha1.h @@ -0,0 +1,28 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#ifndef BITCOIN_CRYPTO_SHA1_H +#define BITCOIN_CRYPTO_SHA1_H + +#include +#include + +/** A hasher class for SHA1. */ +class CSHA1 +{ +private: + uint32_t s[5]; + unsigned char buf[64]; + size_t bytes; + +public: + static const size_t OUTPUT_SIZE = 20; + + CSHA1(); + CSHA1& Write(const unsigned char* data, size_t len); + void Finalize(unsigned char hash[OUTPUT_SIZE]); + CSHA1& Reset(); +}; + +#endif // BITCOIN_CRYPTO_SHA1_H diff --git a/src/crypto/sha256.cpp b/src/crypto/sha256.cpp new file mode 100644 index 00000000..8410e593 --- /dev/null +++ b/src/crypto/sha256.cpp @@ -0,0 +1,189 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include "crypto/sha256.h" + +#include "crypto/common.h" + +#include + +// Internal implementation code. +namespace +{ +/// Internal SHA-256 implementation. +namespace sha256 +{ +uint32_t inline Ch(uint32_t x, uint32_t y, uint32_t z) { return z ^ (x & (y ^ z)); } +uint32_t inline Maj(uint32_t x, uint32_t y, uint32_t z) { return (x & y) | (z & (x | y)); } +uint32_t inline Sigma0(uint32_t x) { return (x >> 2 | x << 30) ^ (x >> 13 | x << 19) ^ (x >> 22 | x << 10); } +uint32_t inline Sigma1(uint32_t x) { return (x >> 6 | x << 26) ^ (x >> 11 | x << 21) ^ (x >> 25 | x << 7); } +uint32_t inline sigma0(uint32_t x) { return (x >> 7 | x << 25) ^ (x >> 18 | x << 14) ^ (x >> 3); } +uint32_t inline sigma1(uint32_t x) { return (x >> 17 | x << 15) ^ (x >> 19 | x << 13) ^ (x >> 10); } + +/** One round of SHA-256. */ +void inline Round(uint32_t a, uint32_t b, uint32_t c, uint32_t& d, uint32_t e, uint32_t f, uint32_t g, uint32_t& h, uint32_t k, uint32_t w) +{ + uint32_t t1 = h + Sigma1(e) + Ch(e, f, g) + k + w; + uint32_t t2 = Sigma0(a) + Maj(a, b, c); + d += t1; + h = t1 + t2; +} + +/** Initialize SHA-256 state. */ +void inline Initialize(uint32_t* s) +{ + s[0] = 0x6a09e667ul; + s[1] = 0xbb67ae85ul; + s[2] = 0x3c6ef372ul; + s[3] = 0xa54ff53aul; + s[4] = 0x510e527ful; + s[5] = 0x9b05688cul; + s[6] = 0x1f83d9abul; + s[7] = 0x5be0cd19ul; +} + +/** Perform one SHA-256 transformation, processing a 64-byte chunk. */ +void Transform(uint32_t* s, const unsigned char* chunk) +{ + uint32_t a = s[0], b = s[1], c = s[2], d = s[3], e = s[4], f = s[5], g = s[6], h = s[7]; + uint32_t w0, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, w11, w12, w13, w14, w15; + + Round(a, b, c, d, e, f, g, h, 0x428a2f98, w0 = ReadBE32(chunk + 0)); + Round(h, a, b, c, d, e, f, g, 0x71374491, w1 = ReadBE32(chunk + 4)); + Round(g, h, a, b, c, d, e, f, 0xb5c0fbcf, w2 = ReadBE32(chunk + 8)); + Round(f, g, h, a, b, c, d, e, 0xe9b5dba5, w3 = ReadBE32(chunk + 12)); + Round(e, f, g, h, a, b, c, d, 0x3956c25b, w4 = ReadBE32(chunk + 16)); + Round(d, e, f, g, h, a, b, c, 0x59f111f1, w5 = ReadBE32(chunk + 20)); + Round(c, d, e, f, g, h, a, b, 0x923f82a4, w6 = ReadBE32(chunk + 24)); + Round(b, c, d, e, f, g, h, a, 0xab1c5ed5, w7 = ReadBE32(chunk + 28)); + Round(a, b, c, d, e, f, g, h, 0xd807aa98, w8 = ReadBE32(chunk + 32)); + Round(h, a, b, c, d, e, f, g, 0x12835b01, w9 = ReadBE32(chunk + 36)); + Round(g, h, a, b, c, d, e, f, 0x243185be, w10 = ReadBE32(chunk + 40)); + Round(f, g, h, a, b, c, d, e, 0x550c7dc3, w11 = ReadBE32(chunk + 44)); + Round(e, f, g, h, a, b, c, d, 0x72be5d74, w12 = ReadBE32(chunk + 48)); + Round(d, e, f, g, h, a, b, c, 0x80deb1fe, w13 = ReadBE32(chunk + 52)); + Round(c, d, e, f, g, h, a, b, 0x9bdc06a7, w14 = ReadBE32(chunk + 56)); + Round(b, c, d, e, f, g, h, a, 0xc19bf174, w15 = ReadBE32(chunk + 60)); + + Round(a, b, c, d, e, f, g, h, 0xe49b69c1, w0 += sigma1(w14) + w9 + sigma0(w1)); + Round(h, a, b, c, d, e, f, g, 0xefbe4786, w1 += sigma1(w15) + w10 + sigma0(w2)); + Round(g, h, a, b, c, d, e, f, 0x0fc19dc6, w2 += sigma1(w0) + w11 + sigma0(w3)); + Round(f, g, h, a, b, c, d, e, 0x240ca1cc, w3 += sigma1(w1) + w12 + sigma0(w4)); + Round(e, f, g, h, a, b, c, d, 0x2de92c6f, w4 += sigma1(w2) + w13 + sigma0(w5)); + Round(d, e, f, g, h, a, b, c, 0x4a7484aa, w5 += sigma1(w3) + w14 + sigma0(w6)); + Round(c, d, e, f, g, h, a, b, 0x5cb0a9dc, w6 += sigma1(w4) + w15 + sigma0(w7)); + Round(b, c, d, e, f, g, h, a, 0x76f988da, w7 += sigma1(w5) + w0 + sigma0(w8)); + Round(a, b, c, d, e, f, g, h, 0x983e5152, w8 += sigma1(w6) + w1 + sigma0(w9)); + Round(h, a, b, c, d, e, f, g, 0xa831c66d, w9 += sigma1(w7) + w2 + sigma0(w10)); + Round(g, h, a, b, c, d, e, f, 0xb00327c8, w10 += sigma1(w8) + w3 + sigma0(w11)); + Round(f, g, h, a, b, c, d, e, 0xbf597fc7, w11 += sigma1(w9) + w4 + sigma0(w12)); + Round(e, f, g, h, a, b, c, d, 0xc6e00bf3, w12 += sigma1(w10) + w5 + sigma0(w13)); + Round(d, e, f, g, h, a, b, c, 0xd5a79147, w13 += sigma1(w11) + w6 + sigma0(w14)); + Round(c, d, e, f, g, h, a, b, 0x06ca6351, w14 += sigma1(w12) + w7 + sigma0(w15)); + Round(b, c, d, e, f, g, h, a, 0x14292967, w15 += sigma1(w13) + w8 + sigma0(w0)); + + Round(a, b, c, d, e, f, g, h, 0x27b70a85, w0 += sigma1(w14) + w9 + sigma0(w1)); + Round(h, a, b, c, d, e, f, g, 0x2e1b2138, w1 += sigma1(w15) + w10 + sigma0(w2)); + Round(g, h, a, b, c, d, e, f, 0x4d2c6dfc, w2 += sigma1(w0) + w11 + sigma0(w3)); + Round(f, g, h, a, b, c, d, e, 0x53380d13, w3 += sigma1(w1) + w12 + sigma0(w4)); + Round(e, f, g, h, a, b, c, d, 0x650a7354, w4 += sigma1(w2) + w13 + sigma0(w5)); + Round(d, e, f, g, h, a, b, c, 0x766a0abb, w5 += sigma1(w3) + w14 + sigma0(w6)); + Round(c, d, e, f, g, h, a, b, 0x81c2c92e, w6 += sigma1(w4) + w15 + sigma0(w7)); + Round(b, c, d, e, f, g, h, a, 0x92722c85, w7 += sigma1(w5) + w0 + sigma0(w8)); + Round(a, b, c, d, e, f, g, h, 0xa2bfe8a1, w8 += sigma1(w6) + w1 + sigma0(w9)); + Round(h, a, b, c, d, e, f, g, 0xa81a664b, w9 += sigma1(w7) + w2 + sigma0(w10)); + Round(g, h, a, b, c, d, e, f, 0xc24b8b70, w10 += sigma1(w8) + w3 + sigma0(w11)); + Round(f, g, h, a, b, c, d, e, 0xc76c51a3, w11 += sigma1(w9) + w4 + sigma0(w12)); + Round(e, f, g, h, a, b, c, d, 0xd192e819, w12 += sigma1(w10) + w5 + sigma0(w13)); + Round(d, e, f, g, h, a, b, c, 0xd6990624, w13 += sigma1(w11) + w6 + sigma0(w14)); + Round(c, d, e, f, g, h, a, b, 0xf40e3585, w14 += sigma1(w12) + w7 + sigma0(w15)); + Round(b, c, d, e, f, g, h, a, 0x106aa070, w15 += sigma1(w13) + w8 + sigma0(w0)); + + Round(a, b, c, d, e, f, g, h, 0x19a4c116, w0 += sigma1(w14) + w9 + sigma0(w1)); + Round(h, a, b, c, d, e, f, g, 0x1e376c08, w1 += sigma1(w15) + w10 + sigma0(w2)); + Round(g, h, a, b, c, d, e, f, 0x2748774c, w2 += sigma1(w0) + w11 + sigma0(w3)); + Round(f, g, h, a, b, c, d, e, 0x34b0bcb5, w3 += sigma1(w1) + w12 + sigma0(w4)); + Round(e, f, g, h, a, b, c, d, 0x391c0cb3, w4 += sigma1(w2) + w13 + sigma0(w5)); + Round(d, e, f, g, h, a, b, c, 0x4ed8aa4a, w5 += sigma1(w3) + w14 + sigma0(w6)); + Round(c, d, e, f, g, h, a, b, 0x5b9cca4f, w6 += sigma1(w4) + w15 + sigma0(w7)); + Round(b, c, d, e, f, g, h, a, 0x682e6ff3, w7 += sigma1(w5) + w0 + sigma0(w8)); + Round(a, b, c, d, e, f, g, h, 0x748f82ee, w8 += sigma1(w6) + w1 + sigma0(w9)); + Round(h, a, b, c, d, e, f, g, 0x78a5636f, w9 += sigma1(w7) + w2 + sigma0(w10)); + Round(g, h, a, b, c, d, e, f, 0x84c87814, w10 += sigma1(w8) + w3 + sigma0(w11)); + Round(f, g, h, a, b, c, d, e, 0x8cc70208, w11 += sigma1(w9) + w4 + sigma0(w12)); + Round(e, f, g, h, a, b, c, d, 0x90befffa, w12 += sigma1(w10) + w5 + sigma0(w13)); + Round(d, e, f, g, h, a, b, c, 0xa4506ceb, w13 += sigma1(w11) + w6 + sigma0(w14)); + Round(c, d, e, f, g, h, a, b, 0xbef9a3f7, w14 + sigma1(w12) + w7 + sigma0(w15)); + Round(b, c, d, e, f, g, h, a, 0xc67178f2, w15 + sigma1(w13) + w8 + sigma0(w0)); + + s[0] += a; + s[1] += b; + s[2] += c; + s[3] += d; + s[4] += e; + s[5] += f; + s[6] += g; + s[7] += h; +} + +} // namespace sha256 +} // namespace + + +////// SHA-256 + +CSHA256::CSHA256() : bytes(0) +{ + sha256::Initialize(s); +} + +CSHA256& CSHA256::Write(const unsigned char* data, size_t len) +{ + const unsigned char* end = data + len; + size_t bufsize = bytes % 64; + if (bufsize && bufsize + len >= 64) { + // Fill the buffer, and process it. + memcpy(buf + bufsize, data, 64 - bufsize); + bytes += 64 - bufsize; + data += 64 - bufsize; + sha256::Transform(s, buf); + bufsize = 0; + } + while (end >= data + 64) { + // Process full chunks directly from the source. + sha256::Transform(s, data); + bytes += 64; + data += 64; + } + if (end > data) { + // Fill the buffer with what remains. + memcpy(buf + bufsize, data, end - data); + bytes += end - data; + } + return *this; +} + +void CSHA256::Finalize(unsigned char hash[OUTPUT_SIZE]) +{ + static const unsigned char pad[64] = {0x80}; + unsigned char sizedesc[8]; + WriteBE64(sizedesc, bytes << 3); + Write(pad, 1 + ((119 - (bytes % 64)) % 64)); + Write(sizedesc, 8); + WriteBE32(hash, s[0]); + WriteBE32(hash + 4, s[1]); + WriteBE32(hash + 8, s[2]); + WriteBE32(hash + 12, s[3]); + WriteBE32(hash + 16, s[4]); + WriteBE32(hash + 20, s[5]); + WriteBE32(hash + 24, s[6]); + WriteBE32(hash + 28, s[7]); +} + +CSHA256& CSHA256::Reset() +{ + bytes = 0; + sha256::Initialize(s); + return *this; +} diff --git a/src/crypto/sha256.h b/src/crypto/sha256.h new file mode 100644 index 00000000..bde1a59b --- /dev/null +++ b/src/crypto/sha256.h @@ -0,0 +1,28 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#ifndef BITCOIN_CRYPTO_SHA256_H +#define BITCOIN_CRYPTO_SHA256_H + +#include +#include + +/** A hasher class for SHA-256. */ +class CSHA256 +{ +private: + uint32_t s[8]; + unsigned char buf[64]; + size_t bytes; + +public: + static const size_t OUTPUT_SIZE = 32; + + CSHA256(); + CSHA256& Write(const unsigned char* data, size_t len); + void Finalize(unsigned char hash[OUTPUT_SIZE]); + CSHA256& Reset(); +}; + +#endif // BITCOIN_CRYPTO_SHA256_H diff --git a/src/crypto/sha512.cpp b/src/crypto/sha512.cpp new file mode 100644 index 00000000..22c3103b --- /dev/null +++ b/src/crypto/sha512.cpp @@ -0,0 +1,207 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include "crypto/sha512.h" + +#include "crypto/common.h" + +#include + +// Internal implementation code. +namespace +{ +/// Internal SHA-512 implementation. +namespace sha512 +{ +uint64_t inline Ch(uint64_t x, uint64_t y, uint64_t z) { return z ^ (x & (y ^ z)); } +uint64_t inline Maj(uint64_t x, uint64_t y, uint64_t z) { return (x & y) | (z & (x | y)); } +uint64_t inline Sigma0(uint64_t x) { return (x >> 28 | x << 36) ^ (x >> 34 | x << 30) ^ (x >> 39 | x << 25); } +uint64_t inline Sigma1(uint64_t x) { return (x >> 14 | x << 50) ^ (x >> 18 | x << 46) ^ (x >> 41 | x << 23); } +uint64_t inline sigma0(uint64_t x) { return (x >> 1 | x << 63) ^ (x >> 8 | x << 56) ^ (x >> 7); } +uint64_t inline sigma1(uint64_t x) { return (x >> 19 | x << 45) ^ (x >> 61 | x << 3) ^ (x >> 6); } + +/** One round of SHA-512. */ +void inline Round(uint64_t a, uint64_t b, uint64_t c, uint64_t& d, uint64_t e, uint64_t f, uint64_t g, uint64_t& h, uint64_t k, uint64_t w) +{ + uint64_t t1 = h + Sigma1(e) + Ch(e, f, g) + k + w; + uint64_t t2 = Sigma0(a) + Maj(a, b, c); + d += t1; + h = t1 + t2; +} + +/** Initialize SHA-256 state. */ +void inline Initialize(uint64_t* s) +{ + s[0] = 0x6a09e667f3bcc908ull; + s[1] = 0xbb67ae8584caa73bull; + s[2] = 0x3c6ef372fe94f82bull; + s[3] = 0xa54ff53a5f1d36f1ull; + s[4] = 0x510e527fade682d1ull; + s[5] = 0x9b05688c2b3e6c1full; + s[6] = 0x1f83d9abfb41bd6bull; + s[7] = 0x5be0cd19137e2179ull; +} + +/** Perform one SHA-512 transformation, processing a 128-byte chunk. */ +void Transform(uint64_t* s, const unsigned char* chunk) +{ + uint64_t a = s[0], b = s[1], c = s[2], d = s[3], e = s[4], f = s[5], g = s[6], h = s[7]; + uint64_t w0, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, w11, w12, w13, w14, w15; + + Round(a, b, c, d, e, f, g, h, 0x428a2f98d728ae22ull, w0 = ReadBE64(chunk + 0)); + Round(h, a, b, c, d, e, f, g, 0x7137449123ef65cdull, w1 = ReadBE64(chunk + 8)); + Round(g, h, a, b, c, d, e, f, 0xb5c0fbcfec4d3b2full, w2 = ReadBE64(chunk + 16)); + Round(f, g, h, a, b, c, d, e, 0xe9b5dba58189dbbcull, w3 = ReadBE64(chunk + 24)); + Round(e, f, g, h, a, b, c, d, 0x3956c25bf348b538ull, w4 = ReadBE64(chunk + 32)); + Round(d, e, f, g, h, a, b, c, 0x59f111f1b605d019ull, w5 = ReadBE64(chunk + 40)); + Round(c, d, e, f, g, h, a, b, 0x923f82a4af194f9bull, w6 = ReadBE64(chunk + 48)); + Round(b, c, d, e, f, g, h, a, 0xab1c5ed5da6d8118ull, w7 = ReadBE64(chunk + 56)); + Round(a, b, c, d, e, f, g, h, 0xd807aa98a3030242ull, w8 = ReadBE64(chunk + 64)); + Round(h, a, b, c, d, e, f, g, 0x12835b0145706fbeull, w9 = ReadBE64(chunk + 72)); + Round(g, h, a, b, c, d, e, f, 0x243185be4ee4b28cull, w10 = ReadBE64(chunk + 80)); + Round(f, g, h, a, b, c, d, e, 0x550c7dc3d5ffb4e2ull, w11 = ReadBE64(chunk + 88)); + Round(e, f, g, h, a, b, c, d, 0x72be5d74f27b896full, w12 = ReadBE64(chunk + 96)); + Round(d, e, f, g, h, a, b, c, 0x80deb1fe3b1696b1ull, w13 = ReadBE64(chunk + 104)); + Round(c, d, e, f, g, h, a, b, 0x9bdc06a725c71235ull, w14 = ReadBE64(chunk + 112)); + Round(b, c, d, e, f, g, h, a, 0xc19bf174cf692694ull, w15 = ReadBE64(chunk + 120)); + + Round(a, b, c, d, e, f, g, h, 0xe49b69c19ef14ad2ull, w0 += sigma1(w14) + w9 + sigma0(w1)); + Round(h, a, b, c, d, e, f, g, 0xefbe4786384f25e3ull, w1 += sigma1(w15) + w10 + sigma0(w2)); + Round(g, h, a, b, c, d, e, f, 0x0fc19dc68b8cd5b5ull, w2 += sigma1(w0) + w11 + sigma0(w3)); + Round(f, g, h, a, b, c, d, e, 0x240ca1cc77ac9c65ull, w3 += sigma1(w1) + w12 + sigma0(w4)); + Round(e, f, g, h, a, b, c, d, 0x2de92c6f592b0275ull, w4 += sigma1(w2) + w13 + sigma0(w5)); + Round(d, e, f, g, h, a, b, c, 0x4a7484aa6ea6e483ull, w5 += sigma1(w3) + w14 + sigma0(w6)); + Round(c, d, e, f, g, h, a, b, 0x5cb0a9dcbd41fbd4ull, w6 += sigma1(w4) + w15 + sigma0(w7)); + Round(b, c, d, e, f, g, h, a, 0x76f988da831153b5ull, w7 += sigma1(w5) + w0 + sigma0(w8)); + Round(a, b, c, d, e, f, g, h, 0x983e5152ee66dfabull, w8 += sigma1(w6) + w1 + sigma0(w9)); + Round(h, a, b, c, d, e, f, g, 0xa831c66d2db43210ull, w9 += sigma1(w7) + w2 + sigma0(w10)); + Round(g, h, a, b, c, d, e, f, 0xb00327c898fb213full, w10 += sigma1(w8) + w3 + sigma0(w11)); + Round(f, g, h, a, b, c, d, e, 0xbf597fc7beef0ee4ull, w11 += sigma1(w9) + w4 + sigma0(w12)); + Round(e, f, g, h, a, b, c, d, 0xc6e00bf33da88fc2ull, w12 += sigma1(w10) + w5 + sigma0(w13)); + Round(d, e, f, g, h, a, b, c, 0xd5a79147930aa725ull, w13 += sigma1(w11) + w6 + sigma0(w14)); + Round(c, d, e, f, g, h, a, b, 0x06ca6351e003826full, w14 += sigma1(w12) + w7 + sigma0(w15)); + Round(b, c, d, e, f, g, h, a, 0x142929670a0e6e70ull, w15 += sigma1(w13) + w8 + sigma0(w0)); + + Round(a, b, c, d, e, f, g, h, 0x27b70a8546d22ffcull, w0 += sigma1(w14) + w9 + sigma0(w1)); + Round(h, a, b, c, d, e, f, g, 0x2e1b21385c26c926ull, w1 += sigma1(w15) + w10 + sigma0(w2)); + Round(g, h, a, b, c, d, e, f, 0x4d2c6dfc5ac42aedull, w2 += sigma1(w0) + w11 + sigma0(w3)); + Round(f, g, h, a, b, c, d, e, 0x53380d139d95b3dfull, w3 += sigma1(w1) + w12 + sigma0(w4)); + Round(e, f, g, h, a, b, c, d, 0x650a73548baf63deull, w4 += sigma1(w2) + w13 + sigma0(w5)); + Round(d, e, f, g, h, a, b, c, 0x766a0abb3c77b2a8ull, w5 += sigma1(w3) + w14 + sigma0(w6)); + Round(c, d, e, f, g, h, a, b, 0x81c2c92e47edaee6ull, w6 += sigma1(w4) + w15 + sigma0(w7)); + Round(b, c, d, e, f, g, h, a, 0x92722c851482353bull, w7 += sigma1(w5) + w0 + sigma0(w8)); + Round(a, b, c, d, e, f, g, h, 0xa2bfe8a14cf10364ull, w8 += sigma1(w6) + w1 + sigma0(w9)); + Round(h, a, b, c, d, e, f, g, 0xa81a664bbc423001ull, w9 += sigma1(w7) + w2 + sigma0(w10)); + Round(g, h, a, b, c, d, e, f, 0xc24b8b70d0f89791ull, w10 += sigma1(w8) + w3 + sigma0(w11)); + Round(f, g, h, a, b, c, d, e, 0xc76c51a30654be30ull, w11 += sigma1(w9) + w4 + sigma0(w12)); + Round(e, f, g, h, a, b, c, d, 0xd192e819d6ef5218ull, w12 += sigma1(w10) + w5 + sigma0(w13)); + Round(d, e, f, g, h, a, b, c, 0xd69906245565a910ull, w13 += sigma1(w11) + w6 + sigma0(w14)); + Round(c, d, e, f, g, h, a, b, 0xf40e35855771202aull, w14 += sigma1(w12) + w7 + sigma0(w15)); + Round(b, c, d, e, f, g, h, a, 0x106aa07032bbd1b8ull, w15 += sigma1(w13) + w8 + sigma0(w0)); + + Round(a, b, c, d, e, f, g, h, 0x19a4c116b8d2d0c8ull, w0 += sigma1(w14) + w9 + sigma0(w1)); + Round(h, a, b, c, d, e, f, g, 0x1e376c085141ab53ull, w1 += sigma1(w15) + w10 + sigma0(w2)); + Round(g, h, a, b, c, d, e, f, 0x2748774cdf8eeb99ull, w2 += sigma1(w0) + w11 + sigma0(w3)); + Round(f, g, h, a, b, c, d, e, 0x34b0bcb5e19b48a8ull, w3 += sigma1(w1) + w12 + sigma0(w4)); + Round(e, f, g, h, a, b, c, d, 0x391c0cb3c5c95a63ull, w4 += sigma1(w2) + w13 + sigma0(w5)); + Round(d, e, f, g, h, a, b, c, 0x4ed8aa4ae3418acbull, w5 += sigma1(w3) + w14 + sigma0(w6)); + Round(c, d, e, f, g, h, a, b, 0x5b9cca4f7763e373ull, w6 += sigma1(w4) + w15 + sigma0(w7)); + Round(b, c, d, e, f, g, h, a, 0x682e6ff3d6b2b8a3ull, w7 += sigma1(w5) + w0 + sigma0(w8)); + Round(a, b, c, d, e, f, g, h, 0x748f82ee5defb2fcull, w8 += sigma1(w6) + w1 + sigma0(w9)); + Round(h, a, b, c, d, e, f, g, 0x78a5636f43172f60ull, w9 += sigma1(w7) + w2 + sigma0(w10)); + Round(g, h, a, b, c, d, e, f, 0x84c87814a1f0ab72ull, w10 += sigma1(w8) + w3 + sigma0(w11)); + Round(f, g, h, a, b, c, d, e, 0x8cc702081a6439ecull, w11 += sigma1(w9) + w4 + sigma0(w12)); + Round(e, f, g, h, a, b, c, d, 0x90befffa23631e28ull, w12 += sigma1(w10) + w5 + sigma0(w13)); + Round(d, e, f, g, h, a, b, c, 0xa4506cebde82bde9ull, w13 += sigma1(w11) + w6 + sigma0(w14)); + Round(c, d, e, f, g, h, a, b, 0xbef9a3f7b2c67915ull, w14 += sigma1(w12) + w7 + sigma0(w15)); + Round(b, c, d, e, f, g, h, a, 0xc67178f2e372532bull, w15 += sigma1(w13) + w8 + sigma0(w0)); + + Round(a, b, c, d, e, f, g, h, 0xca273eceea26619cull, w0 += sigma1(w14) + w9 + sigma0(w1)); + Round(h, a, b, c, d, e, f, g, 0xd186b8c721c0c207ull, w1 += sigma1(w15) + w10 + sigma0(w2)); + Round(g, h, a, b, c, d, e, f, 0xeada7dd6cde0eb1eull, w2 += sigma1(w0) + w11 + sigma0(w3)); + Round(f, g, h, a, b, c, d, e, 0xf57d4f7fee6ed178ull, w3 += sigma1(w1) + w12 + sigma0(w4)); + Round(e, f, g, h, a, b, c, d, 0x06f067aa72176fbaull, w4 += sigma1(w2) + w13 + sigma0(w5)); + Round(d, e, f, g, h, a, b, c, 0x0a637dc5a2c898a6ull, w5 += sigma1(w3) + w14 + sigma0(w6)); + Round(c, d, e, f, g, h, a, b, 0x113f9804bef90daeull, w6 += sigma1(w4) + w15 + sigma0(w7)); + Round(b, c, d, e, f, g, h, a, 0x1b710b35131c471bull, w7 += sigma1(w5) + w0 + sigma0(w8)); + Round(a, b, c, d, e, f, g, h, 0x28db77f523047d84ull, w8 += sigma1(w6) + w1 + sigma0(w9)); + Round(h, a, b, c, d, e, f, g, 0x32caab7b40c72493ull, w9 += sigma1(w7) + w2 + sigma0(w10)); + Round(g, h, a, b, c, d, e, f, 0x3c9ebe0a15c9bebcull, w10 += sigma1(w8) + w3 + sigma0(w11)); + Round(f, g, h, a, b, c, d, e, 0x431d67c49c100d4cull, w11 += sigma1(w9) + w4 + sigma0(w12)); + Round(e, f, g, h, a, b, c, d, 0x4cc5d4becb3e42b6ull, w12 += sigma1(w10) + w5 + sigma0(w13)); + Round(d, e, f, g, h, a, b, c, 0x597f299cfc657e2aull, w13 += sigma1(w11) + w6 + sigma0(w14)); + Round(c, d, e, f, g, h, a, b, 0x5fcb6fab3ad6faecull, w14 + sigma1(w12) + w7 + sigma0(w15)); + Round(b, c, d, e, f, g, h, a, 0x6c44198c4a475817ull, w15 + sigma1(w13) + w8 + sigma0(w0)); + + s[0] += a; + s[1] += b; + s[2] += c; + s[3] += d; + s[4] += e; + s[5] += f; + s[6] += g; + s[7] += h; +} + +} // namespace sha512 + +} // namespace + + +////// SHA-512 + +CSHA512::CSHA512() : bytes(0) +{ + sha512::Initialize(s); +} + +CSHA512& CSHA512::Write(const unsigned char* data, size_t len) +{ + const unsigned char* end = data + len; + size_t bufsize = bytes % 128; + if (bufsize && bufsize + len >= 128) { + // Fill the buffer, and process it. + memcpy(buf + bufsize, data, 128 - bufsize); + bytes += 128 - bufsize; + data += 128 - bufsize; + sha512::Transform(s, buf); + bufsize = 0; + } + while (end >= data + 128) { + // Process full chunks directly from the source. + sha512::Transform(s, data); + data += 128; + bytes += 128; + } + if (end > data) { + // Fill the buffer with what remains. + memcpy(buf + bufsize, data, end - data); + bytes += end - data; + } + return *this; +} + +void CSHA512::Finalize(unsigned char hash[OUTPUT_SIZE]) +{ + static const unsigned char pad[128] = {0x80}; + unsigned char sizedesc[16] = {0x00}; + WriteBE64(sizedesc + 8, bytes << 3); + Write(pad, 1 + ((239 - (bytes % 128)) % 128)); + Write(sizedesc, 16); + WriteBE64(hash, s[0]); + WriteBE64(hash + 8, s[1]); + WriteBE64(hash + 16, s[2]); + WriteBE64(hash + 24, s[3]); + WriteBE64(hash + 32, s[4]); + WriteBE64(hash + 40, s[5]); + WriteBE64(hash + 48, s[6]); + WriteBE64(hash + 56, s[7]); +} + +CSHA512& CSHA512::Reset() +{ + bytes = 0; + sha512::Initialize(s); + return *this; +} diff --git a/src/crypto/sha512.h b/src/crypto/sha512.h new file mode 100644 index 00000000..5566d5db --- /dev/null +++ b/src/crypto/sha512.h @@ -0,0 +1,28 @@ +// Copyright (c) 2014 The Bitcoin developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#ifndef BITCOIN_CRYPTO_SHA512_H +#define BITCOIN_CRYPTO_SHA512_H + +#include +#include + +/** A hasher class for SHA-512. */ +class CSHA512 +{ +private: + uint64_t s[8]; + unsigned char buf[128]; + size_t bytes; + +public: + static const size_t OUTPUT_SIZE = 64; + + CSHA512(); + CSHA512& Write(const unsigned char* data, size_t len); + void Finalize(unsigned char hash[OUTPUT_SIZE]); + CSHA512& Reset(); +}; + +#endif // BITCOIN_CRYPTO_SHA512_H diff --git a/src/entities/asset.cpp b/src/entities/asset.cpp new file mode 100644 index 00000000..d28f95c5 --- /dev/null +++ b/src/entities/asset.cpp @@ -0,0 +1,2218 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" + +#define MC_AST_ASSET_REF_TYPE_OFFSET 32 +#define MC_AST_ASSET_REF_TYPE_SIZE 4 +#define MC_AST_ASSET_SCRIPT_TYPE_OFFSET 44 +#define MC_AST_ASSET_SCRIPT_TYPE_SIZE 4 + +void mc_EntityDBRow::Zero() +{ + memset(this,0,sizeof(mc_EntityDBRow)); +} + +void mc_EntityLedgerRow::Zero() +{ + memset(this,0,sizeof(mc_EntityLedgerRow)); +} + +void mc_EntityDetails::Zero() +{ + memset(this,0,sizeof(mc_EntityDetails)); + m_LedgerRow.Zero(); +} + + + +/** Set initial database object values */ + +void mc_EntityDB::Zero() +{ + m_FileName[0]=0; + m_DB=0; + m_KeyOffset=0; + m_KeySize=36; + m_ValueOffset=36; + if(mc_gState->m_Features->FollowOnIssues()) + { + m_ValueSize=28; + } + else + { + m_ValueSize=12; + } + m_TotalSize=m_KeySize+m_ValueSize; +} + +/** Set database file name */ + +void mc_EntityDB::SetName(const char* name) +{ + if(mc_gState->m_Features->FollowOnIssues()) + { + mc_GetFullFileName(name,"entities",".db",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR,m_FileName); + } + else + { + mc_GetFullFileName(name,"assets",".db",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR,m_FileName); + } +} + +/** Open database */ + +int mc_EntityDB::Open() +{ + + m_DB=new mc_Database; + + m_DB->SetOption("KeySize",0,m_KeySize); + m_DB->SetOption("ValueSize",0,m_ValueSize); + + return m_DB->Open(m_FileName,MC_OPT_DB_DATABASE_CREATE_IF_MISSING | MC_OPT_DB_DATABASE_TRANSACTIONAL | MC_OPT_DB_DATABASE_LEVELDB); +} + +/** Close database */ + +int mc_EntityDB::Close() +{ + if(m_DB) + { + m_DB->Close(); + delete m_DB; + m_DB=NULL; + } + return 0; +} + +/** Set initial ledger values */ + +void mc_EntityLedger::Zero() +{ + m_FileName[0]=0; + m_FileHan=0; + m_KeyOffset=0; + m_KeySize=36; + m_ValueOffset=36; + if(mc_gState->m_Features->FollowOnIssues()) + { + m_ValueSize=60; + } + else + { + m_ValueSize=28; + } + m_TotalSize=m_KeySize+m_ValueSize; +} + +/** Set ledger file name */ + +void mc_EntityLedger::SetName(const char* name) +{ + if(mc_gState->m_Features->FollowOnIssues()) + { + mc_GetFullFileName(name,"entities",".dat",MC_FOM_RELATIVE_TO_DATADIR,m_FileName); + } + else + { + mc_GetFullFileName(name,"assets",".dat",MC_FOM_RELATIVE_TO_DATADIR,m_FileName); + } +} + +/** Open ledger file */ + +int mc_EntityLedger::Open() +{ + if(m_FileHan>0) + { + return m_FileHan; + } + + m_FileHan=open(m_FileName,_O_BINARY | O_RDWR | O_CREAT, S_IRUSR | S_IWUSR); + return m_FileHan; +} + +/** Close ledger file */ + +int mc_EntityLedger::Close() +{ + if(m_FileHan>0) + { + close(m_FileHan); + } + m_FileHan=0; + return 0; +} + +int mc_EntityLedger::GetRow(int64_t pos, mc_EntityLedgerRow* row) +{ + int size; + unsigned char buf[64]; + if(m_FileHan<=0) + { + return MC_ERR_INTERNAL_ERROR; + } + + if(lseek64(m_FileHan,pos,SEEK_SET) != pos) + { + return MC_ERR_NOT_FOUND; + } + + row->Zero(); + + if(mc_gState->m_Features->FollowOnIssues()) + { + if(read(m_FileHan,(unsigned char*)row+m_KeyOffset,m_TotalSize) != m_TotalSize) + { + return MC_ERR_FILE_READ_ERROR; + } + } + else + { + if(read(m_FileHan,buf,m_TotalSize) != m_TotalSize) + { + return MC_ERR_FILE_READ_ERROR; + } + memcpy(row->m_Key,buf+ 0,MC_ENT_KEY_SIZE); + row->m_KeyType=mc_GetLE(buf+32,4); + row->m_Block=mc_GetLE(buf+36,4); + row->m_Offset=mc_GetLE(buf+40,4); + row->m_Quantity=mc_GetLE(buf+44,8); + row->m_PrevPos=mc_GetLE(buf+52,8); + row->m_ScriptSize=mc_GetLE(buf+60,4); + row->m_EntityType=MC_ENT_TYPE_ASSET; + row->m_FirstPos=pos; + row->m_LastPos=0; + row->m_ChainPos=pos; + } + + size=row->m_ScriptSize; + + if((size>0) && (size<=MC_ENT_MAX_SCRIPT_SIZE)) + { + size=mc_AllocSize(size,m_TotalSize,1); + if(read(m_FileHan,row->m_Script,size) != size) + { + return MC_ERR_FILE_READ_ERROR; + } + } + + + return MC_ERR_NOERROR; +} + +int64_t mc_EntityLedger::GetSize() +{ + int64_t pos; + + mc_EntityLedgerRow aldRow; + if(m_FileHan<=0) + { + return 0; + } + + GetRow(0,&aldRow); + pos=aldRow.m_PrevPos; + GetRow(pos,&aldRow); + pos+=mc_AllocSize(m_TotalSize+aldRow.m_ScriptSize,m_TotalSize,1); + + return pos; +} + +int mc_EntityLedger::SetRow(int64_t pos, mc_EntityLedgerRow* row) +{ + int size; + unsigned char buf[64]; + + if(m_FileHan<=0) + { + return -1; + } + + if(lseek64(m_FileHan,pos,SEEK_SET) < 0) + { + return -1; + } + + size=row->m_ScriptSize; + + if((size>=0) && (size<=MC_ENT_MAX_SCRIPT_SIZE)) + { + if(mc_gState->m_Features->FollowOnIssues()) + { + size=mc_AllocSize(size,m_TotalSize,1); + if(write(m_FileHan,row,m_TotalSize) != m_TotalSize) + { + return -1; + } + } + else + { + size=mc_AllocSize(size,m_TotalSize,1); + memset(buf,0,64); + memcpy(buf+ 0,row->m_Key,MC_ENT_KEY_SIZE); + mc_PutLE(buf+32,&(row->m_KeyType),4); + mc_PutLE(buf+36,&(row->m_Block),4); + mc_PutLE(buf+40,&(row->m_Offset),4); + mc_PutLE(buf+44,&(row->m_Quantity),8); + mc_PutLE(buf+52,&(row->m_PrevPos),8); + mc_PutLE(buf+60,&(row->m_ScriptSize),4); + if(write(m_FileHan,buf,m_TotalSize) != m_TotalSize) + { + return -1; + } + } + if(size) + { + if(write(m_FileHan,row->m_Script,size) != size) + { + return -1; + } + } + } + else + { + return -1; + } + + return m_TotalSize+size; +} + +int mc_EntityLedger::SetZeroRow(mc_EntityLedgerRow* row) +{ + return SetRow(0,row); +} + + +int mc_AssetDB::Zero() +{ + m_Database = NULL; + m_Ledger = NULL; + m_MemPool = NULL; + m_Name[0]=0x00; + m_Block=-1; + m_PrevPos=-1; + m_Pos=0; + m_DBRowCount=0; + + return MC_ERR_NOERROR; +} + +int mc_AssetDB::Initialize(const char *name,int mode) +{ + int err,value_len; + int32_t adbBlock,aldBlock; + uint64_t adbLastPos,aldLastPos; + + unsigned char *ptr; + + mc_EntityDBRow adbRow; + mc_EntityLedgerRow aldRow; + + strcpy(m_Name,name); + + err=MC_ERR_NOERROR; + + m_Ledger=new mc_EntityLedger; + m_Database=new mc_EntityDB; + + m_Ledger->SetName(name); + m_Database->SetName(name); + + strcpy(m_Name,name); + + err=m_Database->Open(); + + if(err) + { + return err; + } + + adbBlock=-1; + adbLastPos=0; + + adbRow.Zero(); + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(err) + { + return err; + } + + if(ptr) + { + memcpy((char*)&adbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + adbBlock=adbRow.m_Block; + adbLastPos=adbRow.m_LedgerPos; + } + else + { + adbRow.Zero(); + adbRow.m_Block=(uint32_t)adbBlock; + adbRow.m_LedgerPos=adbLastPos; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,0); + if(err) + { + return err; + } + + err=m_Database->m_DB->Commit(0);//MC_OPT_DB_DATABASE_TRANSACTIONAL + if(err) + { + return err; + } + } + + m_MemPool=new mc_Buffer; + err=m_MemPool->Initialize(m_Ledger->m_KeySize,sizeof(mc_EntityLedgerRow),MC_BUF_MODE_MAP); + + m_Block=adbBlock; + m_PrevPos=adbLastPos; + m_Pos=adbLastPos+m_Ledger->m_TotalSize; + + aldBlock=-1; + aldLastPos=0; + if(m_Ledger->Open() <= 0) + { + return MC_ERR_DBOPEN_ERROR; + } + if(m_Ledger->GetRow(0,&aldRow) == 0) + { + aldBlock=aldRow.m_Block; + aldLastPos=aldRow.m_PrevPos; + m_PrevPos=adbLastPos; + if(m_Ledger->GetRow(m_PrevPos,&aldRow)) + { + return MC_ERR_CORRUPTED; + } + + m_Pos=m_PrevPos+mc_AllocSize(m_Ledger->m_TotalSize+aldRow.m_ScriptSize,m_Ledger->m_TotalSize,1); + } + else + { + aldRow.Zero(); + aldRow.m_Block=(uint32_t)aldBlock; + aldRow.m_PrevPos=aldLastPos; + m_Ledger->SetZeroRow(&aldRow); + } + if(m_Pos != m_Ledger->GetSize()) + { + m_Ledger->Close(); + return MC_ERR_CORRUPTED; + } + + m_Ledger->Close(); + + if(adbBlock != aldBlock) + { + return MC_ERR_CORRUPTED; + } + + if(adbLastPos != aldLastPos) + { + return MC_ERR_CORRUPTED; + } + + return MC_ERR_NOERROR; +} + +void mc_AssetDB::RemoveFiles() +{ + mc_EntityDBRow adbRow; + if(m_Database) + { + adbRow.Zero(); + m_Database->m_DB->Delete((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,0); + m_Database->m_DB->Commit(0); + } + if(m_Ledger) + { + m_Ledger->Close(); + remove(m_Ledger->m_FileName); + } +} + +int mc_AssetDB::Destroy() +{ + if(m_Database) + { + m_Database->Close(); + delete m_Database; + } + + if(m_Ledger) + { + delete m_Ledger; + } + + if(m_MemPool) + { + delete m_MemPool; + } + + Zero(); + + return MC_ERR_NOERROR; +} + +int mc_AssetDB::GetEntity(mc_EntityLedgerRow* row) +{ + int err,value_len,mprow; + int result; + mc_EntityDBRow adbRow; + + unsigned char *ptr; + + adbRow.Zero(); + memcpy(adbRow.m_Key,row->m_Key,MC_ENT_KEY_SIZE); + adbRow.m_KeyType=row->m_KeyType; + ptr=(unsigned char*)m_Database->m_DB->Read((char*)row+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(err) + { + return 0; + } + + if(ptr) + { + memcpy((char*)&adbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + + if(m_Ledger->Open() <= 0) + { + return 0; + } + + result=1; + if(m_Ledger->GetRow(adbRow.m_LedgerPos,row)) + { + result=0; + } + + row->m_ChainPos=adbRow.m_ChainPos; + m_Ledger->Close(); + + return result; + } + + mprow=m_MemPool->Seek((unsigned char*)row); + if(mprow>=0) + { + if( (((mc_EntityLedgerRow*)(m_MemPool->GetRow(mprow)))->m_KeyType & MC_ENT_KEYTYPE_MASK) != MC_ENT_KEYTYPE_TXID) + { + mprow--; + if(mprow>=0) + { + if( (((mc_EntityLedgerRow*)(m_MemPool->GetRow(mprow)))->m_KeyType & MC_ENT_KEYTYPE_MASK) != MC_ENT_KEYTYPE_TXID) + { + mprow--; + if(mprow>=0) + { + if( (((mc_EntityLedgerRow*)(m_MemPool->GetRow(mprow)))->m_KeyType & MC_ENT_KEYTYPE_MASK) != MC_ENT_KEYTYPE_TXID) + { + mprow--; + if(mprow>=0) + { + if( (((mc_EntityLedgerRow*)(m_MemPool->GetRow(mprow)))->m_KeyType & MC_ENT_KEYTYPE_MASK) != MC_ENT_KEYTYPE_TXID) + { + mprow=-1; + } + } + } + } + } + } + } + if(mprow<0) + { + return 0; + } + memcpy(row,m_MemPool->GetRow(mprow),sizeof(mc_EntityLedgerRow)); + return 1; + } + + + return 0; +} + +void mc_EntityDetails::Set(mc_EntityLedgerRow* row) +{ + uint32_t block,i; + int offset,script_size; + uint32_t value_offset; + size_t value_size; + unsigned char dname_buf[6]; + + Zero(); + + memcpy(&m_LedgerRow,row,sizeof(mc_EntityLedgerRow)); + + block=row->m_Block; + offset=row->m_Offset; + script_size=row->m_ScriptSize; + + m_Flags=0; + + if(offset) + { + mc_PutLE(m_Ref,&block,4); + mc_PutLE(m_Ref+4,&offset,4); + for(i=0;im_Key+MC_ENT_KEY_SIZE-1-i); + } + m_Flags |= MC_ENT_FLAG_OFFSET_IS_SET; + } + + if(script_size) + { + value_offset=mc_FindSpecialParamInDetailsScript(m_LedgerRow.m_Script,m_LedgerRow.m_ScriptSize,MC_ENT_SPRM_NAME,&value_size); + if(value_offset == m_LedgerRow.m_ScriptSize) + { + strcpy((char*)dname_buf+1,"name"); + dname_buf[0]=0xff; + value_offset=mc_FindNamedParamInDetailsScript(m_LedgerRow.m_Script,m_LedgerRow.m_ScriptSize,(char*)dname_buf,&value_size); + } + if(mc_gState->m_Features->Streams()) + { + if(value_offset < m_LedgerRow.m_ScriptSize) + { + if(value_size == 2) + { + if((char)m_LedgerRow.m_Script[value_offset] == '*') + { + value_offset=m_LedgerRow.m_ScriptSize; + value_size=0; + } + } + } + } + if(value_offset < m_LedgerRow.m_ScriptSize) + { + memcpy(m_Name,m_LedgerRow.m_Script+value_offset,value_size); + mc_StringLowerCase(m_Name,value_size); + m_Flags |= MC_ENT_FLAG_NAME_IS_SET; + } + } + + mc_ZeroABRaw(m_FullRef); + if(mc_gState->m_Features->ShortTxIDAsAssetRef()) + { + memcpy(m_FullRef+MC_AST_SHORT_TXID_OFFSET,m_LedgerRow.m_Key+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + mc_SetABRefType(m_FullRef,MC_AST_ASSET_REF_TYPE_SHORT_TXID); + } + else + { + memcpy(m_FullRef,m_Ref,MC_AST_ASSET_REF_SIZE); + mc_SetABRefType(m_FullRef,MC_AST_ASSET_REF_TYPE_REF); + } +} + +int mc_AssetDB::InsertStream(const void* txid, int offset, int entity_type, const void *script,size_t script_size, const void* special_script, size_t special_script_size,int update_mempool) +{ + mc_EntityLedgerRow aldRow; + mc_EntityDetails details; + + int pass; + uint32_t value_offset; + size_t value_size; + char stream_name[MC_ENT_MAX_NAME_SIZE+1]; + + aldRow.Zero(); + memcpy(aldRow.m_Key,txid,MC_ENT_KEY_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_TXID; + aldRow.m_Block=m_Block+1; + aldRow.m_Offset=offset; + if(offset<0) + { + aldRow.m_Offset=-(m_MemPool->GetCount()+1); + } + aldRow.m_Quantity=0; + aldRow.m_EntityType=entity_type; + aldRow.m_FirstPos=-1; + aldRow.m_LastPos=0; + aldRow.m_ChainPos=-1; + aldRow.m_PrevPos=-1; + + mc_Script *lpDetails; + lpDetails=new mc_Script; + lpDetails->AddElement(); + + if(special_script_size) + { + lpDetails->SetData((const unsigned char*)special_script,special_script_size); + } + + if(script) + { + value_offset=mc_FindSpecialParamInDetailsScript((unsigned char*)script,script_size,MC_ENT_SPRM_NAME,&value_size); + if(value_offset != script_size) + { + if(value_size) + { + if(*((unsigned char*)script+value_offset+value_size-1)) + { + memcpy(stream_name,(unsigned char*)script+value_offset,value_size); + stream_name[value_size]=0x00; + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_NAME,(unsigned char*)stream_name,strlen(stream_name)+1); + } + } + } + } + + if(lpDetails->m_Size) + { + memcpy(aldRow.m_Script,lpDetails->GetData(0,NULL),lpDetails->m_Size); + } + + aldRow.m_ScriptSize=script_size+lpDetails->m_Size; + if(script_size) + { + memcpy(aldRow.m_Script+lpDetails->m_Size,script,script_size); + } + + delete lpDetails; + + details.Set(&aldRow); + + for(pass=0;pass<1+update_mempool;pass++) + { + memset(aldRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(aldRow.m_Key,details.m_LedgerRow.m_Key,MC_ENT_KEY_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_TXID; + + if(pass) + { + m_MemPool->Add((unsigned char*)&aldRow,(unsigned char*)&aldRow+m_Ledger->m_ValueOffset); + } + else + { + if(GetEntity(&aldRow)) + { + return MC_ERR_FOUND; + } + } + + if(details.m_Flags & MC_ENT_FLAG_OFFSET_IS_SET) + { + memset(aldRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(aldRow.m_Key,details.m_Ref,MC_ENT_REF_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_REF; + + if(pass) + { + m_MemPool->Add((unsigned char*)&aldRow,(unsigned char*)&aldRow+m_Ledger->m_ValueOffset); + } + else + { + if(GetEntity(&aldRow)) + { + return MC_ERR_FOUND; + } + } + } + + if(details.m_Flags & MC_ENT_FLAG_NAME_IS_SET) + { + memset(aldRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(aldRow.m_Key,details.m_Name,MC_ENT_MAX_NAME_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_NAME; + + if(pass) + { + m_MemPool->Add((unsigned char*)&aldRow,(unsigned char*)&aldRow+m_Ledger->m_ValueOffset); + } + else + { + if(GetEntity(&aldRow)) + { + return MC_ERR_FOUND; + } + } + } + + memset(aldRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(aldRow.m_Key,details.m_LedgerRow.m_Key+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_SHORT_TXID; + + if(pass) + { + m_MemPool->Add((unsigned char*)&aldRow,(unsigned char*)&aldRow+m_Ledger->m_ValueOffset); + } + else + { + if(GetEntity(&aldRow)) + { + return MC_ERR_FOUND; + } + } + + + } + + return MC_ERR_NOERROR; +} + + +int mc_AssetDB::InsertAsset(const void* txid, int offset, uint64_t quantity, const char *name, int multiple, const void* script, size_t script_size, const void* special_script, size_t special_script_size,int update_mempool) +{ + mc_EntityLedgerRow aldRow; + mc_EntityDetails details; + + unsigned char dname_buf[6]; + int pass; + uint32_t value_offset; + size_t value_size; + int add_param; + + aldRow.Zero(); + memcpy(aldRow.m_Key,txid,MC_ENT_KEY_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_TXID; + aldRow.m_Block=m_Block+1; + aldRow.m_Offset=offset; + if(offset<0) + { + aldRow.m_Offset=-(m_MemPool->GetCount()+1); + } + aldRow.m_Quantity=quantity; + aldRow.m_EntityType=MC_ENT_TYPE_ASSET; + aldRow.m_FirstPos=-(m_MemPool->GetCount()+1);//-1; // Unconfirmed issue, from 10007 we can create followons for them, so we should differentiate + aldRow.m_LastPos=0; + aldRow.m_ChainPos=-1; + aldRow.m_PrevPos=-1; + + mc_Script *lpDetails; + lpDetails=new mc_Script; + lpDetails->AddElement(); + + if(special_script_size) + { + lpDetails->SetData((const unsigned char*)special_script,special_script_size); + } + + add_param=true; + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + if(script) + { + if(mc_FindSpecialParamInDetailsScript((unsigned char*)script,script_size,MC_ENT_SPRM_ASSET_MULTIPLE,&value_size) != script_size) + { + add_param=false; + } + } + } + + if(add_param) + { + if(mc_gState->m_Features->SpecialParamsInDetailsScript()) + { + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_ASSET_MULTIPLE,(unsigned char*)&multiple,sizeof(multiple)); + } + else + { + if(multiple != 1) + { + lpDetails->SetParamValue("multiple",strlen("multiple"),(unsigned char*)&multiple,sizeof(multiple)); + } + } + } + + add_param=true; + if(script) + { + value_offset=mc_FindSpecialParamInDetailsScript((unsigned char*)script,script_size,MC_ENT_SPRM_NAME,&value_size); + if(value_offset != script_size) + { + if(value_size) + { + if(*((unsigned char*)script+value_offset+value_size-1) == 0) + { + add_param=false; + } + } + } + } + + if(add_param) + { + if(name && (strlen(name) > 0)) + { + if(mc_gState->m_Features->SpecialParamsInDetailsScript()) + { + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_NAME,(unsigned char*)name,strlen(name)+1); + } + else + { + lpDetails->SetParamValue("name",strlen("name"),(unsigned char*)name,strlen(name)); + strcpy((char*)dname_buf+1,"name"); + dname_buf[0]=0xff; + lpDetails->SetParamValue((char*)dname_buf,5,(unsigned char*)name,strlen(name)+1); + } + } + } + + if(lpDetails->m_Size) + { + memcpy(aldRow.m_Script,lpDetails->GetData(0,NULL),lpDetails->m_Size); + } + + aldRow.m_ScriptSize=script_size+lpDetails->m_Size; + if(script) + { + memcpy(aldRow.m_Script+lpDetails->m_Size,script,script_size); + } + + delete lpDetails; + + details.Set(&aldRow); + + for(pass=0;pass<1+update_mempool;pass++) + { + memset(aldRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(aldRow.m_Key,details.m_LedgerRow.m_Key,MC_ENT_KEY_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_TXID; + + if(pass) + { + m_MemPool->Add((unsigned char*)&aldRow,(unsigned char*)&aldRow+m_Ledger->m_ValueOffset); + } + else + { + if(GetEntity(&aldRow)) + { + return MC_ERR_FOUND; + } + } + + + if(details.m_Flags & MC_ENT_FLAG_OFFSET_IS_SET) + { + memset(aldRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(aldRow.m_Key,details.m_Ref,MC_ENT_REF_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_REF; + + if(pass) + { + m_MemPool->Add((unsigned char*)&aldRow,(unsigned char*)&aldRow+m_Ledger->m_ValueOffset); + } + else + { + if(GetEntity(&aldRow)) + { + return MC_ERR_FOUND; + } + } + } + + if(details.m_Flags & MC_ENT_FLAG_NAME_IS_SET) + { + memset(aldRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(aldRow.m_Key,details.m_Name,MC_ENT_MAX_NAME_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_NAME; + + if(pass) + { + m_MemPool->Add((unsigned char*)&aldRow,(unsigned char*)&aldRow+m_Ledger->m_ValueOffset); + } + else + { + if(GetEntity(&aldRow)) + { + return MC_ERR_FOUND; + } + } + } + + memset(aldRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(aldRow.m_Key,details.m_LedgerRow.m_Key+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_SHORT_TXID; + + if(pass) + { + m_MemPool->Add((unsigned char*)&aldRow,(unsigned char*)&aldRow+m_Ledger->m_ValueOffset); + } + else + { + if(GetEntity(&aldRow)) + { + return MC_ERR_FOUND; + } + } + + } + + return MC_ERR_NOERROR; +} + +int mc_AssetDB::InsertAssetFollowOn(const void* txid, int offset, uint64_t quantity, const void* script, size_t script_size, const void* special_script, size_t special_script_size, const void* original_txid, int update_mempool) +{ + mc_EntityLedgerRow aldRow; + + int pass,i; + int64_t size,first_pos,last_pos,tot_pos; + uint64_t value_offset; + size_t value_size; + int64_t total; + uint32_t entity_type; + + aldRow.Zero(); + memcpy(aldRow.m_Key,original_txid,MC_ENT_KEY_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_TXID; + if(!GetEntity(&aldRow)) + { + return MC_ERR_NOT_FOUND; + } + + if(mc_gState->m_Features->ShortTxIDAsAssetRef() == 0) + { + if(aldRow.m_PrevPos < 0) // Unconfirmed genesis for protocol < 10007 + { + return MC_ERR_NOT_FOUND; + } + } + + value_offset=mc_FindSpecialParamInDetailsScript(aldRow.m_Script,aldRow.m_ScriptSize,MC_ENT_SPRM_FOLLOW_ONS,&value_size); + if(value_offset == aldRow.m_ScriptSize) + { + return MC_ERR_NOT_ALLOWED; + } + if( (value_size==0) || (value_size > 4)) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + if(mc_GetLE(aldRow.m_Script+value_offset,value_size) == 0) + { + return MC_ERR_NOT_ALLOWED; + } + + total=GetTotalQuantity(&aldRow); + if((int64_t)(total+quantity)<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + + first_pos=aldRow.m_FirstPos; + last_pos=aldRow.m_ChainPos; + entity_type=aldRow.m_EntityType; + + tot_pos=m_Pos; + + for(i=0;iGetCount();i++) + { + if( (((mc_EntityLedgerRow*)(m_MemPool->GetRow(i)))->m_KeyType & MC_ENT_KEYTYPE_MASK) == MC_ENT_KEYTYPE_TXID) + { + size=m_Ledger->m_TotalSize+mc_AllocSize(((mc_EntityLedgerRow*)(m_MemPool->GetRow(i)))->m_ScriptSize,m_Ledger->m_TotalSize,1); + if( ((mc_EntityLedgerRow*)(m_MemPool->GetRow(i)))->m_KeyType == (MC_ENT_KEYTYPE_FOLLOW_ON | MC_ENT_KEYTYPE_TXID) ) + { + if(((mc_EntityLedgerRow*)(m_MemPool->GetRow(i)))->m_FirstPos == first_pos) + { + last_pos=tot_pos; + } + } + tot_pos+=size; + } + } + + + aldRow.Zero(); + memcpy(aldRow.m_Key,txid,MC_ENT_KEY_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_FOLLOW_ON | MC_ENT_KEYTYPE_TXID; + aldRow.m_Block=m_Block+1; + aldRow.m_Offset=offset; + if(offset<0) + { + aldRow.m_Offset=-(m_MemPool->GetCount()+1); + } + aldRow.m_Quantity=quantity; + aldRow.m_EntityType=entity_type; + aldRow.m_FirstPos=first_pos; + aldRow.m_LastPos=last_pos; + aldRow.m_ChainPos=-1; + aldRow.m_PrevPos=-1; + + mc_Script *lpDetails; + lpDetails=new mc_Script; + lpDetails->AddElement(); + + if(special_script_size) + { + lpDetails->SetData((const unsigned char*)special_script,special_script_size); + } + + if(lpDetails->m_Size) + { + memcpy(aldRow.m_Script,lpDetails->GetData(0,NULL),lpDetails->m_Size); + } + + aldRow.m_ScriptSize=script_size+lpDetails->m_Size; + if(script) + { + memcpy(aldRow.m_Script+lpDetails->m_Size,script,script_size); + } + + delete lpDetails; + + for(pass=0;pass<1+update_mempool;pass++) + { + if(pass) + { + m_MemPool->Add((unsigned char*)&aldRow,(unsigned char*)&aldRow+m_Ledger->m_ValueOffset); + } + else + { + if(GetEntity(&aldRow)) + { + return MC_ERR_FOUND; + } + } + } + + return MC_ERR_NOERROR; + +} + +int mc_AssetDB::Commit() +{ + int i,size,err,value_len; + + mc_EntityDBRow adbRow; + mc_EntityLedgerRow aldRow; + mc_EntityLedgerRow aldGenesisRow; + mc_EntityDetails details; + unsigned char *ptr; + + err=MC_ERR_NOERROR; + + if(m_Ledger->Open() <= 0) + { + return MC_ERR_DBOPEN_ERROR; + } + + if(m_MemPool->GetCount()) + { + if(err == MC_ERR_NOERROR) + { + size=0; + for(i=0;iGetCount();i++) + { + if(err == MC_ERR_NOERROR) + { + memcpy(&aldRow,m_MemPool->GetRow(i),sizeof(mc_EntityLedgerRow)); + aldGenesisRow.Zero(); + if( (aldRow.m_KeyType & MC_ENT_KEYTYPE_MASK) == MC_ENT_KEYTYPE_TXID) + { + m_Pos+=size; + aldRow.m_PrevPos=m_PrevPos; + if(aldRow.m_KeyType == (MC_ENT_KEYTYPE_FOLLOW_ON | MC_ENT_KEYTYPE_TXID)) + { + if(aldRow.m_FirstPos < 0) + { + memcpy(&aldGenesisRow,m_MemPool->GetRow(-aldRow.m_FirstPos-1),sizeof(mc_EntityLedgerRow)); + aldRow.m_LastPos=aldGenesisRow.m_ChainPos; + aldGenesisRow.m_ChainPos=m_Pos; + memcpy(m_MemPool->GetRow(-aldRow.m_FirstPos-1),&aldGenesisRow,sizeof(mc_EntityLedgerRow)); + aldRow.m_FirstPos=aldGenesisRow.m_FirstPos; + } + else + { + err=m_Ledger->GetRow(aldRow.m_FirstPos,&aldGenesisRow); + } + } + else + { + if(aldRow.m_FirstPos < 0) + { + aldRow.m_FirstPos=m_Pos; + } + } + if(aldRow.m_ChainPos < 0) + { + aldRow.m_ChainPos=m_Pos; + } + memcpy(m_MemPool->GetRow(i),&aldRow,sizeof(mc_EntityLedgerRow)); + m_PrevPos=m_Pos; + if(aldRow.m_Offset < 0) + { + err=MC_ERR_INTERNAL_ERROR; + } + else + { + size=m_Ledger->SetRow(m_PrevPos,&aldRow); + if(size<0) + { + err=MC_ERR_INTERNAL_ERROR; + } + } + } + } + + if(err == MC_ERR_NOERROR) + { + adbRow.Zero(); + memcpy(adbRow.m_Key,aldRow.m_Key,MC_ENT_KEY_SIZE); + adbRow.m_KeyType=aldRow.m_KeyType; + adbRow.m_EntityType=aldRow.m_EntityType; + adbRow.m_Block=aldRow.m_Block; + adbRow.m_LedgerPos=m_Pos; + adbRow.m_ChainPos=m_Pos; + + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + + if(err == MC_ERR_NOERROR) + { + if(aldGenesisRow.m_KeyType) + { + details.Set(&aldGenesisRow); + adbRow.Zero(); + memcpy(adbRow.m_Key,aldGenesisRow.m_Key,MC_ENT_KEY_SIZE); + adbRow.m_KeyType=aldGenesisRow.m_KeyType; + adbRow.m_EntityType=aldGenesisRow.m_EntityType; + adbRow.m_Block=aldGenesisRow.m_Block; + adbRow.m_LedgerPos=aldGenesisRow.m_FirstPos; + adbRow.m_ChainPos=m_Pos; + + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_LedgerRow.m_Key+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_SHORT_TXID; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + + if(details.m_Flags & MC_ENT_FLAG_OFFSET_IS_SET) + { + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_Ref,MC_ENT_REF_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_REF; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + if(details.m_Flags & MC_ENT_FLAG_NAME_IS_SET) + { + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_Name,MC_ENT_MAX_NAME_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_NAME; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + } +/* + if(aldRow.m_KeyType == (MC_ENT_KEYTYPE_FOLLOW_ON | MC_ENT_KEYTYPE_TXID)) + { + err=m_Ledger->GetRow(aldRow.m_FirstPos,&aldRow); + details.Set(&aldRow); + + adbRow.Zero(); + + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_LedgerRow.m_Key,MC_ENT_KEY_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_TXID; + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + + if(ptr) + { + memcpy((char*)&adbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + adbRow.m_ChainPos=m_Pos; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(details.m_Flags & MC_ENT_FLAG_OFFSET_IS_SET) + { + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_Ref,MC_ENT_REF_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_REF; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + if(details.m_Flags & MC_ENT_FLAG_NAME_IS_SET) + { + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_Name,MC_ENT_MAX_NAME_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_NAME; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + } + else + { + err=MC_ERR_INTERNAL_ERROR; + } + } + */ + } + } + } + m_Pos+=size; + } + } + + if(err == MC_ERR_NOERROR) + { + m_Ledger->GetRow(0,&aldRow); + aldRow.m_Block=m_Block+1; + aldRow.m_PrevPos=m_PrevPos; + m_Ledger->SetZeroRow(&aldRow); + } + + m_Ledger->Close(); + + if(err == MC_ERR_NOERROR) + { + adbRow.Zero(); + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(ptr) + { + memcpy((char*)&adbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + } + + + adbRow.m_Block=m_Block+1; + adbRow.m_LedgerPos=m_PrevPos; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + + if(err == MC_ERR_NOERROR) + { + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + if(err) + { + RollBack(m_Block); + } + else + { + m_MemPool->Clear(); + m_Block++; + } + + return err; +} + +int mc_AssetDB::RollBack() +{ + return RollBack(m_Block-1); +} + +int mc_AssetDB::RollBack(int block) +{ + int err; + int take_it,value_len; + int64_t this_pos,prev_pos,new_chain_pos; + mc_EntityDBRow adbRow; + mc_EntityLedgerRow aldRow; + mc_EntityDetails details; + unsigned char *ptr; + + err=MC_ERR_NOERROR; + + + ClearMemPool(); + + if(m_Ledger->Open() <= 0) + { + return MC_ERR_DBOPEN_ERROR; + } + + this_pos=m_PrevPos; + take_it=1; + + while(take_it && (this_pos>0)) + { + m_Ledger->GetRow(this_pos,&aldRow); + prev_pos=aldRow.m_PrevPos; + details.Set(&aldRow); + + if(details.m_LedgerRow.m_Block > block) + { + adbRow.Zero(); + memcpy(adbRow.m_Key,aldRow.m_Key,MC_ENT_KEY_SIZE); + adbRow.m_KeyType=aldRow.m_KeyType; + err=m_Database->m_DB->Delete((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + + adbRow.Zero(); + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,aldRow.m_Key+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_SHORT_TXID; + err=m_Database->m_DB->Delete((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + + if((adbRow.m_KeyType & MC_ENT_KEYTYPE_FOLLOW_ON) == 0) + { + if( (details.m_Flags & MC_ENT_FLAG_OFFSET_IS_SET)) + { + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_Ref,MC_ENT_REF_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_REF; + err=m_Database->m_DB->Delete((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + + if(details.m_Flags & MC_ENT_FLAG_NAME_IS_SET) + { + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_Name,MC_ENT_MAX_NAME_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_NAME; + err=m_Database->m_DB->Delete((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + } + + if(err == MC_ERR_NOERROR) + { + if(aldRow.m_KeyType == (MC_ENT_KEYTYPE_FOLLOW_ON | MC_ENT_KEYTYPE_TXID)) + { + new_chain_pos=aldRow.m_LastPos; + err=m_Ledger->GetRow(aldRow.m_FirstPos,&aldRow); + details.Set(&aldRow); + + adbRow.Zero(); + + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_LedgerRow.m_Key,MC_ENT_KEY_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_TXID; + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + + if(ptr) + { + memcpy((char*)&adbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + adbRow.m_ChainPos=new_chain_pos; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_LedgerRow.m_Key+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_SHORT_TXID; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(details.m_Flags & MC_ENT_FLAG_OFFSET_IS_SET) + { + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_Ref,MC_ENT_REF_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_REF; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + if(details.m_Flags & MC_ENT_FLAG_NAME_IS_SET) + { + memset(adbRow.m_Key,0,MC_ENT_KEY_SIZE); + memcpy(adbRow.m_Key,details.m_Name,MC_ENT_MAX_NAME_SIZE); + adbRow.m_KeyType=MC_ENT_KEYTYPE_NAME; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + } + else + { + err=MC_ERR_INTERNAL_ERROR; + } + } + } + } + else + { + take_it=0; + } + if(err) + { + take_it=0; + } + if(take_it) + { + this_pos=prev_pos; + } + } + + m_PrevPos=this_pos; + + if(err == MC_ERR_NOERROR) + { + m_Ledger->GetRow(0,&aldRow); + aldRow.m_Block=block; + aldRow.m_PrevPos=m_PrevPos; + m_Ledger->SetZeroRow(&aldRow); + } + + + if(err == MC_ERR_NOERROR) + { + adbRow.Zero(); + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(ptr) + { + memcpy((char*)&adbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + } + + adbRow.m_Block=block; + adbRow.m_LedgerPos=m_PrevPos; + err=m_Database->m_DB->Write((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&adbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + + if(err == MC_ERR_NOERROR) + { + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + + if(err == MC_ERR_NOERROR) + { + m_Block=block; + m_Ledger->GetRow(m_PrevPos,&aldRow); + m_Pos=m_PrevPos+mc_AllocSize(m_Ledger->m_TotalSize+aldRow.m_ScriptSize,m_Ledger->m_TotalSize,1); + } + + m_Ledger->Close(); + return err; +} + +int mc_AssetDB::ClearMemPool() +{ + mc_EntityLedgerRow aldRow; + + if(m_MemPool) + { + if(m_MemPool->GetCount()) + { + m_MemPool->Clear(); + if(m_Ledger->Open() <= 0) + { + return MC_ERR_DBOPEN_ERROR; + } + + if(m_Ledger->GetRow(0,&aldRow)) + { + return MC_ERR_CORRUPTED; + } + + m_PrevPos=aldRow.m_PrevPos; + if(m_Ledger->GetRow(m_PrevPos,&aldRow)) + { + return MC_ERR_CORRUPTED; + } + + m_Pos=m_PrevPos+mc_AllocSize(m_Ledger->m_TotalSize+aldRow.m_ScriptSize,m_Ledger->m_TotalSize,1); + + m_Ledger->Close(); + } + } + + return MC_ERR_NOERROR; +} + + +int mc_AssetDB::FindEntityByTxID(mc_EntityDetails *entity,const unsigned char* txid) +{ + mc_EntityLedgerRow aldRow; + + entity->Zero(); + aldRow.Zero(); + + memcpy(aldRow.m_Key,txid,MC_ENT_KEY_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_TXID; + + if(GetEntity(&aldRow)) + { + entity->Set(&aldRow); + return 1; + } + + aldRow.m_KeyType=MC_ENT_KEYTYPE_TXID | MC_ENT_KEYTYPE_FOLLOW_ON; + + if(GetEntity(&aldRow)) + { + entity->Set(&aldRow); + return 1; + } + + return 0; +} + +int mc_AssetDB::FindEntityByShortTxID (mc_EntityDetails *entity, const unsigned char* short_txid) +{ + mc_EntityLedgerRow aldRow; + + entity->Zero(); + aldRow.Zero(); + + memcpy(aldRow.m_Key,short_txid,MC_AST_SHORT_TXID_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_SHORT_TXID; + + if(GetEntity(&aldRow)) + { + entity->Set(&aldRow); + return 1; + } + + return 0; +} + + +int mc_AssetDB::FindEntityByRef (mc_EntityDetails *entity,const unsigned char* asset_ref) +{ + mc_EntityLedgerRow aldRow; + + entity->Zero(); + aldRow.Zero(); + + memcpy(aldRow.m_Key,asset_ref,MC_ENT_REF_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_REF; + + if(GetEntity(&aldRow)) + { + entity->Set(&aldRow); + return 1; + } + + return 0; +} + +int mc_AssetDB::FindEntityByFullRef (mc_EntityDetails *entity, unsigned char* full_ref) +{ + entity->Zero(); + switch(mc_GetABRefType(full_ref)) + { + case MC_AST_ASSET_REF_TYPE_REF: + return FindEntityByRef(entity,full_ref); + case MC_AST_ASSET_REF_TYPE_SHORT_TXID: + return FindEntityByShortTxID(entity,full_ref+MC_AST_SHORT_TXID_OFFSET); + case MC_AST_ASSET_REF_TYPE_TXID: + return FindEntityByTxID(entity,full_ref); + } + return false; +} + + +int mc_AssetDB::FindEntityByName(mc_EntityDetails *entity,const char* name) +{ + mc_EntityLedgerRow aldRow; + + entity->Zero(); + aldRow.Zero(); + + memcpy(aldRow.m_Key,name,strlen(name)); + mc_StringLowerCase((char*)(aldRow.m_Key),MC_ENT_MAX_NAME_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_NAME; + + if(GetEntity(&aldRow)) + { + entity->Set(&aldRow); + return 1; + } + + return 0; +} + +int mc_AssetDB::FindEntityByFollowOn(mc_EntityDetails *entity,const unsigned char* txid) +{ + mc_EntityLedgerRow aldRow; + + entity->Zero(); + aldRow.Zero(); + + memcpy(aldRow.m_Key,txid,MC_ENT_KEY_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_FOLLOW_ON | MC_ENT_KEYTYPE_TXID; + + if(GetEntity(&aldRow)) + { + m_Ledger->Open(); + + if(aldRow.m_FirstPos < 0) + { + memcpy(&aldRow,m_MemPool->GetRow(-aldRow.m_FirstPos-1),sizeof(mc_EntityLedgerRow)); + } + else + { + m_Ledger->GetRow(aldRow.m_FirstPos,&aldRow); + } + m_Ledger->Close(); + entity->Set(&aldRow); + return 1; + } + + return 0; +} + + + +const char* mc_EntityDetails::GetName() +{ + uint32_t value_offset; + size_t value_size; + unsigned char dname_buf[6]; + + if(m_LedgerRow.m_ScriptSize) + { + value_offset=mc_FindSpecialParamInDetailsScript(m_LedgerRow.m_Script,m_LedgerRow.m_ScriptSize,MC_ENT_SPRM_NAME,&value_size); + if(value_offset == m_LedgerRow.m_ScriptSize) + { + strcpy((char*)dname_buf+1,"name"); + dname_buf[0]=0xff; + value_offset=mc_FindNamedParamInDetailsScript(m_LedgerRow.m_Script,m_LedgerRow.m_ScriptSize,(char*)dname_buf,&value_size); + } + if(mc_gState->m_Features->Streams()) + { + if(value_offset < m_LedgerRow.m_ScriptSize) + { + if(value_size == 2) + { + if((char)m_LedgerRow.m_Script[value_offset] == '*') + { + value_offset=m_LedgerRow.m_ScriptSize; + value_size=0; + } + } + } + } + if(value_offset < m_LedgerRow.m_ScriptSize) + { + return (char*)(m_LedgerRow.m_Script+value_offset); + } + } + + return m_Name; +} + +const unsigned char* mc_EntityDetails::GetTxID() +{ + return m_LedgerRow.m_Key; +} + +const unsigned char* mc_EntityDetails::GetRef() +{ + return m_Ref; +} + +int mc_EntityDetails::IsUnconfirmedGenesis() +{ + return ((int)mc_GetLE(m_Ref+4,4)<0) ? 1 : 0; +} + +const unsigned char* mc_EntityDetails::GetFullRef() +{ + return m_FullRef; +} + +const unsigned char* mc_EntityDetails::GetShortRef() +{ + if(mc_gState->m_Features->ShortTxIDAsAssetRef()) + { + return GetTxID()+MC_AST_SHORT_TXID_OFFSET; + } + return m_Ref; +} + +const unsigned char* mc_EntityDetails::GetScript() +{ + return m_LedgerRow.m_Script; +} + +int mc_EntityDetails::GetAssetMultiple() +{ + int multiple; + size_t size; + void* ptr; + + multiple=1; + + ptr=NULL; + if(mc_gState->m_Features->SpecialParamsInDetailsScript()) + { + ptr=(void*)GetSpecialParam(MC_ENT_SPRM_ASSET_MULTIPLE,&size); + } + else + { + ptr=(void*)GetParam("multiple",&size); + } + + if(ptr) + { + if(size==sizeof(multiple)) + { + multiple=(int)mc_GetLE(ptr,size); + } + } + + if(multiple <= 0) + { + multiple=1; + } + + return multiple; +} + +int mc_EntityDetails::IsFollowOn() +{ + if(m_LedgerRow.m_KeyType & MC_ENT_KEYTYPE_FOLLOW_ON) + { + return 1; + } + return 0; +} + +int mc_EntityDetails::AllowedFollowOns() +{ + unsigned char *ptr; + size_t bytes; + ptr=(unsigned char *)GetSpecialParam(MC_ENT_SPRM_FOLLOW_ONS,&bytes); + if(ptr) + { + if((bytes>0) && (bytes<=4)) + { + return (int)mc_GetLE(ptr,bytes); + } + } + return 0; +} + +int mc_EntityDetails::AnyoneCanWrite() +{ + unsigned char *ptr; + size_t bytes; + ptr=(unsigned char *)GetSpecialParam(MC_ENT_SPRM_ANYONE_CAN_WRITE,&bytes); + if(ptr) + { + if((bytes>0) && (bytes<=4)) + { + return (int)mc_GetLE(ptr,bytes); + } + } + return 0; +} + + +uint64_t mc_EntityDetails::GetQuantity() +{ + return m_LedgerRow.m_Quantity; +} + +uint32_t mc_EntityDetails::GetEntityType() +{ + return m_LedgerRow.m_EntityType; +} + +int32_t mc_EntityDetails::NextParam(uint32_t offset,uint32_t* param_value_start,size_t *bytes) +{ + int32_t new_offset; + + new_offset=(int32_t)mc_GetParamFromDetailsScript(m_LedgerRow.m_Script,m_LedgerRow.m_ScriptSize,offset,param_value_start,bytes); + if(new_offset == (int32_t)m_LedgerRow.m_ScriptSize) + { + new_offset = -1; + } + + return new_offset; +} + +const void* mc_EntityDetails::GetSpecialParam(uint32_t param,size_t* bytes) +{ + uint32_t offset; + offset=mc_FindSpecialParamInDetailsScript(m_LedgerRow.m_Script,m_LedgerRow.m_ScriptSize,param,bytes); + if(offset == m_LedgerRow.m_ScriptSize) + { + return NULL; + } + return m_LedgerRow.m_Script+offset; +} + +const void* mc_EntityDetails::GetParam(const char *param,size_t* bytes) +{ + uint32_t offset; + offset=mc_FindNamedParamInDetailsScript(m_LedgerRow.m_Script,m_LedgerRow.m_ScriptSize,param,bytes); + if(offset == m_LedgerRow.m_ScriptSize) + { + return NULL; + } + return m_LedgerRow.m_Script+offset; +} + +void mc_AssetDB::Dump() +{ + mc_EntityDBRow adbRow; + mc_EntityLedgerRow aldRow; + unsigned char *ptr; + int dbvalue_len,err,i; + int64_t pos,total; + int size,row_size; + + + + printf("\nDB\n"); + adbRow.Zero(); + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&dbvalue_len,MC_OPT_DB_DATABASE_SEEK_ON_READ,&err); + if(err) + { + return; + } + + row_size=m_Database->m_TotalSize; + while(row_size > 64) + { + row_size /= 2; + } + if(ptr) + { + memcpy((unsigned char*)&adbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueOffset); + while(ptr) + { + mc_MemoryDumpCharSize((unsigned char*)&adbRow,0,m_Database->m_TotalSize,row_size); + ptr=(unsigned char*)m_Database->m_DB->MoveNext(&err); + if(ptr) + { + memcpy((unsigned char*)&adbRow,ptr,m_Database->m_TotalSize); + } + } + } + + printf("Ledger\n"); + + m_Ledger->Open(); + + row_size=m_Ledger->m_TotalSize; + while(row_size > 64) + { + row_size /= 2; + } + + total=m_Ledger->GetSize(); + pos=0; + + while(posGetRow(pos,&aldRow); + + size=mc_AllocSize(m_Ledger->m_TotalSize+aldRow.m_ScriptSize,m_Ledger->m_TotalSize,1); + mc_DumpSize("",&aldRow,size,row_size); + pos+=size; + } + + m_Ledger->Close(); + + printf("MemPool\n"); + for(i=0;iGetCount();i++) + { + memcpy(&aldRow,m_MemPool->GetRow(i),sizeof(mc_EntityLedgerRow)); + size=mc_AllocSize(m_Ledger->m_TotalSize+aldRow.m_ScriptSize,m_Ledger->m_TotalSize,1); + mc_DumpSize("",&aldRow,size,row_size); + pos+=size; + } + +} + +mc_Buffer *mc_AssetDB::GetEntityList(mc_Buffer *old_result,const void* txid,uint32_t entity_type) +{ + mc_EntityDBRow adbRow; + mc_EntityLedgerRow aldRow; + + unsigned char *ptr; + int dbvalue_len,err,i; + mc_Buffer *result; + + if(old_result) + { + result=old_result; + } + else + { + result=new mc_Buffer; + result->Initialize(MC_ENT_KEY_SIZE,MC_ENT_KEY_SIZE,MC_BUF_MODE_DEFAULT); + } + + if(txid) + { + result->Add(txid,(unsigned char *)txid+MC_ENT_KEY_SIZE); + return result; + } + + adbRow.Zero(); + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&adbRow+m_Database->m_KeyOffset,m_Database->m_ValueOffset,&dbvalue_len,MC_OPT_DB_DATABASE_SEEK_ON_READ,&err); + if(err) + { + delete result; + return NULL; + } + + if(ptr) + { + memcpy((unsigned char*)&adbRow+m_Database->m_ValueOffset,ptr,dbvalue_len); + while(ptr) + { + if(mc_gState->m_Features->FollowOnIssues() == 0) + { + adbRow.m_EntityType=MC_ENT_TYPE_ASSET; + } + if(adbRow.m_KeyType == MC_ENT_KEYTYPE_TXID) + { + if( (entity_type == 0) || (adbRow.m_EntityType == entity_type) ) + { + result->Add(adbRow.m_Key,NULL); + } + } + ptr=(unsigned char*)m_Database->m_DB->MoveNext(&err); + if(ptr) + { + memcpy((unsigned char*)&adbRow,ptr,m_Database->m_TotalSize); + } + } + } + + for(i=0;iGetCount();i++) + { + memcpy(&aldRow,m_MemPool->GetRow(i),sizeof(mc_EntityLedgerRow)); + if(aldRow.m_KeyType == MC_ENT_KEYTYPE_TXID) + { + if( (entity_type == 0) || (aldRow.m_EntityType == entity_type) ) + { + result->Add(aldRow.m_Key,NULL); + } + } + } + + return result; +} + +int64_t mc_AssetDB::GetTotalQuantity(mc_EntityDetails *entity) +{ + return GetTotalQuantity(&(entity->m_LedgerRow)); +} + +int64_t mc_AssetDB::GetTotalQuantity(mc_EntityLedgerRow *row) +{ + mc_EntityLedgerRow aldRow; + int64_t pos,first_pos; + int take_it,i; + int64_t total; + + total=0; + pos=row->m_ChainPos; + first_pos=row->m_FirstPos; + + for(i=m_MemPool->GetCount()-1;i>=0;i--) + { + memcpy(&aldRow,m_MemPool->GetRow(i),sizeof(mc_EntityLedgerRow)); + if( (aldRow.m_KeyType & MC_ENT_KEYTYPE_MASK) == MC_ENT_KEYTYPE_TXID) + { + if(aldRow.m_FirstPos == first_pos) + { + total+=aldRow.m_Quantity; + } + } + } + + take_it=1; + + if(first_pos >= 0) + { + m_Ledger->Open(); + while(take_it) + { + m_Ledger->GetRow(pos,&aldRow); + total+=aldRow.m_Quantity; + if(pos != first_pos) + { + pos=aldRow.m_LastPos; + if(pos<=0) + { + take_it=0; + total=0xFFFFFFFFFFFFFFFF; + } + } + else + { + take_it=0; + } + } + m_Ledger->Close(); + } + + return total; +} + +mc_Buffer *mc_AssetDB::GetFollowOns(const void* txid) +{ + mc_EntityLedgerRow aldRow; + int64_t pos,first_pos; + int take_it,i; + mc_Buffer *result; + result=new mc_Buffer; + + result->Initialize(MC_ENT_KEY_SIZE,MC_ENT_KEY_SIZE,MC_BUF_MODE_DEFAULT); + + result->Clear(); + + aldRow.Zero(); + + memcpy(aldRow.m_Key,txid,MC_ENT_KEY_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_TXID; + + if(GetEntity(&aldRow)) + { + pos=aldRow.m_ChainPos; + first_pos=aldRow.m_FirstPos; + + for(i=m_MemPool->GetCount()-1;i>=0;i--) + { + memcpy(&aldRow,m_MemPool->GetRow(i),sizeof(mc_EntityLedgerRow)); + if( (aldRow.m_KeyType & MC_ENT_KEYTYPE_MASK) == MC_ENT_KEYTYPE_TXID) + { + if(aldRow.m_FirstPos != (-i-1)) + { + if(aldRow.m_FirstPos == first_pos) + { + result->Add(aldRow.m_Key,NULL); + } + } + } + } + + if(pos > 0) + { + take_it=1; + + m_Ledger->Open(); + while(take_it) + { + m_Ledger->GetRow(pos,&aldRow); + result->Add(aldRow.m_Key,NULL); + if(pos != first_pos) + { + pos=aldRow.m_LastPos; + if(pos<=0) + { + take_it=0; + result->Clear(); + } + } + else + { + take_it=0; + } + } + m_Ledger->Close(); + } + else + { + result->Add(aldRow.m_Key,NULL); + } + } + + if(result->GetCount() == 0) + { + delete result; + return NULL; + } + + return result; +} + + + +int mc_AssetDB::HasFollowOns(const void* txid) +{ + mc_EntityLedgerRow aldRow; + int64_t first_pos; + int i; + + aldRow.Zero(); + + memcpy(aldRow.m_Key,txid,MC_ENT_KEY_SIZE); + aldRow.m_KeyType=MC_ENT_KEYTYPE_TXID; + + if(GetEntity(&aldRow)) + { + if(aldRow.m_FirstPos >= 0) + { + if(aldRow.m_FirstPos != aldRow.m_ChainPos) + { + return 1; + } + } + first_pos=aldRow.m_FirstPos; + + for(i=0;iGetCount();i++) + { + memcpy(&aldRow,m_MemPool->GetRow(i),sizeof(mc_EntityLedgerRow)); + + if(aldRow.m_FirstPos != (-i-1)) + { + if(aldRow.m_FirstPos == first_pos) + { + if(aldRow.m_KeyType & MC_ENT_KEYTYPE_FOLLOW_ON) + { + return 1; + } + } + } + } + } + + return 0; +} + +void mc_AssetDB::FreeEntityList(mc_Buffer *assets) +{ + if(assets) + { + delete assets; + } +} + +uint32_t mc_GetABScriptType(void *ptr) +{ + return (uint32_t)mc_GetLE((unsigned char*)ptr+MC_AST_ASSET_SCRIPT_TYPE_OFFSET,MC_AST_ASSET_SCRIPT_TYPE_SIZE); +} + +void mc_SetABScriptType(void *ptr,uint32_t type) +{ + mc_PutLE((unsigned char*)ptr+MC_AST_ASSET_SCRIPT_TYPE_OFFSET,&type,MC_AST_ASSET_SCRIPT_TYPE_SIZE); +} + +uint32_t mc_GetABRefType(void *ptr) +{ + return (uint32_t)mc_GetLE((unsigned char*)ptr+MC_AST_ASSET_REF_TYPE_OFFSET,MC_AST_ASSET_REF_TYPE_SIZE); +} + +void mc_SetABRefType(void *ptr,uint32_t type) +{ + mc_PutLE((unsigned char*)ptr+MC_AST_ASSET_REF_TYPE_OFFSET,&type,MC_AST_ASSET_REF_TYPE_SIZE); +} + +int64_t mc_GetABQuantity(void *ptr) +{ + return (int64_t)mc_GetLE((unsigned char*)ptr+MC_AST_ASSET_QUANTITY_OFFSET,MC_AST_ASSET_QUANTITY_SIZE); +} + +void mc_SetABQuantity(void *ptr,int64_t quantity) +{ + mc_PutLE((unsigned char*)ptr+MC_AST_ASSET_QUANTITY_OFFSET,&quantity,MC_AST_ASSET_QUANTITY_SIZE); +} + +unsigned char* mc_GetABRef(void *ptr) +{ + return (unsigned char*)ptr; +} + +void mc_SetABRef(void *ptr,void *ref) +{ + memcpy(ptr,ref,MC_AST_ASSET_BUFFER_REF_SIZE); +} + +void mc_ZeroABRaw(void *ptr) +{ + memset(ptr,0,MC_AST_ASSET_FULLREF_SIZE); +} + +void mc_InitABufferMap(mc_Buffer *buf) +{ + buf->Initialize(MC_AST_ASSET_QUANTITY_OFFSET,MC_AST_ASSET_FULLREF_BUF_SIZE,MC_BUF_MODE_MAP); +} + +void mc_InitABufferDefault(mc_Buffer *buf) +{ + buf->Initialize(MC_AST_ASSET_QUANTITY_OFFSET,MC_AST_ASSET_FULLREF_BUF_SIZE,MC_BUF_MODE_DEFAULT); +} + + diff --git a/src/entities/asset.h b/src/entities/asset.h new file mode 100644 index 00000000..4ea6f8f8 --- /dev/null +++ b/src/entities/asset.h @@ -0,0 +1,264 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAIN_ASSET_H +#define MULTICHAIN_ASSET_H + +#include "utils/declare.h" +#include "utils/dbwrapper.h" + +#define MC_AST_ASSET_REF_SIZE 10 +#define MC_AST_ASSET_BUF_TOTAL_SIZE 22 +#define MC_AST_SHORT_TXID_OFFSET 16 +#define MC_AST_SHORT_TXID_SIZE 16 + +#define MC_AST_ASSET_BUFFER_REF_SIZE 32 +#define MC_AST_ASSET_FULLREF_SIZE 36 +#define MC_AST_ASSET_QUANTITY_OFFSET 36 +#define MC_AST_ASSET_QUANTITY_SIZE 8 +#define MC_AST_ASSET_FULLREF_BUF_SIZE 48 + +#define MC_AST_ASSET_REF_TYPE_REF 0 +#define MC_AST_ASSET_REF_TYPE_SHORT_TXID 1 +#define MC_AST_ASSET_REF_TYPE_TXID 2 + +#define MC_AST_ASSET_REF_TYPE_GENESIS 256 +#define MC_AST_ASSET_REF_TYPE_SPECIAL 512 + + +#define MC_ENT_REF_SIZE 10 +#define MC_ENT_REF_PREFIX_SIZE 2 +#define MC_ENT_MAX_NAME_SIZE 32 +#define MC_ENT_MAX_ITEM_KEY_SIZE 256 +#define MC_ENT_MAX_SCRIPT_SIZE 4096 + +#define MC_ENT_KEY_SIZE 32 +#define MC_ENT_KEYTYPE_TXID 0x00000001 +#define MC_ENT_KEYTYPE_REF 0x00000002 +#define MC_ENT_KEYTYPE_NAME 0x00000003 +#define MC_ENT_KEYTYPE_SHORT_TXID 0x00000004 +#define MC_ENT_KEYTYPE_MASK 0x000000FF +#define MC_ENT_KEYTYPE_FOLLOW_ON 0x00000100 + +#define MC_ENT_TYPE_ANY 0xFF +#define MC_ENT_TYPE_NONE 0x00 +#define MC_ENT_TYPE_ASSET 0x01 +#define MC_ENT_TYPE_STREAM 0x02 +#define MC_ENT_TYPE_MAX 0x0F + +#define MC_ENT_SPRM_NAME 0x01 +#define MC_ENT_SPRM_FOLLOW_ONS 0x02 +#define MC_ENT_SPRM_ISSUER 0x03 +#define MC_ENT_SPRM_ANYONE_CAN_WRITE 0x04 +#define MC_ENT_SPRM_ASSET_MULTIPLE 0x41 + +#define MC_ENT_FLAG_OFFSET_IS_SET 0x00000001 +#define MC_ENT_FLAG_NAME_IS_SET 0x00000010 + + + +/** Database record structure */ + +typedef struct mc_EntityDBRow +{ + unsigned char m_Key[MC_ENT_KEY_SIZE]; // Entity key size - txid/entity-ref/name + uint32_t m_KeyType; // Entity key type - MC_ENT_KEYTYPE_ constants + int32_t m_Block; // Block entity is confirmed in + int64_t m_LedgerPos; // Position in the ledger corresponding to this key + int64_t m_ChainPos; // Position in the ledger corresponding to last object in the chain + uint32_t m_EntityType; // Entity type - MC_ENT_TYPE_ constants + uint32_t m_Reserved1; // Reserved to align to 64 bytes + + void Zero(); +} mc_EntityDBRow; + +/** Database */ + +typedef struct mc_EntityDB +{ + char m_FileName[MC_DCT_DB_MAX_PATH]; // Full file name + mc_Database *m_DB; // Database object + uint32_t m_KeyOffset; // Offset of the key in mc_EntityDBRow structure, 0 + uint32_t m_KeySize; // Size of the database key, 36 + uint32_t m_ValueOffset; // Offset of the value in mc_EntityDBRow structure, 36 + uint32_t m_ValueSize; // Size of the database value, 12 for protocol<=10003,28 otherwise + uint32_t m_TotalSize; // Totals size of the database row + mc_EntityDB() + { + Zero(); + } + + ~mc_EntityDB() + { + Close(); + } + void Zero(); + int Open(); + int Close(); + void SetName(const char *name); +} mc_EntityDB; + +/** Ledger and mempool record structure */ + +typedef struct mc_EntityLedgerRow +{ + unsigned char m_Key[MC_ENT_KEY_SIZE]; // Entity key size - txid/entity-ref/name + uint32_t m_KeyType; // Entity key type - MC_ENT_KEYTYPE_ constants + int32_t m_Block; // Block entity is confirmed in + int32_t m_Offset; // Offset of the entity in the block + uint32_t m_ScriptSize; // Script Size + int64_t m_Quantity; // Total quantity of the entity (including follow-ons) + uint32_t m_EntityType; // Entity type - MC_ENT_TYPE_ constants + uint32_t m_Reserved1; // Reserved to align to 96 bytes + int64_t m_PrevPos; // Position of the previous entity in the ledger + int64_t m_FirstPos; // Position in the ledger corresponding to first object in the chain + int64_t m_LastPos; // Position in the ledger corresponding to last object in the chain before this object + int64_t m_ChainPos; // Position in the ledger corresponding to last object in the chain + unsigned char m_Script[4224]; // Script + + void Zero(); +} mc_EntityLedgerRow; + +/** Entity details structure */ + +typedef struct mc_EntityDetails +{ + unsigned char m_Ref[MC_ENT_REF_SIZE]; // Entity reference + unsigned char m_FullRef[MC_AST_ASSET_QUANTITY_OFFSET]; // Full Entity reference, derived from short txid from v 10007 + char m_Name[MC_ENT_MAX_NAME_SIZE+6]; // Entity name + uint32_t m_Flags; + unsigned char m_Reserved[36]; + mc_EntityLedgerRow m_LedgerRow; + void Zero(); + void Set(mc_EntityLedgerRow *row); + const char* GetName(); + const unsigned char* GetTxID(); + const unsigned char* GetRef(); + const unsigned char* GetFullRef(); + const unsigned char* GetShortRef(); + const unsigned char* GetScript(); + int IsUnconfirmedGenesis(); + int GetAssetMultiple(); + int IsFollowOn(); +// int HasFollowOns(); + int AllowedFollowOns(); + int AnyoneCanWrite(); + uint64_t GetQuantity(); + uint32_t GetEntityType(); + const void* GetSpecialParam(uint32_t param,size_t* bytes); + const void* GetParam(const char *param,size_t* bytes); + int32_t NextParam(uint32_t offset,uint32_t* param_value_start,size_t *bytes); +}mc_EntityDetails; + +/** Ledger */ + +typedef struct mc_EntityLedger +{ + char m_FileName[MC_DCT_DB_MAX_PATH]; // Full file name + int m_FileHan; // File handle + uint32_t m_KeyOffset; // Offset of the key in mc_EntityLedgerRow structure, 0 + uint32_t m_KeySize; // Size of the ledger key, 36 + uint32_t m_ValueOffset; // Offset of the value in mc_EntityLedgerRow structure, 36 + uint32_t m_ValueSize; // Size of the ledger value 28 if protocol<=10003, 60 otherwise + uint32_t m_TotalSize; // Totals size of the ledger row + + mc_EntityLedger() + { + Zero(); + } + + ~mc_EntityLedger() + { + Close(); + } + + void Zero(); + int Open(); + int Close(); + void SetName(const char *name); + int GetRow(int64_t pos,mc_EntityLedgerRow *row); + int64_t GetSize(); + int SetRow(int64_t pos,mc_EntityLedgerRow *row); + int SetZeroRow(mc_EntityLedgerRow *row); + +} mc_EntityLedger; + + +typedef struct mc_AssetDB +{ + mc_EntityDB *m_Database; + mc_EntityLedger *m_Ledger; + + mc_Buffer *m_MemPool; + + char m_Name[MC_PRM_NETWORK_NAME_MAX_SIZE+1]; + int m_Block; + int64_t m_PrevPos; + int64_t m_Pos; + int m_DBRowCount; + + mc_AssetDB() + { + Zero(); + } + + ~mc_AssetDB() + { + Destroy(); + } + + +// External functions + + int Initialize(const char *name,int mode); + + int InsertStream(const void* txid, int offset, int entity_type, const void *script,size_t script_size, const void* special_script, size_t special_script_size,int update_mempool); + int InsertAsset(const void* txid, int offset, uint64_t quantity,const char *name,int multiple,const void *script,size_t script_size, const void* special_script, size_t special_script_size,int update_mempool); + int InsertAssetFollowOn(const void* txid, int offset, uint64_t quantity, const void *script,size_t script_size, const void* special_script, size_t special_script_size,const void* original_txid,int update_mempool); + int Commit(); + int RollBack(int block); + int RollBack(); + int ClearMemPool(); + + int GetEntity(mc_EntityLedgerRow *row); + + int FindEntityByTxID(mc_EntityDetails *entity, const unsigned char* txid); + int FindEntityByShortTxID (mc_EntityDetails *entity, const unsigned char* short_txid); + int FindEntityByRef (mc_EntityDetails *entity, const unsigned char* asset_ref); + int FindEntityByName(mc_EntityDetails *entity, const char* name); + int FindEntityByFollowOn(mc_EntityDetails *entity, const unsigned char* txid); + int FindEntityByFullRef (mc_EntityDetails *entity, unsigned char* full_ref); + + void Dump(); + mc_Buffer *GetEntityList(mc_Buffer *old_result,const void* txid,uint32_t entity_type); + void FreeEntityList(mc_Buffer *entities); + mc_Buffer *GetFollowOns(const void* txid); + int HasFollowOns(const void* txid); + int64_t GetTotalQuantity(mc_EntityDetails *entity); + + void RemoveFiles(); + +//Internal functions + int Zero(); + int Destroy(); + int64_t GetTotalQuantity(mc_EntityLedgerRow *row); + +} mc_AssetDB; + + + +uint32_t mc_GetABScriptType(void *ptr); +void mc_SetABScriptType(void *ptr,uint32_t type); +uint32_t mc_GetABRefType(void *ptr); +void mc_SetABRefType(void *ptr,uint32_t type); +int64_t mc_GetABQuantity(void *ptr); +void mc_SetABQuantity(void *ptr,int64_t quantity); +unsigned char* mc_GetABRef(void *ptr); +void mc_SetABRef(void *ptr,void *ref); +void mc_ZeroABRaw(void *ptr); +void mc_InitABufferMap(mc_Buffer *buf); +void mc_InitABufferDefault(mc_Buffer *buf); + + +#endif /* MULTICHAIN_ASSET_H */ + diff --git a/src/json/LICENSE.txt b/src/json/LICENSE.txt new file mode 100644 index 00000000..797d5363 --- /dev/null +++ b/src/json/LICENSE.txt @@ -0,0 +1,24 @@ +The MIT License + +Copyright (c) 2007 - 2009 John W. Wilkinson + +Permission is hereby granted, free of charge, to any person +obtaining a copy of this software and associated documentation +files (the "Software"), to deal in the Software without +restriction, including without limitation the rights to use, +copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the +Software is furnished to do so, subject to the following +conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES +OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, +WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +OTHER DEALINGS IN THE SOFTWARE. diff --git a/src/json/json_spirit.h b/src/json/json_spirit.h new file mode 100644 index 00000000..ac1879d5 --- /dev/null +++ b/src/json/json_spirit.h @@ -0,0 +1,18 @@ +#ifndef JSON_SPIRIT +#define JSON_SPIRIT + +// Copyright John W. Wilkinson 2007 - 2009. +// Distributed under the MIT License, see accompanying file LICENSE.txt + +// json spirit version 4.03 + +#if defined(_MSC_VER) && (_MSC_VER >= 1020) +# pragma once +#endif + +#include "json_spirit_value.h" +#include "json_spirit_reader.h" +#include "json_spirit_writer.h" +#include "json_spirit_utils.h" + +#endif diff --git a/src/json/json_spirit_error_position.h b/src/json/json_spirit_error_position.h new file mode 100644 index 00000000..17208507 --- /dev/null +++ b/src/json/json_spirit_error_position.h @@ -0,0 +1,54 @@ +#ifndef JSON_SPIRIT_ERROR_POSITION +#define JSON_SPIRIT_ERROR_POSITION + +// Copyright John W. Wilkinson 2007 - 2009. +// Distributed under the MIT License, see accompanying file LICENSE.txt + +// json spirit version 4.03 + +#if defined(_MSC_VER) && (_MSC_VER >= 1020) +# pragma once +#endif + +#include + +namespace json_spirit +{ + // An Error_position exception is thrown by the "read_or_throw" functions below on finding an error. + // Note the "read_or_throw" functions are around 3 times slower than the standard functions "read" + // functions that return a bool. + // + struct Error_position + { + Error_position(); + Error_position( unsigned int line, unsigned int column, const std::string& reason ); + bool operator==( const Error_position& lhs ) const; + unsigned int line_; + unsigned int column_; + std::string reason_; + }; + + inline Error_position::Error_position() + : line_( 0 ) + , column_( 0 ) + { + } + + inline Error_position::Error_position( unsigned int line, unsigned int column, const std::string& reason ) + : line_( line ) + , column_( column ) + , reason_( reason ) + { + } + + inline bool Error_position::operator==( const Error_position& lhs ) const + { + if( this == &lhs ) return true; + + return ( reason_ == lhs.reason_ ) && + ( line_ == lhs.line_ ) && + ( column_ == lhs.column_ ); +} +} + +#endif diff --git a/src/json/json_spirit_reader.cpp b/src/json/json_spirit_reader.cpp new file mode 100644 index 00000000..aa4f6372 --- /dev/null +++ b/src/json/json_spirit_reader.cpp @@ -0,0 +1,137 @@ +// Copyright John W. Wilkinson 2007 - 2009. +// Distributed under the MIT License, see accompanying file LICENSE.txt + +// json spirit version 4.03 + +#include "json_spirit_reader.h" +#include "json_spirit_reader_template.h" + +using namespace json_spirit; + +bool json_spirit::read( const std::string& s, Value& value ) +{ + return read_string( s, value ); +} + +void json_spirit::read_or_throw( const std::string& s, Value& value ) +{ + read_string_or_throw( s, value ); +} + +bool json_spirit::read( std::istream& is, Value& value ) +{ + return read_stream( is, value ); +} + +void json_spirit::read_or_throw( std::istream& is, Value& value ) +{ + read_stream_or_throw( is, value ); +} + +bool json_spirit::read( std::string::const_iterator& begin, std::string::const_iterator end, Value& value ) +{ + return read_range( begin, end, value ); +} + +void json_spirit::read_or_throw( std::string::const_iterator& begin, std::string::const_iterator end, Value& value ) +{ + begin = read_range_or_throw( begin, end, value ); +} + +#ifndef BOOST_NO_STD_WSTRING + +bool json_spirit::read( const std::wstring& s, wValue& value ) +{ + return read_string( s, value ); +} + +void json_spirit::read_or_throw( const std::wstring& s, wValue& value ) +{ + read_string_or_throw( s, value ); +} + +bool json_spirit::read( std::wistream& is, wValue& value ) +{ + return read_stream( is, value ); +} + +void json_spirit::read_or_throw( std::wistream& is, wValue& value ) +{ + read_stream_or_throw( is, value ); +} + +bool json_spirit::read( std::wstring::const_iterator& begin, std::wstring::const_iterator end, wValue& value ) +{ + return read_range( begin, end, value ); +} + +void json_spirit::read_or_throw( std::wstring::const_iterator& begin, std::wstring::const_iterator end, wValue& value ) +{ + begin = read_range_or_throw( begin, end, value ); +} + +#endif + +bool json_spirit::read( const std::string& s, mValue& value ) +{ + return read_string( s, value ); +} + +void json_spirit::read_or_throw( const std::string& s, mValue& value ) +{ + read_string_or_throw( s, value ); +} + +bool json_spirit::read( std::istream& is, mValue& value ) +{ + return read_stream( is, value ); +} + +void json_spirit::read_or_throw( std::istream& is, mValue& value ) +{ + read_stream_or_throw( is, value ); +} + +bool json_spirit::read( std::string::const_iterator& begin, std::string::const_iterator end, mValue& value ) +{ + return read_range( begin, end, value ); +} + +void json_spirit::read_or_throw( std::string::const_iterator& begin, std::string::const_iterator end, mValue& value ) +{ + begin = read_range_or_throw( begin, end, value ); +} + +#ifndef BOOST_NO_STD_WSTRING + +bool json_spirit::read( const std::wstring& s, wmValue& value ) +{ + return read_string( s, value ); +} + +void json_spirit::read_or_throw( const std::wstring& s, wmValue& value ) +{ + read_string_or_throw( s, value ); +} + +bool json_spirit::read( std::wistream& is, wmValue& value ) +{ + return read_stream( is, value ); +} + +void json_spirit::read_or_throw( std::wistream& is, wmValue& value ) +{ + read_stream_or_throw( is, value ); +} + +bool json_spirit::read( std::wstring::const_iterator& begin, std::wstring::const_iterator end, wmValue& value ) +{ + return read_range( begin, end, value ); +} + +void json_spirit::read_or_throw( std::wstring::const_iterator& begin, std::wstring::const_iterator end, wmValue& value ) +{ + begin = read_range_or_throw( begin, end, value ); +} + +#endif diff --git a/src/json/json_spirit_reader.h b/src/json/json_spirit_reader.h new file mode 100644 index 00000000..96494a97 --- /dev/null +++ b/src/json/json_spirit_reader.h @@ -0,0 +1,62 @@ +#ifndef JSON_SPIRIT_READER +#define JSON_SPIRIT_READER + +// Copyright John W. Wilkinson 2007 - 2009. +// Distributed under the MIT License, see accompanying file LICENSE.txt + +// json spirit version 4.03 + +#if defined(_MSC_VER) && (_MSC_VER >= 1020) +# pragma once +#endif + +#include "json_spirit_value.h" +#include "json_spirit_error_position.h" +#include + +namespace json_spirit +{ + // functions to reads a JSON values + + bool read( const std::string& s, Value& value ); + bool read( std::istream& is, Value& value ); + bool read( std::string::const_iterator& begin, std::string::const_iterator end, Value& value ); + + void read_or_throw( const std::string& s, Value& value ); + void read_or_throw( std::istream& is, Value& value ); + void read_or_throw( std::string::const_iterator& begin, std::string::const_iterator end, Value& value ); + +#ifndef BOOST_NO_STD_WSTRING + + bool read( const std::wstring& s, wValue& value ); + bool read( std::wistream& is, wValue& value ); + bool read( std::wstring::const_iterator& begin, std::wstring::const_iterator end, wValue& value ); + + void read_or_throw( const std::wstring& s, wValue& value ); + void read_or_throw( std::wistream& is, wValue& value ); + void read_or_throw( std::wstring::const_iterator& begin, std::wstring::const_iterator end, wValue& value ); + +#endif + + bool read( const std::string& s, mValue& value ); + bool read( std::istream& is, mValue& value ); + bool read( std::string::const_iterator& begin, std::string::const_iterator end, mValue& value ); + + void read_or_throw( const std::string& s, mValue& value ); + void read_or_throw( std::istream& is, mValue& value ); + void read_or_throw( std::string::const_iterator& begin, std::string::const_iterator end, mValue& value ); + +#ifndef BOOST_NO_STD_WSTRING + + bool read( const std::wstring& s, wmValue& value ); + bool read( std::wistream& is, wmValue& value ); + bool read( std::wstring::const_iterator& begin, std::wstring::const_iterator end, wmValue& value ); + + void read_or_throw( const std::wstring& s, wmValue& value ); + void read_or_throw( std::wistream& is, wmValue& value ); + void read_or_throw( std::wstring::const_iterator& begin, std::wstring::const_iterator end, wmValue& value ); + +#endif +} + +#endif diff --git a/src/json/json_spirit_reader_template.h b/src/json/json_spirit_reader_template.h new file mode 100644 index 00000000..46f5892f --- /dev/null +++ b/src/json/json_spirit_reader_template.h @@ -0,0 +1,612 @@ +#ifndef JSON_SPIRIT_READER_TEMPLATE +#define JSON_SPIRIT_READER_TEMPLATE + +// Copyright John W. Wilkinson 2007 - 2009. +// Distributed under the MIT License, see accompanying file LICENSE.txt + +// json spirit version 4.03 + +#include "json_spirit_value.h" +#include "json_spirit_error_position.h" + +//#define BOOST_SPIRIT_THREADSAFE // uncomment for multithreaded use, requires linking to boost.thread + +#include +#include +#include + +#if BOOST_VERSION >= 103800 + #include + #include + #include + #include + #include + #define spirit_namespace boost::spirit::classic +#else + #include + #include + #include + #include + #include + #define spirit_namespace boost::spirit +#endif + +namespace json_spirit +{ + const spirit_namespace::int_parser < int64_t > int64_p = spirit_namespace::int_parser < int64_t >(); + const spirit_namespace::uint_parser< uint64_t > uint64_p = spirit_namespace::uint_parser< uint64_t >(); + + template< class Iter_type > + bool is_eq( Iter_type first, Iter_type last, const char* c_str ) + { + for( Iter_type i = first; i != last; ++i, ++c_str ) + { + if( *c_str == 0 ) return false; + + if( *i != *c_str ) return false; + } + + return true; + } + + template< class Char_type > + Char_type hex_to_num( const Char_type c ) + { + if( ( c >= '0' ) && ( c <= '9' ) ) return c - '0'; + if( ( c >= 'a' ) && ( c <= 'f' ) ) return c - 'a' + 10; + if( ( c >= 'A' ) && ( c <= 'F' ) ) return c - 'A' + 10; + return 0; + } + + template< class Char_type, class Iter_type > + Char_type hex_str_to_char( Iter_type& begin ) + { + const Char_type c1( *( ++begin ) ); + const Char_type c2( *( ++begin ) ); + + return ( hex_to_num( c1 ) << 4 ) + hex_to_num( c2 ); + } + + template< class Char_type, class Iter_type > + Char_type unicode_str_to_char( Iter_type& begin ) + { + const Char_type c1( *( ++begin ) ); + const Char_type c2( *( ++begin ) ); + const Char_type c3( *( ++begin ) ); + const Char_type c4( *( ++begin ) ); + + return ( hex_to_num( c1 ) << 12 ) + + ( hex_to_num( c2 ) << 8 ) + + ( hex_to_num( c3 ) << 4 ) + + hex_to_num( c4 ); + } + + template< class String_type > + void append_esc_char_and_incr_iter( String_type& s, + typename String_type::const_iterator& begin, + typename String_type::const_iterator end ) + { + typedef typename String_type::value_type Char_type; + + const Char_type c2( *begin ); + + switch( c2 ) + { + case 't': s += '\t'; break; + case 'b': s += '\b'; break; + case 'f': s += '\f'; break; + case 'n': s += '\n'; break; + case 'r': s += '\r'; break; + case '\\': s += '\\'; break; + case '/': s += '/'; break; + case '"': s += '"'; break; + case 'x': + { + if( end - begin >= 3 ) // expecting "xHH..." + { + s += hex_str_to_char< Char_type >( begin ); + } + break; + } + case 'u': + { + if( end - begin >= 5 ) // expecting "uHHHH..." + { + s += unicode_str_to_char< Char_type >( begin ); + } + break; + } + } + } + + template< class String_type > + String_type substitute_esc_chars( typename String_type::const_iterator begin, + typename String_type::const_iterator end ) + { + typedef typename String_type::const_iterator Iter_type; + + if( end - begin < 2 ) return String_type( begin, end ); + + String_type result; + + result.reserve( end - begin ); + + const Iter_type end_minus_1( end - 1 ); + + Iter_type substr_start = begin; + Iter_type i = begin; + + for( ; i < end_minus_1; ++i ) + { + if( *i == '\\' ) + { + result.append( substr_start, i ); + + ++i; // skip the '\' + + append_esc_char_and_incr_iter( result, i, end ); + + substr_start = i + 1; + } + } + + result.append( substr_start, end ); + + return result; + } + + template< class String_type > + String_type get_str_( typename String_type::const_iterator begin, + typename String_type::const_iterator end ) + { + assert( end - begin >= 2 ); + + typedef typename String_type::const_iterator Iter_type; + + Iter_type str_without_quotes( ++begin ); + Iter_type end_without_quotes( --end ); + + return substitute_esc_chars< String_type >( str_without_quotes, end_without_quotes ); + } + + inline std::string get_str( std::string::const_iterator begin, std::string::const_iterator end ) + { + return get_str_< std::string >( begin, end ); + } + + inline std::wstring get_str( std::wstring::const_iterator begin, std::wstring::const_iterator end ) + { + return get_str_< std::wstring >( begin, end ); + } + + template< class String_type, class Iter_type > + String_type get_str( Iter_type begin, Iter_type end ) + { + const String_type tmp( begin, end ); // convert multipass iterators to string iterators + + return get_str( tmp.begin(), tmp.end() ); + } + + // this class's methods get called by the spirit parse resulting + // in the creation of a JSON object or array + // + // NB Iter_type could be a std::string iterator, wstring iterator, a position iterator or a multipass iterator + // + template< class Value_type, class Iter_type > + class Semantic_actions + { + public: + + typedef typename Value_type::Config_type Config_type; + typedef typename Config_type::String_type String_type; + typedef typename Config_type::Object_type Object_type; + typedef typename Config_type::Array_type Array_type; + typedef typename String_type::value_type Char_type; + + Semantic_actions( Value_type& value ) + : value_( value ) + , current_p_( 0 ) + { + } + + void begin_obj( Char_type c ) + { + assert( c == '{' ); + + begin_compound< Object_type >(); + } + + void end_obj( Char_type c ) + { + assert( c == '}' ); + + end_compound(); + } + + void begin_array( Char_type c ) + { + assert( c == '[' ); + + begin_compound< Array_type >(); + } + + void end_array( Char_type c ) + { + assert( c == ']' ); + + end_compound(); + } + + void new_name( Iter_type begin, Iter_type end ) + { + assert( current_p_->type() == obj_type ); + + name_ = get_str< String_type >( begin, end ); + } + + void new_str( Iter_type begin, Iter_type end ) + { + add_to_current( get_str< String_type >( begin, end ) ); + } + + void new_true( Iter_type begin, Iter_type end ) + { + assert( is_eq( begin, end, "true" ) ); + + add_to_current( true ); + } + + void new_false( Iter_type begin, Iter_type end ) + { + assert( is_eq( begin, end, "false" ) ); + + add_to_current( false ); + } + + void new_null( Iter_type begin, Iter_type end ) + { + assert( is_eq( begin, end, "null" ) ); + + add_to_current( Value_type() ); + } + + void new_int( int64_t i ) + { + add_to_current( i ); + } + + void new_uint64( uint64_t ui ) + { + add_to_current( ui ); + } + + void new_real( double d ) + { + add_to_current( d ); + } + + private: + + Semantic_actions& operator=( const Semantic_actions& ); + // to prevent "assignment operator could not be generated" warning + + Value_type* add_first( const Value_type& value ) + { + assert( current_p_ == 0 ); + + value_ = value; + current_p_ = &value_; + return current_p_; + } + + template< class Array_or_obj > + void begin_compound() + { + if( current_p_ == 0 ) + { + add_first( Array_or_obj() ); + } + else + { + stack_.push_back( current_p_ ); + + Array_or_obj new_array_or_obj; // avoid copy by building new array or object in place + + current_p_ = add_to_current( new_array_or_obj ); + } + } + + void end_compound() + { + if( current_p_ != &value_ ) + { + current_p_ = stack_.back(); + + stack_.pop_back(); + } + } + + Value_type* add_to_current( const Value_type& value ) + { + if( current_p_ == 0 ) + { + return add_first( value ); + } + else if( current_p_->type() == array_type ) + { + current_p_->get_array().push_back( value ); + + return ¤t_p_->get_array().back(); + } + + assert( current_p_->type() == obj_type ); + + return &Config_type::add( current_p_->get_obj(), name_, value ); + } + + Value_type& value_; // this is the object or array that is being created + Value_type* current_p_; // the child object or array that is currently being constructed + + std::vector< Value_type* > stack_; // previous child objects and arrays + + String_type name_; // of current name/value pair + }; + + template< typename Iter_type > + void throw_error( spirit_namespace::position_iterator< Iter_type > i, const std::string& reason ) + { + throw Error_position( i.get_position().line, i.get_position().column, reason ); + } + + template< typename Iter_type > + void throw_error( Iter_type i, const std::string& reason ) + { + throw reason; + } + + // the spirit grammer + // + template< class Value_type, class Iter_type > + class Json_grammer : public spirit_namespace::grammar< Json_grammer< Value_type, Iter_type > > + { + public: + + typedef Semantic_actions< Value_type, Iter_type > Semantic_actions_t; + + Json_grammer( Semantic_actions_t& semantic_actions ) + : actions_( semantic_actions ) + { + } + + static void throw_not_value( Iter_type begin, Iter_type end ) + { + throw_error( begin, "not a value" ); + } + + static void throw_not_array( Iter_type begin, Iter_type end ) + { + throw_error( begin, "not an array" ); + } + + static void throw_not_object( Iter_type begin, Iter_type end ) + { + throw_error( begin, "not an object" ); + } + + static void throw_not_pair( Iter_type begin, Iter_type end ) + { + throw_error( begin, "not a pair" ); + } + + static void throw_not_colon( Iter_type begin, Iter_type end ) + { + throw_error( begin, "no colon in pair" ); + } + + static void throw_not_string( Iter_type begin, Iter_type end ) + { + throw_error( begin, "not a string" ); + } + + template< typename ScannerT > + class definition + { + public: + + definition( const Json_grammer& self ) + { + using namespace spirit_namespace; + + typedef typename Value_type::String_type::value_type Char_type; + + // first we convert the semantic action class methods to functors with the + // parameter signature expected by spirit + + typedef boost::function< void( Char_type ) > Char_action; + typedef boost::function< void( Iter_type, Iter_type ) > Str_action; + typedef boost::function< void( double ) > Real_action; + typedef boost::function< void( int64_t ) > Int_action; + typedef boost::function< void( uint64_t ) > Uint64_action; + + Char_action begin_obj ( boost::bind( &Semantic_actions_t::begin_obj, &self.actions_, _1 ) ); + Char_action end_obj ( boost::bind( &Semantic_actions_t::end_obj, &self.actions_, _1 ) ); + Char_action begin_array( boost::bind( &Semantic_actions_t::begin_array, &self.actions_, _1 ) ); + Char_action end_array ( boost::bind( &Semantic_actions_t::end_array, &self.actions_, _1 ) ); + Str_action new_name ( boost::bind( &Semantic_actions_t::new_name, &self.actions_, _1, _2 ) ); + Str_action new_str ( boost::bind( &Semantic_actions_t::new_str, &self.actions_, _1, _2 ) ); + Str_action new_true ( boost::bind( &Semantic_actions_t::new_true, &self.actions_, _1, _2 ) ); + Str_action new_false ( boost::bind( &Semantic_actions_t::new_false, &self.actions_, _1, _2 ) ); + Str_action new_null ( boost::bind( &Semantic_actions_t::new_null, &self.actions_, _1, _2 ) ); + Real_action new_real ( boost::bind( &Semantic_actions_t::new_real, &self.actions_, _1 ) ); + Int_action new_int ( boost::bind( &Semantic_actions_t::new_int, &self.actions_, _1 ) ); + Uint64_action new_uint64 ( boost::bind( &Semantic_actions_t::new_uint64, &self.actions_, _1 ) ); + + // actual grammer + + json_ + = value_ | eps_p[ &throw_not_value ] + ; + + value_ + = string_[ new_str ] + | number_ + | object_ + | array_ + | str_p( "true" ) [ new_true ] + | str_p( "false" )[ new_false ] + | str_p( "null" ) [ new_null ] + ; + + object_ + = ch_p('{')[ begin_obj ] + >> !members_ + >> ( ch_p('}')[ end_obj ] | eps_p[ &throw_not_object ] ) + ; + + members_ + = pair_ >> *( ',' >> pair_ ) + ; + + pair_ + = string_[ new_name ] + >> ( ':' | eps_p[ &throw_not_colon ] ) + >> ( value_ | eps_p[ &throw_not_value ] ) + ; + + array_ + = ch_p('[')[ begin_array ] + >> !elements_ + >> ( ch_p(']')[ end_array ] | eps_p[ &throw_not_array ] ) + ; + + elements_ + = value_ >> *( ',' >> value_ ) + ; + + string_ + = lexeme_d // this causes white space inside a string to be retained + [ + confix_p + ( + '"', + *lex_escape_ch_p, + '"' + ) + ] + ; + + number_ + = strict_real_p[ new_real ] + | int64_p [ new_int ] + | uint64_p [ new_uint64 ] + ; + } + + spirit_namespace::rule< ScannerT > json_, object_, members_, pair_, array_, elements_, value_, string_, number_; + + const spirit_namespace::rule< ScannerT >& start() const { return json_; } + }; + + private: + + Json_grammer& operator=( const Json_grammer& ); // to prevent "assignment operator could not be generated" warning + + Semantic_actions_t& actions_; + }; + + template< class Iter_type, class Value_type > + Iter_type read_range_or_throw( Iter_type begin, Iter_type end, Value_type& value ) + { + Semantic_actions< Value_type, Iter_type > semantic_actions( value ); + + const spirit_namespace::parse_info< Iter_type > info = + spirit_namespace::parse( begin, end, + Json_grammer< Value_type, Iter_type >( semantic_actions ), + spirit_namespace::space_p ); + + if( !info.hit ) + { + assert( false ); // in theory exception should already have been thrown + throw_error( info.stop, "error" ); + } + + return info.stop; + } + + template< class Iter_type, class Value_type > + void add_posn_iter_and_read_range_or_throw( Iter_type begin, Iter_type end, Value_type& value ) + { + typedef spirit_namespace::position_iterator< Iter_type > Posn_iter_t; + + const Posn_iter_t posn_begin( begin, end ); + const Posn_iter_t posn_end( end, end ); + + read_range_or_throw( posn_begin, posn_end, value ); + } + + template< class Iter_type, class Value_type > + bool read_range( Iter_type& begin, Iter_type end, Value_type& value ) + { + try + { + begin = read_range_or_throw( begin, end, value ); + + return true; + } + catch( ... ) + { + return false; + } + } + + template< class String_type, class Value_type > + void read_string_or_throw( const String_type& s, Value_type& value ) + { + add_posn_iter_and_read_range_or_throw( s.begin(), s.end(), value ); + } + + template< class String_type, class Value_type > + bool read_string( const String_type& s, Value_type& value ) + { + typename String_type::const_iterator begin = s.begin(); + + return read_range( begin, s.end(), value ); + } + + template< class Istream_type > + struct Multi_pass_iters + { + typedef typename Istream_type::char_type Char_type; + typedef std::istream_iterator< Char_type, Char_type > istream_iter; + typedef spirit_namespace::multi_pass< istream_iter > Mp_iter; + + Multi_pass_iters( Istream_type& is ) + { + is.unsetf( std::ios::skipws ); + + begin_ = spirit_namespace::make_multi_pass( istream_iter( is ) ); + end_ = spirit_namespace::make_multi_pass( istream_iter() ); + } + + Mp_iter begin_; + Mp_iter end_; + }; + + template< class Istream_type, class Value_type > + bool read_stream( Istream_type& is, Value_type& value ) + { + Multi_pass_iters< Istream_type > mp_iters( is ); + + return read_range( mp_iters.begin_, mp_iters.end_, value ); + } + + template< class Istream_type, class Value_type > + void read_stream_or_throw( Istream_type& is, Value_type& value ) + { + const Multi_pass_iters< Istream_type > mp_iters( is ); + + add_posn_iter_and_read_range_or_throw( mp_iters.begin_, mp_iters.end_, value ); + } +} + +#endif diff --git a/src/json/json_spirit_stream_reader.h b/src/json/json_spirit_stream_reader.h new file mode 100644 index 00000000..7e59c9ad --- /dev/null +++ b/src/json/json_spirit_stream_reader.h @@ -0,0 +1,70 @@ +#ifndef JSON_SPIRIT_READ_STREAM +#define JSON_SPIRIT_READ_STREAM + +// Copyright John W. Wilkinson 2007 - 2009. +// Distributed under the MIT License, see accompanying file LICENSE.txt + +// json spirit version 4.03 + +#if defined(_MSC_VER) && (_MSC_VER >= 1020) +# pragma once +#endif + +#include "json_spirit_reader_template.h" + +namespace json_spirit +{ + // these classes allows you to read multiple top level contiguous values from a stream, + // the normal stream read functions have a bug that prevent multiple top level values + // from being read unless they are separated by spaces + + template< class Istream_type, class Value_type > + class Stream_reader + { + public: + + Stream_reader( Istream_type& is ) + : iters_( is ) + { + } + + bool read_next( Value_type& value ) + { + return read_range( iters_.begin_, iters_.end_, value ); + } + + private: + + typedef Multi_pass_iters< Istream_type > Mp_iters; + + Mp_iters iters_; + }; + + template< class Istream_type, class Value_type > + class Stream_reader_thrower + { + public: + + Stream_reader_thrower( Istream_type& is ) + : iters_( is ) + , posn_begin_( iters_.begin_, iters_.end_ ) + , posn_end_( iters_.end_, iters_.end_ ) + { + } + + void read_next( Value_type& value ) + { + posn_begin_ = read_range_or_throw( posn_begin_, posn_end_, value ); + } + + private: + + typedef Multi_pass_iters< Istream_type > Mp_iters; + typedef spirit_namespace::position_iterator< typename Mp_iters::Mp_iter > Posn_iter_t; + + Mp_iters iters_; + Posn_iter_t posn_begin_, posn_end_; + }; +} + +#endif diff --git a/src/json/json_spirit_utils.h b/src/json/json_spirit_utils.h new file mode 100644 index 00000000..553e3b96 --- /dev/null +++ b/src/json/json_spirit_utils.h @@ -0,0 +1,61 @@ +#ifndef JSON_SPIRIT_UTILS +#define JSON_SPIRIT_UTILS + +// Copyright John W. Wilkinson 2007 - 2009. +// Distributed under the MIT License, see accompanying file LICENSE.txt + +// json spirit version 4.03 + +#if defined(_MSC_VER) && (_MSC_VER >= 1020) +# pragma once +#endif + +#include "json_spirit_value.h" +#include + +namespace json_spirit +{ + template< class Obj_t, class Map_t > + void obj_to_map( const Obj_t& obj, Map_t& mp_obj ) + { + mp_obj.clear(); + + for( typename Obj_t::const_iterator i = obj.begin(); i != obj.end(); ++i ) + { + mp_obj[ i->name_ ] = i->value_; + } + } + + template< class Obj_t, class Map_t > + void map_to_obj( const Map_t& mp_obj, Obj_t& obj ) + { + obj.clear(); + + for( typename Map_t::const_iterator i = mp_obj.begin(); i != mp_obj.end(); ++i ) + { + obj.push_back( typename Obj_t::value_type( i->first, i->second ) ); + } + } + + typedef std::map< std::string, Value > Mapped_obj; + +#ifndef BOOST_NO_STD_WSTRING + typedef std::map< std::wstring, wValue > wMapped_obj; +#endif + + template< class Object_type, class String_type > + const typename Object_type::value_type::Value_type& find_value( const Object_type& obj, const String_type& name ) + { + for( typename Object_type::const_iterator i = obj.begin(); i != obj.end(); ++i ) + { + if( i->name_ == name ) + { + return i->value_; + } + } + + return Object_type::value_type::Value_type::null; + } +} + +#endif diff --git a/src/json/json_spirit_value.cpp b/src/json/json_spirit_value.cpp new file mode 100644 index 00000000..44d2f06a --- /dev/null +++ b/src/json/json_spirit_value.cpp @@ -0,0 +1,8 @@ +/* Copyright (c) 2007 John W Wilkinson + + This source code can be used for any purpose as long as + this comment is retained. */ + +// json spirit version 2.00 + +#include "json_spirit_value.h" diff --git a/src/json/json_spirit_value.h b/src/json/json_spirit_value.h new file mode 100644 index 00000000..13cc8921 --- /dev/null +++ b/src/json/json_spirit_value.h @@ -0,0 +1,534 @@ +#ifndef JSON_SPIRIT_VALUE +#define JSON_SPIRIT_VALUE + +// Copyright John W. Wilkinson 2007 - 2009. +// Distributed under the MIT License, see accompanying file LICENSE.txt + +// json spirit version 4.03 + +#if defined(_MSC_VER) && (_MSC_VER >= 1020) +# pragma once +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace json_spirit +{ + enum Value_type{ obj_type, array_type, str_type, bool_type, int_type, real_type, null_type }; + static const char* Value_type_name[]={"obj", "array", "str", "bool", "int", "real", "null"}; + + template< class Config > // Config determines whether the value uses std::string or std::wstring and + // whether JSON Objects are represented as vectors or maps + class Value_impl + { + public: + + typedef Config Config_type; + typedef typename Config::String_type String_type; + typedef typename Config::Object_type Object; + typedef typename Config::Array_type Array; + typedef typename String_type::const_pointer Const_str_ptr; // eg const char* + + Value_impl(); // creates null value + Value_impl( Const_str_ptr value ); + Value_impl( const String_type& value ); + Value_impl( const Object& value ); + Value_impl( const Array& value ); + Value_impl( bool value ); + Value_impl( int value ); + Value_impl( int64_t value ); + Value_impl( uint64_t value ); + Value_impl( double value ); + + Value_impl( const Value_impl& other ); + + bool operator==( const Value_impl& lhs ) const; + + Value_impl& operator=( const Value_impl& lhs ); + + Value_type type() const; + + bool is_uint64() const; + bool is_null() const; + + const String_type& get_str() const; + const Object& get_obj() const; + const Array& get_array() const; + bool get_bool() const; + int get_int() const; + int64_t get_int64() const; + uint64_t get_uint64() const; + double get_real() const; + + Object& get_obj(); + Array& get_array(); + + template< typename T > T get_value() const; // example usage: int i = value.get_value< int >(); + // or double d = value.get_value< double >(); + + static const Value_impl null; + + private: + + void check_type( const Value_type vtype ) const; + + typedef boost::variant< String_type, + boost::recursive_wrapper< Object >, boost::recursive_wrapper< Array >, + bool, int64_t, double > Variant; + + Value_type type_; + Variant v_; + bool is_uint64_; + }; + + // vector objects + + template< class Config > + struct Pair_impl + { + typedef typename Config::String_type String_type; + typedef typename Config::Value_type Value_type; + + Pair_impl( const String_type& name, const Value_type& value ); + + bool operator==( const Pair_impl& lhs ) const; + + String_type name_; + Value_type value_; + }; + + template< class String > + struct Config_vector + { + typedef String String_type; + typedef Value_impl< Config_vector > Value_type; + typedef Pair_impl < Config_vector > Pair_type; + typedef std::vector< Value_type > Array_type; + typedef std::vector< Pair_type > Object_type; + + static Value_type& add( Object_type& obj, const String_type& name, const Value_type& value ) + { + obj.push_back( Pair_type( name , value ) ); + + return obj.back().value_; + } + + static String_type get_name( const Pair_type& pair ) + { + return pair.name_; + } + + static Value_type get_value( const Pair_type& pair ) + { + return pair.value_; + } + }; + + // typedefs for ASCII + + typedef Config_vector< std::string > Config; + + typedef Config::Value_type Value; + typedef Config::Pair_type Pair; + typedef Config::Object_type Object; + typedef Config::Array_type Array; + + // typedefs for Unicode + +#ifndef BOOST_NO_STD_WSTRING + + typedef Config_vector< std::wstring > wConfig; + + typedef wConfig::Value_type wValue; + typedef wConfig::Pair_type wPair; + typedef wConfig::Object_type wObject; + typedef wConfig::Array_type wArray; +#endif + + // map objects + + template< class String > + struct Config_map + { + typedef String String_type; + typedef Value_impl< Config_map > Value_type; + typedef std::vector< Value_type > Array_type; + typedef std::map< String_type, Value_type > Object_type; + typedef typename Object_type::value_type Pair_type; + + static Value_type& add( Object_type& obj, const String_type& name, const Value_type& value ) + { + return obj[ name ] = value; + } + + static String_type get_name( const Pair_type& pair ) + { + return pair.first; + } + + static Value_type get_value( const Pair_type& pair ) + { + return pair.second; + } + }; + + // typedefs for ASCII + + typedef Config_map< std::string > mConfig; + + typedef mConfig::Value_type mValue; + typedef mConfig::Object_type mObject; + typedef mConfig::Array_type mArray; + + // typedefs for Unicode + +#ifndef BOOST_NO_STD_WSTRING + + typedef Config_map< std::wstring > wmConfig; + + typedef wmConfig::Value_type wmValue; + typedef wmConfig::Object_type wmObject; + typedef wmConfig::Array_type wmArray; + +#endif + + /////////////////////////////////////////////////////////////////////////////////////////////// + // + // implementation + + template< class Config > + const Value_impl< Config > Value_impl< Config >::null; + + template< class Config > + Value_impl< Config >::Value_impl() + : type_( null_type ) + , is_uint64_( false ) + { + } + + template< class Config > + Value_impl< Config >::Value_impl( const Const_str_ptr value ) + : type_( str_type ) + , v_( String_type( value ) ) + , is_uint64_( false ) + { + } + + template< class Config > + Value_impl< Config >::Value_impl( const String_type& value ) + : type_( str_type ) + , v_( value ) + , is_uint64_( false ) + { + } + + template< class Config > + Value_impl< Config >::Value_impl( const Object& value ) + : type_( obj_type ) + , v_( value ) + , is_uint64_( false ) + { + } + + template< class Config > + Value_impl< Config >::Value_impl( const Array& value ) + : type_( array_type ) + , v_( value ) + , is_uint64_( false ) + { + } + + template< class Config > + Value_impl< Config >::Value_impl( bool value ) + : type_( bool_type ) + , v_( value ) + , is_uint64_( false ) + { + } + + template< class Config > + Value_impl< Config >::Value_impl( int value ) + : type_( int_type ) + , v_( static_cast< int64_t >( value ) ) + , is_uint64_( false ) + { + } + + template< class Config > + Value_impl< Config >::Value_impl( int64_t value ) + : type_( int_type ) + , v_( value ) + , is_uint64_( false ) + { + } + + template< class Config > + Value_impl< Config >::Value_impl( uint64_t value ) + : type_( int_type ) + , v_( static_cast< int64_t >( value ) ) + , is_uint64_( true ) + { + } + + template< class Config > + Value_impl< Config >::Value_impl( double value ) + : type_( real_type ) + , v_( value ) + , is_uint64_( false ) + { + } + + template< class Config > + Value_impl< Config >::Value_impl( const Value_impl< Config >& other ) + : type_( other.type() ) + , v_( other.v_ ) + , is_uint64_( other.is_uint64_ ) + { + } + + template< class Config > + Value_impl< Config >& Value_impl< Config >::operator=( const Value_impl& lhs ) + { + Value_impl tmp( lhs ); + + std::swap( type_, tmp.type_ ); + std::swap( v_, tmp.v_ ); + std::swap( is_uint64_, tmp.is_uint64_ ); + + return *this; + } + + template< class Config > + bool Value_impl< Config >::operator==( const Value_impl& lhs ) const + { + if( this == &lhs ) return true; + + if( type() != lhs.type() ) return false; + + return v_ == lhs.v_; + } + + template< class Config > + Value_type Value_impl< Config >::type() const + { + return type_; + } + + template< class Config > + bool Value_impl< Config >::is_uint64() const + { + return is_uint64_; + } + + template< class Config > + bool Value_impl< Config >::is_null() const + { + return type() == null_type; + } + + template< class Config > + void Value_impl< Config >::check_type( const Value_type vtype ) const + { + if( type() != vtype ) + { + std::ostringstream os; + + ///// Bitcoin: Tell the types by name instead of by number + os << "value is type " << Value_type_name[type()] << ", expected " << Value_type_name[vtype]; + + throw std::runtime_error( os.str() ); + } + } + + template< class Config > + const typename Config::String_type& Value_impl< Config >::get_str() const + { + check_type( str_type ); + + return *boost::get< String_type >( &v_ ); + } + + template< class Config > + const typename Value_impl< Config >::Object& Value_impl< Config >::get_obj() const + { + check_type( obj_type ); + + return *boost::get< Object >( &v_ ); + } + + template< class Config > + const typename Value_impl< Config >::Array& Value_impl< Config >::get_array() const + { + check_type( array_type ); + + return *boost::get< Array >( &v_ ); + } + + template< class Config > + bool Value_impl< Config >::get_bool() const + { + check_type( bool_type ); + + return boost::get< bool >( v_ ); + } + + template< class Config > + int Value_impl< Config >::get_int() const + { + check_type( int_type ); + + return static_cast< int >( get_int64() ); + } + + template< class Config > + int64_t Value_impl< Config >::get_int64() const + { + check_type( int_type ); + + return boost::get< int64_t >( v_ ); + } + + template< class Config > + uint64_t Value_impl< Config >::get_uint64() const + { + check_type( int_type ); + + return static_cast< uint64_t >( get_int64() ); + } + + template< class Config > + double Value_impl< Config >::get_real() const + { + if( type() == int_type ) + { + return is_uint64() ? static_cast< double >( get_uint64() ) + : static_cast< double >( get_int64() ); + } + + check_type( real_type ); + + return boost::get< double >( v_ ); + } + + template< class Config > + typename Value_impl< Config >::Object& Value_impl< Config >::get_obj() + { + check_type( obj_type ); + + return *boost::get< Object >( &v_ ); + } + + template< class Config > + typename Value_impl< Config >::Array& Value_impl< Config >::get_array() + { + check_type( array_type ); + + return *boost::get< Array >( &v_ ); + } + + template< class Config > + Pair_impl< Config >::Pair_impl( const String_type& name, const Value_type& value ) + : name_( name ) + , value_( value ) + { + } + + template< class Config > + bool Pair_impl< Config >::operator==( const Pair_impl< Config >& lhs ) const + { + if( this == &lhs ) return true; + + return ( name_ == lhs.name_ ) && ( value_ == lhs.value_ ); + } + + // converts a C string, ie. 8 bit char array, to a string object + // + template < class String_type > + String_type to_str( const char* c_str ) + { + String_type result; + + for( const char* p = c_str; *p != 0; ++p ) + { + result += *p; + } + + return result; + } + + // + + namespace internal_ + { + template< typename T > + struct Type_to_type + { + }; + + template< class Value > + int get_value( const Value& value, Type_to_type< int > ) + { + return value.get_int(); + } + + template< class Value > + int64_t get_value( const Value& value, Type_to_type< int64_t > ) + { + return value.get_int64(); + } + + template< class Value > + uint64_t get_value( const Value& value, Type_to_type< uint64_t > ) + { + return value.get_uint64(); + } + + template< class Value > + double get_value( const Value& value, Type_to_type< double > ) + { + return value.get_real(); + } + + template< class Value > + typename Value::String_type get_value( const Value& value, Type_to_type< typename Value::String_type > ) + { + return value.get_str(); + } + + template< class Value > + typename Value::Array get_value( const Value& value, Type_to_type< typename Value::Array > ) + { + return value.get_array(); + } + + template< class Value > + typename Value::Object get_value( const Value& value, Type_to_type< typename Value::Object > ) + { + return value.get_obj(); + } + + template< class Value > + bool get_value( const Value& value, Type_to_type< bool > ) + { + return value.get_bool(); + } + } + + template< class Config > + template< typename T > + T Value_impl< Config >::get_value() const + { + return internal_::get_value( *this, internal_::Type_to_type< T >() ); + } +} + +#endif diff --git a/src/json/json_spirit_writer.cpp b/src/json/json_spirit_writer.cpp new file mode 100644 index 00000000..d24a632c --- /dev/null +++ b/src/json/json_spirit_writer.cpp @@ -0,0 +1,95 @@ +// Copyright John W. Wilkinson 2007 - 2009. +// Distributed under the MIT License, see accompanying file LICENSE.txt + +// json spirit version 4.03 + +#include "json_spirit_writer.h" +#include "json_spirit_writer_template.h" + +void json_spirit::write( const Value& value, std::ostream& os ) +{ + write_stream( value, os, false ); +} + +void json_spirit::write_formatted( const Value& value, std::ostream& os ) +{ + write_stream( value, os, true ); +} + +std::string json_spirit::write( const Value& value ) +{ + return write_string( value, false ); +} + +std::string json_spirit::write_formatted( const Value& value ) +{ + return write_string( value, true ); +} + +#ifndef BOOST_NO_STD_WSTRING + +void json_spirit::write( const wValue& value, std::wostream& os ) +{ + write_stream( value, os, false ); +} + +void json_spirit::write_formatted( const wValue& value, std::wostream& os ) +{ + write_stream( value, os, true ); +} + +std::wstring json_spirit::write( const wValue& value ) +{ + return write_string( value, false ); +} + +std::wstring json_spirit::write_formatted( const wValue& value ) +{ + return write_string( value, true ); +} + +#endif + +void json_spirit::write( const mValue& value, std::ostream& os ) +{ + write_stream( value, os, false ); +} + +void json_spirit::write_formatted( const mValue& value, std::ostream& os ) +{ + write_stream( value, os, true ); +} + +std::string json_spirit::write( const mValue& value ) +{ + return write_string( value, false ); +} + +std::string json_spirit::write_formatted( const mValue& value ) +{ + return write_string( value, true ); +} + +#ifndef BOOST_NO_STD_WSTRING + +void json_spirit::write( const wmValue& value, std::wostream& os ) +{ + write_stream( value, os, false ); +} + +void json_spirit::write_formatted( const wmValue& value, std::wostream& os ) +{ + write_stream( value, os, true ); +} + +std::wstring json_spirit::write( const wmValue& value ) +{ + return write_string( value, false ); +} + +std::wstring json_spirit::write_formatted( const wmValue& value ) +{ + return write_string( value, true ); +} + +#endif diff --git a/src/json/json_spirit_writer.h b/src/json/json_spirit_writer.h new file mode 100644 index 00000000..52e14068 --- /dev/null +++ b/src/json/json_spirit_writer.h @@ -0,0 +1,50 @@ +#ifndef JSON_SPIRIT_WRITER +#define JSON_SPIRIT_WRITER + +// Copyright John W. Wilkinson 2007 - 2009. +// Distributed under the MIT License, see accompanying file LICENSE.txt + +// json spirit version 4.03 + +#if defined(_MSC_VER) && (_MSC_VER >= 1020) +# pragma once +#endif + +#include "json_spirit_value.h" +#include + +namespace json_spirit +{ + // functions to convert JSON Values to text, + // the "formatted" versions add whitespace to format the output nicely + + void write ( const Value& value, std::ostream& os ); + void write_formatted( const Value& value, std::ostream& os ); + std::string write ( const Value& value ); + std::string write_formatted( const Value& value ); + +#ifndef BOOST_NO_STD_WSTRING + + void write ( const wValue& value, std::wostream& os ); + void write_formatted( const wValue& value, std::wostream& os ); + std::wstring write ( const wValue& value ); + std::wstring write_formatted( const wValue& value ); + +#endif + + void write ( const mValue& value, std::ostream& os ); + void write_formatted( const mValue& value, std::ostream& os ); + std::string write ( const mValue& value ); + std::string write_formatted( const mValue& value ); + +#ifndef BOOST_NO_STD_WSTRING + + void write ( const wmValue& value, std::wostream& os ); + void write_formatted( const wmValue& value, std::wostream& os ); + std::wstring write ( const wmValue& value ); + std::wstring write_formatted( const wmValue& value ); + +#endif +} + +#endif diff --git a/src/json/json_spirit_writer_template.h b/src/json/json_spirit_writer_template.h new file mode 100644 index 00000000..6b4978a1 --- /dev/null +++ b/src/json/json_spirit_writer_template.h @@ -0,0 +1,249 @@ +#ifndef JSON_SPIRIT_WRITER_TEMPLATE +#define JSON_SPIRIT_WRITER_TEMPLATE + +// Copyright John W. Wilkinson 2007 - 2009. +// Distributed under the MIT License, see accompanying file LICENSE.txt + +// json spirit version 4.03 + +#include "json_spirit_value.h" + +#include +#include +#include + +namespace json_spirit +{ + inline char to_hex_char( unsigned int c ) + { + assert( c <= 0xF ); + + const char ch = static_cast< char >( c ); + + if( ch < 10 ) return '0' + ch; + + return 'A' - 10 + ch; + } + + template< class String_type > + String_type non_printable_to_string( unsigned int c ) + { + // Silence the warning: typedef ‘Char_type’ locally defined but not used [-Wunused-local-typedefs] + // typedef typename String_type::value_type Char_type; + + String_type result( 6, '\\' ); + + result[1] = 'u'; + + result[ 5 ] = to_hex_char( c & 0x000F ); c >>= 4; + result[ 4 ] = to_hex_char( c & 0x000F ); c >>= 4; + result[ 3 ] = to_hex_char( c & 0x000F ); c >>= 4; + result[ 2 ] = to_hex_char( c & 0x000F ); + + return result; + } + + template< typename Char_type, class String_type > + bool add_esc_char( Char_type c, String_type& s ) + { + switch( c ) + { + case '"': s += to_str< String_type >( "\\\"" ); return true; + case '\\': s += to_str< String_type >( "\\\\" ); return true; + case '\b': s += to_str< String_type >( "\\b" ); return true; + case '\f': s += to_str< String_type >( "\\f" ); return true; + case '\n': s += to_str< String_type >( "\\n" ); return true; + case '\r': s += to_str< String_type >( "\\r" ); return true; + case '\t': s += to_str< String_type >( "\\t" ); return true; + } + + return false; + } + + template< class String_type > + String_type add_esc_chars( const String_type& s ) + { + typedef typename String_type::const_iterator Iter_type; + typedef typename String_type::value_type Char_type; + + String_type result; + + const Iter_type end( s.end() ); + + for( Iter_type i = s.begin(); i != end; ++i ) + { + const Char_type c( *i ); + + if( add_esc_char( c, result ) ) continue; + + const wint_t unsigned_c( ( c >= 0 ) ? c : 256 + c ); + + if( iswprint( unsigned_c ) ) + { + result += c; + } + else + { + result += non_printable_to_string< String_type >( unsigned_c ); + } + } + + return result; + } + + // this class generates the JSON text, + // it keeps track of the indentation level etc. + // + template< class Value_type, class Ostream_type > + class Generator + { + typedef typename Value_type::Config_type Config_type; + typedef typename Config_type::String_type String_type; + typedef typename Config_type::Object_type Object_type; + typedef typename Config_type::Array_type Array_type; + typedef typename String_type::value_type Char_type; + typedef typename Object_type::value_type Obj_member_type; + + public: + + Generator( const Value_type& value, Ostream_type& os, bool pretty ) + : os_( os ) + , indentation_level_( 0 ) + , pretty_( pretty ) + { + output( value ); + } + + private: + + void output( const Value_type& value ) + { + switch( value.type() ) + { + case obj_type: output( value.get_obj() ); break; + case array_type: output( value.get_array() ); break; + case str_type: output( value.get_str() ); break; + case bool_type: output( value.get_bool() ); break; + case int_type: output_int( value ); break; + + /// Bitcoin: Added std::fixed and changed precision from 16 to 8 + case real_type: os_ << std::showpoint << std::fixed << std::setprecision(8) + << value.get_real(); break; + + case null_type: os_ << "null"; break; + default: assert( false ); + } + } + + void output( const Object_type& obj ) + { + output_array_or_obj( obj, '{', '}' ); + } + + void output( const Array_type& arr ) + { + output_array_or_obj( arr, '[', ']' ); + } + + void output( const Obj_member_type& member ) + { + output( Config_type::get_name( member ) ); space(); + os_ << ':'; space(); + output( Config_type::get_value( member ) ); + } + + void output_int( const Value_type& value ) + { + if( value.is_uint64() ) + { + os_ << value.get_uint64(); + } + else + { + os_ << value.get_int64(); + } + } + + void output( const String_type& s ) + { + os_ << '"' << add_esc_chars( s ) << '"'; + } + + void output( bool b ) + { + os_ << to_str< String_type >( b ? "true" : "false" ); + } + + template< class T > + void output_array_or_obj( const T& t, Char_type start_char, Char_type end_char ) + { + os_ << start_char; new_line(); + + ++indentation_level_; + + for( typename T::const_iterator i = t.begin(); i != t.end(); ++i ) + { + indent(); output( *i ); + + typename T::const_iterator next = i; + + if( ++next != t.end()) + { + os_ << ','; + } + + new_line(); + } + + --indentation_level_; + + indent(); os_ << end_char; + } + + void indent() + { + if( !pretty_ ) return; + + for( int i = 0; i < indentation_level_; ++i ) + { + os_ << " "; + } + } + + void space() + { + if( pretty_ ) os_ << ' '; + } + + void new_line() + { + if( pretty_ ) os_ << '\n'; + } + + Generator& operator=( const Generator& ); // to prevent "assignment operator could not be generated" warning + + Ostream_type& os_; + int indentation_level_; + bool pretty_; + }; + + template< class Value_type, class Ostream_type > + void write_stream( const Value_type& value, Ostream_type& os, bool pretty ) + { + Generator< Value_type, Ostream_type >( value, os, pretty ); + } + + template< class Value_type > + typename Value_type::String_type write_string( const Value_type& value, bool pretty ) + { + typedef typename Value_type::String_type::value_type Char_type; + + std::basic_ostringstream< Char_type > os; + + write_stream( value, os, pretty ); + + return os.str(); + } +} + +#endif diff --git a/src/keys/key.cpp b/src/keys/key.cpp new file mode 100644 index 00000000..bc50689e --- /dev/null +++ b/src/keys/key.cpp @@ -0,0 +1,331 @@ +// Copyright (c) 2009-2016 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "keys/key.h" + +#include "crypto/common.h" +#include "crypto/hmac_sha512.h" +#include "keys/pubkey.h" +#include "utils/random.h" + +#include +#include + +static secp256k1_context* secp256k1_context_sign = NULL; + +/** These functions are taken from the libsecp256k1 distribution and are very ugly. */ +static int ec_privkey_import_der(const secp256k1_context* ctx, unsigned char *out32, const unsigned char *privkey, size_t privkeylen) { + const unsigned char *end = privkey + privkeylen; + int lenb = 0; + int len = 0; + memset(out32, 0, 32); + /* sequence header */ + if (end < privkey+1 || *privkey != 0x30) { + return 0; + } + privkey++; + /* sequence length constructor */ + if (end < privkey+1 || !(*privkey & 0x80)) { + return 0; + } + lenb = *privkey & ~0x80; privkey++; + if (lenb < 1 || lenb > 2) { + return 0; + } + if (end < privkey+lenb) { + return 0; + } + /* sequence length */ + len = privkey[lenb-1] | (lenb > 1 ? privkey[lenb-2] << 8 : 0); + privkey += lenb; + if (end < privkey+len) { + return 0; + } + /* sequence element 0: version number (=1) */ + if (end < privkey+3 || privkey[0] != 0x02 || privkey[1] != 0x01 || privkey[2] != 0x01) { + return 0; + } + privkey += 3; + /* sequence element 1: octet string, up to 32 bytes */ + if (end < privkey+2 || privkey[0] != 0x04 || privkey[1] > 0x20 || end < privkey+2+privkey[1]) { + return 0; + } + memcpy(out32 + 32 - privkey[1], privkey + 2, privkey[1]); + if (!secp256k1_ec_seckey_verify(ctx, out32)) { + memset(out32, 0, 32); + return 0; + } + return 1; +} + +static int ec_privkey_export_der(const secp256k1_context *ctx, unsigned char *privkey, size_t *privkeylen, const unsigned char *key32, int compressed) { + secp256k1_pubkey pubkey; + size_t pubkeylen = 0; + if (!secp256k1_ec_pubkey_create(ctx, &pubkey, key32)) { + *privkeylen = 0; + return 0; + } + if (compressed) { + static const unsigned char begin[] = { + 0x30,0x81,0xD3,0x02,0x01,0x01,0x04,0x20 + }; + static const unsigned char middle[] = { + 0xA0,0x81,0x85,0x30,0x81,0x82,0x02,0x01,0x01,0x30,0x2C,0x06,0x07,0x2A,0x86,0x48, + 0xCE,0x3D,0x01,0x01,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFE,0xFF,0xFF,0xFC,0x2F,0x30,0x06,0x04,0x01,0x00,0x04,0x01,0x07,0x04, + 0x21,0x02,0x79,0xBE,0x66,0x7E,0xF9,0xDC,0xBB,0xAC,0x55,0xA0,0x62,0x95,0xCE,0x87, + 0x0B,0x07,0x02,0x9B,0xFC,0xDB,0x2D,0xCE,0x28,0xD9,0x59,0xF2,0x81,0x5B,0x16,0xF8, + 0x17,0x98,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFE,0xBA,0xAE,0xDC,0xE6,0xAF,0x48,0xA0,0x3B,0xBF,0xD2,0x5E, + 0x8C,0xD0,0x36,0x41,0x41,0x02,0x01,0x01,0xA1,0x24,0x03,0x22,0x00 + }; + unsigned char *ptr = privkey; + memcpy(ptr, begin, sizeof(begin)); ptr += sizeof(begin); + memcpy(ptr, key32, 32); ptr += 32; + memcpy(ptr, middle, sizeof(middle)); ptr += sizeof(middle); + pubkeylen = 33; + secp256k1_ec_pubkey_serialize(ctx, ptr, &pubkeylen, &pubkey, SECP256K1_EC_COMPRESSED); + ptr += pubkeylen; + *privkeylen = ptr - privkey; + } else { + static const unsigned char begin[] = { + 0x30,0x82,0x01,0x13,0x02,0x01,0x01,0x04,0x20 + }; + static const unsigned char middle[] = { + 0xA0,0x81,0xA5,0x30,0x81,0xA2,0x02,0x01,0x01,0x30,0x2C,0x06,0x07,0x2A,0x86,0x48, + 0xCE,0x3D,0x01,0x01,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFE,0xFF,0xFF,0xFC,0x2F,0x30,0x06,0x04,0x01,0x00,0x04,0x01,0x07,0x04, + 0x41,0x04,0x79,0xBE,0x66,0x7E,0xF9,0xDC,0xBB,0xAC,0x55,0xA0,0x62,0x95,0xCE,0x87, + 0x0B,0x07,0x02,0x9B,0xFC,0xDB,0x2D,0xCE,0x28,0xD9,0x59,0xF2,0x81,0x5B,0x16,0xF8, + 0x17,0x98,0x48,0x3A,0xDA,0x77,0x26,0xA3,0xC4,0x65,0x5D,0xA4,0xFB,0xFC,0x0E,0x11, + 0x08,0xA8,0xFD,0x17,0xB4,0x48,0xA6,0x85,0x54,0x19,0x9C,0x47,0xD0,0x8F,0xFB,0x10, + 0xD4,0xB8,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFE,0xBA,0xAE,0xDC,0xE6,0xAF,0x48,0xA0,0x3B,0xBF,0xD2,0x5E, + 0x8C,0xD0,0x36,0x41,0x41,0x02,0x01,0x01,0xA1,0x44,0x03,0x42,0x00 + }; + unsigned char *ptr = privkey; + memcpy(ptr, begin, sizeof(begin)); ptr += sizeof(begin); + memcpy(ptr, key32, 32); ptr += 32; + memcpy(ptr, middle, sizeof(middle)); ptr += sizeof(middle); + pubkeylen = 65; + secp256k1_ec_pubkey_serialize(ctx, ptr, &pubkeylen, &pubkey, SECP256K1_EC_UNCOMPRESSED); + ptr += pubkeylen; + *privkeylen = ptr - privkey; + } + return 1; +} + +bool CKey::Check(const unsigned char *vch) { + return secp256k1_ec_seckey_verify(secp256k1_context_sign, vch); +} + +void CKey::MakeNewKey(bool fCompressedIn) { + RandAddSeedPerfmon(); + do { + GetRandBytes(vch, sizeof(vch)); + } while (!Check(vch)); + fValid = true; + fCompressed = fCompressedIn; +} + +bool CKey::SetPrivKey(const CPrivKey &privkey, bool fCompressedIn) { + if (!ec_privkey_import_der(secp256k1_context_sign, (unsigned char*)begin(), &privkey[0], privkey.size())) + return false; + fCompressed = fCompressedIn; + fValid = true; + return true; +} + +CPrivKey CKey::GetPrivKey() const { + assert(fValid); + CPrivKey privkey; + int ret; + size_t privkeylen; + privkey.resize(279); + privkeylen = 279; + ret = ec_privkey_export_der(secp256k1_context_sign, (unsigned char*)&privkey[0], &privkeylen, begin(), fCompressed ? SECP256K1_EC_COMPRESSED : SECP256K1_EC_UNCOMPRESSED); + assert(ret); + privkey.resize(privkeylen); + return privkey; +} + +CPubKey CKey::GetPubKey() const { + assert(fValid); + secp256k1_pubkey pubkey; + size_t clen = 65; + CPubKey result; + int ret = secp256k1_ec_pubkey_create(secp256k1_context_sign, &pubkey, begin()); + assert(ret); + secp256k1_ec_pubkey_serialize(secp256k1_context_sign, (unsigned char*)result.begin(), &clen, &pubkey, fCompressed ? SECP256K1_EC_COMPRESSED : SECP256K1_EC_UNCOMPRESSED); + assert(result.size() == clen); + assert(result.IsValid()); + return result; +} + +bool CKey::Sign(const uint256 &hash, std::vector& vchSig, uint32_t test_case) const { + if (!fValid) + return false; + vchSig.resize(72); + size_t nSigLen = 72; + unsigned char extra_entropy[32] = {0}; + WriteLE32(extra_entropy, test_case); + secp256k1_ecdsa_signature sig; + int ret = secp256k1_ecdsa_sign(secp256k1_context_sign, &sig, hash.begin(), begin(), secp256k1_nonce_function_rfc6979, test_case ? extra_entropy : NULL); + assert(ret); + secp256k1_ecdsa_signature_serialize_der(secp256k1_context_sign, (unsigned char*)&vchSig[0], &nSigLen, &sig); + vchSig.resize(nSigLen); + return true; +} + +bool CKey::VerifyPubKey(const CPubKey& pubkey) const { + if (pubkey.IsCompressed() != fCompressed) { + return false; + } + unsigned char rnd[8]; + std::string str = "Bitcoin key verification\n"; + GetRandBytes(rnd, sizeof(rnd)); + uint256 hash; + CHash256().Write((unsigned char*)str.data(), str.size()).Write(rnd, sizeof(rnd)).Finalize(hash.begin()); + std::vector vchSig; + Sign(hash, vchSig); + return pubkey.Verify(hash, vchSig); +} + +bool CKey::SignCompact(const uint256 &hash, std::vector& vchSig) const { + if (!fValid) + return false; + vchSig.resize(65); + int rec = -1; + secp256k1_ecdsa_recoverable_signature sig; + int ret = secp256k1_ecdsa_sign_recoverable(secp256k1_context_sign, &sig, hash.begin(), begin(), secp256k1_nonce_function_rfc6979, NULL); + assert(ret); + secp256k1_ecdsa_recoverable_signature_serialize_compact(secp256k1_context_sign, (unsigned char*)&vchSig[1], &rec, &sig); + assert(ret); + assert(rec != -1); + vchSig[0] = 27 + rec + (fCompressed ? 4 : 0); + return true; +} + +bool CKey::Load(CPrivKey &privkey, CPubKey &vchPubKey, bool fSkipCheck=false) { + if (!ec_privkey_import_der(secp256k1_context_sign, (unsigned char*)begin(), &privkey[0], privkey.size())) + return false; + fCompressed = vchPubKey.IsCompressed(); + fValid = true; + + if (fSkipCheck) + return true; + + return VerifyPubKey(vchPubKey); +} + +bool CKey::Derive(CKey& keyChild, ChainCode &ccChild, unsigned int nChild, const ChainCode& cc) const { + assert(IsValid()); + assert(IsCompressed()); + unsigned char out[64]; + LockObject(out); + if ((nChild >> 31) == 0) { + CPubKey pubkey = GetPubKey(); + assert(pubkey.begin() + 33 == pubkey.end()); + BIP32Hash(cc, nChild, *pubkey.begin(), pubkey.begin()+1, out); + } else { + assert(begin() + 32 == end()); + BIP32Hash(cc, nChild, 0, begin(), out); + } + memcpy(ccChild.begin(), out+32, 32); + memcpy((unsigned char*)keyChild.begin(), begin(), 32); + bool ret = secp256k1_ec_privkey_tweak_add(secp256k1_context_sign, (unsigned char*)keyChild.begin(), out); + UnlockObject(out); + keyChild.fCompressed = true; + keyChild.fValid = ret; + return ret; +} + +bool CExtKey::Derive(CExtKey &out, unsigned int nChild) const { + out.nDepth = nDepth + 1; + CKeyID id = key.GetPubKey().GetID(); + memcpy(&out.vchFingerprint[0], &id, 4); + out.nChild = nChild; + return key.Derive(out.key, out.chaincode, nChild, chaincode); +} + +void CExtKey::SetMaster(const unsigned char *seed, unsigned int nSeedLen) { + static const unsigned char hashkey[] = {'B','i','t','c','o','i','n',' ','s','e','e','d'}; + unsigned char out[64]; + LockObject(out); + CHMAC_SHA512(hashkey, sizeof(hashkey)).Write(seed, nSeedLen).Finalize(out); + key.Set(&out[0], &out[32], true); + memcpy(chaincode.begin(), &out[32], 32); + UnlockObject(out); + nDepth = 0; + nChild = 0; + memset(vchFingerprint, 0, sizeof(vchFingerprint)); +} + +CExtPubKey CExtKey::Neuter() const { + CExtPubKey ret; + ret.nDepth = nDepth; + memcpy(&ret.vchFingerprint[0], &vchFingerprint[0], 4); + ret.nChild = nChild; + ret.pubkey = key.GetPubKey(); + ret.chaincode = chaincode; + return ret; +} + +void CExtKey::Encode(unsigned char code[74]) const { + code[0] = nDepth; + memcpy(code+1, vchFingerprint, 4); + code[5] = (nChild >> 24) & 0xFF; code[6] = (nChild >> 16) & 0xFF; + code[7] = (nChild >> 8) & 0xFF; code[8] = (nChild >> 0) & 0xFF; + memcpy(code+9, chaincode.begin(), 32); + code[41] = 0; + assert(key.size() == 32); + memcpy(code+42, key.begin(), 32); +} + +void CExtKey::Decode(const unsigned char code[74]) { + nDepth = code[0]; + memcpy(vchFingerprint, code+1, 4); + nChild = (code[5] << 24) | (code[6] << 16) | (code[7] << 8) | code[8]; + memcpy(chaincode.begin(), code+9, 32); + key.Set(code+42, code+74, true); +} + +bool ECC_InitSanityCheck() { + CKey key; + key.MakeNewKey(true); + CPubKey pubkey = key.GetPubKey(); + return key.VerifyPubKey(pubkey); +} + + +void ECC_Start() { + assert(secp256k1_context_sign == NULL); + + secp256k1_context *ctx = secp256k1_context_create(SECP256K1_CONTEXT_SIGN); + assert(ctx != NULL); + + { + // Pass in a random blinding seed to the secp256k1 context. + unsigned char seed[32]; + LockObject(seed); + GetRandBytes(seed, 32); + bool ret = secp256k1_context_randomize(ctx, seed); + assert(ret); + UnlockObject(seed); + } + + secp256k1_context_sign = ctx; +} + +void ECC_Stop() { + secp256k1_context *ctx = secp256k1_context_sign; + secp256k1_context_sign = NULL; + + if (ctx) { + secp256k1_context_destroy(ctx); + } +} diff --git a/src/keys/key.h b/src/keys/key.h new file mode 100644 index 00000000..2a120386 --- /dev/null +++ b/src/keys/key.h @@ -0,0 +1,184 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2016 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_KEY_H +#define BITCOIN_KEY_H + +#include "keys/pubkey.h" +#include "utils/allocators.h" +#include "utils/serialize.h" +#include "structs/uint256.h" + +#include +#include + + +/** + * secp256k1: + * const unsigned int PRIVATE_KEY_SIZE = 279; + * const unsigned int PUBLIC_KEY_SIZE = 65; + * const unsigned int SIGNATURE_SIZE = 72; + * + * see www.keylength.com + * script supports up to 75 for single byte push + */ + +/** + * secure_allocator is defined in allocators.h + * CPrivKey is a serialized private key, with all parameters included (279 bytes) + */ +typedef std::vector > CPrivKey; + +/** An encapsulated private key. */ +class CKey +{ +private: + //! Whether this private key is valid. We check for correctness when modifying the key + //! data, so fValid should always correspond to the actual state. + bool fValid; + + //! Whether the public key corresponding to this private key is (to be) compressed. + bool fCompressed; + + //! The actual byte data + unsigned char vch[32]; + + //! Check whether the 32-byte array pointed to be vch is valid keydata. + bool static Check(const unsigned char* vch); + +public: + //! Construct an invalid private key. + CKey() : fValid(false), fCompressed(false) + { + LockObject(vch); + } + + //! Copy constructor. This is necessary because of memlocking. + CKey(const CKey& secret) : fValid(secret.fValid), fCompressed(secret.fCompressed) + { + LockObject(vch); + memcpy(vch, secret.vch, sizeof(vch)); + } + + //! Destructor (again necessary because of memlocking). + ~CKey() + { + UnlockObject(vch); + } + + friend bool operator==(const CKey& a, const CKey& b) + { + return a.fCompressed == b.fCompressed && a.size() == b.size() && + memcmp(&a.vch[0], &b.vch[0], a.size()) == 0; + } + + //! Initialize using begin and end iterators to byte data. + template + void Set(const T pbegin, const T pend, bool fCompressedIn) + { + if (pend - pbegin != 32) { + fValid = false; + return; + } + if (Check(&pbegin[0])) { + memcpy(vch, (unsigned char*)&pbegin[0], 32); + fValid = true; + fCompressed = fCompressedIn; + } else { + fValid = false; + } + } + + //! Simple read-only vector-like interface. + unsigned int size() const { return (fValid ? 32 : 0); } + const unsigned char* begin() const { return vch; } + const unsigned char* end() const { return vch + size(); } + + //! Check whether this private key is valid. + bool IsValid() const { return fValid; } + + //! Check whether the public key corresponding to this private key is (to be) compressed. + bool IsCompressed() const { return fCompressed; } + + //! Initialize from a CPrivKey (serialized OpenSSL private key data). + bool SetPrivKey(const CPrivKey& vchPrivKey, bool fCompressed); + + //! Generate a new private key using a cryptographic PRNG. + void MakeNewKey(bool fCompressed); + + /** + * Convert the private key to a CPrivKey (serialized OpenSSL private key data). + * This is expensive. + */ + CPrivKey GetPrivKey() const; + + /** + * Compute the public key from a private key. + * This is expensive. + */ + CPubKey GetPubKey() const; + + /** + * Create a DER-serialized signature. + * The test_case parameter tweaks the deterministic nonce. + */ + bool Sign(const uint256& hash, std::vector& vchSig, uint32_t test_case = 0) const; + + /** + * Create a compact signature (65 bytes), which allows reconstructing the used public key. + * The format is one header byte, followed by two times 32 bytes for the serialized r and s values. + * The header byte: 0x1B = first key with even y, 0x1C = first key with odd y, + * 0x1D = second key with even y, 0x1E = second key with odd y, + * add 0x04 for compressed keys. + */ + bool SignCompact(const uint256& hash, std::vector& vchSig) const; + + //! Derive BIP32 child key. + bool Derive(CKey& keyChild, ChainCode &ccChild, unsigned int nChild, const ChainCode& cc) const; + + /** + * Verify thoroughly whether a private key and a public key match. + * This is done using a different mechanism than just regenerating it. + */ + bool VerifyPubKey(const CPubKey& vchPubKey) const; + + //! Load private key and check that public key matches. + bool Load(CPrivKey& privkey, CPubKey& vchPubKey, bool fSkipCheck); + + //! Check whether an element of a signature (r or s) is valid. + static bool CheckSignatureElement(const unsigned char* vch, int len, bool half); +}; + +struct CExtKey { + unsigned char nDepth; + unsigned char vchFingerprint[4]; + unsigned int nChild; + ChainCode chaincode; + CKey key; + + friend bool operator==(const CExtKey& a, const CExtKey& b) + { + return a.nDepth == b.nDepth && memcmp(&a.vchFingerprint[0], &b.vchFingerprint[0], 4) == 0 && a.nChild == b.nChild && + a.chaincode == b.chaincode && a.key == b.key; + } + + void Encode(unsigned char code[74]) const; + void Decode(const unsigned char code[74]); + bool Derive(CExtKey& out, unsigned int nChild) const; + CExtPubKey Neuter() const; + void SetMaster(const unsigned char* seed, unsigned int nSeedLen); +}; + +/** Initialize the elliptic curve support. May not be called twice without calling ECC_Stop first. */ +void ECC_Start(void); + +/** Deinitialize the elliptic curve support. No-op if ECC_Start wasn't called first. */ +void ECC_Stop(void); + +/** Check that required EC support is available at runtime. */ +bool ECC_InitSanityCheck(void); + +#endif // BITCOIN_KEY_H diff --git a/src/keys/pubkey.cpp b/src/keys/pubkey.cpp new file mode 100644 index 00000000..77e3e76d --- /dev/null +++ b/src/keys/pubkey.cpp @@ -0,0 +1,306 @@ +// Copyright (c) 2009-2016 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "keys/pubkey.h" + +#include +#include + +namespace +{ +/* Global secp256k1_context object used for verification. */ +secp256k1_context* secp256k1_context_verify = NULL; +} + +/** This function is taken from the libsecp256k1 distribution and implements + * DER parsing for ECDSA signatures, while supporting an arbitrary subset of + * format violations. + * + * Supported violations include negative integers, excessive padding, garbage + * at the end, and overly long length descriptors. This is safe to use in + * Bitcoin because since the activation of BIP66, signatures are verified to be + * strict DER before being passed to this module, and we know it supports all + * violations present in the blockchain before that point. + */ +static int ecdsa_signature_parse_der_lax(const secp256k1_context* ctx, secp256k1_ecdsa_signature* sig, const unsigned char *input, size_t inputlen) { + size_t rpos, rlen, spos, slen; + size_t pos = 0; + size_t lenbyte; + unsigned char tmpsig[64] = {0}; + int overflow = 0; + + /* Hack to initialize sig with a correctly-parsed but invalid signature. */ + secp256k1_ecdsa_signature_parse_compact(ctx, sig, tmpsig); + + /* Sequence tag byte */ + if (pos == inputlen || input[pos] != 0x30) { + return 0; + } + pos++; + + /* Sequence length bytes */ + if (pos == inputlen) { + return 0; + } + lenbyte = input[pos++]; + if (lenbyte & 0x80) { + lenbyte -= 0x80; + if (pos + lenbyte > inputlen) { + return 0; + } + pos += lenbyte; + } + + /* Integer tag byte for R */ + if (pos == inputlen || input[pos] != 0x02) { + return 0; + } + pos++; + + /* Integer length for R */ + if (pos == inputlen) { + return 0; + } + lenbyte = input[pos++]; + if (lenbyte & 0x80) { + lenbyte -= 0x80; + if (pos + lenbyte > inputlen) { + return 0; + } + while (lenbyte > 0 && input[pos] == 0) { + pos++; + lenbyte--; + } + if (lenbyte >= sizeof(size_t)) { + return 0; + } + rlen = 0; + while (lenbyte > 0) { + rlen = (rlen << 8) + input[pos]; + pos++; + lenbyte--; + } + } else { + rlen = lenbyte; + } + if (rlen > inputlen - pos) { + return 0; + } + rpos = pos; + pos += rlen; + + /* Integer tag byte for S */ + if (pos == inputlen || input[pos] != 0x02) { + return 0; + } + pos++; + + /* Integer length for S */ + if (pos == inputlen) { + return 0; + } + lenbyte = input[pos++]; + if (lenbyte & 0x80) { + lenbyte -= 0x80; + if (pos + lenbyte > inputlen) { + return 0; + } + while (lenbyte > 0 && input[pos] == 0) { + pos++; + lenbyte--; + } + if (lenbyte >= sizeof(size_t)) { + return 0; + } + slen = 0; + while (lenbyte > 0) { + slen = (slen << 8) + input[pos]; + pos++; + lenbyte--; + } + } else { + slen = lenbyte; + } + if (slen > inputlen - pos) { + return 0; + } + spos = pos; + pos += slen; + + /* Ignore leading zeroes in R */ + while (rlen > 0 && input[rpos] == 0) { + rlen--; + rpos++; + } + /* Copy R value */ + if (rlen > 32) { + overflow = 1; + } else { + memcpy(tmpsig + 32 - rlen, input + rpos, rlen); + } + + /* Ignore leading zeroes in S */ + while (slen > 0 && input[spos] == 0) { + slen--; + spos++; + } + /* Copy S value */ + if (slen > 32) { + overflow = 1; + } else { + memcpy(tmpsig + 64 - slen, input + spos, slen); + } + + if (!overflow) { + overflow = !secp256k1_ecdsa_signature_parse_compact(ctx, sig, tmpsig); + } + if (overflow) { + /* Overwrite the result again with a correctly-parsed but invalid + signature if parsing failed. */ + memset(tmpsig, 0, 64); + secp256k1_ecdsa_signature_parse_compact(ctx, sig, tmpsig); + } + return 1; +} + + +bool CPubKey::Verify(const uint256 &hash, const std::vector& vchSig) const { + if (!IsValid()) + return false; + secp256k1_pubkey pubkey; + secp256k1_ecdsa_signature sig; + if (!secp256k1_ec_pubkey_parse(secp256k1_context_verify, &pubkey, &(*this)[0], size())) { + return false; + } + if (vchSig.size() == 0) { + return false; + } + if (!ecdsa_signature_parse_der_lax(secp256k1_context_verify, &sig, &vchSig[0], vchSig.size())) { + return false; + } + /* libsecp256k1's ECDSA verification requires lower-S signatures, which have + * not historically been enforced in Bitcoin, so normalize them first. */ + secp256k1_ecdsa_signature_normalize(secp256k1_context_verify, &sig, &sig); + return secp256k1_ecdsa_verify(secp256k1_context_verify, &sig, hash.begin(), &pubkey); +} + +bool CPubKey::RecoverCompact(const uint256 &hash, const std::vector& vchSig) { + if (vchSig.size() != 65) + return false; + int recid = (vchSig[0] - 27) & 3; + bool fComp = ((vchSig[0] - 27) & 4) != 0; + secp256k1_pubkey pubkey; + secp256k1_ecdsa_recoverable_signature sig; + if (!secp256k1_ecdsa_recoverable_signature_parse_compact(secp256k1_context_verify, &sig, &vchSig[1], recid)) { + return false; + } + if (!secp256k1_ecdsa_recover(secp256k1_context_verify, &pubkey, &sig, hash.begin())) { + return false; + } + unsigned char pub[65]; + size_t publen = 65; + secp256k1_ec_pubkey_serialize(secp256k1_context_verify, pub, &publen, &pubkey, fComp ? SECP256K1_EC_COMPRESSED : SECP256K1_EC_UNCOMPRESSED); + Set(pub, pub + publen); + return true; +} + +bool CPubKey::IsFullyValid() const { + if (!IsValid()) + return false; + secp256k1_pubkey pubkey; + return secp256k1_ec_pubkey_parse(secp256k1_context_verify, &pubkey, &(*this)[0], size()); + return true; +} + +bool CPubKey::Decompress() { + if (!IsValid()) + return false; + secp256k1_pubkey pubkey; + if (!secp256k1_ec_pubkey_parse(secp256k1_context_verify, &pubkey, &(*this)[0], size())) { + return false; + } + unsigned char pub[65]; + size_t publen = 65; + secp256k1_ec_pubkey_serialize(secp256k1_context_verify, pub, &publen, &pubkey, SECP256K1_EC_UNCOMPRESSED); + Set(pub, pub + publen); + return true; +} + +bool CPubKey::Derive(CPubKey& pubkeyChild, ChainCode &ccChild, unsigned int nChild, const ChainCode& cc) const { + assert(IsValid()); + assert((nChild >> 31) == 0); + assert(begin() + 33 == end()); + unsigned char out[64]; + BIP32Hash(cc, nChild, *begin(), begin()+1, out); + memcpy(ccChild.begin(), out+32, 32); + secp256k1_pubkey pubkey; + if (!secp256k1_ec_pubkey_parse(secp256k1_context_verify, &pubkey, &(*this)[0], size())) { + return false; + } + if (!secp256k1_ec_pubkey_tweak_add(secp256k1_context_verify, &pubkey, out)) { + return false; + } + unsigned char pub[33]; + size_t publen = 33; + secp256k1_ec_pubkey_serialize(secp256k1_context_verify, pub, &publen, &pubkey, SECP256K1_EC_COMPRESSED); + pubkeyChild.Set(pub, pub + publen); + return true; +} + +void CExtPubKey::Encode(unsigned char code[74]) const { + code[0] = nDepth; + memcpy(code+1, vchFingerprint, 4); + code[5] = (nChild >> 24) & 0xFF; code[6] = (nChild >> 16) & 0xFF; + code[7] = (nChild >> 8) & 0xFF; code[8] = (nChild >> 0) & 0xFF; + memcpy(code+9, chaincode.begin(), 32); + assert(pubkey.size() == 33); + memcpy(code+41, pubkey.begin(), 33); +} + +void CExtPubKey::Decode(const unsigned char code[74]) { + nDepth = code[0]; + memcpy(vchFingerprint, code+1, 4); + nChild = (code[5] << 24) | (code[6] << 16) | (code[7] << 8) | code[8]; + memcpy(chaincode.begin(), code+9, 32); + pubkey.Set(code+41, code+74); +} + +bool CExtPubKey::Derive(CExtPubKey &out, unsigned int nChild) const { + out.nDepth = nDepth + 1; + CKeyID id = pubkey.GetID(); + memcpy(&out.vchFingerprint[0], &id, 4); + out.nChild = nChild; + return pubkey.Derive(out.pubkey, out.chaincode, nChild, chaincode); +} + +/* static */ bool CPubKey::CheckLowS(const std::vector& vchSig) { + secp256k1_ecdsa_signature sig; + if (!ecdsa_signature_parse_der_lax(secp256k1_context_verify, &sig, &vchSig[0], vchSig.size())) { + return false; + } + return (!secp256k1_ecdsa_signature_normalize(secp256k1_context_verify, NULL, &sig)); +} + +/* static */ int ECCVerifyHandle::refcount = 0; + +ECCVerifyHandle::ECCVerifyHandle() +{ + if (refcount == 0) { + assert(secp256k1_context_verify == NULL); + secp256k1_context_verify = secp256k1_context_create(SECP256K1_CONTEXT_VERIFY); + assert(secp256k1_context_verify != NULL); + } + refcount++; +} + +ECCVerifyHandle::~ECCVerifyHandle() +{ + refcount--; + if (refcount == 0) { + assert(secp256k1_context_verify != NULL); + secp256k1_context_destroy(secp256k1_context_verify); + secp256k1_context_verify = NULL; + } +} diff --git a/src/keys/pubkey.h b/src/keys/pubkey.h new file mode 100644 index 00000000..9d8123a0 --- /dev/null +++ b/src/keys/pubkey.h @@ -0,0 +1,225 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2016 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_PUBKEY_H +#define BITCOIN_PUBKEY_H + +#include "structs/hash.h" +#include "utils/serialize.h" +#include "structs/uint256.h" + +#include +#include + +/** + * secp256k1: + * const unsigned int PRIVATE_KEY_SIZE = 279; + * const unsigned int PUBLIC_KEY_SIZE = 65; + * const unsigned int SIGNATURE_SIZE = 72; + * + * see www.keylength.com + * script supports up to 75 for single byte push + */ + +/** A reference to a CKey: the Hash160 of its serialized public key */ +class CKeyID : public uint160 +{ +public: + CKeyID() : uint160(0) {} + CKeyID(const uint160& in) : uint160(in) {} +}; + +typedef uint256 ChainCode; + +/** An encapsulated public key. */ +class CPubKey +{ +private: + + /** + * Just store the serialized data. + * Its length can very cheaply be computed from the first byte. + */ + unsigned char vch[65]; + + //! Compute the length of a pubkey with a given first byte. + unsigned int static GetLen(unsigned char chHeader) + { + if (chHeader == 2 || chHeader == 3) + return 33; + if (chHeader == 4 || chHeader == 6 || chHeader == 7) + return 65; + return 0; + } + + //! Set this key data to be invalid + void Invalidate() + { + vch[0] = 0xFF; + } + +public: + //! Construct an invalid public key. + CPubKey() + { + Invalidate(); + } + + //! Initialize a public key using begin/end iterators to byte data. + template + void Set(const T pbegin, const T pend) + { + int len = pend == pbegin ? 0 : GetLen(pbegin[0]); + if (len && len == (pend - pbegin)) + memcpy(vch, (unsigned char*)&pbegin[0], len); + else + Invalidate(); + } + + //! Construct a public key using begin/end iterators to byte data. + template + CPubKey(const T pbegin, const T pend) + { + Set(pbegin, pend); + } + + //! Construct a public key from a byte vector. + CPubKey(const std::vector& vch) + { + Set(vch.begin(), vch.end()); + } + + //! Simple read-only vector-like interface to the pubkey data. + unsigned int size() const { return GetLen(vch[0]); } + const unsigned char* begin() const { return vch; } + const unsigned char* end() const { return vch + size(); } + const unsigned char& operator[](unsigned int pos) const { return vch[pos]; } + + //! Comparator implementation. + friend bool operator==(const CPubKey& a, const CPubKey& b) + { + return a.vch[0] == b.vch[0] && + memcmp(a.vch, b.vch, a.size()) == 0; + } + friend bool operator!=(const CPubKey& a, const CPubKey& b) + { + return !(a == b); + } + friend bool operator<(const CPubKey& a, const CPubKey& b) + { + return a.vch[0] < b.vch[0] || + (a.vch[0] == b.vch[0] && memcmp(a.vch, b.vch, a.size()) < 0); + } + + //! Implement serialization, as if this was a byte vector. + unsigned int GetSerializeSize(int nType, int nVersion) const + { + return size() + 1; + } + template + void Serialize(Stream& s, int nType, int nVersion) const + { + unsigned int len = size(); + ::WriteCompactSize(s, len); + s.write((char*)vch, len); + } + template + void Unserialize(Stream& s, int nType, int nVersion) + { + unsigned int len = ::ReadCompactSize(s); + if (len <= 65) { + s.read((char*)vch, len); + } else { + // invalid pubkey, skip available data + char dummy; + while (len--) + s.read(&dummy, 1); + Invalidate(); + } + } + + //! Get the KeyID of this public key (hash of its serialization) + CKeyID GetID() const + { + return CKeyID(Hash160(vch, vch + size())); + } + + //! Get the 256-bit hash of this public key. + uint256 GetHash() const + { + return Hash(vch, vch + size()); + } + + /* + * Check syntactic correctness. + * + * Note that this is consensus critical as CheckSig() calls it! + */ + bool IsValid() const + { + return size() > 0; + } + + //! fully validate whether this is a valid public key (more expensive than IsValid()) + bool IsFullyValid() const; + + //! Check whether this is a compressed public key. + bool IsCompressed() const + { + return size() == 33; + } + + /** + * Verify a DER signature (~72 bytes). + * If this public key is not fully valid, the return value will be false. + */ + bool Verify(const uint256& hash, const std::vector& vchSig) const; + + /** + * Check whether a signature is normalized (lower-S). + */ + static bool CheckLowS(const std::vector& vchSig); + + //! Recover a public key from a compact signature. + bool RecoverCompact(const uint256& hash, const std::vector& vchSig); + + //! Turn this public key into an uncompressed public key. + bool Decompress(); + + //! Derive BIP32 child pubkey. + bool Derive(CPubKey& pubkeyChild, ChainCode &ccChild, unsigned int nChild, const ChainCode& cc) const; +}; + +struct CExtPubKey { + unsigned char nDepth; + unsigned char vchFingerprint[4]; + unsigned int nChild; + ChainCode chaincode; + CPubKey pubkey; + + friend bool operator==(const CExtPubKey &a, const CExtPubKey &b) + { + return a.nDepth == b.nDepth && memcmp(&a.vchFingerprint[0], &b.vchFingerprint[0], 4) == 0 && a.nChild == b.nChild && + a.chaincode == b.chaincode && a.pubkey == b.pubkey; + } + + void Encode(unsigned char code[74]) const; + void Decode(const unsigned char code[74]); + bool Derive(CExtPubKey& out, unsigned int nChild) const; +}; + +/** Users of this module must hold an ECCVerifyHandle. The constructor and + * destructor of these are not allowed to run in parallel, though. */ +class ECCVerifyHandle +{ + static int refcount; + +public: + ECCVerifyHandle(); + ~ECCVerifyHandle(); +}; + +#endif // BITCOIN_PUBKEY_H diff --git a/src/leveldb/.gitignore b/src/leveldb/.gitignore new file mode 100644 index 00000000..71d87a4e --- /dev/null +++ b/src/leveldb/.gitignore @@ -0,0 +1,13 @@ +build_config.mk +*.a +*.o +*.dylib* +*.so +*.so.* +*_test +db_bench +leveldbutil +Release +Debug +Benchmark +vs2010.* diff --git a/src/leveldb/AUTHORS b/src/leveldb/AUTHORS new file mode 100644 index 00000000..2439d7a4 --- /dev/null +++ b/src/leveldb/AUTHORS @@ -0,0 +1,12 @@ +# Names should be added to this file like so: +# Name or Organization + +Google Inc. + +# Initial version authors: +Jeffrey Dean +Sanjay Ghemawat + +# Partial list of contributors: +Kevin Regan +Johan Bilien diff --git a/src/leveldb/CONTRIBUTING.md b/src/leveldb/CONTRIBUTING.md new file mode 100644 index 00000000..cd600ff4 --- /dev/null +++ b/src/leveldb/CONTRIBUTING.md @@ -0,0 +1,36 @@ +# Contributing + +We'd love to accept your code patches! However, before we can take them, we +have to jump a couple of legal hurdles. + +## Contributor License Agreements + +Please fill out either the individual or corporate Contributor License +Agreement as appropriate. + +* If you are an individual writing original source code and you're sure you +own the intellectual property, then sign an [individual CLA](https://developers.google.com/open-source/cla/individual). +* If you work for a company that wants to allow you to contribute your work, +then sign a [corporate CLA](https://developers.google.com/open-source/cla/corporate). + +Follow either of the two links above to access the appropriate CLA and +instructions for how to sign and return it. + +## Submitting a Patch + +1. Sign the contributors license agreement above. +2. Decide which code you want to submit. A submission should be a set of changes +that addresses one issue in the [issue tracker](https://github.com/google/leveldb/issues). +Please don't mix more than one logical change per submission, because it makes +the history hard to follow. If you want to make a change +(e.g. add a sample or feature) that doesn't have a corresponding issue in the +issue tracker, please create one. +3. **Submitting**: When you are ready to submit, send us a Pull Request. Be +sure to include the issue number you fixed and the name you used to sign +the CLA. + +## Writing Code ## + +If your contribution contains code, please make sure that it follows +[the style guide](http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml). +Otherwise we will have to ask you to make changes, and that's no fun for anyone. diff --git a/src/leveldb/LICENSE b/src/leveldb/LICENSE new file mode 100644 index 00000000..8e80208c --- /dev/null +++ b/src/leveldb/LICENSE @@ -0,0 +1,27 @@ +Copyright (c) 2011 The LevelDB Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/src/leveldb/Makefile b/src/leveldb/Makefile new file mode 100644 index 00000000..2bd2cadc --- /dev/null +++ b/src/leveldb/Makefile @@ -0,0 +1,227 @@ +# Copyright (c) 2011 The LevelDB Authors. All rights reserved. +# Use of this source code is governed by a BSD-style license that can be +# found in the LICENSE file. See the AUTHORS file for names of contributors. + +#----------------------------------------------- +# Uncomment exactly one of the lines labelled (A), (B), and (C) below +# to switch between compilation modes. + +# (A) Production use (optimized mode) +OPT ?= -O2 -DNDEBUG +# (B) Debug mode, w/ full line-level debugging symbols +# OPT ?= -g2 +# (C) Profiling mode: opt, but w/debugging symbols +# OPT ?= -O2 -g2 -DNDEBUG +#----------------------------------------------- + +# detect what platform we're building on +$(shell CC="$(CC)" CXX="$(CXX)" TARGET_OS="$(TARGET_OS)" \ + ./build_detect_platform build_config.mk ./) +# this file is generated by the previous line to set build flags and sources +include build_config.mk + +CFLAGS += -I. -I./include $(PLATFORM_CCFLAGS) $(OPT) +CXXFLAGS += -I. -I./include $(PLATFORM_CXXFLAGS) $(OPT) + +LDFLAGS += $(PLATFORM_LDFLAGS) +LIBS += $(PLATFORM_LIBS) + +LIBOBJECTS = $(SOURCES:.cc=.o) +MEMENVOBJECTS = $(MEMENV_SOURCES:.cc=.o) + +TESTUTIL = ./util/testutil.o +TESTHARNESS = ./util/testharness.o $(TESTUTIL) + +# Note: iOS should probably be using libtool, not ar. +ifeq ($(PLATFORM), IOS) +AR=xcrun ar +endif + +TESTS = \ + arena_test \ + autocompact_test \ + bloom_test \ + c_test \ + cache_test \ + coding_test \ + corruption_test \ + crc32c_test \ + db_test \ + dbformat_test \ + env_test \ + filename_test \ + filter_block_test \ + hash_test \ + issue178_test \ + issue200_test \ + log_test \ + memenv_test \ + skiplist_test \ + table_test \ + version_edit_test \ + version_set_test \ + write_batch_test + +PROGRAMS = db_bench leveldbutil $(TESTS) +BENCHMARKS = db_bench_sqlite3 db_bench_tree_db + +LIBRARY = libleveldb.a +MEMENVLIBRARY = libmemenv.a + +default: all + +# Should we build shared libraries? +ifneq ($(PLATFORM_SHARED_EXT),) + +ifneq ($(PLATFORM_SHARED_VERSIONED),true) +SHARED1 = libleveldb.$(PLATFORM_SHARED_EXT) +SHARED2 = $(SHARED1) +SHARED3 = $(SHARED1) +SHARED = $(SHARED1) +else +# Update db.h if you change these. +SHARED_MAJOR = 1 +SHARED_MINOR = 18 +SHARED1 = libleveldb.$(PLATFORM_SHARED_EXT) +SHARED2 = $(SHARED1).$(SHARED_MAJOR) +SHARED3 = $(SHARED1).$(SHARED_MAJOR).$(SHARED_MINOR) +SHARED = $(SHARED1) $(SHARED2) $(SHARED3) +$(SHARED1): $(SHARED3) + ln -fs $(SHARED3) $(SHARED1) +$(SHARED2): $(SHARED3) + ln -fs $(SHARED3) $(SHARED2) +endif + +$(SHARED3): + $(CXX) $(LDFLAGS) $(PLATFORM_SHARED_LDFLAGS)$(SHARED2) $(CXXFLAGS) $(PLATFORM_SHARED_CFLAGS) $(SOURCES) -o $(SHARED3) $(LIBS) + +endif # PLATFORM_SHARED_EXT + +all: $(SHARED) $(LIBRARY) + +check: all $(PROGRAMS) $(TESTS) + for t in $(TESTS); do echo "***** Running $$t"; ./$$t || exit 1; done + +clean: + -rm -f $(PROGRAMS) $(BENCHMARKS) $(LIBRARY) $(SHARED) $(MEMENVLIBRARY) */*.o */*/*.o ios-x86/*/*.o ios-arm/*/*.o build_config.mk + -rm -rf ios-x86/* ios-arm/* + +$(LIBRARY): $(LIBOBJECTS) + rm -f $@ + $(AR) -rs $@ $(LIBOBJECTS) + +db_bench: db/db_bench.o $(LIBOBJECTS) $(TESTUTIL) + $(CXX) $(LDFLAGS) db/db_bench.o $(LIBOBJECTS) $(TESTUTIL) -o $@ $(LIBS) + +db_bench_sqlite3: doc/bench/db_bench_sqlite3.o $(LIBOBJECTS) $(TESTUTIL) + $(CXX) $(LDFLAGS) doc/bench/db_bench_sqlite3.o $(LIBOBJECTS) $(TESTUTIL) -o $@ -lsqlite3 $(LIBS) + +db_bench_tree_db: doc/bench/db_bench_tree_db.o $(LIBOBJECTS) $(TESTUTIL) + $(CXX) $(LDFLAGS) doc/bench/db_bench_tree_db.o $(LIBOBJECTS) $(TESTUTIL) -o $@ -lkyotocabinet $(LIBS) + +leveldbutil: db/leveldb_main.o $(LIBOBJECTS) + $(CXX) $(LDFLAGS) db/leveldb_main.o $(LIBOBJECTS) -o $@ $(LIBS) + +arena_test: util/arena_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) util/arena_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +autocompact_test: db/autocompact_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) db/autocompact_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +bloom_test: util/bloom_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) util/bloom_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +c_test: db/c_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) db/c_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +cache_test: util/cache_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) util/cache_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +coding_test: util/coding_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) util/coding_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +corruption_test: db/corruption_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) db/corruption_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +crc32c_test: util/crc32c_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) util/crc32c_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +db_test: db/db_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) db/db_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +dbformat_test: db/dbformat_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) db/dbformat_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +env_test: util/env_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) util/env_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +filename_test: db/filename_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) db/filename_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +filter_block_test: table/filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) table/filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +hash_test: util/hash_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) util/hash_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +issue178_test: issues/issue178_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) issues/issue178_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +issue200_test: issues/issue200_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) issues/issue200_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +log_test: db/log_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) db/log_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +table_test: table/table_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) table/table_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +skiplist_test: db/skiplist_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) db/skiplist_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +version_edit_test: db/version_edit_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) db/version_edit_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +version_set_test: db/version_set_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) db/version_set_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +write_batch_test: db/write_batch_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(CXX) $(LDFLAGS) db/write_batch_test.o $(LIBOBJECTS) $(TESTHARNESS) -o $@ $(LIBS) + +$(MEMENVLIBRARY) : $(MEMENVOBJECTS) + rm -f $@ + $(AR) -rs $@ $(MEMENVOBJECTS) + +memenv_test : helpers/memenv/memenv_test.o $(MEMENVLIBRARY) $(LIBRARY) $(TESTHARNESS) + $(CXX) $(LDFLAGS) helpers/memenv/memenv_test.o $(MEMENVLIBRARY) $(LIBRARY) $(TESTHARNESS) -o $@ $(LIBS) + +ifeq ($(PLATFORM), IOS) +# For iOS, create universal object files to be used on both the simulator and +# a device. +PLATFORMSROOT=/Applications/Xcode.app/Contents/Developer/Platforms +SIMULATORROOT=$(PLATFORMSROOT)/iPhoneSimulator.platform/Developer +DEVICEROOT=$(PLATFORMSROOT)/iPhoneOS.platform/Developer +IOSVERSION=$(shell defaults read $(PLATFORMSROOT)/iPhoneOS.platform/version CFBundleShortVersionString) +IOSARCH=-arch armv6 -arch armv7 -arch armv7s -arch arm64 + +.cc.o: + mkdir -p ios-x86/$(dir $@) + xcrun -sdk iphonesimulator $(CXX) $(CXXFLAGS) -isysroot $(SIMULATORROOT)/SDKs/iPhoneSimulator$(IOSVERSION).sdk -arch i686 -arch x86_64 -c $< -o ios-x86/$@ + mkdir -p ios-arm/$(dir $@) + xcrun -sdk iphoneos $(CXX) $(CXXFLAGS) -isysroot $(DEVICEROOT)/SDKs/iPhoneOS$(IOSVERSION).sdk $(IOSARCH) -c $< -o ios-arm/$@ + xcrun lipo ios-x86/$@ ios-arm/$@ -create -output $@ + +.c.o: + mkdir -p ios-x86/$(dir $@) + xcrun -sdk iphonesimulator $(CC) $(CFLAGS) -isysroot $(SIMULATORROOT)/SDKs/iPhoneSimulator$(IOSVERSION).sdk -arch i686 -arch x86_64 -c $< -o ios-x86/$@ + mkdir -p ios-arm/$(dir $@) + xcrun -sdk iphoneos $(CC) $(CFLAGS) -isysroot $(DEVICEROOT)/SDKs/iPhoneOS$(IOSVERSION).sdk $(IOSARCH) -c $< -o ios-arm/$@ + xcrun lipo ios-x86/$@ ios-arm/$@ -create -output $@ + +else +.cc.o: + $(CXX) $(CXXFLAGS) -c $< -o $@ + +.c.o: + $(CC) $(CFLAGS) -c $< -o $@ +endif diff --git a/src/leveldb/NEWS b/src/leveldb/NEWS new file mode 100644 index 00000000..3fd99242 --- /dev/null +++ b/src/leveldb/NEWS @@ -0,0 +1,17 @@ +Release 1.2 2011-05-16 +---------------------- + +Fixes for larger databases (tested up to one billion 100-byte entries, +i.e., ~100GB). + +(1) Place hard limit on number of level-0 files. This fixes errors +of the form "too many open files". + +(2) Fixed memtable management. Before the fix, a heavy write burst +could cause unbounded memory usage. + +A fix for a logging bug where the reader would incorrectly complain +about corruption. + +Allow public access to WriteBatch contents so that users can easily +wrap a DB. diff --git a/src/leveldb/README b/src/leveldb/README new file mode 100644 index 00000000..3618adee --- /dev/null +++ b/src/leveldb/README @@ -0,0 +1,51 @@ +leveldb: A key-value store +Authors: Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com) + +The code under this directory implements a system for maintaining a +persistent key/value store. + +See doc/index.html for more explanation. +See doc/impl.html for a brief overview of the implementation. + +The public interface is in include/*.h. Callers should not include or +rely on the details of any other header files in this package. Those +internal APIs may be changed without warning. + +Guide to header files: + +include/db.h + Main interface to the DB: Start here + +include/options.h + Control over the behavior of an entire database, and also + control over the behavior of individual reads and writes. + +include/comparator.h + Abstraction for user-specified comparison function. If you want + just bytewise comparison of keys, you can use the default comparator, + but clients can write their own comparator implementations if they + want custom ordering (e.g. to handle different character + encodings, etc.) + +include/iterator.h + Interface for iterating over data. You can get an iterator + from a DB object. + +include/write_batch.h + Interface for atomically applying multiple updates to a database. + +include/slice.h + A simple module for maintaining a pointer and a length into some + other byte array. + +include/status.h + Status is returned from many of the public interfaces and is used + to report success and various kinds of errors. + +include/env.h + Abstraction of the OS environment. A posix implementation of + this interface is in util/env_posix.cc + +include/table.h +include/table_builder.h + Lower-level modules that most clients probably won't use directly diff --git a/src/leveldb/README.md b/src/leveldb/README.md new file mode 100644 index 00000000..480affb5 --- /dev/null +++ b/src/leveldb/README.md @@ -0,0 +1,138 @@ +**LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.** + +Authors: Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com) + +# Features + * Keys and values are arbitrary byte arrays. + * Data is stored sorted by key. + * Callers can provide a custom comparison function to override the sort order. + * The basic operations are `Put(key,value)`, `Get(key)`, `Delete(key)`. + * Multiple changes can be made in one atomic batch. + * Users can create a transient snapshot to get a consistent view of data. + * Forward and backward iteration is supported over the data. + * Data is automatically compressed using the [Snappy compression library](http://code.google.com/p/snappy). + * External activity (file system operations etc.) is relayed through a virtual interface so users can customize the operating system interactions. + * [Detailed documentation](http://htmlpreview.github.io/?https://github.com/google/leveldb/blob/master/doc/index.html) about how to use the library is included with the source code. + + +# Limitations + * This is not a SQL database. It does not have a relational data model, it does not support SQL queries, and it has no support for indexes. + * Only a single process (possibly multi-threaded) can access a particular database at a time. + * There is no client-server support builtin to the library. An application that needs such support will have to wrap their own server around the library. + +# Performance + +Here is a performance report (with explanations) from the run of the +included db_bench program. The results are somewhat noisy, but should +be enough to get a ballpark performance estimate. + +## Setup + +We use a database with a million entries. Each entry has a 16 byte +key, and a 100 byte value. Values used by the benchmark compress to +about half their original size. + + LevelDB: version 1.1 + Date: Sun May 1 12:11:26 2011 + CPU: 4 x Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz + CPUCache: 4096 KB + Keys: 16 bytes each + Values: 100 bytes each (50 bytes after compression) + Entries: 1000000 + Raw Size: 110.6 MB (estimated) + File Size: 62.9 MB (estimated) + +## Write performance + +The "fill" benchmarks create a brand new database, in either +sequential, or random order. The "fillsync" benchmark flushes data +from the operating system to the disk after every operation; the other +write operations leave the data sitting in the operating system buffer +cache for a while. The "overwrite" benchmark does random writes that +update existing keys in the database. + + fillseq : 1.765 micros/op; 62.7 MB/s + fillsync : 268.409 micros/op; 0.4 MB/s (10000 ops) + fillrandom : 2.460 micros/op; 45.0 MB/s + overwrite : 2.380 micros/op; 46.5 MB/s + +Each "op" above corresponds to a write of a single key/value pair. +I.e., a random write benchmark goes at approximately 400,000 writes per second. + +Each "fillsync" operation costs much less (0.3 millisecond) +than a disk seek (typically 10 milliseconds). We suspect that this is +because the hard disk itself is buffering the update in its memory and +responding before the data has been written to the platter. This may +or may not be safe based on whether or not the hard disk has enough +power to save its memory in the event of a power failure. + +## Read performance + +We list the performance of reading sequentially in both the forward +and reverse direction, and also the performance of a random lookup. +Note that the database created by the benchmark is quite small. +Therefore the report characterizes the performance of leveldb when the +working set fits in memory. The cost of reading a piece of data that +is not present in the operating system buffer cache will be dominated +by the one or two disk seeks needed to fetch the data from disk. +Write performance will be mostly unaffected by whether or not the +working set fits in memory. + + readrandom : 16.677 micros/op; (approximately 60,000 reads per second) + readseq : 0.476 micros/op; 232.3 MB/s + readreverse : 0.724 micros/op; 152.9 MB/s + +LevelDB compacts its underlying storage data in the background to +improve read performance. The results listed above were done +immediately after a lot of random writes. The results after +compactions (which are usually triggered automatically) are better. + + readrandom : 11.602 micros/op; (approximately 85,000 reads per second) + readseq : 0.423 micros/op; 261.8 MB/s + readreverse : 0.663 micros/op; 166.9 MB/s + +Some of the high cost of reads comes from repeated decompression of blocks +read from disk. If we supply enough cache to the leveldb so it can hold the +uncompressed blocks in memory, the read performance improves again: + + readrandom : 9.775 micros/op; (approximately 100,000 reads per second before compaction) + readrandom : 5.215 micros/op; (approximately 190,000 reads per second after compaction) + +## Repository contents + +See doc/index.html for more explanation. See doc/impl.html for a brief overview of the implementation. + +The public interface is in include/*.h. Callers should not include or +rely on the details of any other header files in this package. Those +internal APIs may be changed without warning. + +Guide to header files: + +* **include/db.h**: Main interface to the DB: Start here + +* **include/options.h**: Control over the behavior of an entire database, +and also control over the behavior of individual reads and writes. + +* **include/comparator.h**: Abstraction for user-specified comparison function. +If you want just bytewise comparison of keys, you can use the default +comparator, but clients can write their own comparator implementations if they +want custom ordering (e.g. to handle different character encodings, etc.) + +* **include/iterator.h**: Interface for iterating over data. You can get +an iterator from a DB object. + +* **include/write_batch.h**: Interface for atomically applying multiple +updates to a database. + +* **include/slice.h**: A simple module for maintaining a pointer and a +length into some other byte array. + +* **include/status.h**: Status is returned from many of the public interfaces +and is used to report success and various kinds of errors. + +* **include/env.h**: +Abstraction of the OS environment. A posix implementation of this interface is +in util/env_posix.cc + +* **include/table.h, include/table_builder.h**: Lower-level modules that most +clients probably won't use directly diff --git a/src/leveldb/TODO b/src/leveldb/TODO new file mode 100644 index 00000000..e603c071 --- /dev/null +++ b/src/leveldb/TODO @@ -0,0 +1,14 @@ +ss +- Stats + +db +- Maybe implement DB::BulkDeleteForRange(start_key, end_key) + that would blow away files whose ranges are entirely contained + within [start_key..end_key]? For Chrome, deletion of obsolete + object stores, etc. can be done in the background anyway, so + probably not that important. +- There have been requests for MultiGet. + +After a range is completely deleted, what gets rid of the +corresponding files if we do no future changes to that range. Make +the conditions for triggering compactions fire in more situations? diff --git a/src/leveldb/WINDOWS.md b/src/leveldb/WINDOWS.md new file mode 100644 index 00000000..5b76c244 --- /dev/null +++ b/src/leveldb/WINDOWS.md @@ -0,0 +1,39 @@ +# Building LevelDB On Windows + +## Prereqs + +Install the [Windows Software Development Kit version 7.1](http://www.microsoft.com/downloads/dlx/en-us/listdetailsview.aspx?FamilyID=6b6c21d2-2006-4afa-9702-529fa782d63b). + +Download and extract the [Snappy source distribution](http://snappy.googlecode.com/files/snappy-1.0.5.tar.gz) + +1. Open the "Windows SDK 7.1 Command Prompt" : + Start Menu -> "Microsoft Windows SDK v7.1" > "Windows SDK 7.1 Command Prompt" +2. Change the directory to the leveldb project + +## Building the Static lib + +* 32 bit Version + + setenv /x86 + msbuild.exe /p:Configuration=Release /p:Platform=Win32 /p:Snappy=..\snappy-1.0.5 + +* 64 bit Version + + setenv /x64 + msbuild.exe /p:Configuration=Release /p:Platform=x64 /p:Snappy=..\snappy-1.0.5 + + +## Building and Running the Benchmark app + +* 32 bit Version + + setenv /x86 + msbuild.exe /p:Configuration=Benchmark /p:Platform=Win32 /p:Snappy=..\snappy-1.0.5 + Benchmark\leveldb.exe + +* 64 bit Version + + setenv /x64 + msbuild.exe /p:Configuration=Benchmark /p:Platform=x64 /p:Snappy=..\snappy-1.0.5 + x64\Benchmark\leveldb.exe + diff --git a/src/leveldb/build_detect_platform b/src/leveldb/build_detect_platform new file mode 100755 index 00000000..a1101c1b --- /dev/null +++ b/src/leveldb/build_detect_platform @@ -0,0 +1,231 @@ +#!/bin/sh +# +# Detects OS we're compiling on and outputs a file specified by the first +# argument, which in turn gets read while processing Makefile. +# +# The output will set the following variables: +# CC C Compiler path +# CXX C++ Compiler path +# PLATFORM_LDFLAGS Linker flags +# PLATFORM_LIBS Libraries flags +# PLATFORM_SHARED_EXT Extension for shared libraries +# PLATFORM_SHARED_LDFLAGS Flags for building shared library +# This flag is embedded just before the name +# of the shared library without intervening spaces +# PLATFORM_SHARED_CFLAGS Flags for compiling objects for shared library +# PLATFORM_CCFLAGS C compiler flags +# PLATFORM_CXXFLAGS C++ compiler flags. Will contain: +# PLATFORM_SHARED_VERSIONED Set to 'true' if platform supports versioned +# shared libraries, empty otherwise. +# +# The PLATFORM_CCFLAGS and PLATFORM_CXXFLAGS might include the following: +# +# -DLEVELDB_ATOMIC_PRESENT if is present +# -DLEVELDB_PLATFORM_POSIX for Posix-based platforms +# -DSNAPPY if the Snappy library is present +# + +OUTPUT=$1 +PREFIX=$2 +if test -z "$OUTPUT" || test -z "$PREFIX"; then + echo "usage: $0 " >&2 + exit 1 +fi + +# Delete existing output, if it exists +rm -f $OUTPUT +touch $OUTPUT + +if test -z "$CC"; then + CC=cc +fi + +if test -z "$CXX"; then + CXX=g++ +fi + +if test -z "$TMPDIR"; then + TMPDIR=/tmp +fi + +# Detect OS +if test -z "$TARGET_OS"; then + TARGET_OS=`uname -s` +fi + +COMMON_FLAGS= +CROSS_COMPILE= +PLATFORM_CCFLAGS= +PLATFORM_CXXFLAGS= +PLATFORM_LDFLAGS= +PLATFORM_LIBS= +PLATFORM_SHARED_EXT="so" +PLATFORM_SHARED_LDFLAGS="-shared -Wl,-soname -Wl," +PLATFORM_SHARED_CFLAGS="-fPIC" +PLATFORM_SHARED_VERSIONED=true + +MEMCMP_FLAG= +if [ "$CXX" = "g++" ]; then + # Use libc's memcmp instead of GCC's memcmp. This results in ~40% + # performance improvement on readrandom under gcc 4.4.3 on Linux/x86. + MEMCMP_FLAG="-fno-builtin-memcmp" +fi + +case "$TARGET_OS" in + CYGWIN_*) + PLATFORM=OS_LINUX + COMMON_FLAGS="$MEMCMP_FLAG -lpthread -DOS_LINUX -DCYGWIN" + PLATFORM_LDFLAGS="-lpthread" + PORT_FILE=port/port_posix.cc + ;; + Darwin) + PLATFORM=OS_MACOSX + COMMON_FLAGS="$MEMCMP_FLAG -DOS_MACOSX" + PLATFORM_SHARED_EXT=dylib + [ -z "$INSTALL_PATH" ] && INSTALL_PATH=`pwd` + PLATFORM_SHARED_LDFLAGS="-dynamiclib -install_name $INSTALL_PATH/" + PORT_FILE=port/port_posix.cc + ;; + Linux) + PLATFORM=OS_LINUX + COMMON_FLAGS="$MEMCMP_FLAG -pthread -DOS_LINUX" + PLATFORM_LDFLAGS="-pthread" + PORT_FILE=port/port_posix.cc + ;; + SunOS) + PLATFORM=OS_SOLARIS + COMMON_FLAGS="$MEMCMP_FLAG -D_REENTRANT -DOS_SOLARIS" + PLATFORM_LIBS="-lpthread -lrt" + PORT_FILE=port/port_posix.cc + ;; + FreeBSD) + PLATFORM=OS_FREEBSD + COMMON_FLAGS="$MEMCMP_FLAG -D_REENTRANT -DOS_FREEBSD" + PLATFORM_LIBS="-lpthread" + PORT_FILE=port/port_posix.cc + ;; + GNU/kFreeBSD) + PLATFORM=OS_KFREEBSD + COMMON_FLAGS="$MEMCMP_FLAG -D_REENTRANT -DOS_KFREEBSD" + PLATFORM_LIBS="-lpthread" + PORT_FILE=port/port_posix.cc + ;; + NetBSD) + PLATFORM=OS_NETBSD + COMMON_FLAGS="$MEMCMP_FLAG -D_REENTRANT -DOS_NETBSD" + PLATFORM_LIBS="-lpthread -lgcc_s" + PORT_FILE=port/port_posix.cc + ;; + OpenBSD) + PLATFORM=OS_OPENBSD + COMMON_FLAGS="$MEMCMP_FLAG -D_REENTRANT -DOS_OPENBSD" + PLATFORM_LDFLAGS="-pthread" + PORT_FILE=port/port_posix.cc + ;; + DragonFly) + PLATFORM=OS_DRAGONFLYBSD + COMMON_FLAGS="$MEMCMP_FLAG -D_REENTRANT -DOS_DRAGONFLYBSD" + PLATFORM_LIBS="-lpthread" + PORT_FILE=port/port_posix.cc + ;; + OS_ANDROID_CROSSCOMPILE) + PLATFORM=OS_ANDROID + COMMON_FLAGS="$MEMCMP_FLAG -D_REENTRANT -DOS_ANDROID -DLEVELDB_PLATFORM_POSIX" + PLATFORM_LDFLAGS="" # All pthread features are in the Android C library + PORT_FILE=port/port_posix.cc + CROSS_COMPILE=true + ;; + HP-UX) + PLATFORM=OS_HPUX + COMMON_FLAGS="$MEMCMP_FLAG -D_REENTRANT -DOS_HPUX" + PLATFORM_LDFLAGS="-pthread" + PORT_FILE=port/port_posix.cc + # man ld: +h internal_name + PLATFORM_SHARED_LDFLAGS="-shared -Wl,+h -Wl," + ;; + IOS) + PLATFORM=IOS + COMMON_FLAGS="$MEMCMP_FLAG -DOS_MACOSX" + [ -z "$INSTALL_PATH" ] && INSTALL_PATH=`pwd` + PORT_FILE=port/port_posix.cc + PLATFORM_SHARED_EXT= + PLATFORM_SHARED_LDFLAGS= + PLATFORM_SHARED_CFLAGS= + PLATFORM_SHARED_VERSIONED= + ;; + OS_WINDOWS_CROSSCOMPILE | NATIVE_WINDOWS) + PLATFORM=OS_WINDOWS + COMMON_FLAGS="-fno-builtin-memcmp -D_REENTRANT -DOS_WINDOWS -DLEVELDB_PLATFORM_WINDOWS -DWINVER=0x0500 -D__USE_MINGW_ANSI_STDIO=1" + PLATFORM_SOURCES="util/env_win.cc" + PLATFORM_LIBS="-lshlwapi" + PORT_FILE=port/port_win.cc + CROSS_COMPILE=true + ;; + *) + echo "Unknown platform!" >&2 + exit 1 +esac + +# We want to make a list of all cc files within util, db, table, and helpers +# except for the test and benchmark files. By default, find will output a list +# of all files matching either rule, so we need to append -print to make the +# prune take effect. +DIRS="$PREFIX/db $PREFIX/util $PREFIX/table" + +set -f # temporarily disable globbing so that our patterns aren't expanded +PRUNE_TEST="-name *test*.cc -prune" +PRUNE_BENCH="-name *_bench.cc -prune" +PRUNE_TOOL="-name leveldb_main.cc -prune" +PORTABLE_FILES=`find $DIRS $PRUNE_TEST -o $PRUNE_BENCH -o $PRUNE_TOOL -o -name '*.cc' -print | sort | sed "s,^$PREFIX/,," | tr "\n" " "` + +set +f # re-enable globbing + +# The sources consist of the portable files, plus the platform-specific port +# file. +echo "SOURCES=$PORTABLE_FILES $PORT_FILE" >> $OUTPUT +echo "MEMENV_SOURCES=helpers/memenv/memenv.cc" >> $OUTPUT + +if [ "$CROSS_COMPILE" = "true" ]; then + # Cross-compiling; do not try any compilation tests. + true +else + CXXOUTPUT="${TMPDIR}/leveldb_build_detect_platform-cxx.$$" + + # If -std=c++0x works, use as fallback for when memory barriers + # are not available. + $CXX $CXXFLAGS -std=c++0x -x c++ - -o $CXXOUTPUT 2>/dev/null < + int main() {} +EOF + if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT" + PLATFORM_CXXFLAGS="-std=c++0x" + else + COMMON_FLAGS="$COMMON_FLAGS -DLEVELDB_PLATFORM_POSIX" + fi + + # Test whether tcmalloc is available + $CXX $CXXFLAGS -x c++ - -o $CXXOUTPUT -ltcmalloc 2>/dev/null </dev/null +fi + +PLATFORM_CCFLAGS="$PLATFORM_CCFLAGS $COMMON_FLAGS" +PLATFORM_CXXFLAGS="$PLATFORM_CXXFLAGS $COMMON_FLAGS" + +echo "CC=$CC" >> $OUTPUT +echo "CXX=$CXX" >> $OUTPUT +echo "PLATFORM=$PLATFORM" >> $OUTPUT +echo "PLATFORM_LDFLAGS=$PLATFORM_LDFLAGS" >> $OUTPUT +echo "PLATFORM_LIBS=$PLATFORM_LIBS" >> $OUTPUT +echo "PLATFORM_CCFLAGS=$PLATFORM_CCFLAGS" >> $OUTPUT +echo "PLATFORM_CXXFLAGS=$PLATFORM_CXXFLAGS" >> $OUTPUT +echo "PLATFORM_SHARED_CFLAGS=$PLATFORM_SHARED_CFLAGS" >> $OUTPUT +echo "PLATFORM_SHARED_EXT=$PLATFORM_SHARED_EXT" >> $OUTPUT +echo "PLATFORM_SHARED_LDFLAGS=$PLATFORM_SHARED_LDFLAGS" >> $OUTPUT +echo "PLATFORM_SHARED_VERSIONED=$PLATFORM_SHARED_VERSIONED" >> $OUTPUT diff --git a/src/leveldb/db/autocompact_test.cc b/src/leveldb/db/autocompact_test.cc new file mode 100644 index 00000000..d20a2362 --- /dev/null +++ b/src/leveldb/db/autocompact_test.cc @@ -0,0 +1,118 @@ +// Copyright (c) 2013 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/db.h" +#include "db/db_impl.h" +#include "leveldb/cache.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace leveldb { + +class AutoCompactTest { + public: + std::string dbname_; + Cache* tiny_cache_; + Options options_; + DB* db_; + + AutoCompactTest() { + dbname_ = test::TmpDir() + "/autocompact_test"; + tiny_cache_ = NewLRUCache(100); + options_.block_cache = tiny_cache_; + DestroyDB(dbname_, options_); + options_.create_if_missing = true; + options_.compression = kNoCompression; + ASSERT_OK(DB::Open(options_, dbname_, &db_)); + } + + ~AutoCompactTest() { + delete db_; + DestroyDB(dbname_, Options()); + delete tiny_cache_; + } + + std::string Key(int i) { + char buf[100]; + snprintf(buf, sizeof(buf), "key%06d", i); + return std::string(buf); + } + + uint64_t Size(const Slice& start, const Slice& limit) { + Range r(start, limit); + uint64_t size; + db_->GetApproximateSizes(&r, 1, &size); + return size; + } + + void DoReads(int n); +}; + +static const int kValueSize = 200 * 1024; +static const int kTotalSize = 100 * 1024 * 1024; +static const int kCount = kTotalSize / kValueSize; + +// Read through the first n keys repeatedly and check that they get +// compacted (verified by checking the size of the key space). +void AutoCompactTest::DoReads(int n) { + std::string value(kValueSize, 'x'); + DBImpl* dbi = reinterpret_cast(db_); + + // Fill database + for (int i = 0; i < kCount; i++) { + ASSERT_OK(db_->Put(WriteOptions(), Key(i), value)); + } + ASSERT_OK(dbi->TEST_CompactMemTable()); + + // Delete everything + for (int i = 0; i < kCount; i++) { + ASSERT_OK(db_->Delete(WriteOptions(), Key(i))); + } + ASSERT_OK(dbi->TEST_CompactMemTable()); + + // Get initial measurement of the space we will be reading. + const int64_t initial_size = Size(Key(0), Key(n)); + const int64_t initial_other_size = Size(Key(n), Key(kCount)); + + // Read until size drops significantly. + std::string limit_key = Key(n); + for (int read = 0; true; read++) { + ASSERT_LT(read, 100) << "Taking too long to compact"; + Iterator* iter = db_->NewIterator(ReadOptions()); + for (iter->SeekToFirst(); + iter->Valid() && iter->key().ToString() < limit_key; + iter->Next()) { + // Drop data + } + delete iter; + // Wait a little bit to allow any triggered compactions to complete. + Env::Default()->SleepForMicroseconds(1000000); + uint64_t size = Size(Key(0), Key(n)); + fprintf(stderr, "iter %3d => %7.3f MB [other %7.3f MB]\n", + read+1, size/1048576.0, Size(Key(n), Key(kCount))/1048576.0); + if (size <= initial_size/10) { + break; + } + } + + // Verify that the size of the key space not touched by the reads + // is pretty much unchanged. + const int64_t final_other_size = Size(Key(n), Key(kCount)); + ASSERT_LE(final_other_size, initial_other_size + 1048576); + ASSERT_GE(final_other_size, initial_other_size/5 - 1048576); +} + +TEST(AutoCompactTest, ReadAll) { + DoReads(kCount); +} + +TEST(AutoCompactTest, ReadHalf) { + DoReads(kCount/2); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/db/builder.cc b/src/leveldb/db/builder.cc new file mode 100644 index 00000000..f4198821 --- /dev/null +++ b/src/leveldb/db/builder.cc @@ -0,0 +1,88 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/builder.h" + +#include "db/filename.h" +#include "db/dbformat.h" +#include "db/table_cache.h" +#include "db/version_edit.h" +#include "leveldb/db.h" +#include "leveldb/env.h" +#include "leveldb/iterator.h" + +namespace leveldb { + +Status BuildTable(const std::string& dbname, + Env* env, + const Options& options, + TableCache* table_cache, + Iterator* iter, + FileMetaData* meta) { + Status s; + meta->file_size = 0; + iter->SeekToFirst(); + + std::string fname = TableFileName(dbname, meta->number); + if (iter->Valid()) { + WritableFile* file; + s = env->NewWritableFile(fname, &file); + if (!s.ok()) { + return s; + } + + TableBuilder* builder = new TableBuilder(options, file); + meta->smallest.DecodeFrom(iter->key()); + for (; iter->Valid(); iter->Next()) { + Slice key = iter->key(); + meta->largest.DecodeFrom(key); + builder->Add(key, iter->value()); + } + + // Finish and check for builder errors + if (s.ok()) { + s = builder->Finish(); + if (s.ok()) { + meta->file_size = builder->FileSize(); + assert(meta->file_size > 0); + } + } else { + builder->Abandon(); + } + delete builder; + + // Finish and check for file errors + if (s.ok()) { + s = file->Sync(); + } + if (s.ok()) { + s = file->Close(); + } + delete file; + file = NULL; + + if (s.ok()) { + // Verify that the table is usable + Iterator* it = table_cache->NewIterator(ReadOptions(), + meta->number, + meta->file_size); + s = it->status(); + delete it; + } + } + + // Check for input iterator errors + if (!iter->status().ok()) { + s = iter->status(); + } + + if (s.ok() && meta->file_size > 0) { + // Keep it + } else { + env->DeleteFile(fname); + } + return s; +} + +} // namespace leveldb diff --git a/src/leveldb/db/builder.h b/src/leveldb/db/builder.h new file mode 100644 index 00000000..62431fcf --- /dev/null +++ b/src/leveldb/db/builder.h @@ -0,0 +1,34 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_DB_BUILDER_H_ +#define STORAGE_LEVELDB_DB_BUILDER_H_ + +#include "leveldb/status.h" + +namespace leveldb { + +struct Options; +struct FileMetaData; + +class Env; +class Iterator; +class TableCache; +class VersionEdit; + +// Build a Table file from the contents of *iter. The generated file +// will be named according to meta->number. On success, the rest of +// *meta will be filled with metadata about the generated table. +// If no data is present in *iter, meta->file_size will be set to +// zero, and no Table file will be produced. +extern Status BuildTable(const std::string& dbname, + Env* env, + const Options& options, + TableCache* table_cache, + Iterator* iter, + FileMetaData* meta); + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_BUILDER_H_ diff --git a/src/leveldb/db/c.cc b/src/leveldb/db/c.cc new file mode 100644 index 00000000..08ff0ad9 --- /dev/null +++ b/src/leveldb/db/c.cc @@ -0,0 +1,595 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/c.h" + +#include +#include +#include "leveldb/cache.h" +#include "leveldb/comparator.h" +#include "leveldb/db.h" +#include "leveldb/env.h" +#include "leveldb/filter_policy.h" +#include "leveldb/iterator.h" +#include "leveldb/options.h" +#include "leveldb/status.h" +#include "leveldb/write_batch.h" + +using leveldb::Cache; +using leveldb::Comparator; +using leveldb::CompressionType; +using leveldb::DB; +using leveldb::Env; +using leveldb::FileLock; +using leveldb::FilterPolicy; +using leveldb::Iterator; +using leveldb::kMajorVersion; +using leveldb::kMinorVersion; +using leveldb::Logger; +using leveldb::NewBloomFilterPolicy; +using leveldb::NewLRUCache; +using leveldb::Options; +using leveldb::RandomAccessFile; +using leveldb::Range; +using leveldb::ReadOptions; +using leveldb::SequentialFile; +using leveldb::Slice; +using leveldb::Snapshot; +using leveldb::Status; +using leveldb::WritableFile; +using leveldb::WriteBatch; +using leveldb::WriteOptions; + +extern "C" { + +struct leveldb_t { DB* rep; }; +struct leveldb_iterator_t { Iterator* rep; }; +struct leveldb_writebatch_t { WriteBatch rep; }; +struct leveldb_snapshot_t { const Snapshot* rep; }; +struct leveldb_readoptions_t { ReadOptions rep; }; +struct leveldb_writeoptions_t { WriteOptions rep; }; +struct leveldb_options_t { Options rep; }; +struct leveldb_cache_t { Cache* rep; }; +struct leveldb_seqfile_t { SequentialFile* rep; }; +struct leveldb_randomfile_t { RandomAccessFile* rep; }; +struct leveldb_writablefile_t { WritableFile* rep; }; +struct leveldb_logger_t { Logger* rep; }; +struct leveldb_filelock_t { FileLock* rep; }; + +struct leveldb_comparator_t : public Comparator { + void* state_; + void (*destructor_)(void*); + int (*compare_)( + void*, + const char* a, size_t alen, + const char* b, size_t blen); + const char* (*name_)(void*); + + virtual ~leveldb_comparator_t() { + (*destructor_)(state_); + } + + virtual int Compare(const Slice& a, const Slice& b) const { + return (*compare_)(state_, a.data(), a.size(), b.data(), b.size()); + } + + virtual const char* Name() const { + return (*name_)(state_); + } + + // No-ops since the C binding does not support key shortening methods. + virtual void FindShortestSeparator(std::string*, const Slice&) const { } + virtual void FindShortSuccessor(std::string* key) const { } +}; + +struct leveldb_filterpolicy_t : public FilterPolicy { + void* state_; + void (*destructor_)(void*); + const char* (*name_)(void*); + char* (*create_)( + void*, + const char* const* key_array, const size_t* key_length_array, + int num_keys, + size_t* filter_length); + unsigned char (*key_match_)( + void*, + const char* key, size_t length, + const char* filter, size_t filter_length); + + virtual ~leveldb_filterpolicy_t() { + (*destructor_)(state_); + } + + virtual const char* Name() const { + return (*name_)(state_); + } + + virtual void CreateFilter(const Slice* keys, int n, std::string* dst) const { + std::vector key_pointers(n); + std::vector key_sizes(n); + for (int i = 0; i < n; i++) { + key_pointers[i] = keys[i].data(); + key_sizes[i] = keys[i].size(); + } + size_t len; + char* filter = (*create_)(state_, &key_pointers[0], &key_sizes[0], n, &len); + dst->append(filter, len); + free(filter); + } + + virtual bool KeyMayMatch(const Slice& key, const Slice& filter) const { + return (*key_match_)(state_, key.data(), key.size(), + filter.data(), filter.size()); + } +}; + +struct leveldb_env_t { + Env* rep; + bool is_default; +}; + +static bool SaveError(char** errptr, const Status& s) { + assert(errptr != NULL); + if (s.ok()) { + return false; + } else if (*errptr == NULL) { + *errptr = strdup(s.ToString().c_str()); + } else { + // TODO(sanjay): Merge with existing error? + free(*errptr); + *errptr = strdup(s.ToString().c_str()); + } + return true; +} + +static char* CopyString(const std::string& str) { + char* result = reinterpret_cast(malloc(sizeof(char) * str.size())); + memcpy(result, str.data(), sizeof(char) * str.size()); + return result; +} + +leveldb_t* leveldb_open( + const leveldb_options_t* options, + const char* name, + char** errptr) { + DB* db; + if (SaveError(errptr, DB::Open(options->rep, std::string(name), &db))) { + return NULL; + } + leveldb_t* result = new leveldb_t; + result->rep = db; + return result; +} + +void leveldb_close(leveldb_t* db) { + delete db->rep; + delete db; +} + +void leveldb_put( + leveldb_t* db, + const leveldb_writeoptions_t* options, + const char* key, size_t keylen, + const char* val, size_t vallen, + char** errptr) { + SaveError(errptr, + db->rep->Put(options->rep, Slice(key, keylen), Slice(val, vallen))); +} + +void leveldb_delete( + leveldb_t* db, + const leveldb_writeoptions_t* options, + const char* key, size_t keylen, + char** errptr) { + SaveError(errptr, db->rep->Delete(options->rep, Slice(key, keylen))); +} + + +void leveldb_write( + leveldb_t* db, + const leveldb_writeoptions_t* options, + leveldb_writebatch_t* batch, + char** errptr) { + SaveError(errptr, db->rep->Write(options->rep, &batch->rep)); +} + +char* leveldb_get( + leveldb_t* db, + const leveldb_readoptions_t* options, + const char* key, size_t keylen, + size_t* vallen, + char** errptr) { + char* result = NULL; + std::string tmp; + Status s = db->rep->Get(options->rep, Slice(key, keylen), &tmp); + if (s.ok()) { + *vallen = tmp.size(); + result = CopyString(tmp); + } else { + *vallen = 0; + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + } + return result; +} + +leveldb_iterator_t* leveldb_create_iterator( + leveldb_t* db, + const leveldb_readoptions_t* options) { + leveldb_iterator_t* result = new leveldb_iterator_t; + result->rep = db->rep->NewIterator(options->rep); + return result; +} + +const leveldb_snapshot_t* leveldb_create_snapshot( + leveldb_t* db) { + leveldb_snapshot_t* result = new leveldb_snapshot_t; + result->rep = db->rep->GetSnapshot(); + return result; +} + +void leveldb_release_snapshot( + leveldb_t* db, + const leveldb_snapshot_t* snapshot) { + db->rep->ReleaseSnapshot(snapshot->rep); + delete snapshot; +} + +char* leveldb_property_value( + leveldb_t* db, + const char* propname) { + std::string tmp; + if (db->rep->GetProperty(Slice(propname), &tmp)) { + // We use strdup() since we expect human readable output. + return strdup(tmp.c_str()); + } else { + return NULL; + } +} + +void leveldb_approximate_sizes( + leveldb_t* db, + int num_ranges, + const char* const* range_start_key, const size_t* range_start_key_len, + const char* const* range_limit_key, const size_t* range_limit_key_len, + uint64_t* sizes) { + Range* ranges = new Range[num_ranges]; + for (int i = 0; i < num_ranges; i++) { + ranges[i].start = Slice(range_start_key[i], range_start_key_len[i]); + ranges[i].limit = Slice(range_limit_key[i], range_limit_key_len[i]); + } + db->rep->GetApproximateSizes(ranges, num_ranges, sizes); + delete[] ranges; +} + +void leveldb_compact_range( + leveldb_t* db, + const char* start_key, size_t start_key_len, + const char* limit_key, size_t limit_key_len) { + Slice a, b; + db->rep->CompactRange( + // Pass NULL Slice if corresponding "const char*" is NULL + (start_key ? (a = Slice(start_key, start_key_len), &a) : NULL), + (limit_key ? (b = Slice(limit_key, limit_key_len), &b) : NULL)); +} + +void leveldb_destroy_db( + const leveldb_options_t* options, + const char* name, + char** errptr) { + SaveError(errptr, DestroyDB(name, options->rep)); +} + +void leveldb_repair_db( + const leveldb_options_t* options, + const char* name, + char** errptr) { + SaveError(errptr, RepairDB(name, options->rep)); +} + +void leveldb_iter_destroy(leveldb_iterator_t* iter) { + delete iter->rep; + delete iter; +} + +unsigned char leveldb_iter_valid(const leveldb_iterator_t* iter) { + return iter->rep->Valid(); +} + +void leveldb_iter_seek_to_first(leveldb_iterator_t* iter) { + iter->rep->SeekToFirst(); +} + +void leveldb_iter_seek_to_last(leveldb_iterator_t* iter) { + iter->rep->SeekToLast(); +} + +void leveldb_iter_seek(leveldb_iterator_t* iter, const char* k, size_t klen) { + iter->rep->Seek(Slice(k, klen)); +} + +void leveldb_iter_next(leveldb_iterator_t* iter) { + iter->rep->Next(); +} + +void leveldb_iter_prev(leveldb_iterator_t* iter) { + iter->rep->Prev(); +} + +const char* leveldb_iter_key(const leveldb_iterator_t* iter, size_t* klen) { + Slice s = iter->rep->key(); + *klen = s.size(); + return s.data(); +} + +const char* leveldb_iter_value(const leveldb_iterator_t* iter, size_t* vlen) { + Slice s = iter->rep->value(); + *vlen = s.size(); + return s.data(); +} + +void leveldb_iter_get_error(const leveldb_iterator_t* iter, char** errptr) { + SaveError(errptr, iter->rep->status()); +} + +leveldb_writebatch_t* leveldb_writebatch_create() { + return new leveldb_writebatch_t; +} + +void leveldb_writebatch_destroy(leveldb_writebatch_t* b) { + delete b; +} + +void leveldb_writebatch_clear(leveldb_writebatch_t* b) { + b->rep.Clear(); +} + +void leveldb_writebatch_put( + leveldb_writebatch_t* b, + const char* key, size_t klen, + const char* val, size_t vlen) { + b->rep.Put(Slice(key, klen), Slice(val, vlen)); +} + +void leveldb_writebatch_delete( + leveldb_writebatch_t* b, + const char* key, size_t klen) { + b->rep.Delete(Slice(key, klen)); +} + +void leveldb_writebatch_iterate( + leveldb_writebatch_t* b, + void* state, + void (*put)(void*, const char* k, size_t klen, const char* v, size_t vlen), + void (*deleted)(void*, const char* k, size_t klen)) { + class H : public WriteBatch::Handler { + public: + void* state_; + void (*put_)(void*, const char* k, size_t klen, const char* v, size_t vlen); + void (*deleted_)(void*, const char* k, size_t klen); + virtual void Put(const Slice& key, const Slice& value) { + (*put_)(state_, key.data(), key.size(), value.data(), value.size()); + } + virtual void Delete(const Slice& key) { + (*deleted_)(state_, key.data(), key.size()); + } + }; + H handler; + handler.state_ = state; + handler.put_ = put; + handler.deleted_ = deleted; + b->rep.Iterate(&handler); +} + +leveldb_options_t* leveldb_options_create() { + return new leveldb_options_t; +} + +void leveldb_options_destroy(leveldb_options_t* options) { + delete options; +} + +void leveldb_options_set_comparator( + leveldb_options_t* opt, + leveldb_comparator_t* cmp) { + opt->rep.comparator = cmp; +} + +void leveldb_options_set_filter_policy( + leveldb_options_t* opt, + leveldb_filterpolicy_t* policy) { + opt->rep.filter_policy = policy; +} + +void leveldb_options_set_create_if_missing( + leveldb_options_t* opt, unsigned char v) { + opt->rep.create_if_missing = v; +} + +void leveldb_options_set_error_if_exists( + leveldb_options_t* opt, unsigned char v) { + opt->rep.error_if_exists = v; +} + +void leveldb_options_set_paranoid_checks( + leveldb_options_t* opt, unsigned char v) { + opt->rep.paranoid_checks = v; +} + +void leveldb_options_set_env(leveldb_options_t* opt, leveldb_env_t* env) { + opt->rep.env = (env ? env->rep : NULL); +} + +void leveldb_options_set_info_log(leveldb_options_t* opt, leveldb_logger_t* l) { + opt->rep.info_log = (l ? l->rep : NULL); +} + +void leveldb_options_set_write_buffer_size(leveldb_options_t* opt, size_t s) { + opt->rep.write_buffer_size = s; +} + +void leveldb_options_set_max_open_files(leveldb_options_t* opt, int n) { + opt->rep.max_open_files = n; +} + +void leveldb_options_set_cache(leveldb_options_t* opt, leveldb_cache_t* c) { + opt->rep.block_cache = c->rep; +} + +void leveldb_options_set_block_size(leveldb_options_t* opt, size_t s) { + opt->rep.block_size = s; +} + +void leveldb_options_set_block_restart_interval(leveldb_options_t* opt, int n) { + opt->rep.block_restart_interval = n; +} + +void leveldb_options_set_compression(leveldb_options_t* opt, int t) { + opt->rep.compression = static_cast(t); +} + +leveldb_comparator_t* leveldb_comparator_create( + void* state, + void (*destructor)(void*), + int (*compare)( + void*, + const char* a, size_t alen, + const char* b, size_t blen), + const char* (*name)(void*)) { + leveldb_comparator_t* result = new leveldb_comparator_t; + result->state_ = state; + result->destructor_ = destructor; + result->compare_ = compare; + result->name_ = name; + return result; +} + +void leveldb_comparator_destroy(leveldb_comparator_t* cmp) { + delete cmp; +} + +leveldb_filterpolicy_t* leveldb_filterpolicy_create( + void* state, + void (*destructor)(void*), + char* (*create_filter)( + void*, + const char* const* key_array, const size_t* key_length_array, + int num_keys, + size_t* filter_length), + unsigned char (*key_may_match)( + void*, + const char* key, size_t length, + const char* filter, size_t filter_length), + const char* (*name)(void*)) { + leveldb_filterpolicy_t* result = new leveldb_filterpolicy_t; + result->state_ = state; + result->destructor_ = destructor; + result->create_ = create_filter; + result->key_match_ = key_may_match; + result->name_ = name; + return result; +} + +void leveldb_filterpolicy_destroy(leveldb_filterpolicy_t* filter) { + delete filter; +} + +leveldb_filterpolicy_t* leveldb_filterpolicy_create_bloom(int bits_per_key) { + // Make a leveldb_filterpolicy_t, but override all of its methods so + // they delegate to a NewBloomFilterPolicy() instead of user + // supplied C functions. + struct Wrapper : public leveldb_filterpolicy_t { + const FilterPolicy* rep_; + ~Wrapper() { delete rep_; } + const char* Name() const { return rep_->Name(); } + void CreateFilter(const Slice* keys, int n, std::string* dst) const { + return rep_->CreateFilter(keys, n, dst); + } + bool KeyMayMatch(const Slice& key, const Slice& filter) const { + return rep_->KeyMayMatch(key, filter); + } + static void DoNothing(void*) { } + }; + Wrapper* wrapper = new Wrapper; + wrapper->rep_ = NewBloomFilterPolicy(bits_per_key); + wrapper->state_ = NULL; + wrapper->destructor_ = &Wrapper::DoNothing; + return wrapper; +} + +leveldb_readoptions_t* leveldb_readoptions_create() { + return new leveldb_readoptions_t; +} + +void leveldb_readoptions_destroy(leveldb_readoptions_t* opt) { + delete opt; +} + +void leveldb_readoptions_set_verify_checksums( + leveldb_readoptions_t* opt, + unsigned char v) { + opt->rep.verify_checksums = v; +} + +void leveldb_readoptions_set_fill_cache( + leveldb_readoptions_t* opt, unsigned char v) { + opt->rep.fill_cache = v; +} + +void leveldb_readoptions_set_snapshot( + leveldb_readoptions_t* opt, + const leveldb_snapshot_t* snap) { + opt->rep.snapshot = (snap ? snap->rep : NULL); +} + +leveldb_writeoptions_t* leveldb_writeoptions_create() { + return new leveldb_writeoptions_t; +} + +void leveldb_writeoptions_destroy(leveldb_writeoptions_t* opt) { + delete opt; +} + +void leveldb_writeoptions_set_sync( + leveldb_writeoptions_t* opt, unsigned char v) { + opt->rep.sync = v; +} + +leveldb_cache_t* leveldb_cache_create_lru(size_t capacity) { + leveldb_cache_t* c = new leveldb_cache_t; + c->rep = NewLRUCache(capacity); + return c; +} + +void leveldb_cache_destroy(leveldb_cache_t* cache) { + delete cache->rep; + delete cache; +} + +leveldb_env_t* leveldb_create_default_env() { + leveldb_env_t* result = new leveldb_env_t; + result->rep = Env::Default(); + result->is_default = true; + return result; +} + +void leveldb_env_destroy(leveldb_env_t* env) { + if (!env->is_default) delete env->rep; + delete env; +} + +void leveldb_free(void* ptr) { + free(ptr); +} + +int leveldb_major_version() { + return kMajorVersion; +} + +int leveldb_minor_version() { + return kMinorVersion; +} + +} // end extern "C" diff --git a/src/leveldb/db/c_test.c b/src/leveldb/db/c_test.c new file mode 100644 index 00000000..7cd5ee02 --- /dev/null +++ b/src/leveldb/db/c_test.c @@ -0,0 +1,390 @@ +/* Copyright (c) 2011 The LevelDB Authors. All rights reserved. + Use of this source code is governed by a BSD-style license that can be + found in the LICENSE file. See the AUTHORS file for names of contributors. */ + +#include "leveldb/c.h" + +#include +#include +#include +#include +#include +#include + +const char* phase = ""; +static char dbname[200]; + +static void StartPhase(const char* name) { + fprintf(stderr, "=== Test %s\n", name); + phase = name; +} + +static const char* GetTempDir(void) { + const char* ret = getenv("TEST_TMPDIR"); + if (ret == NULL || ret[0] == '\0') + ret = "/tmp"; + return ret; +} + +#define CheckNoError(err) \ + if ((err) != NULL) { \ + fprintf(stderr, "%s:%d: %s: %s\n", __FILE__, __LINE__, phase, (err)); \ + abort(); \ + } + +#define CheckCondition(cond) \ + if (!(cond)) { \ + fprintf(stderr, "%s:%d: %s: %s\n", __FILE__, __LINE__, phase, #cond); \ + abort(); \ + } + +static void CheckEqual(const char* expected, const char* v, size_t n) { + if (expected == NULL && v == NULL) { + // ok + } else if (expected != NULL && v != NULL && n == strlen(expected) && + memcmp(expected, v, n) == 0) { + // ok + return; + } else { + fprintf(stderr, "%s: expected '%s', got '%s'\n", + phase, + (expected ? expected : "(null)"), + (v ? v : "(null")); + abort(); + } +} + +static void Free(char** ptr) { + if (*ptr) { + free(*ptr); + *ptr = NULL; + } +} + +static void CheckGet( + leveldb_t* db, + const leveldb_readoptions_t* options, + const char* key, + const char* expected) { + char* err = NULL; + size_t val_len; + char* val; + val = leveldb_get(db, options, key, strlen(key), &val_len, &err); + CheckNoError(err); + CheckEqual(expected, val, val_len); + Free(&val); +} + +static void CheckIter(leveldb_iterator_t* iter, + const char* key, const char* val) { + size_t len; + const char* str; + str = leveldb_iter_key(iter, &len); + CheckEqual(key, str, len); + str = leveldb_iter_value(iter, &len); + CheckEqual(val, str, len); +} + +// Callback from leveldb_writebatch_iterate() +static void CheckPut(void* ptr, + const char* k, size_t klen, + const char* v, size_t vlen) { + int* state = (int*) ptr; + CheckCondition(*state < 2); + switch (*state) { + case 0: + CheckEqual("bar", k, klen); + CheckEqual("b", v, vlen); + break; + case 1: + CheckEqual("box", k, klen); + CheckEqual("c", v, vlen); + break; + } + (*state)++; +} + +// Callback from leveldb_writebatch_iterate() +static void CheckDel(void* ptr, const char* k, size_t klen) { + int* state = (int*) ptr; + CheckCondition(*state == 2); + CheckEqual("bar", k, klen); + (*state)++; +} + +static void CmpDestroy(void* arg) { } + +static int CmpCompare(void* arg, const char* a, size_t alen, + const char* b, size_t blen) { + int n = (alen < blen) ? alen : blen; + int r = memcmp(a, b, n); + if (r == 0) { + if (alen < blen) r = -1; + else if (alen > blen) r = +1; + } + return r; +} + +static const char* CmpName(void* arg) { + return "foo"; +} + +// Custom filter policy +static unsigned char fake_filter_result = 1; +static void FilterDestroy(void* arg) { } +static const char* FilterName(void* arg) { + return "TestFilter"; +} +static char* FilterCreate( + void* arg, + const char* const* key_array, const size_t* key_length_array, + int num_keys, + size_t* filter_length) { + *filter_length = 4; + char* result = malloc(4); + memcpy(result, "fake", 4); + return result; +} +unsigned char FilterKeyMatch( + void* arg, + const char* key, size_t length, + const char* filter, size_t filter_length) { + CheckCondition(filter_length == 4); + CheckCondition(memcmp(filter, "fake", 4) == 0); + return fake_filter_result; +} + +int main(int argc, char** argv) { + leveldb_t* db; + leveldb_comparator_t* cmp; + leveldb_cache_t* cache; + leveldb_env_t* env; + leveldb_options_t* options; + leveldb_readoptions_t* roptions; + leveldb_writeoptions_t* woptions; + char* err = NULL; + int run = -1; + + CheckCondition(leveldb_major_version() >= 1); + CheckCondition(leveldb_minor_version() >= 1); + + snprintf(dbname, sizeof(dbname), + "%s/leveldb_c_test-%d", + GetTempDir(), + ((int) geteuid())); + + StartPhase("create_objects"); + cmp = leveldb_comparator_create(NULL, CmpDestroy, CmpCompare, CmpName); + env = leveldb_create_default_env(); + cache = leveldb_cache_create_lru(100000); + + options = leveldb_options_create(); + leveldb_options_set_comparator(options, cmp); + leveldb_options_set_error_if_exists(options, 1); + leveldb_options_set_cache(options, cache); + leveldb_options_set_env(options, env); + leveldb_options_set_info_log(options, NULL); + leveldb_options_set_write_buffer_size(options, 100000); + leveldb_options_set_paranoid_checks(options, 1); + leveldb_options_set_max_open_files(options, 10); + leveldb_options_set_block_size(options, 1024); + leveldb_options_set_block_restart_interval(options, 8); + leveldb_options_set_compression(options, leveldb_no_compression); + + roptions = leveldb_readoptions_create(); + leveldb_readoptions_set_verify_checksums(roptions, 1); + leveldb_readoptions_set_fill_cache(roptions, 0); + + woptions = leveldb_writeoptions_create(); + leveldb_writeoptions_set_sync(woptions, 1); + + StartPhase("destroy"); + leveldb_destroy_db(options, dbname, &err); + Free(&err); + + StartPhase("open_error"); + db = leveldb_open(options, dbname, &err); + CheckCondition(err != NULL); + Free(&err); + + StartPhase("leveldb_free"); + db = leveldb_open(options, dbname, &err); + CheckCondition(err != NULL); + leveldb_free(err); + err = NULL; + + StartPhase("open"); + leveldb_options_set_create_if_missing(options, 1); + db = leveldb_open(options, dbname, &err); + CheckNoError(err); + CheckGet(db, roptions, "foo", NULL); + + StartPhase("put"); + leveldb_put(db, woptions, "foo", 3, "hello", 5, &err); + CheckNoError(err); + CheckGet(db, roptions, "foo", "hello"); + + StartPhase("compactall"); + leveldb_compact_range(db, NULL, 0, NULL, 0); + CheckGet(db, roptions, "foo", "hello"); + + StartPhase("compactrange"); + leveldb_compact_range(db, "a", 1, "z", 1); + CheckGet(db, roptions, "foo", "hello"); + + StartPhase("writebatch"); + { + leveldb_writebatch_t* wb = leveldb_writebatch_create(); + leveldb_writebatch_put(wb, "foo", 3, "a", 1); + leveldb_writebatch_clear(wb); + leveldb_writebatch_put(wb, "bar", 3, "b", 1); + leveldb_writebatch_put(wb, "box", 3, "c", 1); + leveldb_writebatch_delete(wb, "bar", 3); + leveldb_write(db, woptions, wb, &err); + CheckNoError(err); + CheckGet(db, roptions, "foo", "hello"); + CheckGet(db, roptions, "bar", NULL); + CheckGet(db, roptions, "box", "c"); + int pos = 0; + leveldb_writebatch_iterate(wb, &pos, CheckPut, CheckDel); + CheckCondition(pos == 3); + leveldb_writebatch_destroy(wb); + } + + StartPhase("iter"); + { + leveldb_iterator_t* iter = leveldb_create_iterator(db, roptions); + CheckCondition(!leveldb_iter_valid(iter)); + leveldb_iter_seek_to_first(iter); + CheckCondition(leveldb_iter_valid(iter)); + CheckIter(iter, "box", "c"); + leveldb_iter_next(iter); + CheckIter(iter, "foo", "hello"); + leveldb_iter_prev(iter); + CheckIter(iter, "box", "c"); + leveldb_iter_prev(iter); + CheckCondition(!leveldb_iter_valid(iter)); + leveldb_iter_seek_to_last(iter); + CheckIter(iter, "foo", "hello"); + leveldb_iter_seek(iter, "b", 1); + CheckIter(iter, "box", "c"); + leveldb_iter_get_error(iter, &err); + CheckNoError(err); + leveldb_iter_destroy(iter); + } + + StartPhase("approximate_sizes"); + { + int i; + int n = 20000; + char keybuf[100]; + char valbuf[100]; + uint64_t sizes[2]; + const char* start[2] = { "a", "k00000000000000010000" }; + size_t start_len[2] = { 1, 21 }; + const char* limit[2] = { "k00000000000000010000", "z" }; + size_t limit_len[2] = { 21, 1 }; + leveldb_writeoptions_set_sync(woptions, 0); + for (i = 0; i < n; i++) { + snprintf(keybuf, sizeof(keybuf), "k%020d", i); + snprintf(valbuf, sizeof(valbuf), "v%020d", i); + leveldb_put(db, woptions, keybuf, strlen(keybuf), valbuf, strlen(valbuf), + &err); + CheckNoError(err); + } + leveldb_approximate_sizes(db, 2, start, start_len, limit, limit_len, sizes); + CheckCondition(sizes[0] > 0); + CheckCondition(sizes[1] > 0); + } + + StartPhase("property"); + { + char* prop = leveldb_property_value(db, "nosuchprop"); + CheckCondition(prop == NULL); + prop = leveldb_property_value(db, "leveldb.stats"); + CheckCondition(prop != NULL); + Free(&prop); + } + + StartPhase("snapshot"); + { + const leveldb_snapshot_t* snap; + snap = leveldb_create_snapshot(db); + leveldb_delete(db, woptions, "foo", 3, &err); + CheckNoError(err); + leveldb_readoptions_set_snapshot(roptions, snap); + CheckGet(db, roptions, "foo", "hello"); + leveldb_readoptions_set_snapshot(roptions, NULL); + CheckGet(db, roptions, "foo", NULL); + leveldb_release_snapshot(db, snap); + } + + StartPhase("repair"); + { + leveldb_close(db); + leveldb_options_set_create_if_missing(options, 0); + leveldb_options_set_error_if_exists(options, 0); + leveldb_repair_db(options, dbname, &err); + CheckNoError(err); + db = leveldb_open(options, dbname, &err); + CheckNoError(err); + CheckGet(db, roptions, "foo", NULL); + CheckGet(db, roptions, "bar", NULL); + CheckGet(db, roptions, "box", "c"); + leveldb_options_set_create_if_missing(options, 1); + leveldb_options_set_error_if_exists(options, 1); + } + + StartPhase("filter"); + for (run = 0; run < 2; run++) { + // First run uses custom filter, second run uses bloom filter + CheckNoError(err); + leveldb_filterpolicy_t* policy; + if (run == 0) { + policy = leveldb_filterpolicy_create( + NULL, FilterDestroy, FilterCreate, FilterKeyMatch, FilterName); + } else { + policy = leveldb_filterpolicy_create_bloom(10); + } + + // Create new database + leveldb_close(db); + leveldb_destroy_db(options, dbname, &err); + leveldb_options_set_filter_policy(options, policy); + db = leveldb_open(options, dbname, &err); + CheckNoError(err); + leveldb_put(db, woptions, "foo", 3, "foovalue", 8, &err); + CheckNoError(err); + leveldb_put(db, woptions, "bar", 3, "barvalue", 8, &err); + CheckNoError(err); + leveldb_compact_range(db, NULL, 0, NULL, 0); + + fake_filter_result = 1; + CheckGet(db, roptions, "foo", "foovalue"); + CheckGet(db, roptions, "bar", "barvalue"); + if (phase == 0) { + // Must not find value when custom filter returns false + fake_filter_result = 0; + CheckGet(db, roptions, "foo", NULL); + CheckGet(db, roptions, "bar", NULL); + fake_filter_result = 1; + + CheckGet(db, roptions, "foo", "foovalue"); + CheckGet(db, roptions, "bar", "barvalue"); + } + leveldb_options_set_filter_policy(options, NULL); + leveldb_filterpolicy_destroy(policy); + } + + StartPhase("cleanup"); + leveldb_close(db); + leveldb_options_destroy(options); + leveldb_readoptions_destroy(roptions); + leveldb_writeoptions_destroy(woptions); + leveldb_cache_destroy(cache); + leveldb_comparator_destroy(cmp); + leveldb_env_destroy(env); + + fprintf(stderr, "PASS\n"); + return 0; +} diff --git a/src/leveldb/db/corruption_test.cc b/src/leveldb/db/corruption_test.cc new file mode 100644 index 00000000..96afc689 --- /dev/null +++ b/src/leveldb/db/corruption_test.cc @@ -0,0 +1,374 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/db.h" + +#include +#include +#include +#include +#include "leveldb/cache.h" +#include "leveldb/env.h" +#include "leveldb/table.h" +#include "leveldb/write_batch.h" +#include "db/db_impl.h" +#include "db/filename.h" +#include "db/log_format.h" +#include "db/version_set.h" +#include "util/logging.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace leveldb { + +static const int kValueSize = 1000; + +class CorruptionTest { + public: + test::ErrorEnv env_; + std::string dbname_; + Cache* tiny_cache_; + Options options_; + DB* db_; + + CorruptionTest() { + tiny_cache_ = NewLRUCache(100); + options_.env = &env_; + options_.block_cache = tiny_cache_; + dbname_ = test::TmpDir() + "/db_test"; + DestroyDB(dbname_, options_); + + db_ = NULL; + options_.create_if_missing = true; + Reopen(); + options_.create_if_missing = false; + } + + ~CorruptionTest() { + delete db_; + DestroyDB(dbname_, Options()); + delete tiny_cache_; + } + + Status TryReopen() { + delete db_; + db_ = NULL; + return DB::Open(options_, dbname_, &db_); + } + + void Reopen() { + ASSERT_OK(TryReopen()); + } + + void RepairDB() { + delete db_; + db_ = NULL; + ASSERT_OK(::leveldb::RepairDB(dbname_, options_)); + } + + void Build(int n) { + std::string key_space, value_space; + WriteBatch batch; + for (int i = 0; i < n; i++) { + //if ((i % 100) == 0) fprintf(stderr, "@ %d of %d\n", i, n); + Slice key = Key(i, &key_space); + batch.Clear(); + batch.Put(key, Value(i, &value_space)); + WriteOptions options; + // Corrupt() doesn't work without this sync on windows; stat reports 0 for + // the file size. + if (i == n - 1) { + options.sync = true; + } + ASSERT_OK(db_->Write(options, &batch)); + } + } + + void Check(int min_expected, int max_expected) { + int next_expected = 0; + int missed = 0; + int bad_keys = 0; + int bad_values = 0; + int correct = 0; + std::string value_space; + Iterator* iter = db_->NewIterator(ReadOptions()); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + uint64_t key; + Slice in(iter->key()); + if (in == "" || in == "~") { + // Ignore boundary keys. + continue; + } + if (!ConsumeDecimalNumber(&in, &key) || + !in.empty() || + key < next_expected) { + bad_keys++; + continue; + } + missed += (key - next_expected); + next_expected = key + 1; + if (iter->value() != Value(key, &value_space)) { + bad_values++; + } else { + correct++; + } + } + delete iter; + + fprintf(stderr, + "expected=%d..%d; got=%d; bad_keys=%d; bad_values=%d; missed=%d\n", + min_expected, max_expected, correct, bad_keys, bad_values, missed); + ASSERT_LE(min_expected, correct); + ASSERT_GE(max_expected, correct); + } + + void Corrupt(FileType filetype, int offset, int bytes_to_corrupt) { + // Pick file to corrupt + std::vector filenames; + ASSERT_OK(env_.GetChildren(dbname_, &filenames)); + uint64_t number; + FileType type; + std::string fname; + int picked_number = -1; + for (size_t i = 0; i < filenames.size(); i++) { + if (ParseFileName(filenames[i], &number, &type) && + type == filetype && + int(number) > picked_number) { // Pick latest file + fname = dbname_ + "/" + filenames[i]; + picked_number = number; + } + } + ASSERT_TRUE(!fname.empty()) << filetype; + + struct stat sbuf; + if (stat(fname.c_str(), &sbuf) != 0) { + const char* msg = strerror(errno); + ASSERT_TRUE(false) << fname << ": " << msg; + } + + if (offset < 0) { + // Relative to end of file; make it absolute + if (-offset > sbuf.st_size) { + offset = 0; + } else { + offset = sbuf.st_size + offset; + } + } + if (offset > sbuf.st_size) { + offset = sbuf.st_size; + } + if (offset + bytes_to_corrupt > sbuf.st_size) { + bytes_to_corrupt = sbuf.st_size - offset; + } + + // Do it + std::string contents; + Status s = ReadFileToString(Env::Default(), fname, &contents); + ASSERT_TRUE(s.ok()) << s.ToString(); + for (int i = 0; i < bytes_to_corrupt; i++) { + contents[i + offset] ^= 0x80; + } + s = WriteStringToFile(Env::Default(), contents, fname); + ASSERT_TRUE(s.ok()) << s.ToString(); + } + + int Property(const std::string& name) { + std::string property; + int result; + if (db_->GetProperty(name, &property) && + sscanf(property.c_str(), "%d", &result) == 1) { + return result; + } else { + return -1; + } + } + + // Return the ith key + Slice Key(int i, std::string* storage) { + char buf[100]; + snprintf(buf, sizeof(buf), "%016d", i); + storage->assign(buf, strlen(buf)); + return Slice(*storage); + } + + // Return the value to associate with the specified key + Slice Value(int k, std::string* storage) { + Random r(k); + return test::RandomString(&r, kValueSize, storage); + } +}; + +TEST(CorruptionTest, Recovery) { + Build(100); + Check(100, 100); + Corrupt(kLogFile, 19, 1); // WriteBatch tag for first record + Corrupt(kLogFile, log::kBlockSize + 1000, 1); // Somewhere in second block + Reopen(); + + // The 64 records in the first two log blocks are completely lost. + Check(36, 36); +} + +TEST(CorruptionTest, RecoverWriteError) { + env_.writable_file_error_ = true; + Status s = TryReopen(); + ASSERT_TRUE(!s.ok()); +} + +TEST(CorruptionTest, NewFileErrorDuringWrite) { + // Do enough writing to force minor compaction + env_.writable_file_error_ = true; + const int num = 3 + (Options().write_buffer_size / kValueSize); + std::string value_storage; + Status s; + for (int i = 0; s.ok() && i < num; i++) { + WriteBatch batch; + batch.Put("a", Value(100, &value_storage)); + s = db_->Write(WriteOptions(), &batch); + } + ASSERT_TRUE(!s.ok()); + ASSERT_GE(env_.num_writable_file_errors_, 1); + env_.writable_file_error_ = false; + Reopen(); +} + +TEST(CorruptionTest, TableFile) { + Build(100); + DBImpl* dbi = reinterpret_cast(db_); + dbi->TEST_CompactMemTable(); + dbi->TEST_CompactRange(0, NULL, NULL); + dbi->TEST_CompactRange(1, NULL, NULL); + + Corrupt(kTableFile, 100, 1); + Check(90, 99); +} + +TEST(CorruptionTest, TableFileRepair) { + options_.block_size = 2 * kValueSize; // Limit scope of corruption + options_.paranoid_checks = true; + Reopen(); + Build(100); + DBImpl* dbi = reinterpret_cast(db_); + dbi->TEST_CompactMemTable(); + dbi->TEST_CompactRange(0, NULL, NULL); + dbi->TEST_CompactRange(1, NULL, NULL); + + Corrupt(kTableFile, 100, 1); + RepairDB(); + Reopen(); + Check(95, 99); +} + +TEST(CorruptionTest, TableFileIndexData) { + Build(10000); // Enough to build multiple Tables + DBImpl* dbi = reinterpret_cast(db_); + dbi->TEST_CompactMemTable(); + + Corrupt(kTableFile, -2000, 500); + Reopen(); + Check(5000, 9999); +} + +TEST(CorruptionTest, MissingDescriptor) { + Build(1000); + RepairDB(); + Reopen(); + Check(1000, 1000); +} + +TEST(CorruptionTest, SequenceNumberRecovery) { + ASSERT_OK(db_->Put(WriteOptions(), "foo", "v1")); + ASSERT_OK(db_->Put(WriteOptions(), "foo", "v2")); + ASSERT_OK(db_->Put(WriteOptions(), "foo", "v3")); + ASSERT_OK(db_->Put(WriteOptions(), "foo", "v4")); + ASSERT_OK(db_->Put(WriteOptions(), "foo", "v5")); + RepairDB(); + Reopen(); + std::string v; + ASSERT_OK(db_->Get(ReadOptions(), "foo", &v)); + ASSERT_EQ("v5", v); + // Write something. If sequence number was not recovered properly, + // it will be hidden by an earlier write. + ASSERT_OK(db_->Put(WriteOptions(), "foo", "v6")); + ASSERT_OK(db_->Get(ReadOptions(), "foo", &v)); + ASSERT_EQ("v6", v); + Reopen(); + ASSERT_OK(db_->Get(ReadOptions(), "foo", &v)); + ASSERT_EQ("v6", v); +} + +TEST(CorruptionTest, CorruptedDescriptor) { + ASSERT_OK(db_->Put(WriteOptions(), "foo", "hello")); + DBImpl* dbi = reinterpret_cast(db_); + dbi->TEST_CompactMemTable(); + dbi->TEST_CompactRange(0, NULL, NULL); + + Corrupt(kDescriptorFile, 0, 1000); + Status s = TryReopen(); + ASSERT_TRUE(!s.ok()); + + RepairDB(); + Reopen(); + std::string v; + ASSERT_OK(db_->Get(ReadOptions(), "foo", &v)); + ASSERT_EQ("hello", v); +} + +TEST(CorruptionTest, CompactionInputError) { + Build(10); + DBImpl* dbi = reinterpret_cast(db_); + dbi->TEST_CompactMemTable(); + const int last = config::kMaxMemCompactLevel; + ASSERT_EQ(1, Property("leveldb.num-files-at-level" + NumberToString(last))); + + Corrupt(kTableFile, 100, 1); + Check(5, 9); + + // Force compactions by writing lots of values + Build(10000); + Check(10000, 10000); +} + +TEST(CorruptionTest, CompactionInputErrorParanoid) { + options_.paranoid_checks = true; + options_.write_buffer_size = 512 << 10; + Reopen(); + DBImpl* dbi = reinterpret_cast(db_); + + // Make multiple inputs so we need to compact. + for (int i = 0; i < 2; i++) { + Build(10); + dbi->TEST_CompactMemTable(); + Corrupt(kTableFile, 100, 1); + env_.SleepForMicroseconds(100000); + } + dbi->CompactRange(NULL, NULL); + + // Write must fail because of corrupted table + std::string tmp1, tmp2; + Status s = db_->Put(WriteOptions(), Key(5, &tmp1), Value(5, &tmp2)); + ASSERT_TRUE(!s.ok()) << "write did not fail in corrupted paranoid db"; +} + +TEST(CorruptionTest, UnrelatedKeys) { + Build(10); + DBImpl* dbi = reinterpret_cast(db_); + dbi->TEST_CompactMemTable(); + Corrupt(kTableFile, 100, 1); + + std::string tmp1, tmp2; + ASSERT_OK(db_->Put(WriteOptions(), Key(1000, &tmp1), Value(1000, &tmp2))); + std::string v; + ASSERT_OK(db_->Get(ReadOptions(), Key(1000, &tmp1), &v)); + ASSERT_EQ(Value(1000, &tmp2).ToString(), v); + dbi->TEST_CompactMemTable(); + ASSERT_OK(db_->Get(ReadOptions(), Key(1000, &tmp1), &v)); + ASSERT_EQ(Value(1000, &tmp2).ToString(), v); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/db/db_bench.cc b/src/leveldb/db/db_bench.cc new file mode 100644 index 00000000..705a170a --- /dev/null +++ b/src/leveldb/db/db_bench.cc @@ -0,0 +1,978 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include +#include +#include "db/db_impl.h" +#include "db/version_set.h" +#include "leveldb/cache.h" +#include "leveldb/db.h" +#include "leveldb/env.h" +#include "leveldb/write_batch.h" +#include "port/port.h" +#include "util/crc32c.h" +#include "util/histogram.h" +#include "util/mutexlock.h" +#include "util/random.h" +#include "util/testutil.h" + +// Comma-separated list of operations to run in the specified order +// Actual benchmarks: +// fillseq -- write N values in sequential key order in async mode +// fillrandom -- write N values in random key order in async mode +// overwrite -- overwrite N values in random key order in async mode +// fillsync -- write N/100 values in random key order in sync mode +// fill100K -- write N/1000 100K values in random order in async mode +// deleteseq -- delete N keys in sequential order +// deleterandom -- delete N keys in random order +// readseq -- read N times sequentially +// readreverse -- read N times in reverse order +// readrandom -- read N times in random order +// readmissing -- read N missing keys in random order +// readhot -- read N times in random order from 1% section of DB +// seekrandom -- N random seeks +// crc32c -- repeated crc32c of 4K of data +// acquireload -- load N*1000 times +// Meta operations: +// compact -- Compact the entire DB +// stats -- Print DB stats +// sstables -- Print sstable info +// heapprofile -- Dump a heap profile (if supported by this port) +static const char* FLAGS_benchmarks = + "fillseq," + "fillsync," + "fillrandom," + "overwrite," + "readrandom," + "readrandom," // Extra run to allow previous compactions to quiesce + "readseq," + "readreverse," + "compact," + "readrandom," + "readseq," + "readreverse," + "fill100K," + "crc32c," + "snappycomp," + "snappyuncomp," + "acquireload," + ; + +// Number of key/values to place in database +static int FLAGS_num = 1000000; + +// Number of read operations to do. If negative, do FLAGS_num reads. +static int FLAGS_reads = -1; + +// Number of concurrent threads to run. +static int FLAGS_threads = 1; + +// Size of each value +static int FLAGS_value_size = 100; + +// Arrange to generate values that shrink to this fraction of +// their original size after compression +static double FLAGS_compression_ratio = 0.5; + +// Print histogram of operation timings +static bool FLAGS_histogram = false; + +// Number of bytes to buffer in memtable before compacting +// (initialized to default value by "main") +static int FLAGS_write_buffer_size = 0; + +// Number of bytes to use as a cache of uncompressed data. +// Negative means use default settings. +static int FLAGS_cache_size = -1; + +// Maximum number of files to keep open at the same time (use default if == 0) +static int FLAGS_open_files = 0; + +// Bloom filter bits per key. +// Negative means use default settings. +static int FLAGS_bloom_bits = -1; + +// If true, do not destroy the existing database. If you set this +// flag and also specify a benchmark that wants a fresh database, that +// benchmark will fail. +static bool FLAGS_use_existing_db = false; + +// Use the db with the following name. +static const char* FLAGS_db = NULL; + +namespace leveldb { + +namespace { + +// Helper for quickly generating random data. +class RandomGenerator { + private: + std::string data_; + int pos_; + + public: + RandomGenerator() { + // We use a limited amount of data over and over again and ensure + // that it is larger than the compression window (32KB), and also + // large enough to serve all typical value sizes we want to write. + Random rnd(301); + std::string piece; + while (data_.size() < 1048576) { + // Add a short fragment that is as compressible as specified + // by FLAGS_compression_ratio. + test::CompressibleString(&rnd, FLAGS_compression_ratio, 100, &piece); + data_.append(piece); + } + pos_ = 0; + } + + Slice Generate(size_t len) { + if (pos_ + len > data_.size()) { + pos_ = 0; + assert(len < data_.size()); + } + pos_ += len; + return Slice(data_.data() + pos_ - len, len); + } +}; + +static Slice TrimSpace(Slice s) { + size_t start = 0; + while (start < s.size() && isspace(s[start])) { + start++; + } + size_t limit = s.size(); + while (limit > start && isspace(s[limit-1])) { + limit--; + } + return Slice(s.data() + start, limit - start); +} + +static void AppendWithSpace(std::string* str, Slice msg) { + if (msg.empty()) return; + if (!str->empty()) { + str->push_back(' '); + } + str->append(msg.data(), msg.size()); +} + +class Stats { + private: + double start_; + double finish_; + double seconds_; + int done_; + int next_report_; + int64_t bytes_; + double last_op_finish_; + Histogram hist_; + std::string message_; + + public: + Stats() { Start(); } + + void Start() { + next_report_ = 100; + last_op_finish_ = start_; + hist_.Clear(); + done_ = 0; + bytes_ = 0; + seconds_ = 0; + start_ = Env::Default()->NowMicros(); + finish_ = start_; + message_.clear(); + } + + void Merge(const Stats& other) { + hist_.Merge(other.hist_); + done_ += other.done_; + bytes_ += other.bytes_; + seconds_ += other.seconds_; + if (other.start_ < start_) start_ = other.start_; + if (other.finish_ > finish_) finish_ = other.finish_; + + // Just keep the messages from one thread + if (message_.empty()) message_ = other.message_; + } + + void Stop() { + finish_ = Env::Default()->NowMicros(); + seconds_ = (finish_ - start_) * 1e-6; + } + + void AddMessage(Slice msg) { + AppendWithSpace(&message_, msg); + } + + void FinishedSingleOp() { + if (FLAGS_histogram) { + double now = Env::Default()->NowMicros(); + double micros = now - last_op_finish_; + hist_.Add(micros); + if (micros > 20000) { + fprintf(stderr, "long op: %.1f micros%30s\r", micros, ""); + fflush(stderr); + } + last_op_finish_ = now; + } + + done_++; + if (done_ >= next_report_) { + if (next_report_ < 1000) next_report_ += 100; + else if (next_report_ < 5000) next_report_ += 500; + else if (next_report_ < 10000) next_report_ += 1000; + else if (next_report_ < 50000) next_report_ += 5000; + else if (next_report_ < 100000) next_report_ += 10000; + else if (next_report_ < 500000) next_report_ += 50000; + else next_report_ += 100000; + fprintf(stderr, "... finished %d ops%30s\r", done_, ""); + fflush(stderr); + } + } + + void AddBytes(int64_t n) { + bytes_ += n; + } + + void Report(const Slice& name) { + // Pretend at least one op was done in case we are running a benchmark + // that does not call FinishedSingleOp(). + if (done_ < 1) done_ = 1; + + std::string extra; + if (bytes_ > 0) { + // Rate is computed on actual elapsed time, not the sum of per-thread + // elapsed times. + double elapsed = (finish_ - start_) * 1e-6; + char rate[100]; + snprintf(rate, sizeof(rate), "%6.1f MB/s", + (bytes_ / 1048576.0) / elapsed); + extra = rate; + } + AppendWithSpace(&extra, message_); + + fprintf(stdout, "%-12s : %11.3f micros/op;%s%s\n", + name.ToString().c_str(), + seconds_ * 1e6 / done_, + (extra.empty() ? "" : " "), + extra.c_str()); + if (FLAGS_histogram) { + fprintf(stdout, "Microseconds per op:\n%s\n", hist_.ToString().c_str()); + } + fflush(stdout); + } +}; + +// State shared by all concurrent executions of the same benchmark. +struct SharedState { + port::Mutex mu; + port::CondVar cv; + int total; + + // Each thread goes through the following states: + // (1) initializing + // (2) waiting for others to be initialized + // (3) running + // (4) done + + int num_initialized; + int num_done; + bool start; + + SharedState() : cv(&mu) { } +}; + +// Per-thread state for concurrent executions of the same benchmark. +struct ThreadState { + int tid; // 0..n-1 when running in n threads + Random rand; // Has different seeds for different threads + Stats stats; + SharedState* shared; + + ThreadState(int index) + : tid(index), + rand(1000 + index) { + } +}; + +} // namespace + +class Benchmark { + private: + Cache* cache_; + const FilterPolicy* filter_policy_; + DB* db_; + int num_; + int value_size_; + int entries_per_batch_; + WriteOptions write_options_; + int reads_; + int heap_counter_; + + void PrintHeader() { + const int kKeySize = 16; + PrintEnvironment(); + fprintf(stdout, "Keys: %d bytes each\n", kKeySize); + fprintf(stdout, "Values: %d bytes each (%d bytes after compression)\n", + FLAGS_value_size, + static_cast(FLAGS_value_size * FLAGS_compression_ratio + 0.5)); + fprintf(stdout, "Entries: %d\n", num_); + fprintf(stdout, "RawSize: %.1f MB (estimated)\n", + ((static_cast(kKeySize + FLAGS_value_size) * num_) + / 1048576.0)); + fprintf(stdout, "FileSize: %.1f MB (estimated)\n", + (((kKeySize + FLAGS_value_size * FLAGS_compression_ratio) * num_) + / 1048576.0)); + PrintWarnings(); + fprintf(stdout, "------------------------------------------------\n"); + } + + void PrintWarnings() { +#if defined(__GNUC__) && !defined(__OPTIMIZE__) + fprintf(stdout, + "WARNING: Optimization is disabled: benchmarks unnecessarily slow\n" + ); +#endif +#ifndef NDEBUG + fprintf(stdout, + "WARNING: Assertions are enabled; benchmarks unnecessarily slow\n"); +#endif + + // See if snappy is working by attempting to compress a compressible string + const char text[] = "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy"; + std::string compressed; + if (!port::Snappy_Compress(text, sizeof(text), &compressed)) { + fprintf(stdout, "WARNING: Snappy compression is not enabled\n"); + } else if (compressed.size() >= sizeof(text)) { + fprintf(stdout, "WARNING: Snappy compression is not effective\n"); + } + } + + void PrintEnvironment() { + fprintf(stderr, "LevelDB: version %d.%d\n", + kMajorVersion, kMinorVersion); + +#if defined(__linux) + time_t now = time(NULL); + fprintf(stderr, "Date: %s", ctime(&now)); // ctime() adds newline + + FILE* cpuinfo = fopen("/proc/cpuinfo", "r"); + if (cpuinfo != NULL) { + char line[1000]; + int num_cpus = 0; + std::string cpu_type; + std::string cache_size; + while (fgets(line, sizeof(line), cpuinfo) != NULL) { + const char* sep = strchr(line, ':'); + if (sep == NULL) { + continue; + } + Slice key = TrimSpace(Slice(line, sep - 1 - line)); + Slice val = TrimSpace(Slice(sep + 1)); + if (key == "model name") { + ++num_cpus; + cpu_type = val.ToString(); + } else if (key == "cache size") { + cache_size = val.ToString(); + } + } + fclose(cpuinfo); + fprintf(stderr, "CPU: %d * %s\n", num_cpus, cpu_type.c_str()); + fprintf(stderr, "CPUCache: %s\n", cache_size.c_str()); + } +#endif + } + + public: + Benchmark() + : cache_(FLAGS_cache_size >= 0 ? NewLRUCache(FLAGS_cache_size) : NULL), + filter_policy_(FLAGS_bloom_bits >= 0 + ? NewBloomFilterPolicy(FLAGS_bloom_bits) + : NULL), + db_(NULL), + num_(FLAGS_num), + value_size_(FLAGS_value_size), + entries_per_batch_(1), + reads_(FLAGS_reads < 0 ? FLAGS_num : FLAGS_reads), + heap_counter_(0) { + std::vector files; + Env::Default()->GetChildren(FLAGS_db, &files); + for (size_t i = 0; i < files.size(); i++) { + if (Slice(files[i]).starts_with("heap-")) { + Env::Default()->DeleteFile(std::string(FLAGS_db) + "/" + files[i]); + } + } + if (!FLAGS_use_existing_db) { + DestroyDB(FLAGS_db, Options()); + } + } + + ~Benchmark() { + delete db_; + delete cache_; + delete filter_policy_; + } + + void Run() { + PrintHeader(); + Open(); + + const char* benchmarks = FLAGS_benchmarks; + while (benchmarks != NULL) { + const char* sep = strchr(benchmarks, ','); + Slice name; + if (sep == NULL) { + name = benchmarks; + benchmarks = NULL; + } else { + name = Slice(benchmarks, sep - benchmarks); + benchmarks = sep + 1; + } + + // Reset parameters that may be overridden below + num_ = FLAGS_num; + reads_ = (FLAGS_reads < 0 ? FLAGS_num : FLAGS_reads); + value_size_ = FLAGS_value_size; + entries_per_batch_ = 1; + write_options_ = WriteOptions(); + + void (Benchmark::*method)(ThreadState*) = NULL; + bool fresh_db = false; + int num_threads = FLAGS_threads; + + if (name == Slice("fillseq")) { + fresh_db = true; + method = &Benchmark::WriteSeq; + } else if (name == Slice("fillbatch")) { + fresh_db = true; + entries_per_batch_ = 1000; + method = &Benchmark::WriteSeq; + } else if (name == Slice("fillrandom")) { + fresh_db = true; + method = &Benchmark::WriteRandom; + } else if (name == Slice("overwrite")) { + fresh_db = false; + method = &Benchmark::WriteRandom; + } else if (name == Slice("fillsync")) { + fresh_db = true; + num_ /= 1000; + write_options_.sync = true; + method = &Benchmark::WriteRandom; + } else if (name == Slice("fill100K")) { + fresh_db = true; + num_ /= 1000; + value_size_ = 100 * 1000; + method = &Benchmark::WriteRandom; + } else if (name == Slice("readseq")) { + method = &Benchmark::ReadSequential; + } else if (name == Slice("readreverse")) { + method = &Benchmark::ReadReverse; + } else if (name == Slice("readrandom")) { + method = &Benchmark::ReadRandom; + } else if (name == Slice("readmissing")) { + method = &Benchmark::ReadMissing; + } else if (name == Slice("seekrandom")) { + method = &Benchmark::SeekRandom; + } else if (name == Slice("readhot")) { + method = &Benchmark::ReadHot; + } else if (name == Slice("readrandomsmall")) { + reads_ /= 1000; + method = &Benchmark::ReadRandom; + } else if (name == Slice("deleteseq")) { + method = &Benchmark::DeleteSeq; + } else if (name == Slice("deleterandom")) { + method = &Benchmark::DeleteRandom; + } else if (name == Slice("readwhilewriting")) { + num_threads++; // Add extra thread for writing + method = &Benchmark::ReadWhileWriting; + } else if (name == Slice("compact")) { + method = &Benchmark::Compact; + } else if (name == Slice("crc32c")) { + method = &Benchmark::Crc32c; + } else if (name == Slice("acquireload")) { + method = &Benchmark::AcquireLoad; + } else if (name == Slice("snappycomp")) { + method = &Benchmark::SnappyCompress; + } else if (name == Slice("snappyuncomp")) { + method = &Benchmark::SnappyUncompress; + } else if (name == Slice("heapprofile")) { + HeapProfile(); + } else if (name == Slice("stats")) { + PrintStats("leveldb.stats"); + } else if (name == Slice("sstables")) { + PrintStats("leveldb.sstables"); + } else { + if (name != Slice()) { // No error message for empty name + fprintf(stderr, "unknown benchmark '%s'\n", name.ToString().c_str()); + } + } + + if (fresh_db) { + if (FLAGS_use_existing_db) { + fprintf(stdout, "%-12s : skipped (--use_existing_db is true)\n", + name.ToString().c_str()); + method = NULL; + } else { + delete db_; + db_ = NULL; + DestroyDB(FLAGS_db, Options()); + Open(); + } + } + + if (method != NULL) { + RunBenchmark(num_threads, name, method); + } + } + } + + private: + struct ThreadArg { + Benchmark* bm; + SharedState* shared; + ThreadState* thread; + void (Benchmark::*method)(ThreadState*); + }; + + static void ThreadBody(void* v) { + ThreadArg* arg = reinterpret_cast(v); + SharedState* shared = arg->shared; + ThreadState* thread = arg->thread; + { + MutexLock l(&shared->mu); + shared->num_initialized++; + if (shared->num_initialized >= shared->total) { + shared->cv.SignalAll(); + } + while (!shared->start) { + shared->cv.Wait(); + } + } + + thread->stats.Start(); + (arg->bm->*(arg->method))(thread); + thread->stats.Stop(); + + { + MutexLock l(&shared->mu); + shared->num_done++; + if (shared->num_done >= shared->total) { + shared->cv.SignalAll(); + } + } + } + + void RunBenchmark(int n, Slice name, + void (Benchmark::*method)(ThreadState*)) { + SharedState shared; + shared.total = n; + shared.num_initialized = 0; + shared.num_done = 0; + shared.start = false; + + ThreadArg* arg = new ThreadArg[n]; + for (int i = 0; i < n; i++) { + arg[i].bm = this; + arg[i].method = method; + arg[i].shared = &shared; + arg[i].thread = new ThreadState(i); + arg[i].thread->shared = &shared; + Env::Default()->StartThread(ThreadBody, &arg[i]); + } + + shared.mu.Lock(); + while (shared.num_initialized < n) { + shared.cv.Wait(); + } + + shared.start = true; + shared.cv.SignalAll(); + while (shared.num_done < n) { + shared.cv.Wait(); + } + shared.mu.Unlock(); + + for (int i = 1; i < n; i++) { + arg[0].thread->stats.Merge(arg[i].thread->stats); + } + arg[0].thread->stats.Report(name); + + for (int i = 0; i < n; i++) { + delete arg[i].thread; + } + delete[] arg; + } + + void Crc32c(ThreadState* thread) { + // Checksum about 500MB of data total + const int size = 4096; + const char* label = "(4K per op)"; + std::string data(size, 'x'); + int64_t bytes = 0; + uint32_t crc = 0; + while (bytes < 500 * 1048576) { + crc = crc32c::Value(data.data(), size); + thread->stats.FinishedSingleOp(); + bytes += size; + } + // Print so result is not dead + fprintf(stderr, "... crc=0x%x\r", static_cast(crc)); + + thread->stats.AddBytes(bytes); + thread->stats.AddMessage(label); + } + + void AcquireLoad(ThreadState* thread) { + int dummy; + port::AtomicPointer ap(&dummy); + int count = 0; + void *ptr = NULL; + thread->stats.AddMessage("(each op is 1000 loads)"); + while (count < 100000) { + for (int i = 0; i < 1000; i++) { + ptr = ap.Acquire_Load(); + } + count++; + thread->stats.FinishedSingleOp(); + } + if (ptr == NULL) exit(1); // Disable unused variable warning. + } + + void SnappyCompress(ThreadState* thread) { + RandomGenerator gen; + Slice input = gen.Generate(Options().block_size); + int64_t bytes = 0; + int64_t produced = 0; + bool ok = true; + std::string compressed; + while (ok && bytes < 1024 * 1048576) { // Compress 1G + ok = port::Snappy_Compress(input.data(), input.size(), &compressed); + produced += compressed.size(); + bytes += input.size(); + thread->stats.FinishedSingleOp(); + } + + if (!ok) { + thread->stats.AddMessage("(snappy failure)"); + } else { + char buf[100]; + snprintf(buf, sizeof(buf), "(output: %.1f%%)", + (produced * 100.0) / bytes); + thread->stats.AddMessage(buf); + thread->stats.AddBytes(bytes); + } + } + + void SnappyUncompress(ThreadState* thread) { + RandomGenerator gen; + Slice input = gen.Generate(Options().block_size); + std::string compressed; + bool ok = port::Snappy_Compress(input.data(), input.size(), &compressed); + int64_t bytes = 0; + char* uncompressed = new char[input.size()]; + while (ok && bytes < 1024 * 1048576) { // Compress 1G + ok = port::Snappy_Uncompress(compressed.data(), compressed.size(), + uncompressed); + bytes += input.size(); + thread->stats.FinishedSingleOp(); + } + delete[] uncompressed; + + if (!ok) { + thread->stats.AddMessage("(snappy failure)"); + } else { + thread->stats.AddBytes(bytes); + } + } + + void Open() { + assert(db_ == NULL); + Options options; + options.create_if_missing = !FLAGS_use_existing_db; + options.block_cache = cache_; + options.write_buffer_size = FLAGS_write_buffer_size; + options.max_open_files = FLAGS_open_files; + options.filter_policy = filter_policy_; + Status s = DB::Open(options, FLAGS_db, &db_); + if (!s.ok()) { + fprintf(stderr, "open error: %s\n", s.ToString().c_str()); + exit(1); + } + } + + void WriteSeq(ThreadState* thread) { + DoWrite(thread, true); + } + + void WriteRandom(ThreadState* thread) { + DoWrite(thread, false); + } + + void DoWrite(ThreadState* thread, bool seq) { + if (num_ != FLAGS_num) { + char msg[100]; + snprintf(msg, sizeof(msg), "(%d ops)", num_); + thread->stats.AddMessage(msg); + } + + RandomGenerator gen; + WriteBatch batch; + Status s; + int64_t bytes = 0; + for (int i = 0; i < num_; i += entries_per_batch_) { + batch.Clear(); + for (int j = 0; j < entries_per_batch_; j++) { + const int k = seq ? i+j : (thread->rand.Next() % FLAGS_num); + char key[100]; + snprintf(key, sizeof(key), "%016d", k); + batch.Put(key, gen.Generate(value_size_)); + bytes += value_size_ + strlen(key); + thread->stats.FinishedSingleOp(); + } + s = db_->Write(write_options_, &batch); + if (!s.ok()) { + fprintf(stderr, "put error: %s\n", s.ToString().c_str()); + exit(1); + } + } + thread->stats.AddBytes(bytes); + } + + void ReadSequential(ThreadState* thread) { + Iterator* iter = db_->NewIterator(ReadOptions()); + int i = 0; + int64_t bytes = 0; + for (iter->SeekToFirst(); i < reads_ && iter->Valid(); iter->Next()) { + bytes += iter->key().size() + iter->value().size(); + thread->stats.FinishedSingleOp(); + ++i; + } + delete iter; + thread->stats.AddBytes(bytes); + } + + void ReadReverse(ThreadState* thread) { + Iterator* iter = db_->NewIterator(ReadOptions()); + int i = 0; + int64_t bytes = 0; + for (iter->SeekToLast(); i < reads_ && iter->Valid(); iter->Prev()) { + bytes += iter->key().size() + iter->value().size(); + thread->stats.FinishedSingleOp(); + ++i; + } + delete iter; + thread->stats.AddBytes(bytes); + } + + void ReadRandom(ThreadState* thread) { + ReadOptions options; + std::string value; + int found = 0; + for (int i = 0; i < reads_; i++) { + char key[100]; + const int k = thread->rand.Next() % FLAGS_num; + snprintf(key, sizeof(key), "%016d", k); + if (db_->Get(options, key, &value).ok()) { + found++; + } + thread->stats.FinishedSingleOp(); + } + char msg[100]; + snprintf(msg, sizeof(msg), "(%d of %d found)", found, num_); + thread->stats.AddMessage(msg); + } + + void ReadMissing(ThreadState* thread) { + ReadOptions options; + std::string value; + for (int i = 0; i < reads_; i++) { + char key[100]; + const int k = thread->rand.Next() % FLAGS_num; + snprintf(key, sizeof(key), "%016d.", k); + db_->Get(options, key, &value); + thread->stats.FinishedSingleOp(); + } + } + + void ReadHot(ThreadState* thread) { + ReadOptions options; + std::string value; + const int range = (FLAGS_num + 99) / 100; + for (int i = 0; i < reads_; i++) { + char key[100]; + const int k = thread->rand.Next() % range; + snprintf(key, sizeof(key), "%016d", k); + db_->Get(options, key, &value); + thread->stats.FinishedSingleOp(); + } + } + + void SeekRandom(ThreadState* thread) { + ReadOptions options; + int found = 0; + for (int i = 0; i < reads_; i++) { + Iterator* iter = db_->NewIterator(options); + char key[100]; + const int k = thread->rand.Next() % FLAGS_num; + snprintf(key, sizeof(key), "%016d", k); + iter->Seek(key); + if (iter->Valid() && iter->key() == key) found++; + delete iter; + thread->stats.FinishedSingleOp(); + } + char msg[100]; + snprintf(msg, sizeof(msg), "(%d of %d found)", found, num_); + thread->stats.AddMessage(msg); + } + + void DoDelete(ThreadState* thread, bool seq) { + RandomGenerator gen; + WriteBatch batch; + Status s; + for (int i = 0; i < num_; i += entries_per_batch_) { + batch.Clear(); + for (int j = 0; j < entries_per_batch_; j++) { + const int k = seq ? i+j : (thread->rand.Next() % FLAGS_num); + char key[100]; + snprintf(key, sizeof(key), "%016d", k); + batch.Delete(key); + thread->stats.FinishedSingleOp(); + } + s = db_->Write(write_options_, &batch); + if (!s.ok()) { + fprintf(stderr, "del error: %s\n", s.ToString().c_str()); + exit(1); + } + } + } + + void DeleteSeq(ThreadState* thread) { + DoDelete(thread, true); + } + + void DeleteRandom(ThreadState* thread) { + DoDelete(thread, false); + } + + void ReadWhileWriting(ThreadState* thread) { + if (thread->tid > 0) { + ReadRandom(thread); + } else { + // Special thread that keeps writing until other threads are done. + RandomGenerator gen; + while (true) { + { + MutexLock l(&thread->shared->mu); + if (thread->shared->num_done + 1 >= thread->shared->num_initialized) { + // Other threads have finished + break; + } + } + + const int k = thread->rand.Next() % FLAGS_num; + char key[100]; + snprintf(key, sizeof(key), "%016d", k); + Status s = db_->Put(write_options_, key, gen.Generate(value_size_)); + if (!s.ok()) { + fprintf(stderr, "put error: %s\n", s.ToString().c_str()); + exit(1); + } + } + + // Do not count any of the preceding work/delay in stats. + thread->stats.Start(); + } + } + + void Compact(ThreadState* thread) { + db_->CompactRange(NULL, NULL); + } + + void PrintStats(const char* key) { + std::string stats; + if (!db_->GetProperty(key, &stats)) { + stats = "(failed)"; + } + fprintf(stdout, "\n%s\n", stats.c_str()); + } + + static void WriteToFile(void* arg, const char* buf, int n) { + reinterpret_cast(arg)->Append(Slice(buf, n)); + } + + void HeapProfile() { + char fname[100]; + snprintf(fname, sizeof(fname), "%s/heap-%04d", FLAGS_db, ++heap_counter_); + WritableFile* file; + Status s = Env::Default()->NewWritableFile(fname, &file); + if (!s.ok()) { + fprintf(stderr, "%s\n", s.ToString().c_str()); + return; + } + bool ok = port::GetHeapProfile(WriteToFile, file); + delete file; + if (!ok) { + fprintf(stderr, "heap profiling not supported\n"); + Env::Default()->DeleteFile(fname); + } + } +}; + +} // namespace leveldb + +int main(int argc, char** argv) { + FLAGS_write_buffer_size = leveldb::Options().write_buffer_size; + FLAGS_open_files = leveldb::Options().max_open_files; + std::string default_db_path; + + for (int i = 1; i < argc; i++) { + double d; + int n; + char junk; + if (leveldb::Slice(argv[i]).starts_with("--benchmarks=")) { + FLAGS_benchmarks = argv[i] + strlen("--benchmarks="); + } else if (sscanf(argv[i], "--compression_ratio=%lf%c", &d, &junk) == 1) { + FLAGS_compression_ratio = d; + } else if (sscanf(argv[i], "--histogram=%d%c", &n, &junk) == 1 && + (n == 0 || n == 1)) { + FLAGS_histogram = n; + } else if (sscanf(argv[i], "--use_existing_db=%d%c", &n, &junk) == 1 && + (n == 0 || n == 1)) { + FLAGS_use_existing_db = n; + } else if (sscanf(argv[i], "--num=%d%c", &n, &junk) == 1) { + FLAGS_num = n; + } else if (sscanf(argv[i], "--reads=%d%c", &n, &junk) == 1) { + FLAGS_reads = n; + } else if (sscanf(argv[i], "--threads=%d%c", &n, &junk) == 1) { + FLAGS_threads = n; + } else if (sscanf(argv[i], "--value_size=%d%c", &n, &junk) == 1) { + FLAGS_value_size = n; + } else if (sscanf(argv[i], "--write_buffer_size=%d%c", &n, &junk) == 1) { + FLAGS_write_buffer_size = n; + } else if (sscanf(argv[i], "--cache_size=%d%c", &n, &junk) == 1) { + FLAGS_cache_size = n; + } else if (sscanf(argv[i], "--bloom_bits=%d%c", &n, &junk) == 1) { + FLAGS_bloom_bits = n; + } else if (sscanf(argv[i], "--open_files=%d%c", &n, &junk) == 1) { + FLAGS_open_files = n; + } else if (strncmp(argv[i], "--db=", 5) == 0) { + FLAGS_db = argv[i] + 5; + } else { + fprintf(stderr, "Invalid flag '%s'\n", argv[i]); + exit(1); + } + } + + // Choose a location for the test database if none given with --db= + if (FLAGS_db == NULL) { + leveldb::Env::Default()->GetTestDirectory(&default_db_path); + default_db_path += "/dbbench"; + FLAGS_db = default_db_path.c_str(); + } + + leveldb::Benchmark benchmark; + benchmark.Run(); + return 0; +} diff --git a/src/leveldb/db/db_impl.cc b/src/leveldb/db/db_impl.cc new file mode 100644 index 00000000..49b95953 --- /dev/null +++ b/src/leveldb/db/db_impl.cc @@ -0,0 +1,1513 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_impl.h" + +#include +#include +#include +#include +#include +#include +#include "db/builder.h" +#include "db/db_iter.h" +#include "db/dbformat.h" +#include "db/filename.h" +#include "db/log_reader.h" +#include "db/log_writer.h" +#include "db/memtable.h" +#include "db/table_cache.h" +#include "db/version_set.h" +#include "db/write_batch_internal.h" +#include "leveldb/db.h" +#include "leveldb/env.h" +#include "leveldb/status.h" +#include "leveldb/table.h" +#include "leveldb/table_builder.h" +#include "port/port.h" +#include "table/block.h" +#include "table/merger.h" +#include "table/two_level_iterator.h" +#include "util/coding.h" +#include "util/logging.h" +#include "util/mutexlock.h" + +namespace leveldb { + +const int kNumNonTableCacheFiles = 10; + +// Information kept for every waiting writer +struct DBImpl::Writer { + Status status; + WriteBatch* batch; + bool sync; + bool done; + port::CondVar cv; + + explicit Writer(port::Mutex* mu) : cv(mu) { } +}; + +struct DBImpl::CompactionState { + Compaction* const compaction; + + // Sequence numbers < smallest_snapshot are not significant since we + // will never have to service a snapshot below smallest_snapshot. + // Therefore if we have seen a sequence number S <= smallest_snapshot, + // we can drop all entries for the same key with sequence numbers < S. + SequenceNumber smallest_snapshot; + + // Files produced by compaction + struct Output { + uint64_t number; + uint64_t file_size; + InternalKey smallest, largest; + }; + std::vector outputs; + + // State kept for output being generated + WritableFile* outfile; + TableBuilder* builder; + + uint64_t total_bytes; + + Output* current_output() { return &outputs[outputs.size()-1]; } + + explicit CompactionState(Compaction* c) + : compaction(c), + outfile(NULL), + builder(NULL), + total_bytes(0) { + } +}; + +// Fix user-supplied options to be reasonable +template +static void ClipToRange(T* ptr, V minvalue, V maxvalue) { + if (static_cast(*ptr) > maxvalue) *ptr = maxvalue; + if (static_cast(*ptr) < minvalue) *ptr = minvalue; +} +Options SanitizeOptions(const std::string& dbname, + const InternalKeyComparator* icmp, + const InternalFilterPolicy* ipolicy, + const Options& src) { + Options result = src; + result.comparator = icmp; + result.filter_policy = (src.filter_policy != NULL) ? ipolicy : NULL; + ClipToRange(&result.max_open_files, 64 + kNumNonTableCacheFiles, 50000); + ClipToRange(&result.write_buffer_size, 64<<10, 1<<30); + ClipToRange(&result.block_size, 1<<10, 4<<20); + if (result.info_log == NULL) { + // Open a log file in the same directory as the db + src.env->CreateDir(dbname); // In case it does not exist + src.env->RenameFile(InfoLogFileName(dbname), OldInfoLogFileName(dbname)); + Status s = src.env->NewLogger(InfoLogFileName(dbname), &result.info_log); + if (!s.ok()) { + // No place suitable for logging + result.info_log = NULL; + } + } + if (result.block_cache == NULL) { + result.block_cache = NewLRUCache(8 << 20); + } + return result; +} + +DBImpl::DBImpl(const Options& raw_options, const std::string& dbname) + : env_(raw_options.env), + internal_comparator_(raw_options.comparator), + internal_filter_policy_(raw_options.filter_policy), + options_(SanitizeOptions(dbname, &internal_comparator_, + &internal_filter_policy_, raw_options)), + owns_info_log_(options_.info_log != raw_options.info_log), + owns_cache_(options_.block_cache != raw_options.block_cache), + dbname_(dbname), + db_lock_(NULL), + shutting_down_(NULL), + bg_cv_(&mutex_), + mem_(new MemTable(internal_comparator_)), + imm_(NULL), + logfile_(NULL), + logfile_number_(0), + log_(NULL), + seed_(0), + tmp_batch_(new WriteBatch), + bg_compaction_scheduled_(false), + manual_compaction_(NULL) { + mem_->Ref(); + has_imm_.Release_Store(NULL); + + // Reserve ten files or so for other uses and give the rest to TableCache. + const int table_cache_size = options_.max_open_files - kNumNonTableCacheFiles; + table_cache_ = new TableCache(dbname_, &options_, table_cache_size); + + versions_ = new VersionSet(dbname_, &options_, table_cache_, + &internal_comparator_); +} + +DBImpl::~DBImpl() { + // Wait for background work to finish + mutex_.Lock(); + shutting_down_.Release_Store(this); // Any non-NULL value is ok + while (bg_compaction_scheduled_) { + bg_cv_.Wait(); + } + mutex_.Unlock(); + + if (db_lock_ != NULL) { + env_->UnlockFile(db_lock_); + } + + delete versions_; + if (mem_ != NULL) mem_->Unref(); + if (imm_ != NULL) imm_->Unref(); + delete tmp_batch_; + delete log_; + delete logfile_; + delete table_cache_; + + if (owns_info_log_) { + delete options_.info_log; + } + if (owns_cache_) { + delete options_.block_cache; + } +} + +Status DBImpl::NewDB() { + VersionEdit new_db; + new_db.SetComparatorName(user_comparator()->Name()); + new_db.SetLogNumber(0); + new_db.SetNextFile(2); + new_db.SetLastSequence(0); + + const std::string manifest = DescriptorFileName(dbname_, 1); + WritableFile* file; + Status s = env_->NewWritableFile(manifest, &file); + if (!s.ok()) { + return s; + } + { + log::Writer log(file); + std::string record; + new_db.EncodeTo(&record); + s = log.AddRecord(record); + if (s.ok()) { + s = file->Close(); + } + } + delete file; + if (s.ok()) { + // Make "CURRENT" file that points to the new manifest file. + s = SetCurrentFile(env_, dbname_, 1); + } else { + env_->DeleteFile(manifest); + } + return s; +} + +void DBImpl::MaybeIgnoreError(Status* s) const { + if (s->ok() || options_.paranoid_checks) { + // No change needed + } else { + Log(options_.info_log, "Ignoring error %s", s->ToString().c_str()); + *s = Status::OK(); + } +} + +void DBImpl::DeleteObsoleteFiles() { + if (!bg_error_.ok()) { + // After a background error, we don't know whether a new version may + // or may not have been committed, so we cannot safely garbage collect. + return; + } + + // Make a set of all of the live files + std::set live = pending_outputs_; + versions_->AddLiveFiles(&live); + + std::vector filenames; + env_->GetChildren(dbname_, &filenames); // Ignoring errors on purpose + uint64_t number; + FileType type; + for (size_t i = 0; i < filenames.size(); i++) { + if (ParseFileName(filenames[i], &number, &type)) { + bool keep = true; + switch (type) { + case kLogFile: + keep = ((number >= versions_->LogNumber()) || + (number == versions_->PrevLogNumber())); + break; + case kDescriptorFile: + // Keep my manifest file, and any newer incarnations' + // (in case there is a race that allows other incarnations) + keep = (number >= versions_->ManifestFileNumber()); + break; + case kTableFile: + keep = (live.find(number) != live.end()); + break; + case kTempFile: + // Any temp files that are currently being written to must + // be recorded in pending_outputs_, which is inserted into "live" + keep = (live.find(number) != live.end()); + break; + case kCurrentFile: + case kDBLockFile: + case kInfoLogFile: + keep = true; + break; + } + + if (!keep) { + if (type == kTableFile) { + table_cache_->Evict(number); + } + Log(options_.info_log, "Delete type=%d #%lld\n", + int(type), + static_cast(number)); + env_->DeleteFile(dbname_ + "/" + filenames[i]); + } + } + } +} + +Status DBImpl::Recover(VersionEdit* edit) { + mutex_.AssertHeld(); + + // Ignore error from CreateDir since the creation of the DB is + // committed only when the descriptor is created, and this directory + // may already exist from a previous failed creation attempt. + env_->CreateDir(dbname_); + assert(db_lock_ == NULL); + Status s = env_->LockFile(LockFileName(dbname_), &db_lock_); + if (!s.ok()) { + return s; + } + + if (!env_->FileExists(CurrentFileName(dbname_))) { + if (options_.create_if_missing) { + s = NewDB(); + if (!s.ok()) { + return s; + } + } else { + return Status::InvalidArgument( + dbname_, "does not exist (create_if_missing is false)"); + } + } else { + if (options_.error_if_exists) { + return Status::InvalidArgument( + dbname_, "exists (error_if_exists is true)"); + } + } + + s = versions_->Recover(); + if (s.ok()) { + SequenceNumber max_sequence(0); + + // Recover from all newer log files than the ones named in the + // descriptor (new log files may have been added by the previous + // incarnation without registering them in the descriptor). + // + // Note that PrevLogNumber() is no longer used, but we pay + // attention to it in case we are recovering a database + // produced by an older version of leveldb. + const uint64_t min_log = versions_->LogNumber(); + const uint64_t prev_log = versions_->PrevLogNumber(); + std::vector filenames; + s = env_->GetChildren(dbname_, &filenames); + if (!s.ok()) { + return s; + } + std::set expected; + versions_->AddLiveFiles(&expected); + uint64_t number; + FileType type; + std::vector logs; + for (size_t i = 0; i < filenames.size(); i++) { + if (ParseFileName(filenames[i], &number, &type)) { + expected.erase(number); + if (type == kLogFile && ((number >= min_log) || (number == prev_log))) + logs.push_back(number); + } + } + if (!expected.empty()) { + char buf[50]; + snprintf(buf, sizeof(buf), "%d missing files; e.g.", + static_cast(expected.size())); + return Status::Corruption(buf, TableFileName(dbname_, *(expected.begin()))); + } + + // Recover in the order in which the logs were generated + std::sort(logs.begin(), logs.end()); + for (size_t i = 0; i < logs.size(); i++) { + s = RecoverLogFile(logs[i], edit, &max_sequence); + + // The previous incarnation may not have written any MANIFEST + // records after allocating this log number. So we manually + // update the file number allocation counter in VersionSet. + versions_->MarkFileNumberUsed(logs[i]); + } + + if (s.ok()) { + if (versions_->LastSequence() < max_sequence) { + versions_->SetLastSequence(max_sequence); + } + } + } + + return s; +} + +Status DBImpl::RecoverLogFile(uint64_t log_number, + VersionEdit* edit, + SequenceNumber* max_sequence) { + struct LogReporter : public log::Reader::Reporter { + Env* env; + Logger* info_log; + const char* fname; + Status* status; // NULL if options_.paranoid_checks==false + virtual void Corruption(size_t bytes, const Status& s) { + Log(info_log, "%s%s: dropping %d bytes; %s", + (this->status == NULL ? "(ignoring error) " : ""), + fname, static_cast(bytes), s.ToString().c_str()); + if (this->status != NULL && this->status->ok()) *this->status = s; + } + }; + + mutex_.AssertHeld(); + + // Open the log file + std::string fname = LogFileName(dbname_, log_number); + SequentialFile* file; + Status status = env_->NewSequentialFile(fname, &file); + if (!status.ok()) { + MaybeIgnoreError(&status); + return status; + } + + // Create the log reader. + LogReporter reporter; + reporter.env = env_; + reporter.info_log = options_.info_log; + reporter.fname = fname.c_str(); + reporter.status = (options_.paranoid_checks ? &status : NULL); + // We intentionally make log::Reader do checksumming even if + // paranoid_checks==false so that corruptions cause entire commits + // to be skipped instead of propagating bad information (like overly + // large sequence numbers). + log::Reader reader(file, &reporter, true/*checksum*/, + 0/*initial_offset*/); + Log(options_.info_log, "Recovering log #%llu", + (unsigned long long) log_number); + + // Read all the records and add to a memtable + std::string scratch; + Slice record; + WriteBatch batch; + MemTable* mem = NULL; + while (reader.ReadRecord(&record, &scratch) && + status.ok()) { + if (record.size() < 12) { + reporter.Corruption( + record.size(), Status::Corruption("log record too small")); + continue; + } + WriteBatchInternal::SetContents(&batch, record); + + if (mem == NULL) { + mem = new MemTable(internal_comparator_); + mem->Ref(); + } + status = WriteBatchInternal::InsertInto(&batch, mem); + MaybeIgnoreError(&status); + if (!status.ok()) { + break; + } + const SequenceNumber last_seq = + WriteBatchInternal::Sequence(&batch) + + WriteBatchInternal::Count(&batch) - 1; + if (last_seq > *max_sequence) { + *max_sequence = last_seq; + } + + if (mem->ApproximateMemoryUsage() > options_.write_buffer_size) { + status = WriteLevel0Table(mem, edit, NULL); + if (!status.ok()) { + // Reflect errors immediately so that conditions like full + // file-systems cause the DB::Open() to fail. + break; + } + mem->Unref(); + mem = NULL; + } + } + + if (status.ok() && mem != NULL) { + status = WriteLevel0Table(mem, edit, NULL); + // Reflect errors immediately so that conditions like full + // file-systems cause the DB::Open() to fail. + } + + if (mem != NULL) mem->Unref(); + delete file; + return status; +} + +Status DBImpl::WriteLevel0Table(MemTable* mem, VersionEdit* edit, + Version* base) { + mutex_.AssertHeld(); + const uint64_t start_micros = env_->NowMicros(); + FileMetaData meta; + meta.number = versions_->NewFileNumber(); + pending_outputs_.insert(meta.number); + Iterator* iter = mem->NewIterator(); + Log(options_.info_log, "Level-0 table #%llu: started", + (unsigned long long) meta.number); + + Status s; + { + mutex_.Unlock(); + s = BuildTable(dbname_, env_, options_, table_cache_, iter, &meta); + mutex_.Lock(); + } + + Log(options_.info_log, "Level-0 table #%llu: %lld bytes %s", + (unsigned long long) meta.number, + (unsigned long long) meta.file_size, + s.ToString().c_str()); + delete iter; + pending_outputs_.erase(meta.number); + + + // Note that if file_size is zero, the file has been deleted and + // should not be added to the manifest. + int level = 0; + if (s.ok() && meta.file_size > 0) { + const Slice min_user_key = meta.smallest.user_key(); + const Slice max_user_key = meta.largest.user_key(); + if (base != NULL) { + level = base->PickLevelForMemTableOutput(min_user_key, max_user_key); + } + edit->AddFile(level, meta.number, meta.file_size, + meta.smallest, meta.largest); + } + + CompactionStats stats; + stats.micros = env_->NowMicros() - start_micros; + stats.bytes_written = meta.file_size; + stats_[level].Add(stats); + return s; +} + +void DBImpl::CompactMemTable() { + mutex_.AssertHeld(); + assert(imm_ != NULL); + + // Save the contents of the memtable as a new Table + VersionEdit edit; + Version* base = versions_->current(); + base->Ref(); + Status s = WriteLevel0Table(imm_, &edit, base); + base->Unref(); + + if (s.ok() && shutting_down_.Acquire_Load()) { + s = Status::IOError("Deleting DB during memtable compaction"); + } + + // Replace immutable memtable with the generated Table + if (s.ok()) { + edit.SetPrevLogNumber(0); + edit.SetLogNumber(logfile_number_); // Earlier logs no longer needed + s = versions_->LogAndApply(&edit, &mutex_); + } + + if (s.ok()) { + // Commit to the new state + imm_->Unref(); + imm_ = NULL; + has_imm_.Release_Store(NULL); + DeleteObsoleteFiles(); + } else { + RecordBackgroundError(s); + } +} + +void DBImpl::CompactRange(const Slice* begin, const Slice* end) { + int max_level_with_files = 1; + { + MutexLock l(&mutex_); + Version* base = versions_->current(); + for (int level = 1; level < config::kNumLevels; level++) { + if (base->OverlapInLevel(level, begin, end)) { + max_level_with_files = level; + } + } + } + TEST_CompactMemTable(); // TODO(sanjay): Skip if memtable does not overlap + for (int level = 0; level < max_level_with_files; level++) { + TEST_CompactRange(level, begin, end); + } +} + +void DBImpl::TEST_CompactRange(int level, const Slice* begin,const Slice* end) { + assert(level >= 0); + assert(level + 1 < config::kNumLevels); + + InternalKey begin_storage, end_storage; + + ManualCompaction manual; + manual.level = level; + manual.done = false; + if (begin == NULL) { + manual.begin = NULL; + } else { + begin_storage = InternalKey(*begin, kMaxSequenceNumber, kValueTypeForSeek); + manual.begin = &begin_storage; + } + if (end == NULL) { + manual.end = NULL; + } else { + end_storage = InternalKey(*end, 0, static_cast(0)); + manual.end = &end_storage; + } + + MutexLock l(&mutex_); + while (!manual.done && !shutting_down_.Acquire_Load() && bg_error_.ok()) { + if (manual_compaction_ == NULL) { // Idle + manual_compaction_ = &manual; + MaybeScheduleCompaction(); + } else { // Running either my compaction or another compaction. + bg_cv_.Wait(); + } + } + if (manual_compaction_ == &manual) { + // Cancel my manual compaction since we aborted early for some reason. + manual_compaction_ = NULL; + } +} + +Status DBImpl::TEST_CompactMemTable() { + // NULL batch means just wait for earlier writes to be done + Status s = Write(WriteOptions(), NULL); + if (s.ok()) { + // Wait until the compaction completes + MutexLock l(&mutex_); + while (imm_ != NULL && bg_error_.ok()) { + bg_cv_.Wait(); + } + if (imm_ != NULL) { + s = bg_error_; + } + } + return s; +} + +void DBImpl::RecordBackgroundError(const Status& s) { + mutex_.AssertHeld(); + if (bg_error_.ok()) { + bg_error_ = s; + bg_cv_.SignalAll(); + } +} + +void DBImpl::MaybeScheduleCompaction() { + mutex_.AssertHeld(); + if (bg_compaction_scheduled_) { + // Already scheduled + } else if (shutting_down_.Acquire_Load()) { + // DB is being deleted; no more background compactions + } else if (!bg_error_.ok()) { + // Already got an error; no more changes + } else if (imm_ == NULL && + manual_compaction_ == NULL && + !versions_->NeedsCompaction()) { + // No work to be done + } else { + bg_compaction_scheduled_ = true; + env_->Schedule(&DBImpl::BGWork, this); + } +} + +void DBImpl::BGWork(void* db) { + reinterpret_cast(db)->BackgroundCall(); +} + +void DBImpl::BackgroundCall() { + MutexLock l(&mutex_); + assert(bg_compaction_scheduled_); + if (shutting_down_.Acquire_Load()) { + // No more background work when shutting down. + } else if (!bg_error_.ok()) { + // No more background work after a background error. + } else { + BackgroundCompaction(); + } + + bg_compaction_scheduled_ = false; + + // Previous compaction may have produced too many files in a level, + // so reschedule another compaction if needed. + MaybeScheduleCompaction(); + bg_cv_.SignalAll(); +} + +void DBImpl::BackgroundCompaction() { + mutex_.AssertHeld(); + + if (imm_ != NULL) { + CompactMemTable(); + return; + } + + Compaction* c; + bool is_manual = (manual_compaction_ != NULL); + InternalKey manual_end; + if (is_manual) { + ManualCompaction* m = manual_compaction_; + c = versions_->CompactRange(m->level, m->begin, m->end); + m->done = (c == NULL); + if (c != NULL) { + manual_end = c->input(0, c->num_input_files(0) - 1)->largest; + } + Log(options_.info_log, + "Manual compaction at level-%d from %s .. %s; will stop at %s\n", + m->level, + (m->begin ? m->begin->DebugString().c_str() : "(begin)"), + (m->end ? m->end->DebugString().c_str() : "(end)"), + (m->done ? "(end)" : manual_end.DebugString().c_str())); + } else { + c = versions_->PickCompaction(); + } + + Status status; + if (c == NULL) { + // Nothing to do + } else if (!is_manual && c->IsTrivialMove()) { + // Move file to next level + assert(c->num_input_files(0) == 1); + FileMetaData* f = c->input(0, 0); + c->edit()->DeleteFile(c->level(), f->number); + c->edit()->AddFile(c->level() + 1, f->number, f->file_size, + f->smallest, f->largest); + status = versions_->LogAndApply(c->edit(), &mutex_); + if (!status.ok()) { + RecordBackgroundError(status); + } + VersionSet::LevelSummaryStorage tmp; + Log(options_.info_log, "Moved #%lld to level-%d %lld bytes %s: %s\n", + static_cast(f->number), + c->level() + 1, + static_cast(f->file_size), + status.ToString().c_str(), + versions_->LevelSummary(&tmp)); + } else { + CompactionState* compact = new CompactionState(c); + status = DoCompactionWork(compact); + if (!status.ok()) { + RecordBackgroundError(status); + } + CleanupCompaction(compact); + c->ReleaseInputs(); + DeleteObsoleteFiles(); + } + delete c; + + if (status.ok()) { + // Done + } else if (shutting_down_.Acquire_Load()) { + // Ignore compaction errors found during shutting down + } else { + Log(options_.info_log, + "Compaction error: %s", status.ToString().c_str()); + } + + if (is_manual) { + ManualCompaction* m = manual_compaction_; + if (!status.ok()) { + m->done = true; + } + if (!m->done) { + // We only compacted part of the requested range. Update *m + // to the range that is left to be compacted. + m->tmp_storage = manual_end; + m->begin = &m->tmp_storage; + } + manual_compaction_ = NULL; + } +} + +void DBImpl::CleanupCompaction(CompactionState* compact) { + mutex_.AssertHeld(); + if (compact->builder != NULL) { + // May happen if we get a shutdown call in the middle of compaction + compact->builder->Abandon(); + delete compact->builder; + } else { + assert(compact->outfile == NULL); + } + delete compact->outfile; + for (size_t i = 0; i < compact->outputs.size(); i++) { + const CompactionState::Output& out = compact->outputs[i]; + pending_outputs_.erase(out.number); + } + delete compact; +} + +Status DBImpl::OpenCompactionOutputFile(CompactionState* compact) { + assert(compact != NULL); + assert(compact->builder == NULL); + uint64_t file_number; + { + mutex_.Lock(); + file_number = versions_->NewFileNumber(); + pending_outputs_.insert(file_number); + CompactionState::Output out; + out.number = file_number; + out.smallest.Clear(); + out.largest.Clear(); + compact->outputs.push_back(out); + mutex_.Unlock(); + } + + // Make the output file + std::string fname = TableFileName(dbname_, file_number); + Status s = env_->NewWritableFile(fname, &compact->outfile); + if (s.ok()) { + compact->builder = new TableBuilder(options_, compact->outfile); + } + return s; +} + +Status DBImpl::FinishCompactionOutputFile(CompactionState* compact, + Iterator* input) { + assert(compact != NULL); + assert(compact->outfile != NULL); + assert(compact->builder != NULL); + + const uint64_t output_number = compact->current_output()->number; + assert(output_number != 0); + + // Check for iterator errors + Status s = input->status(); + const uint64_t current_entries = compact->builder->NumEntries(); + if (s.ok()) { + s = compact->builder->Finish(); + } else { + compact->builder->Abandon(); + } + const uint64_t current_bytes = compact->builder->FileSize(); + compact->current_output()->file_size = current_bytes; + compact->total_bytes += current_bytes; + delete compact->builder; + compact->builder = NULL; + + // Finish and check for file errors + if (s.ok()) { + s = compact->outfile->Sync(); + } + if (s.ok()) { + s = compact->outfile->Close(); + } + delete compact->outfile; + compact->outfile = NULL; + + if (s.ok() && current_entries > 0) { + // Verify that the table is usable + Iterator* iter = table_cache_->NewIterator(ReadOptions(), + output_number, + current_bytes); + s = iter->status(); + delete iter; + if (s.ok()) { + Log(options_.info_log, + "Generated table #%llu: %lld keys, %lld bytes", + (unsigned long long) output_number, + (unsigned long long) current_entries, + (unsigned long long) current_bytes); + } + } + return s; +} + + +Status DBImpl::InstallCompactionResults(CompactionState* compact) { + mutex_.AssertHeld(); + Log(options_.info_log, "Compacted %d@%d + %d@%d files => %lld bytes", + compact->compaction->num_input_files(0), + compact->compaction->level(), + compact->compaction->num_input_files(1), + compact->compaction->level() + 1, + static_cast(compact->total_bytes)); + + // Add compaction outputs + compact->compaction->AddInputDeletions(compact->compaction->edit()); + const int level = compact->compaction->level(); + for (size_t i = 0; i < compact->outputs.size(); i++) { + const CompactionState::Output& out = compact->outputs[i]; + compact->compaction->edit()->AddFile( + level + 1, + out.number, out.file_size, out.smallest, out.largest); + } + return versions_->LogAndApply(compact->compaction->edit(), &mutex_); +} + +Status DBImpl::DoCompactionWork(CompactionState* compact) { + const uint64_t start_micros = env_->NowMicros(); + int64_t imm_micros = 0; // Micros spent doing imm_ compactions + + Log(options_.info_log, "Compacting %d@%d + %d@%d files", + compact->compaction->num_input_files(0), + compact->compaction->level(), + compact->compaction->num_input_files(1), + compact->compaction->level() + 1); + + assert(versions_->NumLevelFiles(compact->compaction->level()) > 0); + assert(compact->builder == NULL); + assert(compact->outfile == NULL); + if (snapshots_.empty()) { + compact->smallest_snapshot = versions_->LastSequence(); + } else { + compact->smallest_snapshot = snapshots_.oldest()->number_; + } + + // Release mutex while we're actually doing the compaction work + mutex_.Unlock(); + + Iterator* input = versions_->MakeInputIterator(compact->compaction); + input->SeekToFirst(); + Status status; + ParsedInternalKey ikey; + std::string current_user_key; + bool has_current_user_key = false; + SequenceNumber last_sequence_for_key = kMaxSequenceNumber; + for (; input->Valid() && !shutting_down_.Acquire_Load(); ) { + // Prioritize immutable compaction work + if (has_imm_.NoBarrier_Load() != NULL) { + const uint64_t imm_start = env_->NowMicros(); + mutex_.Lock(); + if (imm_ != NULL) { + CompactMemTable(); + bg_cv_.SignalAll(); // Wakeup MakeRoomForWrite() if necessary + } + mutex_.Unlock(); + imm_micros += (env_->NowMicros() - imm_start); + } + + Slice key = input->key(); + if (compact->compaction->ShouldStopBefore(key) && + compact->builder != NULL) { + status = FinishCompactionOutputFile(compact, input); + if (!status.ok()) { + break; + } + } + + // Handle key/value, add to state, etc. + bool drop = false; + if (!ParseInternalKey(key, &ikey)) { + // Do not hide error keys + current_user_key.clear(); + has_current_user_key = false; + last_sequence_for_key = kMaxSequenceNumber; + } else { + if (!has_current_user_key || + user_comparator()->Compare(ikey.user_key, + Slice(current_user_key)) != 0) { + // First occurrence of this user key + current_user_key.assign(ikey.user_key.data(), ikey.user_key.size()); + has_current_user_key = true; + last_sequence_for_key = kMaxSequenceNumber; + } + + if (last_sequence_for_key <= compact->smallest_snapshot) { + // Hidden by an newer entry for same user key + drop = true; // (A) + } else if (ikey.type == kTypeDeletion && + ikey.sequence <= compact->smallest_snapshot && + compact->compaction->IsBaseLevelForKey(ikey.user_key)) { + // For this user key: + // (1) there is no data in higher levels + // (2) data in lower levels will have larger sequence numbers + // (3) data in layers that are being compacted here and have + // smaller sequence numbers will be dropped in the next + // few iterations of this loop (by rule (A) above). + // Therefore this deletion marker is obsolete and can be dropped. + drop = true; + } + + last_sequence_for_key = ikey.sequence; + } +#if 0 + Log(options_.info_log, + " Compact: %s, seq %d, type: %d %d, drop: %d, is_base: %d, " + "%d smallest_snapshot: %d", + ikey.user_key.ToString().c_str(), + (int)ikey.sequence, ikey.type, kTypeValue, drop, + compact->compaction->IsBaseLevelForKey(ikey.user_key), + (int)last_sequence_for_key, (int)compact->smallest_snapshot); +#endif + + if (!drop) { + // Open output file if necessary + if (compact->builder == NULL) { + status = OpenCompactionOutputFile(compact); + if (!status.ok()) { + break; + } + } + if (compact->builder->NumEntries() == 0) { + compact->current_output()->smallest.DecodeFrom(key); + } + compact->current_output()->largest.DecodeFrom(key); + compact->builder->Add(key, input->value()); + + // Close output file if it is big enough + if (compact->builder->FileSize() >= + compact->compaction->MaxOutputFileSize()) { + status = FinishCompactionOutputFile(compact, input); + if (!status.ok()) { + break; + } + } + } + + input->Next(); + } + + if (status.ok() && shutting_down_.Acquire_Load()) { + status = Status::IOError("Deleting DB during compaction"); + } + if (status.ok() && compact->builder != NULL) { + status = FinishCompactionOutputFile(compact, input); + } + if (status.ok()) { + status = input->status(); + } + delete input; + input = NULL; + + CompactionStats stats; + stats.micros = env_->NowMicros() - start_micros - imm_micros; + for (int which = 0; which < 2; which++) { + for (int i = 0; i < compact->compaction->num_input_files(which); i++) { + stats.bytes_read += compact->compaction->input(which, i)->file_size; + } + } + for (size_t i = 0; i < compact->outputs.size(); i++) { + stats.bytes_written += compact->outputs[i].file_size; + } + + mutex_.Lock(); + stats_[compact->compaction->level() + 1].Add(stats); + + if (status.ok()) { + status = InstallCompactionResults(compact); + } + if (!status.ok()) { + RecordBackgroundError(status); + } + VersionSet::LevelSummaryStorage tmp; + Log(options_.info_log, + "compacted to: %s", versions_->LevelSummary(&tmp)); + return status; +} + +namespace { +struct IterState { + port::Mutex* mu; + Version* version; + MemTable* mem; + MemTable* imm; +}; + +static void CleanupIteratorState(void* arg1, void* arg2) { + IterState* state = reinterpret_cast(arg1); + state->mu->Lock(); + state->mem->Unref(); + if (state->imm != NULL) state->imm->Unref(); + state->version->Unref(); + state->mu->Unlock(); + delete state; +} +} // namespace + +Iterator* DBImpl::NewInternalIterator(const ReadOptions& options, + SequenceNumber* latest_snapshot, + uint32_t* seed) { + IterState* cleanup = new IterState; + mutex_.Lock(); + *latest_snapshot = versions_->LastSequence(); + + // Collect together all needed child iterators + std::vector list; + list.push_back(mem_->NewIterator()); + mem_->Ref(); + if (imm_ != NULL) { + list.push_back(imm_->NewIterator()); + imm_->Ref(); + } + versions_->current()->AddIterators(options, &list); + Iterator* internal_iter = + NewMergingIterator(&internal_comparator_, &list[0], list.size()); + versions_->current()->Ref(); + + cleanup->mu = &mutex_; + cleanup->mem = mem_; + cleanup->imm = imm_; + cleanup->version = versions_->current(); + internal_iter->RegisterCleanup(CleanupIteratorState, cleanup, NULL); + + *seed = ++seed_; + mutex_.Unlock(); + return internal_iter; +} + +Iterator* DBImpl::TEST_NewInternalIterator() { + SequenceNumber ignored; + uint32_t ignored_seed; + return NewInternalIterator(ReadOptions(), &ignored, &ignored_seed); +} + +int64_t DBImpl::TEST_MaxNextLevelOverlappingBytes() { + MutexLock l(&mutex_); + return versions_->MaxNextLevelOverlappingBytes(); +} + +Status DBImpl::Get(const ReadOptions& options, + const Slice& key, + std::string* value) { + Status s; + MutexLock l(&mutex_); + SequenceNumber snapshot; + if (options.snapshot != NULL) { + snapshot = reinterpret_cast(options.snapshot)->number_; + } else { + snapshot = versions_->LastSequence(); + } + + MemTable* mem = mem_; + MemTable* imm = imm_; + Version* current = versions_->current(); + mem->Ref(); + if (imm != NULL) imm->Ref(); + current->Ref(); + + bool have_stat_update = false; + Version::GetStats stats; + + // Unlock while reading from files and memtables + { + mutex_.Unlock(); + // First look in the memtable, then in the immutable memtable (if any). + LookupKey lkey(key, snapshot); + if (mem->Get(lkey, value, &s)) { + // Done + } else if (imm != NULL && imm->Get(lkey, value, &s)) { + // Done + } else { + s = current->Get(options, lkey, value, &stats); + have_stat_update = true; + } + mutex_.Lock(); + } + + if (have_stat_update && current->UpdateStats(stats)) { + MaybeScheduleCompaction(); + } + mem->Unref(); + if (imm != NULL) imm->Unref(); + current->Unref(); + return s; +} + +Iterator* DBImpl::NewIterator(const ReadOptions& options) { + SequenceNumber latest_snapshot; + uint32_t seed; + Iterator* iter = NewInternalIterator(options, &latest_snapshot, &seed); + return NewDBIterator( + this, user_comparator(), iter, + (options.snapshot != NULL + ? reinterpret_cast(options.snapshot)->number_ + : latest_snapshot), + seed); +} + +void DBImpl::RecordReadSample(Slice key) { + MutexLock l(&mutex_); + if (versions_->current()->RecordReadSample(key)) { + MaybeScheduleCompaction(); + } +} + +const Snapshot* DBImpl::GetSnapshot() { + MutexLock l(&mutex_); + return snapshots_.New(versions_->LastSequence()); +} + +void DBImpl::ReleaseSnapshot(const Snapshot* s) { + MutexLock l(&mutex_); + snapshots_.Delete(reinterpret_cast(s)); +} + +// Convenience methods +Status DBImpl::Put(const WriteOptions& o, const Slice& key, const Slice& val) { + return DB::Put(o, key, val); +} + +Status DBImpl::Delete(const WriteOptions& options, const Slice& key) { + return DB::Delete(options, key); +} + +Status DBImpl::Write(const WriteOptions& options, WriteBatch* my_batch) { + Writer w(&mutex_); + w.batch = my_batch; + w.sync = options.sync; + w.done = false; + + MutexLock l(&mutex_); + writers_.push_back(&w); + while (!w.done && &w != writers_.front()) { + w.cv.Wait(); + } + if (w.done) { + return w.status; + } + + // May temporarily unlock and wait. + Status status = MakeRoomForWrite(my_batch == NULL); + uint64_t last_sequence = versions_->LastSequence(); + Writer* last_writer = &w; + if (status.ok() && my_batch != NULL) { // NULL batch is for compactions + WriteBatch* updates = BuildBatchGroup(&last_writer); + WriteBatchInternal::SetSequence(updates, last_sequence + 1); + last_sequence += WriteBatchInternal::Count(updates); + + // Add to log and apply to memtable. We can release the lock + // during this phase since &w is currently responsible for logging + // and protects against concurrent loggers and concurrent writes + // into mem_. + { + mutex_.Unlock(); + status = log_->AddRecord(WriteBatchInternal::Contents(updates)); + bool sync_error = false; + if (status.ok() && options.sync) { + status = logfile_->Sync(); + if (!status.ok()) { + sync_error = true; + } + } + if (status.ok()) { + status = WriteBatchInternal::InsertInto(updates, mem_); + } + mutex_.Lock(); + if (sync_error) { + // The state of the log file is indeterminate: the log record we + // just added may or may not show up when the DB is re-opened. + // So we force the DB into a mode where all future writes fail. + RecordBackgroundError(status); + } + } + if (updates == tmp_batch_) tmp_batch_->Clear(); + + versions_->SetLastSequence(last_sequence); + } + + while (true) { + Writer* ready = writers_.front(); + writers_.pop_front(); + if (ready != &w) { + ready->status = status; + ready->done = true; + ready->cv.Signal(); + } + if (ready == last_writer) break; + } + + // Notify new head of write queue + if (!writers_.empty()) { + writers_.front()->cv.Signal(); + } + + return status; +} + +// REQUIRES: Writer list must be non-empty +// REQUIRES: First writer must have a non-NULL batch +WriteBatch* DBImpl::BuildBatchGroup(Writer** last_writer) { + assert(!writers_.empty()); + Writer* first = writers_.front(); + WriteBatch* result = first->batch; + assert(result != NULL); + + size_t size = WriteBatchInternal::ByteSize(first->batch); + + // Allow the group to grow up to a maximum size, but if the + // original write is small, limit the growth so we do not slow + // down the small write too much. + size_t max_size = 1 << 20; + if (size <= (128<<10)) { + max_size = size + (128<<10); + } + + *last_writer = first; + std::deque::iterator iter = writers_.begin(); + ++iter; // Advance past "first" + for (; iter != writers_.end(); ++iter) { + Writer* w = *iter; + if (w->sync && !first->sync) { + // Do not include a sync write into a batch handled by a non-sync write. + break; + } + + if (w->batch != NULL) { + size += WriteBatchInternal::ByteSize(w->batch); + if (size > max_size) { + // Do not make batch too big + break; + } + + // Append to *result + if (result == first->batch) { + // Switch to temporary batch instead of disturbing caller's batch + result = tmp_batch_; + assert(WriteBatchInternal::Count(result) == 0); + WriteBatchInternal::Append(result, first->batch); + } + WriteBatchInternal::Append(result, w->batch); + } + *last_writer = w; + } + return result; +} + +// REQUIRES: mutex_ is held +// REQUIRES: this thread is currently at the front of the writer queue +Status DBImpl::MakeRoomForWrite(bool force) { + mutex_.AssertHeld(); + assert(!writers_.empty()); + bool allow_delay = !force; + Status s; + while (true) { + if (!bg_error_.ok()) { + // Yield previous error + s = bg_error_; + break; + } else if ( + allow_delay && + versions_->NumLevelFiles(0) >= config::kL0_SlowdownWritesTrigger) { + // We are getting close to hitting a hard limit on the number of + // L0 files. Rather than delaying a single write by several + // seconds when we hit the hard limit, start delaying each + // individual write by 1ms to reduce latency variance. Also, + // this delay hands over some CPU to the compaction thread in + // case it is sharing the same core as the writer. + mutex_.Unlock(); + env_->SleepForMicroseconds(1000); + allow_delay = false; // Do not delay a single write more than once + mutex_.Lock(); + } else if (!force && + (mem_->ApproximateMemoryUsage() <= options_.write_buffer_size)) { + // There is room in current memtable + break; + } else if (imm_ != NULL) { + // We have filled up the current memtable, but the previous + // one is still being compacted, so we wait. + Log(options_.info_log, "Current memtable full; waiting...\n"); + bg_cv_.Wait(); + } else if (versions_->NumLevelFiles(0) >= config::kL0_StopWritesTrigger) { + // There are too many level-0 files. + Log(options_.info_log, "Too many L0 files; waiting...\n"); + bg_cv_.Wait(); + } else { + // Attempt to switch to a new memtable and trigger compaction of old + assert(versions_->PrevLogNumber() == 0); + uint64_t new_log_number = versions_->NewFileNumber(); + WritableFile* lfile = NULL; + s = env_->NewWritableFile(LogFileName(dbname_, new_log_number), &lfile); + if (!s.ok()) { + // Avoid chewing through file number space in a tight loop. + versions_->ReuseFileNumber(new_log_number); + break; + } + delete log_; + delete logfile_; + logfile_ = lfile; + logfile_number_ = new_log_number; + log_ = new log::Writer(lfile); + imm_ = mem_; + has_imm_.Release_Store(imm_); + mem_ = new MemTable(internal_comparator_); + mem_->Ref(); + force = false; // Do not force another compaction if have room + MaybeScheduleCompaction(); + } + } + return s; +} + +bool DBImpl::GetProperty(const Slice& property, std::string* value) { + value->clear(); + + MutexLock l(&mutex_); + Slice in = property; + Slice prefix("leveldb."); + if (!in.starts_with(prefix)) return false; + in.remove_prefix(prefix.size()); + + if (in.starts_with("num-files-at-level")) { + in.remove_prefix(strlen("num-files-at-level")); + uint64_t level; + bool ok = ConsumeDecimalNumber(&in, &level) && in.empty(); + if (!ok || level >= config::kNumLevels) { + return false; + } else { + char buf[100]; + snprintf(buf, sizeof(buf), "%d", + versions_->NumLevelFiles(static_cast(level))); + *value = buf; + return true; + } + } else if (in == "stats") { + char buf[200]; + snprintf(buf, sizeof(buf), + " Compactions\n" + "Level Files Size(MB) Time(sec) Read(MB) Write(MB)\n" + "--------------------------------------------------\n" + ); + value->append(buf); + for (int level = 0; level < config::kNumLevels; level++) { + int files = versions_->NumLevelFiles(level); + if (stats_[level].micros > 0 || files > 0) { + snprintf( + buf, sizeof(buf), + "%3d %8d %8.0f %9.0f %8.0f %9.0f\n", + level, + files, + versions_->NumLevelBytes(level) / 1048576.0, + stats_[level].micros / 1e6, + stats_[level].bytes_read / 1048576.0, + stats_[level].bytes_written / 1048576.0); + value->append(buf); + } + } + return true; + } else if (in == "sstables") { + *value = versions_->current()->DebugString(); + return true; + } + + return false; +} + +void DBImpl::GetApproximateSizes( + const Range* range, int n, + uint64_t* sizes) { + // TODO(opt): better implementation + Version* v; + { + MutexLock l(&mutex_); + versions_->current()->Ref(); + v = versions_->current(); + } + + for (int i = 0; i < n; i++) { + // Convert user_key into a corresponding internal key. + InternalKey k1(range[i].start, kMaxSequenceNumber, kValueTypeForSeek); + InternalKey k2(range[i].limit, kMaxSequenceNumber, kValueTypeForSeek); + uint64_t start = versions_->ApproximateOffsetOf(v, k1); + uint64_t limit = versions_->ApproximateOffsetOf(v, k2); + sizes[i] = (limit >= start ? limit - start : 0); + } + + { + MutexLock l(&mutex_); + v->Unref(); + } +} + +// Default implementations of convenience methods that subclasses of DB +// can call if they wish +Status DB::Put(const WriteOptions& opt, const Slice& key, const Slice& value) { + WriteBatch batch; + batch.Put(key, value); + return Write(opt, &batch); +} + +Status DB::Delete(const WriteOptions& opt, const Slice& key) { + WriteBatch batch; + batch.Delete(key); + return Write(opt, &batch); +} + +DB::~DB() { } + +Status DB::Open(const Options& options, const std::string& dbname, + DB** dbptr) { + *dbptr = NULL; + + DBImpl* impl = new DBImpl(options, dbname); + impl->mutex_.Lock(); + VersionEdit edit; + Status s = impl->Recover(&edit); // Handles create_if_missing, error_if_exists + if (s.ok()) { + uint64_t new_log_number = impl->versions_->NewFileNumber(); + WritableFile* lfile; + s = options.env->NewWritableFile(LogFileName(dbname, new_log_number), + &lfile); + if (s.ok()) { + edit.SetLogNumber(new_log_number); + impl->logfile_ = lfile; + impl->logfile_number_ = new_log_number; + impl->log_ = new log::Writer(lfile); + s = impl->versions_->LogAndApply(&edit, &impl->mutex_); + } + if (s.ok()) { + impl->DeleteObsoleteFiles(); + impl->MaybeScheduleCompaction(); + } + } + impl->mutex_.Unlock(); + if (s.ok()) { + *dbptr = impl; + } else { + delete impl; + } + return s; +} + +Snapshot::~Snapshot() { +} + +Status DestroyDB(const std::string& dbname, const Options& options) { + Env* env = options.env; + std::vector filenames; + // Ignore error in case directory does not exist + env->GetChildren(dbname, &filenames); + if (filenames.empty()) { + return Status::OK(); + } + + FileLock* lock; + const std::string lockname = LockFileName(dbname); + Status result = env->LockFile(lockname, &lock); + if (result.ok()) { + uint64_t number; + FileType type; + for (size_t i = 0; i < filenames.size(); i++) { + if (ParseFileName(filenames[i], &number, &type) && + type != kDBLockFile) { // Lock file will be deleted at end + Status del = env->DeleteFile(dbname + "/" + filenames[i]); + if (result.ok() && !del.ok()) { + result = del; + } + } + } + env->UnlockFile(lock); // Ignore error since state is already gone + env->DeleteFile(lockname); + env->DeleteDir(dbname); // Ignore error in case dir contains other files + } + return result; +} + +} // namespace leveldb diff --git a/src/leveldb/db/db_impl.h b/src/leveldb/db/db_impl.h new file mode 100644 index 00000000..cfc99816 --- /dev/null +++ b/src/leveldb/db/db_impl.h @@ -0,0 +1,211 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_DB_DB_IMPL_H_ +#define STORAGE_LEVELDB_DB_DB_IMPL_H_ + +#include +#include +#include "db/dbformat.h" +#include "db/log_writer.h" +#include "db/snapshot.h" +#include "leveldb/db.h" +#include "leveldb/env.h" +#include "port/port.h" +#include "port/thread_annotations.h" + +namespace leveldb { + +class MemTable; +class TableCache; +class Version; +class VersionEdit; +class VersionSet; + +class DBImpl : public DB { + public: + DBImpl(const Options& options, const std::string& dbname); + virtual ~DBImpl(); + + // Implementations of the DB interface + virtual Status Put(const WriteOptions&, const Slice& key, const Slice& value); + virtual Status Delete(const WriteOptions&, const Slice& key); + virtual Status Write(const WriteOptions& options, WriteBatch* updates); + virtual Status Get(const ReadOptions& options, + const Slice& key, + std::string* value); + virtual Iterator* NewIterator(const ReadOptions&); + virtual const Snapshot* GetSnapshot(); + virtual void ReleaseSnapshot(const Snapshot* snapshot); + virtual bool GetProperty(const Slice& property, std::string* value); + virtual void GetApproximateSizes(const Range* range, int n, uint64_t* sizes); + virtual void CompactRange(const Slice* begin, const Slice* end); + + // Extra methods (for testing) that are not in the public DB interface + + // Compact any files in the named level that overlap [*begin,*end] + void TEST_CompactRange(int level, const Slice* begin, const Slice* end); + + // Force current memtable contents to be compacted. + Status TEST_CompactMemTable(); + + // Return an internal iterator over the current state of the database. + // The keys of this iterator are internal keys (see format.h). + // The returned iterator should be deleted when no longer needed. + Iterator* TEST_NewInternalIterator(); + + // Return the maximum overlapping data (in bytes) at next level for any + // file at a level >= 1. + int64_t TEST_MaxNextLevelOverlappingBytes(); + + // Record a sample of bytes read at the specified internal key. + // Samples are taken approximately once every config::kReadBytesPeriod + // bytes. + void RecordReadSample(Slice key); + + private: + friend class DB; + struct CompactionState; + struct Writer; + + Iterator* NewInternalIterator(const ReadOptions&, + SequenceNumber* latest_snapshot, + uint32_t* seed); + + Status NewDB(); + + // Recover the descriptor from persistent storage. May do a significant + // amount of work to recover recently logged updates. Any changes to + // be made to the descriptor are added to *edit. + Status Recover(VersionEdit* edit) EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + void MaybeIgnoreError(Status* s) const; + + // Delete any unneeded files and stale in-memory entries. + void DeleteObsoleteFiles(); + + // Compact the in-memory write buffer to disk. Switches to a new + // log-file/memtable and writes a new descriptor iff successful. + // Errors are recorded in bg_error_. + void CompactMemTable() EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + Status RecoverLogFile(uint64_t log_number, + VersionEdit* edit, + SequenceNumber* max_sequence) + EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + Status WriteLevel0Table(MemTable* mem, VersionEdit* edit, Version* base) + EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + Status MakeRoomForWrite(bool force /* compact even if there is room? */) + EXCLUSIVE_LOCKS_REQUIRED(mutex_); + WriteBatch* BuildBatchGroup(Writer** last_writer); + + void RecordBackgroundError(const Status& s); + + void MaybeScheduleCompaction() EXCLUSIVE_LOCKS_REQUIRED(mutex_); + static void BGWork(void* db); + void BackgroundCall(); + void BackgroundCompaction() EXCLUSIVE_LOCKS_REQUIRED(mutex_); + void CleanupCompaction(CompactionState* compact) + EXCLUSIVE_LOCKS_REQUIRED(mutex_); + Status DoCompactionWork(CompactionState* compact) + EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + Status OpenCompactionOutputFile(CompactionState* compact); + Status FinishCompactionOutputFile(CompactionState* compact, Iterator* input); + Status InstallCompactionResults(CompactionState* compact) + EXCLUSIVE_LOCKS_REQUIRED(mutex_); + + // Constant after construction + Env* const env_; + const InternalKeyComparator internal_comparator_; + const InternalFilterPolicy internal_filter_policy_; + const Options options_; // options_.comparator == &internal_comparator_ + bool owns_info_log_; + bool owns_cache_; + const std::string dbname_; + + // table_cache_ provides its own synchronization + TableCache* table_cache_; + + // Lock over the persistent DB state. Non-NULL iff successfully acquired. + FileLock* db_lock_; + + // State below is protected by mutex_ + port::Mutex mutex_; + port::AtomicPointer shutting_down_; + port::CondVar bg_cv_; // Signalled when background work finishes + MemTable* mem_; + MemTable* imm_; // Memtable being compacted + port::AtomicPointer has_imm_; // So bg thread can detect non-NULL imm_ + WritableFile* logfile_; + uint64_t logfile_number_; + log::Writer* log_; + uint32_t seed_; // For sampling. + + // Queue of writers. + std::deque writers_; + WriteBatch* tmp_batch_; + + SnapshotList snapshots_; + + // Set of table files to protect from deletion because they are + // part of ongoing compactions. + std::set pending_outputs_; + + // Has a background compaction been scheduled or is running? + bool bg_compaction_scheduled_; + + // Information for a manual compaction + struct ManualCompaction { + int level; + bool done; + const InternalKey* begin; // NULL means beginning of key range + const InternalKey* end; // NULL means end of key range + InternalKey tmp_storage; // Used to keep track of compaction progress + }; + ManualCompaction* manual_compaction_; + + VersionSet* versions_; + + // Have we encountered a background error in paranoid mode? + Status bg_error_; + + // Per level compaction stats. stats_[level] stores the stats for + // compactions that produced data for the specified "level". + struct CompactionStats { + int64_t micros; + int64_t bytes_read; + int64_t bytes_written; + + CompactionStats() : micros(0), bytes_read(0), bytes_written(0) { } + + void Add(const CompactionStats& c) { + this->micros += c.micros; + this->bytes_read += c.bytes_read; + this->bytes_written += c.bytes_written; + } + }; + CompactionStats stats_[config::kNumLevels]; + + // No copying allowed + DBImpl(const DBImpl&); + void operator=(const DBImpl&); + + const Comparator* user_comparator() const { + return internal_comparator_.user_comparator(); + } +}; + +// Sanitize db options. The caller should delete result.info_log if +// it is not equal to src.info_log. +extern Options SanitizeOptions(const std::string& db, + const InternalKeyComparator* icmp, + const InternalFilterPolicy* ipolicy, + const Options& src); + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_DB_IMPL_H_ diff --git a/src/leveldb/db/db_iter.cc b/src/leveldb/db/db_iter.cc new file mode 100644 index 00000000..3b2035e9 --- /dev/null +++ b/src/leveldb/db/db_iter.cc @@ -0,0 +1,317 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_iter.h" + +#include "db/filename.h" +#include "db/db_impl.h" +#include "db/dbformat.h" +#include "leveldb/env.h" +#include "leveldb/iterator.h" +#include "port/port.h" +#include "util/logging.h" +#include "util/mutexlock.h" +#include "util/random.h" + +namespace leveldb { + +#if 0 +static void DumpInternalIter(Iterator* iter) { + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ParsedInternalKey k; + if (!ParseInternalKey(iter->key(), &k)) { + fprintf(stderr, "Corrupt '%s'\n", EscapeString(iter->key()).c_str()); + } else { + fprintf(stderr, "@ '%s'\n", k.DebugString().c_str()); + } + } +} +#endif + +namespace { + +// Memtables and sstables that make the DB representation contain +// (userkey,seq,type) => uservalue entries. DBIter +// combines multiple entries for the same userkey found in the DB +// representation into a single entry while accounting for sequence +// numbers, deletion markers, overwrites, etc. +class DBIter: public Iterator { + public: + // Which direction is the iterator currently moving? + // (1) When moving forward, the internal iterator is positioned at + // the exact entry that yields this->key(), this->value() + // (2) When moving backwards, the internal iterator is positioned + // just before all entries whose user key == this->key(). + enum Direction { + kForward, + kReverse + }; + + DBIter(DBImpl* db, const Comparator* cmp, Iterator* iter, SequenceNumber s, + uint32_t seed) + : db_(db), + user_comparator_(cmp), + iter_(iter), + sequence_(s), + direction_(kForward), + valid_(false), + rnd_(seed), + bytes_counter_(RandomPeriod()) { + } + virtual ~DBIter() { + delete iter_; + } + virtual bool Valid() const { return valid_; } + virtual Slice key() const { + assert(valid_); + return (direction_ == kForward) ? ExtractUserKey(iter_->key()) : saved_key_; + } + virtual Slice value() const { + assert(valid_); + return (direction_ == kForward) ? iter_->value() : saved_value_; + } + virtual Status status() const { + if (status_.ok()) { + return iter_->status(); + } else { + return status_; + } + } + + virtual void Next(); + virtual void Prev(); + virtual void Seek(const Slice& target); + virtual void SeekToFirst(); + virtual void SeekToLast(); + + private: + void FindNextUserEntry(bool skipping, std::string* skip); + void FindPrevUserEntry(); + bool ParseKey(ParsedInternalKey* key); + + inline void SaveKey(const Slice& k, std::string* dst) { + dst->assign(k.data(), k.size()); + } + + inline void ClearSavedValue() { + if (saved_value_.capacity() > 1048576) { + std::string empty; + swap(empty, saved_value_); + } else { + saved_value_.clear(); + } + } + + // Pick next gap with average value of config::kReadBytesPeriod. + ssize_t RandomPeriod() { + return rnd_.Uniform(2*config::kReadBytesPeriod); + } + + DBImpl* db_; + const Comparator* const user_comparator_; + Iterator* const iter_; + SequenceNumber const sequence_; + + Status status_; + std::string saved_key_; // == current key when direction_==kReverse + std::string saved_value_; // == current raw value when direction_==kReverse + Direction direction_; + bool valid_; + + Random rnd_; + ssize_t bytes_counter_; + + // No copying allowed + DBIter(const DBIter&); + void operator=(const DBIter&); +}; + +inline bool DBIter::ParseKey(ParsedInternalKey* ikey) { + Slice k = iter_->key(); + ssize_t n = k.size() + iter_->value().size(); + bytes_counter_ -= n; + while (bytes_counter_ < 0) { + bytes_counter_ += RandomPeriod(); + db_->RecordReadSample(k); + } + if (!ParseInternalKey(k, ikey)) { + status_ = Status::Corruption("corrupted internal key in DBIter"); + return false; + } else { + return true; + } +} + +void DBIter::Next() { + assert(valid_); + + if (direction_ == kReverse) { // Switch directions? + direction_ = kForward; + // iter_ is pointing just before the entries for this->key(), + // so advance into the range of entries for this->key() and then + // use the normal skipping code below. + if (!iter_->Valid()) { + iter_->SeekToFirst(); + } else { + iter_->Next(); + } + if (!iter_->Valid()) { + valid_ = false; + saved_key_.clear(); + return; + } + // saved_key_ already contains the key to skip past. + } else { + // Store in saved_key_ the current key so we skip it below. + SaveKey(ExtractUserKey(iter_->key()), &saved_key_); + } + + FindNextUserEntry(true, &saved_key_); +} + +void DBIter::FindNextUserEntry(bool skipping, std::string* skip) { + // Loop until we hit an acceptable entry to yield + assert(iter_->Valid()); + assert(direction_ == kForward); + do { + ParsedInternalKey ikey; + if (ParseKey(&ikey) && ikey.sequence <= sequence_) { + switch (ikey.type) { + case kTypeDeletion: + // Arrange to skip all upcoming entries for this key since + // they are hidden by this deletion. + SaveKey(ikey.user_key, skip); + skipping = true; + break; + case kTypeValue: + if (skipping && + user_comparator_->Compare(ikey.user_key, *skip) <= 0) { + // Entry hidden + } else { + valid_ = true; + saved_key_.clear(); + return; + } + break; + } + } + iter_->Next(); + } while (iter_->Valid()); + saved_key_.clear(); + valid_ = false; +} + +void DBIter::Prev() { + assert(valid_); + + if (direction_ == kForward) { // Switch directions? + // iter_ is pointing at the current entry. Scan backwards until + // the key changes so we can use the normal reverse scanning code. + assert(iter_->Valid()); // Otherwise valid_ would have been false + SaveKey(ExtractUserKey(iter_->key()), &saved_key_); + while (true) { + iter_->Prev(); + if (!iter_->Valid()) { + valid_ = false; + saved_key_.clear(); + ClearSavedValue(); + return; + } + if (user_comparator_->Compare(ExtractUserKey(iter_->key()), + saved_key_) < 0) { + break; + } + } + direction_ = kReverse; + } + + FindPrevUserEntry(); +} + +void DBIter::FindPrevUserEntry() { + assert(direction_ == kReverse); + + ValueType value_type = kTypeDeletion; + if (iter_->Valid()) { + do { + ParsedInternalKey ikey; + if (ParseKey(&ikey) && ikey.sequence <= sequence_) { + if ((value_type != kTypeDeletion) && + user_comparator_->Compare(ikey.user_key, saved_key_) < 0) { + // We encountered a non-deleted value in entries for previous keys, + break; + } + value_type = ikey.type; + if (value_type == kTypeDeletion) { + saved_key_.clear(); + ClearSavedValue(); + } else { + Slice raw_value = iter_->value(); + if (saved_value_.capacity() > raw_value.size() + 1048576) { + std::string empty; + swap(empty, saved_value_); + } + SaveKey(ExtractUserKey(iter_->key()), &saved_key_); + saved_value_.assign(raw_value.data(), raw_value.size()); + } + } + iter_->Prev(); + } while (iter_->Valid()); + } + + if (value_type == kTypeDeletion) { + // End + valid_ = false; + saved_key_.clear(); + ClearSavedValue(); + direction_ = kForward; + } else { + valid_ = true; + } +} + +void DBIter::Seek(const Slice& target) { + direction_ = kForward; + ClearSavedValue(); + saved_key_.clear(); + AppendInternalKey( + &saved_key_, ParsedInternalKey(target, sequence_, kValueTypeForSeek)); + iter_->Seek(saved_key_); + if (iter_->Valid()) { + FindNextUserEntry(false, &saved_key_ /* temporary storage */); + } else { + valid_ = false; + } +} + +void DBIter::SeekToFirst() { + direction_ = kForward; + ClearSavedValue(); + iter_->SeekToFirst(); + if (iter_->Valid()) { + FindNextUserEntry(false, &saved_key_ /* temporary storage */); + } else { + valid_ = false; + } +} + +void DBIter::SeekToLast() { + direction_ = kReverse; + ClearSavedValue(); + iter_->SeekToLast(); + FindPrevUserEntry(); +} + +} // anonymous namespace + +Iterator* NewDBIterator( + DBImpl* db, + const Comparator* user_key_comparator, + Iterator* internal_iter, + SequenceNumber sequence, + uint32_t seed) { + return new DBIter(db, user_key_comparator, internal_iter, sequence, seed); +} + +} // namespace leveldb diff --git a/src/leveldb/db/db_iter.h b/src/leveldb/db/db_iter.h new file mode 100644 index 00000000..04927e93 --- /dev/null +++ b/src/leveldb/db/db_iter.h @@ -0,0 +1,28 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_DB_DB_ITER_H_ +#define STORAGE_LEVELDB_DB_DB_ITER_H_ + +#include +#include "leveldb/db.h" +#include "db/dbformat.h" + +namespace leveldb { + +class DBImpl; + +// Return a new iterator that converts internal keys (yielded by +// "*internal_iter") that were live at the specified "sequence" number +// into appropriate user keys. +extern Iterator* NewDBIterator( + DBImpl* db, + const Comparator* user_key_comparator, + Iterator* internal_iter, + SequenceNumber sequence, + uint32_t seed); + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_DB_ITER_H_ diff --git a/src/leveldb/db/db_test.cc b/src/leveldb/db/db_test.cc new file mode 100644 index 00000000..0fed9137 --- /dev/null +++ b/src/leveldb/db/db_test.cc @@ -0,0 +1,2128 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/db.h" +#include "leveldb/filter_policy.h" +#include "db/db_impl.h" +#include "db/filename.h" +#include "db/version_set.h" +#include "db/write_batch_internal.h" +#include "leveldb/cache.h" +#include "leveldb/env.h" +#include "leveldb/table.h" +#include "util/hash.h" +#include "util/logging.h" +#include "util/mutexlock.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace leveldb { + +static std::string RandomString(Random* rnd, int len) { + std::string r; + test::RandomString(rnd, len, &r); + return r; +} + +namespace { +class AtomicCounter { + private: + port::Mutex mu_; + int count_; + public: + AtomicCounter() : count_(0) { } + void Increment() { + IncrementBy(1); + } + void IncrementBy(int count) { + MutexLock l(&mu_); + count_ += count; + } + int Read() { + MutexLock l(&mu_); + return count_; + } + void Reset() { + MutexLock l(&mu_); + count_ = 0; + } +}; + +void DelayMilliseconds(int millis) { + Env::Default()->SleepForMicroseconds(millis * 1000); +} +} + +// Special Env used to delay background operations +class SpecialEnv : public EnvWrapper { + public: + // sstable/log Sync() calls are blocked while this pointer is non-NULL. + port::AtomicPointer delay_data_sync_; + + // sstable/log Sync() calls return an error. + port::AtomicPointer data_sync_error_; + + // Simulate no-space errors while this pointer is non-NULL. + port::AtomicPointer no_space_; + + // Simulate non-writable file system while this pointer is non-NULL + port::AtomicPointer non_writable_; + + // Force sync of manifest files to fail while this pointer is non-NULL + port::AtomicPointer manifest_sync_error_; + + // Force write to manifest files to fail while this pointer is non-NULL + port::AtomicPointer manifest_write_error_; + + bool count_random_reads_; + AtomicCounter random_read_counter_; + + explicit SpecialEnv(Env* base) : EnvWrapper(base) { + delay_data_sync_.Release_Store(NULL); + data_sync_error_.Release_Store(NULL); + no_space_.Release_Store(NULL); + non_writable_.Release_Store(NULL); + count_random_reads_ = false; + manifest_sync_error_.Release_Store(NULL); + manifest_write_error_.Release_Store(NULL); + } + + Status NewWritableFile(const std::string& f, WritableFile** r) { + class DataFile : public WritableFile { + private: + SpecialEnv* env_; + WritableFile* base_; + + public: + DataFile(SpecialEnv* env, WritableFile* base) + : env_(env), + base_(base) { + } + ~DataFile() { delete base_; } + Status Append(const Slice& data) { + if (env_->no_space_.Acquire_Load() != NULL) { + // Drop writes on the floor + return Status::OK(); + } else { + return base_->Append(data); + } + } + Status Close() { return base_->Close(); } + Status Flush() { return base_->Flush(); } + Status Sync() { + if (env_->data_sync_error_.Acquire_Load() != NULL) { + return Status::IOError("simulated data sync error"); + } + while (env_->delay_data_sync_.Acquire_Load() != NULL) { + DelayMilliseconds(100); + } + return base_->Sync(); + } + }; + class ManifestFile : public WritableFile { + private: + SpecialEnv* env_; + WritableFile* base_; + public: + ManifestFile(SpecialEnv* env, WritableFile* b) : env_(env), base_(b) { } + ~ManifestFile() { delete base_; } + Status Append(const Slice& data) { + if (env_->manifest_write_error_.Acquire_Load() != NULL) { + return Status::IOError("simulated writer error"); + } else { + return base_->Append(data); + } + } + Status Close() { return base_->Close(); } + Status Flush() { return base_->Flush(); } + Status Sync() { + if (env_->manifest_sync_error_.Acquire_Load() != NULL) { + return Status::IOError("simulated sync error"); + } else { + return base_->Sync(); + } + } + }; + + if (non_writable_.Acquire_Load() != NULL) { + return Status::IOError("simulated write error"); + } + + Status s = target()->NewWritableFile(f, r); + if (s.ok()) { + if (strstr(f.c_str(), ".ldb") != NULL || + strstr(f.c_str(), ".log") != NULL) { + *r = new DataFile(this, *r); + } else if (strstr(f.c_str(), "MANIFEST") != NULL) { + *r = new ManifestFile(this, *r); + } + } + return s; + } + + Status NewRandomAccessFile(const std::string& f, RandomAccessFile** r) { + class CountingFile : public RandomAccessFile { + private: + RandomAccessFile* target_; + AtomicCounter* counter_; + public: + CountingFile(RandomAccessFile* target, AtomicCounter* counter) + : target_(target), counter_(counter) { + } + virtual ~CountingFile() { delete target_; } + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + counter_->Increment(); + return target_->Read(offset, n, result, scratch); + } + }; + + Status s = target()->NewRandomAccessFile(f, r); + if (s.ok() && count_random_reads_) { + *r = new CountingFile(*r, &random_read_counter_); + } + return s; + } +}; + +class DBTest { + private: + const FilterPolicy* filter_policy_; + + // Sequence of option configurations to try + enum OptionConfig { + kDefault, + kFilter, + kUncompressed, + kEnd + }; + int option_config_; + + public: + std::string dbname_; + SpecialEnv* env_; + DB* db_; + + Options last_options_; + + DBTest() : option_config_(kDefault), + env_(new SpecialEnv(Env::Default())) { + filter_policy_ = NewBloomFilterPolicy(10); + dbname_ = test::TmpDir() + "/db_test"; + DestroyDB(dbname_, Options()); + db_ = NULL; + Reopen(); + } + + ~DBTest() { + delete db_; + DestroyDB(dbname_, Options()); + delete env_; + delete filter_policy_; + } + + // Switch to a fresh database with the next option configuration to + // test. Return false if there are no more configurations to test. + bool ChangeOptions() { + option_config_++; + if (option_config_ >= kEnd) { + return false; + } else { + DestroyAndReopen(); + return true; + } + } + + // Return the current option configuration. + Options CurrentOptions() { + Options options; + switch (option_config_) { + case kFilter: + options.filter_policy = filter_policy_; + break; + case kUncompressed: + options.compression = kNoCompression; + break; + default: + break; + } + return options; + } + + DBImpl* dbfull() { + return reinterpret_cast(db_); + } + + void Reopen(Options* options = NULL) { + ASSERT_OK(TryReopen(options)); + } + + void Close() { + delete db_; + db_ = NULL; + } + + void DestroyAndReopen(Options* options = NULL) { + delete db_; + db_ = NULL; + DestroyDB(dbname_, Options()); + ASSERT_OK(TryReopen(options)); + } + + Status TryReopen(Options* options) { + delete db_; + db_ = NULL; + Options opts; + if (options != NULL) { + opts = *options; + } else { + opts = CurrentOptions(); + opts.create_if_missing = true; + } + last_options_ = opts; + + return DB::Open(opts, dbname_, &db_); + } + + Status Put(const std::string& k, const std::string& v) { + return db_->Put(WriteOptions(), k, v); + } + + Status Delete(const std::string& k) { + return db_->Delete(WriteOptions(), k); + } + + std::string Get(const std::string& k, const Snapshot* snapshot = NULL) { + ReadOptions options; + options.snapshot = snapshot; + std::string result; + Status s = db_->Get(options, k, &result); + if (s.IsNotFound()) { + result = "NOT_FOUND"; + } else if (!s.ok()) { + result = s.ToString(); + } + return result; + } + + // Return a string that contains all key,value pairs in order, + // formatted like "(k1->v1)(k2->v2)". + std::string Contents() { + std::vector forward; + std::string result; + Iterator* iter = db_->NewIterator(ReadOptions()); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + std::string s = IterStatus(iter); + result.push_back('('); + result.append(s); + result.push_back(')'); + forward.push_back(s); + } + + // Check reverse iteration results are the reverse of forward results + size_t matched = 0; + for (iter->SeekToLast(); iter->Valid(); iter->Prev()) { + ASSERT_LT(matched, forward.size()); + ASSERT_EQ(IterStatus(iter), forward[forward.size() - matched - 1]); + matched++; + } + ASSERT_EQ(matched, forward.size()); + + delete iter; + return result; + } + + std::string AllEntriesFor(const Slice& user_key) { + Iterator* iter = dbfull()->TEST_NewInternalIterator(); + InternalKey target(user_key, kMaxSequenceNumber, kTypeValue); + iter->Seek(target.Encode()); + std::string result; + if (!iter->status().ok()) { + result = iter->status().ToString(); + } else { + result = "[ "; + bool first = true; + while (iter->Valid()) { + ParsedInternalKey ikey; + if (!ParseInternalKey(iter->key(), &ikey)) { + result += "CORRUPTED"; + } else { + if (last_options_.comparator->Compare(ikey.user_key, user_key) != 0) { + break; + } + if (!first) { + result += ", "; + } + first = false; + switch (ikey.type) { + case kTypeValue: + result += iter->value().ToString(); + break; + case kTypeDeletion: + result += "DEL"; + break; + } + } + iter->Next(); + } + if (!first) { + result += " "; + } + result += "]"; + } + delete iter; + return result; + } + + int NumTableFilesAtLevel(int level) { + std::string property; + ASSERT_TRUE( + db_->GetProperty("leveldb.num-files-at-level" + NumberToString(level), + &property)); + return atoi(property.c_str()); + } + + int TotalTableFiles() { + int result = 0; + for (int level = 0; level < config::kNumLevels; level++) { + result += NumTableFilesAtLevel(level); + } + return result; + } + + // Return spread of files per level + std::string FilesPerLevel() { + std::string result; + int last_non_zero_offset = 0; + for (int level = 0; level < config::kNumLevels; level++) { + int f = NumTableFilesAtLevel(level); + char buf[100]; + snprintf(buf, sizeof(buf), "%s%d", (level ? "," : ""), f); + result += buf; + if (f > 0) { + last_non_zero_offset = result.size(); + } + } + result.resize(last_non_zero_offset); + return result; + } + + int CountFiles() { + std::vector files; + env_->GetChildren(dbname_, &files); + return static_cast(files.size()); + } + + uint64_t Size(const Slice& start, const Slice& limit) { + Range r(start, limit); + uint64_t size; + db_->GetApproximateSizes(&r, 1, &size); + return size; + } + + void Compact(const Slice& start, const Slice& limit) { + db_->CompactRange(&start, &limit); + } + + // Do n memtable compactions, each of which produces an sstable + // covering the range [small,large]. + void MakeTables(int n, const std::string& small, const std::string& large) { + for (int i = 0; i < n; i++) { + Put(small, "begin"); + Put(large, "end"); + dbfull()->TEST_CompactMemTable(); + } + } + + // Prevent pushing of new sstables into deeper levels by adding + // tables that cover a specified range to all levels. + void FillLevels(const std::string& smallest, const std::string& largest) { + MakeTables(config::kNumLevels, smallest, largest); + } + + void DumpFileCounts(const char* label) { + fprintf(stderr, "---\n%s:\n", label); + fprintf(stderr, "maxoverlap: %lld\n", + static_cast( + dbfull()->TEST_MaxNextLevelOverlappingBytes())); + for (int level = 0; level < config::kNumLevels; level++) { + int num = NumTableFilesAtLevel(level); + if (num > 0) { + fprintf(stderr, " level %3d : %d files\n", level, num); + } + } + } + + std::string DumpSSTableList() { + std::string property; + db_->GetProperty("leveldb.sstables", &property); + return property; + } + + std::string IterStatus(Iterator* iter) { + std::string result; + if (iter->Valid()) { + result = iter->key().ToString() + "->" + iter->value().ToString(); + } else { + result = "(invalid)"; + } + return result; + } + + bool DeleteAnSSTFile() { + std::vector filenames; + ASSERT_OK(env_->GetChildren(dbname_, &filenames)); + uint64_t number; + FileType type; + for (size_t i = 0; i < filenames.size(); i++) { + if (ParseFileName(filenames[i], &number, &type) && type == kTableFile) { + ASSERT_OK(env_->DeleteFile(TableFileName(dbname_, number))); + return true; + } + } + return false; + } + + // Returns number of files renamed. + int RenameLDBToSST() { + std::vector filenames; + ASSERT_OK(env_->GetChildren(dbname_, &filenames)); + uint64_t number; + FileType type; + int files_renamed = 0; + for (size_t i = 0; i < filenames.size(); i++) { + if (ParseFileName(filenames[i], &number, &type) && type == kTableFile) { + const std::string from = TableFileName(dbname_, number); + const std::string to = SSTTableFileName(dbname_, number); + ASSERT_OK(env_->RenameFile(from, to)); + files_renamed++; + } + } + return files_renamed; + } +}; + +TEST(DBTest, Empty) { + do { + ASSERT_TRUE(db_ != NULL); + ASSERT_EQ("NOT_FOUND", Get("foo")); + } while (ChangeOptions()); +} + +TEST(DBTest, ReadWrite) { + do { + ASSERT_OK(Put("foo", "v1")); + ASSERT_EQ("v1", Get("foo")); + ASSERT_OK(Put("bar", "v2")); + ASSERT_OK(Put("foo", "v3")); + ASSERT_EQ("v3", Get("foo")); + ASSERT_EQ("v2", Get("bar")); + } while (ChangeOptions()); +} + +TEST(DBTest, PutDeleteGet) { + do { + ASSERT_OK(db_->Put(WriteOptions(), "foo", "v1")); + ASSERT_EQ("v1", Get("foo")); + ASSERT_OK(db_->Put(WriteOptions(), "foo", "v2")); + ASSERT_EQ("v2", Get("foo")); + ASSERT_OK(db_->Delete(WriteOptions(), "foo")); + ASSERT_EQ("NOT_FOUND", Get("foo")); + } while (ChangeOptions()); +} + +TEST(DBTest, GetFromImmutableLayer) { + do { + Options options = CurrentOptions(); + options.env = env_; + options.write_buffer_size = 100000; // Small write buffer + Reopen(&options); + + ASSERT_OK(Put("foo", "v1")); + ASSERT_EQ("v1", Get("foo")); + + env_->delay_data_sync_.Release_Store(env_); // Block sync calls + Put("k1", std::string(100000, 'x')); // Fill memtable + Put("k2", std::string(100000, 'y')); // Trigger compaction + ASSERT_EQ("v1", Get("foo")); + env_->delay_data_sync_.Release_Store(NULL); // Release sync calls + } while (ChangeOptions()); +} + +TEST(DBTest, GetFromVersions) { + do { + ASSERT_OK(Put("foo", "v1")); + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ("v1", Get("foo")); + } while (ChangeOptions()); +} + +TEST(DBTest, GetSnapshot) { + do { + // Try with both a short key and a long key + for (int i = 0; i < 2; i++) { + std::string key = (i == 0) ? std::string("foo") : std::string(200, 'x'); + ASSERT_OK(Put(key, "v1")); + const Snapshot* s1 = db_->GetSnapshot(); + ASSERT_OK(Put(key, "v2")); + ASSERT_EQ("v2", Get(key)); + ASSERT_EQ("v1", Get(key, s1)); + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ("v2", Get(key)); + ASSERT_EQ("v1", Get(key, s1)); + db_->ReleaseSnapshot(s1); + } + } while (ChangeOptions()); +} + +TEST(DBTest, GetLevel0Ordering) { + do { + // Check that we process level-0 files in correct order. The code + // below generates two level-0 files where the earlier one comes + // before the later one in the level-0 file list since the earlier + // one has a smaller "smallest" key. + ASSERT_OK(Put("bar", "b")); + ASSERT_OK(Put("foo", "v1")); + dbfull()->TEST_CompactMemTable(); + ASSERT_OK(Put("foo", "v2")); + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ("v2", Get("foo")); + } while (ChangeOptions()); +} + +TEST(DBTest, GetOrderedByLevels) { + do { + ASSERT_OK(Put("foo", "v1")); + Compact("a", "z"); + ASSERT_EQ("v1", Get("foo")); + ASSERT_OK(Put("foo", "v2")); + ASSERT_EQ("v2", Get("foo")); + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ("v2", Get("foo")); + } while (ChangeOptions()); +} + +TEST(DBTest, GetPicksCorrectFile) { + do { + // Arrange to have multiple files in a non-level-0 level. + ASSERT_OK(Put("a", "va")); + Compact("a", "b"); + ASSERT_OK(Put("x", "vx")); + Compact("x", "y"); + ASSERT_OK(Put("f", "vf")); + Compact("f", "g"); + ASSERT_EQ("va", Get("a")); + ASSERT_EQ("vf", Get("f")); + ASSERT_EQ("vx", Get("x")); + } while (ChangeOptions()); +} + +TEST(DBTest, GetEncountersEmptyLevel) { + do { + // Arrange for the following to happen: + // * sstable A in level 0 + // * nothing in level 1 + // * sstable B in level 2 + // Then do enough Get() calls to arrange for an automatic compaction + // of sstable A. A bug would cause the compaction to be marked as + // occurring at level 1 (instead of the correct level 0). + + // Step 1: First place sstables in levels 0 and 2 + int compaction_count = 0; + while (NumTableFilesAtLevel(0) == 0 || + NumTableFilesAtLevel(2) == 0) { + ASSERT_LE(compaction_count, 100) << "could not fill levels 0 and 2"; + compaction_count++; + Put("a", "begin"); + Put("z", "end"); + dbfull()->TEST_CompactMemTable(); + } + + // Step 2: clear level 1 if necessary. + dbfull()->TEST_CompactRange(1, NULL, NULL); + ASSERT_EQ(NumTableFilesAtLevel(0), 1); + ASSERT_EQ(NumTableFilesAtLevel(1), 0); + ASSERT_EQ(NumTableFilesAtLevel(2), 1); + + // Step 3: read a bunch of times + for (int i = 0; i < 1000; i++) { + ASSERT_EQ("NOT_FOUND", Get("missing")); + } + + // Step 4: Wait for compaction to finish + DelayMilliseconds(1000); + + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + } while (ChangeOptions()); +} + +TEST(DBTest, IterEmpty) { + Iterator* iter = db_->NewIterator(ReadOptions()); + + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->Seek("foo"); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + delete iter; +} + +TEST(DBTest, IterSingle) { + ASSERT_OK(Put("a", "va")); + Iterator* iter = db_->NewIterator(ReadOptions()); + + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->Seek(""); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->Seek("a"); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->Seek("b"); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + delete iter; +} + +TEST(DBTest, IterMulti) { + ASSERT_OK(Put("a", "va")); + ASSERT_OK(Put("b", "vb")); + ASSERT_OK(Put("c", "vc")); + Iterator* iter = db_->NewIterator(ReadOptions()); + + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->Seek(""); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Seek("a"); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Seek("ax"); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->Seek("b"); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->Seek("z"); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + // Switch from reverse to forward + iter->SeekToLast(); + iter->Prev(); + iter->Prev(); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + + // Switch from forward to reverse + iter->SeekToFirst(); + iter->Next(); + iter->Next(); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + + // Make sure iter stays at snapshot + ASSERT_OK(Put("a", "va2")); + ASSERT_OK(Put("a2", "va3")); + ASSERT_OK(Put("b", "vb2")); + ASSERT_OK(Put("c", "vc2")); + ASSERT_OK(Delete("b")); + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + delete iter; +} + +TEST(DBTest, IterSmallAndLargeMix) { + ASSERT_OK(Put("a", "va")); + ASSERT_OK(Put("b", std::string(100000, 'b'))); + ASSERT_OK(Put("c", "vc")); + ASSERT_OK(Put("d", std::string(100000, 'd'))); + ASSERT_OK(Put("e", std::string(100000, 'e'))); + + Iterator* iter = db_->NewIterator(ReadOptions()); + + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "b->" + std::string(100000, 'b')); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "d->" + std::string(100000, 'd')); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "e->" + std::string(100000, 'e')); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "e->" + std::string(100000, 'e')); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "d->" + std::string(100000, 'd')); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "b->" + std::string(100000, 'b')); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + delete iter; +} + +TEST(DBTest, IterMultiWithDelete) { + do { + ASSERT_OK(Put("a", "va")); + ASSERT_OK(Put("b", "vb")); + ASSERT_OK(Put("c", "vc")); + ASSERT_OK(Delete("b")); + ASSERT_EQ("NOT_FOUND", Get("b")); + + Iterator* iter = db_->NewIterator(ReadOptions()); + iter->Seek("c"); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "a->va"); + delete iter; + } while (ChangeOptions()); +} + +TEST(DBTest, Recover) { + do { + ASSERT_OK(Put("foo", "v1")); + ASSERT_OK(Put("baz", "v5")); + + Reopen(); + ASSERT_EQ("v1", Get("foo")); + + ASSERT_EQ("v1", Get("foo")); + ASSERT_EQ("v5", Get("baz")); + ASSERT_OK(Put("bar", "v2")); + ASSERT_OK(Put("foo", "v3")); + + Reopen(); + ASSERT_EQ("v3", Get("foo")); + ASSERT_OK(Put("foo", "v4")); + ASSERT_EQ("v4", Get("foo")); + ASSERT_EQ("v2", Get("bar")); + ASSERT_EQ("v5", Get("baz")); + } while (ChangeOptions()); +} + +TEST(DBTest, RecoveryWithEmptyLog) { + do { + ASSERT_OK(Put("foo", "v1")); + ASSERT_OK(Put("foo", "v2")); + Reopen(); + Reopen(); + ASSERT_OK(Put("foo", "v3")); + Reopen(); + ASSERT_EQ("v3", Get("foo")); + } while (ChangeOptions()); +} + +// Check that writes done during a memtable compaction are recovered +// if the database is shutdown during the memtable compaction. +TEST(DBTest, RecoverDuringMemtableCompaction) { + do { + Options options = CurrentOptions(); + options.env = env_; + options.write_buffer_size = 1000000; + Reopen(&options); + + // Trigger a long memtable compaction and reopen the database during it + ASSERT_OK(Put("foo", "v1")); // Goes to 1st log file + ASSERT_OK(Put("big1", std::string(10000000, 'x'))); // Fills memtable + ASSERT_OK(Put("big2", std::string(1000, 'y'))); // Triggers compaction + ASSERT_OK(Put("bar", "v2")); // Goes to new log file + + Reopen(&options); + ASSERT_EQ("v1", Get("foo")); + ASSERT_EQ("v2", Get("bar")); + ASSERT_EQ(std::string(10000000, 'x'), Get("big1")); + ASSERT_EQ(std::string(1000, 'y'), Get("big2")); + } while (ChangeOptions()); +} + +static std::string Key(int i) { + char buf[100]; + snprintf(buf, sizeof(buf), "key%06d", i); + return std::string(buf); +} + +TEST(DBTest, MinorCompactionsHappen) { + Options options = CurrentOptions(); + options.write_buffer_size = 10000; + Reopen(&options); + + const int N = 500; + + int starting_num_tables = TotalTableFiles(); + for (int i = 0; i < N; i++) { + ASSERT_OK(Put(Key(i), Key(i) + std::string(1000, 'v'))); + } + int ending_num_tables = TotalTableFiles(); + ASSERT_GT(ending_num_tables, starting_num_tables); + + for (int i = 0; i < N; i++) { + ASSERT_EQ(Key(i) + std::string(1000, 'v'), Get(Key(i))); + } + + Reopen(); + + for (int i = 0; i < N; i++) { + ASSERT_EQ(Key(i) + std::string(1000, 'v'), Get(Key(i))); + } +} + +TEST(DBTest, RecoverWithLargeLog) { + { + Options options = CurrentOptions(); + Reopen(&options); + ASSERT_OK(Put("big1", std::string(200000, '1'))); + ASSERT_OK(Put("big2", std::string(200000, '2'))); + ASSERT_OK(Put("small3", std::string(10, '3'))); + ASSERT_OK(Put("small4", std::string(10, '4'))); + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + } + + // Make sure that if we re-open with a small write buffer size that + // we flush table files in the middle of a large log file. + Options options = CurrentOptions(); + options.write_buffer_size = 100000; + Reopen(&options); + ASSERT_EQ(NumTableFilesAtLevel(0), 3); + ASSERT_EQ(std::string(200000, '1'), Get("big1")); + ASSERT_EQ(std::string(200000, '2'), Get("big2")); + ASSERT_EQ(std::string(10, '3'), Get("small3")); + ASSERT_EQ(std::string(10, '4'), Get("small4")); + ASSERT_GT(NumTableFilesAtLevel(0), 1); +} + +TEST(DBTest, CompactionsGenerateMultipleFiles) { + Options options = CurrentOptions(); + options.write_buffer_size = 100000000; // Large write buffer + Reopen(&options); + + Random rnd(301); + + // Write 8MB (80 values, each 100K) + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + std::vector values; + for (int i = 0; i < 80; i++) { + values.push_back(RandomString(&rnd, 100000)); + ASSERT_OK(Put(Key(i), values[i])); + } + + // Reopening moves updates to level-0 + Reopen(&options); + dbfull()->TEST_CompactRange(0, NULL, NULL); + + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + ASSERT_GT(NumTableFilesAtLevel(1), 1); + for (int i = 0; i < 80; i++) { + ASSERT_EQ(Get(Key(i)), values[i]); + } +} + +TEST(DBTest, RepeatedWritesToSameKey) { + Options options = CurrentOptions(); + options.env = env_; + options.write_buffer_size = 100000; // Small write buffer + Reopen(&options); + + // We must have at most one file per level except for level-0, + // which may have up to kL0_StopWritesTrigger files. + const int kMaxFiles = config::kNumLevels + config::kL0_StopWritesTrigger; + + Random rnd(301); + std::string value = RandomString(&rnd, 2 * options.write_buffer_size); + for (int i = 0; i < 5 * kMaxFiles; i++) { + Put("key", value); + ASSERT_LE(TotalTableFiles(), kMaxFiles); + fprintf(stderr, "after %d: %d files\n", int(i+1), TotalTableFiles()); + } +} + +TEST(DBTest, SparseMerge) { + Options options = CurrentOptions(); + options.compression = kNoCompression; + Reopen(&options); + + FillLevels("A", "Z"); + + // Suppose there is: + // small amount of data with prefix A + // large amount of data with prefix B + // small amount of data with prefix C + // and that recent updates have made small changes to all three prefixes. + // Check that we do not do a compaction that merges all of B in one shot. + const std::string value(1000, 'x'); + Put("A", "va"); + // Write approximately 100MB of "B" values + for (int i = 0; i < 100000; i++) { + char key[100]; + snprintf(key, sizeof(key), "B%010d", i); + Put(key, value); + } + Put("C", "vc"); + dbfull()->TEST_CompactMemTable(); + dbfull()->TEST_CompactRange(0, NULL, NULL); + + // Make sparse update + Put("A", "va2"); + Put("B100", "bvalue2"); + Put("C", "vc2"); + dbfull()->TEST_CompactMemTable(); + + // Compactions should not cause us to create a situation where + // a file overlaps too much data at the next level. + ASSERT_LE(dbfull()->TEST_MaxNextLevelOverlappingBytes(), 20*1048576); + dbfull()->TEST_CompactRange(0, NULL, NULL); + ASSERT_LE(dbfull()->TEST_MaxNextLevelOverlappingBytes(), 20*1048576); + dbfull()->TEST_CompactRange(1, NULL, NULL); + ASSERT_LE(dbfull()->TEST_MaxNextLevelOverlappingBytes(), 20*1048576); +} + +static bool Between(uint64_t val, uint64_t low, uint64_t high) { + bool result = (val >= low) && (val <= high); + if (!result) { + fprintf(stderr, "Value %llu is not in range [%llu, %llu]\n", + (unsigned long long)(val), + (unsigned long long)(low), + (unsigned long long)(high)); + } + return result; +} + +TEST(DBTest, ApproximateSizes) { + do { + Options options = CurrentOptions(); + options.write_buffer_size = 100000000; // Large write buffer + options.compression = kNoCompression; + DestroyAndReopen(); + + ASSERT_TRUE(Between(Size("", "xyz"), 0, 0)); + Reopen(&options); + ASSERT_TRUE(Between(Size("", "xyz"), 0, 0)); + + // Write 8MB (80 values, each 100K) + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + const int N = 80; + static const int S1 = 100000; + static const int S2 = 105000; // Allow some expansion from metadata + Random rnd(301); + for (int i = 0; i < N; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, S1))); + } + + // 0 because GetApproximateSizes() does not account for memtable space + ASSERT_TRUE(Between(Size("", Key(50)), 0, 0)); + + // Check sizes across recovery by reopening a few times + for (int run = 0; run < 3; run++) { + Reopen(&options); + + for (int compact_start = 0; compact_start < N; compact_start += 10) { + for (int i = 0; i < N; i += 10) { + ASSERT_TRUE(Between(Size("", Key(i)), S1*i, S2*i)); + ASSERT_TRUE(Between(Size("", Key(i)+".suffix"), S1*(i+1), S2*(i+1))); + ASSERT_TRUE(Between(Size(Key(i), Key(i+10)), S1*10, S2*10)); + } + ASSERT_TRUE(Between(Size("", Key(50)), S1*50, S2*50)); + ASSERT_TRUE(Between(Size("", Key(50)+".suffix"), S1*50, S2*50)); + + std::string cstart_str = Key(compact_start); + std::string cend_str = Key(compact_start + 9); + Slice cstart = cstart_str; + Slice cend = cend_str; + dbfull()->TEST_CompactRange(0, &cstart, &cend); + } + + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + ASSERT_GT(NumTableFilesAtLevel(1), 0); + } + } while (ChangeOptions()); +} + +TEST(DBTest, ApproximateSizes_MixOfSmallAndLarge) { + do { + Options options = CurrentOptions(); + options.compression = kNoCompression; + Reopen(); + + Random rnd(301); + std::string big1 = RandomString(&rnd, 100000); + ASSERT_OK(Put(Key(0), RandomString(&rnd, 10000))); + ASSERT_OK(Put(Key(1), RandomString(&rnd, 10000))); + ASSERT_OK(Put(Key(2), big1)); + ASSERT_OK(Put(Key(3), RandomString(&rnd, 10000))); + ASSERT_OK(Put(Key(4), big1)); + ASSERT_OK(Put(Key(5), RandomString(&rnd, 10000))); + ASSERT_OK(Put(Key(6), RandomString(&rnd, 300000))); + ASSERT_OK(Put(Key(7), RandomString(&rnd, 10000))); + + // Check sizes across recovery by reopening a few times + for (int run = 0; run < 3; run++) { + Reopen(&options); + + ASSERT_TRUE(Between(Size("", Key(0)), 0, 0)); + ASSERT_TRUE(Between(Size("", Key(1)), 10000, 11000)); + ASSERT_TRUE(Between(Size("", Key(2)), 20000, 21000)); + ASSERT_TRUE(Between(Size("", Key(3)), 120000, 121000)); + ASSERT_TRUE(Between(Size("", Key(4)), 130000, 131000)); + ASSERT_TRUE(Between(Size("", Key(5)), 230000, 231000)); + ASSERT_TRUE(Between(Size("", Key(6)), 240000, 241000)); + ASSERT_TRUE(Between(Size("", Key(7)), 540000, 541000)); + ASSERT_TRUE(Between(Size("", Key(8)), 550000, 560000)); + + ASSERT_TRUE(Between(Size(Key(3), Key(5)), 110000, 111000)); + + dbfull()->TEST_CompactRange(0, NULL, NULL); + } + } while (ChangeOptions()); +} + +TEST(DBTest, IteratorPinsRef) { + Put("foo", "hello"); + + // Get iterator that will yield the current contents of the DB. + Iterator* iter = db_->NewIterator(ReadOptions()); + + // Write to force compactions + Put("foo", "newvalue1"); + for (int i = 0; i < 100; i++) { + ASSERT_OK(Put(Key(i), Key(i) + std::string(100000, 'v'))); // 100K values + } + Put("foo", "newvalue2"); + + iter->SeekToFirst(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ("foo", iter->key().ToString()); + ASSERT_EQ("hello", iter->value().ToString()); + iter->Next(); + ASSERT_TRUE(!iter->Valid()); + delete iter; +} + +TEST(DBTest, Snapshot) { + do { + Put("foo", "v1"); + const Snapshot* s1 = db_->GetSnapshot(); + Put("foo", "v2"); + const Snapshot* s2 = db_->GetSnapshot(); + Put("foo", "v3"); + const Snapshot* s3 = db_->GetSnapshot(); + + Put("foo", "v4"); + ASSERT_EQ("v1", Get("foo", s1)); + ASSERT_EQ("v2", Get("foo", s2)); + ASSERT_EQ("v3", Get("foo", s3)); + ASSERT_EQ("v4", Get("foo")); + + db_->ReleaseSnapshot(s3); + ASSERT_EQ("v1", Get("foo", s1)); + ASSERT_EQ("v2", Get("foo", s2)); + ASSERT_EQ("v4", Get("foo")); + + db_->ReleaseSnapshot(s1); + ASSERT_EQ("v2", Get("foo", s2)); + ASSERT_EQ("v4", Get("foo")); + + db_->ReleaseSnapshot(s2); + ASSERT_EQ("v4", Get("foo")); + } while (ChangeOptions()); +} + +TEST(DBTest, HiddenValuesAreRemoved) { + do { + Random rnd(301); + FillLevels("a", "z"); + + std::string big = RandomString(&rnd, 50000); + Put("foo", big); + Put("pastfoo", "v"); + const Snapshot* snapshot = db_->GetSnapshot(); + Put("foo", "tiny"); + Put("pastfoo2", "v2"); // Advance sequence number one more + + ASSERT_OK(dbfull()->TEST_CompactMemTable()); + ASSERT_GT(NumTableFilesAtLevel(0), 0); + + ASSERT_EQ(big, Get("foo", snapshot)); + ASSERT_TRUE(Between(Size("", "pastfoo"), 50000, 60000)); + db_->ReleaseSnapshot(snapshot); + ASSERT_EQ(AllEntriesFor("foo"), "[ tiny, " + big + " ]"); + Slice x("x"); + dbfull()->TEST_CompactRange(0, NULL, &x); + ASSERT_EQ(AllEntriesFor("foo"), "[ tiny ]"); + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + ASSERT_GE(NumTableFilesAtLevel(1), 1); + dbfull()->TEST_CompactRange(1, NULL, &x); + ASSERT_EQ(AllEntriesFor("foo"), "[ tiny ]"); + + ASSERT_TRUE(Between(Size("", "pastfoo"), 0, 1000)); + } while (ChangeOptions()); +} + +TEST(DBTest, DeletionMarkers1) { + Put("foo", "v1"); + ASSERT_OK(dbfull()->TEST_CompactMemTable()); + const int last = config::kMaxMemCompactLevel; + ASSERT_EQ(NumTableFilesAtLevel(last), 1); // foo => v1 is now in last level + + // Place a table at level last-1 to prevent merging with preceding mutation + Put("a", "begin"); + Put("z", "end"); + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ(NumTableFilesAtLevel(last), 1); + ASSERT_EQ(NumTableFilesAtLevel(last-1), 1); + + Delete("foo"); + Put("foo", "v2"); + ASSERT_EQ(AllEntriesFor("foo"), "[ v2, DEL, v1 ]"); + ASSERT_OK(dbfull()->TEST_CompactMemTable()); // Moves to level last-2 + ASSERT_EQ(AllEntriesFor("foo"), "[ v2, DEL, v1 ]"); + Slice z("z"); + dbfull()->TEST_CompactRange(last-2, NULL, &z); + // DEL eliminated, but v1 remains because we aren't compacting that level + // (DEL can be eliminated because v2 hides v1). + ASSERT_EQ(AllEntriesFor("foo"), "[ v2, v1 ]"); + dbfull()->TEST_CompactRange(last-1, NULL, NULL); + // Merging last-1 w/ last, so we are the base level for "foo", so + // DEL is removed. (as is v1). + ASSERT_EQ(AllEntriesFor("foo"), "[ v2 ]"); +} + +TEST(DBTest, DeletionMarkers2) { + Put("foo", "v1"); + ASSERT_OK(dbfull()->TEST_CompactMemTable()); + const int last = config::kMaxMemCompactLevel; + ASSERT_EQ(NumTableFilesAtLevel(last), 1); // foo => v1 is now in last level + + // Place a table at level last-1 to prevent merging with preceding mutation + Put("a", "begin"); + Put("z", "end"); + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ(NumTableFilesAtLevel(last), 1); + ASSERT_EQ(NumTableFilesAtLevel(last-1), 1); + + Delete("foo"); + ASSERT_EQ(AllEntriesFor("foo"), "[ DEL, v1 ]"); + ASSERT_OK(dbfull()->TEST_CompactMemTable()); // Moves to level last-2 + ASSERT_EQ(AllEntriesFor("foo"), "[ DEL, v1 ]"); + dbfull()->TEST_CompactRange(last-2, NULL, NULL); + // DEL kept: "last" file overlaps + ASSERT_EQ(AllEntriesFor("foo"), "[ DEL, v1 ]"); + dbfull()->TEST_CompactRange(last-1, NULL, NULL); + // Merging last-1 w/ last, so we are the base level for "foo", so + // DEL is removed. (as is v1). + ASSERT_EQ(AllEntriesFor("foo"), "[ ]"); +} + +TEST(DBTest, OverlapInLevel0) { + do { + ASSERT_EQ(config::kMaxMemCompactLevel, 2) << "Fix test to match config"; + + // Fill levels 1 and 2 to disable the pushing of new memtables to levels > 0. + ASSERT_OK(Put("100", "v100")); + ASSERT_OK(Put("999", "v999")); + dbfull()->TEST_CompactMemTable(); + ASSERT_OK(Delete("100")); + ASSERT_OK(Delete("999")); + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ("0,1,1", FilesPerLevel()); + + // Make files spanning the following ranges in level-0: + // files[0] 200 .. 900 + // files[1] 300 .. 500 + // Note that files are sorted by smallest key. + ASSERT_OK(Put("300", "v300")); + ASSERT_OK(Put("500", "v500")); + dbfull()->TEST_CompactMemTable(); + ASSERT_OK(Put("200", "v200")); + ASSERT_OK(Put("600", "v600")); + ASSERT_OK(Put("900", "v900")); + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ("2,1,1", FilesPerLevel()); + + // Compact away the placeholder files we created initially + dbfull()->TEST_CompactRange(1, NULL, NULL); + dbfull()->TEST_CompactRange(2, NULL, NULL); + ASSERT_EQ("2", FilesPerLevel()); + + // Do a memtable compaction. Before bug-fix, the compaction would + // not detect the overlap with level-0 files and would incorrectly place + // the deletion in a deeper level. + ASSERT_OK(Delete("600")); + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ("3", FilesPerLevel()); + ASSERT_EQ("NOT_FOUND", Get("600")); + } while (ChangeOptions()); +} + +TEST(DBTest, L0_CompactionBug_Issue44_a) { + Reopen(); + ASSERT_OK(Put("b", "v")); + Reopen(); + ASSERT_OK(Delete("b")); + ASSERT_OK(Delete("a")); + Reopen(); + ASSERT_OK(Delete("a")); + Reopen(); + ASSERT_OK(Put("a", "v")); + Reopen(); + Reopen(); + ASSERT_EQ("(a->v)", Contents()); + DelayMilliseconds(1000); // Wait for compaction to finish + ASSERT_EQ("(a->v)", Contents()); +} + +TEST(DBTest, L0_CompactionBug_Issue44_b) { + Reopen(); + Put("",""); + Reopen(); + Delete("e"); + Put("",""); + Reopen(); + Put("c", "cv"); + Reopen(); + Put("",""); + Reopen(); + Put("",""); + DelayMilliseconds(1000); // Wait for compaction to finish + Reopen(); + Put("d","dv"); + Reopen(); + Put("",""); + Reopen(); + Delete("d"); + Delete("b"); + Reopen(); + ASSERT_EQ("(->)(c->cv)", Contents()); + DelayMilliseconds(1000); // Wait for compaction to finish + ASSERT_EQ("(->)(c->cv)", Contents()); +} + +TEST(DBTest, ComparatorCheck) { + class NewComparator : public Comparator { + public: + virtual const char* Name() const { return "leveldb.NewComparator"; } + virtual int Compare(const Slice& a, const Slice& b) const { + return BytewiseComparator()->Compare(a, b); + } + virtual void FindShortestSeparator(std::string* s, const Slice& l) const { + BytewiseComparator()->FindShortestSeparator(s, l); + } + virtual void FindShortSuccessor(std::string* key) const { + BytewiseComparator()->FindShortSuccessor(key); + } + }; + NewComparator cmp; + Options new_options = CurrentOptions(); + new_options.comparator = &cmp; + Status s = TryReopen(&new_options); + ASSERT_TRUE(!s.ok()); + ASSERT_TRUE(s.ToString().find("comparator") != std::string::npos) + << s.ToString(); +} + +TEST(DBTest, CustomComparator) { + class NumberComparator : public Comparator { + public: + virtual const char* Name() const { return "test.NumberComparator"; } + virtual int Compare(const Slice& a, const Slice& b) const { + return ToNumber(a) - ToNumber(b); + } + virtual void FindShortestSeparator(std::string* s, const Slice& l) const { + ToNumber(*s); // Check format + ToNumber(l); // Check format + } + virtual void FindShortSuccessor(std::string* key) const { + ToNumber(*key); // Check format + } + private: + static int ToNumber(const Slice& x) { + // Check that there are no extra characters. + ASSERT_TRUE(x.size() >= 2 && x[0] == '[' && x[x.size()-1] == ']') + << EscapeString(x); + int val; + char ignored; + ASSERT_TRUE(sscanf(x.ToString().c_str(), "[%i]%c", &val, &ignored) == 1) + << EscapeString(x); + return val; + } + }; + NumberComparator cmp; + Options new_options = CurrentOptions(); + new_options.create_if_missing = true; + new_options.comparator = &cmp; + new_options.filter_policy = NULL; // Cannot use bloom filters + new_options.write_buffer_size = 1000; // Compact more often + DestroyAndReopen(&new_options); + ASSERT_OK(Put("[10]", "ten")); + ASSERT_OK(Put("[0x14]", "twenty")); + for (int i = 0; i < 2; i++) { + ASSERT_EQ("ten", Get("[10]")); + ASSERT_EQ("ten", Get("[0xa]")); + ASSERT_EQ("twenty", Get("[20]")); + ASSERT_EQ("twenty", Get("[0x14]")); + ASSERT_EQ("NOT_FOUND", Get("[15]")); + ASSERT_EQ("NOT_FOUND", Get("[0xf]")); + Compact("[0]", "[9999]"); + } + + for (int run = 0; run < 2; run++) { + for (int i = 0; i < 1000; i++) { + char buf[100]; + snprintf(buf, sizeof(buf), "[%d]", i*10); + ASSERT_OK(Put(buf, buf)); + } + Compact("[0]", "[1000000]"); + } +} + +TEST(DBTest, ManualCompaction) { + ASSERT_EQ(config::kMaxMemCompactLevel, 2) + << "Need to update this test to match kMaxMemCompactLevel"; + + MakeTables(3, "p", "q"); + ASSERT_EQ("1,1,1", FilesPerLevel()); + + // Compaction range falls before files + Compact("", "c"); + ASSERT_EQ("1,1,1", FilesPerLevel()); + + // Compaction range falls after files + Compact("r", "z"); + ASSERT_EQ("1,1,1", FilesPerLevel()); + + // Compaction range overlaps files + Compact("p1", "p9"); + ASSERT_EQ("0,0,1", FilesPerLevel()); + + // Populate a different range + MakeTables(3, "c", "e"); + ASSERT_EQ("1,1,2", FilesPerLevel()); + + // Compact just the new range + Compact("b", "f"); + ASSERT_EQ("0,0,2", FilesPerLevel()); + + // Compact all + MakeTables(1, "a", "z"); + ASSERT_EQ("0,1,2", FilesPerLevel()); + db_->CompactRange(NULL, NULL); + ASSERT_EQ("0,0,1", FilesPerLevel()); +} + +TEST(DBTest, DBOpen_Options) { + std::string dbname = test::TmpDir() + "/db_options_test"; + DestroyDB(dbname, Options()); + + // Does not exist, and create_if_missing == false: error + DB* db = NULL; + Options opts; + opts.create_if_missing = false; + Status s = DB::Open(opts, dbname, &db); + ASSERT_TRUE(strstr(s.ToString().c_str(), "does not exist") != NULL); + ASSERT_TRUE(db == NULL); + + // Does not exist, and create_if_missing == true: OK + opts.create_if_missing = true; + s = DB::Open(opts, dbname, &db); + ASSERT_OK(s); + ASSERT_TRUE(db != NULL); + + delete db; + db = NULL; + + // Does exist, and error_if_exists == true: error + opts.create_if_missing = false; + opts.error_if_exists = true; + s = DB::Open(opts, dbname, &db); + ASSERT_TRUE(strstr(s.ToString().c_str(), "exists") != NULL); + ASSERT_TRUE(db == NULL); + + // Does exist, and error_if_exists == false: OK + opts.create_if_missing = true; + opts.error_if_exists = false; + s = DB::Open(opts, dbname, &db); + ASSERT_OK(s); + ASSERT_TRUE(db != NULL); + + delete db; + db = NULL; +} + +TEST(DBTest, Locking) { + DB* db2 = NULL; + Status s = DB::Open(CurrentOptions(), dbname_, &db2); + ASSERT_TRUE(!s.ok()) << "Locking did not prevent re-opening db"; +} + +// Check that number of files does not grow when we are out of space +TEST(DBTest, NoSpace) { + Options options = CurrentOptions(); + options.env = env_; + Reopen(&options); + + ASSERT_OK(Put("foo", "v1")); + ASSERT_EQ("v1", Get("foo")); + Compact("a", "z"); + const int num_files = CountFiles(); + env_->no_space_.Release_Store(env_); // Force out-of-space errors + for (int i = 0; i < 10; i++) { + for (int level = 0; level < config::kNumLevels-1; level++) { + dbfull()->TEST_CompactRange(level, NULL, NULL); + } + } + env_->no_space_.Release_Store(NULL); + ASSERT_LT(CountFiles(), num_files + 3); +} + +TEST(DBTest, NonWritableFileSystem) { + Options options = CurrentOptions(); + options.write_buffer_size = 1000; + options.env = env_; + Reopen(&options); + ASSERT_OK(Put("foo", "v1")); + env_->non_writable_.Release_Store(env_); // Force errors for new files + std::string big(100000, 'x'); + int errors = 0; + for (int i = 0; i < 20; i++) { + fprintf(stderr, "iter %d; errors %d\n", i, errors); + if (!Put("foo", big).ok()) { + errors++; + DelayMilliseconds(100); + } + } + ASSERT_GT(errors, 0); + env_->non_writable_.Release_Store(NULL); +} + +TEST(DBTest, WriteSyncError) { + // Check that log sync errors cause the DB to disallow future writes. + + // (a) Cause log sync calls to fail + Options options = CurrentOptions(); + options.env = env_; + Reopen(&options); + env_->data_sync_error_.Release_Store(env_); + + // (b) Normal write should succeed + WriteOptions w; + ASSERT_OK(db_->Put(w, "k1", "v1")); + ASSERT_EQ("v1", Get("k1")); + + // (c) Do a sync write; should fail + w.sync = true; + ASSERT_TRUE(!db_->Put(w, "k2", "v2").ok()); + ASSERT_EQ("v1", Get("k1")); + ASSERT_EQ("NOT_FOUND", Get("k2")); + + // (d) make sync behave normally + env_->data_sync_error_.Release_Store(NULL); + + // (e) Do a non-sync write; should fail + w.sync = false; + ASSERT_TRUE(!db_->Put(w, "k3", "v3").ok()); + ASSERT_EQ("v1", Get("k1")); + ASSERT_EQ("NOT_FOUND", Get("k2")); + ASSERT_EQ("NOT_FOUND", Get("k3")); +} + +TEST(DBTest, ManifestWriteError) { + // Test for the following problem: + // (a) Compaction produces file F + // (b) Log record containing F is written to MANIFEST file, but Sync() fails + // (c) GC deletes F + // (d) After reopening DB, reads fail since deleted F is named in log record + + // We iterate twice. In the second iteration, everything is the + // same except the log record never makes it to the MANIFEST file. + for (int iter = 0; iter < 2; iter++) { + port::AtomicPointer* error_type = (iter == 0) + ? &env_->manifest_sync_error_ + : &env_->manifest_write_error_; + + // Insert foo=>bar mapping + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + options.error_if_exists = false; + DestroyAndReopen(&options); + ASSERT_OK(Put("foo", "bar")); + ASSERT_EQ("bar", Get("foo")); + + // Memtable compaction (will succeed) + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ("bar", Get("foo")); + const int last = config::kMaxMemCompactLevel; + ASSERT_EQ(NumTableFilesAtLevel(last), 1); // foo=>bar is now in last level + + // Merging compaction (will fail) + error_type->Release_Store(env_); + dbfull()->TEST_CompactRange(last, NULL, NULL); // Should fail + ASSERT_EQ("bar", Get("foo")); + + // Recovery: should not lose data + error_type->Release_Store(NULL); + Reopen(&options); + ASSERT_EQ("bar", Get("foo")); + } +} + +TEST(DBTest, MissingSSTFile) { + ASSERT_OK(Put("foo", "bar")); + ASSERT_EQ("bar", Get("foo")); + + // Dump the memtable to disk. + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ("bar", Get("foo")); + + Close(); + ASSERT_TRUE(DeleteAnSSTFile()); + Options options = CurrentOptions(); + options.paranoid_checks = true; + Status s = TryReopen(&options); + ASSERT_TRUE(!s.ok()); + ASSERT_TRUE(s.ToString().find("issing") != std::string::npos) + << s.ToString(); +} + +TEST(DBTest, StillReadSST) { + ASSERT_OK(Put("foo", "bar")); + ASSERT_EQ("bar", Get("foo")); + + // Dump the memtable to disk. + dbfull()->TEST_CompactMemTable(); + ASSERT_EQ("bar", Get("foo")); + Close(); + ASSERT_GT(RenameLDBToSST(), 0); + Options options = CurrentOptions(); + options.paranoid_checks = true; + Status s = TryReopen(&options); + ASSERT_TRUE(s.ok()); + ASSERT_EQ("bar", Get("foo")); +} + +TEST(DBTest, FilesDeletedAfterCompaction) { + ASSERT_OK(Put("foo", "v2")); + Compact("a", "z"); + const int num_files = CountFiles(); + for (int i = 0; i < 10; i++) { + ASSERT_OK(Put("foo", "v2")); + Compact("a", "z"); + } + ASSERT_EQ(CountFiles(), num_files); +} + +TEST(DBTest, BloomFilter) { + env_->count_random_reads_ = true; + Options options = CurrentOptions(); + options.env = env_; + options.block_cache = NewLRUCache(0); // Prevent cache hits + options.filter_policy = NewBloomFilterPolicy(10); + Reopen(&options); + + // Populate multiple layers + const int N = 10000; + for (int i = 0; i < N; i++) { + ASSERT_OK(Put(Key(i), Key(i))); + } + Compact("a", "z"); + for (int i = 0; i < N; i += 100) { + ASSERT_OK(Put(Key(i), Key(i))); + } + dbfull()->TEST_CompactMemTable(); + + // Prevent auto compactions triggered by seeks + env_->delay_data_sync_.Release_Store(env_); + + // Lookup present keys. Should rarely read from small sstable. + env_->random_read_counter_.Reset(); + for (int i = 0; i < N; i++) { + ASSERT_EQ(Key(i), Get(Key(i))); + } + int reads = env_->random_read_counter_.Read(); + fprintf(stderr, "%d present => %d reads\n", N, reads); + ASSERT_GE(reads, N); + ASSERT_LE(reads, N + 2*N/100); + + // Lookup present keys. Should rarely read from either sstable. + env_->random_read_counter_.Reset(); + for (int i = 0; i < N; i++) { + ASSERT_EQ("NOT_FOUND", Get(Key(i) + ".missing")); + } + reads = env_->random_read_counter_.Read(); + fprintf(stderr, "%d missing => %d reads\n", N, reads); + ASSERT_LE(reads, 3*N/100); + + env_->delay_data_sync_.Release_Store(NULL); + Close(); + delete options.block_cache; + delete options.filter_policy; +} + +// Multi-threaded test: +namespace { + +static const int kNumThreads = 4; +static const int kTestSeconds = 10; +static const int kNumKeys = 1000; + +struct MTState { + DBTest* test; + port::AtomicPointer stop; + port::AtomicPointer counter[kNumThreads]; + port::AtomicPointer thread_done[kNumThreads]; +}; + +struct MTThread { + MTState* state; + int id; +}; + +static void MTThreadBody(void* arg) { + MTThread* t = reinterpret_cast(arg); + int id = t->id; + DB* db = t->state->test->db_; + uintptr_t counter = 0; + fprintf(stderr, "... starting thread %d\n", id); + Random rnd(1000 + id); + std::string value; + char valbuf[1500]; + while (t->state->stop.Acquire_Load() == NULL) { + t->state->counter[id].Release_Store(reinterpret_cast(counter)); + + int key = rnd.Uniform(kNumKeys); + char keybuf[20]; + snprintf(keybuf, sizeof(keybuf), "%016d", key); + + if (rnd.OneIn(2)) { + // Write values of the form . + // We add some padding for force compactions. + snprintf(valbuf, sizeof(valbuf), "%d.%d.%-1000d", + key, id, static_cast(counter)); + ASSERT_OK(db->Put(WriteOptions(), Slice(keybuf), Slice(valbuf))); + } else { + // Read a value and verify that it matches the pattern written above. + Status s = db->Get(ReadOptions(), Slice(keybuf), &value); + if (s.IsNotFound()) { + // Key has not yet been written + } else { + // Check that the writer thread counter is >= the counter in the value + ASSERT_OK(s); + int k, w, c; + ASSERT_EQ(3, sscanf(value.c_str(), "%d.%d.%d", &k, &w, &c)) << value; + ASSERT_EQ(k, key); + ASSERT_GE(w, 0); + ASSERT_LT(w, kNumThreads); + ASSERT_LE(static_cast(c), reinterpret_cast( + t->state->counter[w].Acquire_Load())); + } + } + counter++; + } + t->state->thread_done[id].Release_Store(t); + fprintf(stderr, "... stopping thread %d after %d ops\n", id, int(counter)); +} + +} // namespace + +TEST(DBTest, MultiThreaded) { + do { + // Initialize state + MTState mt; + mt.test = this; + mt.stop.Release_Store(0); + for (int id = 0; id < kNumThreads; id++) { + mt.counter[id].Release_Store(0); + mt.thread_done[id].Release_Store(0); + } + + // Start threads + MTThread thread[kNumThreads]; + for (int id = 0; id < kNumThreads; id++) { + thread[id].state = &mt; + thread[id].id = id; + env_->StartThread(MTThreadBody, &thread[id]); + } + + // Let them run for a while + DelayMilliseconds(kTestSeconds * 1000); + + // Stop the threads and wait for them to finish + mt.stop.Release_Store(&mt); + for (int id = 0; id < kNumThreads; id++) { + while (mt.thread_done[id].Acquire_Load() == NULL) { + DelayMilliseconds(100); + } + } + } while (ChangeOptions()); +} + +namespace { +typedef std::map KVMap; +} + +class ModelDB: public DB { + public: + class ModelSnapshot : public Snapshot { + public: + KVMap map_; + }; + + explicit ModelDB(const Options& options): options_(options) { } + ~ModelDB() { } + virtual Status Put(const WriteOptions& o, const Slice& k, const Slice& v) { + return DB::Put(o, k, v); + } + virtual Status Delete(const WriteOptions& o, const Slice& key) { + return DB::Delete(o, key); + } + virtual Status Get(const ReadOptions& options, + const Slice& key, std::string* value) { + assert(false); // Not implemented + return Status::NotFound(key); + } + virtual Iterator* NewIterator(const ReadOptions& options) { + if (options.snapshot == NULL) { + KVMap* saved = new KVMap; + *saved = map_; + return new ModelIter(saved, true); + } else { + const KVMap* snapshot_state = + &(reinterpret_cast(options.snapshot)->map_); + return new ModelIter(snapshot_state, false); + } + } + virtual const Snapshot* GetSnapshot() { + ModelSnapshot* snapshot = new ModelSnapshot; + snapshot->map_ = map_; + return snapshot; + } + + virtual void ReleaseSnapshot(const Snapshot* snapshot) { + delete reinterpret_cast(snapshot); + } + virtual Status Write(const WriteOptions& options, WriteBatch* batch) { + class Handler : public WriteBatch::Handler { + public: + KVMap* map_; + virtual void Put(const Slice& key, const Slice& value) { + (*map_)[key.ToString()] = value.ToString(); + } + virtual void Delete(const Slice& key) { + map_->erase(key.ToString()); + } + }; + Handler handler; + handler.map_ = &map_; + return batch->Iterate(&handler); + } + + virtual bool GetProperty(const Slice& property, std::string* value) { + return false; + } + virtual void GetApproximateSizes(const Range* r, int n, uint64_t* sizes) { + for (int i = 0; i < n; i++) { + sizes[i] = 0; + } + } + virtual void CompactRange(const Slice* start, const Slice* end) { + } + + private: + class ModelIter: public Iterator { + public: + ModelIter(const KVMap* map, bool owned) + : map_(map), owned_(owned), iter_(map_->end()) { + } + ~ModelIter() { + if (owned_) delete map_; + } + virtual bool Valid() const { return iter_ != map_->end(); } + virtual void SeekToFirst() { iter_ = map_->begin(); } + virtual void SeekToLast() { + if (map_->empty()) { + iter_ = map_->end(); + } else { + iter_ = map_->find(map_->rbegin()->first); + } + } + virtual void Seek(const Slice& k) { + iter_ = map_->lower_bound(k.ToString()); + } + virtual void Next() { ++iter_; } + virtual void Prev() { --iter_; } + virtual Slice key() const { return iter_->first; } + virtual Slice value() const { return iter_->second; } + virtual Status status() const { return Status::OK(); } + private: + const KVMap* const map_; + const bool owned_; // Do we own map_ + KVMap::const_iterator iter_; + }; + const Options options_; + KVMap map_; +}; + +static std::string RandomKey(Random* rnd) { + int len = (rnd->OneIn(3) + ? 1 // Short sometimes to encourage collisions + : (rnd->OneIn(100) ? rnd->Skewed(10) : rnd->Uniform(10))); + return test::RandomKey(rnd, len); +} + +static bool CompareIterators(int step, + DB* model, + DB* db, + const Snapshot* model_snap, + const Snapshot* db_snap) { + ReadOptions options; + options.snapshot = model_snap; + Iterator* miter = model->NewIterator(options); + options.snapshot = db_snap; + Iterator* dbiter = db->NewIterator(options); + bool ok = true; + int count = 0; + for (miter->SeekToFirst(), dbiter->SeekToFirst(); + ok && miter->Valid() && dbiter->Valid(); + miter->Next(), dbiter->Next()) { + count++; + if (miter->key().compare(dbiter->key()) != 0) { + fprintf(stderr, "step %d: Key mismatch: '%s' vs. '%s'\n", + step, + EscapeString(miter->key()).c_str(), + EscapeString(dbiter->key()).c_str()); + ok = false; + break; + } + + if (miter->value().compare(dbiter->value()) != 0) { + fprintf(stderr, "step %d: Value mismatch for key '%s': '%s' vs. '%s'\n", + step, + EscapeString(miter->key()).c_str(), + EscapeString(miter->value()).c_str(), + EscapeString(miter->value()).c_str()); + ok = false; + } + } + + if (ok) { + if (miter->Valid() != dbiter->Valid()) { + fprintf(stderr, "step %d: Mismatch at end of iterators: %d vs. %d\n", + step, miter->Valid(), dbiter->Valid()); + ok = false; + } + } + fprintf(stderr, "%d entries compared: ok=%d\n", count, ok); + delete miter; + delete dbiter; + return ok; +} + +TEST(DBTest, Randomized) { + Random rnd(test::RandomSeed()); + do { + ModelDB model(CurrentOptions()); + const int N = 10000; + const Snapshot* model_snap = NULL; + const Snapshot* db_snap = NULL; + std::string k, v; + for (int step = 0; step < N; step++) { + if (step % 100 == 0) { + fprintf(stderr, "Step %d of %d\n", step, N); + } + // TODO(sanjay): Test Get() works + int p = rnd.Uniform(100); + if (p < 45) { // Put + k = RandomKey(&rnd); + v = RandomString(&rnd, + rnd.OneIn(20) + ? 100 + rnd.Uniform(100) + : rnd.Uniform(8)); + ASSERT_OK(model.Put(WriteOptions(), k, v)); + ASSERT_OK(db_->Put(WriteOptions(), k, v)); + + } else if (p < 90) { // Delete + k = RandomKey(&rnd); + ASSERT_OK(model.Delete(WriteOptions(), k)); + ASSERT_OK(db_->Delete(WriteOptions(), k)); + + + } else { // Multi-element batch + WriteBatch b; + const int num = rnd.Uniform(8); + for (int i = 0; i < num; i++) { + if (i == 0 || !rnd.OneIn(10)) { + k = RandomKey(&rnd); + } else { + // Periodically re-use the same key from the previous iter, so + // we have multiple entries in the write batch for the same key + } + if (rnd.OneIn(2)) { + v = RandomString(&rnd, rnd.Uniform(10)); + b.Put(k, v); + } else { + b.Delete(k); + } + } + ASSERT_OK(model.Write(WriteOptions(), &b)); + ASSERT_OK(db_->Write(WriteOptions(), &b)); + } + + if ((step % 100) == 0) { + ASSERT_TRUE(CompareIterators(step, &model, db_, NULL, NULL)); + ASSERT_TRUE(CompareIterators(step, &model, db_, model_snap, db_snap)); + // Save a snapshot from each DB this time that we'll use next + // time we compare things, to make sure the current state is + // preserved with the snapshot + if (model_snap != NULL) model.ReleaseSnapshot(model_snap); + if (db_snap != NULL) db_->ReleaseSnapshot(db_snap); + + Reopen(); + ASSERT_TRUE(CompareIterators(step, &model, db_, NULL, NULL)); + + model_snap = model.GetSnapshot(); + db_snap = db_->GetSnapshot(); + } + } + if (model_snap != NULL) model.ReleaseSnapshot(model_snap); + if (db_snap != NULL) db_->ReleaseSnapshot(db_snap); + } while (ChangeOptions()); +} + +std::string MakeKey(unsigned int num) { + char buf[30]; + snprintf(buf, sizeof(buf), "%016u", num); + return std::string(buf); +} + +void BM_LogAndApply(int iters, int num_base_files) { + std::string dbname = test::TmpDir() + "/leveldb_test_benchmark"; + DestroyDB(dbname, Options()); + + DB* db = NULL; + Options opts; + opts.create_if_missing = true; + Status s = DB::Open(opts, dbname, &db); + ASSERT_OK(s); + ASSERT_TRUE(db != NULL); + + delete db; + db = NULL; + + Env* env = Env::Default(); + + port::Mutex mu; + MutexLock l(&mu); + + InternalKeyComparator cmp(BytewiseComparator()); + Options options; + VersionSet vset(dbname, &options, NULL, &cmp); + ASSERT_OK(vset.Recover()); + VersionEdit vbase; + uint64_t fnum = 1; + for (int i = 0; i < num_base_files; i++) { + InternalKey start(MakeKey(2*fnum), 1, kTypeValue); + InternalKey limit(MakeKey(2*fnum+1), 1, kTypeDeletion); + vbase.AddFile(2, fnum++, 1 /* file size */, start, limit); + } + ASSERT_OK(vset.LogAndApply(&vbase, &mu)); + + uint64_t start_micros = env->NowMicros(); + + for (int i = 0; i < iters; i++) { + VersionEdit vedit; + vedit.DeleteFile(2, fnum); + InternalKey start(MakeKey(2*fnum), 1, kTypeValue); + InternalKey limit(MakeKey(2*fnum+1), 1, kTypeDeletion); + vedit.AddFile(2, fnum++, 1 /* file size */, start, limit); + vset.LogAndApply(&vedit, &mu); + } + uint64_t stop_micros = env->NowMicros(); + unsigned int us = stop_micros - start_micros; + char buf[16]; + snprintf(buf, sizeof(buf), "%d", num_base_files); + fprintf(stderr, + "BM_LogAndApply/%-6s %8d iters : %9u us (%7.0f us / iter)\n", + buf, iters, us, ((float)us) / iters); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + if (argc > 1 && std::string(argv[1]) == "--benchmark") { + leveldb::BM_LogAndApply(1000, 1); + leveldb::BM_LogAndApply(1000, 100); + leveldb::BM_LogAndApply(1000, 10000); + leveldb::BM_LogAndApply(100, 100000); + return 0; + } + + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/db/dbformat.cc b/src/leveldb/db/dbformat.cc new file mode 100644 index 00000000..20a7ca44 --- /dev/null +++ b/src/leveldb/db/dbformat.cc @@ -0,0 +1,140 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include "db/dbformat.h" +#include "port/port.h" +#include "util/coding.h" + +namespace leveldb { + +static uint64_t PackSequenceAndType(uint64_t seq, ValueType t) { + assert(seq <= kMaxSequenceNumber); + assert(t <= kValueTypeForSeek); + return (seq << 8) | t; +} + +void AppendInternalKey(std::string* result, const ParsedInternalKey& key) { + result->append(key.user_key.data(), key.user_key.size()); + PutFixed64(result, PackSequenceAndType(key.sequence, key.type)); +} + +std::string ParsedInternalKey::DebugString() const { + char buf[50]; + snprintf(buf, sizeof(buf), "' @ %llu : %d", + (unsigned long long) sequence, + int(type)); + std::string result = "'"; + result += EscapeString(user_key.ToString()); + result += buf; + return result; +} + +std::string InternalKey::DebugString() const { + std::string result; + ParsedInternalKey parsed; + if (ParseInternalKey(rep_, &parsed)) { + result = parsed.DebugString(); + } else { + result = "(bad)"; + result.append(EscapeString(rep_)); + } + return result; +} + +const char* InternalKeyComparator::Name() const { + return "leveldb.InternalKeyComparator"; +} + +int InternalKeyComparator::Compare(const Slice& akey, const Slice& bkey) const { + // Order by: + // increasing user key (according to user-supplied comparator) + // decreasing sequence number + // decreasing type (though sequence# should be enough to disambiguate) + int r = user_comparator_->Compare(ExtractUserKey(akey), ExtractUserKey(bkey)); + if (r == 0) { + const uint64_t anum = DecodeFixed64(akey.data() + akey.size() - 8); + const uint64_t bnum = DecodeFixed64(bkey.data() + bkey.size() - 8); + if (anum > bnum) { + r = -1; + } else if (anum < bnum) { + r = +1; + } + } + return r; +} + +void InternalKeyComparator::FindShortestSeparator( + std::string* start, + const Slice& limit) const { + // Attempt to shorten the user portion of the key + Slice user_start = ExtractUserKey(*start); + Slice user_limit = ExtractUserKey(limit); + std::string tmp(user_start.data(), user_start.size()); + user_comparator_->FindShortestSeparator(&tmp, user_limit); + if (tmp.size() < user_start.size() && + user_comparator_->Compare(user_start, tmp) < 0) { + // User key has become shorter physically, but larger logically. + // Tack on the earliest possible number to the shortened user key. + PutFixed64(&tmp, PackSequenceAndType(kMaxSequenceNumber,kValueTypeForSeek)); + assert(this->Compare(*start, tmp) < 0); + assert(this->Compare(tmp, limit) < 0); + start->swap(tmp); + } +} + +void InternalKeyComparator::FindShortSuccessor(std::string* key) const { + Slice user_key = ExtractUserKey(*key); + std::string tmp(user_key.data(), user_key.size()); + user_comparator_->FindShortSuccessor(&tmp); + if (tmp.size() < user_key.size() && + user_comparator_->Compare(user_key, tmp) < 0) { + // User key has become shorter physically, but larger logically. + // Tack on the earliest possible number to the shortened user key. + PutFixed64(&tmp, PackSequenceAndType(kMaxSequenceNumber,kValueTypeForSeek)); + assert(this->Compare(*key, tmp) < 0); + key->swap(tmp); + } +} + +const char* InternalFilterPolicy::Name() const { + return user_policy_->Name(); +} + +void InternalFilterPolicy::CreateFilter(const Slice* keys, int n, + std::string* dst) const { + // We rely on the fact that the code in table.cc does not mind us + // adjusting keys[]. + Slice* mkey = const_cast(keys); + for (int i = 0; i < n; i++) { + mkey[i] = ExtractUserKey(keys[i]); + // TODO(sanjay): Suppress dups? + } + user_policy_->CreateFilter(keys, n, dst); +} + +bool InternalFilterPolicy::KeyMayMatch(const Slice& key, const Slice& f) const { + return user_policy_->KeyMayMatch(ExtractUserKey(key), f); +} + +LookupKey::LookupKey(const Slice& user_key, SequenceNumber s) { + size_t usize = user_key.size(); + size_t needed = usize + 13; // A conservative estimate + char* dst; + if (needed <= sizeof(space_)) { + dst = space_; + } else { + dst = new char[needed]; + } + start_ = dst; + dst = EncodeVarint32(dst, usize + 8); + kstart_ = dst; + memcpy(dst, user_key.data(), usize); + dst += usize; + EncodeFixed64(dst, PackSequenceAndType(s, kValueTypeForSeek)); + dst += 8; + end_ = dst; +} + +} // namespace leveldb diff --git a/src/leveldb/db/dbformat.h b/src/leveldb/db/dbformat.h new file mode 100644 index 00000000..ea897b13 --- /dev/null +++ b/src/leveldb/db/dbformat.h @@ -0,0 +1,230 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_DB_DBFORMAT_H_ +#define STORAGE_LEVELDB_DB_DBFORMAT_H_ + +#include +#include "leveldb/comparator.h" +#include "leveldb/db.h" +#include "leveldb/filter_policy.h" +#include "leveldb/slice.h" +#include "leveldb/table_builder.h" +#include "util/coding.h" +#include "util/logging.h" + +namespace leveldb { + +// Grouping of constants. We may want to make some of these +// parameters set via options. +namespace config { +static const int kNumLevels = 7; + +// Level-0 compaction is started when we hit this many files. +static const int kL0_CompactionTrigger = 4; + +// Soft limit on number of level-0 files. We slow down writes at this point. +static const int kL0_SlowdownWritesTrigger = 8; + +// Maximum number of level-0 files. We stop writes at this point. +static const int kL0_StopWritesTrigger = 12; + +// Maximum level to which a new compacted memtable is pushed if it +// does not create overlap. We try to push to level 2 to avoid the +// relatively expensive level 0=>1 compactions and to avoid some +// expensive manifest file operations. We do not push all the way to +// the largest level since that can generate a lot of wasted disk +// space if the same key space is being repeatedly overwritten. +static const int kMaxMemCompactLevel = 2; + +// Approximate gap in bytes between samples of data read during iteration. +static const int kReadBytesPeriod = 1048576; + +} // namespace config + +class InternalKey; + +// Value types encoded as the last component of internal keys. +// DO NOT CHANGE THESE ENUM VALUES: they are embedded in the on-disk +// data structures. +enum ValueType { + kTypeDeletion = 0x0, + kTypeValue = 0x1 +}; +// kValueTypeForSeek defines the ValueType that should be passed when +// constructing a ParsedInternalKey object for seeking to a particular +// sequence number (since we sort sequence numbers in decreasing order +// and the value type is embedded as the low 8 bits in the sequence +// number in internal keys, we need to use the highest-numbered +// ValueType, not the lowest). +static const ValueType kValueTypeForSeek = kTypeValue; + +typedef uint64_t SequenceNumber; + +// We leave eight bits empty at the bottom so a type and sequence# +// can be packed together into 64-bits. +static const SequenceNumber kMaxSequenceNumber = + ((0x1ull << 56) - 1); + +struct ParsedInternalKey { + Slice user_key; + SequenceNumber sequence; + ValueType type; + + ParsedInternalKey() { } // Intentionally left uninitialized (for speed) + ParsedInternalKey(const Slice& u, const SequenceNumber& seq, ValueType t) + : user_key(u), sequence(seq), type(t) { } + std::string DebugString() const; +}; + +// Return the length of the encoding of "key". +inline size_t InternalKeyEncodingLength(const ParsedInternalKey& key) { + return key.user_key.size() + 8; +} + +// Append the serialization of "key" to *result. +extern void AppendInternalKey(std::string* result, + const ParsedInternalKey& key); + +// Attempt to parse an internal key from "internal_key". On success, +// stores the parsed data in "*result", and returns true. +// +// On error, returns false, leaves "*result" in an undefined state. +extern bool ParseInternalKey(const Slice& internal_key, + ParsedInternalKey* result); + +// Returns the user key portion of an internal key. +inline Slice ExtractUserKey(const Slice& internal_key) { + assert(internal_key.size() >= 8); + return Slice(internal_key.data(), internal_key.size() - 8); +} + +inline ValueType ExtractValueType(const Slice& internal_key) { + assert(internal_key.size() >= 8); + const size_t n = internal_key.size(); + uint64_t num = DecodeFixed64(internal_key.data() + n - 8); + unsigned char c = num & 0xff; + return static_cast(c); +} + +// A comparator for internal keys that uses a specified comparator for +// the user key portion and breaks ties by decreasing sequence number. +class InternalKeyComparator : public Comparator { + private: + const Comparator* user_comparator_; + public: + explicit InternalKeyComparator(const Comparator* c) : user_comparator_(c) { } + virtual const char* Name() const; + virtual int Compare(const Slice& a, const Slice& b) const; + virtual void FindShortestSeparator( + std::string* start, + const Slice& limit) const; + virtual void FindShortSuccessor(std::string* key) const; + + const Comparator* user_comparator() const { return user_comparator_; } + + int Compare(const InternalKey& a, const InternalKey& b) const; +}; + +// Filter policy wrapper that converts from internal keys to user keys +class InternalFilterPolicy : public FilterPolicy { + private: + const FilterPolicy* const user_policy_; + public: + explicit InternalFilterPolicy(const FilterPolicy* p) : user_policy_(p) { } + virtual const char* Name() const; + virtual void CreateFilter(const Slice* keys, int n, std::string* dst) const; + virtual bool KeyMayMatch(const Slice& key, const Slice& filter) const; +}; + +// Modules in this directory should keep internal keys wrapped inside +// the following class instead of plain strings so that we do not +// incorrectly use string comparisons instead of an InternalKeyComparator. +class InternalKey { + private: + std::string rep_; + public: + InternalKey() { } // Leave rep_ as empty to indicate it is invalid + InternalKey(const Slice& user_key, SequenceNumber s, ValueType t) { + AppendInternalKey(&rep_, ParsedInternalKey(user_key, s, t)); + } + + void DecodeFrom(const Slice& s) { rep_.assign(s.data(), s.size()); } + Slice Encode() const { + assert(!rep_.empty()); + return rep_; + } + + Slice user_key() const { return ExtractUserKey(rep_); } + + void SetFrom(const ParsedInternalKey& p) { + rep_.clear(); + AppendInternalKey(&rep_, p); + } + + void Clear() { rep_.clear(); } + + std::string DebugString() const; +}; + +inline int InternalKeyComparator::Compare( + const InternalKey& a, const InternalKey& b) const { + return Compare(a.Encode(), b.Encode()); +} + +inline bool ParseInternalKey(const Slice& internal_key, + ParsedInternalKey* result) { + const size_t n = internal_key.size(); + if (n < 8) return false; + uint64_t num = DecodeFixed64(internal_key.data() + n - 8); + unsigned char c = num & 0xff; + result->sequence = num >> 8; + result->type = static_cast(c); + result->user_key = Slice(internal_key.data(), n - 8); + return (c <= static_cast(kTypeValue)); +} + +// A helper class useful for DBImpl::Get() +class LookupKey { + public: + // Initialize *this for looking up user_key at a snapshot with + // the specified sequence number. + LookupKey(const Slice& user_key, SequenceNumber sequence); + + ~LookupKey(); + + // Return a key suitable for lookup in a MemTable. + Slice memtable_key() const { return Slice(start_, end_ - start_); } + + // Return an internal key (suitable for passing to an internal iterator) + Slice internal_key() const { return Slice(kstart_, end_ - kstart_); } + + // Return the user key + Slice user_key() const { return Slice(kstart_, end_ - kstart_ - 8); } + + private: + // We construct a char array of the form: + // klength varint32 <-- start_ + // userkey char[klength] <-- kstart_ + // tag uint64 + // <-- end_ + // The array is a suitable MemTable key. + // The suffix starting with "userkey" can be used as an InternalKey. + const char* start_; + const char* kstart_; + const char* end_; + char space_[200]; // Avoid allocation for short keys + + // No copying allowed + LookupKey(const LookupKey&); + void operator=(const LookupKey&); +}; + +inline LookupKey::~LookupKey() { + if (start_ != space_) delete[] start_; +} + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_DBFORMAT_H_ diff --git a/src/leveldb/db/dbformat_test.cc b/src/leveldb/db/dbformat_test.cc new file mode 100644 index 00000000..5d82f5d3 --- /dev/null +++ b/src/leveldb/db/dbformat_test.cc @@ -0,0 +1,112 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/dbformat.h" +#include "util/logging.h" +#include "util/testharness.h" + +namespace leveldb { + +static std::string IKey(const std::string& user_key, + uint64_t seq, + ValueType vt) { + std::string encoded; + AppendInternalKey(&encoded, ParsedInternalKey(user_key, seq, vt)); + return encoded; +} + +static std::string Shorten(const std::string& s, const std::string& l) { + std::string result = s; + InternalKeyComparator(BytewiseComparator()).FindShortestSeparator(&result, l); + return result; +} + +static std::string ShortSuccessor(const std::string& s) { + std::string result = s; + InternalKeyComparator(BytewiseComparator()).FindShortSuccessor(&result); + return result; +} + +static void TestKey(const std::string& key, + uint64_t seq, + ValueType vt) { + std::string encoded = IKey(key, seq, vt); + + Slice in(encoded); + ParsedInternalKey decoded("", 0, kTypeValue); + + ASSERT_TRUE(ParseInternalKey(in, &decoded)); + ASSERT_EQ(key, decoded.user_key.ToString()); + ASSERT_EQ(seq, decoded.sequence); + ASSERT_EQ(vt, decoded.type); + + ASSERT_TRUE(!ParseInternalKey(Slice("bar"), &decoded)); +} + +class FormatTest { }; + +TEST(FormatTest, InternalKey_EncodeDecode) { + const char* keys[] = { "", "k", "hello", "longggggggggggggggggggggg" }; + const uint64_t seq[] = { + 1, 2, 3, + (1ull << 8) - 1, 1ull << 8, (1ull << 8) + 1, + (1ull << 16) - 1, 1ull << 16, (1ull << 16) + 1, + (1ull << 32) - 1, 1ull << 32, (1ull << 32) + 1 + }; + for (int k = 0; k < sizeof(keys) / sizeof(keys[0]); k++) { + for (int s = 0; s < sizeof(seq) / sizeof(seq[0]); s++) { + TestKey(keys[k], seq[s], kTypeValue); + TestKey("hello", 1, kTypeDeletion); + } + } +} + +TEST(FormatTest, InternalKeyShortSeparator) { + // When user keys are same + ASSERT_EQ(IKey("foo", 100, kTypeValue), + Shorten(IKey("foo", 100, kTypeValue), + IKey("foo", 99, kTypeValue))); + ASSERT_EQ(IKey("foo", 100, kTypeValue), + Shorten(IKey("foo", 100, kTypeValue), + IKey("foo", 101, kTypeValue))); + ASSERT_EQ(IKey("foo", 100, kTypeValue), + Shorten(IKey("foo", 100, kTypeValue), + IKey("foo", 100, kTypeValue))); + ASSERT_EQ(IKey("foo", 100, kTypeValue), + Shorten(IKey("foo", 100, kTypeValue), + IKey("foo", 100, kTypeDeletion))); + + // When user keys are misordered + ASSERT_EQ(IKey("foo", 100, kTypeValue), + Shorten(IKey("foo", 100, kTypeValue), + IKey("bar", 99, kTypeValue))); + + // When user keys are different, but correctly ordered + ASSERT_EQ(IKey("g", kMaxSequenceNumber, kValueTypeForSeek), + Shorten(IKey("foo", 100, kTypeValue), + IKey("hello", 200, kTypeValue))); + + // When start user key is prefix of limit user key + ASSERT_EQ(IKey("foo", 100, kTypeValue), + Shorten(IKey("foo", 100, kTypeValue), + IKey("foobar", 200, kTypeValue))); + + // When limit user key is prefix of start user key + ASSERT_EQ(IKey("foobar", 100, kTypeValue), + Shorten(IKey("foobar", 100, kTypeValue), + IKey("foo", 200, kTypeValue))); +} + +TEST(FormatTest, InternalKeyShortestSuccessor) { + ASSERT_EQ(IKey("g", kMaxSequenceNumber, kValueTypeForSeek), + ShortSuccessor(IKey("foo", 100, kTypeValue))); + ASSERT_EQ(IKey("\xff\xff", 100, kTypeValue), + ShortSuccessor(IKey("\xff\xff", 100, kTypeValue))); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/db/dumpfile.cc b/src/leveldb/db/dumpfile.cc new file mode 100644 index 00000000..61c47c2f --- /dev/null +++ b/src/leveldb/db/dumpfile.cc @@ -0,0 +1,225 @@ +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include "db/dbformat.h" +#include "db/filename.h" +#include "db/log_reader.h" +#include "db/version_edit.h" +#include "db/write_batch_internal.h" +#include "leveldb/env.h" +#include "leveldb/iterator.h" +#include "leveldb/options.h" +#include "leveldb/status.h" +#include "leveldb/table.h" +#include "leveldb/write_batch.h" +#include "util/logging.h" + +namespace leveldb { + +namespace { + +bool GuessType(const std::string& fname, FileType* type) { + size_t pos = fname.rfind('/'); + std::string basename; + if (pos == std::string::npos) { + basename = fname; + } else { + basename = std::string(fname.data() + pos + 1, fname.size() - pos - 1); + } + uint64_t ignored; + return ParseFileName(basename, &ignored, type); +} + +// Notified when log reader encounters corruption. +class CorruptionReporter : public log::Reader::Reporter { + public: + WritableFile* dst_; + virtual void Corruption(size_t bytes, const Status& status) { + std::string r = "corruption: "; + AppendNumberTo(&r, bytes); + r += " bytes; "; + r += status.ToString(); + r.push_back('\n'); + dst_->Append(r); + } +}; + +// Print contents of a log file. (*func)() is called on every record. +Status PrintLogContents(Env* env, const std::string& fname, + void (*func)(uint64_t, Slice, WritableFile*), + WritableFile* dst) { + SequentialFile* file; + Status s = env->NewSequentialFile(fname, &file); + if (!s.ok()) { + return s; + } + CorruptionReporter reporter; + reporter.dst_ = dst; + log::Reader reader(file, &reporter, true, 0); + Slice record; + std::string scratch; + while (reader.ReadRecord(&record, &scratch)) { + (*func)(reader.LastRecordOffset(), record, dst); + } + delete file; + return Status::OK(); +} + +// Called on every item found in a WriteBatch. +class WriteBatchItemPrinter : public WriteBatch::Handler { + public: + WritableFile* dst_; + virtual void Put(const Slice& key, const Slice& value) { + std::string r = " put '"; + AppendEscapedStringTo(&r, key); + r += "' '"; + AppendEscapedStringTo(&r, value); + r += "'\n"; + dst_->Append(r); + } + virtual void Delete(const Slice& key) { + std::string r = " del '"; + AppendEscapedStringTo(&r, key); + r += "'\n"; + dst_->Append(r); + } +}; + + +// Called on every log record (each one of which is a WriteBatch) +// found in a kLogFile. +static void WriteBatchPrinter(uint64_t pos, Slice record, WritableFile* dst) { + std::string r = "--- offset "; + AppendNumberTo(&r, pos); + r += "; "; + if (record.size() < 12) { + r += "log record length "; + AppendNumberTo(&r, record.size()); + r += " is too small\n"; + dst->Append(r); + return; + } + WriteBatch batch; + WriteBatchInternal::SetContents(&batch, record); + r += "sequence "; + AppendNumberTo(&r, WriteBatchInternal::Sequence(&batch)); + r.push_back('\n'); + dst->Append(r); + WriteBatchItemPrinter batch_item_printer; + batch_item_printer.dst_ = dst; + Status s = batch.Iterate(&batch_item_printer); + if (!s.ok()) { + dst->Append(" error: " + s.ToString() + "\n"); + } +} + +Status DumpLog(Env* env, const std::string& fname, WritableFile* dst) { + return PrintLogContents(env, fname, WriteBatchPrinter, dst); +} + +// Called on every log record (each one of which is a WriteBatch) +// found in a kDescriptorFile. +static void VersionEditPrinter(uint64_t pos, Slice record, WritableFile* dst) { + std::string r = "--- offset "; + AppendNumberTo(&r, pos); + r += "; "; + VersionEdit edit; + Status s = edit.DecodeFrom(record); + if (!s.ok()) { + r += s.ToString(); + r.push_back('\n'); + } else { + r += edit.DebugString(); + } + dst->Append(r); +} + +Status DumpDescriptor(Env* env, const std::string& fname, WritableFile* dst) { + return PrintLogContents(env, fname, VersionEditPrinter, dst); +} + +Status DumpTable(Env* env, const std::string& fname, WritableFile* dst) { + uint64_t file_size; + RandomAccessFile* file = NULL; + Table* table = NULL; + Status s = env->GetFileSize(fname, &file_size); + if (s.ok()) { + s = env->NewRandomAccessFile(fname, &file); + } + if (s.ok()) { + // We use the default comparator, which may or may not match the + // comparator used in this database. However this should not cause + // problems since we only use Table operations that do not require + // any comparisons. In particular, we do not call Seek or Prev. + s = Table::Open(Options(), file, file_size, &table); + } + if (!s.ok()) { + delete table; + delete file; + return s; + } + + ReadOptions ro; + ro.fill_cache = false; + Iterator* iter = table->NewIterator(ro); + std::string r; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + r.clear(); + ParsedInternalKey key; + if (!ParseInternalKey(iter->key(), &key)) { + r = "badkey '"; + AppendEscapedStringTo(&r, iter->key()); + r += "' => '"; + AppendEscapedStringTo(&r, iter->value()); + r += "'\n"; + dst->Append(r); + } else { + r = "'"; + AppendEscapedStringTo(&r, key.user_key); + r += "' @ "; + AppendNumberTo(&r, key.sequence); + r += " : "; + if (key.type == kTypeDeletion) { + r += "del"; + } else if (key.type == kTypeValue) { + r += "val"; + } else { + AppendNumberTo(&r, key.type); + } + r += " => '"; + AppendEscapedStringTo(&r, iter->value()); + r += "'\n"; + dst->Append(r); + } + } + s = iter->status(); + if (!s.ok()) { + dst->Append("iterator error: " + s.ToString() + "\n"); + } + + delete iter; + delete table; + delete file; + return Status::OK(); +} + +} // namespace + +Status DumpFile(Env* env, const std::string& fname, WritableFile* dst) { + FileType ftype; + if (!GuessType(fname, &ftype)) { + return Status::InvalidArgument(fname + ": unknown file type"); + } + switch (ftype) { + case kLogFile: return DumpLog(env, fname, dst); + case kDescriptorFile: return DumpDescriptor(env, fname, dst); + case kTableFile: return DumpTable(env, fname, dst); + default: + break; + } + return Status::InvalidArgument(fname + ": not a dump-able file type"); +} + +} // namespace leveldb diff --git a/src/leveldb/db/filename.cc b/src/leveldb/db/filename.cc new file mode 100644 index 00000000..da32946d --- /dev/null +++ b/src/leveldb/db/filename.cc @@ -0,0 +1,144 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include +#include "db/filename.h" +#include "db/dbformat.h" +#include "leveldb/env.h" +#include "util/logging.h" + +namespace leveldb { + +// A utility routine: write "data" to the named file and Sync() it. +extern Status WriteStringToFileSync(Env* env, const Slice& data, + const std::string& fname); + +static std::string MakeFileName(const std::string& name, uint64_t number, + const char* suffix) { + char buf[100]; + snprintf(buf, sizeof(buf), "/%06llu.%s", + static_cast(number), + suffix); + return name + buf; +} + +std::string LogFileName(const std::string& name, uint64_t number) { + assert(number > 0); + return MakeFileName(name, number, "log"); +} + +std::string TableFileName(const std::string& name, uint64_t number) { + assert(number > 0); + return MakeFileName(name, number, "ldb"); +} + +std::string SSTTableFileName(const std::string& name, uint64_t number) { + assert(number > 0); + return MakeFileName(name, number, "sst"); +} + +std::string DescriptorFileName(const std::string& dbname, uint64_t number) { + assert(number > 0); + char buf[100]; + snprintf(buf, sizeof(buf), "/MANIFEST-%06llu", + static_cast(number)); + return dbname + buf; +} + +std::string CurrentFileName(const std::string& dbname) { + return dbname + "/CURRENT"; +} + +std::string LockFileName(const std::string& dbname) { + return dbname + "/LOCK"; +} + +std::string TempFileName(const std::string& dbname, uint64_t number) { + assert(number > 0); + return MakeFileName(dbname, number, "dbtmp"); +} + +std::string InfoLogFileName(const std::string& dbname) { + return dbname + "/LOG"; +} + +// Return the name of the old info log file for "dbname". +std::string OldInfoLogFileName(const std::string& dbname) { + return dbname + "/LOG.old"; +} + + +// Owned filenames have the form: +// dbname/CURRENT +// dbname/LOCK +// dbname/LOG +// dbname/LOG.old +// dbname/MANIFEST-[0-9]+ +// dbname/[0-9]+.(log|sst|ldb) +bool ParseFileName(const std::string& fname, + uint64_t* number, + FileType* type) { + Slice rest(fname); + if (rest == "CURRENT") { + *number = 0; + *type = kCurrentFile; + } else if (rest == "LOCK") { + *number = 0; + *type = kDBLockFile; + } else if (rest == "LOG" || rest == "LOG.old") { + *number = 0; + *type = kInfoLogFile; + } else if (rest.starts_with("MANIFEST-")) { + rest.remove_prefix(strlen("MANIFEST-")); + uint64_t num; + if (!ConsumeDecimalNumber(&rest, &num)) { + return false; + } + if (!rest.empty()) { + return false; + } + *type = kDescriptorFile; + *number = num; + } else { + // Avoid strtoull() to keep filename format independent of the + // current locale + uint64_t num; + if (!ConsumeDecimalNumber(&rest, &num)) { + return false; + } + Slice suffix = rest; + if (suffix == Slice(".log")) { + *type = kLogFile; + } else if (suffix == Slice(".sst") || suffix == Slice(".ldb")) { + *type = kTableFile; + } else if (suffix == Slice(".dbtmp")) { + *type = kTempFile; + } else { + return false; + } + *number = num; + } + return true; +} + +Status SetCurrentFile(Env* env, const std::string& dbname, + uint64_t descriptor_number) { + // Remove leading "dbname/" and add newline to manifest file name + std::string manifest = DescriptorFileName(dbname, descriptor_number); + Slice contents = manifest; + assert(contents.starts_with(dbname + "/")); + contents.remove_prefix(dbname.size() + 1); + std::string tmp = TempFileName(dbname, descriptor_number); + Status s = WriteStringToFileSync(env, contents.ToString() + "\n", tmp); + if (s.ok()) { + s = env->RenameFile(tmp, CurrentFileName(dbname)); + } + if (!s.ok()) { + env->DeleteFile(tmp); + } + return s; +} + +} // namespace leveldb diff --git a/src/leveldb/db/filename.h b/src/leveldb/db/filename.h new file mode 100644 index 00000000..87a75260 --- /dev/null +++ b/src/leveldb/db/filename.h @@ -0,0 +1,85 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// File names used by DB code + +#ifndef STORAGE_LEVELDB_DB_FILENAME_H_ +#define STORAGE_LEVELDB_DB_FILENAME_H_ + +#include +#include +#include "leveldb/slice.h" +#include "leveldb/status.h" +#include "port/port.h" + +namespace leveldb { + +class Env; + +enum FileType { + kLogFile, + kDBLockFile, + kTableFile, + kDescriptorFile, + kCurrentFile, + kTempFile, + kInfoLogFile // Either the current one, or an old one +}; + +// Return the name of the log file with the specified number +// in the db named by "dbname". The result will be prefixed with +// "dbname". +extern std::string LogFileName(const std::string& dbname, uint64_t number); + +// Return the name of the sstable with the specified number +// in the db named by "dbname". The result will be prefixed with +// "dbname". +extern std::string TableFileName(const std::string& dbname, uint64_t number); + +// Return the legacy file name for an sstable with the specified number +// in the db named by "dbname". The result will be prefixed with +// "dbname". +extern std::string SSTTableFileName(const std::string& dbname, uint64_t number); + +// Return the name of the descriptor file for the db named by +// "dbname" and the specified incarnation number. The result will be +// prefixed with "dbname". +extern std::string DescriptorFileName(const std::string& dbname, + uint64_t number); + +// Return the name of the current file. This file contains the name +// of the current manifest file. The result will be prefixed with +// "dbname". +extern std::string CurrentFileName(const std::string& dbname); + +// Return the name of the lock file for the db named by +// "dbname". The result will be prefixed with "dbname". +extern std::string LockFileName(const std::string& dbname); + +// Return the name of a temporary file owned by the db named "dbname". +// The result will be prefixed with "dbname". +extern std::string TempFileName(const std::string& dbname, uint64_t number); + +// Return the name of the info log file for "dbname". +extern std::string InfoLogFileName(const std::string& dbname); + +// Return the name of the old info log file for "dbname". +extern std::string OldInfoLogFileName(const std::string& dbname); + +// If filename is a leveldb file, store the type of the file in *type. +// The number encoded in the filename is stored in *number. If the +// filename was successfully parsed, returns true. Else return false. +extern bool ParseFileName(const std::string& filename, + uint64_t* number, + FileType* type); + +// Make the CURRENT file point to the descriptor file with the +// specified number. +extern Status SetCurrentFile(Env* env, const std::string& dbname, + uint64_t descriptor_number); + + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_FILENAME_H_ diff --git a/src/leveldb/db/filename_test.cc b/src/leveldb/db/filename_test.cc new file mode 100644 index 00000000..a32556de --- /dev/null +++ b/src/leveldb/db/filename_test.cc @@ -0,0 +1,123 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/filename.h" + +#include "db/dbformat.h" +#include "port/port.h" +#include "util/logging.h" +#include "util/testharness.h" + +namespace leveldb { + +class FileNameTest { }; + +TEST(FileNameTest, Parse) { + Slice db; + FileType type; + uint64_t number; + + // Successful parses + static struct { + const char* fname; + uint64_t number; + FileType type; + } cases[] = { + { "100.log", 100, kLogFile }, + { "0.log", 0, kLogFile }, + { "0.sst", 0, kTableFile }, + { "0.ldb", 0, kTableFile }, + { "CURRENT", 0, kCurrentFile }, + { "LOCK", 0, kDBLockFile }, + { "MANIFEST-2", 2, kDescriptorFile }, + { "MANIFEST-7", 7, kDescriptorFile }, + { "LOG", 0, kInfoLogFile }, + { "LOG.old", 0, kInfoLogFile }, + { "18446744073709551615.log", 18446744073709551615ull, kLogFile }, + }; + for (int i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) { + std::string f = cases[i].fname; + ASSERT_TRUE(ParseFileName(f, &number, &type)) << f; + ASSERT_EQ(cases[i].type, type) << f; + ASSERT_EQ(cases[i].number, number) << f; + } + + // Errors + static const char* errors[] = { + "", + "foo", + "foo-dx-100.log", + ".log", + "", + "manifest", + "CURREN", + "CURRENTX", + "MANIFES", + "MANIFEST", + "MANIFEST-", + "XMANIFEST-3", + "MANIFEST-3x", + "LOC", + "LOCKx", + "LO", + "LOGx", + "18446744073709551616.log", + "184467440737095516150.log", + "100", + "100.", + "100.lop" + }; + for (int i = 0; i < sizeof(errors) / sizeof(errors[0]); i++) { + std::string f = errors[i]; + ASSERT_TRUE(!ParseFileName(f, &number, &type)) << f; + } +} + +TEST(FileNameTest, Construction) { + uint64_t number; + FileType type; + std::string fname; + + fname = CurrentFileName("foo"); + ASSERT_EQ("foo/", std::string(fname.data(), 4)); + ASSERT_TRUE(ParseFileName(fname.c_str() + 4, &number, &type)); + ASSERT_EQ(0, number); + ASSERT_EQ(kCurrentFile, type); + + fname = LockFileName("foo"); + ASSERT_EQ("foo/", std::string(fname.data(), 4)); + ASSERT_TRUE(ParseFileName(fname.c_str() + 4, &number, &type)); + ASSERT_EQ(0, number); + ASSERT_EQ(kDBLockFile, type); + + fname = LogFileName("foo", 192); + ASSERT_EQ("foo/", std::string(fname.data(), 4)); + ASSERT_TRUE(ParseFileName(fname.c_str() + 4, &number, &type)); + ASSERT_EQ(192, number); + ASSERT_EQ(kLogFile, type); + + fname = TableFileName("bar", 200); + ASSERT_EQ("bar/", std::string(fname.data(), 4)); + ASSERT_TRUE(ParseFileName(fname.c_str() + 4, &number, &type)); + ASSERT_EQ(200, number); + ASSERT_EQ(kTableFile, type); + + fname = DescriptorFileName("bar", 100); + ASSERT_EQ("bar/", std::string(fname.data(), 4)); + ASSERT_TRUE(ParseFileName(fname.c_str() + 4, &number, &type)); + ASSERT_EQ(100, number); + ASSERT_EQ(kDescriptorFile, type); + + fname = TempFileName("tmp", 999); + ASSERT_EQ("tmp/", std::string(fname.data(), 4)); + ASSERT_TRUE(ParseFileName(fname.c_str() + 4, &number, &type)); + ASSERT_EQ(999, number); + ASSERT_EQ(kTempFile, type); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/db/leveldb_main.cc b/src/leveldb/db/leveldb_main.cc new file mode 100644 index 00000000..9f4b7dd7 --- /dev/null +++ b/src/leveldb/db/leveldb_main.cc @@ -0,0 +1,64 @@ +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include "leveldb/dumpfile.h" +#include "leveldb/env.h" +#include "leveldb/status.h" + +namespace leveldb { +namespace { + +class StdoutPrinter : public WritableFile { + public: + virtual Status Append(const Slice& data) { + fwrite(data.data(), 1, data.size(), stdout); + return Status::OK(); + } + virtual Status Close() { return Status::OK(); } + virtual Status Flush() { return Status::OK(); } + virtual Status Sync() { return Status::OK(); } +}; + +bool HandleDumpCommand(Env* env, char** files, int num) { + StdoutPrinter printer; + bool ok = true; + for (int i = 0; i < num; i++) { + Status s = DumpFile(env, files[i], &printer); + if (!s.ok()) { + fprintf(stderr, "%s\n", s.ToString().c_str()); + ok = false; + } + } + return ok; +} + +} // namespace +} // namespace leveldb + +static void Usage() { + fprintf( + stderr, + "Usage: leveldbutil command...\n" + " dump files... -- dump contents of specified files\n" + ); +} + +int main(int argc, char** argv) { + leveldb::Env* env = leveldb::Env::Default(); + bool ok = true; + if (argc < 2) { + Usage(); + ok = false; + } else { + std::string command = argv[1]; + if (command == "dump") { + ok = leveldb::HandleDumpCommand(env, argv+2, argc-2); + } else { + Usage(); + ok = false; + } + } + return (ok ? 0 : 1); +} diff --git a/src/leveldb/db/log_format.h b/src/leveldb/db/log_format.h new file mode 100644 index 00000000..a8c06efe --- /dev/null +++ b/src/leveldb/db/log_format.h @@ -0,0 +1,35 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// Log format information shared by reader and writer. +// See ../doc/log_format.txt for more detail. + +#ifndef STORAGE_LEVELDB_DB_LOG_FORMAT_H_ +#define STORAGE_LEVELDB_DB_LOG_FORMAT_H_ + +namespace leveldb { +namespace log { + +enum RecordType { + // Zero is reserved for preallocated files + kZeroType = 0, + + kFullType = 1, + + // For fragments + kFirstType = 2, + kMiddleType = 3, + kLastType = 4 +}; +static const int kMaxRecordType = kLastType; + +static const int kBlockSize = 32768; + +// Header is checksum (4 bytes), length (2 bytes), type (1 byte). +static const int kHeaderSize = 4 + 2 + 1; + +} // namespace log +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_LOG_FORMAT_H_ diff --git a/src/leveldb/db/log_reader.cc b/src/leveldb/db/log_reader.cc new file mode 100644 index 00000000..e44b66c8 --- /dev/null +++ b/src/leveldb/db/log_reader.cc @@ -0,0 +1,266 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/log_reader.h" + +#include +#include "leveldb/env.h" +#include "util/coding.h" +#include "util/crc32c.h" + +namespace leveldb { +namespace log { + +Reader::Reporter::~Reporter() { +} + +Reader::Reader(SequentialFile* file, Reporter* reporter, bool checksum, + uint64_t initial_offset) + : file_(file), + reporter_(reporter), + checksum_(checksum), + backing_store_(new char[kBlockSize]), + buffer_(), + eof_(false), + last_record_offset_(0), + end_of_buffer_offset_(0), + initial_offset_(initial_offset) { +} + +Reader::~Reader() { + delete[] backing_store_; +} + +bool Reader::SkipToInitialBlock() { + size_t offset_in_block = initial_offset_ % kBlockSize; + uint64_t block_start_location = initial_offset_ - offset_in_block; + + // Don't search a block if we'd be in the trailer + if (offset_in_block > kBlockSize - 6) { + offset_in_block = 0; + block_start_location += kBlockSize; + } + + end_of_buffer_offset_ = block_start_location; + + // Skip to start of first block that can contain the initial record + if (block_start_location > 0) { + Status skip_status = file_->Skip(block_start_location); + if (!skip_status.ok()) { + ReportDrop(block_start_location, skip_status); + return false; + } + } + + return true; +} + +bool Reader::ReadRecord(Slice* record, std::string* scratch) { + if (last_record_offset_ < initial_offset_) { + if (!SkipToInitialBlock()) { + return false; + } + } + + scratch->clear(); + record->clear(); + bool in_fragmented_record = false; + // Record offset of the logical record that we're reading + // 0 is a dummy value to make compilers happy + uint64_t prospective_record_offset = 0; + + Slice fragment; + while (true) { + uint64_t physical_record_offset = end_of_buffer_offset_ - buffer_.size(); + const unsigned int record_type = ReadPhysicalRecord(&fragment); + switch (record_type) { + case kFullType: + if (in_fragmented_record) { + // Handle bug in earlier versions of log::Writer where + // it could emit an empty kFirstType record at the tail end + // of a block followed by a kFullType or kFirstType record + // at the beginning of the next block. + if (scratch->empty()) { + in_fragmented_record = false; + } else { + ReportCorruption(scratch->size(), "partial record without end(1)"); + } + } + prospective_record_offset = physical_record_offset; + scratch->clear(); + *record = fragment; + last_record_offset_ = prospective_record_offset; + return true; + + case kFirstType: + if (in_fragmented_record) { + // Handle bug in earlier versions of log::Writer where + // it could emit an empty kFirstType record at the tail end + // of a block followed by a kFullType or kFirstType record + // at the beginning of the next block. + if (scratch->empty()) { + in_fragmented_record = false; + } else { + ReportCorruption(scratch->size(), "partial record without end(2)"); + } + } + prospective_record_offset = physical_record_offset; + scratch->assign(fragment.data(), fragment.size()); + in_fragmented_record = true; + break; + + case kMiddleType: + if (!in_fragmented_record) { + ReportCorruption(fragment.size(), + "missing start of fragmented record(1)"); + } else { + scratch->append(fragment.data(), fragment.size()); + } + break; + + case kLastType: + if (!in_fragmented_record) { + ReportCorruption(fragment.size(), + "missing start of fragmented record(2)"); + } else { + scratch->append(fragment.data(), fragment.size()); + *record = Slice(*scratch); + last_record_offset_ = prospective_record_offset; + return true; + } + break; + + case kEof: + if (in_fragmented_record) { + // This can be caused by the writer dying immediately after + // writing a physical record but before completing the next; don't + // treat it as a corruption, just ignore the entire logical record. + scratch->clear(); + } + return false; + + case kBadRecord: + if (in_fragmented_record) { + ReportCorruption(scratch->size(), "error in middle of record"); + in_fragmented_record = false; + scratch->clear(); + } + break; + + default: { + char buf[40]; + snprintf(buf, sizeof(buf), "unknown record type %u", record_type); + ReportCorruption( + (fragment.size() + (in_fragmented_record ? scratch->size() : 0)), + buf); + in_fragmented_record = false; + scratch->clear(); + break; + } + } + } + return false; +} + +uint64_t Reader::LastRecordOffset() { + return last_record_offset_; +} + +void Reader::ReportCorruption(uint64_t bytes, const char* reason) { + ReportDrop(bytes, Status::Corruption(reason)); +} + +void Reader::ReportDrop(uint64_t bytes, const Status& reason) { + if (reporter_ != NULL && + end_of_buffer_offset_ - buffer_.size() - bytes >= initial_offset_) { + reporter_->Corruption(static_cast(bytes), reason); + } +} + +unsigned int Reader::ReadPhysicalRecord(Slice* result) { + while (true) { + if (buffer_.size() < kHeaderSize) { + if (!eof_) { + // Last read was a full read, so this is a trailer to skip + buffer_.clear(); + Status status = file_->Read(kBlockSize, &buffer_, backing_store_); + end_of_buffer_offset_ += buffer_.size(); + if (!status.ok()) { + buffer_.clear(); + ReportDrop(kBlockSize, status); + eof_ = true; + return kEof; + } else if (buffer_.size() < kBlockSize) { + eof_ = true; + } + continue; + } else { + // Note that if buffer_ is non-empty, we have a truncated header at the + // end of the file, which can be caused by the writer crashing in the + // middle of writing the header. Instead of considering this an error, + // just report EOF. + buffer_.clear(); + return kEof; + } + } + + // Parse the header + const char* header = buffer_.data(); + const uint32_t a = static_cast(header[4]) & 0xff; + const uint32_t b = static_cast(header[5]) & 0xff; + const unsigned int type = header[6]; + const uint32_t length = a | (b << 8); + if (kHeaderSize + length > buffer_.size()) { + size_t drop_size = buffer_.size(); + buffer_.clear(); + if (!eof_) { + ReportCorruption(drop_size, "bad record length"); + return kBadRecord; + } + // If the end of the file has been reached without reading |length| bytes + // of payload, assume the writer died in the middle of writing the record. + // Don't report a corruption. + return kEof; + } + + if (type == kZeroType && length == 0) { + // Skip zero length record without reporting any drops since + // such records are produced by the mmap based writing code in + // env_posix.cc that preallocates file regions. + buffer_.clear(); + return kBadRecord; + } + + // Check crc + if (checksum_) { + uint32_t expected_crc = crc32c::Unmask(DecodeFixed32(header)); + uint32_t actual_crc = crc32c::Value(header + 6, 1 + length); + if (actual_crc != expected_crc) { + // Drop the rest of the buffer since "length" itself may have + // been corrupted and if we trust it, we could find some + // fragment of a real log record that just happens to look + // like a valid log record. + size_t drop_size = buffer_.size(); + buffer_.clear(); + ReportCorruption(drop_size, "checksum mismatch"); + return kBadRecord; + } + } + + buffer_.remove_prefix(kHeaderSize + length); + + // Skip physical record that started before initial_offset_ + if (end_of_buffer_offset_ - buffer_.size() - kHeaderSize - length < + initial_offset_) { + result->clear(); + return kBadRecord; + } + + *result = Slice(header + kHeaderSize, length); + return type; + } +} + +} // namespace log +} // namespace leveldb diff --git a/src/leveldb/db/log_reader.h b/src/leveldb/db/log_reader.h new file mode 100644 index 00000000..6aff7917 --- /dev/null +++ b/src/leveldb/db/log_reader.h @@ -0,0 +1,108 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_DB_LOG_READER_H_ +#define STORAGE_LEVELDB_DB_LOG_READER_H_ + +#include + +#include "db/log_format.h" +#include "leveldb/slice.h" +#include "leveldb/status.h" + +namespace leveldb { + +class SequentialFile; + +namespace log { + +class Reader { + public: + // Interface for reporting errors. + class Reporter { + public: + virtual ~Reporter(); + + // Some corruption was detected. "size" is the approximate number + // of bytes dropped due to the corruption. + virtual void Corruption(size_t bytes, const Status& status) = 0; + }; + + // Create a reader that will return log records from "*file". + // "*file" must remain live while this Reader is in use. + // + // If "reporter" is non-NULL, it is notified whenever some data is + // dropped due to a detected corruption. "*reporter" must remain + // live while this Reader is in use. + // + // If "checksum" is true, verify checksums if available. + // + // The Reader will start reading at the first record located at physical + // position >= initial_offset within the file. + Reader(SequentialFile* file, Reporter* reporter, bool checksum, + uint64_t initial_offset); + + ~Reader(); + + // Read the next record into *record. Returns true if read + // successfully, false if we hit end of the input. May use + // "*scratch" as temporary storage. The contents filled in *record + // will only be valid until the next mutating operation on this + // reader or the next mutation to *scratch. + bool ReadRecord(Slice* record, std::string* scratch); + + // Returns the physical offset of the last record returned by ReadRecord. + // + // Undefined before the first call to ReadRecord. + uint64_t LastRecordOffset(); + + private: + SequentialFile* const file_; + Reporter* const reporter_; + bool const checksum_; + char* const backing_store_; + Slice buffer_; + bool eof_; // Last Read() indicated EOF by returning < kBlockSize + + // Offset of the last record returned by ReadRecord. + uint64_t last_record_offset_; + // Offset of the first location past the end of buffer_. + uint64_t end_of_buffer_offset_; + + // Offset at which to start looking for the first record to return + uint64_t const initial_offset_; + + // Extend record types with the following special values + enum { + kEof = kMaxRecordType + 1, + // Returned whenever we find an invalid physical record. + // Currently there are three situations in which this happens: + // * The record has an invalid CRC (ReadPhysicalRecord reports a drop) + // * The record is a 0-length record (No drop is reported) + // * The record is below constructor's initial_offset (No drop is reported) + kBadRecord = kMaxRecordType + 2 + }; + + // Skips all blocks that are completely before "initial_offset_". + // + // Returns true on success. Handles reporting. + bool SkipToInitialBlock(); + + // Return type, or one of the preceding special values + unsigned int ReadPhysicalRecord(Slice* result); + + // Reports dropped bytes to the reporter. + // buffer_ must be updated to remove the dropped bytes prior to invocation. + void ReportCorruption(uint64_t bytes, const char* reason); + void ReportDrop(uint64_t bytes, const Status& reason); + + // No copying allowed + Reader(const Reader&); + void operator=(const Reader&); +}; + +} // namespace log +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_LOG_READER_H_ diff --git a/src/leveldb/db/log_test.cc b/src/leveldb/db/log_test.cc new file mode 100644 index 00000000..dcf05626 --- /dev/null +++ b/src/leveldb/db/log_test.cc @@ -0,0 +1,530 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/log_reader.h" +#include "db/log_writer.h" +#include "leveldb/env.h" +#include "util/coding.h" +#include "util/crc32c.h" +#include "util/random.h" +#include "util/testharness.h" + +namespace leveldb { +namespace log { + +// Construct a string of the specified length made out of the supplied +// partial string. +static std::string BigString(const std::string& partial_string, size_t n) { + std::string result; + while (result.size() < n) { + result.append(partial_string); + } + result.resize(n); + return result; +} + +// Construct a string from a number +static std::string NumberString(int n) { + char buf[50]; + snprintf(buf, sizeof(buf), "%d.", n); + return std::string(buf); +} + +// Return a skewed potentially long string +static std::string RandomSkewedString(int i, Random* rnd) { + return BigString(NumberString(i), rnd->Skewed(17)); +} + +class LogTest { + private: + class StringDest : public WritableFile { + public: + std::string contents_; + + virtual Status Close() { return Status::OK(); } + virtual Status Flush() { return Status::OK(); } + virtual Status Sync() { return Status::OK(); } + virtual Status Append(const Slice& slice) { + contents_.append(slice.data(), slice.size()); + return Status::OK(); + } + }; + + class StringSource : public SequentialFile { + public: + Slice contents_; + bool force_error_; + bool returned_partial_; + StringSource() : force_error_(false), returned_partial_(false) { } + + virtual Status Read(size_t n, Slice* result, char* scratch) { + ASSERT_TRUE(!returned_partial_) << "must not Read() after eof/error"; + + if (force_error_) { + force_error_ = false; + returned_partial_ = true; + return Status::Corruption("read error"); + } + + if (contents_.size() < n) { + n = contents_.size(); + returned_partial_ = true; + } + *result = Slice(contents_.data(), n); + contents_.remove_prefix(n); + return Status::OK(); + } + + virtual Status Skip(uint64_t n) { + if (n > contents_.size()) { + contents_.clear(); + return Status::NotFound("in-memory file skipepd past end"); + } + + contents_.remove_prefix(n); + + return Status::OK(); + } + }; + + class ReportCollector : public Reader::Reporter { + public: + size_t dropped_bytes_; + std::string message_; + + ReportCollector() : dropped_bytes_(0) { } + virtual void Corruption(size_t bytes, const Status& status) { + dropped_bytes_ += bytes; + message_.append(status.ToString()); + } + }; + + StringDest dest_; + StringSource source_; + ReportCollector report_; + bool reading_; + Writer writer_; + Reader reader_; + + // Record metadata for testing initial offset functionality + static size_t initial_offset_record_sizes_[]; + static uint64_t initial_offset_last_record_offsets_[]; + + public: + LogTest() : reading_(false), + writer_(&dest_), + reader_(&source_, &report_, true/*checksum*/, + 0/*initial_offset*/) { + } + + void Write(const std::string& msg) { + ASSERT_TRUE(!reading_) << "Write() after starting to read"; + writer_.AddRecord(Slice(msg)); + } + + size_t WrittenBytes() const { + return dest_.contents_.size(); + } + + std::string Read() { + if (!reading_) { + reading_ = true; + source_.contents_ = Slice(dest_.contents_); + } + std::string scratch; + Slice record; + if (reader_.ReadRecord(&record, &scratch)) { + return record.ToString(); + } else { + return "EOF"; + } + } + + void IncrementByte(int offset, int delta) { + dest_.contents_[offset] += delta; + } + + void SetByte(int offset, char new_byte) { + dest_.contents_[offset] = new_byte; + } + + void ShrinkSize(int bytes) { + dest_.contents_.resize(dest_.contents_.size() - bytes); + } + + void FixChecksum(int header_offset, int len) { + // Compute crc of type/len/data + uint32_t crc = crc32c::Value(&dest_.contents_[header_offset+6], 1 + len); + crc = crc32c::Mask(crc); + EncodeFixed32(&dest_.contents_[header_offset], crc); + } + + void ForceError() { + source_.force_error_ = true; + } + + size_t DroppedBytes() const { + return report_.dropped_bytes_; + } + + std::string ReportMessage() const { + return report_.message_; + } + + // Returns OK iff recorded error message contains "msg" + std::string MatchError(const std::string& msg) const { + if (report_.message_.find(msg) == std::string::npos) { + return report_.message_; + } else { + return "OK"; + } + } + + void WriteInitialOffsetLog() { + for (int i = 0; i < 4; i++) { + std::string record(initial_offset_record_sizes_[i], + static_cast('a' + i)); + Write(record); + } + } + + void CheckOffsetPastEndReturnsNoRecords(uint64_t offset_past_end) { + WriteInitialOffsetLog(); + reading_ = true; + source_.contents_ = Slice(dest_.contents_); + Reader* offset_reader = new Reader(&source_, &report_, true/*checksum*/, + WrittenBytes() + offset_past_end); + Slice record; + std::string scratch; + ASSERT_TRUE(!offset_reader->ReadRecord(&record, &scratch)); + delete offset_reader; + } + + void CheckInitialOffsetRecord(uint64_t initial_offset, + int expected_record_offset) { + WriteInitialOffsetLog(); + reading_ = true; + source_.contents_ = Slice(dest_.contents_); + Reader* offset_reader = new Reader(&source_, &report_, true/*checksum*/, + initial_offset); + Slice record; + std::string scratch; + ASSERT_TRUE(offset_reader->ReadRecord(&record, &scratch)); + ASSERT_EQ(initial_offset_record_sizes_[expected_record_offset], + record.size()); + ASSERT_EQ(initial_offset_last_record_offsets_[expected_record_offset], + offset_reader->LastRecordOffset()); + ASSERT_EQ((char)('a' + expected_record_offset), record.data()[0]); + delete offset_reader; + } + +}; + +size_t LogTest::initial_offset_record_sizes_[] = + {10000, // Two sizable records in first block + 10000, + 2 * log::kBlockSize - 1000, // Span three blocks + 1}; + +uint64_t LogTest::initial_offset_last_record_offsets_[] = + {0, + kHeaderSize + 10000, + 2 * (kHeaderSize + 10000), + 2 * (kHeaderSize + 10000) + + (2 * log::kBlockSize - 1000) + 3 * kHeaderSize}; + + +TEST(LogTest, Empty) { + ASSERT_EQ("EOF", Read()); +} + +TEST(LogTest, ReadWrite) { + Write("foo"); + Write("bar"); + Write(""); + Write("xxxx"); + ASSERT_EQ("foo", Read()); + ASSERT_EQ("bar", Read()); + ASSERT_EQ("", Read()); + ASSERT_EQ("xxxx", Read()); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ("EOF", Read()); // Make sure reads at eof work +} + +TEST(LogTest, ManyBlocks) { + for (int i = 0; i < 100000; i++) { + Write(NumberString(i)); + } + for (int i = 0; i < 100000; i++) { + ASSERT_EQ(NumberString(i), Read()); + } + ASSERT_EQ("EOF", Read()); +} + +TEST(LogTest, Fragmentation) { + Write("small"); + Write(BigString("medium", 50000)); + Write(BigString("large", 100000)); + ASSERT_EQ("small", Read()); + ASSERT_EQ(BigString("medium", 50000), Read()); + ASSERT_EQ(BigString("large", 100000), Read()); + ASSERT_EQ("EOF", Read()); +} + +TEST(LogTest, MarginalTrailer) { + // Make a trailer that is exactly the same length as an empty record. + const int n = kBlockSize - 2*kHeaderSize; + Write(BigString("foo", n)); + ASSERT_EQ(kBlockSize - kHeaderSize, WrittenBytes()); + Write(""); + Write("bar"); + ASSERT_EQ(BigString("foo", n), Read()); + ASSERT_EQ("", Read()); + ASSERT_EQ("bar", Read()); + ASSERT_EQ("EOF", Read()); +} + +TEST(LogTest, MarginalTrailer2) { + // Make a trailer that is exactly the same length as an empty record. + const int n = kBlockSize - 2*kHeaderSize; + Write(BigString("foo", n)); + ASSERT_EQ(kBlockSize - kHeaderSize, WrittenBytes()); + Write("bar"); + ASSERT_EQ(BigString("foo", n), Read()); + ASSERT_EQ("bar", Read()); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ(0, DroppedBytes()); + ASSERT_EQ("", ReportMessage()); +} + +TEST(LogTest, ShortTrailer) { + const int n = kBlockSize - 2*kHeaderSize + 4; + Write(BigString("foo", n)); + ASSERT_EQ(kBlockSize - kHeaderSize + 4, WrittenBytes()); + Write(""); + Write("bar"); + ASSERT_EQ(BigString("foo", n), Read()); + ASSERT_EQ("", Read()); + ASSERT_EQ("bar", Read()); + ASSERT_EQ("EOF", Read()); +} + +TEST(LogTest, AlignedEof) { + const int n = kBlockSize - 2*kHeaderSize + 4; + Write(BigString("foo", n)); + ASSERT_EQ(kBlockSize - kHeaderSize + 4, WrittenBytes()); + ASSERT_EQ(BigString("foo", n), Read()); + ASSERT_EQ("EOF", Read()); +} + +TEST(LogTest, RandomRead) { + const int N = 500; + Random write_rnd(301); + for (int i = 0; i < N; i++) { + Write(RandomSkewedString(i, &write_rnd)); + } + Random read_rnd(301); + for (int i = 0; i < N; i++) { + ASSERT_EQ(RandomSkewedString(i, &read_rnd), Read()); + } + ASSERT_EQ("EOF", Read()); +} + +// Tests of all the error paths in log_reader.cc follow: + +TEST(LogTest, ReadError) { + Write("foo"); + ForceError(); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ(kBlockSize, DroppedBytes()); + ASSERT_EQ("OK", MatchError("read error")); +} + +TEST(LogTest, BadRecordType) { + Write("foo"); + // Type is stored in header[6] + IncrementByte(6, 100); + FixChecksum(0, 3); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ(3, DroppedBytes()); + ASSERT_EQ("OK", MatchError("unknown record type")); +} + +TEST(LogTest, TruncatedTrailingRecordIsIgnored) { + Write("foo"); + ShrinkSize(4); // Drop all payload as well as a header byte + ASSERT_EQ("EOF", Read()); + // Truncated last record is ignored, not treated as an error. + ASSERT_EQ(0, DroppedBytes()); + ASSERT_EQ("", ReportMessage()); +} + +TEST(LogTest, BadLength) { + const int kPayloadSize = kBlockSize - kHeaderSize; + Write(BigString("bar", kPayloadSize)); + Write("foo"); + // Least significant size byte is stored in header[4]. + IncrementByte(4, 1); + ASSERT_EQ("foo", Read()); + ASSERT_EQ(kBlockSize, DroppedBytes()); + ASSERT_EQ("OK", MatchError("bad record length")); +} + +TEST(LogTest, BadLengthAtEndIsIgnored) { + Write("foo"); + ShrinkSize(1); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ(0, DroppedBytes()); + ASSERT_EQ("", ReportMessage()); +} + +TEST(LogTest, ChecksumMismatch) { + Write("foo"); + IncrementByte(0, 10); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ(10, DroppedBytes()); + ASSERT_EQ("OK", MatchError("checksum mismatch")); +} + +TEST(LogTest, UnexpectedMiddleType) { + Write("foo"); + SetByte(6, kMiddleType); + FixChecksum(0, 3); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ(3, DroppedBytes()); + ASSERT_EQ("OK", MatchError("missing start")); +} + +TEST(LogTest, UnexpectedLastType) { + Write("foo"); + SetByte(6, kLastType); + FixChecksum(0, 3); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ(3, DroppedBytes()); + ASSERT_EQ("OK", MatchError("missing start")); +} + +TEST(LogTest, UnexpectedFullType) { + Write("foo"); + Write("bar"); + SetByte(6, kFirstType); + FixChecksum(0, 3); + ASSERT_EQ("bar", Read()); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ(3, DroppedBytes()); + ASSERT_EQ("OK", MatchError("partial record without end")); +} + +TEST(LogTest, UnexpectedFirstType) { + Write("foo"); + Write(BigString("bar", 100000)); + SetByte(6, kFirstType); + FixChecksum(0, 3); + ASSERT_EQ(BigString("bar", 100000), Read()); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ(3, DroppedBytes()); + ASSERT_EQ("OK", MatchError("partial record without end")); +} + +TEST(LogTest, MissingLastIsIgnored) { + Write(BigString("bar", kBlockSize)); + // Remove the LAST block, including header. + ShrinkSize(14); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ("", ReportMessage()); + ASSERT_EQ(0, DroppedBytes()); +} + +TEST(LogTest, PartialLastIsIgnored) { + Write(BigString("bar", kBlockSize)); + // Cause a bad record length in the LAST block. + ShrinkSize(1); + ASSERT_EQ("EOF", Read()); + ASSERT_EQ("", ReportMessage()); + ASSERT_EQ(0, DroppedBytes()); +} + +TEST(LogTest, ErrorJoinsRecords) { + // Consider two fragmented records: + // first(R1) last(R1) first(R2) last(R2) + // where the middle two fragments disappear. We do not want + // first(R1),last(R2) to get joined and returned as a valid record. + + // Write records that span two blocks + Write(BigString("foo", kBlockSize)); + Write(BigString("bar", kBlockSize)); + Write("correct"); + + // Wipe the middle block + for (int offset = kBlockSize; offset < 2*kBlockSize; offset++) { + SetByte(offset, 'x'); + } + + ASSERT_EQ("correct", Read()); + ASSERT_EQ("EOF", Read()); + const size_t dropped = DroppedBytes(); + ASSERT_LE(dropped, 2*kBlockSize + 100); + ASSERT_GE(dropped, 2*kBlockSize); +} + +TEST(LogTest, ReadStart) { + CheckInitialOffsetRecord(0, 0); +} + +TEST(LogTest, ReadSecondOneOff) { + CheckInitialOffsetRecord(1, 1); +} + +TEST(LogTest, ReadSecondTenThousand) { + CheckInitialOffsetRecord(10000, 1); +} + +TEST(LogTest, ReadSecondStart) { + CheckInitialOffsetRecord(10007, 1); +} + +TEST(LogTest, ReadThirdOneOff) { + CheckInitialOffsetRecord(10008, 2); +} + +TEST(LogTest, ReadThirdStart) { + CheckInitialOffsetRecord(20014, 2); +} + +TEST(LogTest, ReadFourthOneOff) { + CheckInitialOffsetRecord(20015, 3); +} + +TEST(LogTest, ReadFourthFirstBlockTrailer) { + CheckInitialOffsetRecord(log::kBlockSize - 4, 3); +} + +TEST(LogTest, ReadFourthMiddleBlock) { + CheckInitialOffsetRecord(log::kBlockSize + 1, 3); +} + +TEST(LogTest, ReadFourthLastBlock) { + CheckInitialOffsetRecord(2 * log::kBlockSize + 1, 3); +} + +TEST(LogTest, ReadFourthStart) { + CheckInitialOffsetRecord( + 2 * (kHeaderSize + 1000) + (2 * log::kBlockSize - 1000) + 3 * kHeaderSize, + 3); +} + +TEST(LogTest, ReadEnd) { + CheckOffsetPastEndReturnsNoRecords(0); +} + +TEST(LogTest, ReadPastEnd) { + CheckOffsetPastEndReturnsNoRecords(5); +} + +} // namespace log +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/db/log_writer.cc b/src/leveldb/db/log_writer.cc new file mode 100644 index 00000000..2da99ac0 --- /dev/null +++ b/src/leveldb/db/log_writer.cc @@ -0,0 +1,103 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/log_writer.h" + +#include +#include "leveldb/env.h" +#include "util/coding.h" +#include "util/crc32c.h" + +namespace leveldb { +namespace log { + +Writer::Writer(WritableFile* dest) + : dest_(dest), + block_offset_(0) { + for (int i = 0; i <= kMaxRecordType; i++) { + char t = static_cast(i); + type_crc_[i] = crc32c::Value(&t, 1); + } +} + +Writer::~Writer() { +} + +Status Writer::AddRecord(const Slice& slice) { + const char* ptr = slice.data(); + size_t left = slice.size(); + + // Fragment the record if necessary and emit it. Note that if slice + // is empty, we still want to iterate once to emit a single + // zero-length record + Status s; + bool begin = true; + do { + const int leftover = kBlockSize - block_offset_; + assert(leftover >= 0); + if (leftover < kHeaderSize) { + // Switch to a new block + if (leftover > 0) { + // Fill the trailer (literal below relies on kHeaderSize being 7) + assert(kHeaderSize == 7); + dest_->Append(Slice("\x00\x00\x00\x00\x00\x00", leftover)); + } + block_offset_ = 0; + } + + // Invariant: we never leave < kHeaderSize bytes in a block. + assert(kBlockSize - block_offset_ - kHeaderSize >= 0); + + const size_t avail = kBlockSize - block_offset_ - kHeaderSize; + const size_t fragment_length = (left < avail) ? left : avail; + + RecordType type; + const bool end = (left == fragment_length); + if (begin && end) { + type = kFullType; + } else if (begin) { + type = kFirstType; + } else if (end) { + type = kLastType; + } else { + type = kMiddleType; + } + + s = EmitPhysicalRecord(type, ptr, fragment_length); + ptr += fragment_length; + left -= fragment_length; + begin = false; + } while (s.ok() && left > 0); + return s; +} + +Status Writer::EmitPhysicalRecord(RecordType t, const char* ptr, size_t n) { + assert(n <= 0xffff); // Must fit in two bytes + assert(block_offset_ + kHeaderSize + n <= kBlockSize); + + // Format the header + char buf[kHeaderSize]; + buf[4] = static_cast(n & 0xff); + buf[5] = static_cast(n >> 8); + buf[6] = static_cast(t); + + // Compute the crc of the record type and the payload. + uint32_t crc = crc32c::Extend(type_crc_[t], ptr, n); + crc = crc32c::Mask(crc); // Adjust for storage + EncodeFixed32(buf, crc); + + // Write the header and the payload + Status s = dest_->Append(Slice(buf, kHeaderSize)); + if (s.ok()) { + s = dest_->Append(Slice(ptr, n)); + if (s.ok()) { + s = dest_->Flush(); + } + } + block_offset_ += kHeaderSize + n; + return s; +} + +} // namespace log +} // namespace leveldb diff --git a/src/leveldb/db/log_writer.h b/src/leveldb/db/log_writer.h new file mode 100644 index 00000000..a3a954d9 --- /dev/null +++ b/src/leveldb/db/log_writer.h @@ -0,0 +1,48 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_DB_LOG_WRITER_H_ +#define STORAGE_LEVELDB_DB_LOG_WRITER_H_ + +#include +#include "db/log_format.h" +#include "leveldb/slice.h" +#include "leveldb/status.h" + +namespace leveldb { + +class WritableFile; + +namespace log { + +class Writer { + public: + // Create a writer that will append data to "*dest". + // "*dest" must be initially empty. + // "*dest" must remain live while this Writer is in use. + explicit Writer(WritableFile* dest); + ~Writer(); + + Status AddRecord(const Slice& slice); + + private: + WritableFile* dest_; + int block_offset_; // Current offset in block + + // crc32c values for all supported record types. These are + // pre-computed to reduce the overhead of computing the crc of the + // record type stored in the header. + uint32_t type_crc_[kMaxRecordType + 1]; + + Status EmitPhysicalRecord(RecordType type, const char* ptr, size_t length); + + // No copying allowed + Writer(const Writer&); + void operator=(const Writer&); +}; + +} // namespace log +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_LOG_WRITER_H_ diff --git a/src/leveldb/db/memtable.cc b/src/leveldb/db/memtable.cc new file mode 100644 index 00000000..bfec0a7e --- /dev/null +++ b/src/leveldb/db/memtable.cc @@ -0,0 +1,145 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/memtable.h" +#include "db/dbformat.h" +#include "leveldb/comparator.h" +#include "leveldb/env.h" +#include "leveldb/iterator.h" +#include "util/coding.h" + +namespace leveldb { + +static Slice GetLengthPrefixedSlice(const char* data) { + uint32_t len; + const char* p = data; + p = GetVarint32Ptr(p, p + 5, &len); // +5: we assume "p" is not corrupted + return Slice(p, len); +} + +MemTable::MemTable(const InternalKeyComparator& cmp) + : comparator_(cmp), + refs_(0), + table_(comparator_, &arena_) { +} + +MemTable::~MemTable() { + assert(refs_ == 0); +} + +size_t MemTable::ApproximateMemoryUsage() { return arena_.MemoryUsage(); } + +int MemTable::KeyComparator::operator()(const char* aptr, const char* bptr) + const { + // Internal keys are encoded as length-prefixed strings. + Slice a = GetLengthPrefixedSlice(aptr); + Slice b = GetLengthPrefixedSlice(bptr); + return comparator.Compare(a, b); +} + +// Encode a suitable internal key target for "target" and return it. +// Uses *scratch as scratch space, and the returned pointer will point +// into this scratch space. +static const char* EncodeKey(std::string* scratch, const Slice& target) { + scratch->clear(); + PutVarint32(scratch, target.size()); + scratch->append(target.data(), target.size()); + return scratch->data(); +} + +class MemTableIterator: public Iterator { + public: + explicit MemTableIterator(MemTable::Table* table) : iter_(table) { } + + virtual bool Valid() const { return iter_.Valid(); } + virtual void Seek(const Slice& k) { iter_.Seek(EncodeKey(&tmp_, k)); } + virtual void SeekToFirst() { iter_.SeekToFirst(); } + virtual void SeekToLast() { iter_.SeekToLast(); } + virtual void Next() { iter_.Next(); } + virtual void Prev() { iter_.Prev(); } + virtual Slice key() const { return GetLengthPrefixedSlice(iter_.key()); } + virtual Slice value() const { + Slice key_slice = GetLengthPrefixedSlice(iter_.key()); + return GetLengthPrefixedSlice(key_slice.data() + key_slice.size()); + } + + virtual Status status() const { return Status::OK(); } + + private: + MemTable::Table::Iterator iter_; + std::string tmp_; // For passing to EncodeKey + + // No copying allowed + MemTableIterator(const MemTableIterator&); + void operator=(const MemTableIterator&); +}; + +Iterator* MemTable::NewIterator() { + return new MemTableIterator(&table_); +} + +void MemTable::Add(SequenceNumber s, ValueType type, + const Slice& key, + const Slice& value) { + // Format of an entry is concatenation of: + // key_size : varint32 of internal_key.size() + // key bytes : char[internal_key.size()] + // value_size : varint32 of value.size() + // value bytes : char[value.size()] + size_t key_size = key.size(); + size_t val_size = value.size(); + size_t internal_key_size = key_size + 8; + const size_t encoded_len = + VarintLength(internal_key_size) + internal_key_size + + VarintLength(val_size) + val_size; + char* buf = arena_.Allocate(encoded_len); + char* p = EncodeVarint32(buf, internal_key_size); + memcpy(p, key.data(), key_size); + p += key_size; + EncodeFixed64(p, (s << 8) | type); + p += 8; + p = EncodeVarint32(p, val_size); + memcpy(p, value.data(), val_size); + assert((p + val_size) - buf == encoded_len); + table_.Insert(buf); +} + +bool MemTable::Get(const LookupKey& key, std::string* value, Status* s) { + Slice memkey = key.memtable_key(); + Table::Iterator iter(&table_); + iter.Seek(memkey.data()); + if (iter.Valid()) { + // entry format is: + // klength varint32 + // userkey char[klength] + // tag uint64 + // vlength varint32 + // value char[vlength] + // Check that it belongs to same user key. We do not check the + // sequence number since the Seek() call above should have skipped + // all entries with overly large sequence numbers. + const char* entry = iter.key(); + uint32_t key_length; + const char* key_ptr = GetVarint32Ptr(entry, entry+5, &key_length); + if (comparator_.comparator.user_comparator()->Compare( + Slice(key_ptr, key_length - 8), + key.user_key()) == 0) { + // Correct user key + const uint64_t tag = DecodeFixed64(key_ptr + key_length - 8); + switch (static_cast(tag & 0xff)) { + case kTypeValue: { + Slice v = GetLengthPrefixedSlice(key_ptr + key_length); + value->assign(v.data(), v.size()); + return true; + } + case kTypeDeletion: + *s = Status::NotFound(Slice()); + return true; + } + } + } + return false; +} + +} // namespace leveldb diff --git a/src/leveldb/db/memtable.h b/src/leveldb/db/memtable.h new file mode 100644 index 00000000..92e90bb0 --- /dev/null +++ b/src/leveldb/db/memtable.h @@ -0,0 +1,91 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_DB_MEMTABLE_H_ +#define STORAGE_LEVELDB_DB_MEMTABLE_H_ + +#include +#include "leveldb/db.h" +#include "db/dbformat.h" +#include "db/skiplist.h" +#include "util/arena.h" + +namespace leveldb { + +class InternalKeyComparator; +class Mutex; +class MemTableIterator; + +class MemTable { + public: + // MemTables are reference counted. The initial reference count + // is zero and the caller must call Ref() at least once. + explicit MemTable(const InternalKeyComparator& comparator); + + // Increase reference count. + void Ref() { ++refs_; } + + // Drop reference count. Delete if no more references exist. + void Unref() { + --refs_; + assert(refs_ >= 0); + if (refs_ <= 0) { + delete this; + } + } + + // Returns an estimate of the number of bytes of data in use by this + // data structure. + // + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable. + size_t ApproximateMemoryUsage(); + + // Return an iterator that yields the contents of the memtable. + // + // The caller must ensure that the underlying MemTable remains live + // while the returned iterator is live. The keys returned by this + // iterator are internal keys encoded by AppendInternalKey in the + // db/format.{h,cc} module. + Iterator* NewIterator(); + + // Add an entry into memtable that maps key to value at the + // specified sequence number and with the specified type. + // Typically value will be empty if type==kTypeDeletion. + void Add(SequenceNumber seq, ValueType type, + const Slice& key, + const Slice& value); + + // If memtable contains a value for key, store it in *value and return true. + // If memtable contains a deletion for key, store a NotFound() error + // in *status and return true. + // Else, return false. + bool Get(const LookupKey& key, std::string* value, Status* s); + + private: + ~MemTable(); // Private since only Unref() should be used to delete it + + struct KeyComparator { + const InternalKeyComparator comparator; + explicit KeyComparator(const InternalKeyComparator& c) : comparator(c) { } + int operator()(const char* a, const char* b) const; + }; + friend class MemTableIterator; + friend class MemTableBackwardIterator; + + typedef SkipList Table; + + KeyComparator comparator_; + int refs_; + Arena arena_; + Table table_; + + // No copying allowed + MemTable(const MemTable&); + void operator=(const MemTable&); +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_MEMTABLE_H_ diff --git a/src/leveldb/db/repair.cc b/src/leveldb/db/repair.cc new file mode 100644 index 00000000..4cd4bb04 --- /dev/null +++ b/src/leveldb/db/repair.cc @@ -0,0 +1,461 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// We recover the contents of the descriptor from the other files we find. +// (1) Any log files are first converted to tables +// (2) We scan every table to compute +// (a) smallest/largest for the table +// (b) largest sequence number in the table +// (3) We generate descriptor contents: +// - log number is set to zero +// - next-file-number is set to 1 + largest file number we found +// - last-sequence-number is set to largest sequence# found across +// all tables (see 2c) +// - compaction pointers are cleared +// - every table file is added at level 0 +// +// Possible optimization 1: +// (a) Compute total size and use to pick appropriate max-level M +// (b) Sort tables by largest sequence# in the table +// (c) For each table: if it overlaps earlier table, place in level-0, +// else place in level-M. +// Possible optimization 2: +// Store per-table metadata (smallest, largest, largest-seq#, ...) +// in the table's meta section to speed up ScanTable. + +#include "db/builder.h" +#include "db/db_impl.h" +#include "db/dbformat.h" +#include "db/filename.h" +#include "db/log_reader.h" +#include "db/log_writer.h" +#include "db/memtable.h" +#include "db/table_cache.h" +#include "db/version_edit.h" +#include "db/write_batch_internal.h" +#include "leveldb/comparator.h" +#include "leveldb/db.h" +#include "leveldb/env.h" + +namespace leveldb { + +namespace { + +class Repairer { + public: + Repairer(const std::string& dbname, const Options& options) + : dbname_(dbname), + env_(options.env), + icmp_(options.comparator), + ipolicy_(options.filter_policy), + options_(SanitizeOptions(dbname, &icmp_, &ipolicy_, options)), + owns_info_log_(options_.info_log != options.info_log), + owns_cache_(options_.block_cache != options.block_cache), + next_file_number_(1) { + // TableCache can be small since we expect each table to be opened once. + table_cache_ = new TableCache(dbname_, &options_, 10); + } + + ~Repairer() { + delete table_cache_; + if (owns_info_log_) { + delete options_.info_log; + } + if (owns_cache_) { + delete options_.block_cache; + } + } + + Status Run() { + Status status = FindFiles(); + if (status.ok()) { + ConvertLogFilesToTables(); + ExtractMetaData(); + status = WriteDescriptor(); + } + if (status.ok()) { + unsigned long long bytes = 0; + for (size_t i = 0; i < tables_.size(); i++) { + bytes += tables_[i].meta.file_size; + } + Log(options_.info_log, + "**** Repaired leveldb %s; " + "recovered %d files; %llu bytes. " + "Some data may have been lost. " + "****", + dbname_.c_str(), + static_cast(tables_.size()), + bytes); + } + return status; + } + + private: + struct TableInfo { + FileMetaData meta; + SequenceNumber max_sequence; + }; + + std::string const dbname_; + Env* const env_; + InternalKeyComparator const icmp_; + InternalFilterPolicy const ipolicy_; + Options const options_; + bool owns_info_log_; + bool owns_cache_; + TableCache* table_cache_; + VersionEdit edit_; + + std::vector manifests_; + std::vector table_numbers_; + std::vector logs_; + std::vector tables_; + uint64_t next_file_number_; + + Status FindFiles() { + std::vector filenames; + Status status = env_->GetChildren(dbname_, &filenames); + if (!status.ok()) { + return status; + } + if (filenames.empty()) { + return Status::IOError(dbname_, "repair found no files"); + } + + uint64_t number; + FileType type; + for (size_t i = 0; i < filenames.size(); i++) { + if (ParseFileName(filenames[i], &number, &type)) { + if (type == kDescriptorFile) { + manifests_.push_back(filenames[i]); + } else { + if (number + 1 > next_file_number_) { + next_file_number_ = number + 1; + } + if (type == kLogFile) { + logs_.push_back(number); + } else if (type == kTableFile) { + table_numbers_.push_back(number); + } else { + // Ignore other files + } + } + } + } + return status; + } + + void ConvertLogFilesToTables() { + for (size_t i = 0; i < logs_.size(); i++) { + std::string logname = LogFileName(dbname_, logs_[i]); + Status status = ConvertLogToTable(logs_[i]); + if (!status.ok()) { + Log(options_.info_log, "Log #%llu: ignoring conversion error: %s", + (unsigned long long) logs_[i], + status.ToString().c_str()); + } + ArchiveFile(logname); + } + } + + Status ConvertLogToTable(uint64_t log) { + struct LogReporter : public log::Reader::Reporter { + Env* env; + Logger* info_log; + uint64_t lognum; + virtual void Corruption(size_t bytes, const Status& s) { + // We print error messages for corruption, but continue repairing. + Log(info_log, "Log #%llu: dropping %d bytes; %s", + (unsigned long long) lognum, + static_cast(bytes), + s.ToString().c_str()); + } + }; + + // Open the log file + std::string logname = LogFileName(dbname_, log); + SequentialFile* lfile; + Status status = env_->NewSequentialFile(logname, &lfile); + if (!status.ok()) { + return status; + } + + // Create the log reader. + LogReporter reporter; + reporter.env = env_; + reporter.info_log = options_.info_log; + reporter.lognum = log; + // We intentionally make log::Reader do checksumming so that + // corruptions cause entire commits to be skipped instead of + // propagating bad information (like overly large sequence + // numbers). + log::Reader reader(lfile, &reporter, false/*do not checksum*/, + 0/*initial_offset*/); + + // Read all the records and add to a memtable + std::string scratch; + Slice record; + WriteBatch batch; + MemTable* mem = new MemTable(icmp_); + mem->Ref(); + int counter = 0; + while (reader.ReadRecord(&record, &scratch)) { + if (record.size() < 12) { + reporter.Corruption( + record.size(), Status::Corruption("log record too small")); + continue; + } + WriteBatchInternal::SetContents(&batch, record); + status = WriteBatchInternal::InsertInto(&batch, mem); + if (status.ok()) { + counter += WriteBatchInternal::Count(&batch); + } else { + Log(options_.info_log, "Log #%llu: ignoring %s", + (unsigned long long) log, + status.ToString().c_str()); + status = Status::OK(); // Keep going with rest of file + } + } + delete lfile; + + // Do not record a version edit for this conversion to a Table + // since ExtractMetaData() will also generate edits. + FileMetaData meta; + meta.number = next_file_number_++; + Iterator* iter = mem->NewIterator(); + status = BuildTable(dbname_, env_, options_, table_cache_, iter, &meta); + delete iter; + mem->Unref(); + mem = NULL; + if (status.ok()) { + if (meta.file_size > 0) { + table_numbers_.push_back(meta.number); + } + } + Log(options_.info_log, "Log #%llu: %d ops saved to Table #%llu %s", + (unsigned long long) log, + counter, + (unsigned long long) meta.number, + status.ToString().c_str()); + return status; + } + + void ExtractMetaData() { + for (size_t i = 0; i < table_numbers_.size(); i++) { + ScanTable(table_numbers_[i]); + } + } + + Iterator* NewTableIterator(const FileMetaData& meta) { + // Same as compaction iterators: if paranoid_checks are on, turn + // on checksum verification. + ReadOptions r; + r.verify_checksums = options_.paranoid_checks; + return table_cache_->NewIterator(r, meta.number, meta.file_size); + } + + void ScanTable(uint64_t number) { + TableInfo t; + t.meta.number = number; + std::string fname = TableFileName(dbname_, number); + Status status = env_->GetFileSize(fname, &t.meta.file_size); + if (!status.ok()) { + // Try alternate file name. + fname = SSTTableFileName(dbname_, number); + Status s2 = env_->GetFileSize(fname, &t.meta.file_size); + if (s2.ok()) { + status = Status::OK(); + } + } + if (!status.ok()) { + ArchiveFile(TableFileName(dbname_, number)); + ArchiveFile(SSTTableFileName(dbname_, number)); + Log(options_.info_log, "Table #%llu: dropped: %s", + (unsigned long long) t.meta.number, + status.ToString().c_str()); + return; + } + + // Extract metadata by scanning through table. + int counter = 0; + Iterator* iter = NewTableIterator(t.meta); + bool empty = true; + ParsedInternalKey parsed; + t.max_sequence = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + Slice key = iter->key(); + if (!ParseInternalKey(key, &parsed)) { + Log(options_.info_log, "Table #%llu: unparsable key %s", + (unsigned long long) t.meta.number, + EscapeString(key).c_str()); + continue; + } + + counter++; + if (empty) { + empty = false; + t.meta.smallest.DecodeFrom(key); + } + t.meta.largest.DecodeFrom(key); + if (parsed.sequence > t.max_sequence) { + t.max_sequence = parsed.sequence; + } + } + if (!iter->status().ok()) { + status = iter->status(); + } + delete iter; + Log(options_.info_log, "Table #%llu: %d entries %s", + (unsigned long long) t.meta.number, + counter, + status.ToString().c_str()); + + if (status.ok()) { + tables_.push_back(t); + } else { + RepairTable(fname, t); // RepairTable archives input file. + } + } + + void RepairTable(const std::string& src, TableInfo t) { + // We will copy src contents to a new table and then rename the + // new table over the source. + + // Create builder. + std::string copy = TableFileName(dbname_, next_file_number_++); + WritableFile* file; + Status s = env_->NewWritableFile(copy, &file); + if (!s.ok()) { + return; + } + TableBuilder* builder = new TableBuilder(options_, file); + + // Copy data. + Iterator* iter = NewTableIterator(t.meta); + int counter = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + builder->Add(iter->key(), iter->value()); + counter++; + } + delete iter; + + ArchiveFile(src); + if (counter == 0) { + builder->Abandon(); // Nothing to save + } else { + s = builder->Finish(); + if (s.ok()) { + t.meta.file_size = builder->FileSize(); + } + } + delete builder; + builder = NULL; + + if (s.ok()) { + s = file->Close(); + } + delete file; + file = NULL; + + if (counter > 0 && s.ok()) { + std::string orig = TableFileName(dbname_, t.meta.number); + s = env_->RenameFile(copy, orig); + if (s.ok()) { + Log(options_.info_log, "Table #%llu: %d entries repaired", + (unsigned long long) t.meta.number, counter); + tables_.push_back(t); + } + } + if (!s.ok()) { + env_->DeleteFile(copy); + } + } + + Status WriteDescriptor() { + std::string tmp = TempFileName(dbname_, 1); + WritableFile* file; + Status status = env_->NewWritableFile(tmp, &file); + if (!status.ok()) { + return status; + } + + SequenceNumber max_sequence = 0; + for (size_t i = 0; i < tables_.size(); i++) { + if (max_sequence < tables_[i].max_sequence) { + max_sequence = tables_[i].max_sequence; + } + } + + edit_.SetComparatorName(icmp_.user_comparator()->Name()); + edit_.SetLogNumber(0); + edit_.SetNextFile(next_file_number_); + edit_.SetLastSequence(max_sequence); + + for (size_t i = 0; i < tables_.size(); i++) { + // TODO(opt): separate out into multiple levels + const TableInfo& t = tables_[i]; + edit_.AddFile(0, t.meta.number, t.meta.file_size, + t.meta.smallest, t.meta.largest); + } + + //fprintf(stderr, "NewDescriptor:\n%s\n", edit_.DebugString().c_str()); + { + log::Writer log(file); + std::string record; + edit_.EncodeTo(&record); + status = log.AddRecord(record); + } + if (status.ok()) { + status = file->Close(); + } + delete file; + file = NULL; + + if (!status.ok()) { + env_->DeleteFile(tmp); + } else { + // Discard older manifests + for (size_t i = 0; i < manifests_.size(); i++) { + ArchiveFile(dbname_ + "/" + manifests_[i]); + } + + // Install new manifest + status = env_->RenameFile(tmp, DescriptorFileName(dbname_, 1)); + if (status.ok()) { + status = SetCurrentFile(env_, dbname_, 1); + } else { + env_->DeleteFile(tmp); + } + } + return status; + } + + void ArchiveFile(const std::string& fname) { + // Move into another directory. E.g., for + // dir/foo + // rename to + // dir/lost/foo + const char* slash = strrchr(fname.c_str(), '/'); + std::string new_dir; + if (slash != NULL) { + new_dir.assign(fname.data(), slash - fname.data()); + } + new_dir.append("/lost"); + env_->CreateDir(new_dir); // Ignore error + std::string new_file = new_dir; + new_file.append("/"); + new_file.append((slash == NULL) ? fname.c_str() : slash + 1); + Status s = env_->RenameFile(fname, new_file); + Log(options_.info_log, "Archiving %s: %s\n", + fname.c_str(), s.ToString().c_str()); + } +}; +} // namespace + +Status RepairDB(const std::string& dbname, const Options& options) { + Repairer repairer(dbname, options); + return repairer.Run(); +} + +} // namespace leveldb diff --git a/src/leveldb/db/skiplist.h b/src/leveldb/db/skiplist.h new file mode 100644 index 00000000..ed8b0922 --- /dev/null +++ b/src/leveldb/db/skiplist.h @@ -0,0 +1,384 @@ +#ifndef STORAGE_LEVELDB_DB_SKIPLIST_H_ +#define STORAGE_LEVELDB_DB_SKIPLIST_H_ + +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// Thread safety +// ------------- +// +// Writes require external synchronization, most likely a mutex. +// Reads require a guarantee that the SkipList will not be destroyed +// while the read is in progress. Apart from that, reads progress +// without any internal locking or synchronization. +// +// Invariants: +// +// (1) Allocated nodes are never deleted until the SkipList is +// destroyed. This is trivially guaranteed by the code since we +// never delete any skip list nodes. +// +// (2) The contents of a Node except for the next/prev pointers are +// immutable after the Node has been linked into the SkipList. +// Only Insert() modifies the list, and it is careful to initialize +// a node and use release-stores to publish the nodes in one or +// more lists. +// +// ... prev vs. next pointer ordering ... + +#include +#include +#include "port/port.h" +#include "util/arena.h" +#include "util/random.h" + +namespace leveldb { + +class Arena; + +template +class SkipList { + private: + struct Node; + + public: + // Create a new SkipList object that will use "cmp" for comparing keys, + // and will allocate memory using "*arena". Objects allocated in the arena + // must remain allocated for the lifetime of the skiplist object. + explicit SkipList(Comparator cmp, Arena* arena); + + // Insert key into the list. + // REQUIRES: nothing that compares equal to key is currently in the list. + void Insert(const Key& key); + + // Returns true iff an entry that compares equal to key is in the list. + bool Contains(const Key& key) const; + + // Iteration over the contents of a skip list + class Iterator { + public: + // Initialize an iterator over the specified list. + // The returned iterator is not valid. + explicit Iterator(const SkipList* list); + + // Returns true iff the iterator is positioned at a valid node. + bool Valid() const; + + // Returns the key at the current position. + // REQUIRES: Valid() + const Key& key() const; + + // Advances to the next position. + // REQUIRES: Valid() + void Next(); + + // Advances to the previous position. + // REQUIRES: Valid() + void Prev(); + + // Advance to the first entry with a key >= target + void Seek(const Key& target); + + // Position at the first entry in list. + // Final state of iterator is Valid() iff list is not empty. + void SeekToFirst(); + + // Position at the last entry in list. + // Final state of iterator is Valid() iff list is not empty. + void SeekToLast(); + + private: + const SkipList* list_; + Node* node_; + // Intentionally copyable + }; + + private: + enum { kMaxHeight = 12 }; + + // Immutable after construction + Comparator const compare_; + Arena* const arena_; // Arena used for allocations of nodes + + Node* const head_; + + // Modified only by Insert(). Read racily by readers, but stale + // values are ok. + port::AtomicPointer max_height_; // Height of the entire list + + inline int GetMaxHeight() const { + return static_cast( + reinterpret_cast(max_height_.NoBarrier_Load())); + } + + // Read/written only by Insert(). + Random rnd_; + + Node* NewNode(const Key& key, int height); + int RandomHeight(); + bool Equal(const Key& a, const Key& b) const { return (compare_(a, b) == 0); } + + // Return true if key is greater than the data stored in "n" + bool KeyIsAfterNode(const Key& key, Node* n) const; + + // Return the earliest node that comes at or after key. + // Return NULL if there is no such node. + // + // If prev is non-NULL, fills prev[level] with pointer to previous + // node at "level" for every level in [0..max_height_-1]. + Node* FindGreaterOrEqual(const Key& key, Node** prev) const; + + // Return the latest node with a key < key. + // Return head_ if there is no such node. + Node* FindLessThan(const Key& key) const; + + // Return the last node in the list. + // Return head_ if list is empty. + Node* FindLast() const; + + // No copying allowed + SkipList(const SkipList&); + void operator=(const SkipList&); +}; + +// Implementation details follow +template +struct SkipList::Node { + explicit Node(const Key& k) : key(k) { } + + Key const key; + + // Accessors/mutators for links. Wrapped in methods so we can + // add the appropriate barriers as necessary. + Node* Next(int n) { + assert(n >= 0); + // Use an 'acquire load' so that we observe a fully initialized + // version of the returned Node. + return reinterpret_cast(next_[n].Acquire_Load()); + } + void SetNext(int n, Node* x) { + assert(n >= 0); + // Use a 'release store' so that anybody who reads through this + // pointer observes a fully initialized version of the inserted node. + next_[n].Release_Store(x); + } + + // No-barrier variants that can be safely used in a few locations. + Node* NoBarrier_Next(int n) { + assert(n >= 0); + return reinterpret_cast(next_[n].NoBarrier_Load()); + } + void NoBarrier_SetNext(int n, Node* x) { + assert(n >= 0); + next_[n].NoBarrier_Store(x); + } + + private: + // Array of length equal to the node height. next_[0] is lowest level link. + port::AtomicPointer next_[1]; +}; + +template +typename SkipList::Node* +SkipList::NewNode(const Key& key, int height) { + char* mem = arena_->AllocateAligned( + sizeof(Node) + sizeof(port::AtomicPointer) * (height - 1)); + return new (mem) Node(key); +} + +template +inline SkipList::Iterator::Iterator(const SkipList* list) { + list_ = list; + node_ = NULL; +} + +template +inline bool SkipList::Iterator::Valid() const { + return node_ != NULL; +} + +template +inline const Key& SkipList::Iterator::key() const { + assert(Valid()); + return node_->key; +} + +template +inline void SkipList::Iterator::Next() { + assert(Valid()); + node_ = node_->Next(0); +} + +template +inline void SkipList::Iterator::Prev() { + // Instead of using explicit "prev" links, we just search for the + // last node that falls before key. + assert(Valid()); + node_ = list_->FindLessThan(node_->key); + if (node_ == list_->head_) { + node_ = NULL; + } +} + +template +inline void SkipList::Iterator::Seek(const Key& target) { + node_ = list_->FindGreaterOrEqual(target, NULL); +} + +template +inline void SkipList::Iterator::SeekToFirst() { + node_ = list_->head_->Next(0); +} + +template +inline void SkipList::Iterator::SeekToLast() { + node_ = list_->FindLast(); + if (node_ == list_->head_) { + node_ = NULL; + } +} + +template +int SkipList::RandomHeight() { + // Increase height with probability 1 in kBranching + static const unsigned int kBranching = 4; + int height = 1; + while (height < kMaxHeight && ((rnd_.Next() % kBranching) == 0)) { + height++; + } + assert(height > 0); + assert(height <= kMaxHeight); + return height; +} + +template +bool SkipList::KeyIsAfterNode(const Key& key, Node* n) const { + // NULL n is considered infinite + return (n != NULL) && (compare_(n->key, key) < 0); +} + +template +typename SkipList::Node* SkipList::FindGreaterOrEqual(const Key& key, Node** prev) + const { + Node* x = head_; + int level = GetMaxHeight() - 1; + while (true) { + Node* next = x->Next(level); + if (KeyIsAfterNode(key, next)) { + // Keep searching in this list + x = next; + } else { + if (prev != NULL) prev[level] = x; + if (level == 0) { + return next; + } else { + // Switch to next list + level--; + } + } + } +} + +template +typename SkipList::Node* +SkipList::FindLessThan(const Key& key) const { + Node* x = head_; + int level = GetMaxHeight() - 1; + while (true) { + assert(x == head_ || compare_(x->key, key) < 0); + Node* next = x->Next(level); + if (next == NULL || compare_(next->key, key) >= 0) { + if (level == 0) { + return x; + } else { + // Switch to next list + level--; + } + } else { + x = next; + } + } +} + +template +typename SkipList::Node* SkipList::FindLast() + const { + Node* x = head_; + int level = GetMaxHeight() - 1; + while (true) { + Node* next = x->Next(level); + if (next == NULL) { + if (level == 0) { + return x; + } else { + // Switch to next list + level--; + } + } else { + x = next; + } + } +} + +template +SkipList::SkipList(Comparator cmp, Arena* arena) + : compare_(cmp), + arena_(arena), + head_(NewNode(0 /* any key will do */, kMaxHeight)), + max_height_(reinterpret_cast(1)), + rnd_(0xdeadbeef) { + for (int i = 0; i < kMaxHeight; i++) { + head_->SetNext(i, NULL); + } +} + +template +void SkipList::Insert(const Key& key) { + // TODO(opt): We can use a barrier-free variant of FindGreaterOrEqual() + // here since Insert() is externally synchronized. + Node* prev[kMaxHeight]; + Node* x = FindGreaterOrEqual(key, prev); + + // Our data structure does not allow duplicate insertion + assert(x == NULL || !Equal(key, x->key)); + + int height = RandomHeight(); + if (height > GetMaxHeight()) { + for (int i = GetMaxHeight(); i < height; i++) { + prev[i] = head_; + } + //fprintf(stderr, "Change height from %d to %d\n", max_height_, height); + + // It is ok to mutate max_height_ without any synchronization + // with concurrent readers. A concurrent reader that observes + // the new value of max_height_ will see either the old value of + // new level pointers from head_ (NULL), or a new value set in + // the loop below. In the former case the reader will + // immediately drop to the next level since NULL sorts after all + // keys. In the latter case the reader will use the new node. + max_height_.NoBarrier_Store(reinterpret_cast(height)); + } + + x = NewNode(key, height); + for (int i = 0; i < height; i++) { + // NoBarrier_SetNext() suffices since we will add a barrier when + // we publish a pointer to "x" in prev[i]. + x->NoBarrier_SetNext(i, prev[i]->NoBarrier_Next(i)); + prev[i]->SetNext(i, x); + } +} + +template +bool SkipList::Contains(const Key& key) const { + Node* x = FindGreaterOrEqual(key, NULL); + if (x != NULL && Equal(key, x->key)) { + return true; + } else { + return false; + } +} + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_SKIPLIST_H_ diff --git a/src/leveldb/db/skiplist_test.cc b/src/leveldb/db/skiplist_test.cc new file mode 100644 index 00000000..c78f4b4f --- /dev/null +++ b/src/leveldb/db/skiplist_test.cc @@ -0,0 +1,378 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/skiplist.h" +#include +#include "leveldb/env.h" +#include "util/arena.h" +#include "util/hash.h" +#include "util/random.h" +#include "util/testharness.h" + +namespace leveldb { + +typedef uint64_t Key; + +struct Comparator { + int operator()(const Key& a, const Key& b) const { + if (a < b) { + return -1; + } else if (a > b) { + return +1; + } else { + return 0; + } + } +}; + +class SkipTest { }; + +TEST(SkipTest, Empty) { + Arena arena; + Comparator cmp; + SkipList list(cmp, &arena); + ASSERT_TRUE(!list.Contains(10)); + + SkipList::Iterator iter(&list); + ASSERT_TRUE(!iter.Valid()); + iter.SeekToFirst(); + ASSERT_TRUE(!iter.Valid()); + iter.Seek(100); + ASSERT_TRUE(!iter.Valid()); + iter.SeekToLast(); + ASSERT_TRUE(!iter.Valid()); +} + +TEST(SkipTest, InsertAndLookup) { + const int N = 2000; + const int R = 5000; + Random rnd(1000); + std::set keys; + Arena arena; + Comparator cmp; + SkipList list(cmp, &arena); + for (int i = 0; i < N; i++) { + Key key = rnd.Next() % R; + if (keys.insert(key).second) { + list.Insert(key); + } + } + + for (int i = 0; i < R; i++) { + if (list.Contains(i)) { + ASSERT_EQ(keys.count(i), 1); + } else { + ASSERT_EQ(keys.count(i), 0); + } + } + + // Simple iterator tests + { + SkipList::Iterator iter(&list); + ASSERT_TRUE(!iter.Valid()); + + iter.Seek(0); + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*(keys.begin()), iter.key()); + + iter.SeekToFirst(); + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*(keys.begin()), iter.key()); + + iter.SeekToLast(); + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*(keys.rbegin()), iter.key()); + } + + // Forward iteration test + for (int i = 0; i < R; i++) { + SkipList::Iterator iter(&list); + iter.Seek(i); + + // Compare against model iterator + std::set::iterator model_iter = keys.lower_bound(i); + for (int j = 0; j < 3; j++) { + if (model_iter == keys.end()) { + ASSERT_TRUE(!iter.Valid()); + break; + } else { + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*model_iter, iter.key()); + ++model_iter; + iter.Next(); + } + } + } + + // Backward iteration test + { + SkipList::Iterator iter(&list); + iter.SeekToLast(); + + // Compare against model iterator + for (std::set::reverse_iterator model_iter = keys.rbegin(); + model_iter != keys.rend(); + ++model_iter) { + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*model_iter, iter.key()); + iter.Prev(); + } + ASSERT_TRUE(!iter.Valid()); + } +} + +// We want to make sure that with a single writer and multiple +// concurrent readers (with no synchronization other than when a +// reader's iterator is created), the reader always observes all the +// data that was present in the skip list when the iterator was +// constructor. Because insertions are happening concurrently, we may +// also observe new values that were inserted since the iterator was +// constructed, but we should never miss any values that were present +// at iterator construction time. +// +// We generate multi-part keys: +// +// where: +// key is in range [0..K-1] +// gen is a generation number for key +// hash is hash(key,gen) +// +// The insertion code picks a random key, sets gen to be 1 + the last +// generation number inserted for that key, and sets hash to Hash(key,gen). +// +// At the beginning of a read, we snapshot the last inserted +// generation number for each key. We then iterate, including random +// calls to Next() and Seek(). For every key we encounter, we +// check that it is either expected given the initial snapshot or has +// been concurrently added since the iterator started. +class ConcurrentTest { + private: + static const uint32_t K = 4; + + static uint64_t key(Key key) { return (key >> 40); } + static uint64_t gen(Key key) { return (key >> 8) & 0xffffffffu; } + static uint64_t hash(Key key) { return key & 0xff; } + + static uint64_t HashNumbers(uint64_t k, uint64_t g) { + uint64_t data[2] = { k, g }; + return Hash(reinterpret_cast(data), sizeof(data), 0); + } + + static Key MakeKey(uint64_t k, uint64_t g) { + assert(sizeof(Key) == sizeof(uint64_t)); + assert(k <= K); // We sometimes pass K to seek to the end of the skiplist + assert(g <= 0xffffffffu); + return ((k << 40) | (g << 8) | (HashNumbers(k, g) & 0xff)); + } + + static bool IsValidKey(Key k) { + return hash(k) == (HashNumbers(key(k), gen(k)) & 0xff); + } + + static Key RandomTarget(Random* rnd) { + switch (rnd->Next() % 10) { + case 0: + // Seek to beginning + return MakeKey(0, 0); + case 1: + // Seek to end + return MakeKey(K, 0); + default: + // Seek to middle + return MakeKey(rnd->Next() % K, 0); + } + } + + // Per-key generation + struct State { + port::AtomicPointer generation[K]; + void Set(int k, intptr_t v) { + generation[k].Release_Store(reinterpret_cast(v)); + } + intptr_t Get(int k) { + return reinterpret_cast(generation[k].Acquire_Load()); + } + + State() { + for (int k = 0; k < K; k++) { + Set(k, 0); + } + } + }; + + // Current state of the test + State current_; + + Arena arena_; + + // SkipList is not protected by mu_. We just use a single writer + // thread to modify it. + SkipList list_; + + public: + ConcurrentTest() : list_(Comparator(), &arena_) { } + + // REQUIRES: External synchronization + void WriteStep(Random* rnd) { + const uint32_t k = rnd->Next() % K; + const intptr_t g = current_.Get(k) + 1; + const Key key = MakeKey(k, g); + list_.Insert(key); + current_.Set(k, g); + } + + void ReadStep(Random* rnd) { + // Remember the initial committed state of the skiplist. + State initial_state; + for (int k = 0; k < K; k++) { + initial_state.Set(k, current_.Get(k)); + } + + Key pos = RandomTarget(rnd); + SkipList::Iterator iter(&list_); + iter.Seek(pos); + while (true) { + Key current; + if (!iter.Valid()) { + current = MakeKey(K, 0); + } else { + current = iter.key(); + ASSERT_TRUE(IsValidKey(current)) << current; + } + ASSERT_LE(pos, current) << "should not go backwards"; + + // Verify that everything in [pos,current) was not present in + // initial_state. + while (pos < current) { + ASSERT_LT(key(pos), K) << pos; + + // Note that generation 0 is never inserted, so it is ok if + // <*,0,*> is missing. + ASSERT_TRUE((gen(pos) == 0) || + (gen(pos) > initial_state.Get(key(pos))) + ) << "key: " << key(pos) + << "; gen: " << gen(pos) + << "; initgen: " + << initial_state.Get(key(pos)); + + // Advance to next key in the valid key space + if (key(pos) < key(current)) { + pos = MakeKey(key(pos) + 1, 0); + } else { + pos = MakeKey(key(pos), gen(pos) + 1); + } + } + + if (!iter.Valid()) { + break; + } + + if (rnd->Next() % 2) { + iter.Next(); + pos = MakeKey(key(pos), gen(pos) + 1); + } else { + Key new_target = RandomTarget(rnd); + if (new_target > pos) { + pos = new_target; + iter.Seek(new_target); + } + } + } + } +}; +const uint32_t ConcurrentTest::K; + +// Simple test that does single-threaded testing of the ConcurrentTest +// scaffolding. +TEST(SkipTest, ConcurrentWithoutThreads) { + ConcurrentTest test; + Random rnd(test::RandomSeed()); + for (int i = 0; i < 10000; i++) { + test.ReadStep(&rnd); + test.WriteStep(&rnd); + } +} + +class TestState { + public: + ConcurrentTest t_; + int seed_; + port::AtomicPointer quit_flag_; + + enum ReaderState { + STARTING, + RUNNING, + DONE + }; + + explicit TestState(int s) + : seed_(s), + quit_flag_(NULL), + state_(STARTING), + state_cv_(&mu_) {} + + void Wait(ReaderState s) { + mu_.Lock(); + while (state_ != s) { + state_cv_.Wait(); + } + mu_.Unlock(); + } + + void Change(ReaderState s) { + mu_.Lock(); + state_ = s; + state_cv_.Signal(); + mu_.Unlock(); + } + + private: + port::Mutex mu_; + ReaderState state_; + port::CondVar state_cv_; +}; + +static void ConcurrentReader(void* arg) { + TestState* state = reinterpret_cast(arg); + Random rnd(state->seed_); + int64_t reads = 0; + state->Change(TestState::RUNNING); + while (!state->quit_flag_.Acquire_Load()) { + state->t_.ReadStep(&rnd); + ++reads; + } + state->Change(TestState::DONE); +} + +static void RunConcurrent(int run) { + const int seed = test::RandomSeed() + (run * 100); + Random rnd(seed); + const int N = 1000; + const int kSize = 1000; + for (int i = 0; i < N; i++) { + if ((i % 100) == 0) { + fprintf(stderr, "Run %d of %d\n", i, N); + } + TestState state(seed + 1); + Env::Default()->Schedule(ConcurrentReader, &state); + state.Wait(TestState::RUNNING); + for (int i = 0; i < kSize; i++) { + state.t_.WriteStep(&rnd); + } + state.quit_flag_.Release_Store(&state); // Any non-NULL arg will do + state.Wait(TestState::DONE); + } +} + +TEST(SkipTest, Concurrent1) { RunConcurrent(1); } +TEST(SkipTest, Concurrent2) { RunConcurrent(2); } +TEST(SkipTest, Concurrent3) { RunConcurrent(3); } +TEST(SkipTest, Concurrent4) { RunConcurrent(4); } +TEST(SkipTest, Concurrent5) { RunConcurrent(5); } + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/db/snapshot.h b/src/leveldb/db/snapshot.h new file mode 100644 index 00000000..e7f8fd2c --- /dev/null +++ b/src/leveldb/db/snapshot.h @@ -0,0 +1,66 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_DB_SNAPSHOT_H_ +#define STORAGE_LEVELDB_DB_SNAPSHOT_H_ + +#include "leveldb/db.h" + +namespace leveldb { + +class SnapshotList; + +// Snapshots are kept in a doubly-linked list in the DB. +// Each SnapshotImpl corresponds to a particular sequence number. +class SnapshotImpl : public Snapshot { + public: + SequenceNumber number_; // const after creation + + private: + friend class SnapshotList; + + // SnapshotImpl is kept in a doubly-linked circular list + SnapshotImpl* prev_; + SnapshotImpl* next_; + + SnapshotList* list_; // just for sanity checks +}; + +class SnapshotList { + public: + SnapshotList() { + list_.prev_ = &list_; + list_.next_ = &list_; + } + + bool empty() const { return list_.next_ == &list_; } + SnapshotImpl* oldest() const { assert(!empty()); return list_.next_; } + SnapshotImpl* newest() const { assert(!empty()); return list_.prev_; } + + const SnapshotImpl* New(SequenceNumber seq) { + SnapshotImpl* s = new SnapshotImpl; + s->number_ = seq; + s->list_ = this; + s->next_ = &list_; + s->prev_ = list_.prev_; + s->prev_->next_ = s; + s->next_->prev_ = s; + return s; + } + + void Delete(const SnapshotImpl* s) { + assert(s->list_ == this); + s->prev_->next_ = s->next_; + s->next_->prev_ = s->prev_; + delete s; + } + + private: + // Dummy head of doubly-linked list of snapshots + SnapshotImpl list_; +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_SNAPSHOT_H_ diff --git a/src/leveldb/db/table_cache.cc b/src/leveldb/db/table_cache.cc new file mode 100644 index 00000000..e3d82cd3 --- /dev/null +++ b/src/leveldb/db/table_cache.cc @@ -0,0 +1,127 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/table_cache.h" + +#include "db/filename.h" +#include "leveldb/env.h" +#include "leveldb/table.h" +#include "util/coding.h" + +namespace leveldb { + +struct TableAndFile { + RandomAccessFile* file; + Table* table; +}; + +static void DeleteEntry(const Slice& key, void* value) { + TableAndFile* tf = reinterpret_cast(value); + delete tf->table; + delete tf->file; + delete tf; +} + +static void UnrefEntry(void* arg1, void* arg2) { + Cache* cache = reinterpret_cast(arg1); + Cache::Handle* h = reinterpret_cast(arg2); + cache->Release(h); +} + +TableCache::TableCache(const std::string& dbname, + const Options* options, + int entries) + : env_(options->env), + dbname_(dbname), + options_(options), + cache_(NewLRUCache(entries)) { +} + +TableCache::~TableCache() { + delete cache_; +} + +Status TableCache::FindTable(uint64_t file_number, uint64_t file_size, + Cache::Handle** handle) { + Status s; + char buf[sizeof(file_number)]; + EncodeFixed64(buf, file_number); + Slice key(buf, sizeof(buf)); + *handle = cache_->Lookup(key); + if (*handle == NULL) { + std::string fname = TableFileName(dbname_, file_number); + RandomAccessFile* file = NULL; + Table* table = NULL; + s = env_->NewRandomAccessFile(fname, &file); + if (!s.ok()) { + std::string old_fname = SSTTableFileName(dbname_, file_number); + if (env_->NewRandomAccessFile(old_fname, &file).ok()) { + s = Status::OK(); + } + } + if (s.ok()) { + s = Table::Open(*options_, file, file_size, &table); + } + + if (!s.ok()) { + assert(table == NULL); + delete file; + // We do not cache error results so that if the error is transient, + // or somebody repairs the file, we recover automatically. + } else { + TableAndFile* tf = new TableAndFile; + tf->file = file; + tf->table = table; + *handle = cache_->Insert(key, tf, 1, &DeleteEntry); + } + } + return s; +} + +Iterator* TableCache::NewIterator(const ReadOptions& options, + uint64_t file_number, + uint64_t file_size, + Table** tableptr) { + if (tableptr != NULL) { + *tableptr = NULL; + } + + Cache::Handle* handle = NULL; + Status s = FindTable(file_number, file_size, &handle); + if (!s.ok()) { + return NewErrorIterator(s); + } + + Table* table = reinterpret_cast(cache_->Value(handle))->table; + Iterator* result = table->NewIterator(options); + result->RegisterCleanup(&UnrefEntry, cache_, handle); + if (tableptr != NULL) { + *tableptr = table; + } + return result; +} + +Status TableCache::Get(const ReadOptions& options, + uint64_t file_number, + uint64_t file_size, + const Slice& k, + void* arg, + void (*saver)(void*, const Slice&, const Slice&)) { + Cache::Handle* handle = NULL; + Status s = FindTable(file_number, file_size, &handle); + if (s.ok()) { + Table* t = reinterpret_cast(cache_->Value(handle))->table; + s = t->InternalGet(options, k, arg, saver); + cache_->Release(handle); + } + return s; +} + +void TableCache::Evict(uint64_t file_number) { + char buf[sizeof(file_number)]; + EncodeFixed64(buf, file_number); + cache_->Erase(Slice(buf, sizeof(buf))); +} + +} // namespace leveldb diff --git a/src/leveldb/db/table_cache.h b/src/leveldb/db/table_cache.h new file mode 100644 index 00000000..8cf4aaf1 --- /dev/null +++ b/src/leveldb/db/table_cache.h @@ -0,0 +1,61 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// Thread-safe (provides internal synchronization) + +#ifndef STORAGE_LEVELDB_DB_TABLE_CACHE_H_ +#define STORAGE_LEVELDB_DB_TABLE_CACHE_H_ + +#include +#include +#include "db/dbformat.h" +#include "leveldb/cache.h" +#include "leveldb/table.h" +#include "port/port.h" + +namespace leveldb { + +class Env; + +class TableCache { + public: + TableCache(const std::string& dbname, const Options* options, int entries); + ~TableCache(); + + // Return an iterator for the specified file number (the corresponding + // file length must be exactly "file_size" bytes). If "tableptr" is + // non-NULL, also sets "*tableptr" to point to the Table object + // underlying the returned iterator, or NULL if no Table object underlies + // the returned iterator. The returned "*tableptr" object is owned by + // the cache and should not be deleted, and is valid for as long as the + // returned iterator is live. + Iterator* NewIterator(const ReadOptions& options, + uint64_t file_number, + uint64_t file_size, + Table** tableptr = NULL); + + // If a seek to internal key "k" in specified file finds an entry, + // call (*handle_result)(arg, found_key, found_value). + Status Get(const ReadOptions& options, + uint64_t file_number, + uint64_t file_size, + const Slice& k, + void* arg, + void (*handle_result)(void*, const Slice&, const Slice&)); + + // Evict any entry for the specified file number + void Evict(uint64_t file_number); + + private: + Env* const env_; + const std::string dbname_; + const Options* options_; + Cache* cache_; + + Status FindTable(uint64_t file_number, uint64_t file_size, Cache::Handle**); +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_TABLE_CACHE_H_ diff --git a/src/leveldb/db/version_edit.cc b/src/leveldb/db/version_edit.cc new file mode 100644 index 00000000..f10a2d58 --- /dev/null +++ b/src/leveldb/db/version_edit.cc @@ -0,0 +1,266 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/version_edit.h" + +#include "db/version_set.h" +#include "util/coding.h" + +namespace leveldb { + +// Tag numbers for serialized VersionEdit. These numbers are written to +// disk and should not be changed. +enum Tag { + kComparator = 1, + kLogNumber = 2, + kNextFileNumber = 3, + kLastSequence = 4, + kCompactPointer = 5, + kDeletedFile = 6, + kNewFile = 7, + // 8 was used for large value refs + kPrevLogNumber = 9 +}; + +void VersionEdit::Clear() { + comparator_.clear(); + log_number_ = 0; + prev_log_number_ = 0; + last_sequence_ = 0; + next_file_number_ = 0; + has_comparator_ = false; + has_log_number_ = false; + has_prev_log_number_ = false; + has_next_file_number_ = false; + has_last_sequence_ = false; + deleted_files_.clear(); + new_files_.clear(); +} + +void VersionEdit::EncodeTo(std::string* dst) const { + if (has_comparator_) { + PutVarint32(dst, kComparator); + PutLengthPrefixedSlice(dst, comparator_); + } + if (has_log_number_) { + PutVarint32(dst, kLogNumber); + PutVarint64(dst, log_number_); + } + if (has_prev_log_number_) { + PutVarint32(dst, kPrevLogNumber); + PutVarint64(dst, prev_log_number_); + } + if (has_next_file_number_) { + PutVarint32(dst, kNextFileNumber); + PutVarint64(dst, next_file_number_); + } + if (has_last_sequence_) { + PutVarint32(dst, kLastSequence); + PutVarint64(dst, last_sequence_); + } + + for (size_t i = 0; i < compact_pointers_.size(); i++) { + PutVarint32(dst, kCompactPointer); + PutVarint32(dst, compact_pointers_[i].first); // level + PutLengthPrefixedSlice(dst, compact_pointers_[i].second.Encode()); + } + + for (DeletedFileSet::const_iterator iter = deleted_files_.begin(); + iter != deleted_files_.end(); + ++iter) { + PutVarint32(dst, kDeletedFile); + PutVarint32(dst, iter->first); // level + PutVarint64(dst, iter->second); // file number + } + + for (size_t i = 0; i < new_files_.size(); i++) { + const FileMetaData& f = new_files_[i].second; + PutVarint32(dst, kNewFile); + PutVarint32(dst, new_files_[i].first); // level + PutVarint64(dst, f.number); + PutVarint64(dst, f.file_size); + PutLengthPrefixedSlice(dst, f.smallest.Encode()); + PutLengthPrefixedSlice(dst, f.largest.Encode()); + } +} + +static bool GetInternalKey(Slice* input, InternalKey* dst) { + Slice str; + if (GetLengthPrefixedSlice(input, &str)) { + dst->DecodeFrom(str); + return true; + } else { + return false; + } +} + +static bool GetLevel(Slice* input, int* level) { + uint32_t v; + if (GetVarint32(input, &v) && + v < config::kNumLevels) { + *level = v; + return true; + } else { + return false; + } +} + +Status VersionEdit::DecodeFrom(const Slice& src) { + Clear(); + Slice input = src; + const char* msg = NULL; + uint32_t tag; + + // Temporary storage for parsing + int level; + uint64_t number; + FileMetaData f; + Slice str; + InternalKey key; + + while (msg == NULL && GetVarint32(&input, &tag)) { + switch (tag) { + case kComparator: + if (GetLengthPrefixedSlice(&input, &str)) { + comparator_ = str.ToString(); + has_comparator_ = true; + } else { + msg = "comparator name"; + } + break; + + case kLogNumber: + if (GetVarint64(&input, &log_number_)) { + has_log_number_ = true; + } else { + msg = "log number"; + } + break; + + case kPrevLogNumber: + if (GetVarint64(&input, &prev_log_number_)) { + has_prev_log_number_ = true; + } else { + msg = "previous log number"; + } + break; + + case kNextFileNumber: + if (GetVarint64(&input, &next_file_number_)) { + has_next_file_number_ = true; + } else { + msg = "next file number"; + } + break; + + case kLastSequence: + if (GetVarint64(&input, &last_sequence_)) { + has_last_sequence_ = true; + } else { + msg = "last sequence number"; + } + break; + + case kCompactPointer: + if (GetLevel(&input, &level) && + GetInternalKey(&input, &key)) { + compact_pointers_.push_back(std::make_pair(level, key)); + } else { + msg = "compaction pointer"; + } + break; + + case kDeletedFile: + if (GetLevel(&input, &level) && + GetVarint64(&input, &number)) { + deleted_files_.insert(std::make_pair(level, number)); + } else { + msg = "deleted file"; + } + break; + + case kNewFile: + if (GetLevel(&input, &level) && + GetVarint64(&input, &f.number) && + GetVarint64(&input, &f.file_size) && + GetInternalKey(&input, &f.smallest) && + GetInternalKey(&input, &f.largest)) { + new_files_.push_back(std::make_pair(level, f)); + } else { + msg = "new-file entry"; + } + break; + + default: + msg = "unknown tag"; + break; + } + } + + if (msg == NULL && !input.empty()) { + msg = "invalid tag"; + } + + Status result; + if (msg != NULL) { + result = Status::Corruption("VersionEdit", msg); + } + return result; +} + +std::string VersionEdit::DebugString() const { + std::string r; + r.append("VersionEdit {"); + if (has_comparator_) { + r.append("\n Comparator: "); + r.append(comparator_); + } + if (has_log_number_) { + r.append("\n LogNumber: "); + AppendNumberTo(&r, log_number_); + } + if (has_prev_log_number_) { + r.append("\n PrevLogNumber: "); + AppendNumberTo(&r, prev_log_number_); + } + if (has_next_file_number_) { + r.append("\n NextFile: "); + AppendNumberTo(&r, next_file_number_); + } + if (has_last_sequence_) { + r.append("\n LastSeq: "); + AppendNumberTo(&r, last_sequence_); + } + for (size_t i = 0; i < compact_pointers_.size(); i++) { + r.append("\n CompactPointer: "); + AppendNumberTo(&r, compact_pointers_[i].first); + r.append(" "); + r.append(compact_pointers_[i].second.DebugString()); + } + for (DeletedFileSet::const_iterator iter = deleted_files_.begin(); + iter != deleted_files_.end(); + ++iter) { + r.append("\n DeleteFile: "); + AppendNumberTo(&r, iter->first); + r.append(" "); + AppendNumberTo(&r, iter->second); + } + for (size_t i = 0; i < new_files_.size(); i++) { + const FileMetaData& f = new_files_[i].second; + r.append("\n AddFile: "); + AppendNumberTo(&r, new_files_[i].first); + r.append(" "); + AppendNumberTo(&r, f.number); + r.append(" "); + AppendNumberTo(&r, f.file_size); + r.append(" "); + r.append(f.smallest.DebugString()); + r.append(" .. "); + r.append(f.largest.DebugString()); + } + r.append("\n}\n"); + return r; +} + +} // namespace leveldb diff --git a/src/leveldb/db/version_edit.h b/src/leveldb/db/version_edit.h new file mode 100644 index 00000000..eaef77b3 --- /dev/null +++ b/src/leveldb/db/version_edit.h @@ -0,0 +1,107 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_DB_VERSION_EDIT_H_ +#define STORAGE_LEVELDB_DB_VERSION_EDIT_H_ + +#include +#include +#include +#include "db/dbformat.h" + +namespace leveldb { + +class VersionSet; + +struct FileMetaData { + int refs; + int allowed_seeks; // Seeks allowed until compaction + uint64_t number; + uint64_t file_size; // File size in bytes + InternalKey smallest; // Smallest internal key served by table + InternalKey largest; // Largest internal key served by table + + FileMetaData() : refs(0), allowed_seeks(1 << 30), file_size(0) { } +}; + +class VersionEdit { + public: + VersionEdit() { Clear(); } + ~VersionEdit() { } + + void Clear(); + + void SetComparatorName(const Slice& name) { + has_comparator_ = true; + comparator_ = name.ToString(); + } + void SetLogNumber(uint64_t num) { + has_log_number_ = true; + log_number_ = num; + } + void SetPrevLogNumber(uint64_t num) { + has_prev_log_number_ = true; + prev_log_number_ = num; + } + void SetNextFile(uint64_t num) { + has_next_file_number_ = true; + next_file_number_ = num; + } + void SetLastSequence(SequenceNumber seq) { + has_last_sequence_ = true; + last_sequence_ = seq; + } + void SetCompactPointer(int level, const InternalKey& key) { + compact_pointers_.push_back(std::make_pair(level, key)); + } + + // Add the specified file at the specified number. + // REQUIRES: This version has not been saved (see VersionSet::SaveTo) + // REQUIRES: "smallest" and "largest" are smallest and largest keys in file + void AddFile(int level, uint64_t file, + uint64_t file_size, + const InternalKey& smallest, + const InternalKey& largest) { + FileMetaData f; + f.number = file; + f.file_size = file_size; + f.smallest = smallest; + f.largest = largest; + new_files_.push_back(std::make_pair(level, f)); + } + + // Delete the specified "file" from the specified "level". + void DeleteFile(int level, uint64_t file) { + deleted_files_.insert(std::make_pair(level, file)); + } + + void EncodeTo(std::string* dst) const; + Status DecodeFrom(const Slice& src); + + std::string DebugString() const; + + private: + friend class VersionSet; + + typedef std::set< std::pair > DeletedFileSet; + + std::string comparator_; + uint64_t log_number_; + uint64_t prev_log_number_; + uint64_t next_file_number_; + SequenceNumber last_sequence_; + bool has_comparator_; + bool has_log_number_; + bool has_prev_log_number_; + bool has_next_file_number_; + bool has_last_sequence_; + + std::vector< std::pair > compact_pointers_; + DeletedFileSet deleted_files_; + std::vector< std::pair > new_files_; +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_VERSION_EDIT_H_ diff --git a/src/leveldb/db/version_edit_test.cc b/src/leveldb/db/version_edit_test.cc new file mode 100644 index 00000000..280310b4 --- /dev/null +++ b/src/leveldb/db/version_edit_test.cc @@ -0,0 +1,46 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/version_edit.h" +#include "util/testharness.h" + +namespace leveldb { + +static void TestEncodeDecode(const VersionEdit& edit) { + std::string encoded, encoded2; + edit.EncodeTo(&encoded); + VersionEdit parsed; + Status s = parsed.DecodeFrom(encoded); + ASSERT_TRUE(s.ok()) << s.ToString(); + parsed.EncodeTo(&encoded2); + ASSERT_EQ(encoded, encoded2); +} + +class VersionEditTest { }; + +TEST(VersionEditTest, EncodeDecode) { + static const uint64_t kBig = 1ull << 50; + + VersionEdit edit; + for (int i = 0; i < 4; i++) { + TestEncodeDecode(edit); + edit.AddFile(3, kBig + 300 + i, kBig + 400 + i, + InternalKey("foo", kBig + 500 + i, kTypeValue), + InternalKey("zoo", kBig + 600 + i, kTypeDeletion)); + edit.DeleteFile(4, kBig + 700 + i); + edit.SetCompactPointer(i, InternalKey("x", kBig + 900 + i, kTypeValue)); + } + + edit.SetComparatorName("foo"); + edit.SetLogNumber(kBig + 100); + edit.SetNextFile(kBig + 200); + edit.SetLastSequence(kBig + 1000); + TestEncodeDecode(edit); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/db/version_set.cc b/src/leveldb/db/version_set.cc new file mode 100644 index 00000000..aa83df55 --- /dev/null +++ b/src/leveldb/db/version_set.cc @@ -0,0 +1,1484 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/version_set.h" + +#include +#include +#include "db/filename.h" +#include "db/log_reader.h" +#include "db/log_writer.h" +#include "db/memtable.h" +#include "db/table_cache.h" +#include "leveldb/env.h" +#include "leveldb/table_builder.h" +#include "table/merger.h" +#include "table/two_level_iterator.h" +#include "util/coding.h" +#include "util/logging.h" + +namespace leveldb { + +static const int kTargetFileSize = 2 * 1048576; + +// Maximum bytes of overlaps in grandparent (i.e., level+2) before we +// stop building a single file in a level->level+1 compaction. +static const int64_t kMaxGrandParentOverlapBytes = 10 * kTargetFileSize; + +// Maximum number of bytes in all compacted files. We avoid expanding +// the lower level file set of a compaction if it would make the +// total compaction cover more than this many bytes. +static const int64_t kExpandedCompactionByteSizeLimit = 25 * kTargetFileSize; + +static double MaxBytesForLevel(int level) { + // Note: the result for level zero is not really used since we set + // the level-0 compaction threshold based on number of files. + double result = 10 * 1048576.0; // Result for both level-0 and level-1 + while (level > 1) { + result *= 10; + level--; + } + return result; +} + +static uint64_t MaxFileSizeForLevel(int level) { + return kTargetFileSize; // We could vary per level to reduce number of files? +} + +static int64_t TotalFileSize(const std::vector& files) { + int64_t sum = 0; + for (size_t i = 0; i < files.size(); i++) { + sum += files[i]->file_size; + } + return sum; +} + +Version::~Version() { + assert(refs_ == 0); + + // Remove from linked list + prev_->next_ = next_; + next_->prev_ = prev_; + + // Drop references to files + for (int level = 0; level < config::kNumLevels; level++) { + for (size_t i = 0; i < files_[level].size(); i++) { + FileMetaData* f = files_[level][i]; + assert(f->refs > 0); + f->refs--; + if (f->refs <= 0) { + delete f; + } + } + } +} + +int FindFile(const InternalKeyComparator& icmp, + const std::vector& files, + const Slice& key) { + uint32_t left = 0; + uint32_t right = files.size(); + while (left < right) { + uint32_t mid = (left + right) / 2; + const FileMetaData* f = files[mid]; + if (icmp.InternalKeyComparator::Compare(f->largest.Encode(), key) < 0) { + // Key at "mid.largest" is < "target". Therefore all + // files at or before "mid" are uninteresting. + left = mid + 1; + } else { + // Key at "mid.largest" is >= "target". Therefore all files + // after "mid" are uninteresting. + right = mid; + } + } + return right; +} + +static bool AfterFile(const Comparator* ucmp, + const Slice* user_key, const FileMetaData* f) { + // NULL user_key occurs before all keys and is therefore never after *f + return (user_key != NULL && + ucmp->Compare(*user_key, f->largest.user_key()) > 0); +} + +static bool BeforeFile(const Comparator* ucmp, + const Slice* user_key, const FileMetaData* f) { + // NULL user_key occurs after all keys and is therefore never before *f + return (user_key != NULL && + ucmp->Compare(*user_key, f->smallest.user_key()) < 0); +} + +bool SomeFileOverlapsRange( + const InternalKeyComparator& icmp, + bool disjoint_sorted_files, + const std::vector& files, + const Slice* smallest_user_key, + const Slice* largest_user_key) { + const Comparator* ucmp = icmp.user_comparator(); + if (!disjoint_sorted_files) { + // Need to check against all files + for (size_t i = 0; i < files.size(); i++) { + const FileMetaData* f = files[i]; + if (AfterFile(ucmp, smallest_user_key, f) || + BeforeFile(ucmp, largest_user_key, f)) { + // No overlap + } else { + return true; // Overlap + } + } + return false; + } + + // Binary search over file list + uint32_t index = 0; + if (smallest_user_key != NULL) { + // Find the earliest possible internal key for smallest_user_key + InternalKey small(*smallest_user_key, kMaxSequenceNumber,kValueTypeForSeek); + index = FindFile(icmp, files, small.Encode()); + } + + if (index >= files.size()) { + // beginning of range is after all files, so no overlap. + return false; + } + + return !BeforeFile(ucmp, largest_user_key, files[index]); +} + +// An internal iterator. For a given version/level pair, yields +// information about the files in the level. For a given entry, key() +// is the largest key that occurs in the file, and value() is an +// 16-byte value containing the file number and file size, both +// encoded using EncodeFixed64. +class Version::LevelFileNumIterator : public Iterator { + public: + LevelFileNumIterator(const InternalKeyComparator& icmp, + const std::vector* flist) + : icmp_(icmp), + flist_(flist), + index_(flist->size()) { // Marks as invalid + } + virtual bool Valid() const { + return index_ < flist_->size(); + } + virtual void Seek(const Slice& target) { + index_ = FindFile(icmp_, *flist_, target); + } + virtual void SeekToFirst() { index_ = 0; } + virtual void SeekToLast() { + index_ = flist_->empty() ? 0 : flist_->size() - 1; + } + virtual void Next() { + assert(Valid()); + index_++; + } + virtual void Prev() { + assert(Valid()); + if (index_ == 0) { + index_ = flist_->size(); // Marks as invalid + } else { + index_--; + } + } + Slice key() const { + assert(Valid()); + return (*flist_)[index_]->largest.Encode(); + } + Slice value() const { + assert(Valid()); + EncodeFixed64(value_buf_, (*flist_)[index_]->number); + EncodeFixed64(value_buf_+8, (*flist_)[index_]->file_size); + return Slice(value_buf_, sizeof(value_buf_)); + } + virtual Status status() const { return Status::OK(); } + private: + const InternalKeyComparator icmp_; + const std::vector* const flist_; + uint32_t index_; + + // Backing store for value(). Holds the file number and size. + mutable char value_buf_[16]; +}; + +static Iterator* GetFileIterator(void* arg, + const ReadOptions& options, + const Slice& file_value) { + TableCache* cache = reinterpret_cast(arg); + if (file_value.size() != 16) { + return NewErrorIterator( + Status::Corruption("FileReader invoked with unexpected value")); + } else { + return cache->NewIterator(options, + DecodeFixed64(file_value.data()), + DecodeFixed64(file_value.data() + 8)); + } +} + +Iterator* Version::NewConcatenatingIterator(const ReadOptions& options, + int level) const { + return NewTwoLevelIterator( + new LevelFileNumIterator(vset_->icmp_, &files_[level]), + &GetFileIterator, vset_->table_cache_, options); +} + +void Version::AddIterators(const ReadOptions& options, + std::vector* iters) { + // Merge all level zero files together since they may overlap + for (size_t i = 0; i < files_[0].size(); i++) { + iters->push_back( + vset_->table_cache_->NewIterator( + options, files_[0][i]->number, files_[0][i]->file_size)); + } + + // For levels > 0, we can use a concatenating iterator that sequentially + // walks through the non-overlapping files in the level, opening them + // lazily. + for (int level = 1; level < config::kNumLevels; level++) { + if (!files_[level].empty()) { + iters->push_back(NewConcatenatingIterator(options, level)); + } + } +} + +// Callback from TableCache::Get() +namespace { +enum SaverState { + kNotFound, + kFound, + kDeleted, + kCorrupt, +}; +struct Saver { + SaverState state; + const Comparator* ucmp; + Slice user_key; + std::string* value; +}; +} +static void SaveValue(void* arg, const Slice& ikey, const Slice& v) { + Saver* s = reinterpret_cast(arg); + ParsedInternalKey parsed_key; + if (!ParseInternalKey(ikey, &parsed_key)) { + s->state = kCorrupt; + } else { + if (s->ucmp->Compare(parsed_key.user_key, s->user_key) == 0) { + s->state = (parsed_key.type == kTypeValue) ? kFound : kDeleted; + if (s->state == kFound) { + s->value->assign(v.data(), v.size()); + } + } + } +} + +static bool NewestFirst(FileMetaData* a, FileMetaData* b) { + return a->number > b->number; +} + +void Version::ForEachOverlapping(Slice user_key, Slice internal_key, + void* arg, + bool (*func)(void*, int, FileMetaData*)) { + // TODO(sanjay): Change Version::Get() to use this function. + const Comparator* ucmp = vset_->icmp_.user_comparator(); + + // Search level-0 in order from newest to oldest. + std::vector tmp; + tmp.reserve(files_[0].size()); + for (uint32_t i = 0; i < files_[0].size(); i++) { + FileMetaData* f = files_[0][i]; + if (ucmp->Compare(user_key, f->smallest.user_key()) >= 0 && + ucmp->Compare(user_key, f->largest.user_key()) <= 0) { + tmp.push_back(f); + } + } + if (!tmp.empty()) { + std::sort(tmp.begin(), tmp.end(), NewestFirst); + for (uint32_t i = 0; i < tmp.size(); i++) { + if (!(*func)(arg, 0, tmp[i])) { + return; + } + } + } + + // Search other levels. + for (int level = 1; level < config::kNumLevels; level++) { + size_t num_files = files_[level].size(); + if (num_files == 0) continue; + + // Binary search to find earliest index whose largest key >= internal_key. + uint32_t index = FindFile(vset_->icmp_, files_[level], internal_key); + if (index < num_files) { + FileMetaData* f = files_[level][index]; + if (ucmp->Compare(user_key, f->smallest.user_key()) < 0) { + // All of "f" is past any data for user_key + } else { + if (!(*func)(arg, level, f)) { + return; + } + } + } + } +} + +Status Version::Get(const ReadOptions& options, + const LookupKey& k, + std::string* value, + GetStats* stats) { + Slice ikey = k.internal_key(); + Slice user_key = k.user_key(); + const Comparator* ucmp = vset_->icmp_.user_comparator(); + Status s; + + stats->seek_file = NULL; + stats->seek_file_level = -1; + FileMetaData* last_file_read = NULL; + int last_file_read_level = -1; + + // We can search level-by-level since entries never hop across + // levels. Therefore we are guaranteed that if we find data + // in an smaller level, later levels are irrelevant. + std::vector tmp; + FileMetaData* tmp2; + for (int level = 0; level < config::kNumLevels; level++) { + size_t num_files = files_[level].size(); + if (num_files == 0) continue; + + // Get the list of files to search in this level + FileMetaData* const* files = &files_[level][0]; + if (level == 0) { + // Level-0 files may overlap each other. Find all files that + // overlap user_key and process them in order from newest to oldest. + tmp.reserve(num_files); + for (uint32_t i = 0; i < num_files; i++) { + FileMetaData* f = files[i]; + if (ucmp->Compare(user_key, f->smallest.user_key()) >= 0 && + ucmp->Compare(user_key, f->largest.user_key()) <= 0) { + tmp.push_back(f); + } + } + if (tmp.empty()) continue; + + std::sort(tmp.begin(), tmp.end(), NewestFirst); + files = &tmp[0]; + num_files = tmp.size(); + } else { + // Binary search to find earliest index whose largest key >= ikey. + uint32_t index = FindFile(vset_->icmp_, files_[level], ikey); + if (index >= num_files) { + files = NULL; + num_files = 0; + } else { + tmp2 = files[index]; + if (ucmp->Compare(user_key, tmp2->smallest.user_key()) < 0) { + // All of "tmp2" is past any data for user_key + files = NULL; + num_files = 0; + } else { + files = &tmp2; + num_files = 1; + } + } + } + + for (uint32_t i = 0; i < num_files; ++i) { + if (last_file_read != NULL && stats->seek_file == NULL) { + // We have had more than one seek for this read. Charge the 1st file. + stats->seek_file = last_file_read; + stats->seek_file_level = last_file_read_level; + } + + FileMetaData* f = files[i]; + last_file_read = f; + last_file_read_level = level; + + Saver saver; + saver.state = kNotFound; + saver.ucmp = ucmp; + saver.user_key = user_key; + saver.value = value; + s = vset_->table_cache_->Get(options, f->number, f->file_size, + ikey, &saver, SaveValue); + if (!s.ok()) { + return s; + } + switch (saver.state) { + case kNotFound: + break; // Keep searching in other files + case kFound: + return s; + case kDeleted: + s = Status::NotFound(Slice()); // Use empty error message for speed + return s; + case kCorrupt: + s = Status::Corruption("corrupted key for ", user_key); + return s; + } + } + } + + return Status::NotFound(Slice()); // Use an empty error message for speed +} + +bool Version::UpdateStats(const GetStats& stats) { + FileMetaData* f = stats.seek_file; + if (f != NULL) { + f->allowed_seeks--; + if (f->allowed_seeks <= 0 && file_to_compact_ == NULL) { + file_to_compact_ = f; + file_to_compact_level_ = stats.seek_file_level; + return true; + } + } + return false; +} + +bool Version::RecordReadSample(Slice internal_key) { + ParsedInternalKey ikey; + if (!ParseInternalKey(internal_key, &ikey)) { + return false; + } + + struct State { + GetStats stats; // Holds first matching file + int matches; + + static bool Match(void* arg, int level, FileMetaData* f) { + State* state = reinterpret_cast(arg); + state->matches++; + if (state->matches == 1) { + // Remember first match. + state->stats.seek_file = f; + state->stats.seek_file_level = level; + } + // We can stop iterating once we have a second match. + return state->matches < 2; + } + }; + + State state; + state.matches = 0; + ForEachOverlapping(ikey.user_key, internal_key, &state, &State::Match); + + // Must have at least two matches since we want to merge across + // files. But what if we have a single file that contains many + // overwrites and deletions? Should we have another mechanism for + // finding such files? + if (state.matches >= 2) { + // 1MB cost is about 1 seek (see comment in Builder::Apply). + return UpdateStats(state.stats); + } + return false; +} + +void Version::Ref() { + ++refs_; +} + +void Version::Unref() { + assert(this != &vset_->dummy_versions_); + assert(refs_ >= 1); + --refs_; + if (refs_ == 0) { + delete this; + } +} + +bool Version::OverlapInLevel(int level, + const Slice* smallest_user_key, + const Slice* largest_user_key) { + return SomeFileOverlapsRange(vset_->icmp_, (level > 0), files_[level], + smallest_user_key, largest_user_key); +} + +int Version::PickLevelForMemTableOutput( + const Slice& smallest_user_key, + const Slice& largest_user_key) { + int level = 0; + if (!OverlapInLevel(0, &smallest_user_key, &largest_user_key)) { + // Push to next level if there is no overlap in next level, + // and the #bytes overlapping in the level after that are limited. + InternalKey start(smallest_user_key, kMaxSequenceNumber, kValueTypeForSeek); + InternalKey limit(largest_user_key, 0, static_cast(0)); + std::vector overlaps; + while (level < config::kMaxMemCompactLevel) { + if (OverlapInLevel(level + 1, &smallest_user_key, &largest_user_key)) { + break; + } + if (level + 2 < config::kNumLevels) { + // Check that file does not overlap too many grandparent bytes. + GetOverlappingInputs(level + 2, &start, &limit, &overlaps); + const int64_t sum = TotalFileSize(overlaps); + if (sum > kMaxGrandParentOverlapBytes) { + break; + } + } + level++; + } + } + return level; +} + +// Store in "*inputs" all files in "level" that overlap [begin,end] +void Version::GetOverlappingInputs( + int level, + const InternalKey* begin, + const InternalKey* end, + std::vector* inputs) { + assert(level >= 0); + assert(level < config::kNumLevels); + inputs->clear(); + Slice user_begin, user_end; + if (begin != NULL) { + user_begin = begin->user_key(); + } + if (end != NULL) { + user_end = end->user_key(); + } + const Comparator* user_cmp = vset_->icmp_.user_comparator(); + for (size_t i = 0; i < files_[level].size(); ) { + FileMetaData* f = files_[level][i++]; + const Slice file_start = f->smallest.user_key(); + const Slice file_limit = f->largest.user_key(); + if (begin != NULL && user_cmp->Compare(file_limit, user_begin) < 0) { + // "f" is completely before specified range; skip it + } else if (end != NULL && user_cmp->Compare(file_start, user_end) > 0) { + // "f" is completely after specified range; skip it + } else { + inputs->push_back(f); + if (level == 0) { + // Level-0 files may overlap each other. So check if the newly + // added file has expanded the range. If so, restart search. + if (begin != NULL && user_cmp->Compare(file_start, user_begin) < 0) { + user_begin = file_start; + inputs->clear(); + i = 0; + } else if (end != NULL && user_cmp->Compare(file_limit, user_end) > 0) { + user_end = file_limit; + inputs->clear(); + i = 0; + } + } + } + } +} + +std::string Version::DebugString() const { + std::string r; + for (int level = 0; level < config::kNumLevels; level++) { + // E.g., + // --- level 1 --- + // 17:123['a' .. 'd'] + // 20:43['e' .. 'g'] + r.append("--- level "); + AppendNumberTo(&r, level); + r.append(" ---\n"); + const std::vector& files = files_[level]; + for (size_t i = 0; i < files.size(); i++) { + r.push_back(' '); + AppendNumberTo(&r, files[i]->number); + r.push_back(':'); + AppendNumberTo(&r, files[i]->file_size); + r.append("["); + r.append(files[i]->smallest.DebugString()); + r.append(" .. "); + r.append(files[i]->largest.DebugString()); + r.append("]\n"); + } + } + return r; +} + +// A helper class so we can efficiently apply a whole sequence +// of edits to a particular state without creating intermediate +// Versions that contain full copies of the intermediate state. +class VersionSet::Builder { + private: + // Helper to sort by v->files_[file_number].smallest + struct BySmallestKey { + const InternalKeyComparator* internal_comparator; + + bool operator()(FileMetaData* f1, FileMetaData* f2) const { + int r = internal_comparator->Compare(f1->smallest, f2->smallest); + if (r != 0) { + return (r < 0); + } else { + // Break ties by file number + return (f1->number < f2->number); + } + } + }; + + typedef std::set FileSet; + struct LevelState { + std::set deleted_files; + FileSet* added_files; + }; + + VersionSet* vset_; + Version* base_; + LevelState levels_[config::kNumLevels]; + + public: + // Initialize a builder with the files from *base and other info from *vset + Builder(VersionSet* vset, Version* base) + : vset_(vset), + base_(base) { + base_->Ref(); + BySmallestKey cmp; + cmp.internal_comparator = &vset_->icmp_; + for (int level = 0; level < config::kNumLevels; level++) { + levels_[level].added_files = new FileSet(cmp); + } + } + + ~Builder() { + for (int level = 0; level < config::kNumLevels; level++) { + const FileSet* added = levels_[level].added_files; + std::vector to_unref; + to_unref.reserve(added->size()); + for (FileSet::const_iterator it = added->begin(); + it != added->end(); ++it) { + to_unref.push_back(*it); + } + delete added; + for (uint32_t i = 0; i < to_unref.size(); i++) { + FileMetaData* f = to_unref[i]; + f->refs--; + if (f->refs <= 0) { + delete f; + } + } + } + base_->Unref(); + } + + // Apply all of the edits in *edit to the current state. + void Apply(VersionEdit* edit) { + // Update compaction pointers + for (size_t i = 0; i < edit->compact_pointers_.size(); i++) { + const int level = edit->compact_pointers_[i].first; + vset_->compact_pointer_[level] = + edit->compact_pointers_[i].second.Encode().ToString(); + } + + // Delete files + const VersionEdit::DeletedFileSet& del = edit->deleted_files_; + for (VersionEdit::DeletedFileSet::const_iterator iter = del.begin(); + iter != del.end(); + ++iter) { + const int level = iter->first; + const uint64_t number = iter->second; + levels_[level].deleted_files.insert(number); + } + + // Add new files + for (size_t i = 0; i < edit->new_files_.size(); i++) { + const int level = edit->new_files_[i].first; + FileMetaData* f = new FileMetaData(edit->new_files_[i].second); + f->refs = 1; + + // We arrange to automatically compact this file after + // a certain number of seeks. Let's assume: + // (1) One seek costs 10ms + // (2) Writing or reading 1MB costs 10ms (100MB/s) + // (3) A compaction of 1MB does 25MB of IO: + // 1MB read from this level + // 10-12MB read from next level (boundaries may be misaligned) + // 10-12MB written to next level + // This implies that 25 seeks cost the same as the compaction + // of 1MB of data. I.e., one seek costs approximately the + // same as the compaction of 40KB of data. We are a little + // conservative and allow approximately one seek for every 16KB + // of data before triggering a compaction. + f->allowed_seeks = (f->file_size / 16384); + if (f->allowed_seeks < 100) f->allowed_seeks = 100; + + levels_[level].deleted_files.erase(f->number); + levels_[level].added_files->insert(f); + } + } + + // Save the current state in *v. + void SaveTo(Version* v) { + BySmallestKey cmp; + cmp.internal_comparator = &vset_->icmp_; + for (int level = 0; level < config::kNumLevels; level++) { + // Merge the set of added files with the set of pre-existing files. + // Drop any deleted files. Store the result in *v. + const std::vector& base_files = base_->files_[level]; + std::vector::const_iterator base_iter = base_files.begin(); + std::vector::const_iterator base_end = base_files.end(); + const FileSet* added = levels_[level].added_files; + v->files_[level].reserve(base_files.size() + added->size()); + for (FileSet::const_iterator added_iter = added->begin(); + added_iter != added->end(); + ++added_iter) { + // Add all smaller files listed in base_ + for (std::vector::const_iterator bpos + = std::upper_bound(base_iter, base_end, *added_iter, cmp); + base_iter != bpos; + ++base_iter) { + MaybeAddFile(v, level, *base_iter); + } + + MaybeAddFile(v, level, *added_iter); + } + + // Add remaining base files + for (; base_iter != base_end; ++base_iter) { + MaybeAddFile(v, level, *base_iter); + } + +#ifndef NDEBUG + // Make sure there is no overlap in levels > 0 + if (level > 0) { + for (uint32_t i = 1; i < v->files_[level].size(); i++) { + const InternalKey& prev_end = v->files_[level][i-1]->largest; + const InternalKey& this_begin = v->files_[level][i]->smallest; + if (vset_->icmp_.Compare(prev_end, this_begin) >= 0) { + fprintf(stderr, "overlapping ranges in same level %s vs. %s\n", + prev_end.DebugString().c_str(), + this_begin.DebugString().c_str()); + abort(); + } + } + } +#endif + } + } + + void MaybeAddFile(Version* v, int level, FileMetaData* f) { + if (levels_[level].deleted_files.count(f->number) > 0) { + // File is deleted: do nothing + } else { + std::vector* files = &v->files_[level]; + if (level > 0 && !files->empty()) { + // Must not overlap + assert(vset_->icmp_.Compare((*files)[files->size()-1]->largest, + f->smallest) < 0); + } + f->refs++; + files->push_back(f); + } + } +}; + +VersionSet::VersionSet(const std::string& dbname, + const Options* options, + TableCache* table_cache, + const InternalKeyComparator* cmp) + : env_(options->env), + dbname_(dbname), + options_(options), + table_cache_(table_cache), + icmp_(*cmp), + next_file_number_(2), + manifest_file_number_(0), // Filled by Recover() + last_sequence_(0), + log_number_(0), + prev_log_number_(0), + descriptor_file_(NULL), + descriptor_log_(NULL), + dummy_versions_(this), + current_(NULL) { + AppendVersion(new Version(this)); +} + +VersionSet::~VersionSet() { + current_->Unref(); + assert(dummy_versions_.next_ == &dummy_versions_); // List must be empty + delete descriptor_log_; + delete descriptor_file_; +} + +void VersionSet::AppendVersion(Version* v) { + // Make "v" current + assert(v->refs_ == 0); + assert(v != current_); + if (current_ != NULL) { + current_->Unref(); + } + current_ = v; + v->Ref(); + + // Append to linked list + v->prev_ = dummy_versions_.prev_; + v->next_ = &dummy_versions_; + v->prev_->next_ = v; + v->next_->prev_ = v; +} + +Status VersionSet::LogAndApply(VersionEdit* edit, port::Mutex* mu) { + if (edit->has_log_number_) { + assert(edit->log_number_ >= log_number_); + assert(edit->log_number_ < next_file_number_); + } else { + edit->SetLogNumber(log_number_); + } + + if (!edit->has_prev_log_number_) { + edit->SetPrevLogNumber(prev_log_number_); + } + + edit->SetNextFile(next_file_number_); + edit->SetLastSequence(last_sequence_); + + Version* v = new Version(this); + { + Builder builder(this, current_); + builder.Apply(edit); + builder.SaveTo(v); + } + Finalize(v); + + // Initialize new descriptor log file if necessary by creating + // a temporary file that contains a snapshot of the current version. + std::string new_manifest_file; + Status s; + if (descriptor_log_ == NULL) { + // No reason to unlock *mu here since we only hit this path in the + // first call to LogAndApply (when opening the database). + assert(descriptor_file_ == NULL); + new_manifest_file = DescriptorFileName(dbname_, manifest_file_number_); + edit->SetNextFile(next_file_number_); + s = env_->NewWritableFile(new_manifest_file, &descriptor_file_); + if (s.ok()) { + descriptor_log_ = new log::Writer(descriptor_file_); + s = WriteSnapshot(descriptor_log_); + } + } + + // Unlock during expensive MANIFEST log write + { + mu->Unlock(); + + // Write new record to MANIFEST log + if (s.ok()) { + std::string record; + edit->EncodeTo(&record); + s = descriptor_log_->AddRecord(record); + if (s.ok()) { + s = descriptor_file_->Sync(); + } + if (!s.ok()) { + Log(options_->info_log, "MANIFEST write: %s\n", s.ToString().c_str()); + } + } + + // If we just created a new descriptor file, install it by writing a + // new CURRENT file that points to it. + if (s.ok() && !new_manifest_file.empty()) { + s = SetCurrentFile(env_, dbname_, manifest_file_number_); + } + + mu->Lock(); + } + + // Install the new version + if (s.ok()) { + AppendVersion(v); + log_number_ = edit->log_number_; + prev_log_number_ = edit->prev_log_number_; + } else { + delete v; + if (!new_manifest_file.empty()) { + delete descriptor_log_; + delete descriptor_file_; + descriptor_log_ = NULL; + descriptor_file_ = NULL; + env_->DeleteFile(new_manifest_file); + } + } + + return s; +} + +Status VersionSet::Recover() { + struct LogReporter : public log::Reader::Reporter { + Status* status; + virtual void Corruption(size_t bytes, const Status& s) { + if (this->status->ok()) *this->status = s; + } + }; + + // Read "CURRENT" file, which contains a pointer to the current manifest file + std::string current; + Status s = ReadFileToString(env_, CurrentFileName(dbname_), ¤t); + if (!s.ok()) { + return s; + } + if (current.empty() || current[current.size()-1] != '\n') { + return Status::Corruption("CURRENT file does not end with newline"); + } + current.resize(current.size() - 1); + + std::string dscname = dbname_ + "/" + current; + SequentialFile* file; + s = env_->NewSequentialFile(dscname, &file); + if (!s.ok()) { + return s; + } + + bool have_log_number = false; + bool have_prev_log_number = false; + bool have_next_file = false; + bool have_last_sequence = false; + uint64_t next_file = 0; + uint64_t last_sequence = 0; + uint64_t log_number = 0; + uint64_t prev_log_number = 0; + Builder builder(this, current_); + + { + LogReporter reporter; + reporter.status = &s; + log::Reader reader(file, &reporter, true/*checksum*/, 0/*initial_offset*/); + Slice record; + std::string scratch; + while (reader.ReadRecord(&record, &scratch) && s.ok()) { + VersionEdit edit; + s = edit.DecodeFrom(record); + if (s.ok()) { + if (edit.has_comparator_ && + edit.comparator_ != icmp_.user_comparator()->Name()) { + s = Status::InvalidArgument( + edit.comparator_ + " does not match existing comparator ", + icmp_.user_comparator()->Name()); + } + } + + if (s.ok()) { + builder.Apply(&edit); + } + + if (edit.has_log_number_) { + log_number = edit.log_number_; + have_log_number = true; + } + + if (edit.has_prev_log_number_) { + prev_log_number = edit.prev_log_number_; + have_prev_log_number = true; + } + + if (edit.has_next_file_number_) { + next_file = edit.next_file_number_; + have_next_file = true; + } + + if (edit.has_last_sequence_) { + last_sequence = edit.last_sequence_; + have_last_sequence = true; + } + } + } + delete file; + file = NULL; + + if (s.ok()) { + if (!have_next_file) { + s = Status::Corruption("no meta-nextfile entry in descriptor"); + } else if (!have_log_number) { + s = Status::Corruption("no meta-lognumber entry in descriptor"); + } else if (!have_last_sequence) { + s = Status::Corruption("no last-sequence-number entry in descriptor"); + } + + if (!have_prev_log_number) { + prev_log_number = 0; + } + + MarkFileNumberUsed(prev_log_number); + MarkFileNumberUsed(log_number); + } + + if (s.ok()) { + Version* v = new Version(this); + builder.SaveTo(v); + // Install recovered version + Finalize(v); + AppendVersion(v); + manifest_file_number_ = next_file; + next_file_number_ = next_file + 1; + last_sequence_ = last_sequence; + log_number_ = log_number; + prev_log_number_ = prev_log_number; + } + + return s; +} + +void VersionSet::MarkFileNumberUsed(uint64_t number) { + if (next_file_number_ <= number) { + next_file_number_ = number + 1; + } +} + +void VersionSet::Finalize(Version* v) { + // Precomputed best level for next compaction + int best_level = -1; + double best_score = -1; + + for (int level = 0; level < config::kNumLevels-1; level++) { + double score; + if (level == 0) { + // We treat level-0 specially by bounding the number of files + // instead of number of bytes for two reasons: + // + // (1) With larger write-buffer sizes, it is nice not to do too + // many level-0 compactions. + // + // (2) The files in level-0 are merged on every read and + // therefore we wish to avoid too many files when the individual + // file size is small (perhaps because of a small write-buffer + // setting, or very high compression ratios, or lots of + // overwrites/deletions). + score = v->files_[level].size() / + static_cast(config::kL0_CompactionTrigger); + } else { + // Compute the ratio of current size to size limit. + const uint64_t level_bytes = TotalFileSize(v->files_[level]); + score = static_cast(level_bytes) / MaxBytesForLevel(level); + } + + if (score > best_score) { + best_level = level; + best_score = score; + } + } + + v->compaction_level_ = best_level; + v->compaction_score_ = best_score; +} + +Status VersionSet::WriteSnapshot(log::Writer* log) { + // TODO: Break up into multiple records to reduce memory usage on recovery? + + // Save metadata + VersionEdit edit; + edit.SetComparatorName(icmp_.user_comparator()->Name()); + + // Save compaction pointers + for (int level = 0; level < config::kNumLevels; level++) { + if (!compact_pointer_[level].empty()) { + InternalKey key; + key.DecodeFrom(compact_pointer_[level]); + edit.SetCompactPointer(level, key); + } + } + + // Save files + for (int level = 0; level < config::kNumLevels; level++) { + const std::vector& files = current_->files_[level]; + for (size_t i = 0; i < files.size(); i++) { + const FileMetaData* f = files[i]; + edit.AddFile(level, f->number, f->file_size, f->smallest, f->largest); + } + } + + std::string record; + edit.EncodeTo(&record); + return log->AddRecord(record); +} + +int VersionSet::NumLevelFiles(int level) const { + assert(level >= 0); + assert(level < config::kNumLevels); + return current_->files_[level].size(); +} + +const char* VersionSet::LevelSummary(LevelSummaryStorage* scratch) const { + // Update code if kNumLevels changes + assert(config::kNumLevels == 7); + snprintf(scratch->buffer, sizeof(scratch->buffer), + "files[ %d %d %d %d %d %d %d ]", + int(current_->files_[0].size()), + int(current_->files_[1].size()), + int(current_->files_[2].size()), + int(current_->files_[3].size()), + int(current_->files_[4].size()), + int(current_->files_[5].size()), + int(current_->files_[6].size())); + return scratch->buffer; +} + +uint64_t VersionSet::ApproximateOffsetOf(Version* v, const InternalKey& ikey) { + uint64_t result = 0; + for (int level = 0; level < config::kNumLevels; level++) { + const std::vector& files = v->files_[level]; + for (size_t i = 0; i < files.size(); i++) { + if (icmp_.Compare(files[i]->largest, ikey) <= 0) { + // Entire file is before "ikey", so just add the file size + result += files[i]->file_size; + } else if (icmp_.Compare(files[i]->smallest, ikey) > 0) { + // Entire file is after "ikey", so ignore + if (level > 0) { + // Files other than level 0 are sorted by meta->smallest, so + // no further files in this level will contain data for + // "ikey". + break; + } + } else { + // "ikey" falls in the range for this table. Add the + // approximate offset of "ikey" within the table. + Table* tableptr; + Iterator* iter = table_cache_->NewIterator( + ReadOptions(), files[i]->number, files[i]->file_size, &tableptr); + if (tableptr != NULL) { + result += tableptr->ApproximateOffsetOf(ikey.Encode()); + } + delete iter; + } + } + } + return result; +} + +void VersionSet::AddLiveFiles(std::set* live) { + for (Version* v = dummy_versions_.next_; + v != &dummy_versions_; + v = v->next_) { + for (int level = 0; level < config::kNumLevels; level++) { + const std::vector& files = v->files_[level]; + for (size_t i = 0; i < files.size(); i++) { + live->insert(files[i]->number); + } + } + } +} + +int64_t VersionSet::NumLevelBytes(int level) const { + assert(level >= 0); + assert(level < config::kNumLevels); + return TotalFileSize(current_->files_[level]); +} + +int64_t VersionSet::MaxNextLevelOverlappingBytes() { + int64_t result = 0; + std::vector overlaps; + for (int level = 1; level < config::kNumLevels - 1; level++) { + for (size_t i = 0; i < current_->files_[level].size(); i++) { + const FileMetaData* f = current_->files_[level][i]; + current_->GetOverlappingInputs(level+1, &f->smallest, &f->largest, + &overlaps); + const int64_t sum = TotalFileSize(overlaps); + if (sum > result) { + result = sum; + } + } + } + return result; +} + +// Stores the minimal range that covers all entries in inputs in +// *smallest, *largest. +// REQUIRES: inputs is not empty +void VersionSet::GetRange(const std::vector& inputs, + InternalKey* smallest, + InternalKey* largest) { + assert(!inputs.empty()); + smallest->Clear(); + largest->Clear(); + for (size_t i = 0; i < inputs.size(); i++) { + FileMetaData* f = inputs[i]; + if (i == 0) { + *smallest = f->smallest; + *largest = f->largest; + } else { + if (icmp_.Compare(f->smallest, *smallest) < 0) { + *smallest = f->smallest; + } + if (icmp_.Compare(f->largest, *largest) > 0) { + *largest = f->largest; + } + } + } +} + +// Stores the minimal range that covers all entries in inputs1 and inputs2 +// in *smallest, *largest. +// REQUIRES: inputs is not empty +void VersionSet::GetRange2(const std::vector& inputs1, + const std::vector& inputs2, + InternalKey* smallest, + InternalKey* largest) { + std::vector all = inputs1; + all.insert(all.end(), inputs2.begin(), inputs2.end()); + GetRange(all, smallest, largest); +} + +Iterator* VersionSet::MakeInputIterator(Compaction* c) { + ReadOptions options; + options.verify_checksums = options_->paranoid_checks; + options.fill_cache = false; + + // Level-0 files have to be merged together. For other levels, + // we will make a concatenating iterator per level. + // TODO(opt): use concatenating iterator for level-0 if there is no overlap + const int space = (c->level() == 0 ? c->inputs_[0].size() + 1 : 2); + Iterator** list = new Iterator*[space]; + int num = 0; + for (int which = 0; which < 2; which++) { + if (!c->inputs_[which].empty()) { + if (c->level() + which == 0) { + const std::vector& files = c->inputs_[which]; + for (size_t i = 0; i < files.size(); i++) { + list[num++] = table_cache_->NewIterator( + options, files[i]->number, files[i]->file_size); + } + } else { + // Create concatenating iterator for the files from this level + list[num++] = NewTwoLevelIterator( + new Version::LevelFileNumIterator(icmp_, &c->inputs_[which]), + &GetFileIterator, table_cache_, options); + } + } + } + assert(num <= space); + Iterator* result = NewMergingIterator(&icmp_, list, num); + delete[] list; + return result; +} + +Compaction* VersionSet::PickCompaction() { + Compaction* c; + int level; + + // We prefer compactions triggered by too much data in a level over + // the compactions triggered by seeks. + const bool size_compaction = (current_->compaction_score_ >= 1); + const bool seek_compaction = (current_->file_to_compact_ != NULL); + if (size_compaction) { + level = current_->compaction_level_; + assert(level >= 0); + assert(level+1 < config::kNumLevels); + c = new Compaction(level); + + // Pick the first file that comes after compact_pointer_[level] + for (size_t i = 0; i < current_->files_[level].size(); i++) { + FileMetaData* f = current_->files_[level][i]; + if (compact_pointer_[level].empty() || + icmp_.Compare(f->largest.Encode(), compact_pointer_[level]) > 0) { + c->inputs_[0].push_back(f); + break; + } + } + if (c->inputs_[0].empty()) { + // Wrap-around to the beginning of the key space + c->inputs_[0].push_back(current_->files_[level][0]); + } + } else if (seek_compaction) { + level = current_->file_to_compact_level_; + c = new Compaction(level); + c->inputs_[0].push_back(current_->file_to_compact_); + } else { + return NULL; + } + + c->input_version_ = current_; + c->input_version_->Ref(); + + // Files in level 0 may overlap each other, so pick up all overlapping ones + if (level == 0) { + InternalKey smallest, largest; + GetRange(c->inputs_[0], &smallest, &largest); + // Note that the next call will discard the file we placed in + // c->inputs_[0] earlier and replace it with an overlapping set + // which will include the picked file. + current_->GetOverlappingInputs(0, &smallest, &largest, &c->inputs_[0]); + assert(!c->inputs_[0].empty()); + } + + SetupOtherInputs(c); + + return c; +} + +void VersionSet::SetupOtherInputs(Compaction* c) { + const int level = c->level(); + InternalKey smallest, largest; + GetRange(c->inputs_[0], &smallest, &largest); + + current_->GetOverlappingInputs(level+1, &smallest, &largest, &c->inputs_[1]); + + // Get entire range covered by compaction + InternalKey all_start, all_limit; + GetRange2(c->inputs_[0], c->inputs_[1], &all_start, &all_limit); + + // See if we can grow the number of inputs in "level" without + // changing the number of "level+1" files we pick up. + if (!c->inputs_[1].empty()) { + std::vector expanded0; + current_->GetOverlappingInputs(level, &all_start, &all_limit, &expanded0); + const int64_t inputs0_size = TotalFileSize(c->inputs_[0]); + const int64_t inputs1_size = TotalFileSize(c->inputs_[1]); + const int64_t expanded0_size = TotalFileSize(expanded0); + if (expanded0.size() > c->inputs_[0].size() && + inputs1_size + expanded0_size < kExpandedCompactionByteSizeLimit) { + InternalKey new_start, new_limit; + GetRange(expanded0, &new_start, &new_limit); + std::vector expanded1; + current_->GetOverlappingInputs(level+1, &new_start, &new_limit, + &expanded1); + if (expanded1.size() == c->inputs_[1].size()) { + Log(options_->info_log, + "Expanding@%d %d+%d (%ld+%ld bytes) to %d+%d (%ld+%ld bytes)\n", + level, + int(c->inputs_[0].size()), + int(c->inputs_[1].size()), + long(inputs0_size), long(inputs1_size), + int(expanded0.size()), + int(expanded1.size()), + long(expanded0_size), long(inputs1_size)); + smallest = new_start; + largest = new_limit; + c->inputs_[0] = expanded0; + c->inputs_[1] = expanded1; + GetRange2(c->inputs_[0], c->inputs_[1], &all_start, &all_limit); + } + } + } + + // Compute the set of grandparent files that overlap this compaction + // (parent == level+1; grandparent == level+2) + if (level + 2 < config::kNumLevels) { + current_->GetOverlappingInputs(level + 2, &all_start, &all_limit, + &c->grandparents_); + } + + if (false) { + Log(options_->info_log, "Compacting %d '%s' .. '%s'", + level, + smallest.DebugString().c_str(), + largest.DebugString().c_str()); + } + + // Update the place where we will do the next compaction for this level. + // We update this immediately instead of waiting for the VersionEdit + // to be applied so that if the compaction fails, we will try a different + // key range next time. + compact_pointer_[level] = largest.Encode().ToString(); + c->edit_.SetCompactPointer(level, largest); +} + +Compaction* VersionSet::CompactRange( + int level, + const InternalKey* begin, + const InternalKey* end) { + std::vector inputs; + current_->GetOverlappingInputs(level, begin, end, &inputs); + if (inputs.empty()) { + return NULL; + } + + // Avoid compacting too much in one shot in case the range is large. + // But we cannot do this for level-0 since level-0 files can overlap + // and we must not pick one file and drop another older file if the + // two files overlap. + if (level > 0) { + const uint64_t limit = MaxFileSizeForLevel(level); + uint64_t total = 0; + for (size_t i = 0; i < inputs.size(); i++) { + uint64_t s = inputs[i]->file_size; + total += s; + if (total >= limit) { + inputs.resize(i + 1); + break; + } + } + } + + Compaction* c = new Compaction(level); + c->input_version_ = current_; + c->input_version_->Ref(); + c->inputs_[0] = inputs; + SetupOtherInputs(c); + return c; +} + +Compaction::Compaction(int level) + : level_(level), + max_output_file_size_(MaxFileSizeForLevel(level)), + input_version_(NULL), + grandparent_index_(0), + seen_key_(false), + overlapped_bytes_(0) { + for (int i = 0; i < config::kNumLevels; i++) { + level_ptrs_[i] = 0; + } +} + +Compaction::~Compaction() { + if (input_version_ != NULL) { + input_version_->Unref(); + } +} + +bool Compaction::IsTrivialMove() const { + // Avoid a move if there is lots of overlapping grandparent data. + // Otherwise, the move could create a parent file that will require + // a very expensive merge later on. + return (num_input_files(0) == 1 && + num_input_files(1) == 0 && + TotalFileSize(grandparents_) <= kMaxGrandParentOverlapBytes); +} + +void Compaction::AddInputDeletions(VersionEdit* edit) { + for (int which = 0; which < 2; which++) { + for (size_t i = 0; i < inputs_[which].size(); i++) { + edit->DeleteFile(level_ + which, inputs_[which][i]->number); + } + } +} + +bool Compaction::IsBaseLevelForKey(const Slice& user_key) { + // Maybe use binary search to find right entry instead of linear search? + const Comparator* user_cmp = input_version_->vset_->icmp_.user_comparator(); + for (int lvl = level_ + 2; lvl < config::kNumLevels; lvl++) { + const std::vector& files = input_version_->files_[lvl]; + for (; level_ptrs_[lvl] < files.size(); ) { + FileMetaData* f = files[level_ptrs_[lvl]]; + if (user_cmp->Compare(user_key, f->largest.user_key()) <= 0) { + // We've advanced far enough + if (user_cmp->Compare(user_key, f->smallest.user_key()) >= 0) { + // Key falls in this file's range, so definitely not base level + return false; + } + break; + } + level_ptrs_[lvl]++; + } + } + return true; +} + +bool Compaction::ShouldStopBefore(const Slice& internal_key) { + // Scan to find earliest grandparent file that contains key. + const InternalKeyComparator* icmp = &input_version_->vset_->icmp_; + while (grandparent_index_ < grandparents_.size() && + icmp->Compare(internal_key, + grandparents_[grandparent_index_]->largest.Encode()) > 0) { + if (seen_key_) { + overlapped_bytes_ += grandparents_[grandparent_index_]->file_size; + } + grandparent_index_++; + } + seen_key_ = true; + + if (overlapped_bytes_ > kMaxGrandParentOverlapBytes) { + // Too much overlap for current output; start new output + overlapped_bytes_ = 0; + return true; + } else { + return false; + } +} + +void Compaction::ReleaseInputs() { + if (input_version_ != NULL) { + input_version_->Unref(); + input_version_ = NULL; + } +} + +} // namespace leveldb diff --git a/src/leveldb/db/version_set.h b/src/leveldb/db/version_set.h new file mode 100644 index 00000000..8dc14b8e --- /dev/null +++ b/src/leveldb/db/version_set.h @@ -0,0 +1,396 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// The representation of a DBImpl consists of a set of Versions. The +// newest version is called "current". Older versions may be kept +// around to provide a consistent view to live iterators. +// +// Each Version keeps track of a set of Table files per level. The +// entire set of versions is maintained in a VersionSet. +// +// Version,VersionSet are thread-compatible, but require external +// synchronization on all accesses. + +#ifndef STORAGE_LEVELDB_DB_VERSION_SET_H_ +#define STORAGE_LEVELDB_DB_VERSION_SET_H_ + +#include +#include +#include +#include "db/dbformat.h" +#include "db/version_edit.h" +#include "port/port.h" +#include "port/thread_annotations.h" + +namespace leveldb { + +namespace log { class Writer; } + +class Compaction; +class Iterator; +class MemTable; +class TableBuilder; +class TableCache; +class Version; +class VersionSet; +class WritableFile; + +// Return the smallest index i such that files[i]->largest >= key. +// Return files.size() if there is no such file. +// REQUIRES: "files" contains a sorted list of non-overlapping files. +extern int FindFile(const InternalKeyComparator& icmp, + const std::vector& files, + const Slice& key); + +// Returns true iff some file in "files" overlaps the user key range +// [*smallest,*largest]. +// smallest==NULL represents a key smaller than all keys in the DB. +// largest==NULL represents a key largest than all keys in the DB. +// REQUIRES: If disjoint_sorted_files, files[] contains disjoint ranges +// in sorted order. +extern bool SomeFileOverlapsRange( + const InternalKeyComparator& icmp, + bool disjoint_sorted_files, + const std::vector& files, + const Slice* smallest_user_key, + const Slice* largest_user_key); + +class Version { + public: + // Append to *iters a sequence of iterators that will + // yield the contents of this Version when merged together. + // REQUIRES: This version has been saved (see VersionSet::SaveTo) + void AddIterators(const ReadOptions&, std::vector* iters); + + // Lookup the value for key. If found, store it in *val and + // return OK. Else return a non-OK status. Fills *stats. + // REQUIRES: lock is not held + struct GetStats { + FileMetaData* seek_file; + int seek_file_level; + }; + Status Get(const ReadOptions&, const LookupKey& key, std::string* val, + GetStats* stats); + + // Adds "stats" into the current state. Returns true if a new + // compaction may need to be triggered, false otherwise. + // REQUIRES: lock is held + bool UpdateStats(const GetStats& stats); + + // Record a sample of bytes read at the specified internal key. + // Samples are taken approximately once every config::kReadBytesPeriod + // bytes. Returns true if a new compaction may need to be triggered. + // REQUIRES: lock is held + bool RecordReadSample(Slice key); + + // Reference count management (so Versions do not disappear out from + // under live iterators) + void Ref(); + void Unref(); + + void GetOverlappingInputs( + int level, + const InternalKey* begin, // NULL means before all keys + const InternalKey* end, // NULL means after all keys + std::vector* inputs); + + // Returns true iff some file in the specified level overlaps + // some part of [*smallest_user_key,*largest_user_key]. + // smallest_user_key==NULL represents a key smaller than all keys in the DB. + // largest_user_key==NULL represents a key largest than all keys in the DB. + bool OverlapInLevel(int level, + const Slice* smallest_user_key, + const Slice* largest_user_key); + + // Return the level at which we should place a new memtable compaction + // result that covers the range [smallest_user_key,largest_user_key]. + int PickLevelForMemTableOutput(const Slice& smallest_user_key, + const Slice& largest_user_key); + + int NumFiles(int level) const { return files_[level].size(); } + + // Return a human readable string that describes this version's contents. + std::string DebugString() const; + + private: + friend class Compaction; + friend class VersionSet; + + class LevelFileNumIterator; + Iterator* NewConcatenatingIterator(const ReadOptions&, int level) const; + + // Call func(arg, level, f) for every file that overlaps user_key in + // order from newest to oldest. If an invocation of func returns + // false, makes no more calls. + // + // REQUIRES: user portion of internal_key == user_key. + void ForEachOverlapping(Slice user_key, Slice internal_key, + void* arg, + bool (*func)(void*, int, FileMetaData*)); + + VersionSet* vset_; // VersionSet to which this Version belongs + Version* next_; // Next version in linked list + Version* prev_; // Previous version in linked list + int refs_; // Number of live refs to this version + + // List of files per level + std::vector files_[config::kNumLevels]; + + // Next file to compact based on seek stats. + FileMetaData* file_to_compact_; + int file_to_compact_level_; + + // Level that should be compacted next and its compaction score. + // Score < 1 means compaction is not strictly needed. These fields + // are initialized by Finalize(). + double compaction_score_; + int compaction_level_; + + explicit Version(VersionSet* vset) + : vset_(vset), next_(this), prev_(this), refs_(0), + file_to_compact_(NULL), + file_to_compact_level_(-1), + compaction_score_(-1), + compaction_level_(-1) { + } + + ~Version(); + + // No copying allowed + Version(const Version&); + void operator=(const Version&); +}; + +class VersionSet { + public: + VersionSet(const std::string& dbname, + const Options* options, + TableCache* table_cache, + const InternalKeyComparator*); + ~VersionSet(); + + // Apply *edit to the current version to form a new descriptor that + // is both saved to persistent state and installed as the new + // current version. Will release *mu while actually writing to the file. + // REQUIRES: *mu is held on entry. + // REQUIRES: no other thread concurrently calls LogAndApply() + Status LogAndApply(VersionEdit* edit, port::Mutex* mu) + EXCLUSIVE_LOCKS_REQUIRED(mu); + + // Recover the last saved descriptor from persistent storage. + Status Recover(); + + // Return the current version. + Version* current() const { return current_; } + + // Return the current manifest file number + uint64_t ManifestFileNumber() const { return manifest_file_number_; } + + // Allocate and return a new file number + uint64_t NewFileNumber() { return next_file_number_++; } + + // Arrange to reuse "file_number" unless a newer file number has + // already been allocated. + // REQUIRES: "file_number" was returned by a call to NewFileNumber(). + void ReuseFileNumber(uint64_t file_number) { + if (next_file_number_ == file_number + 1) { + next_file_number_ = file_number; + } + } + + // Return the number of Table files at the specified level. + int NumLevelFiles(int level) const; + + // Return the combined file size of all files at the specified level. + int64_t NumLevelBytes(int level) const; + + // Return the last sequence number. + uint64_t LastSequence() const { return last_sequence_; } + + // Set the last sequence number to s. + void SetLastSequence(uint64_t s) { + assert(s >= last_sequence_); + last_sequence_ = s; + } + + // Mark the specified file number as used. + void MarkFileNumberUsed(uint64_t number); + + // Return the current log file number. + uint64_t LogNumber() const { return log_number_; } + + // Return the log file number for the log file that is currently + // being compacted, or zero if there is no such log file. + uint64_t PrevLogNumber() const { return prev_log_number_; } + + // Pick level and inputs for a new compaction. + // Returns NULL if there is no compaction to be done. + // Otherwise returns a pointer to a heap-allocated object that + // describes the compaction. Caller should delete the result. + Compaction* PickCompaction(); + + // Return a compaction object for compacting the range [begin,end] in + // the specified level. Returns NULL if there is nothing in that + // level that overlaps the specified range. Caller should delete + // the result. + Compaction* CompactRange( + int level, + const InternalKey* begin, + const InternalKey* end); + + // Return the maximum overlapping data (in bytes) at next level for any + // file at a level >= 1. + int64_t MaxNextLevelOverlappingBytes(); + + // Create an iterator that reads over the compaction inputs for "*c". + // The caller should delete the iterator when no longer needed. + Iterator* MakeInputIterator(Compaction* c); + + // Returns true iff some level needs a compaction. + bool NeedsCompaction() const { + Version* v = current_; + return (v->compaction_score_ >= 1) || (v->file_to_compact_ != NULL); + } + + // Add all files listed in any live version to *live. + // May also mutate some internal state. + void AddLiveFiles(std::set* live); + + // Return the approximate offset in the database of the data for + // "key" as of version "v". + uint64_t ApproximateOffsetOf(Version* v, const InternalKey& key); + + // Return a human-readable short (single-line) summary of the number + // of files per level. Uses *scratch as backing store. + struct LevelSummaryStorage { + char buffer[100]; + }; + const char* LevelSummary(LevelSummaryStorage* scratch) const; + + private: + class Builder; + + friend class Compaction; + friend class Version; + + void Finalize(Version* v); + + void GetRange(const std::vector& inputs, + InternalKey* smallest, + InternalKey* largest); + + void GetRange2(const std::vector& inputs1, + const std::vector& inputs2, + InternalKey* smallest, + InternalKey* largest); + + void SetupOtherInputs(Compaction* c); + + // Save current contents to *log + Status WriteSnapshot(log::Writer* log); + + void AppendVersion(Version* v); + + Env* const env_; + const std::string dbname_; + const Options* const options_; + TableCache* const table_cache_; + const InternalKeyComparator icmp_; + uint64_t next_file_number_; + uint64_t manifest_file_number_; + uint64_t last_sequence_; + uint64_t log_number_; + uint64_t prev_log_number_; // 0 or backing store for memtable being compacted + + // Opened lazily + WritableFile* descriptor_file_; + log::Writer* descriptor_log_; + Version dummy_versions_; // Head of circular doubly-linked list of versions. + Version* current_; // == dummy_versions_.prev_ + + // Per-level key at which the next compaction at that level should start. + // Either an empty string, or a valid InternalKey. + std::string compact_pointer_[config::kNumLevels]; + + // No copying allowed + VersionSet(const VersionSet&); + void operator=(const VersionSet&); +}; + +// A Compaction encapsulates information about a compaction. +class Compaction { + public: + ~Compaction(); + + // Return the level that is being compacted. Inputs from "level" + // and "level+1" will be merged to produce a set of "level+1" files. + int level() const { return level_; } + + // Return the object that holds the edits to the descriptor done + // by this compaction. + VersionEdit* edit() { return &edit_; } + + // "which" must be either 0 or 1 + int num_input_files(int which) const { return inputs_[which].size(); } + + // Return the ith input file at "level()+which" ("which" must be 0 or 1). + FileMetaData* input(int which, int i) const { return inputs_[which][i]; } + + // Maximum size of files to build during this compaction. + uint64_t MaxOutputFileSize() const { return max_output_file_size_; } + + // Is this a trivial compaction that can be implemented by just + // moving a single input file to the next level (no merging or splitting) + bool IsTrivialMove() const; + + // Add all inputs to this compaction as delete operations to *edit. + void AddInputDeletions(VersionEdit* edit); + + // Returns true if the information we have available guarantees that + // the compaction is producing data in "level+1" for which no data exists + // in levels greater than "level+1". + bool IsBaseLevelForKey(const Slice& user_key); + + // Returns true iff we should stop building the current output + // before processing "internal_key". + bool ShouldStopBefore(const Slice& internal_key); + + // Release the input version for the compaction, once the compaction + // is successful. + void ReleaseInputs(); + + private: + friend class Version; + friend class VersionSet; + + explicit Compaction(int level); + + int level_; + uint64_t max_output_file_size_; + Version* input_version_; + VersionEdit edit_; + + // Each compaction reads inputs from "level_" and "level_+1" + std::vector inputs_[2]; // The two sets of inputs + + // State used to check for number of of overlapping grandparent files + // (parent == level_ + 1, grandparent == level_ + 2) + std::vector grandparents_; + size_t grandparent_index_; // Index in grandparent_starts_ + bool seen_key_; // Some output key has been seen + int64_t overlapped_bytes_; // Bytes of overlap between current output + // and grandparent files + + // State for implementing IsBaseLevelForKey + + // level_ptrs_ holds indices into input_version_->levels_: our state + // is that we are positioned at one of the file ranges for each + // higher level than the ones involved in this compaction (i.e. for + // all L >= level_ + 2). + size_t level_ptrs_[config::kNumLevels]; +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_DB_VERSION_SET_H_ diff --git a/src/leveldb/db/version_set_test.cc b/src/leveldb/db/version_set_test.cc new file mode 100644 index 00000000..501e34d1 --- /dev/null +++ b/src/leveldb/db/version_set_test.cc @@ -0,0 +1,179 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/version_set.h" +#include "util/logging.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace leveldb { + +class FindFileTest { + public: + std::vector files_; + bool disjoint_sorted_files_; + + FindFileTest() : disjoint_sorted_files_(true) { } + + ~FindFileTest() { + for (int i = 0; i < files_.size(); i++) { + delete files_[i]; + } + } + + void Add(const char* smallest, const char* largest, + SequenceNumber smallest_seq = 100, + SequenceNumber largest_seq = 100) { + FileMetaData* f = new FileMetaData; + f->number = files_.size() + 1; + f->smallest = InternalKey(smallest, smallest_seq, kTypeValue); + f->largest = InternalKey(largest, largest_seq, kTypeValue); + files_.push_back(f); + } + + int Find(const char* key) { + InternalKey target(key, 100, kTypeValue); + InternalKeyComparator cmp(BytewiseComparator()); + return FindFile(cmp, files_, target.Encode()); + } + + bool Overlaps(const char* smallest, const char* largest) { + InternalKeyComparator cmp(BytewiseComparator()); + Slice s(smallest != NULL ? smallest : ""); + Slice l(largest != NULL ? largest : ""); + return SomeFileOverlapsRange(cmp, disjoint_sorted_files_, files_, + (smallest != NULL ? &s : NULL), + (largest != NULL ? &l : NULL)); + } +}; + +TEST(FindFileTest, Empty) { + ASSERT_EQ(0, Find("foo")); + ASSERT_TRUE(! Overlaps("a", "z")); + ASSERT_TRUE(! Overlaps(NULL, "z")); + ASSERT_TRUE(! Overlaps("a", NULL)); + ASSERT_TRUE(! Overlaps(NULL, NULL)); +} + +TEST(FindFileTest, Single) { + Add("p", "q"); + ASSERT_EQ(0, Find("a")); + ASSERT_EQ(0, Find("p")); + ASSERT_EQ(0, Find("p1")); + ASSERT_EQ(0, Find("q")); + ASSERT_EQ(1, Find("q1")); + ASSERT_EQ(1, Find("z")); + + ASSERT_TRUE(! Overlaps("a", "b")); + ASSERT_TRUE(! Overlaps("z1", "z2")); + ASSERT_TRUE(Overlaps("a", "p")); + ASSERT_TRUE(Overlaps("a", "q")); + ASSERT_TRUE(Overlaps("a", "z")); + ASSERT_TRUE(Overlaps("p", "p1")); + ASSERT_TRUE(Overlaps("p", "q")); + ASSERT_TRUE(Overlaps("p", "z")); + ASSERT_TRUE(Overlaps("p1", "p2")); + ASSERT_TRUE(Overlaps("p1", "z")); + ASSERT_TRUE(Overlaps("q", "q")); + ASSERT_TRUE(Overlaps("q", "q1")); + + ASSERT_TRUE(! Overlaps(NULL, "j")); + ASSERT_TRUE(! Overlaps("r", NULL)); + ASSERT_TRUE(Overlaps(NULL, "p")); + ASSERT_TRUE(Overlaps(NULL, "p1")); + ASSERT_TRUE(Overlaps("q", NULL)); + ASSERT_TRUE(Overlaps(NULL, NULL)); +} + + +TEST(FindFileTest, Multiple) { + Add("150", "200"); + Add("200", "250"); + Add("300", "350"); + Add("400", "450"); + ASSERT_EQ(0, Find("100")); + ASSERT_EQ(0, Find("150")); + ASSERT_EQ(0, Find("151")); + ASSERT_EQ(0, Find("199")); + ASSERT_EQ(0, Find("200")); + ASSERT_EQ(1, Find("201")); + ASSERT_EQ(1, Find("249")); + ASSERT_EQ(1, Find("250")); + ASSERT_EQ(2, Find("251")); + ASSERT_EQ(2, Find("299")); + ASSERT_EQ(2, Find("300")); + ASSERT_EQ(2, Find("349")); + ASSERT_EQ(2, Find("350")); + ASSERT_EQ(3, Find("351")); + ASSERT_EQ(3, Find("400")); + ASSERT_EQ(3, Find("450")); + ASSERT_EQ(4, Find("451")); + + ASSERT_TRUE(! Overlaps("100", "149")); + ASSERT_TRUE(! Overlaps("251", "299")); + ASSERT_TRUE(! Overlaps("451", "500")); + ASSERT_TRUE(! Overlaps("351", "399")); + + ASSERT_TRUE(Overlaps("100", "150")); + ASSERT_TRUE(Overlaps("100", "200")); + ASSERT_TRUE(Overlaps("100", "300")); + ASSERT_TRUE(Overlaps("100", "400")); + ASSERT_TRUE(Overlaps("100", "500")); + ASSERT_TRUE(Overlaps("375", "400")); + ASSERT_TRUE(Overlaps("450", "450")); + ASSERT_TRUE(Overlaps("450", "500")); +} + +TEST(FindFileTest, MultipleNullBoundaries) { + Add("150", "200"); + Add("200", "250"); + Add("300", "350"); + Add("400", "450"); + ASSERT_TRUE(! Overlaps(NULL, "149")); + ASSERT_TRUE(! Overlaps("451", NULL)); + ASSERT_TRUE(Overlaps(NULL, NULL)); + ASSERT_TRUE(Overlaps(NULL, "150")); + ASSERT_TRUE(Overlaps(NULL, "199")); + ASSERT_TRUE(Overlaps(NULL, "200")); + ASSERT_TRUE(Overlaps(NULL, "201")); + ASSERT_TRUE(Overlaps(NULL, "400")); + ASSERT_TRUE(Overlaps(NULL, "800")); + ASSERT_TRUE(Overlaps("100", NULL)); + ASSERT_TRUE(Overlaps("200", NULL)); + ASSERT_TRUE(Overlaps("449", NULL)); + ASSERT_TRUE(Overlaps("450", NULL)); +} + +TEST(FindFileTest, OverlapSequenceChecks) { + Add("200", "200", 5000, 3000); + ASSERT_TRUE(! Overlaps("199", "199")); + ASSERT_TRUE(! Overlaps("201", "300")); + ASSERT_TRUE(Overlaps("200", "200")); + ASSERT_TRUE(Overlaps("190", "200")); + ASSERT_TRUE(Overlaps("200", "210")); +} + +TEST(FindFileTest, OverlappingFiles) { + Add("150", "600"); + Add("400", "500"); + disjoint_sorted_files_ = false; + ASSERT_TRUE(! Overlaps("100", "149")); + ASSERT_TRUE(! Overlaps("601", "700")); + ASSERT_TRUE(Overlaps("100", "150")); + ASSERT_TRUE(Overlaps("100", "200")); + ASSERT_TRUE(Overlaps("100", "300")); + ASSERT_TRUE(Overlaps("100", "400")); + ASSERT_TRUE(Overlaps("100", "500")); + ASSERT_TRUE(Overlaps("375", "400")); + ASSERT_TRUE(Overlaps("450", "450")); + ASSERT_TRUE(Overlaps("450", "500")); + ASSERT_TRUE(Overlaps("450", "700")); + ASSERT_TRUE(Overlaps("600", "700")); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/db/write_batch.cc b/src/leveldb/db/write_batch.cc new file mode 100644 index 00000000..33f4a425 --- /dev/null +++ b/src/leveldb/db/write_batch.cc @@ -0,0 +1,147 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// WriteBatch::rep_ := +// sequence: fixed64 +// count: fixed32 +// data: record[count] +// record := +// kTypeValue varstring varstring | +// kTypeDeletion varstring +// varstring := +// len: varint32 +// data: uint8[len] + +#include "leveldb/write_batch.h" + +#include "leveldb/db.h" +#include "db/dbformat.h" +#include "db/memtable.h" +#include "db/write_batch_internal.h" +#include "util/coding.h" + +namespace leveldb { + +// WriteBatch header has an 8-byte sequence number followed by a 4-byte count. +static const size_t kHeader = 12; + +WriteBatch::WriteBatch() { + Clear(); +} + +WriteBatch::~WriteBatch() { } + +WriteBatch::Handler::~Handler() { } + +void WriteBatch::Clear() { + rep_.clear(); + rep_.resize(kHeader); +} + +Status WriteBatch::Iterate(Handler* handler) const { + Slice input(rep_); + if (input.size() < kHeader) { + return Status::Corruption("malformed WriteBatch (too small)"); + } + + input.remove_prefix(kHeader); + Slice key, value; + int found = 0; + while (!input.empty()) { + found++; + char tag = input[0]; + input.remove_prefix(1); + switch (tag) { + case kTypeValue: + if (GetLengthPrefixedSlice(&input, &key) && + GetLengthPrefixedSlice(&input, &value)) { + handler->Put(key, value); + } else { + return Status::Corruption("bad WriteBatch Put"); + } + break; + case kTypeDeletion: + if (GetLengthPrefixedSlice(&input, &key)) { + handler->Delete(key); + } else { + return Status::Corruption("bad WriteBatch Delete"); + } + break; + default: + return Status::Corruption("unknown WriteBatch tag"); + } + } + if (found != WriteBatchInternal::Count(this)) { + return Status::Corruption("WriteBatch has wrong count"); + } else { + return Status::OK(); + } +} + +int WriteBatchInternal::Count(const WriteBatch* b) { + return DecodeFixed32(b->rep_.data() + 8); +} + +void WriteBatchInternal::SetCount(WriteBatch* b, int n) { + EncodeFixed32(&b->rep_[8], n); +} + +SequenceNumber WriteBatchInternal::Sequence(const WriteBatch* b) { + return SequenceNumber(DecodeFixed64(b->rep_.data())); +} + +void WriteBatchInternal::SetSequence(WriteBatch* b, SequenceNumber seq) { + EncodeFixed64(&b->rep_[0], seq); +} + +void WriteBatch::Put(const Slice& key, const Slice& value) { + WriteBatchInternal::SetCount(this, WriteBatchInternal::Count(this) + 1); + rep_.push_back(static_cast(kTypeValue)); + PutLengthPrefixedSlice(&rep_, key); + PutLengthPrefixedSlice(&rep_, value); +} + +void WriteBatch::Delete(const Slice& key) { + WriteBatchInternal::SetCount(this, WriteBatchInternal::Count(this) + 1); + rep_.push_back(static_cast(kTypeDeletion)); + PutLengthPrefixedSlice(&rep_, key); +} + +namespace { +class MemTableInserter : public WriteBatch::Handler { + public: + SequenceNumber sequence_; + MemTable* mem_; + + virtual void Put(const Slice& key, const Slice& value) { + mem_->Add(sequence_, kTypeValue, key, value); + sequence_++; + } + virtual void Delete(const Slice& key) { + mem_->Add(sequence_, kTypeDeletion, key, Slice()); + sequence_++; + } +}; +} // namespace + +Status WriteBatchInternal::InsertInto(const WriteBatch* b, + MemTable* memtable) { + MemTableInserter inserter; + inserter.sequence_ = WriteBatchInternal::Sequence(b); + inserter.mem_ = memtable; + return b->Iterate(&inserter); +} + +void WriteBatchInternal::SetContents(WriteBatch* b, const Slice& contents) { + assert(contents.size() >= kHeader); + b->rep_.assign(contents.data(), contents.size()); +} + +void WriteBatchInternal::Append(WriteBatch* dst, const WriteBatch* src) { + SetCount(dst, Count(dst) + Count(src)); + assert(src->rep_.size() >= kHeader); + dst->rep_.append(src->rep_.data() + kHeader, src->rep_.size() - kHeader); +} + +} // namespace leveldb diff --git a/src/leveldb/db/write_batch_internal.h b/src/leveldb/db/write_batch_internal.h new file mode 100644 index 00000000..310a3c89 --- /dev/null +++ b/src/leveldb/db/write_batch_internal.h @@ -0,0 +1,49 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_DB_WRITE_BATCH_INTERNAL_H_ +#define STORAGE_LEVELDB_DB_WRITE_BATCH_INTERNAL_H_ + +#include "leveldb/write_batch.h" + +namespace leveldb { + +class MemTable; + +// WriteBatchInternal provides static methods for manipulating a +// WriteBatch that we don't want in the public WriteBatch interface. +class WriteBatchInternal { + public: + // Return the number of entries in the batch. + static int Count(const WriteBatch* batch); + + // Set the count for the number of entries in the batch. + static void SetCount(WriteBatch* batch, int n); + + // Return the sequence number for the start of this batch. + static SequenceNumber Sequence(const WriteBatch* batch); + + // Store the specified number as the sequence number for the start of + // this batch. + static void SetSequence(WriteBatch* batch, SequenceNumber seq); + + static Slice Contents(const WriteBatch* batch) { + return Slice(batch->rep_); + } + + static size_t ByteSize(const WriteBatch* batch) { + return batch->rep_.size(); + } + + static void SetContents(WriteBatch* batch, const Slice& contents); + + static Status InsertInto(const WriteBatch* batch, MemTable* memtable); + + static void Append(WriteBatch* dst, const WriteBatch* src); +}; + +} // namespace leveldb + + +#endif // STORAGE_LEVELDB_DB_WRITE_BATCH_INTERNAL_H_ diff --git a/src/leveldb/db/write_batch_test.cc b/src/leveldb/db/write_batch_test.cc new file mode 100644 index 00000000..9064e3d8 --- /dev/null +++ b/src/leveldb/db/write_batch_test.cc @@ -0,0 +1,120 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/db.h" + +#include "db/memtable.h" +#include "db/write_batch_internal.h" +#include "leveldb/env.h" +#include "util/logging.h" +#include "util/testharness.h" + +namespace leveldb { + +static std::string PrintContents(WriteBatch* b) { + InternalKeyComparator cmp(BytewiseComparator()); + MemTable* mem = new MemTable(cmp); + mem->Ref(); + std::string state; + Status s = WriteBatchInternal::InsertInto(b, mem); + int count = 0; + Iterator* iter = mem->NewIterator(); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ParsedInternalKey ikey; + ASSERT_TRUE(ParseInternalKey(iter->key(), &ikey)); + switch (ikey.type) { + case kTypeValue: + state.append("Put("); + state.append(ikey.user_key.ToString()); + state.append(", "); + state.append(iter->value().ToString()); + state.append(")"); + count++; + break; + case kTypeDeletion: + state.append("Delete("); + state.append(ikey.user_key.ToString()); + state.append(")"); + count++; + break; + } + state.append("@"); + state.append(NumberToString(ikey.sequence)); + } + delete iter; + if (!s.ok()) { + state.append("ParseError()"); + } else if (count != WriteBatchInternal::Count(b)) { + state.append("CountMismatch()"); + } + mem->Unref(); + return state; +} + +class WriteBatchTest { }; + +TEST(WriteBatchTest, Empty) { + WriteBatch batch; + ASSERT_EQ("", PrintContents(&batch)); + ASSERT_EQ(0, WriteBatchInternal::Count(&batch)); +} + +TEST(WriteBatchTest, Multiple) { + WriteBatch batch; + batch.Put(Slice("foo"), Slice("bar")); + batch.Delete(Slice("box")); + batch.Put(Slice("baz"), Slice("boo")); + WriteBatchInternal::SetSequence(&batch, 100); + ASSERT_EQ(100, WriteBatchInternal::Sequence(&batch)); + ASSERT_EQ(3, WriteBatchInternal::Count(&batch)); + ASSERT_EQ("Put(baz, boo)@102" + "Delete(box)@101" + "Put(foo, bar)@100", + PrintContents(&batch)); +} + +TEST(WriteBatchTest, Corruption) { + WriteBatch batch; + batch.Put(Slice("foo"), Slice("bar")); + batch.Delete(Slice("box")); + WriteBatchInternal::SetSequence(&batch, 200); + Slice contents = WriteBatchInternal::Contents(&batch); + WriteBatchInternal::SetContents(&batch, + Slice(contents.data(),contents.size()-1)); + ASSERT_EQ("Put(foo, bar)@200" + "ParseError()", + PrintContents(&batch)); +} + +TEST(WriteBatchTest, Append) { + WriteBatch b1, b2; + WriteBatchInternal::SetSequence(&b1, 200); + WriteBatchInternal::SetSequence(&b2, 300); + WriteBatchInternal::Append(&b1, &b2); + ASSERT_EQ("", + PrintContents(&b1)); + b2.Put("a", "va"); + WriteBatchInternal::Append(&b1, &b2); + ASSERT_EQ("Put(a, va)@200", + PrintContents(&b1)); + b2.Clear(); + b2.Put("b", "vb"); + WriteBatchInternal::Append(&b1, &b2); + ASSERT_EQ("Put(a, va)@200" + "Put(b, vb)@201", + PrintContents(&b1)); + b2.Delete("foo"); + WriteBatchInternal::Append(&b1, &b2); + ASSERT_EQ("Put(a, va)@200" + "Put(b, vb)@202" + "Put(b, vb)@201" + "Delete(foo)@203", + PrintContents(&b1)); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/doc/bench/db_bench_sqlite3.cc b/src/leveldb/doc/bench/db_bench_sqlite3.cc new file mode 100644 index 00000000..e63aaa8d --- /dev/null +++ b/src/leveldb/doc/bench/db_bench_sqlite3.cc @@ -0,0 +1,718 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include +#include +#include "util/histogram.h" +#include "util/random.h" +#include "util/testutil.h" + +// Comma-separated list of operations to run in the specified order +// Actual benchmarks: +// +// fillseq -- write N values in sequential key order in async mode +// fillseqsync -- write N/100 values in sequential key order in sync mode +// fillseqbatch -- batch write N values in sequential key order in async mode +// fillrandom -- write N values in random key order in async mode +// fillrandsync -- write N/100 values in random key order in sync mode +// fillrandbatch -- batch write N values in sequential key order in async mode +// overwrite -- overwrite N values in random key order in async mode +// fillrand100K -- write N/1000 100K values in random order in async mode +// fillseq100K -- write N/1000 100K values in sequential order in async mode +// readseq -- read N times sequentially +// readrandom -- read N times in random order +// readrand100K -- read N/1000 100K values in sequential order in async mode +static const char* FLAGS_benchmarks = + "fillseq," + "fillseqsync," + "fillseqbatch," + "fillrandom," + "fillrandsync," + "fillrandbatch," + "overwrite," + "overwritebatch," + "readrandom," + "readseq," + "fillrand100K," + "fillseq100K," + "readseq," + "readrand100K," + ; + +// Number of key/values to place in database +static int FLAGS_num = 1000000; + +// Number of read operations to do. If negative, do FLAGS_num reads. +static int FLAGS_reads = -1; + +// Size of each value +static int FLAGS_value_size = 100; + +// Print histogram of operation timings +static bool FLAGS_histogram = false; + +// Arrange to generate values that shrink to this fraction of +// their original size after compression +static double FLAGS_compression_ratio = 0.5; + +// Page size. Default 1 KB. +static int FLAGS_page_size = 1024; + +// Number of pages. +// Default cache size = FLAGS_page_size * FLAGS_num_pages = 4 MB. +static int FLAGS_num_pages = 4096; + +// If true, do not destroy the existing database. If you set this +// flag and also specify a benchmark that wants a fresh database, that +// benchmark will fail. +static bool FLAGS_use_existing_db = false; + +// If true, we allow batch writes to occur +static bool FLAGS_transaction = true; + +// If true, we enable Write-Ahead Logging +static bool FLAGS_WAL_enabled = true; + +// Use the db with the following name. +static const char* FLAGS_db = NULL; + +inline +static void ExecErrorCheck(int status, char *err_msg) { + if (status != SQLITE_OK) { + fprintf(stderr, "SQL error: %s\n", err_msg); + sqlite3_free(err_msg); + exit(1); + } +} + +inline +static void StepErrorCheck(int status) { + if (status != SQLITE_DONE) { + fprintf(stderr, "SQL step error: status = %d\n", status); + exit(1); + } +} + +inline +static void ErrorCheck(int status) { + if (status != SQLITE_OK) { + fprintf(stderr, "sqlite3 error: status = %d\n", status); + exit(1); + } +} + +inline +static void WalCheckpoint(sqlite3* db_) { + // Flush all writes to disk + if (FLAGS_WAL_enabled) { + sqlite3_wal_checkpoint_v2(db_, NULL, SQLITE_CHECKPOINT_FULL, NULL, NULL); + } +} + +namespace leveldb { + +// Helper for quickly generating random data. +namespace { +class RandomGenerator { + private: + std::string data_; + int pos_; + + public: + RandomGenerator() { + // We use a limited amount of data over and over again and ensure + // that it is larger than the compression window (32KB), and also + // large enough to serve all typical value sizes we want to write. + Random rnd(301); + std::string piece; + while (data_.size() < 1048576) { + // Add a short fragment that is as compressible as specified + // by FLAGS_compression_ratio. + test::CompressibleString(&rnd, FLAGS_compression_ratio, 100, &piece); + data_.append(piece); + } + pos_ = 0; + } + + Slice Generate(int len) { + if (pos_ + len > data_.size()) { + pos_ = 0; + assert(len < data_.size()); + } + pos_ += len; + return Slice(data_.data() + pos_ - len, len); + } +}; + +static Slice TrimSpace(Slice s) { + int start = 0; + while (start < s.size() && isspace(s[start])) { + start++; + } + int limit = s.size(); + while (limit > start && isspace(s[limit-1])) { + limit--; + } + return Slice(s.data() + start, limit - start); +} + +} // namespace + +class Benchmark { + private: + sqlite3* db_; + int db_num_; + int num_; + int reads_; + double start_; + double last_op_finish_; + int64_t bytes_; + std::string message_; + Histogram hist_; + RandomGenerator gen_; + Random rand_; + + // State kept for progress messages + int done_; + int next_report_; // When to report next + + void PrintHeader() { + const int kKeySize = 16; + PrintEnvironment(); + fprintf(stdout, "Keys: %d bytes each\n", kKeySize); + fprintf(stdout, "Values: %d bytes each\n", FLAGS_value_size); + fprintf(stdout, "Entries: %d\n", num_); + fprintf(stdout, "RawSize: %.1f MB (estimated)\n", + ((static_cast(kKeySize + FLAGS_value_size) * num_) + / 1048576.0)); + PrintWarnings(); + fprintf(stdout, "------------------------------------------------\n"); + } + + void PrintWarnings() { +#if defined(__GNUC__) && !defined(__OPTIMIZE__) + fprintf(stdout, + "WARNING: Optimization is disabled: benchmarks unnecessarily slow\n" + ); +#endif +#ifndef NDEBUG + fprintf(stdout, + "WARNING: Assertions are enabled; benchmarks unnecessarily slow\n"); +#endif + } + + void PrintEnvironment() { + fprintf(stderr, "SQLite: version %s\n", SQLITE_VERSION); + +#if defined(__linux) + time_t now = time(NULL); + fprintf(stderr, "Date: %s", ctime(&now)); // ctime() adds newline + + FILE* cpuinfo = fopen("/proc/cpuinfo", "r"); + if (cpuinfo != NULL) { + char line[1000]; + int num_cpus = 0; + std::string cpu_type; + std::string cache_size; + while (fgets(line, sizeof(line), cpuinfo) != NULL) { + const char* sep = strchr(line, ':'); + if (sep == NULL) { + continue; + } + Slice key = TrimSpace(Slice(line, sep - 1 - line)); + Slice val = TrimSpace(Slice(sep + 1)); + if (key == "model name") { + ++num_cpus; + cpu_type = val.ToString(); + } else if (key == "cache size") { + cache_size = val.ToString(); + } + } + fclose(cpuinfo); + fprintf(stderr, "CPU: %d * %s\n", num_cpus, cpu_type.c_str()); + fprintf(stderr, "CPUCache: %s\n", cache_size.c_str()); + } +#endif + } + + void Start() { + start_ = Env::Default()->NowMicros() * 1e-6; + bytes_ = 0; + message_.clear(); + last_op_finish_ = start_; + hist_.Clear(); + done_ = 0; + next_report_ = 100; + } + + void FinishedSingleOp() { + if (FLAGS_histogram) { + double now = Env::Default()->NowMicros() * 1e-6; + double micros = (now - last_op_finish_) * 1e6; + hist_.Add(micros); + if (micros > 20000) { + fprintf(stderr, "long op: %.1f micros%30s\r", micros, ""); + fflush(stderr); + } + last_op_finish_ = now; + } + + done_++; + if (done_ >= next_report_) { + if (next_report_ < 1000) next_report_ += 100; + else if (next_report_ < 5000) next_report_ += 500; + else if (next_report_ < 10000) next_report_ += 1000; + else if (next_report_ < 50000) next_report_ += 5000; + else if (next_report_ < 100000) next_report_ += 10000; + else if (next_report_ < 500000) next_report_ += 50000; + else next_report_ += 100000; + fprintf(stderr, "... finished %d ops%30s\r", done_, ""); + fflush(stderr); + } + } + + void Stop(const Slice& name) { + double finish = Env::Default()->NowMicros() * 1e-6; + + // Pretend at least one op was done in case we are running a benchmark + // that does not call FinishedSingleOp(). + if (done_ < 1) done_ = 1; + + if (bytes_ > 0) { + char rate[100]; + snprintf(rate, sizeof(rate), "%6.1f MB/s", + (bytes_ / 1048576.0) / (finish - start_)); + if (!message_.empty()) { + message_ = std::string(rate) + " " + message_; + } else { + message_ = rate; + } + } + + fprintf(stdout, "%-12s : %11.3f micros/op;%s%s\n", + name.ToString().c_str(), + (finish - start_) * 1e6 / done_, + (message_.empty() ? "" : " "), + message_.c_str()); + if (FLAGS_histogram) { + fprintf(stdout, "Microseconds per op:\n%s\n", hist_.ToString().c_str()); + } + fflush(stdout); + } + + public: + enum Order { + SEQUENTIAL, + RANDOM + }; + enum DBState { + FRESH, + EXISTING + }; + + Benchmark() + : db_(NULL), + db_num_(0), + num_(FLAGS_num), + reads_(FLAGS_reads < 0 ? FLAGS_num : FLAGS_reads), + bytes_(0), + rand_(301) { + std::vector files; + std::string test_dir; + Env::Default()->GetTestDirectory(&test_dir); + Env::Default()->GetChildren(test_dir, &files); + if (!FLAGS_use_existing_db) { + for (int i = 0; i < files.size(); i++) { + if (Slice(files[i]).starts_with("dbbench_sqlite3")) { + std::string file_name(test_dir); + file_name += "/"; + file_name += files[i]; + Env::Default()->DeleteFile(file_name.c_str()); + } + } + } + } + + ~Benchmark() { + int status = sqlite3_close(db_); + ErrorCheck(status); + } + + void Run() { + PrintHeader(); + Open(); + + const char* benchmarks = FLAGS_benchmarks; + while (benchmarks != NULL) { + const char* sep = strchr(benchmarks, ','); + Slice name; + if (sep == NULL) { + name = benchmarks; + benchmarks = NULL; + } else { + name = Slice(benchmarks, sep - benchmarks); + benchmarks = sep + 1; + } + + bytes_ = 0; + Start(); + + bool known = true; + bool write_sync = false; + if (name == Slice("fillseq")) { + Write(write_sync, SEQUENTIAL, FRESH, num_, FLAGS_value_size, 1); + WalCheckpoint(db_); + } else if (name == Slice("fillseqbatch")) { + Write(write_sync, SEQUENTIAL, FRESH, num_, FLAGS_value_size, 1000); + WalCheckpoint(db_); + } else if (name == Slice("fillrandom")) { + Write(write_sync, RANDOM, FRESH, num_, FLAGS_value_size, 1); + WalCheckpoint(db_); + } else if (name == Slice("fillrandbatch")) { + Write(write_sync, RANDOM, FRESH, num_, FLAGS_value_size, 1000); + WalCheckpoint(db_); + } else if (name == Slice("overwrite")) { + Write(write_sync, RANDOM, EXISTING, num_, FLAGS_value_size, 1); + WalCheckpoint(db_); + } else if (name == Slice("overwritebatch")) { + Write(write_sync, RANDOM, EXISTING, num_, FLAGS_value_size, 1000); + WalCheckpoint(db_); + } else if (name == Slice("fillrandsync")) { + write_sync = true; + Write(write_sync, RANDOM, FRESH, num_ / 100, FLAGS_value_size, 1); + WalCheckpoint(db_); + } else if (name == Slice("fillseqsync")) { + write_sync = true; + Write(write_sync, SEQUENTIAL, FRESH, num_ / 100, FLAGS_value_size, 1); + WalCheckpoint(db_); + } else if (name == Slice("fillrand100K")) { + Write(write_sync, RANDOM, FRESH, num_ / 1000, 100 * 1000, 1); + WalCheckpoint(db_); + } else if (name == Slice("fillseq100K")) { + Write(write_sync, SEQUENTIAL, FRESH, num_ / 1000, 100 * 1000, 1); + WalCheckpoint(db_); + } else if (name == Slice("readseq")) { + ReadSequential(); + } else if (name == Slice("readrandom")) { + Read(RANDOM, 1); + } else if (name == Slice("readrand100K")) { + int n = reads_; + reads_ /= 1000; + Read(RANDOM, 1); + reads_ = n; + } else { + known = false; + if (name != Slice()) { // No error message for empty name + fprintf(stderr, "unknown benchmark '%s'\n", name.ToString().c_str()); + } + } + if (known) { + Stop(name); + } + } + } + + void Open() { + assert(db_ == NULL); + + int status; + char file_name[100]; + char* err_msg = NULL; + db_num_++; + + // Open database + std::string tmp_dir; + Env::Default()->GetTestDirectory(&tmp_dir); + snprintf(file_name, sizeof(file_name), + "%s/dbbench_sqlite3-%d.db", + tmp_dir.c_str(), + db_num_); + status = sqlite3_open(file_name, &db_); + if (status) { + fprintf(stderr, "open error: %s\n", sqlite3_errmsg(db_)); + exit(1); + } + + // Change SQLite cache size + char cache_size[100]; + snprintf(cache_size, sizeof(cache_size), "PRAGMA cache_size = %d", + FLAGS_num_pages); + status = sqlite3_exec(db_, cache_size, NULL, NULL, &err_msg); + ExecErrorCheck(status, err_msg); + + // FLAGS_page_size is defaulted to 1024 + if (FLAGS_page_size != 1024) { + char page_size[100]; + snprintf(page_size, sizeof(page_size), "PRAGMA page_size = %d", + FLAGS_page_size); + status = sqlite3_exec(db_, page_size, NULL, NULL, &err_msg); + ExecErrorCheck(status, err_msg); + } + + // Change journal mode to WAL if WAL enabled flag is on + if (FLAGS_WAL_enabled) { + std::string WAL_stmt = "PRAGMA journal_mode = WAL"; + + // LevelDB's default cache size is a combined 4 MB + std::string WAL_checkpoint = "PRAGMA wal_autocheckpoint = 4096"; + status = sqlite3_exec(db_, WAL_stmt.c_str(), NULL, NULL, &err_msg); + ExecErrorCheck(status, err_msg); + status = sqlite3_exec(db_, WAL_checkpoint.c_str(), NULL, NULL, &err_msg); + ExecErrorCheck(status, err_msg); + } + + // Change locking mode to exclusive and create tables/index for database + std::string locking_stmt = "PRAGMA locking_mode = EXCLUSIVE"; + std::string create_stmt = + "CREATE TABLE test (key blob, value blob, PRIMARY KEY(key))"; + std::string stmt_array[] = { locking_stmt, create_stmt }; + int stmt_array_length = sizeof(stmt_array) / sizeof(std::string); + for (int i = 0; i < stmt_array_length; i++) { + status = sqlite3_exec(db_, stmt_array[i].c_str(), NULL, NULL, &err_msg); + ExecErrorCheck(status, err_msg); + } + } + + void Write(bool write_sync, Order order, DBState state, + int num_entries, int value_size, int entries_per_batch) { + // Create new database if state == FRESH + if (state == FRESH) { + if (FLAGS_use_existing_db) { + message_ = "skipping (--use_existing_db is true)"; + return; + } + sqlite3_close(db_); + db_ = NULL; + Open(); + Start(); + } + + if (num_entries != num_) { + char msg[100]; + snprintf(msg, sizeof(msg), "(%d ops)", num_entries); + message_ = msg; + } + + char* err_msg = NULL; + int status; + + sqlite3_stmt *replace_stmt, *begin_trans_stmt, *end_trans_stmt; + std::string replace_str = "REPLACE INTO test (key, value) VALUES (?, ?)"; + std::string begin_trans_str = "BEGIN TRANSACTION;"; + std::string end_trans_str = "END TRANSACTION;"; + + // Check for synchronous flag in options + std::string sync_stmt = (write_sync) ? "PRAGMA synchronous = FULL" : + "PRAGMA synchronous = OFF"; + status = sqlite3_exec(db_, sync_stmt.c_str(), NULL, NULL, &err_msg); + ExecErrorCheck(status, err_msg); + + // Preparing sqlite3 statements + status = sqlite3_prepare_v2(db_, replace_str.c_str(), -1, + &replace_stmt, NULL); + ErrorCheck(status); + status = sqlite3_prepare_v2(db_, begin_trans_str.c_str(), -1, + &begin_trans_stmt, NULL); + ErrorCheck(status); + status = sqlite3_prepare_v2(db_, end_trans_str.c_str(), -1, + &end_trans_stmt, NULL); + ErrorCheck(status); + + bool transaction = (entries_per_batch > 1); + for (int i = 0; i < num_entries; i += entries_per_batch) { + // Begin write transaction + if (FLAGS_transaction && transaction) { + status = sqlite3_step(begin_trans_stmt); + StepErrorCheck(status); + status = sqlite3_reset(begin_trans_stmt); + ErrorCheck(status); + } + + // Create and execute SQL statements + for (int j = 0; j < entries_per_batch; j++) { + const char* value = gen_.Generate(value_size).data(); + + // Create values for key-value pair + const int k = (order == SEQUENTIAL) ? i + j : + (rand_.Next() % num_entries); + char key[100]; + snprintf(key, sizeof(key), "%016d", k); + + // Bind KV values into replace_stmt + status = sqlite3_bind_blob(replace_stmt, 1, key, 16, SQLITE_STATIC); + ErrorCheck(status); + status = sqlite3_bind_blob(replace_stmt, 2, value, + value_size, SQLITE_STATIC); + ErrorCheck(status); + + // Execute replace_stmt + bytes_ += value_size + strlen(key); + status = sqlite3_step(replace_stmt); + StepErrorCheck(status); + + // Reset SQLite statement for another use + status = sqlite3_clear_bindings(replace_stmt); + ErrorCheck(status); + status = sqlite3_reset(replace_stmt); + ErrorCheck(status); + + FinishedSingleOp(); + } + + // End write transaction + if (FLAGS_transaction && transaction) { + status = sqlite3_step(end_trans_stmt); + StepErrorCheck(status); + status = sqlite3_reset(end_trans_stmt); + ErrorCheck(status); + } + } + + status = sqlite3_finalize(replace_stmt); + ErrorCheck(status); + status = sqlite3_finalize(begin_trans_stmt); + ErrorCheck(status); + status = sqlite3_finalize(end_trans_stmt); + ErrorCheck(status); + } + + void Read(Order order, int entries_per_batch) { + int status; + sqlite3_stmt *read_stmt, *begin_trans_stmt, *end_trans_stmt; + + std::string read_str = "SELECT * FROM test WHERE key = ?"; + std::string begin_trans_str = "BEGIN TRANSACTION;"; + std::string end_trans_str = "END TRANSACTION;"; + + // Preparing sqlite3 statements + status = sqlite3_prepare_v2(db_, begin_trans_str.c_str(), -1, + &begin_trans_stmt, NULL); + ErrorCheck(status); + status = sqlite3_prepare_v2(db_, end_trans_str.c_str(), -1, + &end_trans_stmt, NULL); + ErrorCheck(status); + status = sqlite3_prepare_v2(db_, read_str.c_str(), -1, &read_stmt, NULL); + ErrorCheck(status); + + bool transaction = (entries_per_batch > 1); + for (int i = 0; i < reads_; i += entries_per_batch) { + // Begin read transaction + if (FLAGS_transaction && transaction) { + status = sqlite3_step(begin_trans_stmt); + StepErrorCheck(status); + status = sqlite3_reset(begin_trans_stmt); + ErrorCheck(status); + } + + // Create and execute SQL statements + for (int j = 0; j < entries_per_batch; j++) { + // Create key value + char key[100]; + int k = (order == SEQUENTIAL) ? i + j : (rand_.Next() % reads_); + snprintf(key, sizeof(key), "%016d", k); + + // Bind key value into read_stmt + status = sqlite3_bind_blob(read_stmt, 1, key, 16, SQLITE_STATIC); + ErrorCheck(status); + + // Execute read statement + while ((status = sqlite3_step(read_stmt)) == SQLITE_ROW) {} + StepErrorCheck(status); + + // Reset SQLite statement for another use + status = sqlite3_clear_bindings(read_stmt); + ErrorCheck(status); + status = sqlite3_reset(read_stmt); + ErrorCheck(status); + FinishedSingleOp(); + } + + // End read transaction + if (FLAGS_transaction && transaction) { + status = sqlite3_step(end_trans_stmt); + StepErrorCheck(status); + status = sqlite3_reset(end_trans_stmt); + ErrorCheck(status); + } + } + + status = sqlite3_finalize(read_stmt); + ErrorCheck(status); + status = sqlite3_finalize(begin_trans_stmt); + ErrorCheck(status); + status = sqlite3_finalize(end_trans_stmt); + ErrorCheck(status); + } + + void ReadSequential() { + int status; + sqlite3_stmt *pStmt; + std::string read_str = "SELECT * FROM test ORDER BY key"; + + status = sqlite3_prepare_v2(db_, read_str.c_str(), -1, &pStmt, NULL); + ErrorCheck(status); + for (int i = 0; i < reads_ && SQLITE_ROW == sqlite3_step(pStmt); i++) { + bytes_ += sqlite3_column_bytes(pStmt, 1) + sqlite3_column_bytes(pStmt, 2); + FinishedSingleOp(); + } + + status = sqlite3_finalize(pStmt); + ErrorCheck(status); + } + +}; + +} // namespace leveldb + +int main(int argc, char** argv) { + std::string default_db_path; + for (int i = 1; i < argc; i++) { + double d; + int n; + char junk; + if (leveldb::Slice(argv[i]).starts_with("--benchmarks=")) { + FLAGS_benchmarks = argv[i] + strlen("--benchmarks="); + } else if (sscanf(argv[i], "--histogram=%d%c", &n, &junk) == 1 && + (n == 0 || n == 1)) { + FLAGS_histogram = n; + } else if (sscanf(argv[i], "--compression_ratio=%lf%c", &d, &junk) == 1) { + FLAGS_compression_ratio = d; + } else if (sscanf(argv[i], "--use_existing_db=%d%c", &n, &junk) == 1 && + (n == 0 || n == 1)) { + FLAGS_use_existing_db = n; + } else if (sscanf(argv[i], "--num=%d%c", &n, &junk) == 1) { + FLAGS_num = n; + } else if (sscanf(argv[i], "--reads=%d%c", &n, &junk) == 1) { + FLAGS_reads = n; + } else if (sscanf(argv[i], "--value_size=%d%c", &n, &junk) == 1) { + FLAGS_value_size = n; + } else if (leveldb::Slice(argv[i]) == leveldb::Slice("--no_transaction")) { + FLAGS_transaction = false; + } else if (sscanf(argv[i], "--page_size=%d%c", &n, &junk) == 1) { + FLAGS_page_size = n; + } else if (sscanf(argv[i], "--num_pages=%d%c", &n, &junk) == 1) { + FLAGS_num_pages = n; + } else if (sscanf(argv[i], "--WAL_enabled=%d%c", &n, &junk) == 1 && + (n == 0 || n == 1)) { + FLAGS_WAL_enabled = n; + } else if (strncmp(argv[i], "--db=", 5) == 0) { + FLAGS_db = argv[i] + 5; + } else { + fprintf(stderr, "Invalid flag '%s'\n", argv[i]); + exit(1); + } + } + + // Choose a location for the test database if none given with --db= + if (FLAGS_db == NULL) { + leveldb::Env::Default()->GetTestDirectory(&default_db_path); + default_db_path += "/dbbench"; + FLAGS_db = default_db_path.c_str(); + } + + leveldb::Benchmark benchmark; + benchmark.Run(); + return 0; +} diff --git a/src/leveldb/doc/bench/db_bench_tree_db.cc b/src/leveldb/doc/bench/db_bench_tree_db.cc new file mode 100644 index 00000000..4ca381f1 --- /dev/null +++ b/src/leveldb/doc/bench/db_bench_tree_db.cc @@ -0,0 +1,528 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include +#include +#include "util/histogram.h" +#include "util/random.h" +#include "util/testutil.h" + +// Comma-separated list of operations to run in the specified order +// Actual benchmarks: +// +// fillseq -- write N values in sequential key order in async mode +// fillrandom -- write N values in random key order in async mode +// overwrite -- overwrite N values in random key order in async mode +// fillseqsync -- write N/100 values in sequential key order in sync mode +// fillrandsync -- write N/100 values in random key order in sync mode +// fillrand100K -- write N/1000 100K values in random order in async mode +// fillseq100K -- write N/1000 100K values in seq order in async mode +// readseq -- read N times sequentially +// readseq100K -- read N/1000 100K values in sequential order in async mode +// readrand100K -- read N/1000 100K values in sequential order in async mode +// readrandom -- read N times in random order +static const char* FLAGS_benchmarks = + "fillseq," + "fillseqsync," + "fillrandsync," + "fillrandom," + "overwrite," + "readrandom," + "readseq," + "fillrand100K," + "fillseq100K," + "readseq100K," + "readrand100K," + ; + +// Number of key/values to place in database +static int FLAGS_num = 1000000; + +// Number of read operations to do. If negative, do FLAGS_num reads. +static int FLAGS_reads = -1; + +// Size of each value +static int FLAGS_value_size = 100; + +// Arrange to generate values that shrink to this fraction of +// their original size after compression +static double FLAGS_compression_ratio = 0.5; + +// Print histogram of operation timings +static bool FLAGS_histogram = false; + +// Cache size. Default 4 MB +static int FLAGS_cache_size = 4194304; + +// Page size. Default 1 KB +static int FLAGS_page_size = 1024; + +// If true, do not destroy the existing database. If you set this +// flag and also specify a benchmark that wants a fresh database, that +// benchmark will fail. +static bool FLAGS_use_existing_db = false; + +// Compression flag. If true, compression is on. If false, compression +// is off. +static bool FLAGS_compression = true; + +// Use the db with the following name. +static const char* FLAGS_db = NULL; + +inline +static void DBSynchronize(kyotocabinet::TreeDB* db_) +{ + // Synchronize will flush writes to disk + if (!db_->synchronize()) { + fprintf(stderr, "synchronize error: %s\n", db_->error().name()); + } +} + +namespace leveldb { + +// Helper for quickly generating random data. +namespace { +class RandomGenerator { + private: + std::string data_; + int pos_; + + public: + RandomGenerator() { + // We use a limited amount of data over and over again and ensure + // that it is larger than the compression window (32KB), and also + // large enough to serve all typical value sizes we want to write. + Random rnd(301); + std::string piece; + while (data_.size() < 1048576) { + // Add a short fragment that is as compressible as specified + // by FLAGS_compression_ratio. + test::CompressibleString(&rnd, FLAGS_compression_ratio, 100, &piece); + data_.append(piece); + } + pos_ = 0; + } + + Slice Generate(int len) { + if (pos_ + len > data_.size()) { + pos_ = 0; + assert(len < data_.size()); + } + pos_ += len; + return Slice(data_.data() + pos_ - len, len); + } +}; + +static Slice TrimSpace(Slice s) { + int start = 0; + while (start < s.size() && isspace(s[start])) { + start++; + } + int limit = s.size(); + while (limit > start && isspace(s[limit-1])) { + limit--; + } + return Slice(s.data() + start, limit - start); +} + +} // namespace + +class Benchmark { + private: + kyotocabinet::TreeDB* db_; + int db_num_; + int num_; + int reads_; + double start_; + double last_op_finish_; + int64_t bytes_; + std::string message_; + Histogram hist_; + RandomGenerator gen_; + Random rand_; + kyotocabinet::LZOCompressor comp_; + + // State kept for progress messages + int done_; + int next_report_; // When to report next + + void PrintHeader() { + const int kKeySize = 16; + PrintEnvironment(); + fprintf(stdout, "Keys: %d bytes each\n", kKeySize); + fprintf(stdout, "Values: %d bytes each (%d bytes after compression)\n", + FLAGS_value_size, + static_cast(FLAGS_value_size * FLAGS_compression_ratio + 0.5)); + fprintf(stdout, "Entries: %d\n", num_); + fprintf(stdout, "RawSize: %.1f MB (estimated)\n", + ((static_cast(kKeySize + FLAGS_value_size) * num_) + / 1048576.0)); + fprintf(stdout, "FileSize: %.1f MB (estimated)\n", + (((kKeySize + FLAGS_value_size * FLAGS_compression_ratio) * num_) + / 1048576.0)); + PrintWarnings(); + fprintf(stdout, "------------------------------------------------\n"); + } + + void PrintWarnings() { +#if defined(__GNUC__) && !defined(__OPTIMIZE__) + fprintf(stdout, + "WARNING: Optimization is disabled: benchmarks unnecessarily slow\n" + ); +#endif +#ifndef NDEBUG + fprintf(stdout, + "WARNING: Assertions are enabled; benchmarks unnecessarily slow\n"); +#endif + } + + void PrintEnvironment() { + fprintf(stderr, "Kyoto Cabinet: version %s, lib ver %d, lib rev %d\n", + kyotocabinet::VERSION, kyotocabinet::LIBVER, kyotocabinet::LIBREV); + +#if defined(__linux) + time_t now = time(NULL); + fprintf(stderr, "Date: %s", ctime(&now)); // ctime() adds newline + + FILE* cpuinfo = fopen("/proc/cpuinfo", "r"); + if (cpuinfo != NULL) { + char line[1000]; + int num_cpus = 0; + std::string cpu_type; + std::string cache_size; + while (fgets(line, sizeof(line), cpuinfo) != NULL) { + const char* sep = strchr(line, ':'); + if (sep == NULL) { + continue; + } + Slice key = TrimSpace(Slice(line, sep - 1 - line)); + Slice val = TrimSpace(Slice(sep + 1)); + if (key == "model name") { + ++num_cpus; + cpu_type = val.ToString(); + } else if (key == "cache size") { + cache_size = val.ToString(); + } + } + fclose(cpuinfo); + fprintf(stderr, "CPU: %d * %s\n", num_cpus, cpu_type.c_str()); + fprintf(stderr, "CPUCache: %s\n", cache_size.c_str()); + } +#endif + } + + void Start() { + start_ = Env::Default()->NowMicros() * 1e-6; + bytes_ = 0; + message_.clear(); + last_op_finish_ = start_; + hist_.Clear(); + done_ = 0; + next_report_ = 100; + } + + void FinishedSingleOp() { + if (FLAGS_histogram) { + double now = Env::Default()->NowMicros() * 1e-6; + double micros = (now - last_op_finish_) * 1e6; + hist_.Add(micros); + if (micros > 20000) { + fprintf(stderr, "long op: %.1f micros%30s\r", micros, ""); + fflush(stderr); + } + last_op_finish_ = now; + } + + done_++; + if (done_ >= next_report_) { + if (next_report_ < 1000) next_report_ += 100; + else if (next_report_ < 5000) next_report_ += 500; + else if (next_report_ < 10000) next_report_ += 1000; + else if (next_report_ < 50000) next_report_ += 5000; + else if (next_report_ < 100000) next_report_ += 10000; + else if (next_report_ < 500000) next_report_ += 50000; + else next_report_ += 100000; + fprintf(stderr, "... finished %d ops%30s\r", done_, ""); + fflush(stderr); + } + } + + void Stop(const Slice& name) { + double finish = Env::Default()->NowMicros() * 1e-6; + + // Pretend at least one op was done in case we are running a benchmark + // that does not call FinishedSingleOp(). + if (done_ < 1) done_ = 1; + + if (bytes_ > 0) { + char rate[100]; + snprintf(rate, sizeof(rate), "%6.1f MB/s", + (bytes_ / 1048576.0) / (finish - start_)); + if (!message_.empty()) { + message_ = std::string(rate) + " " + message_; + } else { + message_ = rate; + } + } + + fprintf(stdout, "%-12s : %11.3f micros/op;%s%s\n", + name.ToString().c_str(), + (finish - start_) * 1e6 / done_, + (message_.empty() ? "" : " "), + message_.c_str()); + if (FLAGS_histogram) { + fprintf(stdout, "Microseconds per op:\n%s\n", hist_.ToString().c_str()); + } + fflush(stdout); + } + + public: + enum Order { + SEQUENTIAL, + RANDOM + }; + enum DBState { + FRESH, + EXISTING + }; + + Benchmark() + : db_(NULL), + num_(FLAGS_num), + reads_(FLAGS_reads < 0 ? FLAGS_num : FLAGS_reads), + bytes_(0), + rand_(301) { + std::vector files; + std::string test_dir; + Env::Default()->GetTestDirectory(&test_dir); + Env::Default()->GetChildren(test_dir.c_str(), &files); + if (!FLAGS_use_existing_db) { + for (int i = 0; i < files.size(); i++) { + if (Slice(files[i]).starts_with("dbbench_polyDB")) { + std::string file_name(test_dir); + file_name += "/"; + file_name += files[i]; + Env::Default()->DeleteFile(file_name.c_str()); + } + } + } + } + + ~Benchmark() { + if (!db_->close()) { + fprintf(stderr, "close error: %s\n", db_->error().name()); + } + } + + void Run() { + PrintHeader(); + Open(false); + + const char* benchmarks = FLAGS_benchmarks; + while (benchmarks != NULL) { + const char* sep = strchr(benchmarks, ','); + Slice name; + if (sep == NULL) { + name = benchmarks; + benchmarks = NULL; + } else { + name = Slice(benchmarks, sep - benchmarks); + benchmarks = sep + 1; + } + + Start(); + + bool known = true; + bool write_sync = false; + if (name == Slice("fillseq")) { + Write(write_sync, SEQUENTIAL, FRESH, num_, FLAGS_value_size, 1); + DBSynchronize(db_); + } else if (name == Slice("fillrandom")) { + Write(write_sync, RANDOM, FRESH, num_, FLAGS_value_size, 1); + DBSynchronize(db_); + } else if (name == Slice("overwrite")) { + Write(write_sync, RANDOM, EXISTING, num_, FLAGS_value_size, 1); + DBSynchronize(db_); + } else if (name == Slice("fillrandsync")) { + write_sync = true; + Write(write_sync, RANDOM, FRESH, num_ / 100, FLAGS_value_size, 1); + DBSynchronize(db_); + } else if (name == Slice("fillseqsync")) { + write_sync = true; + Write(write_sync, SEQUENTIAL, FRESH, num_ / 100, FLAGS_value_size, 1); + DBSynchronize(db_); + } else if (name == Slice("fillrand100K")) { + Write(write_sync, RANDOM, FRESH, num_ / 1000, 100 * 1000, 1); + DBSynchronize(db_); + } else if (name == Slice("fillseq100K")) { + Write(write_sync, SEQUENTIAL, FRESH, num_ / 1000, 100 * 1000, 1); + DBSynchronize(db_); + } else if (name == Slice("readseq")) { + ReadSequential(); + } else if (name == Slice("readrandom")) { + ReadRandom(); + } else if (name == Slice("readrand100K")) { + int n = reads_; + reads_ /= 1000; + ReadRandom(); + reads_ = n; + } else if (name == Slice("readseq100K")) { + int n = reads_; + reads_ /= 1000; + ReadSequential(); + reads_ = n; + } else { + known = false; + if (name != Slice()) { // No error message for empty name + fprintf(stderr, "unknown benchmark '%s'\n", name.ToString().c_str()); + } + } + if (known) { + Stop(name); + } + } + } + + private: + void Open(bool sync) { + assert(db_ == NULL); + + // Initialize db_ + db_ = new kyotocabinet::TreeDB(); + char file_name[100]; + db_num_++; + std::string test_dir; + Env::Default()->GetTestDirectory(&test_dir); + snprintf(file_name, sizeof(file_name), + "%s/dbbench_polyDB-%d.kct", + test_dir.c_str(), + db_num_); + + // Create tuning options and open the database + int open_options = kyotocabinet::PolyDB::OWRITER | + kyotocabinet::PolyDB::OCREATE; + int tune_options = kyotocabinet::TreeDB::TSMALL | + kyotocabinet::TreeDB::TLINEAR; + if (FLAGS_compression) { + tune_options |= kyotocabinet::TreeDB::TCOMPRESS; + db_->tune_compressor(&comp_); + } + db_->tune_options(tune_options); + db_->tune_page_cache(FLAGS_cache_size); + db_->tune_page(FLAGS_page_size); + db_->tune_map(256LL<<20); + if (sync) { + open_options |= kyotocabinet::PolyDB::OAUTOSYNC; + } + if (!db_->open(file_name, open_options)) { + fprintf(stderr, "open error: %s\n", db_->error().name()); + } + } + + void Write(bool sync, Order order, DBState state, + int num_entries, int value_size, int entries_per_batch) { + // Create new database if state == FRESH + if (state == FRESH) { + if (FLAGS_use_existing_db) { + message_ = "skipping (--use_existing_db is true)"; + return; + } + delete db_; + db_ = NULL; + Open(sync); + Start(); // Do not count time taken to destroy/open + } + + if (num_entries != num_) { + char msg[100]; + snprintf(msg, sizeof(msg), "(%d ops)", num_entries); + message_ = msg; + } + + // Write to database + for (int i = 0; i < num_entries; i++) + { + const int k = (order == SEQUENTIAL) ? i : (rand_.Next() % num_entries); + char key[100]; + snprintf(key, sizeof(key), "%016d", k); + bytes_ += value_size + strlen(key); + std::string cpp_key = key; + if (!db_->set(cpp_key, gen_.Generate(value_size).ToString())) { + fprintf(stderr, "set error: %s\n", db_->error().name()); + } + FinishedSingleOp(); + } + } + + void ReadSequential() { + kyotocabinet::DB::Cursor* cur = db_->cursor(); + cur->jump(); + std::string ckey, cvalue; + while (cur->get(&ckey, &cvalue, true)) { + bytes_ += ckey.size() + cvalue.size(); + FinishedSingleOp(); + } + delete cur; + } + + void ReadRandom() { + std::string value; + for (int i = 0; i < reads_; i++) { + char key[100]; + const int k = rand_.Next() % reads_; + snprintf(key, sizeof(key), "%016d", k); + db_->get(key, &value); + FinishedSingleOp(); + } + } +}; + +} // namespace leveldb + +int main(int argc, char** argv) { + std::string default_db_path; + for (int i = 1; i < argc; i++) { + double d; + int n; + char junk; + if (leveldb::Slice(argv[i]).starts_with("--benchmarks=")) { + FLAGS_benchmarks = argv[i] + strlen("--benchmarks="); + } else if (sscanf(argv[i], "--compression_ratio=%lf%c", &d, &junk) == 1) { + FLAGS_compression_ratio = d; + } else if (sscanf(argv[i], "--histogram=%d%c", &n, &junk) == 1 && + (n == 0 || n == 1)) { + FLAGS_histogram = n; + } else if (sscanf(argv[i], "--num=%d%c", &n, &junk) == 1) { + FLAGS_num = n; + } else if (sscanf(argv[i], "--reads=%d%c", &n, &junk) == 1) { + FLAGS_reads = n; + } else if (sscanf(argv[i], "--value_size=%d%c", &n, &junk) == 1) { + FLAGS_value_size = n; + } else if (sscanf(argv[i], "--cache_size=%d%c", &n, &junk) == 1) { + FLAGS_cache_size = n; + } else if (sscanf(argv[i], "--page_size=%d%c", &n, &junk) == 1) { + FLAGS_page_size = n; + } else if (sscanf(argv[i], "--compression=%d%c", &n, &junk) == 1 && + (n == 0 || n == 1)) { + FLAGS_compression = (n == 1) ? true : false; + } else if (strncmp(argv[i], "--db=", 5) == 0) { + FLAGS_db = argv[i] + 5; + } else { + fprintf(stderr, "Invalid flag '%s'\n", argv[i]); + exit(1); + } + } + + // Choose a location for the test database if none given with --db= + if (FLAGS_db == NULL) { + leveldb::Env::Default()->GetTestDirectory(&default_db_path); + default_db_path += "/dbbench"; + FLAGS_db = default_db_path.c_str(); + } + + leveldb::Benchmark benchmark; + benchmark.Run(); + return 0; +} diff --git a/src/leveldb/doc/benchmark.html b/src/leveldb/doc/benchmark.html new file mode 100644 index 00000000..c4639772 --- /dev/null +++ b/src/leveldb/doc/benchmark.html @@ -0,0 +1,459 @@ + + + +LevelDB Benchmarks + + + + +

LevelDB Benchmarks

+

Google, July 2011

+
+ +

In order to test LevelDB's performance, we benchmark it against other well-established database implementations. We compare LevelDB (revision 39) against SQLite3 (version 3.7.6.3) and Kyoto Cabinet's (version 1.2.67) TreeDB (a B+Tree based key-value store). We would like to acknowledge Scott Hess and Mikio Hirabayashi for their suggestions and contributions to the SQLite3 and Kyoto Cabinet benchmarks, respectively.

+ +

Benchmarks were all performed on a six-core Intel(R) Xeon(R) CPU X5650 @ 2.67GHz, with 12288 KB of total L3 cache and 12 GB of DDR3 RAM at 1333 MHz. (Note that LevelDB uses at most two CPUs since the benchmarks are single threaded: one to run the benchmark, and one for background compactions.) We ran the benchmarks on two machines (with identical processors), one with an Ext3 file system and one with an Ext4 file system. The machine with the Ext3 file system has a SATA Hitachi HDS721050CLA362 hard drive. The machine with the Ext4 file system has a SATA Samsung HD502HJ hard drive. Both hard drives spin at 7200 RPM and have hard drive write-caching enabled (using `hdparm -W 1 [device]`). The numbers reported below are the median of three measurements.

+ +

Benchmark Source Code

+

We wrote benchmark tools for SQLite and Kyoto TreeDB based on LevelDB's db_bench. The code for each of the benchmarks resides here:

+ + +

Custom Build Specifications

+
    +
  • LevelDB: LevelDB was compiled with the tcmalloc library and the Snappy compression library (revision 33). Assertions were disabled.
    • +
  • TreeDB: TreeDB was compiled using the LZO compression library (version 2.03). Furthermore, we enabled the TSMALL and TLINEAR options when opening the database in order to reduce the footprint of each record.
    • +
  • SQLite: We tuned SQLite's performance, by setting its locking mode to exclusive. We also enabled SQLite's write-ahead logging.
    • +
+ +

1. Baseline Performance

+

This section gives the baseline performance of all the +databases. Following sections show how performance changes as various +parameters are varied. For the baseline:

+
    +
  • Each database is allowed 4 MB of cache memory.
    • +
  • Databases are opened in asynchronous write mode. + (LevelDB's sync option, TreeDB's OAUTOSYNC option, and + SQLite3's synchronous options are all turned off). I.e., + every write is pushed to the operating system, but the + benchmark does not wait for the write to reach the disk.
    • +
  • Keys are 16 bytes each.
    • +
  • Value are 100 bytes each (with enough redundancy so that + a simple compressor shrinks them to 50% of their original + size).
    • +
  • Sequential reads/writes traverse the key space in increasing order.
    • +
  • Random reads/writes traverse the key space in random order.
    • +
+ +

A. Sequential Reads

+ + + + + + + + + + +
LevelDB4,030,000 ops/sec
 
Kyoto TreeDB1,010,000 ops/sec
 
SQLite3383,000 ops/sec
 
+

B. Random Reads

+ + + + + + + + + + +
LevelDB129,000 ops/sec
 
Kyoto TreeDB151,000 ops/sec
 
SQLite3134,000 ops/sec
 
+

C. Sequential Writes

+ + + + + + + + + + +
LevelDB779,000 ops/sec
 
Kyoto TreeDB342,000 ops/sec
 
SQLite348,600 ops/sec
 
+

D. Random Writes

+ + + + + + + + + + +
LevelDB164,000 ops/sec
 
Kyoto TreeDB88,500 ops/sec
 
SQLite39,860 ops/sec
 
+ +

LevelDB outperforms both SQLite3 and TreeDB in sequential and random write operations and sequential read operations. Kyoto Cabinet has the fastest random read operations.

+ +

2. Write Performance under Different Configurations

+

A. Large Values

+

For this benchmark, we start with an empty database, and write 100,000 byte values (~50% compressible). To keep the benchmark running time reasonable, we stop after writing 1000 values.

+

Sequential Writes

+ + + + + + + + + + +
LevelDB1,100 ops/sec
 
Kyoto TreeDB1,000 ops/sec
 
SQLite31,600 ops/sec
 
+

Random Writes

+ + + + + + + + + + +
LevelDB480 ops/sec
 
Kyoto TreeDB1,100 ops/sec
 
SQLite31,600 ops/sec
 
+

LevelDB doesn't perform as well with large values of 100,000 bytes each. This is because LevelDB writes keys and values at least twice: first time to the transaction log, and second time (during a compaction) to a sorted file. +With larger values, LevelDB's per-operation efficiency is swamped by the +cost of extra copies of large values.

+

B. Batch Writes

+

A batch write is a set of writes that are applied atomically to the underlying database. A single batch of N writes may be significantly faster than N individual writes. The following benchmark writes one thousand batches where each batch contains one thousand 100-byte values. TreeDB does not support batch writes and is omitted from this benchmark.

+

Sequential Writes

+ + + + + + + + + +
LevelDB840,000 entries/sec
 
(1.08x baseline)
SQLite3124,000 entries/sec
 
(2.55x baseline)
+

Random Writes

+ + + + + + + + + +
LevelDB221,000 entries/sec
 
(1.35x baseline)
SQLite322,000 entries/sec
 
(2.23x baseline)
+ +

Because of the way LevelDB persistent storage is organized, batches of +random writes are not much slower (only a factor of 4x) than batches +of sequential writes.

+ +

C. Synchronous Writes

+

In the following benchmark, we enable the synchronous writing modes +of all of the databases. Since this change significantly slows down the +benchmark, we stop after 10,000 writes. For synchronous write tests, we've +disabled hard drive write-caching (using `hdparm -W 0 [device]`).

+
    +
  • For LevelDB, we set WriteOptions.sync = true.
    • +
  • In TreeDB, we enabled TreeDB's OAUTOSYNC option.
    • +
  • For SQLite3, we set "PRAGMA synchronous = FULL".
    • +
+

Sequential Writes

+ + + + + + + + + + + + + +
LevelDB100 ops/sec
 
(0.003x baseline)
Kyoto TreeDB7 ops/sec
 
(0.0004x baseline)
SQLite388 ops/sec
 
(0.002x baseline)
+

Random Writes

+ + + + + + + + + + + + + +
LevelDB100 ops/sec
 
(0.015x baseline)
Kyoto TreeDB8 ops/sec
 
(0.001x baseline)
SQLite388 ops/sec
 
(0.009x baseline)
+ +

Also see the ext4 performance numbers below +since synchronous writes behave significantly differently +on ext3 and ext4.

+ +

D. Turning Compression Off

+ +

In the baseline measurements, LevelDB and TreeDB were using +light-weight compression +(Snappy for LevelDB, +and LZO for +TreeDB). SQLite3, by default does not use compression. The +experiments below show what happens when compression is disabled in +all of the databases (the SQLite3 numbers are just a copy of +its baseline measurements):

+ +

Sequential Writes

+ + + + + + + + + + + + + +
LevelDB594,000 ops/sec
 
(0.76x baseline)
Kyoto TreeDB485,000 ops/sec
 
(1.42x baseline)
SQLite348,600 ops/sec
 
(1.00x baseline)
+

Random Writes

+ + + + + + + + + + + + + +
LevelDB135,000 ops/sec
 
(0.82x baseline)
Kyoto TreeDB159,000 ops/sec
 
(1.80x baseline)
SQLite39,860 ops/sec
 
(1.00x baseline)
+ +

LevelDB's write performance is better with compression than without +since compression decreases the amount of data that has to be written +to disk. Therefore LevelDB users can leave compression enabled in +most scenarios without having worry about a tradeoff between space +usage and performance. TreeDB's performance on the other hand is +better without compression than with compression. Presumably this is +because TreeDB's compression library (LZO) is more expensive than +LevelDB's compression library (Snappy).

+ +

E. Using More Memory

+

We increased the overall cache size for each database to 128 MB. For LevelDB, we partitioned 128 MB into a 120 MB write buffer and 8 MB of cache (up from 2 MB of write buffer and 2 MB of cache). For SQLite3, we kept the page size at 1024 bytes, but increased the number of pages to 131,072 (up from 4096). For TreeDB, we also kept the page size at 1024 bytes, but increased the cache size to 128 MB (up from 4 MB).

+

Sequential Writes

+ + + + + + + + + + + + + +
LevelDB812,000 ops/sec
 
(1.04x baseline)
Kyoto TreeDB321,000 ops/sec
 
(0.94x baseline)
SQLite348,500 ops/sec
 
(1.00x baseline)
+

Random Writes

+ + + + + + + + + + + + + +
LevelDB355,000 ops/sec
 
(2.16x baseline)
Kyoto TreeDB284,000 ops/sec
 
(3.21x baseline)
SQLite39,670 ops/sec
 
(0.98x baseline)
+ +

SQLite's performance does not change substantially when compared to +the baseline, but the random write performance for both LevelDB and +TreeDB increases significantly. LevelDB's performance improves +because a larger write buffer reduces the need to merge sorted files +(since it creates a smaller number of larger sorted files). TreeDB's +performance goes up because the entire database is available in memory +for fast in-place updates.

+ +

3. Read Performance under Different Configurations

+

A. Larger Caches

+

We increased the overall memory usage to 128 MB for each database. +For LevelDB, we allocated 8 MB to LevelDB's write buffer and 120 MB +to LevelDB's cache. The other databases don't differentiate between a +write buffer and a cache, so we simply set their cache size to 128 +MB.

+

Sequential Reads

+ + + + + + + + + + + + + +
LevelDB5,210,000 ops/sec
 
(1.29x baseline)
Kyoto TreeDB1,070,000 ops/sec
 
(1.06x baseline)
SQLite3609,000 ops/sec
 
(1.59x baseline)
+ +

Random Reads

+ + + + + + + + + + + + + +
LevelDB190,000 ops/sec
 
(1.47x baseline)
Kyoto TreeDB463,000 ops/sec
 
(3.07x baseline)
SQLite3186,000 ops/sec
 
(1.39x baseline)
+ +

As expected, the read performance of all of the databases increases +when the caches are enlarged. In particular, TreeDB seems to make +very effective use of a cache that is large enough to hold the entire +database.

+ +

B. No Compression Reads

+

For this benchmark, we populated a database with 1 million entries consisting of 16 byte keys and 100 byte values. We compiled LevelDB and Kyoto Cabinet without compression support, so results that are read out from the database are already uncompressed. We've listed the SQLite3 baseline read performance as a point of comparison.

+

Sequential Reads

+ + + + + + + + + + + + + +
LevelDB4,880,000 ops/sec
 
(1.21x baseline)
Kyoto TreeDB1,230,000 ops/sec
 
(3.60x baseline)
SQLite3383,000 ops/sec
 
(1.00x baseline)
+

Random Reads

+ + + + + + + + + + + + + +
LevelDB149,000 ops/sec
 
(1.16x baseline)
Kyoto TreeDB175,000 ops/sec
 
(1.16x baseline)
SQLite3134,000 ops/sec
 
(1.00x baseline)
+ +

Performance of both LevelDB and TreeDB improves a small amount when +compression is disabled. Note however that under different workloads, +performance may very well be better with compression if it allows more +of the working set to fit in memory.

+ +

Note about Ext4 Filesystems

+

The preceding numbers are for an ext3 file system. Synchronous writes are much slower under ext4 (LevelDB drops to ~31 writes / second and TreeDB drops to ~5 writes / second; SQLite3's synchronous writes do not noticeably drop) due to ext4's different handling of fsync / msync calls. Even LevelDB's asynchronous write performance drops somewhat since it spreads its storage across multiple files and issues fsync calls when switching to a new file.

+ +

Acknowledgements

+

Jeff Dean and Sanjay Ghemawat wrote LevelDB. Kevin Tseng wrote and compiled these benchmarks. Mikio Hirabayashi, Scott Hess, and Gabor Cselle provided help and advice.

+ + diff --git a/src/leveldb/doc/doc.css b/src/leveldb/doc/doc.css new file mode 100644 index 00000000..700c564e --- /dev/null +++ b/src/leveldb/doc/doc.css @@ -0,0 +1,89 @@ +body { + margin-left: 0.5in; + margin-right: 0.5in; + background: white; + color: black; +} + +h1 { + margin-left: -0.2in; + font-size: 14pt; +} +h2 { + margin-left: -0in; + font-size: 12pt; +} +h3 { + margin-left: -0in; +} +h4 { + margin-left: -0in; +} +hr { + margin-left: -0in; +} + +/* Definition lists: definition term bold */ +dt { + font-weight: bold; +} + +address { + text-align: center; +} +code,samp,var { + color: blue; +} +kbd { + color: #600000; +} +div.note p { + float: right; + width: 3in; + margin-right: 0%; + padding: 1px; + border: 2px solid #6060a0; + background-color: #fffff0; +} + +ul { + margin-top: -0em; + margin-bottom: -0em; +} + +ol { + margin-top: -0em; + margin-bottom: -0em; +} + +UL.nobullets { + list-style-type: none; + list-style-image: none; + margin-left: -1em; +} + +p { + margin: 1em 0 1em 0; + padding: 0 0 0 0; +} + +pre { + line-height: 1.3em; + padding: 0.4em 0 0.8em 0; + margin: 0 0 0 0; + border: 0 0 0 0; + color: blue; +} + +.datatable { + margin-left: auto; + margin-right: auto; + margin-top: 2em; + margin-bottom: 2em; + border: 1px solid; +} + +.datatable td,th { + padding: 0 0.5em 0 0.5em; + text-align: right; +} diff --git a/src/leveldb/doc/impl.html b/src/leveldb/doc/impl.html new file mode 100644 index 00000000..6a468be0 --- /dev/null +++ b/src/leveldb/doc/impl.html @@ -0,0 +1,213 @@ + + + + +Leveldb file layout and compactions + + + + +

Files

+ +The implementation of leveldb is similar in spirit to the +representation of a single + +Bigtable tablet (section 5.3). +However the organization of the files that make up the representation +is somewhat different and is explained below. + +

+Each database is represented by a set of files stored in a directory. +There are several different types of files as documented below: +

+

Log files

+

+A log file (*.log) stores a sequence of recent updates. Each update +is appended to the current log file. When the log file reaches a +pre-determined size (approximately 4MB by default), it is converted +to a sorted table (see below) and a new log file is created for future +updates. +

+A copy of the current log file is kept in an in-memory structure (the +memtable). This copy is consulted on every read so that read +operations reflect all logged updates. +

+

Sorted tables

+

+A sorted table (*.sst) stores a sequence of entries sorted by key. +Each entry is either a value for the key, or a deletion marker for the +key. (Deletion markers are kept around to hide obsolete values +present in older sorted tables). +

+The set of sorted tables are organized into a sequence of levels. The +sorted table generated from a log file is placed in a special young +level (also called level-0). When the number of young files exceeds a +certain threshold (currently four), all of the young files are merged +together with all of the overlapping level-1 files to produce a +sequence of new level-1 files (we create a new level-1 file for every +2MB of data.) +

+Files in the young level may contain overlapping keys. However files +in other levels have distinct non-overlapping key ranges. Consider +level number L where L >= 1. When the combined size of files in +level-L exceeds (10^L) MB (i.e., 10MB for level-1, 100MB for level-2, +...), one file in level-L, and all of the overlapping files in +level-(L+1) are merged to form a set of new files for level-(L+1). +These merges have the effect of gradually migrating new updates from +the young level to the largest level using only bulk reads and writes +(i.e., minimizing expensive seeks). + +

Manifest

+

+A MANIFEST file lists the set of sorted tables that make up each +level, the corresponding key ranges, and other important metadata. +A new MANIFEST file (with a new number embedded in the file name) +is created whenever the database is reopened. The MANIFEST file is +formatted as a log, and changes made to the serving state (as files +are added or removed) are appended to this log. +

+

Current

+

+CURRENT is a simple text file that contains the name of the latest +MANIFEST file. +

+

Info logs

+

+Informational messages are printed to files named LOG and LOG.old. +

+

Others

+

+Other files used for miscellaneous purposes may also be present +(LOCK, *.dbtmp). + +

Level 0

+When the log file grows above a certain size (1MB by default): +
    +
  • Create a brand new memtable and log file and direct future updates here +
  • In the background: +
      +
    • Write the contents of the previous memtable to an sstable +
    • Discard the memtable +
    • Delete the old log file and the old memtable +
    • Add the new sstable to the young (level-0) level. +
    +
+ +

Compactions

+ +

+When the size of level L exceeds its limit, we compact it in a +background thread. The compaction picks a file from level L and all +overlapping files from the next level L+1. Note that if a level-L +file overlaps only part of a level-(L+1) file, the entire file at +level-(L+1) is used as an input to the compaction and will be +discarded after the compaction. Aside: because level-0 is special +(files in it may overlap each other), we treat compactions from +level-0 to level-1 specially: a level-0 compaction may pick more than +one level-0 file in case some of these files overlap each other. + +

+A compaction merges the contents of the picked files to produce a +sequence of level-(L+1) files. We switch to producing a new +level-(L+1) file after the current output file has reached the target +file size (2MB). We also switch to a new output file when the key +range of the current output file has grown enough to overlap more than +ten level-(L+2) files. This last rule ensures that a later compaction +of a level-(L+1) file will not pick up too much data from level-(L+2). + +

+The old files are discarded and the new files are added to the serving +state. + +

+Compactions for a particular level rotate through the key space. In +more detail, for each level L, we remember the ending key of the last +compaction at level L. The next compaction for level L will pick the +first file that starts after this key (wrapping around to the +beginning of the key space if there is no such file). + +

+Compactions drop overwritten values. They also drop deletion markers +if there are no higher numbered levels that contain a file whose range +overlaps the current key. + +

Timing

+ +Level-0 compactions will read up to four 1MB files from level-0, and +at worst all the level-1 files (10MB). I.e., we will read 14MB and +write 14MB. + +

+Other than the special level-0 compactions, we will pick one 2MB file +from level L. In the worst case, this will overlap ~ 12 files from +level L+1 (10 because level-(L+1) is ten times the size of level-L, +and another two at the boundaries since the file ranges at level-L +will usually not be aligned with the file ranges at level-L+1). The +compaction will therefore read 26MB and write 26MB. Assuming a disk +IO rate of 100MB/s (ballpark range for modern drives), the worst +compaction cost will be approximately 0.5 second. + +

+If we throttle the background writing to something small, say 10% of +the full 100MB/s speed, a compaction may take up to 5 seconds. If the +user is writing at 10MB/s, we might build up lots of level-0 files +(~50 to hold the 5*10MB). This may significantly increase the cost of +reads due to the overhead of merging more files together on every +read. + +

+Solution 1: To reduce this problem, we might want to increase the log +switching threshold when the number of level-0 files is large. Though +the downside is that the larger this threshold, the more memory we will +need to hold the corresponding memtable. + +

+Solution 2: We might want to decrease write rate artificially when the +number of level-0 files goes up. + +

+Solution 3: We work on reducing the cost of very wide merges. +Perhaps most of the level-0 files will have their blocks sitting +uncompressed in the cache and we will only need to worry about the +O(N) complexity in the merging iterator. + +

Number of files

+ +Instead of always making 2MB files, we could make larger files for +larger levels to reduce the total file count, though at the expense of +more bursty compactions. Alternatively, we could shard the set of +files into multiple directories. + +

+An experiment on an ext3 filesystem on Feb 04, 2011 shows +the following timings to do 100K file opens in directories with +varying number of files: + + + + + +
Files in directoryMicroseconds to open a file
10009
1000010
10000016
+So maybe even the sharding is not necessary on modern filesystems? + +

Recovery

+ +
    +
  • Read CURRENT to find name of the latest committed MANIFEST +
  • Read the named MANIFEST file +
  • Clean up stale files +
  • We could open all sstables here, but it is probably better to be lazy... +
  • Convert log chunk to a new level-0 sstable +
  • Start directing new writes to a new log file with recovered sequence# +
+ +

Garbage collection of files

+ +DeleteObsoleteFiles() is called at the end of every +compaction and at the end of recovery. It finds the names of all +files in the database. It deletes all log files that are not the +current log file. It deletes all table files that are not referenced +from some level and are not the output of an active compaction. + + + diff --git a/src/leveldb/doc/index.html b/src/leveldb/doc/index.html new file mode 100644 index 00000000..3ed0ed9d --- /dev/null +++ b/src/leveldb/doc/index.html @@ -0,0 +1,549 @@ + + + + +Leveldb + + + +

Leveldb

+
Jeff Dean, Sanjay Ghemawat
+

+The leveldb library provides a persistent key value store. Keys and +values are arbitrary byte arrays. The keys are ordered within the key +value store according to a user-specified comparator function. + +

+

Opening A Database

+

+A leveldb database has a name which corresponds to a file system +directory. All of the contents of database are stored in this +directory. The following example shows how to open a database, +creating it if necessary: +

+

+  #include <assert>
+  #include "leveldb/db.h"
+
+  leveldb::DB* db;
+  leveldb::Options options;
+  options.create_if_missing = true;
+  leveldb::Status status = leveldb::DB::Open(options, "/tmp/testdb", &db);
+  assert(status.ok());
+  ...
+
+If you want to raise an error if the database already exists, add +the following line before the leveldb::DB::Open call: +
+  options.error_if_exists = true;
+
+

Status

+

+You may have noticed the leveldb::Status type above. Values of this +type are returned by most functions in leveldb that may encounter an +error. You can check if such a result is ok, and also print an +associated error message: +

+

+   leveldb::Status s = ...;
+   if (!s.ok()) cerr << s.ToString() << endl;
+
+

Closing A Database

+

+When you are done with a database, just delete the database object. +Example: +

+

+  ... open the db as described above ...
+  ... do something with db ...
+  delete db;
+
+

Reads And Writes

+

+The database provides Put, Delete, and Get methods to +modify/query the database. For example, the following code +moves the value stored under key1 to key2. +

+  std::string value;
+  leveldb::Status s = db->Get(leveldb::ReadOptions(), key1, &value);
+  if (s.ok()) s = db->Put(leveldb::WriteOptions(), key2, value);
+  if (s.ok()) s = db->Delete(leveldb::WriteOptions(), key1);
+
+ +

Atomic Updates

+

+Note that if the process dies after the Put of key2 but before the +delete of key1, the same value may be left stored under multiple keys. +Such problems can be avoided by using the WriteBatch class to +atomically apply a set of updates: +

+

+  #include "leveldb/write_batch.h"
+  ...
+  std::string value;
+  leveldb::Status s = db->Get(leveldb::ReadOptions(), key1, &value);
+  if (s.ok()) {
+    leveldb::WriteBatch batch;
+    batch.Delete(key1);
+    batch.Put(key2, value);
+    s = db->Write(leveldb::WriteOptions(), &batch);
+  }
+
+The WriteBatch holds a sequence of edits to be made to the database, +and these edits within the batch are applied in order. Note that we +called Delete before Put so that if key1 is identical to key2, +we do not end up erroneously dropping the value entirely. +

+Apart from its atomicity benefits, WriteBatch may also be used to +speed up bulk updates by placing lots of individual mutations into the +same batch. + +

Synchronous Writes

+By default, each write to leveldb is asynchronous: it +returns after pushing the write from the process into the operating +system. The transfer from operating system memory to the underlying +persistent storage happens asynchronously. The sync flag +can be turned on for a particular write to make the write operation +not return until the data being written has been pushed all the way to +persistent storage. (On Posix systems, this is implemented by calling +either fsync(...) or fdatasync(...) or +msync(..., MS_SYNC) before the write operation returns.) +
+  leveldb::WriteOptions write_options;
+  write_options.sync = true;
+  db->Put(write_options, ...);
+
+Asynchronous writes are often more than a thousand times as fast as +synchronous writes. The downside of asynchronous writes is that a +crash of the machine may cause the last few updates to be lost. Note +that a crash of just the writing process (i.e., not a reboot) will not +cause any loss since even when sync is false, an update +is pushed from the process memory into the operating system before it +is considered done. + +

+Asynchronous writes can often be used safely. For example, when +loading a large amount of data into the database you can handle lost +updates by restarting the bulk load after a crash. A hybrid scheme is +also possible where every Nth write is synchronous, and in the event +of a crash, the bulk load is restarted just after the last synchronous +write finished by the previous run. (The synchronous write can update +a marker that describes where to restart on a crash.) + +

+WriteBatch provides an alternative to asynchronous writes. +Multiple updates may be placed in the same WriteBatch and +applied together using a synchronous write (i.e., +write_options.sync is set to true). The extra cost of +the synchronous write will be amortized across all of the writes in +the batch. + +

+

Concurrency

+

+A database may only be opened by one process at a time. +The leveldb implementation acquires a lock from the +operating system to prevent misuse. Within a single process, the +same leveldb::DB object may be safely shared by multiple +concurrent threads. I.e., different threads may write into or fetch +iterators or call Get on the same database without any +external synchronization (the leveldb implementation will +automatically do the required synchronization). However other objects +(like Iterator and WriteBatch) may require external synchronization. +If two threads share such an object, they must protect access to it +using their own locking protocol. More details are available in +the public header files. +

+

Iteration

+

+The following example demonstrates how to print all key,value pairs +in a database. +

+

+  leveldb::Iterator* it = db->NewIterator(leveldb::ReadOptions());
+  for (it->SeekToFirst(); it->Valid(); it->Next()) {
+    cout << it->key().ToString() << ": "  << it->value().ToString() << endl;
+  }
+  assert(it->status().ok());  // Check for any errors found during the scan
+  delete it;
+
+The following variation shows how to process just the keys in the +range [start,limit): +

+

+  for (it->Seek(start);
+       it->Valid() && it->key().ToString() < limit;
+       it->Next()) {
+    ...
+  }
+
+You can also process entries in reverse order. (Caveat: reverse +iteration may be somewhat slower than forward iteration.) +

+

+  for (it->SeekToLast(); it->Valid(); it->Prev()) {
+    ...
+  }
+
+

Snapshots

+

+Snapshots provide consistent read-only views over the entire state of +the key-value store. ReadOptions::snapshot may be non-NULL to indicate +that a read should operate on a particular version of the DB state. +If ReadOptions::snapshot is NULL, the read will operate on an +implicit snapshot of the current state. +

+Snapshots are created by the DB::GetSnapshot() method: +

+

+  leveldb::ReadOptions options;
+  options.snapshot = db->GetSnapshot();
+  ... apply some updates to db ...
+  leveldb::Iterator* iter = db->NewIterator(options);
+  ... read using iter to view the state when the snapshot was created ...
+  delete iter;
+  db->ReleaseSnapshot(options.snapshot);
+
+Note that when a snapshot is no longer needed, it should be released +using the DB::ReleaseSnapshot interface. This allows the +implementation to get rid of state that was being maintained just to +support reading as of that snapshot. +

Slice

+

+The return value of the it->key() and it->value() calls above +are instances of the leveldb::Slice type. Slice is a simple +structure that contains a length and a pointer to an external byte +array. Returning a Slice is a cheaper alternative to returning a +std::string since we do not need to copy potentially large keys and +values. In addition, leveldb methods do not return null-terminated +C-style strings since leveldb keys and values are allowed to +contain '\0' bytes. +

+C++ strings and null-terminated C-style strings can be easily converted +to a Slice: +

+

+   leveldb::Slice s1 = "hello";
+
+   std::string str("world");
+   leveldb::Slice s2 = str;
+
+A Slice can be easily converted back to a C++ string: +
+   std::string str = s1.ToString();
+   assert(str == std::string("hello"));
+
+Be careful when using Slices since it is up to the caller to ensure that +the external byte array into which the Slice points remains live while +the Slice is in use. For example, the following is buggy: +

+

+   leveldb::Slice slice;
+   if (...) {
+     std::string str = ...;
+     slice = str;
+   }
+   Use(slice);
+
+When the if statement goes out of scope, str will be destroyed and the +backing storage for slice will disappear. +

+

Comparators

+

+The preceding examples used the default ordering function for key, +which orders bytes lexicographically. You can however supply a custom +comparator when opening a database. For example, suppose each +database key consists of two numbers and we should sort by the first +number, breaking ties by the second number. First, define a proper +subclass of leveldb::Comparator that expresses these rules: +

+

+  class TwoPartComparator : public leveldb::Comparator {
+   public:
+    // Three-way comparison function:
+    //   if a < b: negative result
+    //   if a > b: positive result
+    //   else: zero result
+    int Compare(const leveldb::Slice& a, const leveldb::Slice& b) const {
+      int a1, a2, b1, b2;
+      ParseKey(a, &a1, &a2);
+      ParseKey(b, &b1, &b2);
+      if (a1 < b1) return -1;
+      if (a1 > b1) return +1;
+      if (a2 < b2) return -1;
+      if (a2 > b2) return +1;
+      return 0;
+    }
+
+    // Ignore the following methods for now:
+    const char* Name() const { return "TwoPartComparator"; }
+    void FindShortestSeparator(std::string*, const leveldb::Slice&) const { }
+    void FindShortSuccessor(std::string*) const { }
+  };
+
+Now create a database using this custom comparator: +

+

+  TwoPartComparator cmp;
+  leveldb::DB* db;
+  leveldb::Options options;
+  options.create_if_missing = true;
+  options.comparator = &cmp;
+  leveldb::Status status = leveldb::DB::Open(options, "/tmp/testdb", &db);
+  ...
+
+

Backwards compatibility

+

+The result of the comparator's Name method is attached to the +database when it is created, and is checked on every subsequent +database open. If the name changes, the leveldb::DB::Open call will +fail. Therefore, change the name if and only if the new key format +and comparison function are incompatible with existing databases, and +it is ok to discard the contents of all existing databases. +

+You can however still gradually evolve your key format over time with +a little bit of pre-planning. For example, you could store a version +number at the end of each key (one byte should suffice for most uses). +When you wish to switch to a new key format (e.g., adding an optional +third part to the keys processed by TwoPartComparator), +(a) keep the same comparator name (b) increment the version number +for new keys (c) change the comparator function so it uses the +version numbers found in the keys to decide how to interpret them. +

+

Performance

+

+Performance can be tuned by changing the default values of the +types defined in include/leveldb/options.h. + +

+

Block size

+

+leveldb groups adjacent keys together into the same block and such a +block is the unit of transfer to and from persistent storage. The +default block size is approximately 4096 uncompressed bytes. +Applications that mostly do bulk scans over the contents of the +database may wish to increase this size. Applications that do a lot +of point reads of small values may wish to switch to a smaller block +size if performance measurements indicate an improvement. There isn't +much benefit in using blocks smaller than one kilobyte, or larger than +a few megabytes. Also note that compression will be more effective +with larger block sizes. +

+

Compression

+

+Each block is individually compressed before being written to +persistent storage. Compression is on by default since the default +compression method is very fast, and is automatically disabled for +uncompressible data. In rare cases, applications may want to disable +compression entirely, but should only do so if benchmarks show a +performance improvement: +

+

+  leveldb::Options options;
+  options.compression = leveldb::kNoCompression;
+  ... leveldb::DB::Open(options, name, ...) ....
+
+

Cache

+

+The contents of the database are stored in a set of files in the +filesystem and each file stores a sequence of compressed blocks. If +options.cache is non-NULL, it is used to cache frequently used +uncompressed block contents. +

+

+  #include "leveldb/cache.h"
+
+  leveldb::Options options;
+  options.cache = leveldb::NewLRUCache(100 * 1048576);  // 100MB cache
+  leveldb::DB* db;
+  leveldb::DB::Open(options, name, &db);
+  ... use the db ...
+  delete db
+  delete options.cache;
+
+Note that the cache holds uncompressed data, and therefore it should +be sized according to application level data sizes, without any +reduction from compression. (Caching of compressed blocks is left to +the operating system buffer cache, or any custom Env +implementation provided by the client.) +

+When performing a bulk read, the application may wish to disable +caching so that the data processed by the bulk read does not end up +displacing most of the cached contents. A per-iterator option can be +used to achieve this: +

+

+  leveldb::ReadOptions options;
+  options.fill_cache = false;
+  leveldb::Iterator* it = db->NewIterator(options);
+  for (it->SeekToFirst(); it->Valid(); it->Next()) {
+    ...
+  }
+
+

Key Layout

+

+Note that the unit of disk transfer and caching is a block. Adjacent +keys (according to the database sort order) will usually be placed in +the same block. Therefore the application can improve its performance +by placing keys that are accessed together near each other and placing +infrequently used keys in a separate region of the key space. +

+For example, suppose we are implementing a simple file system on top +of leveldb. The types of entries we might wish to store are: +

+

+   filename -> permission-bits, length, list of file_block_ids
+   file_block_id -> data
+
+We might want to prefix filename keys with one letter (say '/') and the +file_block_id keys with a different letter (say '0') so that scans +over just the metadata do not force us to fetch and cache bulky file +contents. +

+

Filters

+

+Because of the way leveldb data is organized on disk, +a single Get() call may involve multiple reads from disk. +The optional FilterPolicy mechanism can be used to reduce +the number of disk reads substantially. +

+   leveldb::Options options;
+   options.filter_policy = NewBloomFilterPolicy(10);
+   leveldb::DB* db;
+   leveldb::DB::Open(options, "/tmp/testdb", &db);
+   ... use the database ...
+   delete db;
+   delete options.filter_policy;
+
+The preceding code associates a +Bloom filter +based filtering policy with the database. Bloom filter based +filtering relies on keeping some number of bits of data in memory per +key (in this case 10 bits per key since that is the argument we passed +to NewBloomFilterPolicy). This filter will reduce the number of unnecessary +disk reads needed for Get() calls by a factor of +approximately a 100. Increasing the bits per key will lead to a +larger reduction at the cost of more memory usage. We recommend that +applications whose working set does not fit in memory and that do a +lot of random reads set a filter policy. +

+If you are using a custom comparator, you should ensure that the filter +policy you are using is compatible with your comparator. For example, +consider a comparator that ignores trailing spaces when comparing keys. +NewBloomFilterPolicy must not be used with such a comparator. +Instead, the application should provide a custom filter policy that +also ignores trailing spaces. For example: +

+  class CustomFilterPolicy : public leveldb::FilterPolicy {
+   private:
+    FilterPolicy* builtin_policy_;
+   public:
+    CustomFilterPolicy() : builtin_policy_(NewBloomFilterPolicy(10)) { }
+    ~CustomFilterPolicy() { delete builtin_policy_; }
+
+    const char* Name() const { return "IgnoreTrailingSpacesFilter"; }
+
+    void CreateFilter(const Slice* keys, int n, std::string* dst) const {
+      // Use builtin bloom filter code after removing trailing spaces
+      std::vector<Slice> trimmed(n);
+      for (int i = 0; i < n; i++) {
+        trimmed[i] = RemoveTrailingSpaces(keys[i]);
+      }
+      return builtin_policy_->CreateFilter(&trimmed[i], n, dst);
+    }
+
+    bool KeyMayMatch(const Slice& key, const Slice& filter) const {
+      // Use builtin bloom filter code after removing trailing spaces
+      return builtin_policy_->KeyMayMatch(RemoveTrailingSpaces(key), filter);
+    }
+  };
+
+

+Advanced applications may provide a filter policy that does not use +a bloom filter but uses some other mechanism for summarizing a set +of keys. See leveldb/filter_policy.h for detail. +

+

Checksums

+

+leveldb associates checksums with all data it stores in the file system. +There are two separate controls provided over how aggressively these +checksums are verified: +

+

    +
  • ReadOptions::verify_checksums may be set to true to force + checksum verification of all data that is read from the file system on + behalf of a particular read. By default, no such verification is + done. +

    +

  • Options::paranoid_checks may be set to true before opening a + database to make the database implementation raise an error as soon as + it detects an internal corruption. Depending on which portion of the + database has been corrupted, the error may be raised when the database + is opened, or later by another database operation. By default, + paranoid checking is off so that the database can be used even if + parts of its persistent storage have been corrupted. +

    + If a database is corrupted (perhaps it cannot be opened when + paranoid checking is turned on), the leveldb::RepairDB function + may be used to recover as much of the data as possible +

    +

+

Approximate Sizes

+

+The GetApproximateSizes method can used to get the approximate +number of bytes of file system space used by one or more key ranges. +

+

+   leveldb::Range ranges[2];
+   ranges[0] = leveldb::Range("a", "c");
+   ranges[1] = leveldb::Range("x", "z");
+   uint64_t sizes[2];
+   leveldb::Status s = db->GetApproximateSizes(ranges, 2, sizes);
+
+The preceding call will set sizes[0] to the approximate number of +bytes of file system space used by the key range [a..c) and +sizes[1] to the approximate number of bytes used by the key range +[x..z). +

+

Environment

+

+All file operations (and other operating system calls) issued by the +leveldb implementation are routed through a leveldb::Env object. +Sophisticated clients may wish to provide their own Env +implementation to get better control. For example, an application may +introduce artificial delays in the file IO paths to limit the impact +of leveldb on other activities in the system. +

+

+  class SlowEnv : public leveldb::Env {
+    .. implementation of the Env interface ...
+  };
+
+  SlowEnv env;
+  leveldb::Options options;
+  options.env = &env;
+  Status s = leveldb::DB::Open(options, ...);
+
+

Porting

+

+leveldb may be ported to a new platform by providing platform +specific implementations of the types/methods/functions exported by +leveldb/port/port.h. See leveldb/port/port_example.h for more +details. +

+In addition, the new platform may need a new default leveldb::Env +implementation. See leveldb/util/env_posix.h for an example. + +

Other Information

+ +

+Details about the leveldb implementation may be found in +the following documents: +

+ + + diff --git a/src/leveldb/doc/log_format.txt b/src/leveldb/doc/log_format.txt new file mode 100644 index 00000000..4cca5ef6 --- /dev/null +++ b/src/leveldb/doc/log_format.txt @@ -0,0 +1,75 @@ +The log file contents are a sequence of 32KB blocks. The only +exception is that the tail of the file may contain a partial block. + +Each block consists of a sequence of records: + block := record* trailer? + record := + checksum: uint32 // crc32c of type and data[] ; little-endian + length: uint16 // little-endian + type: uint8 // One of FULL, FIRST, MIDDLE, LAST + data: uint8[length] + +A record never starts within the last six bytes of a block (since it +won't fit). Any leftover bytes here form the trailer, which must +consist entirely of zero bytes and must be skipped by readers. + +Aside: if exactly seven bytes are left in the current block, and a new +non-zero length record is added, the writer must emit a FIRST record +(which contains zero bytes of user data) to fill up the trailing seven +bytes of the block and then emit all of the user data in subsequent +blocks. + +More types may be added in the future. Some Readers may skip record +types they do not understand, others may report that some data was +skipped. + +FULL == 1 +FIRST == 2 +MIDDLE == 3 +LAST == 4 + +The FULL record contains the contents of an entire user record. + +FIRST, MIDDLE, LAST are types used for user records that have been +split into multiple fragments (typically because of block boundaries). +FIRST is the type of the first fragment of a user record, LAST is the +type of the last fragment of a user record, and MIDDLE is the type of +all interior fragments of a user record. + +Example: consider a sequence of user records: + A: length 1000 + B: length 97270 + C: length 8000 +A will be stored as a FULL record in the first block. + +B will be split into three fragments: first fragment occupies the rest +of the first block, second fragment occupies the entirety of the +second block, and the third fragment occupies a prefix of the third +block. This will leave six bytes free in the third block, which will +be left empty as the trailer. + +C will be stored as a FULL record in the fourth block. + +=================== + +Some benefits over the recordio format: + +(1) We do not need any heuristics for resyncing - just go to next +block boundary and scan. If there is a corruption, skip to the next +block. As a side-benefit, we do not get confused when part of the +contents of one log file are embedded as a record inside another log +file. + +(2) Splitting at approximate boundaries (e.g., for mapreduce) is +simple: find the next block boundary and skip records until we +hit a FULL or FIRST record. + +(3) We do not need extra buffering for large records. + +Some downsides compared to recordio format: + +(1) No packing of tiny records. This could be fixed by adding a new +record type, so it is a shortcoming of the current implementation, +not necessarily the format. + +(2) No compression. Again, this could be fixed by adding new record types. diff --git a/src/leveldb/doc/table_format.txt b/src/leveldb/doc/table_format.txt new file mode 100644 index 00000000..ca8f9b44 --- /dev/null +++ b/src/leveldb/doc/table_format.txt @@ -0,0 +1,104 @@ +File format +=========== + + + [data block 1] + [data block 2] + ... + [data block N] + [meta block 1] + ... + [meta block K] + [metaindex block] + [index block] + [Footer] (fixed size; starts at file_size - sizeof(Footer)) + + +The file contains internal pointers. Each such pointer is called +a BlockHandle and contains the following information: + offset: varint64 + size: varint64 +See https://developers.google.com/protocol-buffers/docs/encoding#varints +for an explanation of varint64 format. + +(1) The sequence of key/value pairs in the file are stored in sorted +order and partitioned into a sequence of data blocks. These blocks +come one after another at the beginning of the file. Each data block +is formatted according to the code in block_builder.cc, and then +optionally compressed. + +(2) After the data blocks we store a bunch of meta blocks. The +supported meta block types are described below. More meta block types +may be added in the future. Each meta block is again formatted using +block_builder.cc and then optionally compressed. + +(3) A "metaindex" block. It contains one entry for every other meta +block where the key is the name of the meta block and the value is a +BlockHandle pointing to that meta block. + +(4) An "index" block. This block contains one entry per data block, +where the key is a string >= last key in that data block and before +the first key in the successive data block. The value is the +BlockHandle for the data block. + +(6) At the very end of the file is a fixed length footer that contains +the BlockHandle of the metaindex and index blocks as well as a magic number. + metaindex_handle: char[p]; // Block handle for metaindex + index_handle: char[q]; // Block handle for index + padding: char[40-p-q]; // zeroed bytes to make fixed length + // (40==2*BlockHandle::kMaxEncodedLength) + magic: fixed64; // == 0xdb4775248b80fb57 (little-endian) + +"filter" Meta Block +------------------- + +If a "FilterPolicy" was specified when the database was opened, a +filter block is stored in each table. The "metaindex" block contains +an entry that maps from "filter." to the BlockHandle for the filter +block where "" is the string returned by the filter policy's +"Name()" method. + +The filter block stores a sequence of filters, where filter i contains +the output of FilterPolicy::CreateFilter() on all keys that are stored +in a block whose file offset falls within the range + + [ i*base ... (i+1)*base-1 ] + +Currently, "base" is 2KB. So for example, if blocks X and Y start in +the range [ 0KB .. 2KB-1 ], all of the keys in X and Y will be +converted to a filter by calling FilterPolicy::CreateFilter(), and the +resulting filter will be stored as the first filter in the filter +block. + +The filter block is formatted as follows: + + [filter 0] + [filter 1] + [filter 2] + ... + [filter N-1] + + [offset of filter 0] : 4 bytes + [offset of filter 1] : 4 bytes + [offset of filter 2] : 4 bytes + ... + [offset of filter N-1] : 4 bytes + + [offset of beginning of offset array] : 4 bytes + lg(base) : 1 byte + +The offset array at the end of the filter block allows efficient +mapping from a data block offset to the corresponding filter. + +"stats" Meta Block +------------------ + +This meta block contains a bunch of stats. The key is the name +of the statistic. The value contains the statistic. +TODO(postrelease): record following stats. + data size + index size + key size (uncompressed) + value size (uncompressed) + number of entries + number of data blocks diff --git a/src/leveldb/helpers/memenv/memenv.cc b/src/leveldb/helpers/memenv/memenv.cc new file mode 100644 index 00000000..3aecb12f --- /dev/null +++ b/src/leveldb/helpers/memenv/memenv.cc @@ -0,0 +1,387 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#define SIZE_MAX (~(size_t)0) // MCHN + +#include "helpers/memenv/memenv.h" + +#include "leveldb/env.h" +#include "leveldb/status.h" +#include "port/port.h" +#include "util/mutexlock.h" +#include +#include +#include +#include + +namespace leveldb { + +namespace { + +class FileState { + public: + // FileStates are reference counted. The initial reference count is zero + // and the caller must call Ref() at least once. + FileState() : refs_(0), size_(0) {} + + // Increase the reference count. + void Ref() { + MutexLock lock(&refs_mutex_); + ++refs_; + } + + // Decrease the reference count. Delete if this is the last reference. + void Unref() { + bool do_delete = false; + + { + MutexLock lock(&refs_mutex_); + --refs_; + assert(refs_ >= 0); + if (refs_ <= 0) { + do_delete = true; + } + } + + if (do_delete) { + delete this; + } + } + + uint64_t Size() const { return size_; } + + Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const { + if (offset > size_) { + return Status::IOError("Offset greater than file size."); + } + const uint64_t available = size_ - offset; + if (n > available) { + n = static_cast(available); + } + if (n == 0) { + *result = Slice(); + return Status::OK(); + } + + assert(offset / kBlockSize <= SIZE_MAX); + size_t block = static_cast(offset / kBlockSize); + size_t block_offset = offset % kBlockSize; + + if (n <= kBlockSize - block_offset) { + // The requested bytes are all in the first block. + *result = Slice(blocks_[block] + block_offset, n); + return Status::OK(); + } + + size_t bytes_to_copy = n; + char* dst = scratch; + + while (bytes_to_copy > 0) { + size_t avail = kBlockSize - block_offset; + if (avail > bytes_to_copy) { + avail = bytes_to_copy; + } + memcpy(dst, blocks_[block] + block_offset, avail); + + bytes_to_copy -= avail; + dst += avail; + block++; + block_offset = 0; + } + + *result = Slice(scratch, n); + return Status::OK(); + } + + Status Append(const Slice& data) { + const char* src = data.data(); + size_t src_len = data.size(); + + while (src_len > 0) { + size_t avail; + size_t offset = size_ % kBlockSize; + + if (offset != 0) { + // There is some room in the last block. + avail = kBlockSize - offset; + } else { + // No room in the last block; push new one. + blocks_.push_back(new char[kBlockSize]); + avail = kBlockSize; + } + + if (avail > src_len) { + avail = src_len; + } + memcpy(blocks_.back() + offset, src, avail); + src_len -= avail; + src += avail; + size_ += avail; + } + + return Status::OK(); + } + + private: + // Private since only Unref() should be used to delete it. + ~FileState() { + for (std::vector::iterator i = blocks_.begin(); i != blocks_.end(); + ++i) { + delete [] *i; + } + } + + // No copying allowed. + FileState(const FileState&); + void operator=(const FileState&); + + port::Mutex refs_mutex_; + int refs_; // Protected by refs_mutex_; + + // The following fields are not protected by any mutex. They are only mutable + // while the file is being written, and concurrent access is not allowed + // to writable files. + std::vector blocks_; + uint64_t size_; + + enum { kBlockSize = 8 * 1024 }; +}; + +class SequentialFileImpl : public SequentialFile { + public: + explicit SequentialFileImpl(FileState* file) : file_(file), pos_(0) { + file_->Ref(); + } + + ~SequentialFileImpl() { + file_->Unref(); + } + + virtual Status Read(size_t n, Slice* result, char* scratch) { + Status s = file_->Read(pos_, n, result, scratch); + if (s.ok()) { + pos_ += result->size(); + } + return s; + } + + virtual Status Skip(uint64_t n) { + if (pos_ > file_->Size()) { + return Status::IOError("pos_ > file_->Size()"); + } + const uint64_t available = file_->Size() - pos_; + if (n > available) { + n = available; + } + pos_ += n; + return Status::OK(); + } + + private: + FileState* file_; + uint64_t pos_; +}; + +class RandomAccessFileImpl : public RandomAccessFile { + public: + explicit RandomAccessFileImpl(FileState* file) : file_(file) { + file_->Ref(); + } + + ~RandomAccessFileImpl() { + file_->Unref(); + } + + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + return file_->Read(offset, n, result, scratch); + } + + private: + FileState* file_; +}; + +class WritableFileImpl : public WritableFile { + public: + WritableFileImpl(FileState* file) : file_(file) { + file_->Ref(); + } + + ~WritableFileImpl() { + file_->Unref(); + } + + virtual Status Append(const Slice& data) { + return file_->Append(data); + } + + virtual Status Close() { return Status::OK(); } + virtual Status Flush() { return Status::OK(); } + virtual Status Sync() { return Status::OK(); } + + private: + FileState* file_; +}; + +class NoOpLogger : public Logger { + public: + virtual void Logv(const char* format, va_list ap) { } +}; + +class InMemoryEnv : public EnvWrapper { + public: + explicit InMemoryEnv(Env* base_env) : EnvWrapper(base_env) { } + + virtual ~InMemoryEnv() { + for (FileSystem::iterator i = file_map_.begin(); i != file_map_.end(); ++i){ + i->second->Unref(); + } + } + + // Partial implementation of the Env interface. + virtual Status NewSequentialFile(const std::string& fname, + SequentialFile** result) { + MutexLock lock(&mutex_); + if (file_map_.find(fname) == file_map_.end()) { + *result = NULL; + return Status::IOError(fname, "File not found"); + } + + *result = new SequentialFileImpl(file_map_[fname]); + return Status::OK(); + } + + virtual Status NewRandomAccessFile(const std::string& fname, + RandomAccessFile** result) { + MutexLock lock(&mutex_); + if (file_map_.find(fname) == file_map_.end()) { + *result = NULL; + return Status::IOError(fname, "File not found"); + } + + *result = new RandomAccessFileImpl(file_map_[fname]); + return Status::OK(); + } + + virtual Status NewWritableFile(const std::string& fname, + WritableFile** result) { + MutexLock lock(&mutex_); + if (file_map_.find(fname) != file_map_.end()) { + DeleteFileInternal(fname); + } + + FileState* file = new FileState(); + file->Ref(); + file_map_[fname] = file; + + *result = new WritableFileImpl(file); + return Status::OK(); + } + + virtual bool FileExists(const std::string& fname) { + MutexLock lock(&mutex_); + return file_map_.find(fname) != file_map_.end(); + } + + virtual Status GetChildren(const std::string& dir, + std::vector* result) { + MutexLock lock(&mutex_); + result->clear(); + + for (FileSystem::iterator i = file_map_.begin(); i != file_map_.end(); ++i){ + const std::string& filename = i->first; + + if (filename.size() >= dir.size() + 1 && filename[dir.size()] == '/' && + Slice(filename).starts_with(Slice(dir))) { + result->push_back(filename.substr(dir.size() + 1)); + } + } + + return Status::OK(); + } + + void DeleteFileInternal(const std::string& fname) { + if (file_map_.find(fname) == file_map_.end()) { + return; + } + + file_map_[fname]->Unref(); + file_map_.erase(fname); + } + + virtual Status DeleteFile(const std::string& fname) { + MutexLock lock(&mutex_); + if (file_map_.find(fname) == file_map_.end()) { + return Status::IOError(fname, "File not found"); + } + + DeleteFileInternal(fname); + return Status::OK(); + } + + virtual Status CreateDir(const std::string& dirname) { + return Status::OK(); + } + + virtual Status DeleteDir(const std::string& dirname) { + return Status::OK(); + } + + virtual Status GetFileSize(const std::string& fname, uint64_t* file_size) { + MutexLock lock(&mutex_); + if (file_map_.find(fname) == file_map_.end()) { + return Status::IOError(fname, "File not found"); + } + + *file_size = file_map_[fname]->Size(); + return Status::OK(); + } + + virtual Status RenameFile(const std::string& src, + const std::string& target) { + MutexLock lock(&mutex_); + if (file_map_.find(src) == file_map_.end()) { + return Status::IOError(src, "File not found"); + } + + DeleteFileInternal(target); + file_map_[target] = file_map_[src]; + file_map_.erase(src); + return Status::OK(); + } + + virtual Status LockFile(const std::string& fname, FileLock** lock) { + *lock = new FileLock; + return Status::OK(); + } + + virtual Status UnlockFile(FileLock* lock) { + delete lock; + return Status::OK(); + } + + virtual Status GetTestDirectory(std::string* path) { + *path = "/test"; + return Status::OK(); + } + + virtual Status NewLogger(const std::string& fname, Logger** result) { + *result = new NoOpLogger; + return Status::OK(); + } + + private: + // Map from filenames to FileState objects, representing a simple file system. + typedef std::map FileSystem; + port::Mutex mutex_; + FileSystem file_map_; // Protected by mutex_. +}; + +} // namespace + +Env* NewMemEnv(Env* base_env) { + return new InMemoryEnv(base_env); +} + +} // namespace leveldb diff --git a/src/leveldb/helpers/memenv/memenv.h b/src/leveldb/helpers/memenv/memenv.h new file mode 100644 index 00000000..03b88de7 --- /dev/null +++ b/src/leveldb/helpers/memenv/memenv.h @@ -0,0 +1,20 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_HELPERS_MEMENV_MEMENV_H_ +#define STORAGE_LEVELDB_HELPERS_MEMENV_MEMENV_H_ + +namespace leveldb { + +class Env; + +// Returns a new environment that stores its data in memory and delegates +// all non-file-storage tasks to base_env. The caller must delete the result +// when it is no longer needed. +// *base_env must remain live while the result is in use. +Env* NewMemEnv(Env* base_env); + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_HELPERS_MEMENV_MEMENV_H_ diff --git a/src/leveldb/helpers/memenv/memenv_test.cc b/src/leveldb/helpers/memenv/memenv_test.cc new file mode 100644 index 00000000..a44310fe --- /dev/null +++ b/src/leveldb/helpers/memenv/memenv_test.cc @@ -0,0 +1,232 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "helpers/memenv/memenv.h" + +#include "db/db_impl.h" +#include "leveldb/db.h" +#include "leveldb/env.h" +#include "util/testharness.h" +#include +#include + +namespace leveldb { + +class MemEnvTest { + public: + Env* env_; + + MemEnvTest() + : env_(NewMemEnv(Env::Default())) { + } + ~MemEnvTest() { + delete env_; + } +}; + +TEST(MemEnvTest, Basics) { + uint64_t file_size; + WritableFile* writable_file; + std::vector children; + + ASSERT_OK(env_->CreateDir("/dir")); + + // Check that the directory is empty. + ASSERT_TRUE(!env_->FileExists("/dir/non_existent")); + ASSERT_TRUE(!env_->GetFileSize("/dir/non_existent", &file_size).ok()); + ASSERT_OK(env_->GetChildren("/dir", &children)); + ASSERT_EQ(0, children.size()); + + // Create a file. + ASSERT_OK(env_->NewWritableFile("/dir/f", &writable_file)); + delete writable_file; + + // Check that the file exists. + ASSERT_TRUE(env_->FileExists("/dir/f")); + ASSERT_OK(env_->GetFileSize("/dir/f", &file_size)); + ASSERT_EQ(0, file_size); + ASSERT_OK(env_->GetChildren("/dir", &children)); + ASSERT_EQ(1, children.size()); + ASSERT_EQ("f", children[0]); + + // Write to the file. + ASSERT_OK(env_->NewWritableFile("/dir/f", &writable_file)); + ASSERT_OK(writable_file->Append("abc")); + delete writable_file; + + // Check for expected size. + ASSERT_OK(env_->GetFileSize("/dir/f", &file_size)); + ASSERT_EQ(3, file_size); + + // Check that renaming works. + ASSERT_TRUE(!env_->RenameFile("/dir/non_existent", "/dir/g").ok()); + ASSERT_OK(env_->RenameFile("/dir/f", "/dir/g")); + ASSERT_TRUE(!env_->FileExists("/dir/f")); + ASSERT_TRUE(env_->FileExists("/dir/g")); + ASSERT_OK(env_->GetFileSize("/dir/g", &file_size)); + ASSERT_EQ(3, file_size); + + // Check that opening non-existent file fails. + SequentialFile* seq_file; + RandomAccessFile* rand_file; + ASSERT_TRUE(!env_->NewSequentialFile("/dir/non_existent", &seq_file).ok()); + ASSERT_TRUE(!seq_file); + ASSERT_TRUE(!env_->NewRandomAccessFile("/dir/non_existent", &rand_file).ok()); + ASSERT_TRUE(!rand_file); + + // Check that deleting works. + ASSERT_TRUE(!env_->DeleteFile("/dir/non_existent").ok()); + ASSERT_OK(env_->DeleteFile("/dir/g")); + ASSERT_TRUE(!env_->FileExists("/dir/g")); + ASSERT_OK(env_->GetChildren("/dir", &children)); + ASSERT_EQ(0, children.size()); + ASSERT_OK(env_->DeleteDir("/dir")); +} + +TEST(MemEnvTest, ReadWrite) { + WritableFile* writable_file; + SequentialFile* seq_file; + RandomAccessFile* rand_file; + Slice result; + char scratch[100]; + + ASSERT_OK(env_->CreateDir("/dir")); + + ASSERT_OK(env_->NewWritableFile("/dir/f", &writable_file)); + ASSERT_OK(writable_file->Append("hello ")); + ASSERT_OK(writable_file->Append("world")); + delete writable_file; + + // Read sequentially. + ASSERT_OK(env_->NewSequentialFile("/dir/f", &seq_file)); + ASSERT_OK(seq_file->Read(5, &result, scratch)); // Read "hello". + ASSERT_EQ(0, result.compare("hello")); + ASSERT_OK(seq_file->Skip(1)); + ASSERT_OK(seq_file->Read(1000, &result, scratch)); // Read "world". + ASSERT_EQ(0, result.compare("world")); + ASSERT_OK(seq_file->Read(1000, &result, scratch)); // Try reading past EOF. + ASSERT_EQ(0, result.size()); + ASSERT_OK(seq_file->Skip(100)); // Try to skip past end of file. + ASSERT_OK(seq_file->Read(1000, &result, scratch)); + ASSERT_EQ(0, result.size()); + delete seq_file; + + // Random reads. + ASSERT_OK(env_->NewRandomAccessFile("/dir/f", &rand_file)); + ASSERT_OK(rand_file->Read(6, 5, &result, scratch)); // Read "world". + ASSERT_EQ(0, result.compare("world")); + ASSERT_OK(rand_file->Read(0, 5, &result, scratch)); // Read "hello". + ASSERT_EQ(0, result.compare("hello")); + ASSERT_OK(rand_file->Read(10, 100, &result, scratch)); // Read "d". + ASSERT_EQ(0, result.compare("d")); + + // Too high offset. + ASSERT_TRUE(!rand_file->Read(1000, 5, &result, scratch).ok()); + delete rand_file; +} + +TEST(MemEnvTest, Locks) { + FileLock* lock; + + // These are no-ops, but we test they return success. + ASSERT_OK(env_->LockFile("some file", &lock)); + ASSERT_OK(env_->UnlockFile(lock)); +} + +TEST(MemEnvTest, Misc) { + std::string test_dir; + ASSERT_OK(env_->GetTestDirectory(&test_dir)); + ASSERT_TRUE(!test_dir.empty()); + + WritableFile* writable_file; + ASSERT_OK(env_->NewWritableFile("/a/b", &writable_file)); + + // These are no-ops, but we test they return success. + ASSERT_OK(writable_file->Sync()); + ASSERT_OK(writable_file->Flush()); + ASSERT_OK(writable_file->Close()); + delete writable_file; +} + +TEST(MemEnvTest, LargeWrite) { + const size_t kWriteSize = 300 * 1024; + char* scratch = new char[kWriteSize * 2]; + + std::string write_data; + for (size_t i = 0; i < kWriteSize; ++i) { + write_data.append(1, static_cast(i)); + } + + WritableFile* writable_file; + ASSERT_OK(env_->NewWritableFile("/dir/f", &writable_file)); + ASSERT_OK(writable_file->Append("foo")); + ASSERT_OK(writable_file->Append(write_data)); + delete writable_file; + + SequentialFile* seq_file; + Slice result; + ASSERT_OK(env_->NewSequentialFile("/dir/f", &seq_file)); + ASSERT_OK(seq_file->Read(3, &result, scratch)); // Read "foo". + ASSERT_EQ(0, result.compare("foo")); + + size_t read = 0; + std::string read_data; + while (read < kWriteSize) { + ASSERT_OK(seq_file->Read(kWriteSize - read, &result, scratch)); + read_data.append(result.data(), result.size()); + read += result.size(); + } + ASSERT_TRUE(write_data == read_data); + delete seq_file; + delete [] scratch; +} + +TEST(MemEnvTest, DBTest) { + Options options; + options.create_if_missing = true; + options.env = env_; + DB* db; + + const Slice keys[] = {Slice("aaa"), Slice("bbb"), Slice("ccc")}; + const Slice vals[] = {Slice("foo"), Slice("bar"), Slice("baz")}; + + ASSERT_OK(DB::Open(options, "/dir/db", &db)); + for (size_t i = 0; i < 3; ++i) { + ASSERT_OK(db->Put(WriteOptions(), keys[i], vals[i])); + } + + for (size_t i = 0; i < 3; ++i) { + std::string res; + ASSERT_OK(db->Get(ReadOptions(), keys[i], &res)); + ASSERT_TRUE(res == vals[i]); + } + + Iterator* iterator = db->NewIterator(ReadOptions()); + iterator->SeekToFirst(); + for (size_t i = 0; i < 3; ++i) { + ASSERT_TRUE(iterator->Valid()); + ASSERT_TRUE(keys[i] == iterator->key()); + ASSERT_TRUE(vals[i] == iterator->value()); + iterator->Next(); + } + ASSERT_TRUE(!iterator->Valid()); + delete iterator; + + DBImpl* dbi = reinterpret_cast(db); + ASSERT_OK(dbi->TEST_CompactMemTable()); + + for (size_t i = 0; i < 3; ++i) { + std::string res; + ASSERT_OK(db->Get(ReadOptions(), keys[i], &res)); + ASSERT_TRUE(res == vals[i]); + } + + delete db; +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/include/leveldb/c.h b/src/leveldb/include/leveldb/c.h new file mode 100644 index 00000000..1048fe3b --- /dev/null +++ b/src/leveldb/include/leveldb/c.h @@ -0,0 +1,290 @@ +/* Copyright (c) 2011 The LevelDB Authors. All rights reserved. + Use of this source code is governed by a BSD-style license that can be + found in the LICENSE file. See the AUTHORS file for names of contributors. + + C bindings for leveldb. May be useful as a stable ABI that can be + used by programs that keep leveldb in a shared library, or for + a JNI api. + + Does not support: + . getters for the option types + . custom comparators that implement key shortening + . custom iter, db, env, cache implementations using just the C bindings + + Some conventions: + + (1) We expose just opaque struct pointers and functions to clients. + This allows us to change internal representations without having to + recompile clients. + + (2) For simplicity, there is no equivalent to the Slice type. Instead, + the caller has to pass the pointer and length as separate + arguments. + + (3) Errors are represented by a null-terminated c string. NULL + means no error. All operations that can raise an error are passed + a "char** errptr" as the last argument. One of the following must + be true on entry: + *errptr == NULL + *errptr points to a malloc()ed null-terminated error message + (On Windows, *errptr must have been malloc()-ed by this library.) + On success, a leveldb routine leaves *errptr unchanged. + On failure, leveldb frees the old value of *errptr and + set *errptr to a malloc()ed error message. + + (4) Bools have the type unsigned char (0 == false; rest == true) + + (5) All of the pointer arguments must be non-NULL. +*/ + +#ifndef STORAGE_LEVELDB_INCLUDE_C_H_ +#define STORAGE_LEVELDB_INCLUDE_C_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include + +/* Exported types */ + +typedef struct leveldb_t leveldb_t; +typedef struct leveldb_cache_t leveldb_cache_t; +typedef struct leveldb_comparator_t leveldb_comparator_t; +typedef struct leveldb_env_t leveldb_env_t; +typedef struct leveldb_filelock_t leveldb_filelock_t; +typedef struct leveldb_filterpolicy_t leveldb_filterpolicy_t; +typedef struct leveldb_iterator_t leveldb_iterator_t; +typedef struct leveldb_logger_t leveldb_logger_t; +typedef struct leveldb_options_t leveldb_options_t; +typedef struct leveldb_randomfile_t leveldb_randomfile_t; +typedef struct leveldb_readoptions_t leveldb_readoptions_t; +typedef struct leveldb_seqfile_t leveldb_seqfile_t; +typedef struct leveldb_snapshot_t leveldb_snapshot_t; +typedef struct leveldb_writablefile_t leveldb_writablefile_t; +typedef struct leveldb_writebatch_t leveldb_writebatch_t; +typedef struct leveldb_writeoptions_t leveldb_writeoptions_t; + +/* DB operations */ + +extern leveldb_t* leveldb_open( + const leveldb_options_t* options, + const char* name, + char** errptr); + +extern void leveldb_close(leveldb_t* db); + +extern void leveldb_put( + leveldb_t* db, + const leveldb_writeoptions_t* options, + const char* key, size_t keylen, + const char* val, size_t vallen, + char** errptr); + +extern void leveldb_delete( + leveldb_t* db, + const leveldb_writeoptions_t* options, + const char* key, size_t keylen, + char** errptr); + +extern void leveldb_write( + leveldb_t* db, + const leveldb_writeoptions_t* options, + leveldb_writebatch_t* batch, + char** errptr); + +/* Returns NULL if not found. A malloc()ed array otherwise. + Stores the length of the array in *vallen. */ +extern char* leveldb_get( + leveldb_t* db, + const leveldb_readoptions_t* options, + const char* key, size_t keylen, + size_t* vallen, + char** errptr); + +extern leveldb_iterator_t* leveldb_create_iterator( + leveldb_t* db, + const leveldb_readoptions_t* options); + +extern const leveldb_snapshot_t* leveldb_create_snapshot( + leveldb_t* db); + +extern void leveldb_release_snapshot( + leveldb_t* db, + const leveldb_snapshot_t* snapshot); + +/* Returns NULL if property name is unknown. + Else returns a pointer to a malloc()-ed null-terminated value. */ +extern char* leveldb_property_value( + leveldb_t* db, + const char* propname); + +extern void leveldb_approximate_sizes( + leveldb_t* db, + int num_ranges, + const char* const* range_start_key, const size_t* range_start_key_len, + const char* const* range_limit_key, const size_t* range_limit_key_len, + uint64_t* sizes); + +extern void leveldb_compact_range( + leveldb_t* db, + const char* start_key, size_t start_key_len, + const char* limit_key, size_t limit_key_len); + +/* Management operations */ + +extern void leveldb_destroy_db( + const leveldb_options_t* options, + const char* name, + char** errptr); + +extern void leveldb_repair_db( + const leveldb_options_t* options, + const char* name, + char** errptr); + +/* Iterator */ + +extern void leveldb_iter_destroy(leveldb_iterator_t*); +extern unsigned char leveldb_iter_valid(const leveldb_iterator_t*); +extern void leveldb_iter_seek_to_first(leveldb_iterator_t*); +extern void leveldb_iter_seek_to_last(leveldb_iterator_t*); +extern void leveldb_iter_seek(leveldb_iterator_t*, const char* k, size_t klen); +extern void leveldb_iter_next(leveldb_iterator_t*); +extern void leveldb_iter_prev(leveldb_iterator_t*); +extern const char* leveldb_iter_key(const leveldb_iterator_t*, size_t* klen); +extern const char* leveldb_iter_value(const leveldb_iterator_t*, size_t* vlen); +extern void leveldb_iter_get_error(const leveldb_iterator_t*, char** errptr); + +/* Write batch */ + +extern leveldb_writebatch_t* leveldb_writebatch_create(); +extern void leveldb_writebatch_destroy(leveldb_writebatch_t*); +extern void leveldb_writebatch_clear(leveldb_writebatch_t*); +extern void leveldb_writebatch_put( + leveldb_writebatch_t*, + const char* key, size_t klen, + const char* val, size_t vlen); +extern void leveldb_writebatch_delete( + leveldb_writebatch_t*, + const char* key, size_t klen); +extern void leveldb_writebatch_iterate( + leveldb_writebatch_t*, + void* state, + void (*put)(void*, const char* k, size_t klen, const char* v, size_t vlen), + void (*deleted)(void*, const char* k, size_t klen)); + +/* Options */ + +extern leveldb_options_t* leveldb_options_create(); +extern void leveldb_options_destroy(leveldb_options_t*); +extern void leveldb_options_set_comparator( + leveldb_options_t*, + leveldb_comparator_t*); +extern void leveldb_options_set_filter_policy( + leveldb_options_t*, + leveldb_filterpolicy_t*); +extern void leveldb_options_set_create_if_missing( + leveldb_options_t*, unsigned char); +extern void leveldb_options_set_error_if_exists( + leveldb_options_t*, unsigned char); +extern void leveldb_options_set_paranoid_checks( + leveldb_options_t*, unsigned char); +extern void leveldb_options_set_env(leveldb_options_t*, leveldb_env_t*); +extern void leveldb_options_set_info_log(leveldb_options_t*, leveldb_logger_t*); +extern void leveldb_options_set_write_buffer_size(leveldb_options_t*, size_t); +extern void leveldb_options_set_max_open_files(leveldb_options_t*, int); +extern void leveldb_options_set_cache(leveldb_options_t*, leveldb_cache_t*); +extern void leveldb_options_set_block_size(leveldb_options_t*, size_t); +extern void leveldb_options_set_block_restart_interval(leveldb_options_t*, int); + +enum { + leveldb_no_compression = 0, + leveldb_snappy_compression = 1 +}; +extern void leveldb_options_set_compression(leveldb_options_t*, int); + +/* Comparator */ + +extern leveldb_comparator_t* leveldb_comparator_create( + void* state, + void (*destructor)(void*), + int (*compare)( + void*, + const char* a, size_t alen, + const char* b, size_t blen), + const char* (*name)(void*)); +extern void leveldb_comparator_destroy(leveldb_comparator_t*); + +/* Filter policy */ + +extern leveldb_filterpolicy_t* leveldb_filterpolicy_create( + void* state, + void (*destructor)(void*), + char* (*create_filter)( + void*, + const char* const* key_array, const size_t* key_length_array, + int num_keys, + size_t* filter_length), + unsigned char (*key_may_match)( + void*, + const char* key, size_t length, + const char* filter, size_t filter_length), + const char* (*name)(void*)); +extern void leveldb_filterpolicy_destroy(leveldb_filterpolicy_t*); + +extern leveldb_filterpolicy_t* leveldb_filterpolicy_create_bloom( + int bits_per_key); + +/* Read options */ + +extern leveldb_readoptions_t* leveldb_readoptions_create(); +extern void leveldb_readoptions_destroy(leveldb_readoptions_t*); +extern void leveldb_readoptions_set_verify_checksums( + leveldb_readoptions_t*, + unsigned char); +extern void leveldb_readoptions_set_fill_cache( + leveldb_readoptions_t*, unsigned char); +extern void leveldb_readoptions_set_snapshot( + leveldb_readoptions_t*, + const leveldb_snapshot_t*); + +/* Write options */ + +extern leveldb_writeoptions_t* leveldb_writeoptions_create(); +extern void leveldb_writeoptions_destroy(leveldb_writeoptions_t*); +extern void leveldb_writeoptions_set_sync( + leveldb_writeoptions_t*, unsigned char); + +/* Cache */ + +extern leveldb_cache_t* leveldb_cache_create_lru(size_t capacity); +extern void leveldb_cache_destroy(leveldb_cache_t* cache); + +/* Env */ + +extern leveldb_env_t* leveldb_create_default_env(); +extern void leveldb_env_destroy(leveldb_env_t*); + +/* Utility */ + +/* Calls free(ptr). + REQUIRES: ptr was malloc()-ed and returned by one of the routines + in this file. Note that in certain cases (typically on Windows), you + may need to call this routine instead of free(ptr) to dispose of + malloc()-ed memory returned by this library. */ +extern void leveldb_free(void* ptr); + +/* Return the major version number for this release. */ +extern int leveldb_major_version(); + +/* Return the minor version number for this release. */ +extern int leveldb_minor_version(); + +#ifdef __cplusplus +} /* end extern "C" */ +#endif + +#endif /* STORAGE_LEVELDB_INCLUDE_C_H_ */ diff --git a/src/leveldb/include/leveldb/cache.h b/src/leveldb/include/leveldb/cache.h new file mode 100644 index 00000000..1a201e5e --- /dev/null +++ b/src/leveldb/include/leveldb/cache.h @@ -0,0 +1,99 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// A Cache is an interface that maps keys to values. It has internal +// synchronization and may be safely accessed concurrently from +// multiple threads. It may automatically evict entries to make room +// for new entries. Values have a specified charge against the cache +// capacity. For example, a cache where the values are variable +// length strings, may use the length of the string as the charge for +// the string. +// +// A builtin cache implementation with a least-recently-used eviction +// policy is provided. Clients may use their own implementations if +// they want something more sophisticated (like scan-resistance, a +// custom eviction policy, variable cache sizing, etc.) + +#ifndef STORAGE_LEVELDB_INCLUDE_CACHE_H_ +#define STORAGE_LEVELDB_INCLUDE_CACHE_H_ + +#include +#include "leveldb/slice.h" + +namespace leveldb { + +class Cache; + +// Create a new cache with a fixed size capacity. This implementation +// of Cache uses a least-recently-used eviction policy. +extern Cache* NewLRUCache(size_t capacity); + +class Cache { + public: + Cache() { } + + // Destroys all existing entries by calling the "deleter" + // function that was passed to the constructor. + virtual ~Cache(); + + // Opaque handle to an entry stored in the cache. + struct Handle { }; + + // Insert a mapping from key->value into the cache and assign it + // the specified charge against the total cache capacity. + // + // Returns a handle that corresponds to the mapping. The caller + // must call this->Release(handle) when the returned mapping is no + // longer needed. + // + // When the inserted entry is no longer needed, the key and + // value will be passed to "deleter". + virtual Handle* Insert(const Slice& key, void* value, size_t charge, + void (*deleter)(const Slice& key, void* value)) = 0; + + // If the cache has no mapping for "key", returns NULL. + // + // Else return a handle that corresponds to the mapping. The caller + // must call this->Release(handle) when the returned mapping is no + // longer needed. + virtual Handle* Lookup(const Slice& key) = 0; + + // Release a mapping returned by a previous Lookup(). + // REQUIRES: handle must not have been released yet. + // REQUIRES: handle must have been returned by a method on *this. + virtual void Release(Handle* handle) = 0; + + // Return the value encapsulated in a handle returned by a + // successful Lookup(). + // REQUIRES: handle must not have been released yet. + // REQUIRES: handle must have been returned by a method on *this. + virtual void* Value(Handle* handle) = 0; + + // If the cache contains entry for key, erase it. Note that the + // underlying entry will be kept around until all existing handles + // to it have been released. + virtual void Erase(const Slice& key) = 0; + + // Return a new numeric id. May be used by multiple clients who are + // sharing the same cache to partition the key space. Typically the + // client will allocate a new id at startup and prepend the id to + // its cache keys. + virtual uint64_t NewId() = 0; + + private: + void LRU_Remove(Handle* e); + void LRU_Append(Handle* e); + void Unref(Handle* e); + + struct Rep; + Rep* rep_; + + // No copying allowed + Cache(const Cache&); + void operator=(const Cache&); +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_INCLUDE_CACHE_H_ diff --git a/src/leveldb/include/leveldb/comparator.h b/src/leveldb/include/leveldb/comparator.h new file mode 100644 index 00000000..556b984c --- /dev/null +++ b/src/leveldb/include/leveldb/comparator.h @@ -0,0 +1,63 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_INCLUDE_COMPARATOR_H_ +#define STORAGE_LEVELDB_INCLUDE_COMPARATOR_H_ + +#include + +namespace leveldb { + +class Slice; + +// A Comparator object provides a total order across slices that are +// used as keys in an sstable or a database. A Comparator implementation +// must be thread-safe since leveldb may invoke its methods concurrently +// from multiple threads. +class Comparator { + public: + virtual ~Comparator(); + + // Three-way comparison. Returns value: + // < 0 iff "a" < "b", + // == 0 iff "a" == "b", + // > 0 iff "a" > "b" + virtual int Compare(const Slice& a, const Slice& b) const = 0; + + // The name of the comparator. Used to check for comparator + // mismatches (i.e., a DB created with one comparator is + // accessed using a different comparator. + // + // The client of this package should switch to a new name whenever + // the comparator implementation changes in a way that will cause + // the relative ordering of any two keys to change. + // + // Names starting with "leveldb." are reserved and should not be used + // by any clients of this package. + virtual const char* Name() const = 0; + + // Advanced functions: these are used to reduce the space requirements + // for internal data structures like index blocks. + + // If *start < limit, changes *start to a short string in [start,limit). + // Simple comparator implementations may return with *start unchanged, + // i.e., an implementation of this method that does nothing is correct. + virtual void FindShortestSeparator( + std::string* start, + const Slice& limit) const = 0; + + // Changes *key to a short string >= *key. + // Simple comparator implementations may return with *key unchanged, + // i.e., an implementation of this method that does nothing is correct. + virtual void FindShortSuccessor(std::string* key) const = 0; +}; + +// Return a builtin comparator that uses lexicographic byte-wise +// ordering. The result remains the property of this module and +// must not be deleted. +extern const Comparator* BytewiseComparator(); + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_INCLUDE_COMPARATOR_H_ diff --git a/src/leveldb/include/leveldb/db.h b/src/leveldb/include/leveldb/db.h new file mode 100644 index 00000000..4c169bf2 --- /dev/null +++ b/src/leveldb/include/leveldb/db.h @@ -0,0 +1,161 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_INCLUDE_DB_H_ +#define STORAGE_LEVELDB_INCLUDE_DB_H_ + +#include +#include +#include "leveldb/iterator.h" +#include "leveldb/options.h" + +namespace leveldb { + +// Update Makefile if you change these +static const int kMajorVersion = 1; +static const int kMinorVersion = 18; + +struct Options; +struct ReadOptions; +struct WriteOptions; +class WriteBatch; + +// Abstract handle to particular state of a DB. +// A Snapshot is an immutable object and can therefore be safely +// accessed from multiple threads without any external synchronization. +class Snapshot { + protected: + virtual ~Snapshot(); +}; + +// A range of keys +struct Range { + Slice start; // Included in the range + Slice limit; // Not included in the range + + Range() { } + Range(const Slice& s, const Slice& l) : start(s), limit(l) { } +}; + +// A DB is a persistent ordered map from keys to values. +// A DB is safe for concurrent access from multiple threads without +// any external synchronization. +class DB { + public: + // Open the database with the specified "name". + // Stores a pointer to a heap-allocated database in *dbptr and returns + // OK on success. + // Stores NULL in *dbptr and returns a non-OK status on error. + // Caller should delete *dbptr when it is no longer needed. + static Status Open(const Options& options, + const std::string& name, + DB** dbptr); + + DB() { } + virtual ~DB(); + + // Set the database entry for "key" to "value". Returns OK on success, + // and a non-OK status on error. + // Note: consider setting options.sync = true. + virtual Status Put(const WriteOptions& options, + const Slice& key, + const Slice& value) = 0; + + // Remove the database entry (if any) for "key". Returns OK on + // success, and a non-OK status on error. It is not an error if "key" + // did not exist in the database. + // Note: consider setting options.sync = true. + virtual Status Delete(const WriteOptions& options, const Slice& key) = 0; + + // Apply the specified updates to the database. + // Returns OK on success, non-OK on failure. + // Note: consider setting options.sync = true. + virtual Status Write(const WriteOptions& options, WriteBatch* updates) = 0; + + // If the database contains an entry for "key" store the + // corresponding value in *value and return OK. + // + // If there is no entry for "key" leave *value unchanged and return + // a status for which Status::IsNotFound() returns true. + // + // May return some other Status on an error. + virtual Status Get(const ReadOptions& options, + const Slice& key, std::string* value) = 0; + + // Return a heap-allocated iterator over the contents of the database. + // The result of NewIterator() is initially invalid (caller must + // call one of the Seek methods on the iterator before using it). + // + // Caller should delete the iterator when it is no longer needed. + // The returned iterator should be deleted before this db is deleted. + virtual Iterator* NewIterator(const ReadOptions& options) = 0; + + // Return a handle to the current DB state. Iterators created with + // this handle will all observe a stable snapshot of the current DB + // state. The caller must call ReleaseSnapshot(result) when the + // snapshot is no longer needed. + virtual const Snapshot* GetSnapshot() = 0; + + // Release a previously acquired snapshot. The caller must not + // use "snapshot" after this call. + virtual void ReleaseSnapshot(const Snapshot* snapshot) = 0; + + // DB implementations can export properties about their state + // via this method. If "property" is a valid property understood by this + // DB implementation, fills "*value" with its current value and returns + // true. Otherwise returns false. + // + // + // Valid property names include: + // + // "leveldb.num-files-at-level" - return the number of files at level , + // where is an ASCII representation of a level number (e.g. "0"). + // "leveldb.stats" - returns a multi-line string that describes statistics + // about the internal operation of the DB. + // "leveldb.sstables" - returns a multi-line string that describes all + // of the sstables that make up the db contents. + virtual bool GetProperty(const Slice& property, std::string* value) = 0; + + // For each i in [0,n-1], store in "sizes[i]", the approximate + // file system space used by keys in "[range[i].start .. range[i].limit)". + // + // Note that the returned sizes measure file system space usage, so + // if the user data compresses by a factor of ten, the returned + // sizes will be one-tenth the size of the corresponding user data size. + // + // The results may not include the sizes of recently written data. + virtual void GetApproximateSizes(const Range* range, int n, + uint64_t* sizes) = 0; + + // Compact the underlying storage for the key range [*begin,*end]. + // In particular, deleted and overwritten versions are discarded, + // and the data is rearranged to reduce the cost of operations + // needed to access the data. This operation should typically only + // be invoked by users who understand the underlying implementation. + // + // begin==NULL is treated as a key before all keys in the database. + // end==NULL is treated as a key after all keys in the database. + // Therefore the following call will compact the entire database: + // db->CompactRange(NULL, NULL); + virtual void CompactRange(const Slice* begin, const Slice* end) = 0; + + private: + // No copying allowed + DB(const DB&); + void operator=(const DB&); +}; + +// Destroy the contents of the specified database. +// Be very careful using this method. +Status DestroyDB(const std::string& name, const Options& options); + +// If a DB cannot be opened, you may attempt to call this method to +// resurrect as much of the contents of the database as possible. +// Some data may be lost, so be careful when calling this function +// on a database that contains important information. +Status RepairDB(const std::string& dbname, const Options& options); + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_INCLUDE_DB_H_ diff --git a/src/leveldb/include/leveldb/dumpfile.h b/src/leveldb/include/leveldb/dumpfile.h new file mode 100644 index 00000000..3f97fda1 --- /dev/null +++ b/src/leveldb/include/leveldb/dumpfile.h @@ -0,0 +1,25 @@ +// Copyright (c) 2014 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_INCLUDE_DUMPFILE_H_ +#define STORAGE_LEVELDB_INCLUDE_DUMPFILE_H_ + +#include +#include "leveldb/env.h" +#include "leveldb/status.h" + +namespace leveldb { + +// Dump the contents of the file named by fname in text format to +// *dst. Makes a sequence of dst->Append() calls; each call is passed +// the newline-terminated text corresponding to a single item found +// in the file. +// +// Returns a non-OK result if fname does not name a leveldb storage +// file, or if the file cannot be read. +Status DumpFile(Env* env, const std::string& fname, WritableFile* dst); + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_INCLUDE_DUMPFILE_H_ diff --git a/src/leveldb/include/leveldb/env.h b/src/leveldb/include/leveldb/env.h new file mode 100644 index 00000000..f709514d --- /dev/null +++ b/src/leveldb/include/leveldb/env.h @@ -0,0 +1,333 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// An Env is an interface used by the leveldb implementation to access +// operating system functionality like the filesystem etc. Callers +// may wish to provide a custom Env object when opening a database to +// get fine gain control; e.g., to rate limit file system operations. +// +// All Env implementations are safe for concurrent access from +// multiple threads without any external synchronization. + +#ifndef STORAGE_LEVELDB_INCLUDE_ENV_H_ +#define STORAGE_LEVELDB_INCLUDE_ENV_H_ + +#include +#include +#include +#include +#include "leveldb/status.h" + +namespace leveldb { + +class FileLock; +class Logger; +class RandomAccessFile; +class SequentialFile; +class Slice; +class WritableFile; + +class Env { + public: + Env() { } + virtual ~Env(); + + // Return a default environment suitable for the current operating + // system. Sophisticated users may wish to provide their own Env + // implementation instead of relying on this default environment. + // + // The result of Default() belongs to leveldb and must never be deleted. + static Env* Default(); + + // Create a brand new sequentially-readable file with the specified name. + // On success, stores a pointer to the new file in *result and returns OK. + // On failure stores NULL in *result and returns non-OK. If the file does + // not exist, returns a non-OK status. + // + // The returned file will only be accessed by one thread at a time. + virtual Status NewSequentialFile(const std::string& fname, + SequentialFile** result) = 0; + + // Create a brand new random access read-only file with the + // specified name. On success, stores a pointer to the new file in + // *result and returns OK. On failure stores NULL in *result and + // returns non-OK. If the file does not exist, returns a non-OK + // status. + // + // The returned file may be concurrently accessed by multiple threads. + virtual Status NewRandomAccessFile(const std::string& fname, + RandomAccessFile** result) = 0; + + // Create an object that writes to a new file with the specified + // name. Deletes any existing file with the same name and creates a + // new file. On success, stores a pointer to the new file in + // *result and returns OK. On failure stores NULL in *result and + // returns non-OK. + // + // The returned file will only be accessed by one thread at a time. + virtual Status NewWritableFile(const std::string& fname, + WritableFile** result) = 0; + + // Returns true iff the named file exists. + virtual bool FileExists(const std::string& fname) = 0; + + // Store in *result the names of the children of the specified directory. + // The names are relative to "dir". + // Original contents of *results are dropped. + virtual Status GetChildren(const std::string& dir, + std::vector* result) = 0; + + // Delete the named file. + virtual Status DeleteFile(const std::string& fname) = 0; + + // Create the specified directory. + virtual Status CreateDir(const std::string& dirname) = 0; + + // Delete the specified directory. + virtual Status DeleteDir(const std::string& dirname) = 0; + + // Store the size of fname in *file_size. + virtual Status GetFileSize(const std::string& fname, uint64_t* file_size) = 0; + + // Rename file src to target. + virtual Status RenameFile(const std::string& src, + const std::string& target) = 0; + + // Lock the specified file. Used to prevent concurrent access to + // the same db by multiple processes. On failure, stores NULL in + // *lock and returns non-OK. + // + // On success, stores a pointer to the object that represents the + // acquired lock in *lock and returns OK. The caller should call + // UnlockFile(*lock) to release the lock. If the process exits, + // the lock will be automatically released. + // + // If somebody else already holds the lock, finishes immediately + // with a failure. I.e., this call does not wait for existing locks + // to go away. + // + // May create the named file if it does not already exist. + virtual Status LockFile(const std::string& fname, FileLock** lock) = 0; + + // Release the lock acquired by a previous successful call to LockFile. + // REQUIRES: lock was returned by a successful LockFile() call + // REQUIRES: lock has not already been unlocked. + virtual Status UnlockFile(FileLock* lock) = 0; + + // Arrange to run "(*function)(arg)" once in a background thread. + // + // "function" may run in an unspecified thread. Multiple functions + // added to the same Env may run concurrently in different threads. + // I.e., the caller may not assume that background work items are + // serialized. + virtual void Schedule( + void (*function)(void* arg), + void* arg) = 0; + + // Start a new thread, invoking "function(arg)" within the new thread. + // When "function(arg)" returns, the thread will be destroyed. + virtual void StartThread(void (*function)(void* arg), void* arg) = 0; + + // *path is set to a temporary directory that can be used for testing. It may + // or many not have just been created. The directory may or may not differ + // between runs of the same process, but subsequent calls will return the + // same directory. + virtual Status GetTestDirectory(std::string* path) = 0; + + // Create and return a log file for storing informational messages. + virtual Status NewLogger(const std::string& fname, Logger** result) = 0; + + // Returns the number of micro-seconds since some fixed point in time. Only + // useful for computing deltas of time. + virtual uint64_t NowMicros() = 0; + + // Sleep/delay the thread for the prescribed number of micro-seconds. + virtual void SleepForMicroseconds(int micros) = 0; + + private: + // No copying allowed + Env(const Env&); + void operator=(const Env&); +}; + +// A file abstraction for reading sequentially through a file +class SequentialFile { + public: + SequentialFile() { } + virtual ~SequentialFile(); + + // Read up to "n" bytes from the file. "scratch[0..n-1]" may be + // written by this routine. Sets "*result" to the data that was + // read (including if fewer than "n" bytes were successfully read). + // May set "*result" to point at data in "scratch[0..n-1]", so + // "scratch[0..n-1]" must be live when "*result" is used. + // If an error was encountered, returns a non-OK status. + // + // REQUIRES: External synchronization + virtual Status Read(size_t n, Slice* result, char* scratch) = 0; + + // Skip "n" bytes from the file. This is guaranteed to be no + // slower that reading the same data, but may be faster. + // + // If end of file is reached, skipping will stop at the end of the + // file, and Skip will return OK. + // + // REQUIRES: External synchronization + virtual Status Skip(uint64_t n) = 0; + + private: + // No copying allowed + SequentialFile(const SequentialFile&); + void operator=(const SequentialFile&); +}; + +// A file abstraction for randomly reading the contents of a file. +class RandomAccessFile { + public: + RandomAccessFile() { } + virtual ~RandomAccessFile(); + + // Read up to "n" bytes from the file starting at "offset". + // "scratch[0..n-1]" may be written by this routine. Sets "*result" + // to the data that was read (including if fewer than "n" bytes were + // successfully read). May set "*result" to point at data in + // "scratch[0..n-1]", so "scratch[0..n-1]" must be live when + // "*result" is used. If an error was encountered, returns a non-OK + // status. + // + // Safe for concurrent use by multiple threads. + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const = 0; + + private: + // No copying allowed + RandomAccessFile(const RandomAccessFile&); + void operator=(const RandomAccessFile&); +}; + +// A file abstraction for sequential writing. The implementation +// must provide buffering since callers may append small fragments +// at a time to the file. +class WritableFile { + public: + WritableFile() { } + virtual ~WritableFile(); + + virtual Status Append(const Slice& data) = 0; + virtual Status Close() = 0; + virtual Status Flush() = 0; + virtual Status Sync() = 0; + + private: + // No copying allowed + WritableFile(const WritableFile&); + void operator=(const WritableFile&); +}; + +// An interface for writing log messages. +class Logger { + public: + Logger() { } + virtual ~Logger(); + + // Write an entry to the log file with the specified format. + virtual void Logv(const char* format, va_list ap) = 0; + + private: + // No copying allowed + Logger(const Logger&); + void operator=(const Logger&); +}; + + +// Identifies a locked file. +class FileLock { + public: + FileLock() { } + virtual ~FileLock(); + private: + // No copying allowed + FileLock(const FileLock&); + void operator=(const FileLock&); +}; + +// Log the specified data to *info_log if info_log is non-NULL. +extern void Log(Logger* info_log, const char* format, ...) +# if defined(__GNUC__) || defined(__clang__) + __attribute__((__format__ (__printf__, 2, 3))) +# endif + ; + +// A utility routine: write "data" to the named file. +extern Status WriteStringToFile(Env* env, const Slice& data, + const std::string& fname); + +// A utility routine: read contents of named file into *data +extern Status ReadFileToString(Env* env, const std::string& fname, + std::string* data); + +// An implementation of Env that forwards all calls to another Env. +// May be useful to clients who wish to override just part of the +// functionality of another Env. +class EnvWrapper : public Env { + public: + // Initialize an EnvWrapper that delegates all calls to *t + explicit EnvWrapper(Env* t) : target_(t) { } + virtual ~EnvWrapper(); + + // Return the target to which this Env forwards all calls + Env* target() const { return target_; } + + // The following text is boilerplate that forwards all methods to target() + Status NewSequentialFile(const std::string& f, SequentialFile** r) { + return target_->NewSequentialFile(f, r); + } + Status NewRandomAccessFile(const std::string& f, RandomAccessFile** r) { + return target_->NewRandomAccessFile(f, r); + } + Status NewWritableFile(const std::string& f, WritableFile** r) { + return target_->NewWritableFile(f, r); + } + bool FileExists(const std::string& f) { return target_->FileExists(f); } + Status GetChildren(const std::string& dir, std::vector* r) { + return target_->GetChildren(dir, r); + } + Status DeleteFile(const std::string& f) { return target_->DeleteFile(f); } + Status CreateDir(const std::string& d) { return target_->CreateDir(d); } + Status DeleteDir(const std::string& d) { return target_->DeleteDir(d); } + Status GetFileSize(const std::string& f, uint64_t* s) { + return target_->GetFileSize(f, s); + } + Status RenameFile(const std::string& s, const std::string& t) { + return target_->RenameFile(s, t); + } + Status LockFile(const std::string& f, FileLock** l) { + return target_->LockFile(f, l); + } + Status UnlockFile(FileLock* l) { return target_->UnlockFile(l); } + void Schedule(void (*f)(void*), void* a) { + return target_->Schedule(f, a); + } + void StartThread(void (*f)(void*), void* a) { + return target_->StartThread(f, a); + } + virtual Status GetTestDirectory(std::string* path) { + return target_->GetTestDirectory(path); + } + virtual Status NewLogger(const std::string& fname, Logger** result) { + return target_->NewLogger(fname, result); + } + uint64_t NowMicros() { + return target_->NowMicros(); + } + void SleepForMicroseconds(int micros) { + target_->SleepForMicroseconds(micros); + } + private: + Env* target_; +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_INCLUDE_ENV_H_ diff --git a/src/leveldb/include/leveldb/filter_policy.h b/src/leveldb/include/leveldb/filter_policy.h new file mode 100644 index 00000000..1fba0800 --- /dev/null +++ b/src/leveldb/include/leveldb/filter_policy.h @@ -0,0 +1,70 @@ +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// A database can be configured with a custom FilterPolicy object. +// This object is responsible for creating a small filter from a set +// of keys. These filters are stored in leveldb and are consulted +// automatically by leveldb to decide whether or not to read some +// information from disk. In many cases, a filter can cut down the +// number of disk seeks form a handful to a single disk seek per +// DB::Get() call. +// +// Most people will want to use the builtin bloom filter support (see +// NewBloomFilterPolicy() below). + +#ifndef STORAGE_LEVELDB_INCLUDE_FILTER_POLICY_H_ +#define STORAGE_LEVELDB_INCLUDE_FILTER_POLICY_H_ + +#include + +namespace leveldb { + +class Slice; + +class FilterPolicy { + public: + virtual ~FilterPolicy(); + + // Return the name of this policy. Note that if the filter encoding + // changes in an incompatible way, the name returned by this method + // must be changed. Otherwise, old incompatible filters may be + // passed to methods of this type. + virtual const char* Name() const = 0; + + // keys[0,n-1] contains a list of keys (potentially with duplicates) + // that are ordered according to the user supplied comparator. + // Append a filter that summarizes keys[0,n-1] to *dst. + // + // Warning: do not change the initial contents of *dst. Instead, + // append the newly constructed filter to *dst. + virtual void CreateFilter(const Slice* keys, int n, std::string* dst) + const = 0; + + // "filter" contains the data appended by a preceding call to + // CreateFilter() on this class. This method must return true if + // the key was in the list of keys passed to CreateFilter(). + // This method may return true or false if the key was not on the + // list, but it should aim to return false with a high probability. + virtual bool KeyMayMatch(const Slice& key, const Slice& filter) const = 0; +}; + +// Return a new filter policy that uses a bloom filter with approximately +// the specified number of bits per key. A good value for bits_per_key +// is 10, which yields a filter with ~ 1% false positive rate. +// +// Callers must delete the result after any database that is using the +// result has been closed. +// +// Note: if you are using a custom comparator that ignores some parts +// of the keys being compared, you must not use NewBloomFilterPolicy() +// and must provide your own FilterPolicy that also ignores the +// corresponding parts of the keys. For example, if the comparator +// ignores trailing spaces, it would be incorrect to use a +// FilterPolicy (like NewBloomFilterPolicy) that does not ignore +// trailing spaces in keys. +extern const FilterPolicy* NewBloomFilterPolicy(int bits_per_key); + +} + +#endif // STORAGE_LEVELDB_INCLUDE_FILTER_POLICY_H_ diff --git a/src/leveldb/include/leveldb/iterator.h b/src/leveldb/include/leveldb/iterator.h new file mode 100644 index 00000000..76aced04 --- /dev/null +++ b/src/leveldb/include/leveldb/iterator.h @@ -0,0 +1,100 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// An iterator yields a sequence of key/value pairs from a source. +// The following class defines the interface. Multiple implementations +// are provided by this library. In particular, iterators are provided +// to access the contents of a Table or a DB. +// +// Multiple threads can invoke const methods on an Iterator without +// external synchronization, but if any of the threads may call a +// non-const method, all threads accessing the same Iterator must use +// external synchronization. + +#ifndef STORAGE_LEVELDB_INCLUDE_ITERATOR_H_ +#define STORAGE_LEVELDB_INCLUDE_ITERATOR_H_ + +#include "leveldb/slice.h" +#include "leveldb/status.h" + +namespace leveldb { + +class Iterator { + public: + Iterator(); + virtual ~Iterator(); + + // An iterator is either positioned at a key/value pair, or + // not valid. This method returns true iff the iterator is valid. + virtual bool Valid() const = 0; + + // Position at the first key in the source. The iterator is Valid() + // after this call iff the source is not empty. + virtual void SeekToFirst() = 0; + + // Position at the last key in the source. The iterator is + // Valid() after this call iff the source is not empty. + virtual void SeekToLast() = 0; + + // Position at the first key in the source that at or past target + // The iterator is Valid() after this call iff the source contains + // an entry that comes at or past target. + virtual void Seek(const Slice& target) = 0; + + // Moves to the next entry in the source. After this call, Valid() is + // true iff the iterator was not positioned at the last entry in the source. + // REQUIRES: Valid() + virtual void Next() = 0; + + // Moves to the previous entry in the source. After this call, Valid() is + // true iff the iterator was not positioned at the first entry in source. + // REQUIRES: Valid() + virtual void Prev() = 0; + + // Return the key for the current entry. The underlying storage for + // the returned slice is valid only until the next modification of + // the iterator. + // REQUIRES: Valid() + virtual Slice key() const = 0; + + // Return the value for the current entry. The underlying storage for + // the returned slice is valid only until the next modification of + // the iterator. + // REQUIRES: Valid() + virtual Slice value() const = 0; + + // If an error has occurred, return it. Else return an ok status. + virtual Status status() const = 0; + + // Clients are allowed to register function/arg1/arg2 triples that + // will be invoked when this iterator is destroyed. + // + // Note that unlike all of the preceding methods, this method is + // not abstract and therefore clients should not override it. + typedef void (*CleanupFunction)(void* arg1, void* arg2); + void RegisterCleanup(CleanupFunction function, void* arg1, void* arg2); + + private: + struct Cleanup { + CleanupFunction function; + void* arg1; + void* arg2; + Cleanup* next; + }; + Cleanup cleanup_; + + // No copying allowed + Iterator(const Iterator&); + void operator=(const Iterator&); +}; + +// Return an empty iterator (yields nothing). +extern Iterator* NewEmptyIterator(); + +// Return an empty iterator with the specified status. +extern Iterator* NewErrorIterator(const Status& status); + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_INCLUDE_ITERATOR_H_ diff --git a/src/leveldb/include/leveldb/options.h b/src/leveldb/include/leveldb/options.h new file mode 100644 index 00000000..7c9b9734 --- /dev/null +++ b/src/leveldb/include/leveldb/options.h @@ -0,0 +1,195 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_INCLUDE_OPTIONS_H_ +#define STORAGE_LEVELDB_INCLUDE_OPTIONS_H_ + +#include + +namespace leveldb { + +class Cache; +class Comparator; +class Env; +class FilterPolicy; +class Logger; +class Snapshot; + +// DB contents are stored in a set of blocks, each of which holds a +// sequence of key,value pairs. Each block may be compressed before +// being stored in a file. The following enum describes which +// compression method (if any) is used to compress a block. +enum CompressionType { + // NOTE: do not change the values of existing entries, as these are + // part of the persistent format on disk. + kNoCompression = 0x0, + kSnappyCompression = 0x1 +}; + +// Options to control the behavior of a database (passed to DB::Open) +struct Options { + // ------------------- + // Parameters that affect behavior + + // Comparator used to define the order of keys in the table. + // Default: a comparator that uses lexicographic byte-wise ordering + // + // REQUIRES: The client must ensure that the comparator supplied + // here has the same name and orders keys *exactly* the same as the + // comparator provided to previous open calls on the same DB. + const Comparator* comparator; + + // If true, the database will be created if it is missing. + // Default: false + bool create_if_missing; + + // If true, an error is raised if the database already exists. + // Default: false + bool error_if_exists; + + // If true, the implementation will do aggressive checking of the + // data it is processing and will stop early if it detects any + // errors. This may have unforeseen ramifications: for example, a + // corruption of one DB entry may cause a large number of entries to + // become unreadable or for the entire DB to become unopenable. + // Default: false + bool paranoid_checks; + + // Use the specified object to interact with the environment, + // e.g. to read/write files, schedule background work, etc. + // Default: Env::Default() + Env* env; + + // Any internal progress/error information generated by the db will + // be written to info_log if it is non-NULL, or to a file stored + // in the same directory as the DB contents if info_log is NULL. + // Default: NULL + Logger* info_log; + + // ------------------- + // Parameters that affect performance + + // Amount of data to build up in memory (backed by an unsorted log + // on disk) before converting to a sorted on-disk file. + // + // Larger values increase performance, especially during bulk loads. + // Up to two write buffers may be held in memory at the same time, + // so you may wish to adjust this parameter to control memory usage. + // Also, a larger write buffer will result in a longer recovery time + // the next time the database is opened. + // + // Default: 4MB + size_t write_buffer_size; + + // Number of open files that can be used by the DB. You may need to + // increase this if your database has a large working set (budget + // one open file per 2MB of working set). + // + // Default: 1000 + int max_open_files; + + // Control over blocks (user data is stored in a set of blocks, and + // a block is the unit of reading from disk). + + // If non-NULL, use the specified cache for blocks. + // If NULL, leveldb will automatically create and use an 8MB internal cache. + // Default: NULL + Cache* block_cache; + + // Approximate size of user data packed per block. Note that the + // block size specified here corresponds to uncompressed data. The + // actual size of the unit read from disk may be smaller if + // compression is enabled. This parameter can be changed dynamically. + // + // Default: 4K + size_t block_size; + + // Number of keys between restart points for delta encoding of keys. + // This parameter can be changed dynamically. Most clients should + // leave this parameter alone. + // + // Default: 16 + int block_restart_interval; + + // Compress blocks using the specified compression algorithm. This + // parameter can be changed dynamically. + // + // Default: kSnappyCompression, which gives lightweight but fast + // compression. + // + // Typical speeds of kSnappyCompression on an Intel(R) Core(TM)2 2.4GHz: + // ~200-500MB/s compression + // ~400-800MB/s decompression + // Note that these speeds are significantly faster than most + // persistent storage speeds, and therefore it is typically never + // worth switching to kNoCompression. Even if the input data is + // incompressible, the kSnappyCompression implementation will + // efficiently detect that and will switch to uncompressed mode. + CompressionType compression; + + // If non-NULL, use the specified filter policy to reduce disk reads. + // Many applications will benefit from passing the result of + // NewBloomFilterPolicy() here. + // + // Default: NULL + const FilterPolicy* filter_policy; + + // Create an Options object with default values for all fields. + Options(); +}; + +// Options that control read operations +struct ReadOptions { + // If true, all data read from underlying storage will be + // verified against corresponding checksums. + // Default: false + bool verify_checksums; + + // Should the data read for this iteration be cached in memory? + // Callers may wish to set this field to false for bulk scans. + // Default: true + bool fill_cache; + + // If "snapshot" is non-NULL, read as of the supplied snapshot + // (which must belong to the DB that is being read and which must + // not have been released). If "snapshot" is NULL, use an implicit + // snapshot of the state at the beginning of this read operation. + // Default: NULL + const Snapshot* snapshot; + + ReadOptions() + : verify_checksums(false), + fill_cache(true), + snapshot(NULL) { + } +}; + +// Options that control write operations +struct WriteOptions { + // If true, the write will be flushed from the operating system + // buffer cache (by calling WritableFile::Sync()) before the write + // is considered complete. If this flag is true, writes will be + // slower. + // + // If this flag is false, and the machine crashes, some recent + // writes may be lost. Note that if it is just the process that + // crashes (i.e., the machine does not reboot), no writes will be + // lost even if sync==false. + // + // In other words, a DB write with sync==false has similar + // crash semantics as the "write()" system call. A DB write + // with sync==true has similar crash semantics to a "write()" + // system call followed by "fsync()". + // + // Default: false + bool sync; + + WriteOptions() + : sync(false) { + } +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_INCLUDE_OPTIONS_H_ diff --git a/src/leveldb/include/leveldb/slice.h b/src/leveldb/include/leveldb/slice.h new file mode 100644 index 00000000..bc367986 --- /dev/null +++ b/src/leveldb/include/leveldb/slice.h @@ -0,0 +1,109 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// Slice is a simple structure containing a pointer into some external +// storage and a size. The user of a Slice must ensure that the slice +// is not used after the corresponding external storage has been +// deallocated. +// +// Multiple threads can invoke const methods on a Slice without +// external synchronization, but if any of the threads may call a +// non-const method, all threads accessing the same Slice must use +// external synchronization. + +#ifndef STORAGE_LEVELDB_INCLUDE_SLICE_H_ +#define STORAGE_LEVELDB_INCLUDE_SLICE_H_ + +#include +#include +#include +#include + +namespace leveldb { + +class Slice { + public: + // Create an empty slice. + Slice() : data_(""), size_(0) { } + + // Create a slice that refers to d[0,n-1]. + Slice(const char* d, size_t n) : data_(d), size_(n) { } + + // Create a slice that refers to the contents of "s" + Slice(const std::string& s) : data_(s.data()), size_(s.size()) { } + + // Create a slice that refers to s[0,strlen(s)-1] + Slice(const char* s) : data_(s), size_(strlen(s)) { } + + // Return a pointer to the beginning of the referenced data + const char* data() const { return data_; } + + // Return the length (in bytes) of the referenced data + size_t size() const { return size_; } + + // Return true iff the length of the referenced data is zero + bool empty() const { return size_ == 0; } + + // Return the ith byte in the referenced data. + // REQUIRES: n < size() + char operator[](size_t n) const { + assert(n < size()); + return data_[n]; + } + + // Change this slice to refer to an empty array + void clear() { data_ = ""; size_ = 0; } + + // Drop the first "n" bytes from this slice. + void remove_prefix(size_t n) { + assert(n <= size()); + data_ += n; + size_ -= n; + } + + // Return a string that contains the copy of the referenced data. + std::string ToString() const { return std::string(data_, size_); } + + // Three-way comparison. Returns value: + // < 0 iff "*this" < "b", + // == 0 iff "*this" == "b", + // > 0 iff "*this" > "b" + int compare(const Slice& b) const; + + // Return true iff "x" is a prefix of "*this" + bool starts_with(const Slice& x) const { + return ((size_ >= x.size_) && + (memcmp(data_, x.data_, x.size_) == 0)); + } + + private: + const char* data_; + size_t size_; + + // Intentionally copyable +}; + +inline bool operator==(const Slice& x, const Slice& y) { + return ((x.size() == y.size()) && + (memcmp(x.data(), y.data(), x.size()) == 0)); +} + +inline bool operator!=(const Slice& x, const Slice& y) { + return !(x == y); +} + +inline int Slice::compare(const Slice& b) const { + const size_t min_len = (size_ < b.size_) ? size_ : b.size_; + int r = memcmp(data_, b.data_, min_len); + if (r == 0) { + if (size_ < b.size_) r = -1; + else if (size_ > b.size_) r = +1; + } + return r; +} + +} // namespace leveldb + + +#endif // STORAGE_LEVELDB_INCLUDE_SLICE_H_ diff --git a/src/leveldb/include/leveldb/status.h b/src/leveldb/include/leveldb/status.h new file mode 100644 index 00000000..11dbd4b4 --- /dev/null +++ b/src/leveldb/include/leveldb/status.h @@ -0,0 +1,106 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// A Status encapsulates the result of an operation. It may indicate success, +// or it may indicate an error with an associated error message. +// +// Multiple threads can invoke const methods on a Status without +// external synchronization, but if any of the threads may call a +// non-const method, all threads accessing the same Status must use +// external synchronization. + +#ifndef STORAGE_LEVELDB_INCLUDE_STATUS_H_ +#define STORAGE_LEVELDB_INCLUDE_STATUS_H_ + +#include +#include "leveldb/slice.h" + +namespace leveldb { + +class Status { + public: + // Create a success status. + Status() : state_(NULL) { } + ~Status() { delete[] state_; } + + // Copy the specified status. + Status(const Status& s); + void operator=(const Status& s); + + // Return a success status. + static Status OK() { return Status(); } + + // Return error status of an appropriate type. + static Status NotFound(const Slice& msg, const Slice& msg2 = Slice()) { + return Status(kNotFound, msg, msg2); + } + static Status Corruption(const Slice& msg, const Slice& msg2 = Slice()) { + return Status(kCorruption, msg, msg2); + } + static Status NotSupported(const Slice& msg, const Slice& msg2 = Slice()) { + return Status(kNotSupported, msg, msg2); + } + static Status InvalidArgument(const Slice& msg, const Slice& msg2 = Slice()) { + return Status(kInvalidArgument, msg, msg2); + } + static Status IOError(const Slice& msg, const Slice& msg2 = Slice()) { + return Status(kIOError, msg, msg2); + } + + // Returns true iff the status indicates success. + bool ok() const { return (state_ == NULL); } + + // Returns true iff the status indicates a NotFound error. + bool IsNotFound() const { return code() == kNotFound; } + + // Returns true iff the status indicates a Corruption error. + bool IsCorruption() const { return code() == kCorruption; } + + // Returns true iff the status indicates an IOError. + bool IsIOError() const { return code() == kIOError; } + + // Return a string representation of this status suitable for printing. + // Returns the string "OK" for success. + std::string ToString() const; + + private: + // OK status has a NULL state_. Otherwise, state_ is a new[] array + // of the following form: + // state_[0..3] == length of message + // state_[4] == code + // state_[5..] == message + const char* state_; + + enum Code { + kOk = 0, + kNotFound = 1, + kCorruption = 2, + kNotSupported = 3, + kInvalidArgument = 4, + kIOError = 5 + }; + + Code code() const { + return (state_ == NULL) ? kOk : static_cast(state_[4]); + } + + Status(Code code, const Slice& msg, const Slice& msg2); + static const char* CopyState(const char* s); +}; + +inline Status::Status(const Status& s) { + state_ = (s.state_ == NULL) ? NULL : CopyState(s.state_); +} +inline void Status::operator=(const Status& s) { + // The following condition catches both aliasing (when this == &s), + // and the common case where both s and *this are ok. + if (state_ != s.state_) { + delete[] state_; + state_ = (s.state_ == NULL) ? NULL : CopyState(s.state_); + } +} + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_INCLUDE_STATUS_H_ diff --git a/src/leveldb/include/leveldb/table.h b/src/leveldb/include/leveldb/table.h new file mode 100644 index 00000000..a9746c3f --- /dev/null +++ b/src/leveldb/include/leveldb/table.h @@ -0,0 +1,85 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_INCLUDE_TABLE_H_ +#define STORAGE_LEVELDB_INCLUDE_TABLE_H_ + +#include +#include "leveldb/iterator.h" + +namespace leveldb { + +class Block; +class BlockHandle; +class Footer; +struct Options; +class RandomAccessFile; +struct ReadOptions; +class TableCache; + +// A Table is a sorted map from strings to strings. Tables are +// immutable and persistent. A Table may be safely accessed from +// multiple threads without external synchronization. +class Table { + public: + // Attempt to open the table that is stored in bytes [0..file_size) + // of "file", and read the metadata entries necessary to allow + // retrieving data from the table. + // + // If successful, returns ok and sets "*table" to the newly opened + // table. The client should delete "*table" when no longer needed. + // If there was an error while initializing the table, sets "*table" + // to NULL and returns a non-ok status. Does not take ownership of + // "*source", but the client must ensure that "source" remains live + // for the duration of the returned table's lifetime. + // + // *file must remain live while this Table is in use. + static Status Open(const Options& options, + RandomAccessFile* file, + uint64_t file_size, + Table** table); + + ~Table(); + + // Returns a new iterator over the table contents. + // The result of NewIterator() is initially invalid (caller must + // call one of the Seek methods on the iterator before using it). + Iterator* NewIterator(const ReadOptions&) const; + + // Given a key, return an approximate byte offset in the file where + // the data for that key begins (or would begin if the key were + // present in the file). The returned value is in terms of file + // bytes, and so includes effects like compression of the underlying data. + // E.g., the approximate offset of the last key in the table will + // be close to the file length. + uint64_t ApproximateOffsetOf(const Slice& key) const; + + private: + struct Rep; + Rep* rep_; + + explicit Table(Rep* rep) { rep_ = rep; } + static Iterator* BlockReader(void*, const ReadOptions&, const Slice&); + + // Calls (*handle_result)(arg, ...) with the entry found after a call + // to Seek(key). May not make such a call if filter policy says + // that key is not present. + friend class TableCache; + Status InternalGet( + const ReadOptions&, const Slice& key, + void* arg, + void (*handle_result)(void* arg, const Slice& k, const Slice& v)); + + + void ReadMeta(const Footer& footer); + void ReadFilter(const Slice& filter_handle_value); + + // No copying allowed + Table(const Table&); + void operator=(const Table&); +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_INCLUDE_TABLE_H_ diff --git a/src/leveldb/include/leveldb/table_builder.h b/src/leveldb/include/leveldb/table_builder.h new file mode 100644 index 00000000..5fd1dc71 --- /dev/null +++ b/src/leveldb/include/leveldb/table_builder.h @@ -0,0 +1,92 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// TableBuilder provides the interface used to build a Table +// (an immutable and sorted map from keys to values). +// +// Multiple threads can invoke const methods on a TableBuilder without +// external synchronization, but if any of the threads may call a +// non-const method, all threads accessing the same TableBuilder must use +// external synchronization. + +#ifndef STORAGE_LEVELDB_INCLUDE_TABLE_BUILDER_H_ +#define STORAGE_LEVELDB_INCLUDE_TABLE_BUILDER_H_ + +#include +#include "leveldb/options.h" +#include "leveldb/status.h" + +namespace leveldb { + +class BlockBuilder; +class BlockHandle; +class WritableFile; + +class TableBuilder { + public: + // Create a builder that will store the contents of the table it is + // building in *file. Does not close the file. It is up to the + // caller to close the file after calling Finish(). + TableBuilder(const Options& options, WritableFile* file); + + // REQUIRES: Either Finish() or Abandon() has been called. + ~TableBuilder(); + + // Change the options used by this builder. Note: only some of the + // option fields can be changed after construction. If a field is + // not allowed to change dynamically and its value in the structure + // passed to the constructor is different from its value in the + // structure passed to this method, this method will return an error + // without changing any fields. + Status ChangeOptions(const Options& options); + + // Add key,value to the table being constructed. + // REQUIRES: key is after any previously added key according to comparator. + // REQUIRES: Finish(), Abandon() have not been called + void Add(const Slice& key, const Slice& value); + + // Advanced operation: flush any buffered key/value pairs to file. + // Can be used to ensure that two adjacent entries never live in + // the same data block. Most clients should not need to use this method. + // REQUIRES: Finish(), Abandon() have not been called + void Flush(); + + // Return non-ok iff some error has been detected. + Status status() const; + + // Finish building the table. Stops using the file passed to the + // constructor after this function returns. + // REQUIRES: Finish(), Abandon() have not been called + Status Finish(); + + // Indicate that the contents of this builder should be abandoned. Stops + // using the file passed to the constructor after this function returns. + // If the caller is not going to call Finish(), it must call Abandon() + // before destroying this builder. + // REQUIRES: Finish(), Abandon() have not been called + void Abandon(); + + // Number of calls to Add() so far. + uint64_t NumEntries() const; + + // Size of the file generated so far. If invoked after a successful + // Finish() call, returns the size of the final generated file. + uint64_t FileSize() const; + + private: + bool ok() const { return status().ok(); } + void WriteBlock(BlockBuilder* block, BlockHandle* handle); + void WriteRawBlock(const Slice& data, CompressionType, BlockHandle* handle); + + struct Rep; + Rep* rep_; + + // No copying allowed + TableBuilder(const TableBuilder&); + void operator=(const TableBuilder&); +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_INCLUDE_TABLE_BUILDER_H_ diff --git a/src/leveldb/include/leveldb/write_batch.h b/src/leveldb/include/leveldb/write_batch.h new file mode 100644 index 00000000..ee9aab68 --- /dev/null +++ b/src/leveldb/include/leveldb/write_batch.h @@ -0,0 +1,64 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// WriteBatch holds a collection of updates to apply atomically to a DB. +// +// The updates are applied in the order in which they are added +// to the WriteBatch. For example, the value of "key" will be "v3" +// after the following batch is written: +// +// batch.Put("key", "v1"); +// batch.Delete("key"); +// batch.Put("key", "v2"); +// batch.Put("key", "v3"); +// +// Multiple threads can invoke const methods on a WriteBatch without +// external synchronization, but if any of the threads may call a +// non-const method, all threads accessing the same WriteBatch must use +// external synchronization. + +#ifndef STORAGE_LEVELDB_INCLUDE_WRITE_BATCH_H_ +#define STORAGE_LEVELDB_INCLUDE_WRITE_BATCH_H_ + +#include +#include "leveldb/status.h" + +namespace leveldb { + +class Slice; + +class WriteBatch { + public: + WriteBatch(); + ~WriteBatch(); + + // Store the mapping "key->value" in the database. + void Put(const Slice& key, const Slice& value); + + // If the database contains a mapping for "key", erase it. Else do nothing. + void Delete(const Slice& key); + + // Clear all updates buffered in this batch. + void Clear(); + + // Support for iterating over the contents of a batch. + class Handler { + public: + virtual ~Handler(); + virtual void Put(const Slice& key, const Slice& value) = 0; + virtual void Delete(const Slice& key) = 0; + }; + Status Iterate(Handler* handler) const; + + private: + friend class WriteBatchInternal; + + std::string rep_; // See comment in write_batch.cc for the format of rep_ + + // Intentionally copyable +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_INCLUDE_WRITE_BATCH_H_ diff --git a/src/leveldb/issues/issue178_test.cc b/src/leveldb/issues/issue178_test.cc new file mode 100644 index 00000000..1b1cf8bb --- /dev/null +++ b/src/leveldb/issues/issue178_test.cc @@ -0,0 +1,92 @@ +// Copyright (c) 2013 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +// Test for issue 178: a manual compaction causes deleted data to reappear. +#include +#include +#include + +#include "leveldb/db.h" +#include "leveldb/write_batch.h" +#include "util/testharness.h" + +namespace { + +const int kNumKeys = 1100000; + +std::string Key1(int i) { + char buf[100]; + snprintf(buf, sizeof(buf), "my_key_%d", i); + return buf; +} + +std::string Key2(int i) { + return Key1(i) + "_xxx"; +} + +class Issue178 { }; + +TEST(Issue178, Test) { + // Get rid of any state from an old run. + std::string dbpath = leveldb::test::TmpDir() + "/leveldb_cbug_test"; + DestroyDB(dbpath, leveldb::Options()); + + // Open database. Disable compression since it affects the creation + // of layers and the code below is trying to test against a very + // specific scenario. + leveldb::DB* db; + leveldb::Options db_options; + db_options.create_if_missing = true; + db_options.compression = leveldb::kNoCompression; + ASSERT_OK(leveldb::DB::Open(db_options, dbpath, &db)); + + // create first key range + leveldb::WriteBatch batch; + for (size_t i = 0; i < kNumKeys; i++) { + batch.Put(Key1(i), "value for range 1 key"); + } + ASSERT_OK(db->Write(leveldb::WriteOptions(), &batch)); + + // create second key range + batch.Clear(); + for (size_t i = 0; i < kNumKeys; i++) { + batch.Put(Key2(i), "value for range 2 key"); + } + ASSERT_OK(db->Write(leveldb::WriteOptions(), &batch)); + + // delete second key range + batch.Clear(); + for (size_t i = 0; i < kNumKeys; i++) { + batch.Delete(Key2(i)); + } + ASSERT_OK(db->Write(leveldb::WriteOptions(), &batch)); + + // compact database + std::string start_key = Key1(0); + std::string end_key = Key1(kNumKeys - 1); + leveldb::Slice least(start_key.data(), start_key.size()); + leveldb::Slice greatest(end_key.data(), end_key.size()); + + // commenting out the line below causes the example to work correctly + db->CompactRange(&least, &greatest); + + // count the keys + leveldb::Iterator* iter = db->NewIterator(leveldb::ReadOptions()); + size_t num_keys = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + num_keys++; + } + delete iter; + ASSERT_EQ(kNumKeys, num_keys) << "Bad number of keys"; + + // close database + delete db; + DestroyDB(dbpath, leveldb::Options()); +} + +} // anonymous namespace + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/issues/issue200_test.cc b/src/leveldb/issues/issue200_test.cc new file mode 100644 index 00000000..1cec79f4 --- /dev/null +++ b/src/leveldb/issues/issue200_test.cc @@ -0,0 +1,59 @@ +// Copyright (c) 2013 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +// Test for issue 200: when iterator switches direction from backward +// to forward, the current key can be yielded unexpectedly if a new +// mutation has been added just before the current key. + +#include "leveldb/db.h" +#include "util/testharness.h" + +namespace leveldb { + +class Issue200 { }; + +TEST(Issue200, Test) { + // Get rid of any state from an old run. + std::string dbpath = test::TmpDir() + "/leveldb_issue200_test"; + DestroyDB(dbpath, Options()); + + DB *db; + Options options; + options.create_if_missing = true; + ASSERT_OK(DB::Open(options, dbpath, &db)); + + WriteOptions write_options; + ASSERT_OK(db->Put(write_options, "1", "b")); + ASSERT_OK(db->Put(write_options, "2", "c")); + ASSERT_OK(db->Put(write_options, "3", "d")); + ASSERT_OK(db->Put(write_options, "4", "e")); + ASSERT_OK(db->Put(write_options, "5", "f")); + + ReadOptions read_options; + Iterator *iter = db->NewIterator(read_options); + + // Add an element that should not be reflected in the iterator. + ASSERT_OK(db->Put(write_options, "25", "cd")); + + iter->Seek("5"); + ASSERT_EQ(iter->key().ToString(), "5"); + iter->Prev(); + ASSERT_EQ(iter->key().ToString(), "4"); + iter->Prev(); + ASSERT_EQ(iter->key().ToString(), "3"); + iter->Next(); + ASSERT_EQ(iter->key().ToString(), "4"); + iter->Next(); + ASSERT_EQ(iter->key().ToString(), "5"); + + delete iter; + delete db; + DestroyDB(dbpath, options); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/port/README b/src/leveldb/port/README new file mode 100644 index 00000000..422563e2 --- /dev/null +++ b/src/leveldb/port/README @@ -0,0 +1,10 @@ +This directory contains interfaces and implementations that isolate the +rest of the package from platform details. + +Code in the rest of the package includes "port.h" from this directory. +"port.h" in turn includes a platform specific "port_.h" file +that provides the platform specific implementation. + +See port_posix.h for an example of what must be provided in a platform +specific header file. + diff --git a/src/leveldb/port/atomic_pointer.h b/src/leveldb/port/atomic_pointer.h new file mode 100644 index 00000000..9bf091f7 --- /dev/null +++ b/src/leveldb/port/atomic_pointer.h @@ -0,0 +1,223 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +// AtomicPointer provides storage for a lock-free pointer. +// Platform-dependent implementation of AtomicPointer: +// - If the platform provides a cheap barrier, we use it with raw pointers +// - If is present (on newer versions of gcc, it is), we use +// a -based AtomicPointer. However we prefer the memory +// barrier based version, because at least on a gcc 4.4 32-bit build +// on linux, we have encountered a buggy implementation. +// Also, some implementations are much slower than a memory-barrier +// based implementation (~16ns for based acquire-load vs. ~1ns for +// a barrier based acquire-load). +// This code is based on atomicops-internals-* in Google's perftools: +// http://code.google.com/p/google-perftools/source/browse/#svn%2Ftrunk%2Fsrc%2Fbase + +#ifndef PORT_ATOMIC_POINTER_H_ +#define PORT_ATOMIC_POINTER_H_ + +#include +#ifdef LEVELDB_ATOMIC_PRESENT +#include +#endif +#ifdef OS_WIN +#include +#endif +#ifdef OS_MACOSX +#include +#endif + +#if defined(_M_X64) || defined(__x86_64__) +#define ARCH_CPU_X86_FAMILY 1 +#elif defined(_M_IX86) || defined(__i386__) || defined(__i386) +#define ARCH_CPU_X86_FAMILY 1 +#elif defined(__ARMEL__) +#define ARCH_CPU_ARM_FAMILY 1 +#elif defined(__ppc__) || defined(__powerpc__) || defined(__powerpc64__) +#define ARCH_CPU_PPC_FAMILY 1 +#endif + +namespace leveldb { +namespace port { + +// Define MemoryBarrier() if available +// Windows on x86 +#if defined(OS_WIN) && defined(COMPILER_MSVC) && defined(ARCH_CPU_X86_FAMILY) +// windows.h already provides a MemoryBarrier(void) macro +// http://msdn.microsoft.com/en-us/library/ms684208(v=vs.85).aspx +#define LEVELDB_HAVE_MEMORY_BARRIER + +// Mac OS +#elif defined(OS_MACOSX) +inline void MemoryBarrier() { + OSMemoryBarrier(); +} +#define LEVELDB_HAVE_MEMORY_BARRIER + +// Gcc on x86 +#elif defined(ARCH_CPU_X86_FAMILY) && defined(__GNUC__) +inline void MemoryBarrier() { + // See http://gcc.gnu.org/ml/gcc/2003-04/msg01180.html for a discussion on + // this idiom. Also see http://en.wikipedia.org/wiki/Memory_ordering. + __asm__ __volatile__("" : : : "memory"); +} +#define LEVELDB_HAVE_MEMORY_BARRIER + +// Sun Studio +#elif defined(ARCH_CPU_X86_FAMILY) && defined(__SUNPRO_CC) +inline void MemoryBarrier() { + // See http://gcc.gnu.org/ml/gcc/2003-04/msg01180.html for a discussion on + // this idiom. Also see http://en.wikipedia.org/wiki/Memory_ordering. + asm volatile("" : : : "memory"); +} +#define LEVELDB_HAVE_MEMORY_BARRIER + +// ARM Linux +#elif defined(ARCH_CPU_ARM_FAMILY) && defined(__linux__) +typedef void (*LinuxKernelMemoryBarrierFunc)(void); +// The Linux ARM kernel provides a highly optimized device-specific memory +// barrier function at a fixed memory address that is mapped in every +// user-level process. +// +// This beats using CPU-specific instructions which are, on single-core +// devices, un-necessary and very costly (e.g. ARMv7-A "dmb" takes more +// than 180ns on a Cortex-A8 like the one on a Nexus One). Benchmarking +// shows that the extra function call cost is completely negligible on +// multi-core devices. +// +inline void MemoryBarrier() { + (*(LinuxKernelMemoryBarrierFunc)0xffff0fa0)(); +} +#define LEVELDB_HAVE_MEMORY_BARRIER + +// PPC +#elif defined(ARCH_CPU_PPC_FAMILY) && defined(__GNUC__) +inline void MemoryBarrier() { + // TODO for some powerpc expert: is there a cheaper suitable variant? + // Perhaps by having separate barriers for acquire and release ops. + asm volatile("sync" : : : "memory"); +} +#define LEVELDB_HAVE_MEMORY_BARRIER + +#endif + +// AtomicPointer built using platform-specific MemoryBarrier() +#if defined(LEVELDB_HAVE_MEMORY_BARRIER) +class AtomicPointer { + private: + void* rep_; + public: + AtomicPointer() { } + explicit AtomicPointer(void* p) : rep_(p) {} + inline void* NoBarrier_Load() const { return rep_; } + inline void NoBarrier_Store(void* v) { rep_ = v; } + inline void* Acquire_Load() const { + void* result = rep_; + MemoryBarrier(); + return result; + } + inline void Release_Store(void* v) { + MemoryBarrier(); + rep_ = v; + } +}; + +// AtomicPointer based on +#elif defined(LEVELDB_ATOMIC_PRESENT) +class AtomicPointer { + private: + std::atomic rep_; + public: + AtomicPointer() { } + explicit AtomicPointer(void* v) : rep_(v) { } + inline void* Acquire_Load() const { + return rep_.load(std::memory_order_acquire); + } + inline void Release_Store(void* v) { + rep_.store(v, std::memory_order_release); + } + inline void* NoBarrier_Load() const { + return rep_.load(std::memory_order_relaxed); + } + inline void NoBarrier_Store(void* v) { + rep_.store(v, std::memory_order_relaxed); + } +}; + +// Atomic pointer based on sparc memory barriers +#elif defined(__sparcv9) && defined(__GNUC__) +class AtomicPointer { + private: + void* rep_; + public: + AtomicPointer() { } + explicit AtomicPointer(void* v) : rep_(v) { } + inline void* Acquire_Load() const { + void* val; + __asm__ __volatile__ ( + "ldx [%[rep_]], %[val] \n\t" + "membar #LoadLoad|#LoadStore \n\t" + : [val] "=r" (val) + : [rep_] "r" (&rep_) + : "memory"); + return val; + } + inline void Release_Store(void* v) { + __asm__ __volatile__ ( + "membar #LoadStore|#StoreStore \n\t" + "stx %[v], [%[rep_]] \n\t" + : + : [rep_] "r" (&rep_), [v] "r" (v) + : "memory"); + } + inline void* NoBarrier_Load() const { return rep_; } + inline void NoBarrier_Store(void* v) { rep_ = v; } +}; + +// Atomic pointer based on ia64 acq/rel +#elif defined(__ia64) && defined(__GNUC__) +class AtomicPointer { + private: + void* rep_; + public: + AtomicPointer() { } + explicit AtomicPointer(void* v) : rep_(v) { } + inline void* Acquire_Load() const { + void* val ; + __asm__ __volatile__ ( + "ld8.acq %[val] = [%[rep_]] \n\t" + : [val] "=r" (val) + : [rep_] "r" (&rep_) + : "memory" + ); + return val; + } + inline void Release_Store(void* v) { + __asm__ __volatile__ ( + "st8.rel [%[rep_]] = %[v] \n\t" + : + : [rep_] "r" (&rep_), [v] "r" (v) + : "memory" + ); + } + inline void* NoBarrier_Load() const { return rep_; } + inline void NoBarrier_Store(void* v) { rep_ = v; } +}; + +// We have neither MemoryBarrier(), nor +#else +#error Please implement AtomicPointer for this platform. + +#endif + +#undef LEVELDB_HAVE_MEMORY_BARRIER +#undef ARCH_CPU_X86_FAMILY +#undef ARCH_CPU_ARM_FAMILY +#undef ARCH_CPU_PPC_FAMILY + +} // namespace port +} // namespace leveldb + +#endif // PORT_ATOMIC_POINTER_H_ diff --git a/src/leveldb/port/port.h b/src/leveldb/port/port.h new file mode 100644 index 00000000..4baafa8e --- /dev/null +++ b/src/leveldb/port/port.h @@ -0,0 +1,21 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_PORT_PORT_H_ +#define STORAGE_LEVELDB_PORT_PORT_H_ + +#include + +// Include the appropriate platform specific file below. If you are +// porting to a new platform, see "port_example.h" for documentation +// of what the new port_.h file must provide. +#if defined(LEVELDB_PLATFORM_POSIX) +# include "port/port_posix.h" +#elif defined(LEVELDB_PLATFORM_CHROMIUM) +# include "port/port_chromium.h" +#elif defined(LEVELDB_PLATFORM_WINDOWS) +# include "port/port_win.h" +#endif + +#endif // STORAGE_LEVELDB_PORT_PORT_H_ diff --git a/src/leveldb/port/port_example.h b/src/leveldb/port/port_example.h new file mode 100644 index 00000000..ab9e489b --- /dev/null +++ b/src/leveldb/port/port_example.h @@ -0,0 +1,135 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// This file contains the specification, but not the implementations, +// of the types/operations/etc. that should be defined by a platform +// specific port_.h file. Use this file as a reference for +// how to port this package to a new platform. + +#ifndef STORAGE_LEVELDB_PORT_PORT_EXAMPLE_H_ +#define STORAGE_LEVELDB_PORT_PORT_EXAMPLE_H_ + +namespace leveldb { +namespace port { + +// TODO(jorlow): Many of these belong more in the environment class rather than +// here. We should try moving them and see if it affects perf. + +// The following boolean constant must be true on a little-endian machine +// and false otherwise. +static const bool kLittleEndian = true /* or some other expression */; + +// ------------------ Threading ------------------- + +// A Mutex represents an exclusive lock. +class Mutex { + public: + Mutex(); + ~Mutex(); + + // Lock the mutex. Waits until other lockers have exited. + // Will deadlock if the mutex is already locked by this thread. + void Lock(); + + // Unlock the mutex. + // REQUIRES: This mutex was locked by this thread. + void Unlock(); + + // Optionally crash if this thread does not hold this mutex. + // The implementation must be fast, especially if NDEBUG is + // defined. The implementation is allowed to skip all checks. + void AssertHeld(); +}; + +class CondVar { + public: + explicit CondVar(Mutex* mu); + ~CondVar(); + + // Atomically release *mu and block on this condition variable until + // either a call to SignalAll(), or a call to Signal() that picks + // this thread to wakeup. + // REQUIRES: this thread holds *mu + void Wait(); + + // If there are some threads waiting, wake up at least one of them. + void Signal(); + + // Wake up all waiting threads. + void SignallAll(); +}; + +// Thread-safe initialization. +// Used as follows: +// static port::OnceType init_control = LEVELDB_ONCE_INIT; +// static void Initializer() { ... do something ...; } +// ... +// port::InitOnce(&init_control, &Initializer); +typedef intptr_t OnceType; +#define LEVELDB_ONCE_INIT 0 +extern void InitOnce(port::OnceType*, void (*initializer)()); + +// A type that holds a pointer that can be read or written atomically +// (i.e., without word-tearing.) +class AtomicPointer { + private: + intptr_t rep_; + public: + // Initialize to arbitrary value + AtomicPointer(); + + // Initialize to hold v + explicit AtomicPointer(void* v) : rep_(v) { } + + // Read and return the stored pointer with the guarantee that no + // later memory access (read or write) by this thread can be + // reordered ahead of this read. + void* Acquire_Load() const; + + // Set v as the stored pointer with the guarantee that no earlier + // memory access (read or write) by this thread can be reordered + // after this store. + void Release_Store(void* v); + + // Read the stored pointer with no ordering guarantees. + void* NoBarrier_Load() const; + + // Set va as the stored pointer with no ordering guarantees. + void NoBarrier_Store(void* v); +}; + +// ------------------ Compression ------------------- + +// Store the snappy compression of "input[0,input_length-1]" in *output. +// Returns false if snappy is not supported by this port. +extern bool Snappy_Compress(const char* input, size_t input_length, + std::string* output); + +// If input[0,input_length-1] looks like a valid snappy compressed +// buffer, store the size of the uncompressed data in *result and +// return true. Else return false. +extern bool Snappy_GetUncompressedLength(const char* input, size_t length, + size_t* result); + +// Attempt to snappy uncompress input[0,input_length-1] into *output. +// Returns true if successful, false if the input is invalid lightweight +// compressed data. +// +// REQUIRES: at least the first "n" bytes of output[] must be writable +// where "n" is the result of a successful call to +// Snappy_GetUncompressedLength. +extern bool Snappy_Uncompress(const char* input_data, size_t input_length, + char* output); + +// ------------------ Miscellaneous ------------------- + +// If heap profiling is not supported, returns false. +// Else repeatedly calls (*func)(arg, data, n) and then returns true. +// The concatenation of all "data[0,n-1]" fragments is the heap profile. +extern bool GetHeapProfile(void (*func)(void*, const char*, int), void* arg); + +} // namespace port +} // namespace leveldb + +#endif // STORAGE_LEVELDB_PORT_PORT_EXAMPLE_H_ diff --git a/src/leveldb/port/port_posix.cc b/src/leveldb/port/port_posix.cc new file mode 100644 index 00000000..5ba127a5 --- /dev/null +++ b/src/leveldb/port/port_posix.cc @@ -0,0 +1,54 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "port/port_posix.h" + +#include +#include +#include +#include "util/logging.h" + +namespace leveldb { +namespace port { + +static void PthreadCall(const char* label, int result) { + if (result != 0) { + fprintf(stderr, "pthread %s: %s\n", label, strerror(result)); + abort(); + } +} + +Mutex::Mutex() { PthreadCall("init mutex", pthread_mutex_init(&mu_, NULL)); } + +Mutex::~Mutex() { PthreadCall("destroy mutex", pthread_mutex_destroy(&mu_)); } + +void Mutex::Lock() { PthreadCall("lock", pthread_mutex_lock(&mu_)); } + +void Mutex::Unlock() { PthreadCall("unlock", pthread_mutex_unlock(&mu_)); } + +CondVar::CondVar(Mutex* mu) + : mu_(mu) { + PthreadCall("init cv", pthread_cond_init(&cv_, NULL)); +} + +CondVar::~CondVar() { PthreadCall("destroy cv", pthread_cond_destroy(&cv_)); } + +void CondVar::Wait() { + PthreadCall("wait", pthread_cond_wait(&cv_, &mu_->mu_)); +} + +void CondVar::Signal() { + PthreadCall("signal", pthread_cond_signal(&cv_)); +} + +void CondVar::SignalAll() { + PthreadCall("broadcast", pthread_cond_broadcast(&cv_)); +} + +void InitOnce(OnceType* once, void (*initializer)()) { + PthreadCall("once", pthread_once(once, initializer)); +} + +} // namespace port +} // namespace leveldb diff --git a/src/leveldb/port/port_posix.h b/src/leveldb/port/port_posix.h new file mode 100644 index 00000000..ccca9939 --- /dev/null +++ b/src/leveldb/port/port_posix.h @@ -0,0 +1,158 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// See port_example.h for documentation for the following types/functions. + +#ifndef STORAGE_LEVELDB_PORT_PORT_POSIX_H_ +#define STORAGE_LEVELDB_PORT_PORT_POSIX_H_ + +#undef PLATFORM_IS_LITTLE_ENDIAN +#if defined(OS_MACOSX) + #include + #if defined(__DARWIN_LITTLE_ENDIAN) && defined(__DARWIN_BYTE_ORDER) + #define PLATFORM_IS_LITTLE_ENDIAN \ + (__DARWIN_BYTE_ORDER == __DARWIN_LITTLE_ENDIAN) + #endif +#elif defined(OS_SOLARIS) + #include + #ifdef _LITTLE_ENDIAN + #define PLATFORM_IS_LITTLE_ENDIAN true + #else + #define PLATFORM_IS_LITTLE_ENDIAN false + #endif +#elif defined(OS_FREEBSD) || defined(OS_OPENBSD) ||\ + defined(OS_NETBSD) || defined(OS_DRAGONFLYBSD) + #include + #include + #define PLATFORM_IS_LITTLE_ENDIAN (_BYTE_ORDER == _LITTLE_ENDIAN) +#elif defined(OS_HPUX) + #define PLATFORM_IS_LITTLE_ENDIAN false +#elif defined(OS_ANDROID) + // Due to a bug in the NDK x86 definition, + // _BYTE_ORDER must be used instead of __BYTE_ORDER on Android. + // See http://code.google.com/p/android/issues/detail?id=39824 + #include + #define PLATFORM_IS_LITTLE_ENDIAN (_BYTE_ORDER == _LITTLE_ENDIAN) +#else + #include +#endif + +#include +#ifdef SNAPPY +#include +#endif +#include +#include +#include "port/atomic_pointer.h" + +#ifndef PLATFORM_IS_LITTLE_ENDIAN +#define PLATFORM_IS_LITTLE_ENDIAN (__BYTE_ORDER == __LITTLE_ENDIAN) +#endif + +#if defined(OS_MACOSX) || defined(OS_SOLARIS) || defined(OS_FREEBSD) ||\ + defined(OS_NETBSD) || defined(OS_OPENBSD) || defined(OS_DRAGONFLYBSD) ||\ + defined(OS_ANDROID) || defined(OS_HPUX) || defined(CYGWIN) +// Use fread/fwrite/fflush on platforms without _unlocked variants +#define fread_unlocked fread +#define fwrite_unlocked fwrite +#define fflush_unlocked fflush +#endif + +#if defined(OS_FREEBSD) ||\ + defined(OS_OPENBSD) || defined(OS_DRAGONFLYBSD) +// Use fsync() on platforms without fdatasync() +#define fdatasync fsync +#endif + +#if defined(OS_MACOSX) +#define fdatasync(fd) fcntl(fd, F_FULLFSYNC, 0) +#endif + +#if defined(OS_ANDROID) && __ANDROID_API__ < 9 +// fdatasync() was only introduced in API level 9 on Android. Use fsync() +// when targetting older platforms. +#define fdatasync fsync +#endif + +namespace leveldb { +namespace port { + +static const bool kLittleEndian = PLATFORM_IS_LITTLE_ENDIAN; +#undef PLATFORM_IS_LITTLE_ENDIAN + +class CondVar; + +class Mutex { + public: + Mutex(); + ~Mutex(); + + void Lock(); + void Unlock(); + void AssertHeld() { } + + private: + friend class CondVar; + pthread_mutex_t mu_; + + // No copying + Mutex(const Mutex&); + void operator=(const Mutex&); +}; + +class CondVar { + public: + explicit CondVar(Mutex* mu); + ~CondVar(); + void Wait(); + void Signal(); + void SignalAll(); + private: + pthread_cond_t cv_; + Mutex* mu_; +}; + +typedef pthread_once_t OnceType; +#define LEVELDB_ONCE_INIT PTHREAD_ONCE_INIT +extern void InitOnce(OnceType* once, void (*initializer)()); + +inline bool Snappy_Compress(const char* input, size_t length, + ::std::string* output) { +#ifdef SNAPPY + output->resize(snappy::MaxCompressedLength(length)); + size_t outlen; + snappy::RawCompress(input, length, &(*output)[0], &outlen); + output->resize(outlen); + return true; +#endif + + return false; +} + +inline bool Snappy_GetUncompressedLength(const char* input, size_t length, + size_t* result) { +#ifdef SNAPPY + return snappy::GetUncompressedLength(input, length, result); +#else + return false; +#endif +} + +inline bool Snappy_Uncompress(const char* input, size_t length, + char* output) { +#ifdef SNAPPY + return snappy::RawUncompress(input, length, output); +#else + return false; +#endif +} + +inline bool GetHeapProfile(void (*func)(void*, const char*, int), void* arg) { + return false; +} + +} // namespace port +} // namespace leveldb + +#endif // STORAGE_LEVELDB_PORT_PORT_POSIX_H_ diff --git a/src/leveldb/port/port_win.cc b/src/leveldb/port/port_win.cc new file mode 100644 index 00000000..1b0f060a --- /dev/null +++ b/src/leveldb/port/port_win.cc @@ -0,0 +1,147 @@ +// LevelDB Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// See port_example.h for documentation for the following types/functions. + +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// * Neither the name of the University of California, Berkeley nor the +// names of its contributors may be used to endorse or promote products +// derived from this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +// WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +// DISCLAIMED. IN NO EVENT SHALL THE REGENTS AND CONTRIBUTORS BE LIABLE FOR ANY +// DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +// (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +// LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +// ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +// SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// + +#include "port/port_win.h" + +#include +#include + +namespace leveldb { +namespace port { + +Mutex::Mutex() : + cs_(NULL) { + assert(!cs_); + cs_ = static_cast(new CRITICAL_SECTION()); + ::InitializeCriticalSection(static_cast(cs_)); + assert(cs_); +} + +Mutex::~Mutex() { + assert(cs_); + ::DeleteCriticalSection(static_cast(cs_)); + delete static_cast(cs_); + cs_ = NULL; + assert(!cs_); +} + +void Mutex::Lock() { + assert(cs_); + ::EnterCriticalSection(static_cast(cs_)); +} + +void Mutex::Unlock() { + assert(cs_); + ::LeaveCriticalSection(static_cast(cs_)); +} + +void Mutex::AssertHeld() { + assert(cs_); + assert(1); +} + +CondVar::CondVar(Mutex* mu) : + waiting_(0), + mu_(mu), + sem1_(::CreateSemaphore(NULL, 0, 10000, NULL)), + sem2_(::CreateSemaphore(NULL, 0, 10000, NULL)) { + assert(mu_); +} + +CondVar::~CondVar() { + ::CloseHandle(sem1_); + ::CloseHandle(sem2_); +} + +void CondVar::Wait() { + mu_->AssertHeld(); + + wait_mtx_.Lock(); + ++waiting_; + wait_mtx_.Unlock(); + + mu_->Unlock(); + + // initiate handshake + ::WaitForSingleObject(sem1_, INFINITE); + ::ReleaseSemaphore(sem2_, 1, NULL); + mu_->Lock(); +} + +void CondVar::Signal() { + wait_mtx_.Lock(); + if (waiting_ > 0) { + --waiting_; + + // finalize handshake + ::ReleaseSemaphore(sem1_, 1, NULL); + ::WaitForSingleObject(sem2_, INFINITE); + } + wait_mtx_.Unlock(); +} + +void CondVar::SignalAll() { + wait_mtx_.Lock(); + ::ReleaseSemaphore(sem1_, waiting_, NULL); + while(waiting_ > 0) { + --waiting_; + ::WaitForSingleObject(sem2_, INFINITE); + } + wait_mtx_.Unlock(); +} + +AtomicPointer::AtomicPointer(void* v) { + Release_Store(v); +} + +void InitOnce(OnceType* once, void (*initializer)()) { + once->InitOnce(initializer); +} + +void* AtomicPointer::Acquire_Load() const { + void * p = NULL; + InterlockedExchangePointer(&p, rep_); + return p; +} + +void AtomicPointer::Release_Store(void* v) { + InterlockedExchangePointer(&rep_, v); +} + +void* AtomicPointer::NoBarrier_Load() const { + return rep_; +} + +void AtomicPointer::NoBarrier_Store(void* v) { + rep_ = v; +} + +} +} diff --git a/src/leveldb/port/port_win.h b/src/leveldb/port/port_win.h new file mode 100644 index 00000000..45bf2f0e --- /dev/null +++ b/src/leveldb/port/port_win.h @@ -0,0 +1,174 @@ +// LevelDB Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// See port_example.h for documentation for the following types/functions. + +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// * Neither the name of the University of California, Berkeley nor the +// names of its contributors may be used to endorse or promote products +// derived from this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +// WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +// DISCLAIMED. IN NO EVENT SHALL THE REGENTS AND CONTRIBUTORS BE LIABLE FOR ANY +// DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +// (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +// LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +// ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +// SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// + +#ifndef STORAGE_LEVELDB_PORT_PORT_WIN_H_ +#define STORAGE_LEVELDB_PORT_PORT_WIN_H_ + +#ifdef _MSC_VER +#define snprintf _snprintf +#define close _close +#define fread_unlocked _fread_nolock +#endif + +#include +#include +#ifdef SNAPPY +#include +#endif + +namespace leveldb { +namespace port { + +// Windows is little endian (for now :p) +static const bool kLittleEndian = true; + +class CondVar; + +class Mutex { + public: + Mutex(); + ~Mutex(); + + void Lock(); + void Unlock(); + void AssertHeld(); + + private: + friend class CondVar; + // critical sections are more efficient than mutexes + // but they are not recursive and can only be used to synchronize threads within the same process + // we use opaque void * to avoid including windows.h in port_win.h + void * cs_; + + // No copying + Mutex(const Mutex&); + void operator=(const Mutex&); +}; + +// the Win32 API offers a dependable condition variable mechanism, but only starting with +// Windows 2008 and Vista +// no matter what we will implement our own condition variable with a semaphore +// implementation as described in a paper written by Andrew D. Birrell in 2003 +class CondVar { + public: + explicit CondVar(Mutex* mu); + ~CondVar(); + void Wait(); + void Signal(); + void SignalAll(); + private: + Mutex* mu_; + + Mutex wait_mtx_; + long waiting_; + + void * sem1_; + void * sem2_; + + +}; + +class OnceType { +public: +// OnceType() : init_(false) {} + OnceType(const OnceType &once) : init_(once.init_) {} + OnceType(bool f) : init_(f) {} + void InitOnce(void (*initializer)()) { + mutex_.Lock(); + if (!init_) { + init_ = true; + initializer(); + } + mutex_.Unlock(); + } + +private: + bool init_; + Mutex mutex_; +}; + +#define LEVELDB_ONCE_INIT false +extern void InitOnce(port::OnceType*, void (*initializer)()); + +// Storage for a lock-free pointer +class AtomicPointer { + private: + void * rep_; + public: + AtomicPointer() : rep_(NULL) { } + explicit AtomicPointer(void* v); + void* Acquire_Load() const; + + void Release_Store(void* v); + + void* NoBarrier_Load() const; + + void NoBarrier_Store(void* v); +}; + +inline bool Snappy_Compress(const char* input, size_t length, + ::std::string* output) { +#ifdef SNAPPY + output->resize(snappy::MaxCompressedLength(length)); + size_t outlen; + snappy::RawCompress(input, length, &(*output)[0], &outlen); + output->resize(outlen); + return true; +#endif + + return false; +} + +inline bool Snappy_GetUncompressedLength(const char* input, size_t length, + size_t* result) { +#ifdef SNAPPY + return snappy::GetUncompressedLength(input, length, result); +#else + return false; +#endif +} + +inline bool Snappy_Uncompress(const char* input, size_t length, + char* output) { +#ifdef SNAPPY + return snappy::RawUncompress(input, length, output); +#else + return false; +#endif +} + +inline bool GetHeapProfile(void (*func)(void*, const char*, int), void* arg) { + return false; +} + +} +} + +#endif // STORAGE_LEVELDB_PORT_PORT_WIN_H_ diff --git a/src/leveldb/port/thread_annotations.h b/src/leveldb/port/thread_annotations.h new file mode 100644 index 00000000..9470ef58 --- /dev/null +++ b/src/leveldb/port/thread_annotations.h @@ -0,0 +1,60 @@ +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_PORT_THREAD_ANNOTATIONS_H_ +#define STORAGE_LEVELDB_PORT_THREAD_ANNOTATIONS_H_ + +// Some environments provide custom macros to aid in static thread-safety +// analysis. Provide empty definitions of such macros unless they are already +// defined. + +#ifndef EXCLUSIVE_LOCKS_REQUIRED +#define EXCLUSIVE_LOCKS_REQUIRED(...) +#endif + +#ifndef SHARED_LOCKS_REQUIRED +#define SHARED_LOCKS_REQUIRED(...) +#endif + +#ifndef LOCKS_EXCLUDED +#define LOCKS_EXCLUDED(...) +#endif + +#ifndef LOCK_RETURNED +#define LOCK_RETURNED(x) +#endif + +#ifndef LOCKABLE +#define LOCKABLE +#endif + +#ifndef SCOPED_LOCKABLE +#define SCOPED_LOCKABLE +#endif + +#ifndef EXCLUSIVE_LOCK_FUNCTION +#define EXCLUSIVE_LOCK_FUNCTION(...) +#endif + +#ifndef SHARED_LOCK_FUNCTION +#define SHARED_LOCK_FUNCTION(...) +#endif + +#ifndef EXCLUSIVE_TRYLOCK_FUNCTION +#define EXCLUSIVE_TRYLOCK_FUNCTION(...) +#endif + +#ifndef SHARED_TRYLOCK_FUNCTION +#define SHARED_TRYLOCK_FUNCTION(...) +#endif + +#ifndef UNLOCK_FUNCTION +#define UNLOCK_FUNCTION(...) +#endif + +#ifndef NO_THREAD_SAFETY_ANALYSIS +#define NO_THREAD_SAFETY_ANALYSIS +#endif + +#endif // STORAGE_LEVELDB_PORT_THREAD_ANNOTATIONS_H_ diff --git a/src/leveldb/port/win/stdint.h b/src/leveldb/port/win/stdint.h new file mode 100644 index 00000000..39edd0db --- /dev/null +++ b/src/leveldb/port/win/stdint.h @@ -0,0 +1,24 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +// MSVC didn't ship with this file until the 2010 version. + +#ifndef STORAGE_LEVELDB_PORT_WIN_STDINT_H_ +#define STORAGE_LEVELDB_PORT_WIN_STDINT_H_ + +#if !defined(_MSC_VER) +#error This file should only be included when compiling with MSVC. +#endif + +// Define C99 equivalent types. +typedef signed char int8_t; +typedef signed short int16_t; +typedef signed int int32_t; +typedef signed long long int64_t; +typedef unsigned char uint8_t; +typedef unsigned short uint16_t; +typedef unsigned int uint32_t; +typedef unsigned long long uint64_t; + +#endif // STORAGE_LEVELDB_PORT_WIN_STDINT_H_ diff --git a/src/leveldb/table/block.cc b/src/leveldb/table/block.cc new file mode 100644 index 00000000..43e402c9 --- /dev/null +++ b/src/leveldb/table/block.cc @@ -0,0 +1,268 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// Decodes the blocks generated by block_builder.cc. + +#include "table/block.h" + +#include +#include +#include "leveldb/comparator.h" +#include "table/format.h" +#include "util/coding.h" +#include "util/logging.h" + +namespace leveldb { + +inline uint32_t Block::NumRestarts() const { + assert(size_ >= sizeof(uint32_t)); + return DecodeFixed32(data_ + size_ - sizeof(uint32_t)); +} + +Block::Block(const BlockContents& contents) + : data_(contents.data.data()), + size_(contents.data.size()), + owned_(contents.heap_allocated) { + if (size_ < sizeof(uint32_t)) { + size_ = 0; // Error marker + } else { + size_t max_restarts_allowed = (size_-sizeof(uint32_t)) / sizeof(uint32_t); + if (NumRestarts() > max_restarts_allowed) { + // The size is too small for NumRestarts() + size_ = 0; + } else { + restart_offset_ = size_ - (1 + NumRestarts()) * sizeof(uint32_t); + } + } +} + +Block::~Block() { + if (owned_) { + delete[] data_; + } +} + +// Helper routine: decode the next block entry starting at "p", +// storing the number of shared key bytes, non_shared key bytes, +// and the length of the value in "*shared", "*non_shared", and +// "*value_length", respectively. Will not dereference past "limit". +// +// If any errors are detected, returns NULL. Otherwise, returns a +// pointer to the key delta (just past the three decoded values). +static inline const char* DecodeEntry(const char* p, const char* limit, + uint32_t* shared, + uint32_t* non_shared, + uint32_t* value_length) { + if (limit - p < 3) return NULL; + *shared = reinterpret_cast(p)[0]; + *non_shared = reinterpret_cast(p)[1]; + *value_length = reinterpret_cast(p)[2]; + if ((*shared | *non_shared | *value_length) < 128) { + // Fast path: all three values are encoded in one byte each + p += 3; + } else { + if ((p = GetVarint32Ptr(p, limit, shared)) == NULL) return NULL; + if ((p = GetVarint32Ptr(p, limit, non_shared)) == NULL) return NULL; + if ((p = GetVarint32Ptr(p, limit, value_length)) == NULL) return NULL; + } + + if (static_cast(limit - p) < (*non_shared + *value_length)) { + return NULL; + } + return p; +} + +class Block::Iter : public Iterator { + private: + const Comparator* const comparator_; + const char* const data_; // underlying block contents + uint32_t const restarts_; // Offset of restart array (list of fixed32) + uint32_t const num_restarts_; // Number of uint32_t entries in restart array + + // current_ is offset in data_ of current entry. >= restarts_ if !Valid + uint32_t current_; + uint32_t restart_index_; // Index of restart block in which current_ falls + std::string key_; + Slice value_; + Status status_; + + inline int Compare(const Slice& a, const Slice& b) const { + return comparator_->Compare(a, b); + } + + // Return the offset in data_ just past the end of the current entry. + inline uint32_t NextEntryOffset() const { + return (value_.data() + value_.size()) - data_; + } + + uint32_t GetRestartPoint(uint32_t index) { + assert(index < num_restarts_); + return DecodeFixed32(data_ + restarts_ + index * sizeof(uint32_t)); + } + + void SeekToRestartPoint(uint32_t index) { + key_.clear(); + restart_index_ = index; + // current_ will be fixed by ParseNextKey(); + + // ParseNextKey() starts at the end of value_, so set value_ accordingly + uint32_t offset = GetRestartPoint(index); + value_ = Slice(data_ + offset, 0); + } + + public: + Iter(const Comparator* comparator, + const char* data, + uint32_t restarts, + uint32_t num_restarts) + : comparator_(comparator), + data_(data), + restarts_(restarts), + num_restarts_(num_restarts), + current_(restarts_), + restart_index_(num_restarts_) { + assert(num_restarts_ > 0); + } + + virtual bool Valid() const { return current_ < restarts_; } + virtual Status status() const { return status_; } + virtual Slice key() const { + assert(Valid()); + return key_; + } + virtual Slice value() const { + assert(Valid()); + return value_; + } + + virtual void Next() { + assert(Valid()); + ParseNextKey(); + } + + virtual void Prev() { + assert(Valid()); + + // Scan backwards to a restart point before current_ + const uint32_t original = current_; + while (GetRestartPoint(restart_index_) >= original) { + if (restart_index_ == 0) { + // No more entries + current_ = restarts_; + restart_index_ = num_restarts_; + return; + } + restart_index_--; + } + + SeekToRestartPoint(restart_index_); + do { + // Loop until end of current entry hits the start of original entry + } while (ParseNextKey() && NextEntryOffset() < original); + } + + virtual void Seek(const Slice& target) { + // Binary search in restart array to find the last restart point + // with a key < target + uint32_t left = 0; + uint32_t right = num_restarts_ - 1; + while (left < right) { + uint32_t mid = (left + right + 1) / 2; + uint32_t region_offset = GetRestartPoint(mid); + uint32_t shared, non_shared, value_length; + const char* key_ptr = DecodeEntry(data_ + region_offset, + data_ + restarts_, + &shared, &non_shared, &value_length); + if (key_ptr == NULL || (shared != 0)) { + CorruptionError(); + return; + } + Slice mid_key(key_ptr, non_shared); + if (Compare(mid_key, target) < 0) { + // Key at "mid" is smaller than "target". Therefore all + // blocks before "mid" are uninteresting. + left = mid; + } else { + // Key at "mid" is >= "target". Therefore all blocks at or + // after "mid" are uninteresting. + right = mid - 1; + } + } + + // Linear search (within restart block) for first key >= target + SeekToRestartPoint(left); + while (true) { + if (!ParseNextKey()) { + return; + } + if (Compare(key_, target) >= 0) { + return; + } + } + } + + virtual void SeekToFirst() { + SeekToRestartPoint(0); + ParseNextKey(); + } + + virtual void SeekToLast() { + SeekToRestartPoint(num_restarts_ - 1); + while (ParseNextKey() && NextEntryOffset() < restarts_) { + // Keep skipping + } + } + + private: + void CorruptionError() { + current_ = restarts_; + restart_index_ = num_restarts_; + status_ = Status::Corruption("bad entry in block"); + key_.clear(); + value_.clear(); + } + + bool ParseNextKey() { + current_ = NextEntryOffset(); + const char* p = data_ + current_; + const char* limit = data_ + restarts_; // Restarts come right after data + if (p >= limit) { + // No more entries to return. Mark as invalid. + current_ = restarts_; + restart_index_ = num_restarts_; + return false; + } + + // Decode next entry + uint32_t shared, non_shared, value_length; + p = DecodeEntry(p, limit, &shared, &non_shared, &value_length); + if (p == NULL || key_.size() < shared) { + CorruptionError(); + return false; + } else { + key_.resize(shared); + key_.append(p, non_shared); + value_ = Slice(p + non_shared, value_length); + while (restart_index_ + 1 < num_restarts_ && + GetRestartPoint(restart_index_ + 1) < current_) { + ++restart_index_; + } + return true; + } + } +}; + +Iterator* Block::NewIterator(const Comparator* cmp) { + if (size_ < sizeof(uint32_t)) { + return NewErrorIterator(Status::Corruption("bad block contents")); + } + const uint32_t num_restarts = NumRestarts(); + if (num_restarts == 0) { + return NewEmptyIterator(); + } else { + return new Iter(cmp, data_, restart_offset_, num_restarts); + } +} + +} // namespace leveldb diff --git a/src/leveldb/table/block.h b/src/leveldb/table/block.h new file mode 100644 index 00000000..2493eb9f --- /dev/null +++ b/src/leveldb/table/block.h @@ -0,0 +1,44 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_TABLE_BLOCK_H_ +#define STORAGE_LEVELDB_TABLE_BLOCK_H_ + +#include +#include +#include "leveldb/iterator.h" + +namespace leveldb { + +struct BlockContents; +class Comparator; + +class Block { + public: + // Initialize the block with the specified contents. + explicit Block(const BlockContents& contents); + + ~Block(); + + size_t size() const { return size_; } + Iterator* NewIterator(const Comparator* comparator); + + private: + uint32_t NumRestarts() const; + + const char* data_; + size_t size_; + uint32_t restart_offset_; // Offset in data_ of restart array + bool owned_; // Block owns data_[] + + // No copying allowed + Block(const Block&); + void operator=(const Block&); + + class Iter; +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_TABLE_BLOCK_H_ diff --git a/src/leveldb/table/block_builder.cc b/src/leveldb/table/block_builder.cc new file mode 100644 index 00000000..db660cd0 --- /dev/null +++ b/src/leveldb/table/block_builder.cc @@ -0,0 +1,109 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// BlockBuilder generates blocks where keys are prefix-compressed: +// +// When we store a key, we drop the prefix shared with the previous +// string. This helps reduce the space requirement significantly. +// Furthermore, once every K keys, we do not apply the prefix +// compression and store the entire key. We call this a "restart +// point". The tail end of the block stores the offsets of all of the +// restart points, and can be used to do a binary search when looking +// for a particular key. Values are stored as-is (without compression) +// immediately following the corresponding key. +// +// An entry for a particular key-value pair has the form: +// shared_bytes: varint32 +// unshared_bytes: varint32 +// value_length: varint32 +// key_delta: char[unshared_bytes] +// value: char[value_length] +// shared_bytes == 0 for restart points. +// +// The trailer of the block has the form: +// restarts: uint32[num_restarts] +// num_restarts: uint32 +// restarts[i] contains the offset within the block of the ith restart point. + +#include "table/block_builder.h" + +#include +#include +#include "leveldb/comparator.h" +#include "leveldb/table_builder.h" +#include "util/coding.h" + +namespace leveldb { + +BlockBuilder::BlockBuilder(const Options* options) + : options_(options), + restarts_(), + counter_(0), + finished_(false) { + assert(options->block_restart_interval >= 1); + restarts_.push_back(0); // First restart point is at offset 0 +} + +void BlockBuilder::Reset() { + buffer_.clear(); + restarts_.clear(); + restarts_.push_back(0); // First restart point is at offset 0 + counter_ = 0; + finished_ = false; + last_key_.clear(); +} + +size_t BlockBuilder::CurrentSizeEstimate() const { + return (buffer_.size() + // Raw data buffer + restarts_.size() * sizeof(uint32_t) + // Restart array + sizeof(uint32_t)); // Restart array length +} + +Slice BlockBuilder::Finish() { + // Append restart array + for (size_t i = 0; i < restarts_.size(); i++) { + PutFixed32(&buffer_, restarts_[i]); + } + PutFixed32(&buffer_, restarts_.size()); + finished_ = true; + return Slice(buffer_); +} + +void BlockBuilder::Add(const Slice& key, const Slice& value) { + Slice last_key_piece(last_key_); + assert(!finished_); + assert(counter_ <= options_->block_restart_interval); + assert(buffer_.empty() // No values yet? + || options_->comparator->Compare(key, last_key_piece) > 0); + size_t shared = 0; + if (counter_ < options_->block_restart_interval) { + // See how much sharing to do with previous string + const size_t min_length = std::min(last_key_piece.size(), key.size()); + while ((shared < min_length) && (last_key_piece[shared] == key[shared])) { + shared++; + } + } else { + // Restart compression + restarts_.push_back(buffer_.size()); + counter_ = 0; + } + const size_t non_shared = key.size() - shared; + + // Add "" to buffer_ + PutVarint32(&buffer_, shared); + PutVarint32(&buffer_, non_shared); + PutVarint32(&buffer_, value.size()); + + // Add string delta to buffer_ followed by value + buffer_.append(key.data() + shared, non_shared); + buffer_.append(value.data(), value.size()); + + // Update state + last_key_.resize(shared); + last_key_.append(key.data() + shared, non_shared); + assert(Slice(last_key_) == key); + counter_++; +} + +} // namespace leveldb diff --git a/src/leveldb/table/block_builder.h b/src/leveldb/table/block_builder.h new file mode 100644 index 00000000..4fbcb339 --- /dev/null +++ b/src/leveldb/table/block_builder.h @@ -0,0 +1,57 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_TABLE_BLOCK_BUILDER_H_ +#define STORAGE_LEVELDB_TABLE_BLOCK_BUILDER_H_ + +#include + +#include +#include "leveldb/slice.h" + +namespace leveldb { + +struct Options; + +class BlockBuilder { + public: + explicit BlockBuilder(const Options* options); + + // Reset the contents as if the BlockBuilder was just constructed. + void Reset(); + + // REQUIRES: Finish() has not been called since the last call to Reset(). + // REQUIRES: key is larger than any previously added key + void Add(const Slice& key, const Slice& value); + + // Finish building the block and return a slice that refers to the + // block contents. The returned slice will remain valid for the + // lifetime of this builder or until Reset() is called. + Slice Finish(); + + // Returns an estimate of the current (uncompressed) size of the block + // we are building. + size_t CurrentSizeEstimate() const; + + // Return true iff no entries have been added since the last Reset() + bool empty() const { + return buffer_.empty(); + } + + private: + const Options* options_; + std::string buffer_; // Destination buffer + std::vector restarts_; // Restart points + int counter_; // Number of entries emitted since restart + bool finished_; // Has Finish() been called? + std::string last_key_; + + // No copying allowed + BlockBuilder(const BlockBuilder&); + void operator=(const BlockBuilder&); +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_TABLE_BLOCK_BUILDER_H_ diff --git a/src/leveldb/table/filter_block.cc b/src/leveldb/table/filter_block.cc new file mode 100644 index 00000000..203e15c8 --- /dev/null +++ b/src/leveldb/table/filter_block.cc @@ -0,0 +1,111 @@ +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "table/filter_block.h" + +#include "leveldb/filter_policy.h" +#include "util/coding.h" + +namespace leveldb { + +// See doc/table_format.txt for an explanation of the filter block format. + +// Generate new filter every 2KB of data +static const size_t kFilterBaseLg = 11; +static const size_t kFilterBase = 1 << kFilterBaseLg; + +FilterBlockBuilder::FilterBlockBuilder(const FilterPolicy* policy) + : policy_(policy) { +} + +void FilterBlockBuilder::StartBlock(uint64_t block_offset) { + uint64_t filter_index = (block_offset / kFilterBase); + assert(filter_index >= filter_offsets_.size()); + while (filter_index > filter_offsets_.size()) { + GenerateFilter(); + } +} + +void FilterBlockBuilder::AddKey(const Slice& key) { + Slice k = key; + start_.push_back(keys_.size()); + keys_.append(k.data(), k.size()); +} + +Slice FilterBlockBuilder::Finish() { + if (!start_.empty()) { + GenerateFilter(); + } + + // Append array of per-filter offsets + const uint32_t array_offset = result_.size(); + for (size_t i = 0; i < filter_offsets_.size(); i++) { + PutFixed32(&result_, filter_offsets_[i]); + } + + PutFixed32(&result_, array_offset); + result_.push_back(kFilterBaseLg); // Save encoding parameter in result + return Slice(result_); +} + +void FilterBlockBuilder::GenerateFilter() { + const size_t num_keys = start_.size(); + if (num_keys == 0) { + // Fast path if there are no keys for this filter + filter_offsets_.push_back(result_.size()); + return; + } + + // Make list of keys from flattened key structure + start_.push_back(keys_.size()); // Simplify length computation + tmp_keys_.resize(num_keys); + for (size_t i = 0; i < num_keys; i++) { + const char* base = keys_.data() + start_[i]; + size_t length = start_[i+1] - start_[i]; + tmp_keys_[i] = Slice(base, length); + } + + // Generate filter for current set of keys and append to result_. + filter_offsets_.push_back(result_.size()); + policy_->CreateFilter(&tmp_keys_[0], num_keys, &result_); + + tmp_keys_.clear(); + keys_.clear(); + start_.clear(); +} + +FilterBlockReader::FilterBlockReader(const FilterPolicy* policy, + const Slice& contents) + : policy_(policy), + data_(NULL), + offset_(NULL), + num_(0), + base_lg_(0) { + size_t n = contents.size(); + if (n < 5) return; // 1 byte for base_lg_ and 4 for start of offset array + base_lg_ = contents[n-1]; + uint32_t last_word = DecodeFixed32(contents.data() + n - 5); + if (last_word > n - 5) return; + data_ = contents.data(); + offset_ = data_ + last_word; + num_ = (n - 5 - last_word) / 4; +} + +bool FilterBlockReader::KeyMayMatch(uint64_t block_offset, const Slice& key) { + uint64_t index = block_offset >> base_lg_; + if (index < num_) { + uint32_t start = DecodeFixed32(offset_ + index*4); + uint32_t limit = DecodeFixed32(offset_ + index*4 + 4); + if (start <= limit && limit <= (offset_ - data_)) { + Slice filter = Slice(data_ + start, limit - start); + return policy_->KeyMayMatch(key, filter); + } else if (start == limit) { + // Empty filters do not match any keys + return false; + } + } + return true; // Errors are treated as potential matches +} + +} diff --git a/src/leveldb/table/filter_block.h b/src/leveldb/table/filter_block.h new file mode 100644 index 00000000..c67d010b --- /dev/null +++ b/src/leveldb/table/filter_block.h @@ -0,0 +1,68 @@ +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// A filter block is stored near the end of a Table file. It contains +// filters (e.g., bloom filters) for all data blocks in the table combined +// into a single filter block. + +#ifndef STORAGE_LEVELDB_TABLE_FILTER_BLOCK_H_ +#define STORAGE_LEVELDB_TABLE_FILTER_BLOCK_H_ + +#include +#include +#include +#include +#include "leveldb/slice.h" +#include "util/hash.h" + +namespace leveldb { + +class FilterPolicy; + +// A FilterBlockBuilder is used to construct all of the filters for a +// particular Table. It generates a single string which is stored as +// a special block in the Table. +// +// The sequence of calls to FilterBlockBuilder must match the regexp: +// (StartBlock AddKey*)* Finish +class FilterBlockBuilder { + public: + explicit FilterBlockBuilder(const FilterPolicy*); + + void StartBlock(uint64_t block_offset); + void AddKey(const Slice& key); + Slice Finish(); + + private: + void GenerateFilter(); + + const FilterPolicy* policy_; + std::string keys_; // Flattened key contents + std::vector start_; // Starting index in keys_ of each key + std::string result_; // Filter data computed so far + std::vector tmp_keys_; // policy_->CreateFilter() argument + std::vector filter_offsets_; + + // No copying allowed + FilterBlockBuilder(const FilterBlockBuilder&); + void operator=(const FilterBlockBuilder&); +}; + +class FilterBlockReader { + public: + // REQUIRES: "contents" and *policy must stay live while *this is live. + FilterBlockReader(const FilterPolicy* policy, const Slice& contents); + bool KeyMayMatch(uint64_t block_offset, const Slice& key); + + private: + const FilterPolicy* policy_; + const char* data_; // Pointer to filter data (at block-start) + const char* offset_; // Pointer to beginning of offset array (at block-end) + size_t num_; // Number of entries in offset array + size_t base_lg_; // Encoding parameter (see kFilterBaseLg in .cc file) +}; + +} + +#endif // STORAGE_LEVELDB_TABLE_FILTER_BLOCK_H_ diff --git a/src/leveldb/table/filter_block_test.cc b/src/leveldb/table/filter_block_test.cc new file mode 100644 index 00000000..8c4a4741 --- /dev/null +++ b/src/leveldb/table/filter_block_test.cc @@ -0,0 +1,128 @@ +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "table/filter_block.h" + +#include "leveldb/filter_policy.h" +#include "util/coding.h" +#include "util/hash.h" +#include "util/logging.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace leveldb { + +// For testing: emit an array with one hash value per key +class TestHashFilter : public FilterPolicy { + public: + virtual const char* Name() const { + return "TestHashFilter"; + } + + virtual void CreateFilter(const Slice* keys, int n, std::string* dst) const { + for (int i = 0; i < n; i++) { + uint32_t h = Hash(keys[i].data(), keys[i].size(), 1); + PutFixed32(dst, h); + } + } + + virtual bool KeyMayMatch(const Slice& key, const Slice& filter) const { + uint32_t h = Hash(key.data(), key.size(), 1); + for (size_t i = 0; i + 4 <= filter.size(); i += 4) { + if (h == DecodeFixed32(filter.data() + i)) { + return true; + } + } + return false; + } +}; + +class FilterBlockTest { + public: + TestHashFilter policy_; +}; + +TEST(FilterBlockTest, EmptyBuilder) { + FilterBlockBuilder builder(&policy_); + Slice block = builder.Finish(); + ASSERT_EQ("\\x00\\x00\\x00\\x00\\x0b", EscapeString(block)); + FilterBlockReader reader(&policy_, block); + ASSERT_TRUE(reader.KeyMayMatch(0, "foo")); + ASSERT_TRUE(reader.KeyMayMatch(100000, "foo")); +} + +TEST(FilterBlockTest, SingleChunk) { + FilterBlockBuilder builder(&policy_); + builder.StartBlock(100); + builder.AddKey("foo"); + builder.AddKey("bar"); + builder.AddKey("box"); + builder.StartBlock(200); + builder.AddKey("box"); + builder.StartBlock(300); + builder.AddKey("hello"); + Slice block = builder.Finish(); + FilterBlockReader reader(&policy_, block); + ASSERT_TRUE(reader.KeyMayMatch(100, "foo")); + ASSERT_TRUE(reader.KeyMayMatch(100, "bar")); + ASSERT_TRUE(reader.KeyMayMatch(100, "box")); + ASSERT_TRUE(reader.KeyMayMatch(100, "hello")); + ASSERT_TRUE(reader.KeyMayMatch(100, "foo")); + ASSERT_TRUE(! reader.KeyMayMatch(100, "missing")); + ASSERT_TRUE(! reader.KeyMayMatch(100, "other")); +} + +TEST(FilterBlockTest, MultiChunk) { + FilterBlockBuilder builder(&policy_); + + // First filter + builder.StartBlock(0); + builder.AddKey("foo"); + builder.StartBlock(2000); + builder.AddKey("bar"); + + // Second filter + builder.StartBlock(3100); + builder.AddKey("box"); + + // Third filter is empty + + // Last filter + builder.StartBlock(9000); + builder.AddKey("box"); + builder.AddKey("hello"); + + Slice block = builder.Finish(); + FilterBlockReader reader(&policy_, block); + + // Check first filter + ASSERT_TRUE(reader.KeyMayMatch(0, "foo")); + ASSERT_TRUE(reader.KeyMayMatch(2000, "bar")); + ASSERT_TRUE(! reader.KeyMayMatch(0, "box")); + ASSERT_TRUE(! reader.KeyMayMatch(0, "hello")); + + // Check second filter + ASSERT_TRUE(reader.KeyMayMatch(3100, "box")); + ASSERT_TRUE(! reader.KeyMayMatch(3100, "foo")); + ASSERT_TRUE(! reader.KeyMayMatch(3100, "bar")); + ASSERT_TRUE(! reader.KeyMayMatch(3100, "hello")); + + // Check third filter (empty) + ASSERT_TRUE(! reader.KeyMayMatch(4100, "foo")); + ASSERT_TRUE(! reader.KeyMayMatch(4100, "bar")); + ASSERT_TRUE(! reader.KeyMayMatch(4100, "box")); + ASSERT_TRUE(! reader.KeyMayMatch(4100, "hello")); + + // Check last filter + ASSERT_TRUE(reader.KeyMayMatch(9000, "box")); + ASSERT_TRUE(reader.KeyMayMatch(9000, "hello")); + ASSERT_TRUE(! reader.KeyMayMatch(9000, "foo")); + ASSERT_TRUE(! reader.KeyMayMatch(9000, "bar")); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/table/format.cc b/src/leveldb/table/format.cc new file mode 100644 index 00000000..aa63144c --- /dev/null +++ b/src/leveldb/table/format.cc @@ -0,0 +1,145 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "table/format.h" + +#include "leveldb/env.h" +#include "port/port.h" +#include "table/block.h" +#include "util/coding.h" +#include "util/crc32c.h" + +namespace leveldb { + +void BlockHandle::EncodeTo(std::string* dst) const { + // Sanity check that all fields have been set + assert(offset_ != ~static_cast(0)); + assert(size_ != ~static_cast(0)); + PutVarint64(dst, offset_); + PutVarint64(dst, size_); +} + +Status BlockHandle::DecodeFrom(Slice* input) { + if (GetVarint64(input, &offset_) && + GetVarint64(input, &size_)) { + return Status::OK(); + } else { + return Status::Corruption("bad block handle"); + } +} + +void Footer::EncodeTo(std::string* dst) const { +#ifndef NDEBUG + const size_t original_size = dst->size(); +#endif + metaindex_handle_.EncodeTo(dst); + index_handle_.EncodeTo(dst); + dst->resize(2 * BlockHandle::kMaxEncodedLength); // Padding + PutFixed32(dst, static_cast(kTableMagicNumber & 0xffffffffu)); + PutFixed32(dst, static_cast(kTableMagicNumber >> 32)); + assert(dst->size() == original_size + kEncodedLength); +} + +Status Footer::DecodeFrom(Slice* input) { + const char* magic_ptr = input->data() + kEncodedLength - 8; + const uint32_t magic_lo = DecodeFixed32(magic_ptr); + const uint32_t magic_hi = DecodeFixed32(magic_ptr + 4); + const uint64_t magic = ((static_cast(magic_hi) << 32) | + (static_cast(magic_lo))); + if (magic != kTableMagicNumber) { + return Status::Corruption("not an sstable (bad magic number)"); + } + + Status result = metaindex_handle_.DecodeFrom(input); + if (result.ok()) { + result = index_handle_.DecodeFrom(input); + } + if (result.ok()) { + // We skip over any leftover data (just padding for now) in "input" + const char* end = magic_ptr + 8; + *input = Slice(end, input->data() + input->size() - end); + } + return result; +} + +Status ReadBlock(RandomAccessFile* file, + const ReadOptions& options, + const BlockHandle& handle, + BlockContents* result) { + result->data = Slice(); + result->cachable = false; + result->heap_allocated = false; + + // Read the block contents as well as the type/crc footer. + // See table_builder.cc for the code that built this structure. + size_t n = static_cast(handle.size()); + char* buf = new char[n + kBlockTrailerSize]; + Slice contents; + Status s = file->Read(handle.offset(), n + kBlockTrailerSize, &contents, buf); + if (!s.ok()) { + delete[] buf; + return s; + } + if (contents.size() != n + kBlockTrailerSize) { + delete[] buf; + return Status::Corruption("truncated block read"); + } + + // Check the crc of the type and the block contents + const char* data = contents.data(); // Pointer to where Read put the data + if (options.verify_checksums) { + const uint32_t crc = crc32c::Unmask(DecodeFixed32(data + n + 1)); + const uint32_t actual = crc32c::Value(data, n + 1); + if (actual != crc) { + delete[] buf; + s = Status::Corruption("block checksum mismatch"); + return s; + } + } + + switch (data[n]) { + case kNoCompression: + if (data != buf) { + // File implementation gave us pointer to some other data. + // Use it directly under the assumption that it will be live + // while the file is open. + delete[] buf; + result->data = Slice(data, n); + result->heap_allocated = false; + result->cachable = false; // Do not double-cache + } else { + result->data = Slice(buf, n); + result->heap_allocated = true; + result->cachable = true; + } + + // Ok + break; + case kSnappyCompression: { + size_t ulength = 0; + if (!port::Snappy_GetUncompressedLength(data, n, &ulength)) { + delete[] buf; + return Status::Corruption("corrupted compressed block contents"); + } + char* ubuf = new char[ulength]; + if (!port::Snappy_Uncompress(data, n, ubuf)) { + delete[] buf; + delete[] ubuf; + return Status::Corruption("corrupted compressed block contents"); + } + delete[] buf; + result->data = Slice(ubuf, ulength); + result->heap_allocated = true; + result->cachable = true; + break; + } + default: + delete[] buf; + return Status::Corruption("bad block type"); + } + + return Status::OK(); +} + +} // namespace leveldb diff --git a/src/leveldb/table/format.h b/src/leveldb/table/format.h new file mode 100644 index 00000000..6c0b80c0 --- /dev/null +++ b/src/leveldb/table/format.h @@ -0,0 +1,108 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_TABLE_FORMAT_H_ +#define STORAGE_LEVELDB_TABLE_FORMAT_H_ + +#include +#include +#include "leveldb/slice.h" +#include "leveldb/status.h" +#include "leveldb/table_builder.h" + +namespace leveldb { + +class Block; +class RandomAccessFile; +struct ReadOptions; + +// BlockHandle is a pointer to the extent of a file that stores a data +// block or a meta block. +class BlockHandle { + public: + BlockHandle(); + + // The offset of the block in the file. + uint64_t offset() const { return offset_; } + void set_offset(uint64_t offset) { offset_ = offset; } + + // The size of the stored block + uint64_t size() const { return size_; } + void set_size(uint64_t size) { size_ = size; } + + void EncodeTo(std::string* dst) const; + Status DecodeFrom(Slice* input); + + // Maximum encoding length of a BlockHandle + enum { kMaxEncodedLength = 10 + 10 }; + + private: + uint64_t offset_; + uint64_t size_; +}; + +// Footer encapsulates the fixed information stored at the tail +// end of every table file. +class Footer { + public: + Footer() { } + + // The block handle for the metaindex block of the table + const BlockHandle& metaindex_handle() const { return metaindex_handle_; } + void set_metaindex_handle(const BlockHandle& h) { metaindex_handle_ = h; } + + // The block handle for the index block of the table + const BlockHandle& index_handle() const { + return index_handle_; + } + void set_index_handle(const BlockHandle& h) { + index_handle_ = h; + } + + void EncodeTo(std::string* dst) const; + Status DecodeFrom(Slice* input); + + // Encoded length of a Footer. Note that the serialization of a + // Footer will always occupy exactly this many bytes. It consists + // of two block handles and a magic number. + enum { + kEncodedLength = 2*BlockHandle::kMaxEncodedLength + 8 + }; + + private: + BlockHandle metaindex_handle_; + BlockHandle index_handle_; +}; + +// kTableMagicNumber was picked by running +// echo http://code.google.com/p/leveldb/ | sha1sum +// and taking the leading 64 bits. +static const uint64_t kTableMagicNumber = 0xdb4775248b80fb57ull; + +// 1-byte type + 32-bit crc +static const size_t kBlockTrailerSize = 5; + +struct BlockContents { + Slice data; // Actual contents of data + bool cachable; // True iff data can be cached + bool heap_allocated; // True iff caller should delete[] data.data() +}; + +// Read the block identified by "handle" from "file". On failure +// return non-OK. On success fill *result and return OK. +extern Status ReadBlock(RandomAccessFile* file, + const ReadOptions& options, + const BlockHandle& handle, + BlockContents* result); + +// Implementation details follow. Clients should ignore, + +inline BlockHandle::BlockHandle() + : offset_(~static_cast(0)), + size_(~static_cast(0)) { +} + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_TABLE_FORMAT_H_ diff --git a/src/leveldb/table/iterator.cc b/src/leveldb/table/iterator.cc new file mode 100644 index 00000000..3d1c87fd --- /dev/null +++ b/src/leveldb/table/iterator.cc @@ -0,0 +1,67 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/iterator.h" + +namespace leveldb { + +Iterator::Iterator() { + cleanup_.function = NULL; + cleanup_.next = NULL; +} + +Iterator::~Iterator() { + if (cleanup_.function != NULL) { + (*cleanup_.function)(cleanup_.arg1, cleanup_.arg2); + for (Cleanup* c = cleanup_.next; c != NULL; ) { + (*c->function)(c->arg1, c->arg2); + Cleanup* next = c->next; + delete c; + c = next; + } + } +} + +void Iterator::RegisterCleanup(CleanupFunction func, void* arg1, void* arg2) { + assert(func != NULL); + Cleanup* c; + if (cleanup_.function == NULL) { + c = &cleanup_; + } else { + c = new Cleanup; + c->next = cleanup_.next; + cleanup_.next = c; + } + c->function = func; + c->arg1 = arg1; + c->arg2 = arg2; +} + +namespace { +class EmptyIterator : public Iterator { + public: + EmptyIterator(const Status& s) : status_(s) { } + virtual bool Valid() const { return false; } + virtual void Seek(const Slice& target) { } + virtual void SeekToFirst() { } + virtual void SeekToLast() { } + virtual void Next() { assert(false); } + virtual void Prev() { assert(false); } + Slice key() const { assert(false); return Slice(); } + Slice value() const { assert(false); return Slice(); } + virtual Status status() const { return status_; } + private: + Status status_; +}; +} // namespace + +Iterator* NewEmptyIterator() { + return new EmptyIterator(Status::OK()); +} + +Iterator* NewErrorIterator(const Status& status) { + return new EmptyIterator(status); +} + +} // namespace leveldb diff --git a/src/leveldb/table/iterator_wrapper.h b/src/leveldb/table/iterator_wrapper.h new file mode 100644 index 00000000..9e16b3db --- /dev/null +++ b/src/leveldb/table/iterator_wrapper.h @@ -0,0 +1,63 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_TABLE_ITERATOR_WRAPPER_H_ +#define STORAGE_LEVELDB_TABLE_ITERATOR_WRAPPER_H_ + +namespace leveldb { + +// A internal wrapper class with an interface similar to Iterator that +// caches the valid() and key() results for an underlying iterator. +// This can help avoid virtual function calls and also gives better +// cache locality. +class IteratorWrapper { + public: + IteratorWrapper(): iter_(NULL), valid_(false) { } + explicit IteratorWrapper(Iterator* iter): iter_(NULL) { + Set(iter); + } + ~IteratorWrapper() { delete iter_; } + Iterator* iter() const { return iter_; } + + // Takes ownership of "iter" and will delete it when destroyed, or + // when Set() is invoked again. + void Set(Iterator* iter) { + delete iter_; + iter_ = iter; + if (iter_ == NULL) { + valid_ = false; + } else { + Update(); + } + } + + + // Iterator interface methods + bool Valid() const { return valid_; } + Slice key() const { assert(Valid()); return key_; } + Slice value() const { assert(Valid()); return iter_->value(); } + // Methods below require iter() != NULL + Status status() const { assert(iter_); return iter_->status(); } + void Next() { assert(iter_); iter_->Next(); Update(); } + void Prev() { assert(iter_); iter_->Prev(); Update(); } + void Seek(const Slice& k) { assert(iter_); iter_->Seek(k); Update(); } + void SeekToFirst() { assert(iter_); iter_->SeekToFirst(); Update(); } + void SeekToLast() { assert(iter_); iter_->SeekToLast(); Update(); } + + private: + void Update() { + valid_ = iter_->Valid(); + if (valid_) { + key_ = iter_->key(); + } + } + + Iterator* iter_; + bool valid_; + Slice key_; +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_TABLE_ITERATOR_WRAPPER_H_ diff --git a/src/leveldb/table/merger.cc b/src/leveldb/table/merger.cc new file mode 100644 index 00000000..2dde4dc2 --- /dev/null +++ b/src/leveldb/table/merger.cc @@ -0,0 +1,197 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "table/merger.h" + +#include "leveldb/comparator.h" +#include "leveldb/iterator.h" +#include "table/iterator_wrapper.h" + +namespace leveldb { + +namespace { +class MergingIterator : public Iterator { + public: + MergingIterator(const Comparator* comparator, Iterator** children, int n) + : comparator_(comparator), + children_(new IteratorWrapper[n]), + n_(n), + current_(NULL), + direction_(kForward) { + for (int i = 0; i < n; i++) { + children_[i].Set(children[i]); + } + } + + virtual ~MergingIterator() { + delete[] children_; + } + + virtual bool Valid() const { + return (current_ != NULL); + } + + virtual void SeekToFirst() { + for (int i = 0; i < n_; i++) { + children_[i].SeekToFirst(); + } + FindSmallest(); + direction_ = kForward; + } + + virtual void SeekToLast() { + for (int i = 0; i < n_; i++) { + children_[i].SeekToLast(); + } + FindLargest(); + direction_ = kReverse; + } + + virtual void Seek(const Slice& target) { + for (int i = 0; i < n_; i++) { + children_[i].Seek(target); + } + FindSmallest(); + direction_ = kForward; + } + + virtual void Next() { + assert(Valid()); + + // Ensure that all children are positioned after key(). + // If we are moving in the forward direction, it is already + // true for all of the non-current_ children since current_ is + // the smallest child and key() == current_->key(). Otherwise, + // we explicitly position the non-current_ children. + if (direction_ != kForward) { + for (int i = 0; i < n_; i++) { + IteratorWrapper* child = &children_[i]; + if (child != current_) { + child->Seek(key()); + if (child->Valid() && + comparator_->Compare(key(), child->key()) == 0) { + child->Next(); + } + } + } + direction_ = kForward; + } + + current_->Next(); + FindSmallest(); + } + + virtual void Prev() { + assert(Valid()); + + // Ensure that all children are positioned before key(). + // If we are moving in the reverse direction, it is already + // true for all of the non-current_ children since current_ is + // the largest child and key() == current_->key(). Otherwise, + // we explicitly position the non-current_ children. + if (direction_ != kReverse) { + for (int i = 0; i < n_; i++) { + IteratorWrapper* child = &children_[i]; + if (child != current_) { + child->Seek(key()); + if (child->Valid()) { + // Child is at first entry >= key(). Step back one to be < key() + child->Prev(); + } else { + // Child has no entries >= key(). Position at last entry. + child->SeekToLast(); + } + } + } + direction_ = kReverse; + } + + current_->Prev(); + FindLargest(); + } + + virtual Slice key() const { + assert(Valid()); + return current_->key(); + } + + virtual Slice value() const { + assert(Valid()); + return current_->value(); + } + + virtual Status status() const { + Status status; + for (int i = 0; i < n_; i++) { + status = children_[i].status(); + if (!status.ok()) { + break; + } + } + return status; + } + + private: + void FindSmallest(); + void FindLargest(); + + // We might want to use a heap in case there are lots of children. + // For now we use a simple array since we expect a very small number + // of children in leveldb. + const Comparator* comparator_; + IteratorWrapper* children_; + int n_; + IteratorWrapper* current_; + + // Which direction is the iterator moving? + enum Direction { + kForward, + kReverse + }; + Direction direction_; +}; + +void MergingIterator::FindSmallest() { + IteratorWrapper* smallest = NULL; + for (int i = 0; i < n_; i++) { + IteratorWrapper* child = &children_[i]; + if (child->Valid()) { + if (smallest == NULL) { + smallest = child; + } else if (comparator_->Compare(child->key(), smallest->key()) < 0) { + smallest = child; + } + } + } + current_ = smallest; +} + +void MergingIterator::FindLargest() { + IteratorWrapper* largest = NULL; + for (int i = n_-1; i >= 0; i--) { + IteratorWrapper* child = &children_[i]; + if (child->Valid()) { + if (largest == NULL) { + largest = child; + } else if (comparator_->Compare(child->key(), largest->key()) > 0) { + largest = child; + } + } + } + current_ = largest; +} +} // namespace + +Iterator* NewMergingIterator(const Comparator* cmp, Iterator** list, int n) { + assert(n >= 0); + if (n == 0) { + return NewEmptyIterator(); + } else if (n == 1) { + return list[0]; + } else { + return new MergingIterator(cmp, list, n); + } +} + +} // namespace leveldb diff --git a/src/leveldb/table/merger.h b/src/leveldb/table/merger.h new file mode 100644 index 00000000..91ddd80f --- /dev/null +++ b/src/leveldb/table/merger.h @@ -0,0 +1,26 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_TABLE_MERGER_H_ +#define STORAGE_LEVELDB_TABLE_MERGER_H_ + +namespace leveldb { + +class Comparator; +class Iterator; + +// Return an iterator that provided the union of the data in +// children[0,n-1]. Takes ownership of the child iterators and +// will delete them when the result iterator is deleted. +// +// The result does no duplicate suppression. I.e., if a particular +// key is present in K child iterators, it will be yielded K times. +// +// REQUIRES: n >= 0 +extern Iterator* NewMergingIterator( + const Comparator* comparator, Iterator** children, int n); + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_TABLE_MERGER_H_ diff --git a/src/leveldb/table/table.cc b/src/leveldb/table/table.cc new file mode 100644 index 00000000..dff8a825 --- /dev/null +++ b/src/leveldb/table/table.cc @@ -0,0 +1,285 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/table.h" + +#include "leveldb/cache.h" +#include "leveldb/comparator.h" +#include "leveldb/env.h" +#include "leveldb/filter_policy.h" +#include "leveldb/options.h" +#include "table/block.h" +#include "table/filter_block.h" +#include "table/format.h" +#include "table/two_level_iterator.h" +#include "util/coding.h" + +namespace leveldb { + +struct Table::Rep { + ~Rep() { + delete filter; + delete [] filter_data; + delete index_block; + } + + Options options; + Status status; + RandomAccessFile* file; + uint64_t cache_id; + FilterBlockReader* filter; + const char* filter_data; + + BlockHandle metaindex_handle; // Handle to metaindex_block: saved from footer + Block* index_block; +}; + +Status Table::Open(const Options& options, + RandomAccessFile* file, + uint64_t size, + Table** table) { + *table = NULL; + if (size < Footer::kEncodedLength) { + return Status::Corruption("file is too short to be an sstable"); + } + + char footer_space[Footer::kEncodedLength]; + Slice footer_input; + Status s = file->Read(size - Footer::kEncodedLength, Footer::kEncodedLength, + &footer_input, footer_space); + if (!s.ok()) return s; + + Footer footer; + s = footer.DecodeFrom(&footer_input); + if (!s.ok()) return s; + + // Read the index block + BlockContents contents; + Block* index_block = NULL; + if (s.ok()) { + ReadOptions opt; + if (options.paranoid_checks) { + opt.verify_checksums = true; + } + s = ReadBlock(file, opt, footer.index_handle(), &contents); + if (s.ok()) { + index_block = new Block(contents); + } + } + + if (s.ok()) { + // We've successfully read the footer and the index block: we're + // ready to serve requests. + Rep* rep = new Table::Rep; + rep->options = options; + rep->file = file; + rep->metaindex_handle = footer.metaindex_handle(); + rep->index_block = index_block; + rep->cache_id = (options.block_cache ? options.block_cache->NewId() : 0); + rep->filter_data = NULL; + rep->filter = NULL; + *table = new Table(rep); + (*table)->ReadMeta(footer); + } else { + if (index_block) delete index_block; + } + + return s; +} + +void Table::ReadMeta(const Footer& footer) { + if (rep_->options.filter_policy == NULL) { + return; // Do not need any metadata + } + + // TODO(sanjay): Skip this if footer.metaindex_handle() size indicates + // it is an empty block. + ReadOptions opt; + if (rep_->options.paranoid_checks) { + opt.verify_checksums = true; + } + BlockContents contents; + if (!ReadBlock(rep_->file, opt, footer.metaindex_handle(), &contents).ok()) { + // Do not propagate errors since meta info is not needed for operation + return; + } + Block* meta = new Block(contents); + + Iterator* iter = meta->NewIterator(BytewiseComparator()); + std::string key = "filter."; + key.append(rep_->options.filter_policy->Name()); + iter->Seek(key); + if (iter->Valid() && iter->key() == Slice(key)) { + ReadFilter(iter->value()); + } + delete iter; + delete meta; +} + +void Table::ReadFilter(const Slice& filter_handle_value) { + Slice v = filter_handle_value; + BlockHandle filter_handle; + if (!filter_handle.DecodeFrom(&v).ok()) { + return; + } + + // We might want to unify with ReadBlock() if we start + // requiring checksum verification in Table::Open. + ReadOptions opt; + if (rep_->options.paranoid_checks) { + opt.verify_checksums = true; + } + BlockContents block; + if (!ReadBlock(rep_->file, opt, filter_handle, &block).ok()) { + return; + } + if (block.heap_allocated) { + rep_->filter_data = block.data.data(); // Will need to delete later + } + rep_->filter = new FilterBlockReader(rep_->options.filter_policy, block.data); +} + +Table::~Table() { + delete rep_; +} + +static void DeleteBlock(void* arg, void* ignored) { + delete reinterpret_cast(arg); +} + +static void DeleteCachedBlock(const Slice& key, void* value) { + Block* block = reinterpret_cast(value); + delete block; +} + +static void ReleaseBlock(void* arg, void* h) { + Cache* cache = reinterpret_cast(arg); + Cache::Handle* handle = reinterpret_cast(h); + cache->Release(handle); +} + +// Convert an index iterator value (i.e., an encoded BlockHandle) +// into an iterator over the contents of the corresponding block. +Iterator* Table::BlockReader(void* arg, + const ReadOptions& options, + const Slice& index_value) { + Table* table = reinterpret_cast(arg); + Cache* block_cache = table->rep_->options.block_cache; + Block* block = NULL; + Cache::Handle* cache_handle = NULL; + + BlockHandle handle; + Slice input = index_value; + Status s = handle.DecodeFrom(&input); + // We intentionally allow extra stuff in index_value so that we + // can add more features in the future. + + if (s.ok()) { + BlockContents contents; + if (block_cache != NULL) { + char cache_key_buffer[16]; + EncodeFixed64(cache_key_buffer, table->rep_->cache_id); + EncodeFixed64(cache_key_buffer+8, handle.offset()); + Slice key(cache_key_buffer, sizeof(cache_key_buffer)); + cache_handle = block_cache->Lookup(key); + if (cache_handle != NULL) { + block = reinterpret_cast(block_cache->Value(cache_handle)); + } else { + s = ReadBlock(table->rep_->file, options, handle, &contents); + if (s.ok()) { + block = new Block(contents); + if (contents.cachable && options.fill_cache) { + cache_handle = block_cache->Insert( + key, block, block->size(), &DeleteCachedBlock); + } + } + } + } else { + s = ReadBlock(table->rep_->file, options, handle, &contents); + if (s.ok()) { + block = new Block(contents); + } + } + } + + Iterator* iter; + if (block != NULL) { + iter = block->NewIterator(table->rep_->options.comparator); + if (cache_handle == NULL) { + iter->RegisterCleanup(&DeleteBlock, block, NULL); + } else { + iter->RegisterCleanup(&ReleaseBlock, block_cache, cache_handle); + } + } else { + iter = NewErrorIterator(s); + } + return iter; +} + +Iterator* Table::NewIterator(const ReadOptions& options) const { + return NewTwoLevelIterator( + rep_->index_block->NewIterator(rep_->options.comparator), + &Table::BlockReader, const_cast(this), options); +} + +Status Table::InternalGet(const ReadOptions& options, const Slice& k, + void* arg, + void (*saver)(void*, const Slice&, const Slice&)) { + Status s; + Iterator* iiter = rep_->index_block->NewIterator(rep_->options.comparator); + iiter->Seek(k); + if (iiter->Valid()) { + Slice handle_value = iiter->value(); + FilterBlockReader* filter = rep_->filter; + BlockHandle handle; + if (filter != NULL && + handle.DecodeFrom(&handle_value).ok() && + !filter->KeyMayMatch(handle.offset(), k)) { + // Not found + } else { + Iterator* block_iter = BlockReader(this, options, iiter->value()); + block_iter->Seek(k); + if (block_iter->Valid()) { + (*saver)(arg, block_iter->key(), block_iter->value()); + } + s = block_iter->status(); + delete block_iter; + } + } + if (s.ok()) { + s = iiter->status(); + } + delete iiter; + return s; +} + + +uint64_t Table::ApproximateOffsetOf(const Slice& key) const { + Iterator* index_iter = + rep_->index_block->NewIterator(rep_->options.comparator); + index_iter->Seek(key); + uint64_t result; + if (index_iter->Valid()) { + BlockHandle handle; + Slice input = index_iter->value(); + Status s = handle.DecodeFrom(&input); + if (s.ok()) { + result = handle.offset(); + } else { + // Strange: we can't decode the block handle in the index block. + // We'll just return the offset of the metaindex block, which is + // close to the whole file size for this case. + result = rep_->metaindex_handle.offset(); + } + } else { + // key is past the last key in the file. Approximate the offset + // by returning the offset of the metaindex block (which is + // right near the end of the file). + result = rep_->metaindex_handle.offset(); + } + delete index_iter; + return result; +} + +} // namespace leveldb diff --git a/src/leveldb/table/table_builder.cc b/src/leveldb/table/table_builder.cc new file mode 100644 index 00000000..62002c84 --- /dev/null +++ b/src/leveldb/table/table_builder.cc @@ -0,0 +1,270 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/table_builder.h" + +#include +#include "leveldb/comparator.h" +#include "leveldb/env.h" +#include "leveldb/filter_policy.h" +#include "leveldb/options.h" +#include "table/block_builder.h" +#include "table/filter_block.h" +#include "table/format.h" +#include "util/coding.h" +#include "util/crc32c.h" + +namespace leveldb { + +struct TableBuilder::Rep { + Options options; + Options index_block_options; + WritableFile* file; + uint64_t offset; + Status status; + BlockBuilder data_block; + BlockBuilder index_block; + std::string last_key; + int64_t num_entries; + bool closed; // Either Finish() or Abandon() has been called. + FilterBlockBuilder* filter_block; + + // We do not emit the index entry for a block until we have seen the + // first key for the next data block. This allows us to use shorter + // keys in the index block. For example, consider a block boundary + // between the keys "the quick brown fox" and "the who". We can use + // "the r" as the key for the index block entry since it is >= all + // entries in the first block and < all entries in subsequent + // blocks. + // + // Invariant: r->pending_index_entry is true only if data_block is empty. + bool pending_index_entry; + BlockHandle pending_handle; // Handle to add to index block + + std::string compressed_output; + + Rep(const Options& opt, WritableFile* f) + : options(opt), + index_block_options(opt), + file(f), + offset(0), + data_block(&options), + index_block(&index_block_options), + num_entries(0), + closed(false), + filter_block(opt.filter_policy == NULL ? NULL + : new FilterBlockBuilder(opt.filter_policy)), + pending_index_entry(false) { + index_block_options.block_restart_interval = 1; + } +}; + +TableBuilder::TableBuilder(const Options& options, WritableFile* file) + : rep_(new Rep(options, file)) { + if (rep_->filter_block != NULL) { + rep_->filter_block->StartBlock(0); + } +} + +TableBuilder::~TableBuilder() { + assert(rep_->closed); // Catch errors where caller forgot to call Finish() + delete rep_->filter_block; + delete rep_; +} + +Status TableBuilder::ChangeOptions(const Options& options) { + // Note: if more fields are added to Options, update + // this function to catch changes that should not be allowed to + // change in the middle of building a Table. + if (options.comparator != rep_->options.comparator) { + return Status::InvalidArgument("changing comparator while building table"); + } + + // Note that any live BlockBuilders point to rep_->options and therefore + // will automatically pick up the updated options. + rep_->options = options; + rep_->index_block_options = options; + rep_->index_block_options.block_restart_interval = 1; + return Status::OK(); +} + +void TableBuilder::Add(const Slice& key, const Slice& value) { + Rep* r = rep_; + assert(!r->closed); + if (!ok()) return; + if (r->num_entries > 0) { + assert(r->options.comparator->Compare(key, Slice(r->last_key)) > 0); + } + + if (r->pending_index_entry) { + assert(r->data_block.empty()); + r->options.comparator->FindShortestSeparator(&r->last_key, key); + std::string handle_encoding; + r->pending_handle.EncodeTo(&handle_encoding); + r->index_block.Add(r->last_key, Slice(handle_encoding)); + r->pending_index_entry = false; + } + + if (r->filter_block != NULL) { + r->filter_block->AddKey(key); + } + + r->last_key.assign(key.data(), key.size()); + r->num_entries++; + r->data_block.Add(key, value); + + const size_t estimated_block_size = r->data_block.CurrentSizeEstimate(); + if (estimated_block_size >= r->options.block_size) { + Flush(); + } +} + +void TableBuilder::Flush() { + Rep* r = rep_; + assert(!r->closed); + if (!ok()) return; + if (r->data_block.empty()) return; + assert(!r->pending_index_entry); + WriteBlock(&r->data_block, &r->pending_handle); + if (ok()) { + r->pending_index_entry = true; + r->status = r->file->Flush(); + } + if (r->filter_block != NULL) { + r->filter_block->StartBlock(r->offset); + } +} + +void TableBuilder::WriteBlock(BlockBuilder* block, BlockHandle* handle) { + // File format contains a sequence of blocks where each block has: + // block_data: uint8[n] + // type: uint8 + // crc: uint32 + assert(ok()); + Rep* r = rep_; + Slice raw = block->Finish(); + + Slice block_contents; + CompressionType type = r->options.compression; + // TODO(postrelease): Support more compression options: zlib? + switch (type) { + case kNoCompression: + block_contents = raw; + break; + + case kSnappyCompression: { + std::string* compressed = &r->compressed_output; + if (port::Snappy_Compress(raw.data(), raw.size(), compressed) && + compressed->size() < raw.size() - (raw.size() / 8u)) { + block_contents = *compressed; + } else { + // Snappy not supported, or compressed less than 12.5%, so just + // store uncompressed form + block_contents = raw; + type = kNoCompression; + } + break; + } + } + WriteRawBlock(block_contents, type, handle); + r->compressed_output.clear(); + block->Reset(); +} + +void TableBuilder::WriteRawBlock(const Slice& block_contents, + CompressionType type, + BlockHandle* handle) { + Rep* r = rep_; + handle->set_offset(r->offset); + handle->set_size(block_contents.size()); + r->status = r->file->Append(block_contents); + if (r->status.ok()) { + char trailer[kBlockTrailerSize]; + trailer[0] = type; + uint32_t crc = crc32c::Value(block_contents.data(), block_contents.size()); + crc = crc32c::Extend(crc, trailer, 1); // Extend crc to cover block type + EncodeFixed32(trailer+1, crc32c::Mask(crc)); + r->status = r->file->Append(Slice(trailer, kBlockTrailerSize)); + if (r->status.ok()) { + r->offset += block_contents.size() + kBlockTrailerSize; + } + } +} + +Status TableBuilder::status() const { + return rep_->status; +} + +Status TableBuilder::Finish() { + Rep* r = rep_; + Flush(); + assert(!r->closed); + r->closed = true; + + BlockHandle filter_block_handle, metaindex_block_handle, index_block_handle; + + // Write filter block + if (ok() && r->filter_block != NULL) { + WriteRawBlock(r->filter_block->Finish(), kNoCompression, + &filter_block_handle); + } + + // Write metaindex block + if (ok()) { + BlockBuilder meta_index_block(&r->options); + if (r->filter_block != NULL) { + // Add mapping from "filter.Name" to location of filter data + std::string key = "filter."; + key.append(r->options.filter_policy->Name()); + std::string handle_encoding; + filter_block_handle.EncodeTo(&handle_encoding); + meta_index_block.Add(key, handle_encoding); + } + + // TODO(postrelease): Add stats and other meta blocks + WriteBlock(&meta_index_block, &metaindex_block_handle); + } + + // Write index block + if (ok()) { + if (r->pending_index_entry) { + r->options.comparator->FindShortSuccessor(&r->last_key); + std::string handle_encoding; + r->pending_handle.EncodeTo(&handle_encoding); + r->index_block.Add(r->last_key, Slice(handle_encoding)); + r->pending_index_entry = false; + } + WriteBlock(&r->index_block, &index_block_handle); + } + + // Write footer + if (ok()) { + Footer footer; + footer.set_metaindex_handle(metaindex_block_handle); + footer.set_index_handle(index_block_handle); + std::string footer_encoding; + footer.EncodeTo(&footer_encoding); + r->status = r->file->Append(footer_encoding); + if (r->status.ok()) { + r->offset += footer_encoding.size(); + } + } + return r->status; +} + +void TableBuilder::Abandon() { + Rep* r = rep_; + assert(!r->closed); + r->closed = true; +} + +uint64_t TableBuilder::NumEntries() const { + return rep_->num_entries; +} + +uint64_t TableBuilder::FileSize() const { + return rep_->offset; +} + +} // namespace leveldb diff --git a/src/leveldb/table/table_test.cc b/src/leveldb/table/table_test.cc new file mode 100644 index 00000000..c723bf84 --- /dev/null +++ b/src/leveldb/table/table_test.cc @@ -0,0 +1,868 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/table.h" + +#include +#include +#include "db/dbformat.h" +#include "db/memtable.h" +#include "db/write_batch_internal.h" +#include "leveldb/db.h" +#include "leveldb/env.h" +#include "leveldb/iterator.h" +#include "leveldb/table_builder.h" +#include "table/block.h" +#include "table/block_builder.h" +#include "table/format.h" +#include "util/random.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace leveldb { + +// Return reverse of "key". +// Used to test non-lexicographic comparators. +static std::string Reverse(const Slice& key) { + std::string str(key.ToString()); + std::string rev(""); + for (std::string::reverse_iterator rit = str.rbegin(); + rit != str.rend(); ++rit) { + rev.push_back(*rit); + } + return rev; +} + +namespace { +class ReverseKeyComparator : public Comparator { + public: + virtual const char* Name() const { + return "leveldb.ReverseBytewiseComparator"; + } + + virtual int Compare(const Slice& a, const Slice& b) const { + return BytewiseComparator()->Compare(Reverse(a), Reverse(b)); + } + + virtual void FindShortestSeparator( + std::string* start, + const Slice& limit) const { + std::string s = Reverse(*start); + std::string l = Reverse(limit); + BytewiseComparator()->FindShortestSeparator(&s, l); + *start = Reverse(s); + } + + virtual void FindShortSuccessor(std::string* key) const { + std::string s = Reverse(*key); + BytewiseComparator()->FindShortSuccessor(&s); + *key = Reverse(s); + } +}; +} // namespace +static ReverseKeyComparator reverse_key_comparator; + +static void Increment(const Comparator* cmp, std::string* key) { + if (cmp == BytewiseComparator()) { + key->push_back('\0'); + } else { + assert(cmp == &reverse_key_comparator); + std::string rev = Reverse(*key); + rev.push_back('\0'); + *key = Reverse(rev); + } +} + +// An STL comparator that uses a Comparator +namespace { +struct STLLessThan { + const Comparator* cmp; + + STLLessThan() : cmp(BytewiseComparator()) { } + STLLessThan(const Comparator* c) : cmp(c) { } + bool operator()(const std::string& a, const std::string& b) const { + return cmp->Compare(Slice(a), Slice(b)) < 0; + } +}; +} // namespace + +class StringSink: public WritableFile { + public: + ~StringSink() { } + + const std::string& contents() const { return contents_; } + + virtual Status Close() { return Status::OK(); } + virtual Status Flush() { return Status::OK(); } + virtual Status Sync() { return Status::OK(); } + + virtual Status Append(const Slice& data) { + contents_.append(data.data(), data.size()); + return Status::OK(); + } + + private: + std::string contents_; +}; + + +class StringSource: public RandomAccessFile { + public: + StringSource(const Slice& contents) + : contents_(contents.data(), contents.size()) { + } + + virtual ~StringSource() { } + + uint64_t Size() const { return contents_.size(); } + + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + if (offset > contents_.size()) { + return Status::InvalidArgument("invalid Read offset"); + } + if (offset + n > contents_.size()) { + n = contents_.size() - offset; + } + memcpy(scratch, &contents_[offset], n); + *result = Slice(scratch, n); + return Status::OK(); + } + + private: + std::string contents_; +}; + +typedef std::map KVMap; + +// Helper class for tests to unify the interface between +// BlockBuilder/TableBuilder and Block/Table. +class Constructor { + public: + explicit Constructor(const Comparator* cmp) : data_(STLLessThan(cmp)) { } + virtual ~Constructor() { } + + void Add(const std::string& key, const Slice& value) { + data_[key] = value.ToString(); + } + + // Finish constructing the data structure with all the keys that have + // been added so far. Returns the keys in sorted order in "*keys" + // and stores the key/value pairs in "*kvmap" + void Finish(const Options& options, + std::vector* keys, + KVMap* kvmap) { + *kvmap = data_; + keys->clear(); + for (KVMap::const_iterator it = data_.begin(); + it != data_.end(); + ++it) { + keys->push_back(it->first); + } + data_.clear(); + Status s = FinishImpl(options, *kvmap); + ASSERT_TRUE(s.ok()) << s.ToString(); + } + + // Construct the data structure from the data in "data" + virtual Status FinishImpl(const Options& options, const KVMap& data) = 0; + + virtual Iterator* NewIterator() const = 0; + + virtual const KVMap& data() { return data_; } + + virtual DB* db() const { return NULL; } // Overridden in DBConstructor + + private: + KVMap data_; +}; + +class BlockConstructor: public Constructor { + public: + explicit BlockConstructor(const Comparator* cmp) + : Constructor(cmp), + comparator_(cmp), + block_(NULL) { } + ~BlockConstructor() { + delete block_; + } + virtual Status FinishImpl(const Options& options, const KVMap& data) { + delete block_; + block_ = NULL; + BlockBuilder builder(&options); + + for (KVMap::const_iterator it = data.begin(); + it != data.end(); + ++it) { + builder.Add(it->first, it->second); + } + // Open the block + data_ = builder.Finish().ToString(); + BlockContents contents; + contents.data = data_; + contents.cachable = false; + contents.heap_allocated = false; + block_ = new Block(contents); + return Status::OK(); + } + virtual Iterator* NewIterator() const { + return block_->NewIterator(comparator_); + } + + private: + const Comparator* comparator_; + std::string data_; + Block* block_; + + BlockConstructor(); +}; + +class TableConstructor: public Constructor { + public: + TableConstructor(const Comparator* cmp) + : Constructor(cmp), + source_(NULL), table_(NULL) { + } + ~TableConstructor() { + Reset(); + } + virtual Status FinishImpl(const Options& options, const KVMap& data) { + Reset(); + StringSink sink; + TableBuilder builder(options, &sink); + + for (KVMap::const_iterator it = data.begin(); + it != data.end(); + ++it) { + builder.Add(it->first, it->second); + ASSERT_TRUE(builder.status().ok()); + } + Status s = builder.Finish(); + ASSERT_TRUE(s.ok()) << s.ToString(); + + ASSERT_EQ(sink.contents().size(), builder.FileSize()); + + // Open the table + source_ = new StringSource(sink.contents()); + Options table_options; + table_options.comparator = options.comparator; + return Table::Open(table_options, source_, sink.contents().size(), &table_); + } + + virtual Iterator* NewIterator() const { + return table_->NewIterator(ReadOptions()); + } + + uint64_t ApproximateOffsetOf(const Slice& key) const { + return table_->ApproximateOffsetOf(key); + } + + private: + void Reset() { + delete table_; + delete source_; + table_ = NULL; + source_ = NULL; + } + + StringSource* source_; + Table* table_; + + TableConstructor(); +}; + +// A helper class that converts internal format keys into user keys +class KeyConvertingIterator: public Iterator { + public: + explicit KeyConvertingIterator(Iterator* iter) : iter_(iter) { } + virtual ~KeyConvertingIterator() { delete iter_; } + virtual bool Valid() const { return iter_->Valid(); } + virtual void Seek(const Slice& target) { + ParsedInternalKey ikey(target, kMaxSequenceNumber, kTypeValue); + std::string encoded; + AppendInternalKey(&encoded, ikey); + iter_->Seek(encoded); + } + virtual void SeekToFirst() { iter_->SeekToFirst(); } + virtual void SeekToLast() { iter_->SeekToLast(); } + virtual void Next() { iter_->Next(); } + virtual void Prev() { iter_->Prev(); } + + virtual Slice key() const { + assert(Valid()); + ParsedInternalKey key; + if (!ParseInternalKey(iter_->key(), &key)) { + status_ = Status::Corruption("malformed internal key"); + return Slice("corrupted key"); + } + return key.user_key; + } + + virtual Slice value() const { return iter_->value(); } + virtual Status status() const { + return status_.ok() ? iter_->status() : status_; + } + + private: + mutable Status status_; + Iterator* iter_; + + // No copying allowed + KeyConvertingIterator(const KeyConvertingIterator&); + void operator=(const KeyConvertingIterator&); +}; + +class MemTableConstructor: public Constructor { + public: + explicit MemTableConstructor(const Comparator* cmp) + : Constructor(cmp), + internal_comparator_(cmp) { + memtable_ = new MemTable(internal_comparator_); + memtable_->Ref(); + } + ~MemTableConstructor() { + memtable_->Unref(); + } + virtual Status FinishImpl(const Options& options, const KVMap& data) { + memtable_->Unref(); + memtable_ = new MemTable(internal_comparator_); + memtable_->Ref(); + int seq = 1; + for (KVMap::const_iterator it = data.begin(); + it != data.end(); + ++it) { + memtable_->Add(seq, kTypeValue, it->first, it->second); + seq++; + } + return Status::OK(); + } + virtual Iterator* NewIterator() const { + return new KeyConvertingIterator(memtable_->NewIterator()); + } + + private: + InternalKeyComparator internal_comparator_; + MemTable* memtable_; +}; + +class DBConstructor: public Constructor { + public: + explicit DBConstructor(const Comparator* cmp) + : Constructor(cmp), + comparator_(cmp) { + db_ = NULL; + NewDB(); + } + ~DBConstructor() { + delete db_; + } + virtual Status FinishImpl(const Options& options, const KVMap& data) { + delete db_; + db_ = NULL; + NewDB(); + for (KVMap::const_iterator it = data.begin(); + it != data.end(); + ++it) { + WriteBatch batch; + batch.Put(it->first, it->second); + ASSERT_TRUE(db_->Write(WriteOptions(), &batch).ok()); + } + return Status::OK(); + } + virtual Iterator* NewIterator() const { + return db_->NewIterator(ReadOptions()); + } + + virtual DB* db() const { return db_; } + + private: + void NewDB() { + std::string name = test::TmpDir() + "/table_testdb"; + + Options options; + options.comparator = comparator_; + Status status = DestroyDB(name, options); + ASSERT_TRUE(status.ok()) << status.ToString(); + + options.create_if_missing = true; + options.error_if_exists = true; + options.write_buffer_size = 10000; // Something small to force merging + status = DB::Open(options, name, &db_); + ASSERT_TRUE(status.ok()) << status.ToString(); + } + + const Comparator* comparator_; + DB* db_; +}; + +enum TestType { + TABLE_TEST, + BLOCK_TEST, + MEMTABLE_TEST, + DB_TEST +}; + +struct TestArgs { + TestType type; + bool reverse_compare; + int restart_interval; +}; + +static const TestArgs kTestArgList[] = { + { TABLE_TEST, false, 16 }, + { TABLE_TEST, false, 1 }, + { TABLE_TEST, false, 1024 }, + { TABLE_TEST, true, 16 }, + { TABLE_TEST, true, 1 }, + { TABLE_TEST, true, 1024 }, + + { BLOCK_TEST, false, 16 }, + { BLOCK_TEST, false, 1 }, + { BLOCK_TEST, false, 1024 }, + { BLOCK_TEST, true, 16 }, + { BLOCK_TEST, true, 1 }, + { BLOCK_TEST, true, 1024 }, + + // Restart interval does not matter for memtables + { MEMTABLE_TEST, false, 16 }, + { MEMTABLE_TEST, true, 16 }, + + // Do not bother with restart interval variations for DB + { DB_TEST, false, 16 }, + { DB_TEST, true, 16 }, +}; +static const int kNumTestArgs = sizeof(kTestArgList) / sizeof(kTestArgList[0]); + +class Harness { + public: + Harness() : constructor_(NULL) { } + + void Init(const TestArgs& args) { + delete constructor_; + constructor_ = NULL; + options_ = Options(); + + options_.block_restart_interval = args.restart_interval; + // Use shorter block size for tests to exercise block boundary + // conditions more. + options_.block_size = 256; + if (args.reverse_compare) { + options_.comparator = &reverse_key_comparator; + } + switch (args.type) { + case TABLE_TEST: + constructor_ = new TableConstructor(options_.comparator); + break; + case BLOCK_TEST: + constructor_ = new BlockConstructor(options_.comparator); + break; + case MEMTABLE_TEST: + constructor_ = new MemTableConstructor(options_.comparator); + break; + case DB_TEST: + constructor_ = new DBConstructor(options_.comparator); + break; + } + } + + ~Harness() { + delete constructor_; + } + + void Add(const std::string& key, const std::string& value) { + constructor_->Add(key, value); + } + + void Test(Random* rnd) { + std::vector keys; + KVMap data; + constructor_->Finish(options_, &keys, &data); + + TestForwardScan(keys, data); + TestBackwardScan(keys, data); + TestRandomAccess(rnd, keys, data); + } + + void TestForwardScan(const std::vector& keys, + const KVMap& data) { + Iterator* iter = constructor_->NewIterator(); + ASSERT_TRUE(!iter->Valid()); + iter->SeekToFirst(); + for (KVMap::const_iterator model_iter = data.begin(); + model_iter != data.end(); + ++model_iter) { + ASSERT_EQ(ToString(data, model_iter), ToString(iter)); + iter->Next(); + } + ASSERT_TRUE(!iter->Valid()); + delete iter; + } + + void TestBackwardScan(const std::vector& keys, + const KVMap& data) { + Iterator* iter = constructor_->NewIterator(); + ASSERT_TRUE(!iter->Valid()); + iter->SeekToLast(); + for (KVMap::const_reverse_iterator model_iter = data.rbegin(); + model_iter != data.rend(); + ++model_iter) { + ASSERT_EQ(ToString(data, model_iter), ToString(iter)); + iter->Prev(); + } + ASSERT_TRUE(!iter->Valid()); + delete iter; + } + + void TestRandomAccess(Random* rnd, + const std::vector& keys, + const KVMap& data) { + static const bool kVerbose = false; + Iterator* iter = constructor_->NewIterator(); + ASSERT_TRUE(!iter->Valid()); + KVMap::const_iterator model_iter = data.begin(); + if (kVerbose) fprintf(stderr, "---\n"); + for (int i = 0; i < 200; i++) { + const int toss = rnd->Uniform(5); + switch (toss) { + case 0: { + if (iter->Valid()) { + if (kVerbose) fprintf(stderr, "Next\n"); + iter->Next(); + ++model_iter; + ASSERT_EQ(ToString(data, model_iter), ToString(iter)); + } + break; + } + + case 1: { + if (kVerbose) fprintf(stderr, "SeekToFirst\n"); + iter->SeekToFirst(); + model_iter = data.begin(); + ASSERT_EQ(ToString(data, model_iter), ToString(iter)); + break; + } + + case 2: { + std::string key = PickRandomKey(rnd, keys); + model_iter = data.lower_bound(key); + if (kVerbose) fprintf(stderr, "Seek '%s'\n", + EscapeString(key).c_str()); + iter->Seek(Slice(key)); + ASSERT_EQ(ToString(data, model_iter), ToString(iter)); + break; + } + + case 3: { + if (iter->Valid()) { + if (kVerbose) fprintf(stderr, "Prev\n"); + iter->Prev(); + if (model_iter == data.begin()) { + model_iter = data.end(); // Wrap around to invalid value + } else { + --model_iter; + } + ASSERT_EQ(ToString(data, model_iter), ToString(iter)); + } + break; + } + + case 4: { + if (kVerbose) fprintf(stderr, "SeekToLast\n"); + iter->SeekToLast(); + if (keys.empty()) { + model_iter = data.end(); + } else { + std::string last = data.rbegin()->first; + model_iter = data.lower_bound(last); + } + ASSERT_EQ(ToString(data, model_iter), ToString(iter)); + break; + } + } + } + delete iter; + } + + std::string ToString(const KVMap& data, const KVMap::const_iterator& it) { + if (it == data.end()) { + return "END"; + } else { + return "'" + it->first + "->" + it->second + "'"; + } + } + + std::string ToString(const KVMap& data, + const KVMap::const_reverse_iterator& it) { + if (it == data.rend()) { + return "END"; + } else { + return "'" + it->first + "->" + it->second + "'"; + } + } + + std::string ToString(const Iterator* it) { + if (!it->Valid()) { + return "END"; + } else { + return "'" + it->key().ToString() + "->" + it->value().ToString() + "'"; + } + } + + std::string PickRandomKey(Random* rnd, const std::vector& keys) { + if (keys.empty()) { + return "foo"; + } else { + const int index = rnd->Uniform(keys.size()); + std::string result = keys[index]; + switch (rnd->Uniform(3)) { + case 0: + // Return an existing key + break; + case 1: { + // Attempt to return something smaller than an existing key + if (result.size() > 0 && result[result.size()-1] > '\0') { + result[result.size()-1]--; + } + break; + } + case 2: { + // Return something larger than an existing key + Increment(options_.comparator, &result); + break; + } + } + return result; + } + } + + // Returns NULL if not running against a DB + DB* db() const { return constructor_->db(); } + + private: + Options options_; + Constructor* constructor_; +}; + +// Test empty table/block. +TEST(Harness, Empty) { + for (int i = 0; i < kNumTestArgs; i++) { + Init(kTestArgList[i]); + Random rnd(test::RandomSeed() + 1); + Test(&rnd); + } +} + +// Special test for a block with no restart entries. The C++ leveldb +// code never generates such blocks, but the Java version of leveldb +// seems to. +TEST(Harness, ZeroRestartPointsInBlock) { + char data[sizeof(uint32_t)]; + memset(data, 0, sizeof(data)); + BlockContents contents; + contents.data = Slice(data, sizeof(data)); + contents.cachable = false; + contents.heap_allocated = false; + Block block(contents); + Iterator* iter = block.NewIterator(BytewiseComparator()); + iter->SeekToFirst(); + ASSERT_TRUE(!iter->Valid()); + iter->SeekToLast(); + ASSERT_TRUE(!iter->Valid()); + iter->Seek("foo"); + ASSERT_TRUE(!iter->Valid()); + delete iter; +} + +// Test the empty key +TEST(Harness, SimpleEmptyKey) { + for (int i = 0; i < kNumTestArgs; i++) { + Init(kTestArgList[i]); + Random rnd(test::RandomSeed() + 1); + Add("", "v"); + Test(&rnd); + } +} + +TEST(Harness, SimpleSingle) { + for (int i = 0; i < kNumTestArgs; i++) { + Init(kTestArgList[i]); + Random rnd(test::RandomSeed() + 2); + Add("abc", "v"); + Test(&rnd); + } +} + +TEST(Harness, SimpleMulti) { + for (int i = 0; i < kNumTestArgs; i++) { + Init(kTestArgList[i]); + Random rnd(test::RandomSeed() + 3); + Add("abc", "v"); + Add("abcd", "v"); + Add("ac", "v2"); + Test(&rnd); + } +} + +TEST(Harness, SimpleSpecialKey) { + for (int i = 0; i < kNumTestArgs; i++) { + Init(kTestArgList[i]); + Random rnd(test::RandomSeed() + 4); + Add("\xff\xff", "v3"); + Test(&rnd); + } +} + +TEST(Harness, Randomized) { + for (int i = 0; i < kNumTestArgs; i++) { + Init(kTestArgList[i]); + Random rnd(test::RandomSeed() + 5); + for (int num_entries = 0; num_entries < 2000; + num_entries += (num_entries < 50 ? 1 : 200)) { + if ((num_entries % 10) == 0) { + fprintf(stderr, "case %d of %d: num_entries = %d\n", + (i + 1), int(kNumTestArgs), num_entries); + } + for (int e = 0; e < num_entries; e++) { + std::string v; + Add(test::RandomKey(&rnd, rnd.Skewed(4)), + test::RandomString(&rnd, rnd.Skewed(5), &v).ToString()); + } + Test(&rnd); + } + } +} + +TEST(Harness, RandomizedLongDB) { + Random rnd(test::RandomSeed()); + TestArgs args = { DB_TEST, false, 16 }; + Init(args); + int num_entries = 100000; + for (int e = 0; e < num_entries; e++) { + std::string v; + Add(test::RandomKey(&rnd, rnd.Skewed(4)), + test::RandomString(&rnd, rnd.Skewed(5), &v).ToString()); + } + Test(&rnd); + + // We must have created enough data to force merging + int files = 0; + for (int level = 0; level < config::kNumLevels; level++) { + std::string value; + char name[100]; + snprintf(name, sizeof(name), "leveldb.num-files-at-level%d", level); + ASSERT_TRUE(db()->GetProperty(name, &value)); + files += atoi(value.c_str()); + } + ASSERT_GT(files, 0); +} + +class MemTableTest { }; + +TEST(MemTableTest, Simple) { + InternalKeyComparator cmp(BytewiseComparator()); + MemTable* memtable = new MemTable(cmp); + memtable->Ref(); + WriteBatch batch; + WriteBatchInternal::SetSequence(&batch, 100); + batch.Put(std::string("k1"), std::string("v1")); + batch.Put(std::string("k2"), std::string("v2")); + batch.Put(std::string("k3"), std::string("v3")); + batch.Put(std::string("largekey"), std::string("vlarge")); + ASSERT_TRUE(WriteBatchInternal::InsertInto(&batch, memtable).ok()); + + Iterator* iter = memtable->NewIterator(); + iter->SeekToFirst(); + while (iter->Valid()) { + fprintf(stderr, "key: '%s' -> '%s'\n", + iter->key().ToString().c_str(), + iter->value().ToString().c_str()); + iter->Next(); + } + + delete iter; + memtable->Unref(); +} + +static bool Between(uint64_t val, uint64_t low, uint64_t high) { + bool result = (val >= low) && (val <= high); + if (!result) { + fprintf(stderr, "Value %llu is not in range [%llu, %llu]\n", + (unsigned long long)(val), + (unsigned long long)(low), + (unsigned long long)(high)); + } + return result; +} + +class TableTest { }; + +TEST(TableTest, ApproximateOffsetOfPlain) { + TableConstructor c(BytewiseComparator()); + c.Add("k01", "hello"); + c.Add("k02", "hello2"); + c.Add("k03", std::string(10000, 'x')); + c.Add("k04", std::string(200000, 'x')); + c.Add("k05", std::string(300000, 'x')); + c.Add("k06", "hello3"); + c.Add("k07", std::string(100000, 'x')); + std::vector keys; + KVMap kvmap; + Options options; + options.block_size = 1024; + options.compression = kNoCompression; + c.Finish(options, &keys, &kvmap); + + ASSERT_TRUE(Between(c.ApproximateOffsetOf("abc"), 0, 0)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k01"), 0, 0)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k01a"), 0, 0)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k02"), 0, 0)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k03"), 0, 0)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k04"), 10000, 11000)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k04a"), 210000, 211000)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k05"), 210000, 211000)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k06"), 510000, 511000)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k07"), 510000, 511000)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("xyz"), 610000, 612000)); + +} + +static bool SnappyCompressionSupported() { + std::string out; + Slice in = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + return port::Snappy_Compress(in.data(), in.size(), &out); +} + +TEST(TableTest, ApproximateOffsetOfCompressed) { + if (!SnappyCompressionSupported()) { + fprintf(stderr, "skipping compression tests\n"); + return; + } + + Random rnd(301); + TableConstructor c(BytewiseComparator()); + std::string tmp; + c.Add("k01", "hello"); + c.Add("k02", test::CompressibleString(&rnd, 0.25, 10000, &tmp)); + c.Add("k03", "hello3"); + c.Add("k04", test::CompressibleString(&rnd, 0.25, 10000, &tmp)); + std::vector keys; + KVMap kvmap; + Options options; + options.block_size = 1024; + options.compression = kSnappyCompression; + c.Finish(options, &keys, &kvmap); + + ASSERT_TRUE(Between(c.ApproximateOffsetOf("abc"), 0, 0)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k01"), 0, 0)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k02"), 0, 0)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k03"), 2000, 3000)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k04"), 2000, 3000)); + ASSERT_TRUE(Between(c.ApproximateOffsetOf("xyz"), 4000, 6000)); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/table/two_level_iterator.cc b/src/leveldb/table/two_level_iterator.cc new file mode 100644 index 00000000..7822ebab --- /dev/null +++ b/src/leveldb/table/two_level_iterator.cc @@ -0,0 +1,182 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "table/two_level_iterator.h" + +#include "leveldb/table.h" +#include "table/block.h" +#include "table/format.h" +#include "table/iterator_wrapper.h" + +namespace leveldb { + +namespace { + +typedef Iterator* (*BlockFunction)(void*, const ReadOptions&, const Slice&); + +class TwoLevelIterator: public Iterator { + public: + TwoLevelIterator( + Iterator* index_iter, + BlockFunction block_function, + void* arg, + const ReadOptions& options); + + virtual ~TwoLevelIterator(); + + virtual void Seek(const Slice& target); + virtual void SeekToFirst(); + virtual void SeekToLast(); + virtual void Next(); + virtual void Prev(); + + virtual bool Valid() const { + return data_iter_.Valid(); + } + virtual Slice key() const { + assert(Valid()); + return data_iter_.key(); + } + virtual Slice value() const { + assert(Valid()); + return data_iter_.value(); + } + virtual Status status() const { + // It'd be nice if status() returned a const Status& instead of a Status + if (!index_iter_.status().ok()) { + return index_iter_.status(); + } else if (data_iter_.iter() != NULL && !data_iter_.status().ok()) { + return data_iter_.status(); + } else { + return status_; + } + } + + private: + void SaveError(const Status& s) { + if (status_.ok() && !s.ok()) status_ = s; + } + void SkipEmptyDataBlocksForward(); + void SkipEmptyDataBlocksBackward(); + void SetDataIterator(Iterator* data_iter); + void InitDataBlock(); + + BlockFunction block_function_; + void* arg_; + const ReadOptions options_; + Status status_; + IteratorWrapper index_iter_; + IteratorWrapper data_iter_; // May be NULL + // If data_iter_ is non-NULL, then "data_block_handle_" holds the + // "index_value" passed to block_function_ to create the data_iter_. + std::string data_block_handle_; +}; + +TwoLevelIterator::TwoLevelIterator( + Iterator* index_iter, + BlockFunction block_function, + void* arg, + const ReadOptions& options) + : block_function_(block_function), + arg_(arg), + options_(options), + index_iter_(index_iter), + data_iter_(NULL) { +} + +TwoLevelIterator::~TwoLevelIterator() { +} + +void TwoLevelIterator::Seek(const Slice& target) { + index_iter_.Seek(target); + InitDataBlock(); + if (data_iter_.iter() != NULL) data_iter_.Seek(target); + SkipEmptyDataBlocksForward(); +} + +void TwoLevelIterator::SeekToFirst() { + index_iter_.SeekToFirst(); + InitDataBlock(); + if (data_iter_.iter() != NULL) data_iter_.SeekToFirst(); + SkipEmptyDataBlocksForward(); +} + +void TwoLevelIterator::SeekToLast() { + index_iter_.SeekToLast(); + InitDataBlock(); + if (data_iter_.iter() != NULL) data_iter_.SeekToLast(); + SkipEmptyDataBlocksBackward(); +} + +void TwoLevelIterator::Next() { + assert(Valid()); + data_iter_.Next(); + SkipEmptyDataBlocksForward(); +} + +void TwoLevelIterator::Prev() { + assert(Valid()); + data_iter_.Prev(); + SkipEmptyDataBlocksBackward(); +} + + +void TwoLevelIterator::SkipEmptyDataBlocksForward() { + while (data_iter_.iter() == NULL || !data_iter_.Valid()) { + // Move to next block + if (!index_iter_.Valid()) { + SetDataIterator(NULL); + return; + } + index_iter_.Next(); + InitDataBlock(); + if (data_iter_.iter() != NULL) data_iter_.SeekToFirst(); + } +} + +void TwoLevelIterator::SkipEmptyDataBlocksBackward() { + while (data_iter_.iter() == NULL || !data_iter_.Valid()) { + // Move to next block + if (!index_iter_.Valid()) { + SetDataIterator(NULL); + return; + } + index_iter_.Prev(); + InitDataBlock(); + if (data_iter_.iter() != NULL) data_iter_.SeekToLast(); + } +} + +void TwoLevelIterator::SetDataIterator(Iterator* data_iter) { + if (data_iter_.iter() != NULL) SaveError(data_iter_.status()); + data_iter_.Set(data_iter); +} + +void TwoLevelIterator::InitDataBlock() { + if (!index_iter_.Valid()) { + SetDataIterator(NULL); + } else { + Slice handle = index_iter_.value(); + if (data_iter_.iter() != NULL && handle.compare(data_block_handle_) == 0) { + // data_iter_ is already constructed with this iterator, so + // no need to change anything + } else { + Iterator* iter = (*block_function_)(arg_, options_, handle); + data_block_handle_.assign(handle.data(), handle.size()); + SetDataIterator(iter); + } + } +} + +} // namespace + +Iterator* NewTwoLevelIterator( + Iterator* index_iter, + BlockFunction block_function, + void* arg, + const ReadOptions& options) { + return new TwoLevelIterator(index_iter, block_function, arg, options); +} + +} // namespace leveldb diff --git a/src/leveldb/table/two_level_iterator.h b/src/leveldb/table/two_level_iterator.h new file mode 100644 index 00000000..629ca345 --- /dev/null +++ b/src/leveldb/table/two_level_iterator.h @@ -0,0 +1,34 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_TABLE_TWO_LEVEL_ITERATOR_H_ +#define STORAGE_LEVELDB_TABLE_TWO_LEVEL_ITERATOR_H_ + +#include "leveldb/iterator.h" + +namespace leveldb { + +struct ReadOptions; + +// Return a new two level iterator. A two-level iterator contains an +// index iterator whose values point to a sequence of blocks where +// each block is itself a sequence of key,value pairs. The returned +// two-level iterator yields the concatenation of all key/value pairs +// in the sequence of blocks. Takes ownership of "index_iter" and +// will delete it when no longer needed. +// +// Uses a supplied function to convert an index_iter value into +// an iterator over the contents of the corresponding block. +extern Iterator* NewTwoLevelIterator( + Iterator* index_iter, + Iterator* (*block_function)( + void* arg, + const ReadOptions& options, + const Slice& index_value), + void* arg, + const ReadOptions& options); + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_TABLE_TWO_LEVEL_ITERATOR_H_ diff --git a/src/leveldb/util/arena.cc b/src/leveldb/util/arena.cc new file mode 100644 index 00000000..9367f714 --- /dev/null +++ b/src/leveldb/util/arena.cc @@ -0,0 +1,68 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "util/arena.h" +#include + +namespace leveldb { + +static const int kBlockSize = 4096; + +Arena::Arena() { + blocks_memory_ = 0; + alloc_ptr_ = NULL; // First allocation will allocate a block + alloc_bytes_remaining_ = 0; +} + +Arena::~Arena() { + for (size_t i = 0; i < blocks_.size(); i++) { + delete[] blocks_[i]; + } +} + +char* Arena::AllocateFallback(size_t bytes) { + if (bytes > kBlockSize / 4) { + // Object is more than a quarter of our block size. Allocate it separately + // to avoid wasting too much space in leftover bytes. + char* result = AllocateNewBlock(bytes); + return result; + } + + // We waste the remaining space in the current block. + alloc_ptr_ = AllocateNewBlock(kBlockSize); + alloc_bytes_remaining_ = kBlockSize; + + char* result = alloc_ptr_; + alloc_ptr_ += bytes; + alloc_bytes_remaining_ -= bytes; + return result; +} + +char* Arena::AllocateAligned(size_t bytes) { + const int align = (sizeof(void*) > 8) ? sizeof(void*) : 8; + assert((align & (align-1)) == 0); // Pointer size should be a power of 2 + size_t current_mod = reinterpret_cast(alloc_ptr_) & (align-1); + size_t slop = (current_mod == 0 ? 0 : align - current_mod); + size_t needed = bytes + slop; + char* result; + if (needed <= alloc_bytes_remaining_) { + result = alloc_ptr_ + slop; + alloc_ptr_ += needed; + alloc_bytes_remaining_ -= needed; + } else { + // AllocateFallback always returned aligned memory + result = AllocateFallback(bytes); + } + assert((reinterpret_cast(result) & (align-1)) == 0); + return result; +} + +char* Arena::AllocateNewBlock(size_t block_bytes) { + char* result = new char[block_bytes]; + blocks_memory_ += block_bytes; + blocks_.push_back(result); + return result; +} + +} // namespace leveldb diff --git a/src/leveldb/util/arena.h b/src/leveldb/util/arena.h new file mode 100644 index 00000000..73bbf1cb --- /dev/null +++ b/src/leveldb/util/arena.h @@ -0,0 +1,68 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_UTIL_ARENA_H_ +#define STORAGE_LEVELDB_UTIL_ARENA_H_ + +#include +#include +#include +#include + +namespace leveldb { + +class Arena { + public: + Arena(); + ~Arena(); + + // Return a pointer to a newly allocated memory block of "bytes" bytes. + char* Allocate(size_t bytes); + + // Allocate memory with the normal alignment guarantees provided by malloc + char* AllocateAligned(size_t bytes); + + // Returns an estimate of the total memory usage of data allocated + // by the arena (including space allocated but not yet used for user + // allocations). + size_t MemoryUsage() const { + return blocks_memory_ + blocks_.capacity() * sizeof(char*); + } + + private: + char* AllocateFallback(size_t bytes); + char* AllocateNewBlock(size_t block_bytes); + + // Allocation state + char* alloc_ptr_; + size_t alloc_bytes_remaining_; + + // Array of new[] allocated memory blocks + std::vector blocks_; + + // Bytes of memory in blocks allocated so far + size_t blocks_memory_; + + // No copying allowed + Arena(const Arena&); + void operator=(const Arena&); +}; + +inline char* Arena::Allocate(size_t bytes) { + // The semantics of what to return are a bit messy if we allow + // 0-byte allocations, so we disallow them here (we don't need + // them for our internal use). + assert(bytes > 0); + if (bytes <= alloc_bytes_remaining_) { + char* result = alloc_ptr_; + alloc_ptr_ += bytes; + alloc_bytes_remaining_ -= bytes; + return result; + } + return AllocateFallback(bytes); +} + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_UTIL_ARENA_H_ diff --git a/src/leveldb/util/arena_test.cc b/src/leveldb/util/arena_test.cc new file mode 100644 index 00000000..58e870ec --- /dev/null +++ b/src/leveldb/util/arena_test.cc @@ -0,0 +1,68 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "util/arena.h" + +#include "util/random.h" +#include "util/testharness.h" + +namespace leveldb { + +class ArenaTest { }; + +TEST(ArenaTest, Empty) { + Arena arena; +} + +TEST(ArenaTest, Simple) { + std::vector > allocated; + Arena arena; + const int N = 100000; + size_t bytes = 0; + Random rnd(301); + for (int i = 0; i < N; i++) { + size_t s; + if (i % (N / 10) == 0) { + s = i; + } else { + s = rnd.OneIn(4000) ? rnd.Uniform(6000) : + (rnd.OneIn(10) ? rnd.Uniform(100) : rnd.Uniform(20)); + } + if (s == 0) { + // Our arena disallows size 0 allocations. + s = 1; + } + char* r; + if (rnd.OneIn(10)) { + r = arena.AllocateAligned(s); + } else { + r = arena.Allocate(s); + } + + for (size_t b = 0; b < s; b++) { + // Fill the "i"th allocation with a known bit pattern + r[b] = i % 256; + } + bytes += s; + allocated.push_back(std::make_pair(s, r)); + ASSERT_GE(arena.MemoryUsage(), bytes); + if (i > N/10) { + ASSERT_LE(arena.MemoryUsage(), bytes * 1.10); + } + } + for (size_t i = 0; i < allocated.size(); i++) { + size_t num_bytes = allocated[i].first; + const char* p = allocated[i].second; + for (size_t b = 0; b < num_bytes; b++) { + // Check the "i"th allocation for the known bit pattern + ASSERT_EQ(int(p[b]) & 0xff, i % 256); + } + } +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/util/bloom.cc b/src/leveldb/util/bloom.cc new file mode 100644 index 00000000..a27a2ace --- /dev/null +++ b/src/leveldb/util/bloom.cc @@ -0,0 +1,95 @@ +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/filter_policy.h" + +#include "leveldb/slice.h" +#include "util/hash.h" + +namespace leveldb { + +namespace { +static uint32_t BloomHash(const Slice& key) { + return Hash(key.data(), key.size(), 0xbc9f1d34); +} + +class BloomFilterPolicy : public FilterPolicy { + private: + size_t bits_per_key_; + size_t k_; + + public: + explicit BloomFilterPolicy(int bits_per_key) + : bits_per_key_(bits_per_key) { + // We intentionally round down to reduce probing cost a little bit + k_ = static_cast(bits_per_key * 0.69); // 0.69 =~ ln(2) + if (k_ < 1) k_ = 1; + if (k_ > 30) k_ = 30; + } + + virtual const char* Name() const { + return "leveldb.BuiltinBloomFilter2"; + } + + virtual void CreateFilter(const Slice* keys, int n, std::string* dst) const { + // Compute bloom filter size (in both bits and bytes) + size_t bits = n * bits_per_key_; + + // For small n, we can see a very high false positive rate. Fix it + // by enforcing a minimum bloom filter length. + if (bits < 64) bits = 64; + + size_t bytes = (bits + 7) / 8; + bits = bytes * 8; + + const size_t init_size = dst->size(); + dst->resize(init_size + bytes, 0); + dst->push_back(static_cast(k_)); // Remember # of probes in filter + char* array = &(*dst)[init_size]; + for (size_t i = 0; i < n; i++) { + // Use double-hashing to generate a sequence of hash values. + // See analysis in [Kirsch,Mitzenmacher 2006]. + uint32_t h = BloomHash(keys[i]); + const uint32_t delta = (h >> 17) | (h << 15); // Rotate right 17 bits + for (size_t j = 0; j < k_; j++) { + const uint32_t bitpos = h % bits; + array[bitpos/8] |= (1 << (bitpos % 8)); + h += delta; + } + } + } + + virtual bool KeyMayMatch(const Slice& key, const Slice& bloom_filter) const { + const size_t len = bloom_filter.size(); + if (len < 2) return false; + + const char* array = bloom_filter.data(); + const size_t bits = (len - 1) * 8; + + // Use the encoded k so that we can read filters generated by + // bloom filters created using different parameters. + const size_t k = array[len-1]; + if (k > 30) { + // Reserved for potentially new encodings for short bloom filters. + // Consider it a match. + return true; + } + + uint32_t h = BloomHash(key); + const uint32_t delta = (h >> 17) | (h << 15); // Rotate right 17 bits + for (size_t j = 0; j < k; j++) { + const uint32_t bitpos = h % bits; + if ((array[bitpos/8] & (1 << (bitpos % 8))) == 0) return false; + h += delta; + } + return true; + } +}; +} + +const FilterPolicy* NewBloomFilterPolicy(int bits_per_key) { + return new BloomFilterPolicy(bits_per_key); +} + +} // namespace leveldb diff --git a/src/leveldb/util/bloom_test.cc b/src/leveldb/util/bloom_test.cc new file mode 100644 index 00000000..77fb1b31 --- /dev/null +++ b/src/leveldb/util/bloom_test.cc @@ -0,0 +1,161 @@ +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/filter_policy.h" + +#include "util/coding.h" +#include "util/logging.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace leveldb { + +static const int kVerbose = 1; + +static Slice Key(int i, char* buffer) { + EncodeFixed32(buffer, i); + return Slice(buffer, sizeof(uint32_t)); +} + +class BloomTest { + private: + const FilterPolicy* policy_; + std::string filter_; + std::vector keys_; + + public: + BloomTest() : policy_(NewBloomFilterPolicy(10)) { } + + ~BloomTest() { + delete policy_; + } + + void Reset() { + keys_.clear(); + filter_.clear(); + } + + void Add(const Slice& s) { + keys_.push_back(s.ToString()); + } + + void Build() { + std::vector key_slices; + for (size_t i = 0; i < keys_.size(); i++) { + key_slices.push_back(Slice(keys_[i])); + } + filter_.clear(); + policy_->CreateFilter(&key_slices[0], key_slices.size(), &filter_); + keys_.clear(); + if (kVerbose >= 2) DumpFilter(); + } + + size_t FilterSize() const { + return filter_.size(); + } + + void DumpFilter() { + fprintf(stderr, "F("); + for (size_t i = 0; i+1 < filter_.size(); i++) { + const unsigned int c = static_cast(filter_[i]); + for (int j = 0; j < 8; j++) { + fprintf(stderr, "%c", (c & (1 <KeyMayMatch(s, filter_); + } + + double FalsePositiveRate() { + char buffer[sizeof(int)]; + int result = 0; + for (int i = 0; i < 10000; i++) { + if (Matches(Key(i + 1000000000, buffer))) { + result++; + } + } + return result / 10000.0; + } +}; + +TEST(BloomTest, EmptyFilter) { + ASSERT_TRUE(! Matches("hello")); + ASSERT_TRUE(! Matches("world")); +} + +TEST(BloomTest, Small) { + Add("hello"); + Add("world"); + ASSERT_TRUE(Matches("hello")); + ASSERT_TRUE(Matches("world")); + ASSERT_TRUE(! Matches("x")); + ASSERT_TRUE(! Matches("foo")); +} + +static int NextLength(int length) { + if (length < 10) { + length += 1; + } else if (length < 100) { + length += 10; + } else if (length < 1000) { + length += 100; + } else { + length += 1000; + } + return length; +} + +TEST(BloomTest, VaryingLengths) { + char buffer[sizeof(int)]; + + // Count number of filters that significantly exceed the false positive rate + int mediocre_filters = 0; + int good_filters = 0; + + for (int length = 1; length <= 10000; length = NextLength(length)) { + Reset(); + for (int i = 0; i < length; i++) { + Add(Key(i, buffer)); + } + Build(); + + ASSERT_LE(FilterSize(), static_cast((length * 10 / 8) + 40)) + << length; + + // All added keys must match + for (int i = 0; i < length; i++) { + ASSERT_TRUE(Matches(Key(i, buffer))) + << "Length " << length << "; key " << i; + } + + // Check false positive rate + double rate = FalsePositiveRate(); + if (kVerbose >= 1) { + fprintf(stderr, "False positives: %5.2f%% @ length = %6d ; bytes = %6d\n", + rate*100.0, length, static_cast(FilterSize())); + } + ASSERT_LE(rate, 0.02); // Must not be over 2% + if (rate > 0.0125) mediocre_filters++; // Allowed, but not too often + else good_filters++; + } + if (kVerbose >= 1) { + fprintf(stderr, "Filters: %d good, %d mediocre\n", + good_filters, mediocre_filters); + } + ASSERT_LE(mediocre_filters, good_filters/5); +} + +// Different bits-per-byte + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/util/cache.cc b/src/leveldb/util/cache.cc new file mode 100644 index 00000000..8b197bc0 --- /dev/null +++ b/src/leveldb/util/cache.cc @@ -0,0 +1,325 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include +#include + +#include "leveldb/cache.h" +#include "port/port.h" +#include "util/hash.h" +#include "util/mutexlock.h" + +namespace leveldb { + +Cache::~Cache() { +} + +namespace { + +// LRU cache implementation + +// An entry is a variable length heap-allocated structure. Entries +// are kept in a circular doubly linked list ordered by access time. +struct LRUHandle { + void* value; + void (*deleter)(const Slice&, void* value); + LRUHandle* next_hash; + LRUHandle* next; + LRUHandle* prev; + size_t charge; // TODO(opt): Only allow uint32_t? + size_t key_length; + uint32_t refs; + uint32_t hash; // Hash of key(); used for fast sharding and comparisons + char key_data[1]; // Beginning of key + + Slice key() const { + // For cheaper lookups, we allow a temporary Handle object + // to store a pointer to a key in "value". + if (next == this) { + return *(reinterpret_cast(value)); + } else { + return Slice(key_data, key_length); + } + } +}; + +// We provide our own simple hash table since it removes a whole bunch +// of porting hacks and is also faster than some of the built-in hash +// table implementations in some of the compiler/runtime combinations +// we have tested. E.g., readrandom speeds up by ~5% over the g++ +// 4.4.3's builtin hashtable. +class HandleTable { + public: + HandleTable() : length_(0), elems_(0), list_(NULL) { Resize(); } + ~HandleTable() { delete[] list_; } + + LRUHandle* Lookup(const Slice& key, uint32_t hash) { + return *FindPointer(key, hash); + } + + LRUHandle* Insert(LRUHandle* h) { + LRUHandle** ptr = FindPointer(h->key(), h->hash); + LRUHandle* old = *ptr; + h->next_hash = (old == NULL ? NULL : old->next_hash); + *ptr = h; + if (old == NULL) { + ++elems_; + if (elems_ > length_) { + // Since each cache entry is fairly large, we aim for a small + // average linked list length (<= 1). + Resize(); + } + } + return old; + } + + LRUHandle* Remove(const Slice& key, uint32_t hash) { + LRUHandle** ptr = FindPointer(key, hash); + LRUHandle* result = *ptr; + if (result != NULL) { + *ptr = result->next_hash; + --elems_; + } + return result; + } + + private: + // The table consists of an array of buckets where each bucket is + // a linked list of cache entries that hash into the bucket. + uint32_t length_; + uint32_t elems_; + LRUHandle** list_; + + // Return a pointer to slot that points to a cache entry that + // matches key/hash. If there is no such cache entry, return a + // pointer to the trailing slot in the corresponding linked list. + LRUHandle** FindPointer(const Slice& key, uint32_t hash) { + LRUHandle** ptr = &list_[hash & (length_ - 1)]; + while (*ptr != NULL && + ((*ptr)->hash != hash || key != (*ptr)->key())) { + ptr = &(*ptr)->next_hash; + } + return ptr; + } + + void Resize() { + uint32_t new_length = 4; + while (new_length < elems_) { + new_length *= 2; + } + LRUHandle** new_list = new LRUHandle*[new_length]; + memset(new_list, 0, sizeof(new_list[0]) * new_length); + uint32_t count = 0; + for (uint32_t i = 0; i < length_; i++) { + LRUHandle* h = list_[i]; + while (h != NULL) { + LRUHandle* next = h->next_hash; + uint32_t hash = h->hash; + LRUHandle** ptr = &new_list[hash & (new_length - 1)]; + h->next_hash = *ptr; + *ptr = h; + h = next; + count++; + } + } + assert(elems_ == count); + delete[] list_; + list_ = new_list; + length_ = new_length; + } +}; + +// A single shard of sharded cache. +class LRUCache { + public: + LRUCache(); + ~LRUCache(); + + // Separate from constructor so caller can easily make an array of LRUCache + void SetCapacity(size_t capacity) { capacity_ = capacity; } + + // Like Cache methods, but with an extra "hash" parameter. + Cache::Handle* Insert(const Slice& key, uint32_t hash, + void* value, size_t charge, + void (*deleter)(const Slice& key, void* value)); + Cache::Handle* Lookup(const Slice& key, uint32_t hash); + void Release(Cache::Handle* handle); + void Erase(const Slice& key, uint32_t hash); + + private: + void LRU_Remove(LRUHandle* e); + void LRU_Append(LRUHandle* e); + void Unref(LRUHandle* e); + + // Initialized before use. + size_t capacity_; + + // mutex_ protects the following state. + port::Mutex mutex_; + size_t usage_; + + // Dummy head of LRU list. + // lru.prev is newest entry, lru.next is oldest entry. + LRUHandle lru_; + + HandleTable table_; +}; + +LRUCache::LRUCache() + : usage_(0) { + // Make empty circular linked list + lru_.next = &lru_; + lru_.prev = &lru_; +} + +LRUCache::~LRUCache() { + for (LRUHandle* e = lru_.next; e != &lru_; ) { + LRUHandle* next = e->next; + assert(e->refs == 1); // Error if caller has an unreleased handle + Unref(e); + e = next; + } +} + +void LRUCache::Unref(LRUHandle* e) { + assert(e->refs > 0); + e->refs--; + if (e->refs <= 0) { + usage_ -= e->charge; + (*e->deleter)(e->key(), e->value); + free(e); + } +} + +void LRUCache::LRU_Remove(LRUHandle* e) { + e->next->prev = e->prev; + e->prev->next = e->next; +} + +void LRUCache::LRU_Append(LRUHandle* e) { + // Make "e" newest entry by inserting just before lru_ + e->next = &lru_; + e->prev = lru_.prev; + e->prev->next = e; + e->next->prev = e; +} + +Cache::Handle* LRUCache::Lookup(const Slice& key, uint32_t hash) { + MutexLock l(&mutex_); + LRUHandle* e = table_.Lookup(key, hash); + if (e != NULL) { + e->refs++; + LRU_Remove(e); + LRU_Append(e); + } + return reinterpret_cast(e); +} + +void LRUCache::Release(Cache::Handle* handle) { + MutexLock l(&mutex_); + Unref(reinterpret_cast(handle)); +} + +Cache::Handle* LRUCache::Insert( + const Slice& key, uint32_t hash, void* value, size_t charge, + void (*deleter)(const Slice& key, void* value)) { + MutexLock l(&mutex_); + + LRUHandle* e = reinterpret_cast( + malloc(sizeof(LRUHandle)-1 + key.size())); + e->value = value; + e->deleter = deleter; + e->charge = charge; + e->key_length = key.size(); + e->hash = hash; + e->refs = 2; // One from LRUCache, one for the returned handle + memcpy(e->key_data, key.data(), key.size()); + LRU_Append(e); + usage_ += charge; + + LRUHandle* old = table_.Insert(e); + if (old != NULL) { + LRU_Remove(old); + Unref(old); + } + + while (usage_ > capacity_ && lru_.next != &lru_) { + LRUHandle* old = lru_.next; + LRU_Remove(old); + table_.Remove(old->key(), old->hash); + Unref(old); + } + + return reinterpret_cast(e); +} + +void LRUCache::Erase(const Slice& key, uint32_t hash) { + MutexLock l(&mutex_); + LRUHandle* e = table_.Remove(key, hash); + if (e != NULL) { + LRU_Remove(e); + Unref(e); + } +} + +static const int kNumShardBits = 4; +static const int kNumShards = 1 << kNumShardBits; + +class ShardedLRUCache : public Cache { + private: + LRUCache shard_[kNumShards]; + port::Mutex id_mutex_; + uint64_t last_id_; + + static inline uint32_t HashSlice(const Slice& s) { + return Hash(s.data(), s.size(), 0); + } + + static uint32_t Shard(uint32_t hash) { + return hash >> (32 - kNumShardBits); + } + + public: + explicit ShardedLRUCache(size_t capacity) + : last_id_(0) { + const size_t per_shard = (capacity + (kNumShards - 1)) / kNumShards; + for (int s = 0; s < kNumShards; s++) { + shard_[s].SetCapacity(per_shard); + } + } + virtual ~ShardedLRUCache() { } + virtual Handle* Insert(const Slice& key, void* value, size_t charge, + void (*deleter)(const Slice& key, void* value)) { + const uint32_t hash = HashSlice(key); + return shard_[Shard(hash)].Insert(key, hash, value, charge, deleter); + } + virtual Handle* Lookup(const Slice& key) { + const uint32_t hash = HashSlice(key); + return shard_[Shard(hash)].Lookup(key, hash); + } + virtual void Release(Handle* handle) { + LRUHandle* h = reinterpret_cast(handle); + shard_[Shard(h->hash)].Release(handle); + } + virtual void Erase(const Slice& key) { + const uint32_t hash = HashSlice(key); + shard_[Shard(hash)].Erase(key, hash); + } + virtual void* Value(Handle* handle) { + return reinterpret_cast(handle)->value; + } + virtual uint64_t NewId() { + MutexLock l(&id_mutex_); + return ++(last_id_); + } +}; + +} // end anonymous namespace + +Cache* NewLRUCache(size_t capacity) { + return new ShardedLRUCache(capacity); +} + +} // namespace leveldb diff --git a/src/leveldb/util/cache_test.cc b/src/leveldb/util/cache_test.cc new file mode 100644 index 00000000..43716715 --- /dev/null +++ b/src/leveldb/util/cache_test.cc @@ -0,0 +1,186 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/cache.h" + +#include +#include "util/coding.h" +#include "util/testharness.h" + +namespace leveldb { + +// Conversions between numeric keys/values and the types expected by Cache. +static std::string EncodeKey(int k) { + std::string result; + PutFixed32(&result, k); + return result; +} +static int DecodeKey(const Slice& k) { + assert(k.size() == 4); + return DecodeFixed32(k.data()); +} +static void* EncodeValue(uintptr_t v) { return reinterpret_cast(v); } +static int DecodeValue(void* v) { return reinterpret_cast(v); } + +class CacheTest { + public: + static CacheTest* current_; + + static void Deleter(const Slice& key, void* v) { + current_->deleted_keys_.push_back(DecodeKey(key)); + current_->deleted_values_.push_back(DecodeValue(v)); + } + + static const int kCacheSize = 1000; + std::vector deleted_keys_; + std::vector deleted_values_; + Cache* cache_; + + CacheTest() : cache_(NewLRUCache(kCacheSize)) { + current_ = this; + } + + ~CacheTest() { + delete cache_; + } + + int Lookup(int key) { + Cache::Handle* handle = cache_->Lookup(EncodeKey(key)); + const int r = (handle == NULL) ? -1 : DecodeValue(cache_->Value(handle)); + if (handle != NULL) { + cache_->Release(handle); + } + return r; + } + + void Insert(int key, int value, int charge = 1) { + cache_->Release(cache_->Insert(EncodeKey(key), EncodeValue(value), charge, + &CacheTest::Deleter)); + } + + void Erase(int key) { + cache_->Erase(EncodeKey(key)); + } +}; +CacheTest* CacheTest::current_; + +TEST(CacheTest, HitAndMiss) { + ASSERT_EQ(-1, Lookup(100)); + + Insert(100, 101); + ASSERT_EQ(101, Lookup(100)); + ASSERT_EQ(-1, Lookup(200)); + ASSERT_EQ(-1, Lookup(300)); + + Insert(200, 201); + ASSERT_EQ(101, Lookup(100)); + ASSERT_EQ(201, Lookup(200)); + ASSERT_EQ(-1, Lookup(300)); + + Insert(100, 102); + ASSERT_EQ(102, Lookup(100)); + ASSERT_EQ(201, Lookup(200)); + ASSERT_EQ(-1, Lookup(300)); + + ASSERT_EQ(1, deleted_keys_.size()); + ASSERT_EQ(100, deleted_keys_[0]); + ASSERT_EQ(101, deleted_values_[0]); +} + +TEST(CacheTest, Erase) { + Erase(200); + ASSERT_EQ(0, deleted_keys_.size()); + + Insert(100, 101); + Insert(200, 201); + Erase(100); + ASSERT_EQ(-1, Lookup(100)); + ASSERT_EQ(201, Lookup(200)); + ASSERT_EQ(1, deleted_keys_.size()); + ASSERT_EQ(100, deleted_keys_[0]); + ASSERT_EQ(101, deleted_values_[0]); + + Erase(100); + ASSERT_EQ(-1, Lookup(100)); + ASSERT_EQ(201, Lookup(200)); + ASSERT_EQ(1, deleted_keys_.size()); +} + +TEST(CacheTest, EntriesArePinned) { + Insert(100, 101); + Cache::Handle* h1 = cache_->Lookup(EncodeKey(100)); + ASSERT_EQ(101, DecodeValue(cache_->Value(h1))); + + Insert(100, 102); + Cache::Handle* h2 = cache_->Lookup(EncodeKey(100)); + ASSERT_EQ(102, DecodeValue(cache_->Value(h2))); + ASSERT_EQ(0, deleted_keys_.size()); + + cache_->Release(h1); + ASSERT_EQ(1, deleted_keys_.size()); + ASSERT_EQ(100, deleted_keys_[0]); + ASSERT_EQ(101, deleted_values_[0]); + + Erase(100); + ASSERT_EQ(-1, Lookup(100)); + ASSERT_EQ(1, deleted_keys_.size()); + + cache_->Release(h2); + ASSERT_EQ(2, deleted_keys_.size()); + ASSERT_EQ(100, deleted_keys_[1]); + ASSERT_EQ(102, deleted_values_[1]); +} + +TEST(CacheTest, EvictionPolicy) { + Insert(100, 101); + Insert(200, 201); + + // Frequently used entry must be kept around + for (int i = 0; i < kCacheSize + 100; i++) { + Insert(1000+i, 2000+i); + ASSERT_EQ(2000+i, Lookup(1000+i)); + ASSERT_EQ(101, Lookup(100)); + } + ASSERT_EQ(101, Lookup(100)); + ASSERT_EQ(-1, Lookup(200)); +} + +TEST(CacheTest, HeavyEntries) { + // Add a bunch of light and heavy entries and then count the combined + // size of items still in the cache, which must be approximately the + // same as the total capacity. + const int kLight = 1; + const int kHeavy = 10; + int added = 0; + int index = 0; + while (added < 2*kCacheSize) { + const int weight = (index & 1) ? kLight : kHeavy; + Insert(index, 1000+index, weight); + added += weight; + index++; + } + + int cached_weight = 0; + for (int i = 0; i < index; i++) { + const int weight = (i & 1 ? kLight : kHeavy); + int r = Lookup(i); + if (r >= 0) { + cached_weight += weight; + ASSERT_EQ(1000+i, r); + } + } + ASSERT_LE(cached_weight, kCacheSize + kCacheSize/10); +} + +TEST(CacheTest, NewId) { + uint64_t a = cache_->NewId(); + uint64_t b = cache_->NewId(); + ASSERT_NE(a, b); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/util/coding.cc b/src/leveldb/util/coding.cc new file mode 100644 index 00000000..21e3186d --- /dev/null +++ b/src/leveldb/util/coding.cc @@ -0,0 +1,194 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "util/coding.h" + +namespace leveldb { + +void EncodeFixed32(char* buf, uint32_t value) { + if (port::kLittleEndian) { + memcpy(buf, &value, sizeof(value)); + } else { + buf[0] = value & 0xff; + buf[1] = (value >> 8) & 0xff; + buf[2] = (value >> 16) & 0xff; + buf[3] = (value >> 24) & 0xff; + } +} + +void EncodeFixed64(char* buf, uint64_t value) { + if (port::kLittleEndian) { + memcpy(buf, &value, sizeof(value)); + } else { + buf[0] = value & 0xff; + buf[1] = (value >> 8) & 0xff; + buf[2] = (value >> 16) & 0xff; + buf[3] = (value >> 24) & 0xff; + buf[4] = (value >> 32) & 0xff; + buf[5] = (value >> 40) & 0xff; + buf[6] = (value >> 48) & 0xff; + buf[7] = (value >> 56) & 0xff; + } +} + +void PutFixed32(std::string* dst, uint32_t value) { + char buf[sizeof(value)]; + EncodeFixed32(buf, value); + dst->append(buf, sizeof(buf)); +} + +void PutFixed64(std::string* dst, uint64_t value) { + char buf[sizeof(value)]; + EncodeFixed64(buf, value); + dst->append(buf, sizeof(buf)); +} + +char* EncodeVarint32(char* dst, uint32_t v) { + // Operate on characters as unsigneds + unsigned char* ptr = reinterpret_cast(dst); + static const int B = 128; + if (v < (1<<7)) { + *(ptr++) = v; + } else if (v < (1<<14)) { + *(ptr++) = v | B; + *(ptr++) = v>>7; + } else if (v < (1<<21)) { + *(ptr++) = v | B; + *(ptr++) = (v>>7) | B; + *(ptr++) = v>>14; + } else if (v < (1<<28)) { + *(ptr++) = v | B; + *(ptr++) = (v>>7) | B; + *(ptr++) = (v>>14) | B; + *(ptr++) = v>>21; + } else { + *(ptr++) = v | B; + *(ptr++) = (v>>7) | B; + *(ptr++) = (v>>14) | B; + *(ptr++) = (v>>21) | B; + *(ptr++) = v>>28; + } + return reinterpret_cast(ptr); +} + +void PutVarint32(std::string* dst, uint32_t v) { + char buf[5]; + char* ptr = EncodeVarint32(buf, v); + dst->append(buf, ptr - buf); +} + +char* EncodeVarint64(char* dst, uint64_t v) { + static const int B = 128; + unsigned char* ptr = reinterpret_cast(dst); + while (v >= B) { + *(ptr++) = (v & (B-1)) | B; + v >>= 7; + } + *(ptr++) = static_cast(v); + return reinterpret_cast(ptr); +} + +void PutVarint64(std::string* dst, uint64_t v) { + char buf[10]; + char* ptr = EncodeVarint64(buf, v); + dst->append(buf, ptr - buf); +} + +void PutLengthPrefixedSlice(std::string* dst, const Slice& value) { + PutVarint32(dst, value.size()); + dst->append(value.data(), value.size()); +} + +int VarintLength(uint64_t v) { + int len = 1; + while (v >= 128) { + v >>= 7; + len++; + } + return len; +} + +const char* GetVarint32PtrFallback(const char* p, + const char* limit, + uint32_t* value) { + uint32_t result = 0; + for (uint32_t shift = 0; shift <= 28 && p < limit; shift += 7) { + uint32_t byte = *(reinterpret_cast(p)); + p++; + if (byte & 128) { + // More bytes are present + result |= ((byte & 127) << shift); + } else { + result |= (byte << shift); + *value = result; + return reinterpret_cast(p); + } + } + return NULL; +} + +bool GetVarint32(Slice* input, uint32_t* value) { + const char* p = input->data(); + const char* limit = p + input->size(); + const char* q = GetVarint32Ptr(p, limit, value); + if (q == NULL) { + return false; + } else { + *input = Slice(q, limit - q); + return true; + } +} + +const char* GetVarint64Ptr(const char* p, const char* limit, uint64_t* value) { + uint64_t result = 0; + for (uint32_t shift = 0; shift <= 63 && p < limit; shift += 7) { + uint64_t byte = *(reinterpret_cast(p)); + p++; + if (byte & 128) { + // More bytes are present + result |= ((byte & 127) << shift); + } else { + result |= (byte << shift); + *value = result; + return reinterpret_cast(p); + } + } + return NULL; +} + +bool GetVarint64(Slice* input, uint64_t* value) { + const char* p = input->data(); + const char* limit = p + input->size(); + const char* q = GetVarint64Ptr(p, limit, value); + if (q == NULL) { + return false; + } else { + *input = Slice(q, limit - q); + return true; + } +} + +const char* GetLengthPrefixedSlice(const char* p, const char* limit, + Slice* result) { + uint32_t len; + p = GetVarint32Ptr(p, limit, &len); + if (p == NULL) return NULL; + if (p + len > limit) return NULL; + *result = Slice(p, len); + return p + len; +} + +bool GetLengthPrefixedSlice(Slice* input, Slice* result) { + uint32_t len; + if (GetVarint32(input, &len) && + input->size() >= len) { + *result = Slice(input->data(), len); + input->remove_prefix(len); + return true; + } else { + return false; + } +} + +} // namespace leveldb diff --git a/src/leveldb/util/coding.h b/src/leveldb/util/coding.h new file mode 100644 index 00000000..3993c4a7 --- /dev/null +++ b/src/leveldb/util/coding.h @@ -0,0 +1,104 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// Endian-neutral encoding: +// * Fixed-length numbers are encoded with least-significant byte first +// * In addition we support variable length "varint" encoding +// * Strings are encoded prefixed by their length in varint format + +#ifndef STORAGE_LEVELDB_UTIL_CODING_H_ +#define STORAGE_LEVELDB_UTIL_CODING_H_ + +#include +#include +#include +#include "leveldb/slice.h" +#include "port/port.h" + +namespace leveldb { + +// Standard Put... routines append to a string +extern void PutFixed32(std::string* dst, uint32_t value); +extern void PutFixed64(std::string* dst, uint64_t value); +extern void PutVarint32(std::string* dst, uint32_t value); +extern void PutVarint64(std::string* dst, uint64_t value); +extern void PutLengthPrefixedSlice(std::string* dst, const Slice& value); + +// Standard Get... routines parse a value from the beginning of a Slice +// and advance the slice past the parsed value. +extern bool GetVarint32(Slice* input, uint32_t* value); +extern bool GetVarint64(Slice* input, uint64_t* value); +extern bool GetLengthPrefixedSlice(Slice* input, Slice* result); + +// Pointer-based variants of GetVarint... These either store a value +// in *v and return a pointer just past the parsed value, or return +// NULL on error. These routines only look at bytes in the range +// [p..limit-1] +extern const char* GetVarint32Ptr(const char* p,const char* limit, uint32_t* v); +extern const char* GetVarint64Ptr(const char* p,const char* limit, uint64_t* v); + +// Returns the length of the varint32 or varint64 encoding of "v" +extern int VarintLength(uint64_t v); + +// Lower-level versions of Put... that write directly into a character buffer +// REQUIRES: dst has enough space for the value being written +extern void EncodeFixed32(char* dst, uint32_t value); +extern void EncodeFixed64(char* dst, uint64_t value); + +// Lower-level versions of Put... that write directly into a character buffer +// and return a pointer just past the last byte written. +// REQUIRES: dst has enough space for the value being written +extern char* EncodeVarint32(char* dst, uint32_t value); +extern char* EncodeVarint64(char* dst, uint64_t value); + +// Lower-level versions of Get... that read directly from a character buffer +// without any bounds checking. + +inline uint32_t DecodeFixed32(const char* ptr) { + if (port::kLittleEndian) { + // Load the raw bytes + uint32_t result; + memcpy(&result, ptr, sizeof(result)); // gcc optimizes this to a plain load + return result; + } else { + return ((static_cast(static_cast(ptr[0]))) + | (static_cast(static_cast(ptr[1])) << 8) + | (static_cast(static_cast(ptr[2])) << 16) + | (static_cast(static_cast(ptr[3])) << 24)); + } +} + +inline uint64_t DecodeFixed64(const char* ptr) { + if (port::kLittleEndian) { + // Load the raw bytes + uint64_t result; + memcpy(&result, ptr, sizeof(result)); // gcc optimizes this to a plain load + return result; + } else { + uint64_t lo = DecodeFixed32(ptr); + uint64_t hi = DecodeFixed32(ptr + 4); + return (hi << 32) | lo; + } +} + +// Internal routine for use by fallback path of GetVarint32Ptr +extern const char* GetVarint32PtrFallback(const char* p, + const char* limit, + uint32_t* value); +inline const char* GetVarint32Ptr(const char* p, + const char* limit, + uint32_t* value) { + if (p < limit) { + uint32_t result = *(reinterpret_cast(p)); + if ((result & 128) == 0) { + *value = result; + return p + 1; + } + } + return GetVarint32PtrFallback(p, limit, value); +} + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_UTIL_CODING_H_ diff --git a/src/leveldb/util/coding_test.cc b/src/leveldb/util/coding_test.cc new file mode 100644 index 00000000..521541ea --- /dev/null +++ b/src/leveldb/util/coding_test.cc @@ -0,0 +1,196 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "util/coding.h" + +#include "util/testharness.h" + +namespace leveldb { + +class Coding { }; + +TEST(Coding, Fixed32) { + std::string s; + for (uint32_t v = 0; v < 100000; v++) { + PutFixed32(&s, v); + } + + const char* p = s.data(); + for (uint32_t v = 0; v < 100000; v++) { + uint32_t actual = DecodeFixed32(p); + ASSERT_EQ(v, actual); + p += sizeof(uint32_t); + } +} + +TEST(Coding, Fixed64) { + std::string s; + for (int power = 0; power <= 63; power++) { + uint64_t v = static_cast(1) << power; + PutFixed64(&s, v - 1); + PutFixed64(&s, v + 0); + PutFixed64(&s, v + 1); + } + + const char* p = s.data(); + for (int power = 0; power <= 63; power++) { + uint64_t v = static_cast(1) << power; + uint64_t actual; + actual = DecodeFixed64(p); + ASSERT_EQ(v-1, actual); + p += sizeof(uint64_t); + + actual = DecodeFixed64(p); + ASSERT_EQ(v+0, actual); + p += sizeof(uint64_t); + + actual = DecodeFixed64(p); + ASSERT_EQ(v+1, actual); + p += sizeof(uint64_t); + } +} + +// Test that encoding routines generate little-endian encodings +TEST(Coding, EncodingOutput) { + std::string dst; + PutFixed32(&dst, 0x04030201); + ASSERT_EQ(4, dst.size()); + ASSERT_EQ(0x01, static_cast(dst[0])); + ASSERT_EQ(0x02, static_cast(dst[1])); + ASSERT_EQ(0x03, static_cast(dst[2])); + ASSERT_EQ(0x04, static_cast(dst[3])); + + dst.clear(); + PutFixed64(&dst, 0x0807060504030201ull); + ASSERT_EQ(8, dst.size()); + ASSERT_EQ(0x01, static_cast(dst[0])); + ASSERT_EQ(0x02, static_cast(dst[1])); + ASSERT_EQ(0x03, static_cast(dst[2])); + ASSERT_EQ(0x04, static_cast(dst[3])); + ASSERT_EQ(0x05, static_cast(dst[4])); + ASSERT_EQ(0x06, static_cast(dst[5])); + ASSERT_EQ(0x07, static_cast(dst[6])); + ASSERT_EQ(0x08, static_cast(dst[7])); +} + +TEST(Coding, Varint32) { + std::string s; + for (uint32_t i = 0; i < (32 * 32); i++) { + uint32_t v = (i / 32) << (i % 32); + PutVarint32(&s, v); + } + + const char* p = s.data(); + const char* limit = p + s.size(); + for (uint32_t i = 0; i < (32 * 32); i++) { + uint32_t expected = (i / 32) << (i % 32); + uint32_t actual; + const char* start = p; + p = GetVarint32Ptr(p, limit, &actual); + ASSERT_TRUE(p != NULL); + ASSERT_EQ(expected, actual); + ASSERT_EQ(VarintLength(actual), p - start); + } + ASSERT_EQ(p, s.data() + s.size()); +} + +TEST(Coding, Varint64) { + // Construct the list of values to check + std::vector values; + // Some special values + values.push_back(0); + values.push_back(100); + values.push_back(~static_cast(0)); + values.push_back(~static_cast(0) - 1); + for (uint32_t k = 0; k < 64; k++) { + // Test values near powers of two + const uint64_t power = 1ull << k; + values.push_back(power); + values.push_back(power-1); + values.push_back(power+1); + } + + std::string s; + for (size_t i = 0; i < values.size(); i++) { + PutVarint64(&s, values[i]); + } + + const char* p = s.data(); + const char* limit = p + s.size(); + for (size_t i = 0; i < values.size(); i++) { + ASSERT_TRUE(p < limit); + uint64_t actual; + const char* start = p; + p = GetVarint64Ptr(p, limit, &actual); + ASSERT_TRUE(p != NULL); + ASSERT_EQ(values[i], actual); + ASSERT_EQ(VarintLength(actual), p - start); + } + ASSERT_EQ(p, limit); + +} + +TEST(Coding, Varint32Overflow) { + uint32_t result; + std::string input("\x81\x82\x83\x84\x85\x11"); + ASSERT_TRUE(GetVarint32Ptr(input.data(), input.data() + input.size(), &result) + == NULL); +} + +TEST(Coding, Varint32Truncation) { + uint32_t large_value = (1u << 31) + 100; + std::string s; + PutVarint32(&s, large_value); + uint32_t result; + for (size_t len = 0; len < s.size() - 1; len++) { + ASSERT_TRUE(GetVarint32Ptr(s.data(), s.data() + len, &result) == NULL); + } + ASSERT_TRUE(GetVarint32Ptr(s.data(), s.data() + s.size(), &result) != NULL); + ASSERT_EQ(large_value, result); +} + +TEST(Coding, Varint64Overflow) { + uint64_t result; + std::string input("\x81\x82\x83\x84\x85\x81\x82\x83\x84\x85\x11"); + ASSERT_TRUE(GetVarint64Ptr(input.data(), input.data() + input.size(), &result) + == NULL); +} + +TEST(Coding, Varint64Truncation) { + uint64_t large_value = (1ull << 63) + 100ull; + std::string s; + PutVarint64(&s, large_value); + uint64_t result; + for (size_t len = 0; len < s.size() - 1; len++) { + ASSERT_TRUE(GetVarint64Ptr(s.data(), s.data() + len, &result) == NULL); + } + ASSERT_TRUE(GetVarint64Ptr(s.data(), s.data() + s.size(), &result) != NULL); + ASSERT_EQ(large_value, result); +} + +TEST(Coding, Strings) { + std::string s; + PutLengthPrefixedSlice(&s, Slice("")); + PutLengthPrefixedSlice(&s, Slice("foo")); + PutLengthPrefixedSlice(&s, Slice("bar")); + PutLengthPrefixedSlice(&s, Slice(std::string(200, 'x'))); + + Slice input(s); + Slice v; + ASSERT_TRUE(GetLengthPrefixedSlice(&input, &v)); + ASSERT_EQ("", v.ToString()); + ASSERT_TRUE(GetLengthPrefixedSlice(&input, &v)); + ASSERT_EQ("foo", v.ToString()); + ASSERT_TRUE(GetLengthPrefixedSlice(&input, &v)); + ASSERT_EQ("bar", v.ToString()); + ASSERT_TRUE(GetLengthPrefixedSlice(&input, &v)); + ASSERT_EQ(std::string(200, 'x'), v.ToString()); + ASSERT_EQ("", input.ToString()); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/util/comparator.cc b/src/leveldb/util/comparator.cc new file mode 100644 index 00000000..4b7b5724 --- /dev/null +++ b/src/leveldb/util/comparator.cc @@ -0,0 +1,81 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include +#include "leveldb/comparator.h" +#include "leveldb/slice.h" +#include "port/port.h" +#include "util/logging.h" + +namespace leveldb { + +Comparator::~Comparator() { } + +namespace { +class BytewiseComparatorImpl : public Comparator { + public: + BytewiseComparatorImpl() { } + + virtual const char* Name() const { + return "leveldb.BytewiseComparator"; + } + + virtual int Compare(const Slice& a, const Slice& b) const { + return a.compare(b); + } + + virtual void FindShortestSeparator( + std::string* start, + const Slice& limit) const { + // Find length of common prefix + size_t min_length = std::min(start->size(), limit.size()); + size_t diff_index = 0; + while ((diff_index < min_length) && + ((*start)[diff_index] == limit[diff_index])) { + diff_index++; + } + + if (diff_index >= min_length) { + // Do not shorten if one string is a prefix of the other + } else { + uint8_t diff_byte = static_cast((*start)[diff_index]); + if (diff_byte < static_cast(0xff) && + diff_byte + 1 < static_cast(limit[diff_index])) { + (*start)[diff_index]++; + start->resize(diff_index + 1); + assert(Compare(*start, limit) < 0); + } + } + } + + virtual void FindShortSuccessor(std::string* key) const { + // Find first character that can be incremented + size_t n = key->size(); + for (size_t i = 0; i < n; i++) { + const uint8_t byte = (*key)[i]; + if (byte != static_cast(0xff)) { + (*key)[i] = byte + 1; + key->resize(i+1); + return; + } + } + // *key is a run of 0xffs. Leave it alone. + } +}; +} // namespace + +static port::OnceType once = LEVELDB_ONCE_INIT; +static const Comparator* bytewise; + +static void InitModule() { + bytewise = new BytewiseComparatorImpl; +} + +const Comparator* BytewiseComparator() { + port::InitOnce(&once, InitModule); + return bytewise; +} + +} // namespace leveldb diff --git a/src/leveldb/util/crc32c.cc b/src/leveldb/util/crc32c.cc new file mode 100644 index 00000000..6db9e770 --- /dev/null +++ b/src/leveldb/util/crc32c.cc @@ -0,0 +1,332 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// A portable implementation of crc32c, optimized to handle +// four bytes at a time. + +#include "util/crc32c.h" + +#include +#include "util/coding.h" + +namespace leveldb { +namespace crc32c { + +static const uint32_t table0_[256] = { + 0x00000000, 0xf26b8303, 0xe13b70f7, 0x1350f3f4, + 0xc79a971f, 0x35f1141c, 0x26a1e7e8, 0xd4ca64eb, + 0x8ad958cf, 0x78b2dbcc, 0x6be22838, 0x9989ab3b, + 0x4d43cfd0, 0xbf284cd3, 0xac78bf27, 0x5e133c24, + 0x105ec76f, 0xe235446c, 0xf165b798, 0x030e349b, + 0xd7c45070, 0x25afd373, 0x36ff2087, 0xc494a384, + 0x9a879fa0, 0x68ec1ca3, 0x7bbcef57, 0x89d76c54, + 0x5d1d08bf, 0xaf768bbc, 0xbc267848, 0x4e4dfb4b, + 0x20bd8ede, 0xd2d60ddd, 0xc186fe29, 0x33ed7d2a, + 0xe72719c1, 0x154c9ac2, 0x061c6936, 0xf477ea35, + 0xaa64d611, 0x580f5512, 0x4b5fa6e6, 0xb93425e5, + 0x6dfe410e, 0x9f95c20d, 0x8cc531f9, 0x7eaeb2fa, + 0x30e349b1, 0xc288cab2, 0xd1d83946, 0x23b3ba45, + 0xf779deae, 0x05125dad, 0x1642ae59, 0xe4292d5a, + 0xba3a117e, 0x4851927d, 0x5b016189, 0xa96ae28a, + 0x7da08661, 0x8fcb0562, 0x9c9bf696, 0x6ef07595, + 0x417b1dbc, 0xb3109ebf, 0xa0406d4b, 0x522bee48, + 0x86e18aa3, 0x748a09a0, 0x67dafa54, 0x95b17957, + 0xcba24573, 0x39c9c670, 0x2a993584, 0xd8f2b687, + 0x0c38d26c, 0xfe53516f, 0xed03a29b, 0x1f682198, + 0x5125dad3, 0xa34e59d0, 0xb01eaa24, 0x42752927, + 0x96bf4dcc, 0x64d4cecf, 0x77843d3b, 0x85efbe38, + 0xdbfc821c, 0x2997011f, 0x3ac7f2eb, 0xc8ac71e8, + 0x1c661503, 0xee0d9600, 0xfd5d65f4, 0x0f36e6f7, + 0x61c69362, 0x93ad1061, 0x80fde395, 0x72966096, + 0xa65c047d, 0x5437877e, 0x4767748a, 0xb50cf789, + 0xeb1fcbad, 0x197448ae, 0x0a24bb5a, 0xf84f3859, + 0x2c855cb2, 0xdeeedfb1, 0xcdbe2c45, 0x3fd5af46, + 0x7198540d, 0x83f3d70e, 0x90a324fa, 0x62c8a7f9, + 0xb602c312, 0x44694011, 0x5739b3e5, 0xa55230e6, + 0xfb410cc2, 0x092a8fc1, 0x1a7a7c35, 0xe811ff36, + 0x3cdb9bdd, 0xceb018de, 0xdde0eb2a, 0x2f8b6829, + 0x82f63b78, 0x709db87b, 0x63cd4b8f, 0x91a6c88c, + 0x456cac67, 0xb7072f64, 0xa457dc90, 0x563c5f93, + 0x082f63b7, 0xfa44e0b4, 0xe9141340, 0x1b7f9043, + 0xcfb5f4a8, 0x3dde77ab, 0x2e8e845f, 0xdce5075c, + 0x92a8fc17, 0x60c37f14, 0x73938ce0, 0x81f80fe3, + 0x55326b08, 0xa759e80b, 0xb4091bff, 0x466298fc, + 0x1871a4d8, 0xea1a27db, 0xf94ad42f, 0x0b21572c, + 0xdfeb33c7, 0x2d80b0c4, 0x3ed04330, 0xccbbc033, + 0xa24bb5a6, 0x502036a5, 0x4370c551, 0xb11b4652, + 0x65d122b9, 0x97baa1ba, 0x84ea524e, 0x7681d14d, + 0x2892ed69, 0xdaf96e6a, 0xc9a99d9e, 0x3bc21e9d, + 0xef087a76, 0x1d63f975, 0x0e330a81, 0xfc588982, + 0xb21572c9, 0x407ef1ca, 0x532e023e, 0xa145813d, + 0x758fe5d6, 0x87e466d5, 0x94b49521, 0x66df1622, + 0x38cc2a06, 0xcaa7a905, 0xd9f75af1, 0x2b9cd9f2, + 0xff56bd19, 0x0d3d3e1a, 0x1e6dcdee, 0xec064eed, + 0xc38d26c4, 0x31e6a5c7, 0x22b65633, 0xd0ddd530, + 0x0417b1db, 0xf67c32d8, 0xe52cc12c, 0x1747422f, + 0x49547e0b, 0xbb3ffd08, 0xa86f0efc, 0x5a048dff, + 0x8ecee914, 0x7ca56a17, 0x6ff599e3, 0x9d9e1ae0, + 0xd3d3e1ab, 0x21b862a8, 0x32e8915c, 0xc083125f, + 0x144976b4, 0xe622f5b7, 0xf5720643, 0x07198540, + 0x590ab964, 0xab613a67, 0xb831c993, 0x4a5a4a90, + 0x9e902e7b, 0x6cfbad78, 0x7fab5e8c, 0x8dc0dd8f, + 0xe330a81a, 0x115b2b19, 0x020bd8ed, 0xf0605bee, + 0x24aa3f05, 0xd6c1bc06, 0xc5914ff2, 0x37faccf1, + 0x69e9f0d5, 0x9b8273d6, 0x88d28022, 0x7ab90321, + 0xae7367ca, 0x5c18e4c9, 0x4f48173d, 0xbd23943e, + 0xf36e6f75, 0x0105ec76, 0x12551f82, 0xe03e9c81, + 0x34f4f86a, 0xc69f7b69, 0xd5cf889d, 0x27a40b9e, + 0x79b737ba, 0x8bdcb4b9, 0x988c474d, 0x6ae7c44e, + 0xbe2da0a5, 0x4c4623a6, 0x5f16d052, 0xad7d5351 +}; +static const uint32_t table1_[256] = { + 0x00000000, 0x13a29877, 0x274530ee, 0x34e7a899, + 0x4e8a61dc, 0x5d28f9ab, 0x69cf5132, 0x7a6dc945, + 0x9d14c3b8, 0x8eb65bcf, 0xba51f356, 0xa9f36b21, + 0xd39ea264, 0xc03c3a13, 0xf4db928a, 0xe7790afd, + 0x3fc5f181, 0x2c6769f6, 0x1880c16f, 0x0b225918, + 0x714f905d, 0x62ed082a, 0x560aa0b3, 0x45a838c4, + 0xa2d13239, 0xb173aa4e, 0x859402d7, 0x96369aa0, + 0xec5b53e5, 0xfff9cb92, 0xcb1e630b, 0xd8bcfb7c, + 0x7f8be302, 0x6c297b75, 0x58ced3ec, 0x4b6c4b9b, + 0x310182de, 0x22a31aa9, 0x1644b230, 0x05e62a47, + 0xe29f20ba, 0xf13db8cd, 0xc5da1054, 0xd6788823, + 0xac154166, 0xbfb7d911, 0x8b507188, 0x98f2e9ff, + 0x404e1283, 0x53ec8af4, 0x670b226d, 0x74a9ba1a, + 0x0ec4735f, 0x1d66eb28, 0x298143b1, 0x3a23dbc6, + 0xdd5ad13b, 0xcef8494c, 0xfa1fe1d5, 0xe9bd79a2, + 0x93d0b0e7, 0x80722890, 0xb4958009, 0xa737187e, + 0xff17c604, 0xecb55e73, 0xd852f6ea, 0xcbf06e9d, + 0xb19da7d8, 0xa23f3faf, 0x96d89736, 0x857a0f41, + 0x620305bc, 0x71a19dcb, 0x45463552, 0x56e4ad25, + 0x2c896460, 0x3f2bfc17, 0x0bcc548e, 0x186eccf9, + 0xc0d23785, 0xd370aff2, 0xe797076b, 0xf4359f1c, + 0x8e585659, 0x9dface2e, 0xa91d66b7, 0xbabffec0, + 0x5dc6f43d, 0x4e646c4a, 0x7a83c4d3, 0x69215ca4, + 0x134c95e1, 0x00ee0d96, 0x3409a50f, 0x27ab3d78, + 0x809c2506, 0x933ebd71, 0xa7d915e8, 0xb47b8d9f, + 0xce1644da, 0xddb4dcad, 0xe9537434, 0xfaf1ec43, + 0x1d88e6be, 0x0e2a7ec9, 0x3acdd650, 0x296f4e27, + 0x53028762, 0x40a01f15, 0x7447b78c, 0x67e52ffb, + 0xbf59d487, 0xacfb4cf0, 0x981ce469, 0x8bbe7c1e, + 0xf1d3b55b, 0xe2712d2c, 0xd69685b5, 0xc5341dc2, + 0x224d173f, 0x31ef8f48, 0x050827d1, 0x16aabfa6, + 0x6cc776e3, 0x7f65ee94, 0x4b82460d, 0x5820de7a, + 0xfbc3faf9, 0xe861628e, 0xdc86ca17, 0xcf245260, + 0xb5499b25, 0xa6eb0352, 0x920cabcb, 0x81ae33bc, + 0x66d73941, 0x7575a136, 0x419209af, 0x523091d8, + 0x285d589d, 0x3bffc0ea, 0x0f186873, 0x1cbaf004, + 0xc4060b78, 0xd7a4930f, 0xe3433b96, 0xf0e1a3e1, + 0x8a8c6aa4, 0x992ef2d3, 0xadc95a4a, 0xbe6bc23d, + 0x5912c8c0, 0x4ab050b7, 0x7e57f82e, 0x6df56059, + 0x1798a91c, 0x043a316b, 0x30dd99f2, 0x237f0185, + 0x844819fb, 0x97ea818c, 0xa30d2915, 0xb0afb162, + 0xcac27827, 0xd960e050, 0xed8748c9, 0xfe25d0be, + 0x195cda43, 0x0afe4234, 0x3e19eaad, 0x2dbb72da, + 0x57d6bb9f, 0x447423e8, 0x70938b71, 0x63311306, + 0xbb8de87a, 0xa82f700d, 0x9cc8d894, 0x8f6a40e3, + 0xf50789a6, 0xe6a511d1, 0xd242b948, 0xc1e0213f, + 0x26992bc2, 0x353bb3b5, 0x01dc1b2c, 0x127e835b, + 0x68134a1e, 0x7bb1d269, 0x4f567af0, 0x5cf4e287, + 0x04d43cfd, 0x1776a48a, 0x23910c13, 0x30339464, + 0x4a5e5d21, 0x59fcc556, 0x6d1b6dcf, 0x7eb9f5b8, + 0x99c0ff45, 0x8a626732, 0xbe85cfab, 0xad2757dc, + 0xd74a9e99, 0xc4e806ee, 0xf00fae77, 0xe3ad3600, + 0x3b11cd7c, 0x28b3550b, 0x1c54fd92, 0x0ff665e5, + 0x759baca0, 0x663934d7, 0x52de9c4e, 0x417c0439, + 0xa6050ec4, 0xb5a796b3, 0x81403e2a, 0x92e2a65d, + 0xe88f6f18, 0xfb2df76f, 0xcfca5ff6, 0xdc68c781, + 0x7b5fdfff, 0x68fd4788, 0x5c1aef11, 0x4fb87766, + 0x35d5be23, 0x26772654, 0x12908ecd, 0x013216ba, + 0xe64b1c47, 0xf5e98430, 0xc10e2ca9, 0xd2acb4de, + 0xa8c17d9b, 0xbb63e5ec, 0x8f844d75, 0x9c26d502, + 0x449a2e7e, 0x5738b609, 0x63df1e90, 0x707d86e7, + 0x0a104fa2, 0x19b2d7d5, 0x2d557f4c, 0x3ef7e73b, + 0xd98eedc6, 0xca2c75b1, 0xfecbdd28, 0xed69455f, + 0x97048c1a, 0x84a6146d, 0xb041bcf4, 0xa3e32483 +}; +static const uint32_t table2_[256] = { + 0x00000000, 0xa541927e, 0x4f6f520d, 0xea2ec073, + 0x9edea41a, 0x3b9f3664, 0xd1b1f617, 0x74f06469, + 0x38513ec5, 0x9d10acbb, 0x773e6cc8, 0xd27ffeb6, + 0xa68f9adf, 0x03ce08a1, 0xe9e0c8d2, 0x4ca15aac, + 0x70a27d8a, 0xd5e3eff4, 0x3fcd2f87, 0x9a8cbdf9, + 0xee7cd990, 0x4b3d4bee, 0xa1138b9d, 0x045219e3, + 0x48f3434f, 0xedb2d131, 0x079c1142, 0xa2dd833c, + 0xd62de755, 0x736c752b, 0x9942b558, 0x3c032726, + 0xe144fb14, 0x4405696a, 0xae2ba919, 0x0b6a3b67, + 0x7f9a5f0e, 0xdadbcd70, 0x30f50d03, 0x95b49f7d, + 0xd915c5d1, 0x7c5457af, 0x967a97dc, 0x333b05a2, + 0x47cb61cb, 0xe28af3b5, 0x08a433c6, 0xade5a1b8, + 0x91e6869e, 0x34a714e0, 0xde89d493, 0x7bc846ed, + 0x0f382284, 0xaa79b0fa, 0x40577089, 0xe516e2f7, + 0xa9b7b85b, 0x0cf62a25, 0xe6d8ea56, 0x43997828, + 0x37691c41, 0x92288e3f, 0x78064e4c, 0xdd47dc32, + 0xc76580d9, 0x622412a7, 0x880ad2d4, 0x2d4b40aa, + 0x59bb24c3, 0xfcfab6bd, 0x16d476ce, 0xb395e4b0, + 0xff34be1c, 0x5a752c62, 0xb05bec11, 0x151a7e6f, + 0x61ea1a06, 0xc4ab8878, 0x2e85480b, 0x8bc4da75, + 0xb7c7fd53, 0x12866f2d, 0xf8a8af5e, 0x5de93d20, + 0x29195949, 0x8c58cb37, 0x66760b44, 0xc337993a, + 0x8f96c396, 0x2ad751e8, 0xc0f9919b, 0x65b803e5, + 0x1148678c, 0xb409f5f2, 0x5e273581, 0xfb66a7ff, + 0x26217bcd, 0x8360e9b3, 0x694e29c0, 0xcc0fbbbe, + 0xb8ffdfd7, 0x1dbe4da9, 0xf7908dda, 0x52d11fa4, + 0x1e704508, 0xbb31d776, 0x511f1705, 0xf45e857b, + 0x80aee112, 0x25ef736c, 0xcfc1b31f, 0x6a802161, + 0x56830647, 0xf3c29439, 0x19ec544a, 0xbcadc634, + 0xc85da25d, 0x6d1c3023, 0x8732f050, 0x2273622e, + 0x6ed23882, 0xcb93aafc, 0x21bd6a8f, 0x84fcf8f1, + 0xf00c9c98, 0x554d0ee6, 0xbf63ce95, 0x1a225ceb, + 0x8b277743, 0x2e66e53d, 0xc448254e, 0x6109b730, + 0x15f9d359, 0xb0b84127, 0x5a968154, 0xffd7132a, + 0xb3764986, 0x1637dbf8, 0xfc191b8b, 0x595889f5, + 0x2da8ed9c, 0x88e97fe2, 0x62c7bf91, 0xc7862def, + 0xfb850ac9, 0x5ec498b7, 0xb4ea58c4, 0x11abcaba, + 0x655baed3, 0xc01a3cad, 0x2a34fcde, 0x8f756ea0, + 0xc3d4340c, 0x6695a672, 0x8cbb6601, 0x29faf47f, + 0x5d0a9016, 0xf84b0268, 0x1265c21b, 0xb7245065, + 0x6a638c57, 0xcf221e29, 0x250cde5a, 0x804d4c24, + 0xf4bd284d, 0x51fcba33, 0xbbd27a40, 0x1e93e83e, + 0x5232b292, 0xf77320ec, 0x1d5de09f, 0xb81c72e1, + 0xccec1688, 0x69ad84f6, 0x83834485, 0x26c2d6fb, + 0x1ac1f1dd, 0xbf8063a3, 0x55aea3d0, 0xf0ef31ae, + 0x841f55c7, 0x215ec7b9, 0xcb7007ca, 0x6e3195b4, + 0x2290cf18, 0x87d15d66, 0x6dff9d15, 0xc8be0f6b, + 0xbc4e6b02, 0x190ff97c, 0xf321390f, 0x5660ab71, + 0x4c42f79a, 0xe90365e4, 0x032da597, 0xa66c37e9, + 0xd29c5380, 0x77ddc1fe, 0x9df3018d, 0x38b293f3, + 0x7413c95f, 0xd1525b21, 0x3b7c9b52, 0x9e3d092c, + 0xeacd6d45, 0x4f8cff3b, 0xa5a23f48, 0x00e3ad36, + 0x3ce08a10, 0x99a1186e, 0x738fd81d, 0xd6ce4a63, + 0xa23e2e0a, 0x077fbc74, 0xed517c07, 0x4810ee79, + 0x04b1b4d5, 0xa1f026ab, 0x4bdee6d8, 0xee9f74a6, + 0x9a6f10cf, 0x3f2e82b1, 0xd50042c2, 0x7041d0bc, + 0xad060c8e, 0x08479ef0, 0xe2695e83, 0x4728ccfd, + 0x33d8a894, 0x96993aea, 0x7cb7fa99, 0xd9f668e7, + 0x9557324b, 0x3016a035, 0xda386046, 0x7f79f238, + 0x0b899651, 0xaec8042f, 0x44e6c45c, 0xe1a75622, + 0xdda47104, 0x78e5e37a, 0x92cb2309, 0x378ab177, + 0x437ad51e, 0xe63b4760, 0x0c158713, 0xa954156d, + 0xe5f54fc1, 0x40b4ddbf, 0xaa9a1dcc, 0x0fdb8fb2, + 0x7b2bebdb, 0xde6a79a5, 0x3444b9d6, 0x91052ba8 +}; +static const uint32_t table3_[256] = { + 0x00000000, 0xdd45aab8, 0xbf672381, 0x62228939, + 0x7b2231f3, 0xa6679b4b, 0xc4451272, 0x1900b8ca, + 0xf64463e6, 0x2b01c95e, 0x49234067, 0x9466eadf, + 0x8d665215, 0x5023f8ad, 0x32017194, 0xef44db2c, + 0xe964b13d, 0x34211b85, 0x560392bc, 0x8b463804, + 0x924680ce, 0x4f032a76, 0x2d21a34f, 0xf06409f7, + 0x1f20d2db, 0xc2657863, 0xa047f15a, 0x7d025be2, + 0x6402e328, 0xb9474990, 0xdb65c0a9, 0x06206a11, + 0xd725148b, 0x0a60be33, 0x6842370a, 0xb5079db2, + 0xac072578, 0x71428fc0, 0x136006f9, 0xce25ac41, + 0x2161776d, 0xfc24ddd5, 0x9e0654ec, 0x4343fe54, + 0x5a43469e, 0x8706ec26, 0xe524651f, 0x3861cfa7, + 0x3e41a5b6, 0xe3040f0e, 0x81268637, 0x5c632c8f, + 0x45639445, 0x98263efd, 0xfa04b7c4, 0x27411d7c, + 0xc805c650, 0x15406ce8, 0x7762e5d1, 0xaa274f69, + 0xb327f7a3, 0x6e625d1b, 0x0c40d422, 0xd1057e9a, + 0xaba65fe7, 0x76e3f55f, 0x14c17c66, 0xc984d6de, + 0xd0846e14, 0x0dc1c4ac, 0x6fe34d95, 0xb2a6e72d, + 0x5de23c01, 0x80a796b9, 0xe2851f80, 0x3fc0b538, + 0x26c00df2, 0xfb85a74a, 0x99a72e73, 0x44e284cb, + 0x42c2eeda, 0x9f874462, 0xfda5cd5b, 0x20e067e3, + 0x39e0df29, 0xe4a57591, 0x8687fca8, 0x5bc25610, + 0xb4868d3c, 0x69c32784, 0x0be1aebd, 0xd6a40405, + 0xcfa4bccf, 0x12e11677, 0x70c39f4e, 0xad8635f6, + 0x7c834b6c, 0xa1c6e1d4, 0xc3e468ed, 0x1ea1c255, + 0x07a17a9f, 0xdae4d027, 0xb8c6591e, 0x6583f3a6, + 0x8ac7288a, 0x57828232, 0x35a00b0b, 0xe8e5a1b3, + 0xf1e51979, 0x2ca0b3c1, 0x4e823af8, 0x93c79040, + 0x95e7fa51, 0x48a250e9, 0x2a80d9d0, 0xf7c57368, + 0xeec5cba2, 0x3380611a, 0x51a2e823, 0x8ce7429b, + 0x63a399b7, 0xbee6330f, 0xdcc4ba36, 0x0181108e, + 0x1881a844, 0xc5c402fc, 0xa7e68bc5, 0x7aa3217d, + 0x52a0c93f, 0x8fe56387, 0xedc7eabe, 0x30824006, + 0x2982f8cc, 0xf4c75274, 0x96e5db4d, 0x4ba071f5, + 0xa4e4aad9, 0x79a10061, 0x1b838958, 0xc6c623e0, + 0xdfc69b2a, 0x02833192, 0x60a1b8ab, 0xbde41213, + 0xbbc47802, 0x6681d2ba, 0x04a35b83, 0xd9e6f13b, + 0xc0e649f1, 0x1da3e349, 0x7f816a70, 0xa2c4c0c8, + 0x4d801be4, 0x90c5b15c, 0xf2e73865, 0x2fa292dd, + 0x36a22a17, 0xebe780af, 0x89c50996, 0x5480a32e, + 0x8585ddb4, 0x58c0770c, 0x3ae2fe35, 0xe7a7548d, + 0xfea7ec47, 0x23e246ff, 0x41c0cfc6, 0x9c85657e, + 0x73c1be52, 0xae8414ea, 0xcca69dd3, 0x11e3376b, + 0x08e38fa1, 0xd5a62519, 0xb784ac20, 0x6ac10698, + 0x6ce16c89, 0xb1a4c631, 0xd3864f08, 0x0ec3e5b0, + 0x17c35d7a, 0xca86f7c2, 0xa8a47efb, 0x75e1d443, + 0x9aa50f6f, 0x47e0a5d7, 0x25c22cee, 0xf8878656, + 0xe1873e9c, 0x3cc29424, 0x5ee01d1d, 0x83a5b7a5, + 0xf90696d8, 0x24433c60, 0x4661b559, 0x9b241fe1, + 0x8224a72b, 0x5f610d93, 0x3d4384aa, 0xe0062e12, + 0x0f42f53e, 0xd2075f86, 0xb025d6bf, 0x6d607c07, + 0x7460c4cd, 0xa9256e75, 0xcb07e74c, 0x16424df4, + 0x106227e5, 0xcd278d5d, 0xaf050464, 0x7240aedc, + 0x6b401616, 0xb605bcae, 0xd4273597, 0x09629f2f, + 0xe6264403, 0x3b63eebb, 0x59416782, 0x8404cd3a, + 0x9d0475f0, 0x4041df48, 0x22635671, 0xff26fcc9, + 0x2e238253, 0xf36628eb, 0x9144a1d2, 0x4c010b6a, + 0x5501b3a0, 0x88441918, 0xea669021, 0x37233a99, + 0xd867e1b5, 0x05224b0d, 0x6700c234, 0xba45688c, + 0xa345d046, 0x7e007afe, 0x1c22f3c7, 0xc167597f, + 0xc747336e, 0x1a0299d6, 0x782010ef, 0xa565ba57, + 0xbc65029d, 0x6120a825, 0x0302211c, 0xde478ba4, + 0x31035088, 0xec46fa30, 0x8e647309, 0x5321d9b1, + 0x4a21617b, 0x9764cbc3, 0xf54642fa, 0x2803e842 +}; + +// Used to fetch a naturally-aligned 32-bit word in little endian byte-order +static inline uint32_t LE_LOAD32(const uint8_t *p) { + return DecodeFixed32(reinterpret_cast(p)); +} + +uint32_t Extend(uint32_t crc, const char* buf, size_t size) { + const uint8_t *p = reinterpret_cast(buf); + const uint8_t *e = p + size; + uint32_t l = crc ^ 0xffffffffu; + +#define STEP1 do { \ + int c = (l & 0xff) ^ *p++; \ + l = table0_[c] ^ (l >> 8); \ +} while (0) +#define STEP4 do { \ + uint32_t c = l ^ LE_LOAD32(p); \ + p += 4; \ + l = table3_[c & 0xff] ^ \ + table2_[(c >> 8) & 0xff] ^ \ + table1_[(c >> 16) & 0xff] ^ \ + table0_[c >> 24]; \ +} while (0) + + // Point x at first 4-byte aligned byte in string. This might be + // just past the end of the string. + const uintptr_t pval = reinterpret_cast(p); + const uint8_t* x = reinterpret_cast(((pval + 3) >> 2) << 2); + if (x <= e) { + // Process bytes until finished or p is 4-byte aligned + while (p != x) { + STEP1; + } + } + // Process bytes 16 at a time + while ((e-p) >= 16) { + STEP4; STEP4; STEP4; STEP4; + } + // Process bytes 4 at a time + while ((e-p) >= 4) { + STEP4; + } + // Process the last few bytes + while (p != e) { + STEP1; + } +#undef STEP4 +#undef STEP1 + return l ^ 0xffffffffu; +} + +} // namespace crc32c +} // namespace leveldb diff --git a/src/leveldb/util/crc32c.h b/src/leveldb/util/crc32c.h new file mode 100644 index 00000000..1d7e5c07 --- /dev/null +++ b/src/leveldb/util/crc32c.h @@ -0,0 +1,45 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_UTIL_CRC32C_H_ +#define STORAGE_LEVELDB_UTIL_CRC32C_H_ + +#include +#include + +namespace leveldb { +namespace crc32c { + +// Return the crc32c of concat(A, data[0,n-1]) where init_crc is the +// crc32c of some string A. Extend() is often used to maintain the +// crc32c of a stream of data. +extern uint32_t Extend(uint32_t init_crc, const char* data, size_t n); + +// Return the crc32c of data[0,n-1] +inline uint32_t Value(const char* data, size_t n) { + return Extend(0, data, n); +} + +static const uint32_t kMaskDelta = 0xa282ead8ul; + +// Return a masked representation of crc. +// +// Motivation: it is problematic to compute the CRC of a string that +// contains embedded CRCs. Therefore we recommend that CRCs stored +// somewhere (e.g., in files) should be masked before being stored. +inline uint32_t Mask(uint32_t crc) { + // Rotate right by 15 bits and add a constant. + return ((crc >> 15) | (crc << 17)) + kMaskDelta; +} + +// Return the crc whose masked representation is masked_crc. +inline uint32_t Unmask(uint32_t masked_crc) { + uint32_t rot = masked_crc - kMaskDelta; + return ((rot >> 17) | (rot << 15)); +} + +} // namespace crc32c +} // namespace leveldb + +#endif // STORAGE_LEVELDB_UTIL_CRC32C_H_ diff --git a/src/leveldb/util/crc32c_test.cc b/src/leveldb/util/crc32c_test.cc new file mode 100644 index 00000000..4b957ee1 --- /dev/null +++ b/src/leveldb/util/crc32c_test.cc @@ -0,0 +1,72 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "util/crc32c.h" +#include "util/testharness.h" + +namespace leveldb { +namespace crc32c { + +class CRC { }; + +TEST(CRC, StandardResults) { + // From rfc3720 section B.4. + char buf[32]; + + memset(buf, 0, sizeof(buf)); + ASSERT_EQ(0x8a9136aa, Value(buf, sizeof(buf))); + + memset(buf, 0xff, sizeof(buf)); + ASSERT_EQ(0x62a8ab43, Value(buf, sizeof(buf))); + + for (int i = 0; i < 32; i++) { + buf[i] = i; + } + ASSERT_EQ(0x46dd794e, Value(buf, sizeof(buf))); + + for (int i = 0; i < 32; i++) { + buf[i] = 31 - i; + } + ASSERT_EQ(0x113fdb5c, Value(buf, sizeof(buf))); + + unsigned char data[48] = { + 0x01, 0xc0, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x14, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x04, 0x00, + 0x00, 0x00, 0x00, 0x14, + 0x00, 0x00, 0x00, 0x18, + 0x28, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x02, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + }; + ASSERT_EQ(0xd9963a56, Value(reinterpret_cast(data), sizeof(data))); +} + +TEST(CRC, Values) { + ASSERT_NE(Value("a", 1), Value("foo", 3)); +} + +TEST(CRC, Extend) { + ASSERT_EQ(Value("hello world", 11), + Extend(Value("hello ", 6), "world", 5)); +} + +TEST(CRC, Mask) { + uint32_t crc = Value("foo", 3); + ASSERT_NE(crc, Mask(crc)); + ASSERT_NE(crc, Mask(Mask(crc))); + ASSERT_EQ(crc, Unmask(Mask(crc))); + ASSERT_EQ(crc, Unmask(Unmask(Mask(Mask(crc))))); +} + +} // namespace crc32c +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/util/env.cc b/src/leveldb/util/env.cc new file mode 100644 index 00000000..c2600e96 --- /dev/null +++ b/src/leveldb/util/env.cc @@ -0,0 +1,96 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/env.h" + +namespace leveldb { + +Env::~Env() { +} + +SequentialFile::~SequentialFile() { +} + +RandomAccessFile::~RandomAccessFile() { +} + +WritableFile::~WritableFile() { +} + +Logger::~Logger() { +} + +FileLock::~FileLock() { +} + +void Log(Logger* info_log, const char* format, ...) { + if (info_log != NULL) { + va_list ap; + va_start(ap, format); + info_log->Logv(format, ap); + va_end(ap); + } +} + +static Status DoWriteStringToFile(Env* env, const Slice& data, + const std::string& fname, + bool should_sync) { + WritableFile* file; + Status s = env->NewWritableFile(fname, &file); + if (!s.ok()) { + return s; + } + s = file->Append(data); + if (s.ok() && should_sync) { + s = file->Sync(); + } + if (s.ok()) { + s = file->Close(); + } + delete file; // Will auto-close if we did not close above + if (!s.ok()) { + env->DeleteFile(fname); + } + return s; +} + +Status WriteStringToFile(Env* env, const Slice& data, + const std::string& fname) { + return DoWriteStringToFile(env, data, fname, false); +} + +Status WriteStringToFileSync(Env* env, const Slice& data, + const std::string& fname) { + return DoWriteStringToFile(env, data, fname, true); +} + +Status ReadFileToString(Env* env, const std::string& fname, std::string* data) { + data->clear(); + SequentialFile* file; + Status s = env->NewSequentialFile(fname, &file); + if (!s.ok()) { + return s; + } + static const int kBufferSize = 8192; + char* space = new char[kBufferSize]; + while (true) { + Slice fragment; + s = file->Read(kBufferSize, &fragment, space); + if (!s.ok()) { + break; + } + data->append(fragment.data(), fragment.size()); + if (fragment.empty()) { + break; + } + } + delete[] space; + delete file; + return s; +} + +EnvWrapper::~EnvWrapper() { +} + +} // namespace leveldb diff --git a/src/leveldb/util/env_posix.cc b/src/leveldb/util/env_posix.cc new file mode 100644 index 00000000..ba266786 --- /dev/null +++ b/src/leveldb/util/env_posix.cc @@ -0,0 +1,608 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#if !defined(LEVELDB_PLATFORM_WINDOWS) + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "leveldb/env.h" +#include "leveldb/slice.h" +#include "port/port.h" +#include "util/logging.h" +#include "util/mutexlock.h" +#include "util/posix_logger.h" + +namespace leveldb { + +namespace { + +static Status IOError(const std::string& context, int err_number) { + return Status::IOError(context, strerror(err_number)); +} + +class PosixSequentialFile: public SequentialFile { + private: + std::string filename_; + FILE* file_; + + public: + PosixSequentialFile(const std::string& fname, FILE* f) + : filename_(fname), file_(f) { } + virtual ~PosixSequentialFile() { fclose(file_); } + + virtual Status Read(size_t n, Slice* result, char* scratch) { + Status s; + size_t r = fread_unlocked(scratch, 1, n, file_); + *result = Slice(scratch, r); + if (r < n) { + if (feof(file_)) { + // We leave status as ok if we hit the end of the file + } else { + // A partial read with an error: return a non-ok status + s = IOError(filename_, errno); + } + } + return s; + } + + virtual Status Skip(uint64_t n) { + if (fseek(file_, n, SEEK_CUR)) { + return IOError(filename_, errno); + } + return Status::OK(); + } +}; + +// pread() based random-access +class PosixRandomAccessFile: public RandomAccessFile { + private: + std::string filename_; + int fd_; + + public: + PosixRandomAccessFile(const std::string& fname, int fd) + : filename_(fname), fd_(fd) { } + virtual ~PosixRandomAccessFile() { close(fd_); } + + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + Status s; + ssize_t r = pread(fd_, scratch, n, static_cast(offset)); + *result = Slice(scratch, (r < 0) ? 0 : r); + if (r < 0) { + // An error: return a non-ok status + s = IOError(filename_, errno); + } + return s; + } +}; + +// Helper class to limit mmap file usage so that we do not end up +// running out virtual memory or running into kernel performance +// problems for very large databases. +class MmapLimiter { + public: + // Up to 1000 mmaps for 64-bit binaries; none for smaller pointer sizes. + MmapLimiter() { + SetAllowed(sizeof(void*) >= 8 ? 1000 : 0); + } + + // If another mmap slot is available, acquire it and return true. + // Else return false. + bool Acquire() { + if (GetAllowed() <= 0) { + return false; + } + MutexLock l(&mu_); + intptr_t x = GetAllowed(); + if (x <= 0) { + return false; + } else { + SetAllowed(x - 1); + return true; + } + } + + // Release a slot acquired by a previous call to Acquire() that returned true. + void Release() { + MutexLock l(&mu_); + SetAllowed(GetAllowed() + 1); + } + + private: + port::Mutex mu_; + port::AtomicPointer allowed_; + + intptr_t GetAllowed() const { + return reinterpret_cast(allowed_.Acquire_Load()); + } + + // REQUIRES: mu_ must be held + void SetAllowed(intptr_t v) { + allowed_.Release_Store(reinterpret_cast(v)); + } + + MmapLimiter(const MmapLimiter&); + void operator=(const MmapLimiter&); +}; + +// mmap() based random-access +class PosixMmapReadableFile: public RandomAccessFile { + private: + std::string filename_; + void* mmapped_region_; + size_t length_; + MmapLimiter* limiter_; + + public: + // base[0,length-1] contains the mmapped contents of the file. + PosixMmapReadableFile(const std::string& fname, void* base, size_t length, + MmapLimiter* limiter) + : filename_(fname), mmapped_region_(base), length_(length), + limiter_(limiter) { + } + + virtual ~PosixMmapReadableFile() { + munmap(mmapped_region_, length_); + limiter_->Release(); + } + + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + Status s; + if (offset + n > length_) { + *result = Slice(); + s = IOError(filename_, EINVAL); + } else { + *result = Slice(reinterpret_cast(mmapped_region_) + offset, n); + } + return s; + } +}; + +class PosixWritableFile : public WritableFile { + private: + std::string filename_; + FILE* file_; + + public: + PosixWritableFile(const std::string& fname, FILE* f) + : filename_(fname), file_(f) { } + + ~PosixWritableFile() { + if (file_ != NULL) { + // Ignoring any potential errors + fclose(file_); + } + } + + virtual Status Append(const Slice& data) { + size_t r = fwrite_unlocked(data.data(), 1, data.size(), file_); + if (r != data.size()) { + return IOError(filename_, errno); + } + return Status::OK(); + } + + virtual Status Close() { + Status result; + if (fclose(file_) != 0) { + result = IOError(filename_, errno); + } + file_ = NULL; + return result; + } + + virtual Status Flush() { + if (fflush_unlocked(file_) != 0) { + return IOError(filename_, errno); + } + return Status::OK(); + } + + Status SyncDirIfManifest() { + const char* f = filename_.c_str(); + const char* sep = strrchr(f, '/'); + Slice basename; + std::string dir; + if (sep == NULL) { + dir = "."; + basename = f; + } else { + dir = std::string(f, sep - f); + basename = sep + 1; + } + Status s; + if (basename.starts_with("MANIFEST")) { + int fd = open(dir.c_str(), O_RDONLY); + if (fd < 0) { + s = IOError(dir, errno); + } else { + if (fsync(fd) < 0) { + s = IOError(dir, errno); + } + close(fd); + } + } + return s; + } + + virtual Status Sync() { + // Ensure new files referred to by the manifest are in the filesystem. + Status s = SyncDirIfManifest(); + if (!s.ok()) { + return s; + } + if (fflush_unlocked(file_) != 0 || + fdatasync(fileno(file_)) != 0) { + s = Status::IOError(filename_, strerror(errno)); + } + return s; + } +}; + +static int LockOrUnlock(int fd, bool lock) { + errno = 0; + struct flock f; + memset(&f, 0, sizeof(f)); + f.l_type = (lock ? F_WRLCK : F_UNLCK); + f.l_whence = SEEK_SET; + f.l_start = 0; + f.l_len = 0; // Lock/unlock entire file + return fcntl(fd, F_SETLK, &f); +} + +class PosixFileLock : public FileLock { + public: + int fd_; + std::string name_; +}; + +// Set of locked files. We keep a separate set instead of just +// relying on fcntrl(F_SETLK) since fcntl(F_SETLK) does not provide +// any protection against multiple uses from the same process. +class PosixLockTable { + private: + port::Mutex mu_; + std::set locked_files_; + public: + bool Insert(const std::string& fname) { + MutexLock l(&mu_); + return locked_files_.insert(fname).second; + } + void Remove(const std::string& fname) { + MutexLock l(&mu_); + locked_files_.erase(fname); + } +}; + +class PosixEnv : public Env { + public: + PosixEnv(); + virtual ~PosixEnv() { + char msg[] = "Destroying Env::Default()\n"; + fwrite(msg, 1, sizeof(msg), stderr); + abort(); + } + + virtual Status NewSequentialFile(const std::string& fname, + SequentialFile** result) { + FILE* f = fopen(fname.c_str(), "r"); + if (f == NULL) { + *result = NULL; + return IOError(fname, errno); + } else { + *result = new PosixSequentialFile(fname, f); + return Status::OK(); + } + } + + virtual Status NewRandomAccessFile(const std::string& fname, + RandomAccessFile** result) { + *result = NULL; + Status s; + int fd = open(fname.c_str(), O_RDONLY); + if (fd < 0) { + s = IOError(fname, errno); + } else if (mmap_limit_.Acquire()) { + uint64_t size; + s = GetFileSize(fname, &size); + if (s.ok()) { + void* base = mmap(NULL, size, PROT_READ, MAP_SHARED, fd, 0); + if (base != MAP_FAILED) { + *result = new PosixMmapReadableFile(fname, base, size, &mmap_limit_); + } else { + s = IOError(fname, errno); + } + } + close(fd); + if (!s.ok()) { + mmap_limit_.Release(); + } + } else { + *result = new PosixRandomAccessFile(fname, fd); + } + return s; + } + + virtual Status NewWritableFile(const std::string& fname, + WritableFile** result) { + Status s; + FILE* f = fopen(fname.c_str(), "w"); + if (f == NULL) { + *result = NULL; + s = IOError(fname, errno); + } else { + *result = new PosixWritableFile(fname, f); + } + return s; + } + + virtual bool FileExists(const std::string& fname) { + return access(fname.c_str(), F_OK) == 0; + } + + virtual Status GetChildren(const std::string& dir, + std::vector* result) { + result->clear(); + DIR* d = opendir(dir.c_str()); + if (d == NULL) { + return IOError(dir, errno); + } + struct dirent* entry; + while ((entry = readdir(d)) != NULL) { + result->push_back(entry->d_name); + } + closedir(d); + return Status::OK(); + } + + virtual Status DeleteFile(const std::string& fname) { + Status result; + if (unlink(fname.c_str()) != 0) { + result = IOError(fname, errno); + } + return result; + } + + virtual Status CreateDir(const std::string& name) { + Status result; + if (mkdir(name.c_str(), 0755) != 0) { + result = IOError(name, errno); + } + return result; + } + + virtual Status DeleteDir(const std::string& name) { + Status result; + if (rmdir(name.c_str()) != 0) { + result = IOError(name, errno); + } + return result; + } + + virtual Status GetFileSize(const std::string& fname, uint64_t* size) { + Status s; + struct stat sbuf; + if (stat(fname.c_str(), &sbuf) != 0) { + *size = 0; + s = IOError(fname, errno); + } else { + *size = sbuf.st_size; + } + return s; + } + + virtual Status RenameFile(const std::string& src, const std::string& target) { + Status result; + if (rename(src.c_str(), target.c_str()) != 0) { + result = IOError(src, errno); + } + return result; + } + + virtual Status LockFile(const std::string& fname, FileLock** lock) { + *lock = NULL; + Status result; + int fd = open(fname.c_str(), O_RDWR | O_CREAT, 0644); + if (fd < 0) { + result = IOError(fname, errno); + } else if (!locks_.Insert(fname)) { + close(fd); + result = Status::IOError("lock " + fname, "already held by process"); + } else if (LockOrUnlock(fd, true) == -1) { + result = IOError("lock " + fname, errno); + close(fd); + locks_.Remove(fname); + } else { + PosixFileLock* my_lock = new PosixFileLock; + my_lock->fd_ = fd; + my_lock->name_ = fname; + *lock = my_lock; + } + return result; + } + + virtual Status UnlockFile(FileLock* lock) { + PosixFileLock* my_lock = reinterpret_cast(lock); + Status result; + if (LockOrUnlock(my_lock->fd_, false) == -1) { + result = IOError("unlock", errno); + } + locks_.Remove(my_lock->name_); + close(my_lock->fd_); + delete my_lock; + return result; + } + + virtual void Schedule(void (*function)(void*), void* arg); + + virtual void StartThread(void (*function)(void* arg), void* arg); + + virtual Status GetTestDirectory(std::string* result) { + const char* env = getenv("TEST_TMPDIR"); + if (env && env[0] != '\0') { + *result = env; + } else { + char buf[100]; + snprintf(buf, sizeof(buf), "/tmp/leveldbtest-%d", int(geteuid())); + *result = buf; + } + // Directory may already exist + CreateDir(*result); + return Status::OK(); + } + + static uint64_t gettid() { + pthread_t tid = pthread_self(); + uint64_t thread_id = 0; + memcpy(&thread_id, &tid, std::min(sizeof(thread_id), sizeof(tid))); + return thread_id; + } + + virtual Status NewLogger(const std::string& fname, Logger** result) { + FILE* f = fopen(fname.c_str(), "w"); + if (f == NULL) { + *result = NULL; + return IOError(fname, errno); + } else { + *result = new PosixLogger(f, &PosixEnv::gettid); + return Status::OK(); + } + } + + virtual uint64_t NowMicros() { + struct timeval tv; + gettimeofday(&tv, NULL); + return static_cast(tv.tv_sec) * 1000000 + tv.tv_usec; + } + + virtual void SleepForMicroseconds(int micros) { + usleep(micros); + } + + private: + void PthreadCall(const char* label, int result) { + if (result != 0) { + fprintf(stderr, "pthread %s: %s\n", label, strerror(result)); + abort(); + } + } + + // BGThread() is the body of the background thread + void BGThread(); + static void* BGThreadWrapper(void* arg) { + reinterpret_cast(arg)->BGThread(); + return NULL; + } + + pthread_mutex_t mu_; + pthread_cond_t bgsignal_; + pthread_t bgthread_; + bool started_bgthread_; + + // Entry per Schedule() call + struct BGItem { void* arg; void (*function)(void*); }; + typedef std::deque BGQueue; + BGQueue queue_; + + PosixLockTable locks_; + MmapLimiter mmap_limit_; +}; + +PosixEnv::PosixEnv() : started_bgthread_(false) { + PthreadCall("mutex_init", pthread_mutex_init(&mu_, NULL)); + PthreadCall("cvar_init", pthread_cond_init(&bgsignal_, NULL)); +} + +void PosixEnv::Schedule(void (*function)(void*), void* arg) { + PthreadCall("lock", pthread_mutex_lock(&mu_)); + + // Start background thread if necessary + if (!started_bgthread_) { + started_bgthread_ = true; + PthreadCall( + "create thread", + pthread_create(&bgthread_, NULL, &PosixEnv::BGThreadWrapper, this)); + } + + // If the queue is currently empty, the background thread may currently be + // waiting. + if (queue_.empty()) { + PthreadCall("signal", pthread_cond_signal(&bgsignal_)); + } + + // Add to priority queue + queue_.push_back(BGItem()); + queue_.back().function = function; + queue_.back().arg = arg; + + PthreadCall("unlock", pthread_mutex_unlock(&mu_)); +} + +void PosixEnv::BGThread() { + while (true) { + // Wait until there is an item that is ready to run + PthreadCall("lock", pthread_mutex_lock(&mu_)); + while (queue_.empty()) { + PthreadCall("wait", pthread_cond_wait(&bgsignal_, &mu_)); + } + + void (*function)(void*) = queue_.front().function; + void* arg = queue_.front().arg; + queue_.pop_front(); + + PthreadCall("unlock", pthread_mutex_unlock(&mu_)); + (*function)(arg); + } +} + +namespace { +struct StartThreadState { + void (*user_function)(void*); + void* arg; +}; +} +static void* StartThreadWrapper(void* arg) { + StartThreadState* state = reinterpret_cast(arg); + state->user_function(state->arg); + delete state; + return NULL; +} + +void PosixEnv::StartThread(void (*function)(void* arg), void* arg) { + pthread_t t; + StartThreadState* state = new StartThreadState; + state->user_function = function; + state->arg = arg; + PthreadCall("start thread", + pthread_create(&t, NULL, &StartThreadWrapper, state)); +} + +} // namespace + +static pthread_once_t once = PTHREAD_ONCE_INIT; +static Env* default_env; +static void InitDefaultEnv() { default_env = new PosixEnv; } + +Env* Env::Default() { + pthread_once(&once, InitDefaultEnv); + return default_env; +} + +} // namespace leveldb + +#endif diff --git a/src/leveldb/util/env_test.cc b/src/leveldb/util/env_test.cc new file mode 100644 index 00000000..b72cb443 --- /dev/null +++ b/src/leveldb/util/env_test.cc @@ -0,0 +1,104 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/env.h" + +#include "port/port.h" +#include "util/testharness.h" + +namespace leveldb { + +static const int kDelayMicros = 100000; + +class EnvPosixTest { + private: + port::Mutex mu_; + std::string events_; + + public: + Env* env_; + EnvPosixTest() : env_(Env::Default()) { } +}; + +static void SetBool(void* ptr) { + reinterpret_cast(ptr)->NoBarrier_Store(ptr); +} + +TEST(EnvPosixTest, RunImmediately) { + port::AtomicPointer called (NULL); + env_->Schedule(&SetBool, &called); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_TRUE(called.NoBarrier_Load() != NULL); +} + +TEST(EnvPosixTest, RunMany) { + port::AtomicPointer last_id (NULL); + + struct CB { + port::AtomicPointer* last_id_ptr; // Pointer to shared slot + uintptr_t id; // Order# for the execution of this callback + + CB(port::AtomicPointer* p, int i) : last_id_ptr(p), id(i) { } + + static void Run(void* v) { + CB* cb = reinterpret_cast(v); + void* cur = cb->last_id_ptr->NoBarrier_Load(); + ASSERT_EQ(cb->id-1, reinterpret_cast(cur)); + cb->last_id_ptr->Release_Store(reinterpret_cast(cb->id)); + } + }; + + // Schedule in different order than start time + CB cb1(&last_id, 1); + CB cb2(&last_id, 2); + CB cb3(&last_id, 3); + CB cb4(&last_id, 4); + env_->Schedule(&CB::Run, &cb1); + env_->Schedule(&CB::Run, &cb2); + env_->Schedule(&CB::Run, &cb3); + env_->Schedule(&CB::Run, &cb4); + + Env::Default()->SleepForMicroseconds(kDelayMicros); + void* cur = last_id.Acquire_Load(); + ASSERT_EQ(4, reinterpret_cast(cur)); +} + +struct State { + port::Mutex mu; + int val; + int num_running; +}; + +static void ThreadBody(void* arg) { + State* s = reinterpret_cast(arg); + s->mu.Lock(); + s->val += 1; + s->num_running -= 1; + s->mu.Unlock(); +} + +TEST(EnvPosixTest, StartThread) { + State state; + state.val = 0; + state.num_running = 3; + for (int i = 0; i < 3; i++) { + env_->StartThread(&ThreadBody, &state); + } + while (true) { + state.mu.Lock(); + int num = state.num_running; + state.mu.Unlock(); + if (num == 0) { + break; + } + Env::Default()->SleepForMicroseconds(kDelayMicros); + } + ASSERT_EQ(state.val, 3); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/util/env_win.cc b/src/leveldb/util/env_win.cc new file mode 100644 index 00000000..ef2ecae8 --- /dev/null +++ b/src/leveldb/util/env_win.cc @@ -0,0 +1,1031 @@ +// This file contains source that originates from: +// http://code.google.com/p/leveldbwin/source/browse/trunk/win32_impl_src/env_win32.h +// http://code.google.com/p/leveldbwin/source/browse/trunk/win32_impl_src/port_win32.cc +// Those files dont' have any explict license headers but the +// project (http://code.google.com/p/leveldbwin/) lists the 'New BSD License' +// as the license. +#if defined(LEVELDB_PLATFORM_WINDOWS) +#include + + +#include "leveldb/env.h" + +#include "port/port.h" +#include "leveldb/slice.h" +#include "util/logging.h" + +#include +#include +#include +#include +#include +#include +#include + +#ifdef max +#undef max +#endif + +#ifndef va_copy +#define va_copy(d,s) ((d) = (s)) +#endif + +#if defined DeleteFile +#undef DeleteFile +#endif + +//Declarations +namespace leveldb +{ + +namespace Win32 +{ + +#define DISALLOW_COPY_AND_ASSIGN(TypeName) \ + TypeName(const TypeName&); \ + void operator=(const TypeName&) + +std::string GetCurrentDir(); +std::wstring GetCurrentDirW(); + +static const std::string CurrentDir = GetCurrentDir(); +static const std::wstring CurrentDirW = GetCurrentDirW(); + +std::string& ModifyPath(std::string& path); +std::wstring& ModifyPath(std::wstring& path); + +std::string GetLastErrSz(); +std::wstring GetLastErrSzW(); + +size_t GetPageSize(); + +typedef void (*ScheduleProc)(void*) ; + +struct WorkItemWrapper +{ + WorkItemWrapper(ScheduleProc proc_,void* content_); + ScheduleProc proc; + void* pContent; +}; + +DWORD WINAPI WorkItemWrapperProc(LPVOID pContent); + +class Win32SequentialFile : public SequentialFile +{ +public: + friend class Win32Env; + virtual ~Win32SequentialFile(); + virtual Status Read(size_t n, Slice* result, char* scratch); + virtual Status Skip(uint64_t n); + BOOL isEnable(); +private: + BOOL _Init(); + void _CleanUp(); + Win32SequentialFile(const std::string& fname); + std::string _filename; + ::HANDLE _hFile; + DISALLOW_COPY_AND_ASSIGN(Win32SequentialFile); +}; + +class Win32RandomAccessFile : public RandomAccessFile +{ +public: + friend class Win32Env; + virtual ~Win32RandomAccessFile(); + virtual Status Read(uint64_t offset, size_t n, Slice* result,char* scratch) const; + BOOL isEnable(); +private: + BOOL _Init(LPCWSTR path); + void _CleanUp(); + Win32RandomAccessFile(const std::string& fname); + HANDLE _hFile; + const std::string _filename; + DISALLOW_COPY_AND_ASSIGN(Win32RandomAccessFile); +}; + +class Win32MapFile : public WritableFile +{ +public: + Win32MapFile(const std::string& fname); + + ~Win32MapFile(); + virtual Status Append(const Slice& data); + virtual Status Close(); + virtual Status Flush(); + virtual Status Sync(); + BOOL isEnable(); +private: + std::string _filename; + HANDLE _hFile; + size_t _page_size; + size_t _map_size; // How much extra memory to map at a time + char* _base; // The mapped region + HANDLE _base_handle; + char* _limit; // Limit of the mapped region + char* _dst; // Where to write next (in range [base_,limit_]) + char* _last_sync; // Where have we synced up to + uint64_t _file_offset; // Offset of base_ in file + //LARGE_INTEGER file_offset_; + // Have we done an munmap of unsynced data? + bool _pending_sync; + + // Roundup x to a multiple of y + static size_t _Roundup(size_t x, size_t y); + size_t _TruncateToPageBoundary(size_t s); + bool _UnmapCurrentRegion(); + bool _MapNewRegion(); + DISALLOW_COPY_AND_ASSIGN(Win32MapFile); + BOOL _Init(LPCWSTR Path); +}; + +class Win32FileLock : public FileLock +{ +public: + friend class Win32Env; + virtual ~Win32FileLock(); + BOOL isEnable(); +private: + BOOL _Init(LPCWSTR path); + void _CleanUp(); + Win32FileLock(const std::string& fname); + HANDLE _hFile; + std::string _filename; + DISALLOW_COPY_AND_ASSIGN(Win32FileLock); +}; + +class Win32Logger : public Logger +{ +public: + friend class Win32Env; + virtual ~Win32Logger(); + virtual void Logv(const char* format, va_list ap); +private: + explicit Win32Logger(WritableFile* pFile); + WritableFile* _pFileProxy; + DISALLOW_COPY_AND_ASSIGN(Win32Logger); +}; + +class Win32Env : public Env +{ +public: + Win32Env(); + virtual ~Win32Env(); + virtual Status NewSequentialFile(const std::string& fname, + SequentialFile** result); + + virtual Status NewRandomAccessFile(const std::string& fname, + RandomAccessFile** result); + virtual Status NewWritableFile(const std::string& fname, + WritableFile** result); + + virtual bool FileExists(const std::string& fname); + + virtual Status GetChildren(const std::string& dir, + std::vector* result); + + virtual Status DeleteFile(const std::string& fname); + + virtual Status CreateDir(const std::string& dirname); + + virtual Status DeleteDir(const std::string& dirname); + + virtual Status GetFileSize(const std::string& fname, uint64_t* file_size); + + virtual Status RenameFile(const std::string& src, + const std::string& target); + + virtual Status LockFile(const std::string& fname, FileLock** lock); + + virtual Status UnlockFile(FileLock* lock); + + virtual void Schedule( + void (*function)(void* arg), + void* arg); + + virtual void StartThread(void (*function)(void* arg), void* arg); + + virtual Status GetTestDirectory(std::string* path); + + //virtual void Logv(WritableFile* log, const char* format, va_list ap); + + virtual Status NewLogger(const std::string& fname, Logger** result); + + virtual uint64_t NowMicros(); + + virtual void SleepForMicroseconds(int micros); +}; + +void ToWidePath(const std::string& value, std::wstring& target) { + wchar_t buffer[MAX_PATH]; + MultiByteToWideChar(CP_ACP, 0, value.c_str(), -1, buffer, MAX_PATH); + target = buffer; +} + +void ToNarrowPath(const std::wstring& value, std::string& target) { + char buffer[MAX_PATH]; + WideCharToMultiByte(CP_ACP, 0, value.c_str(), -1, buffer, MAX_PATH, NULL, NULL); + target = buffer; +} + +std::string GetCurrentDir() +{ + CHAR path[MAX_PATH]; + ::GetModuleFileNameA(::GetModuleHandleA(NULL),path,MAX_PATH); + *strrchr(path,'\\') = 0; + return std::string(path); +} + +std::wstring GetCurrentDirW() +{ + WCHAR path[MAX_PATH]; + ::GetModuleFileNameW(::GetModuleHandleW(NULL),path,MAX_PATH); + *wcsrchr(path,L'\\') = 0; + return std::wstring(path); +} + +std::string& ModifyPath(std::string& path) +{ + if(path[0] == '/' || path[0] == '\\'){ + path = CurrentDir + path; + } + std::replace(path.begin(),path.end(),'/','\\'); + + return path; +} + +std::wstring& ModifyPath(std::wstring& path) +{ + if(path[0] == L'/' || path[0] == L'\\'){ + path = CurrentDirW + path; + } + std::replace(path.begin(),path.end(),L'/',L'\\'); + return path; +} + +std::string GetLastErrSz() +{ + LPWSTR lpMsgBuf; + FormatMessageW( + FORMAT_MESSAGE_ALLOCATE_BUFFER | + FORMAT_MESSAGE_FROM_SYSTEM | + FORMAT_MESSAGE_IGNORE_INSERTS, + NULL, + GetLastError(), + 0, // Default language + (LPWSTR) &lpMsgBuf, + 0, + NULL + ); + std::string Err; + ToNarrowPath(lpMsgBuf, Err); + LocalFree( lpMsgBuf ); + return Err; +} + +std::wstring GetLastErrSzW() +{ + LPVOID lpMsgBuf; + FormatMessageW( + FORMAT_MESSAGE_ALLOCATE_BUFFER | + FORMAT_MESSAGE_FROM_SYSTEM | + FORMAT_MESSAGE_IGNORE_INSERTS, + NULL, + GetLastError(), + 0, // Default language + (LPWSTR) &lpMsgBuf, + 0, + NULL + ); + std::wstring Err = (LPCWSTR)lpMsgBuf; + LocalFree(lpMsgBuf); + return Err; +} + +WorkItemWrapper::WorkItemWrapper( ScheduleProc proc_,void* content_ ) : + proc(proc_),pContent(content_) +{ + +} + +DWORD WINAPI WorkItemWrapperProc(LPVOID pContent) +{ + WorkItemWrapper* item = static_cast(pContent); + ScheduleProc TempProc = item->proc; + void* arg = item->pContent; + delete item; + TempProc(arg); + return 0; +} + +size_t GetPageSize() +{ + SYSTEM_INFO si; + GetSystemInfo(&si); + return std::max(si.dwPageSize,si.dwAllocationGranularity); +} + +const size_t g_PageSize = GetPageSize(); + + +Win32SequentialFile::Win32SequentialFile( const std::string& fname ) : + _filename(fname),_hFile(NULL) +{ + _Init(); +} + +Win32SequentialFile::~Win32SequentialFile() +{ + _CleanUp(); +} + +Status Win32SequentialFile::Read( size_t n, Slice* result, char* scratch ) +{ + Status sRet; + DWORD hasRead = 0; + if(_hFile && ReadFile(_hFile,scratch,n,&hasRead,NULL) ){ + *result = Slice(scratch,hasRead); + } else { + sRet = Status::IOError(_filename, Win32::GetLastErrSz() ); + } + return sRet; +} + +Status Win32SequentialFile::Skip( uint64_t n ) +{ + Status sRet; + LARGE_INTEGER Move,NowPointer; + Move.QuadPart = n; + if(!SetFilePointerEx(_hFile,Move,&NowPointer,FILE_CURRENT)){ + sRet = Status::IOError(_filename,Win32::GetLastErrSz()); + } + return sRet; +} + +BOOL Win32SequentialFile::isEnable() +{ + return _hFile ? TRUE : FALSE; +} + +BOOL Win32SequentialFile::_Init() +{ + std::wstring path; + ToWidePath(_filename, path); + _hFile = CreateFileW(path.c_str(), + GENERIC_READ, + FILE_SHARE_READ, + NULL, + OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL, + NULL); + return _hFile ? TRUE : FALSE; +} + +void Win32SequentialFile::_CleanUp() +{ + if(_hFile){ + CloseHandle(_hFile); + _hFile = NULL; + } +} + +Win32RandomAccessFile::Win32RandomAccessFile( const std::string& fname ) : + _filename(fname),_hFile(NULL) +{ + std::wstring path; + ToWidePath(fname, path); + _Init( path.c_str() ); +} + +Win32RandomAccessFile::~Win32RandomAccessFile() +{ + _CleanUp(); +} + +Status Win32RandomAccessFile::Read(uint64_t offset,size_t n,Slice* result,char* scratch) const +{ + Status sRet; + OVERLAPPED ol = {0}; + ZeroMemory(&ol,sizeof(ol)); + ol.Offset = (DWORD)offset; + ol.OffsetHigh = (DWORD)(offset >> 32); + DWORD hasRead = 0; + if(!ReadFile(_hFile,scratch,n,&hasRead,&ol)) + sRet = Status::IOError(_filename,Win32::GetLastErrSz()); + else + *result = Slice(scratch,hasRead); + return sRet; +} + +BOOL Win32RandomAccessFile::_Init( LPCWSTR path ) +{ + BOOL bRet = FALSE; + if(!_hFile) + _hFile = ::CreateFileW(path,GENERIC_READ,FILE_SHARE_READ,NULL,OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL | FILE_FLAG_RANDOM_ACCESS,NULL); + if(!_hFile || _hFile == INVALID_HANDLE_VALUE ) + _hFile = NULL; + else + bRet = TRUE; + return bRet; +} + +BOOL Win32RandomAccessFile::isEnable() +{ + return _hFile ? TRUE : FALSE; +} + +void Win32RandomAccessFile::_CleanUp() +{ + if(_hFile){ + ::CloseHandle(_hFile); + _hFile = NULL; + } +} + +size_t Win32MapFile::_Roundup( size_t x, size_t y ) +{ + return ((x + y - 1) / y) * y; +} + +size_t Win32MapFile::_TruncateToPageBoundary( size_t s ) +{ + s -= (s & (_page_size - 1)); + assert((s % _page_size) == 0); + return s; +} + +bool Win32MapFile::_UnmapCurrentRegion() +{ + bool result = true; + if (_base != NULL) { + if (_last_sync < _limit) { + // Defer syncing this data until next Sync() call, if any + _pending_sync = true; + } + if (!UnmapViewOfFile(_base) || !CloseHandle(_base_handle)) + result = false; + _file_offset += _limit - _base; + _base = NULL; + _base_handle = NULL; + _limit = NULL; + _last_sync = NULL; + _dst = NULL; + // Increase the amount we map the next time, but capped at 1MB + if (_map_size < (1<<20)) { + _map_size *= 2; + } + } + return result; +} + +bool Win32MapFile::_MapNewRegion() +{ + assert(_base == NULL); + //LONG newSizeHigh = (LONG)((file_offset_ + map_size_) >> 32); + //LONG newSizeLow = (LONG)((file_offset_ + map_size_) & 0xFFFFFFFF); + DWORD off_hi = (DWORD)(_file_offset >> 32); + DWORD off_lo = (DWORD)(_file_offset & 0xFFFFFFFF); + LARGE_INTEGER newSize; + newSize.QuadPart = _file_offset + _map_size; + SetFilePointerEx(_hFile, newSize, NULL, FILE_BEGIN); + SetEndOfFile(_hFile); + + _base_handle = CreateFileMappingA( + _hFile, + NULL, + PAGE_READWRITE, + 0, + 0, + 0); + if (_base_handle != NULL) { + _base = (char*) MapViewOfFile(_base_handle, + FILE_MAP_ALL_ACCESS, + off_hi, + off_lo, + _map_size); + if (_base != NULL) { + _limit = _base + _map_size; + _dst = _base; + _last_sync = _base; + return true; + } + } + return false; +} + +Win32MapFile::Win32MapFile( const std::string& fname) : + _filename(fname), + _hFile(NULL), + _page_size(Win32::g_PageSize), + _map_size(_Roundup(65536, Win32::g_PageSize)), + _base(NULL), + _base_handle(NULL), + _limit(NULL), + _dst(NULL), + _last_sync(NULL), + _file_offset(0), + _pending_sync(false) +{ + std::wstring path; + ToWidePath(fname, path); + _Init(path.c_str()); + assert((Win32::g_PageSize & (Win32::g_PageSize - 1)) == 0); +} + +Status Win32MapFile::Append( const Slice& data ) +{ + const char* src = data.data(); + size_t left = data.size(); + Status s; + while (left > 0) { + assert(_base <= _dst); + assert(_dst <= _limit); + size_t avail = _limit - _dst; + if (avail == 0) { + if (!_UnmapCurrentRegion() || + !_MapNewRegion()) { + return Status::IOError("WinMmapFile.Append::UnmapCurrentRegion or MapNewRegion: ", Win32::GetLastErrSz()); + } + } + size_t n = (left <= avail) ? left : avail; + memcpy(_dst, src, n); + _dst += n; + src += n; + left -= n; + } + return s; +} + +Status Win32MapFile::Close() +{ + Status s; + size_t unused = _limit - _dst; + if (!_UnmapCurrentRegion()) { + s = Status::IOError("WinMmapFile.Close::UnmapCurrentRegion: ",Win32::GetLastErrSz()); + } else if (unused > 0) { + // Trim the extra space at the end of the file + LARGE_INTEGER newSize; + newSize.QuadPart = _file_offset - unused; + if (!SetFilePointerEx(_hFile, newSize, NULL, FILE_BEGIN)) { + s = Status::IOError("WinMmapFile.Close::SetFilePointer: ",Win32::GetLastErrSz()); + } else + SetEndOfFile(_hFile); + } + if (!CloseHandle(_hFile)) { + if (s.ok()) { + s = Status::IOError("WinMmapFile.Close::CloseHandle: ", Win32::GetLastErrSz()); + } + } + _hFile = INVALID_HANDLE_VALUE; + _base = NULL; + _base_handle = NULL; + _limit = NULL; + + return s; +} + +Status Win32MapFile::Sync() +{ + Status s; + if (_pending_sync) { + // Some unmapped data was not synced + _pending_sync = false; + if (!FlushFileBuffers(_hFile)) { + s = Status::IOError("WinMmapFile.Sync::FlushFileBuffers: ",Win32::GetLastErrSz()); + } + } + if (_dst > _last_sync) { + // Find the beginnings of the pages that contain the first and last + // bytes to be synced. + size_t p1 = _TruncateToPageBoundary(_last_sync - _base); + size_t p2 = _TruncateToPageBoundary(_dst - _base - 1); + _last_sync = _dst; + if (!FlushViewOfFile(_base + p1, p2 - p1 + _page_size)) { + s = Status::IOError("WinMmapFile.Sync::FlushViewOfFile: ",Win32::GetLastErrSz()); + } + } + return s; +} + +Status Win32MapFile::Flush() +{ + return Status::OK(); +} + +Win32MapFile::~Win32MapFile() +{ + if (_hFile != INVALID_HANDLE_VALUE) { + Win32MapFile::Close(); + } +} + +BOOL Win32MapFile::_Init( LPCWSTR Path ) +{ + DWORD Flag = PathFileExistsW(Path) ? OPEN_EXISTING : CREATE_ALWAYS; + _hFile = CreateFileW(Path, + GENERIC_READ | GENERIC_WRITE, + FILE_SHARE_READ|FILE_SHARE_DELETE|FILE_SHARE_WRITE, + NULL, + Flag, + FILE_ATTRIBUTE_NORMAL, + NULL); + if(!_hFile || _hFile == INVALID_HANDLE_VALUE) + return FALSE; + else + return TRUE; +} + +BOOL Win32MapFile::isEnable() +{ + return _hFile ? TRUE : FALSE; +} + +Win32FileLock::Win32FileLock( const std::string& fname ) : + _hFile(NULL),_filename(fname) +{ + std::wstring path; + ToWidePath(fname, path); + _Init(path.c_str()); +} + +Win32FileLock::~Win32FileLock() +{ + _CleanUp(); +} + +BOOL Win32FileLock::_Init( LPCWSTR path ) +{ + BOOL bRet = FALSE; + if(!_hFile) + _hFile = ::CreateFileW(path,0,0,NULL,CREATE_ALWAYS,FILE_ATTRIBUTE_NORMAL,NULL); + if(!_hFile || _hFile == INVALID_HANDLE_VALUE ){ + _hFile = NULL; + } + else + bRet = TRUE; + return bRet; +} + +void Win32FileLock::_CleanUp() +{ + ::CloseHandle(_hFile); + _hFile = NULL; +} + +BOOL Win32FileLock::isEnable() +{ + return _hFile ? TRUE : FALSE; +} + +Win32Logger::Win32Logger(WritableFile* pFile) : _pFileProxy(pFile) +{ + assert(_pFileProxy); +} + +Win32Logger::~Win32Logger() +{ + if(_pFileProxy) + delete _pFileProxy; +} + +void Win32Logger::Logv( const char* format, va_list ap ) +{ + uint64_t thread_id = ::GetCurrentThreadId(); + + // We try twice: the first time with a fixed-size stack allocated buffer, + // and the second time with a much larger dynamically allocated buffer. + char buffer[500]; + for (int iter = 0; iter < 2; iter++) { + char* base; + int bufsize; + if (iter == 0) { + bufsize = sizeof(buffer); + base = buffer; + } else { + bufsize = 30000; + base = new char[bufsize]; + } + char* p = base; + char* limit = base + bufsize; + + SYSTEMTIME st; + GetLocalTime(&st); + p += snprintf(p, limit - p, + "%04d/%02d/%02d-%02d:%02d:%02d.%06d %llx ", + int(st.wYear), + int(st.wMonth), + int(st.wDay), + int(st.wHour), + int(st.wMinute), + int(st.wMinute), + int(st.wMilliseconds), + static_cast(thread_id)); + + // Print the message + if (p < limit) { + va_list backup_ap; + va_copy(backup_ap, ap); + p += vsnprintf(p, limit - p, format, backup_ap); + va_end(backup_ap); + } + + // Truncate to available space if necessary + if (p >= limit) { + if (iter == 0) { + continue; // Try again with larger buffer + } else { + p = limit - 1; + } + } + + // Add newline if necessary + if (p == base || p[-1] != '\n') { + *p++ = '\n'; + } + + assert(p <= limit); + DWORD hasWritten = 0; + if(_pFileProxy){ + _pFileProxy->Append(Slice(base, p - base)); + _pFileProxy->Flush(); + } + if (base != buffer) { + delete[] base; + } + break; + } +} + +bool Win32Env::FileExists(const std::string& fname) +{ + std::string path = fname; + std::wstring wpath; + ToWidePath(ModifyPath(path), wpath); + return ::PathFileExistsW(wpath.c_str()) ? true : false; +} + +Status Win32Env::GetChildren(const std::string& dir, std::vector* result) +{ + Status sRet; + ::WIN32_FIND_DATAW wfd; + std::string path = dir; + ModifyPath(path); + path += "\\*.*"; + std::wstring wpath; + ToWidePath(path, wpath); + + ::HANDLE hFind = ::FindFirstFileW(wpath.c_str() ,&wfd); + if(hFind && hFind != INVALID_HANDLE_VALUE){ + BOOL hasNext = TRUE; + std::string child; + while(hasNext){ + ToNarrowPath(wfd.cFileName, child); + if(child != ".." && child != ".") { + result->push_back(child); + } + hasNext = ::FindNextFileW(hFind,&wfd); + } + ::FindClose(hFind); + } + else + sRet = Status::IOError(dir,"Could not get children."); + return sRet; +} + +void Win32Env::SleepForMicroseconds( int micros ) +{ + ::Sleep((micros + 999) /1000); +} + + +Status Win32Env::DeleteFile( const std::string& fname ) +{ + Status sRet; + std::string path = fname; + std::wstring wpath; + ToWidePath(ModifyPath(path), wpath); + + if(!::DeleteFileW(wpath.c_str())) { + sRet = Status::IOError(path, "Could not delete file."); + } + return sRet; +} + +Status Win32Env::GetFileSize( const std::string& fname, uint64_t* file_size ) +{ + Status sRet; + std::string path = fname; + std::wstring wpath; + ToWidePath(ModifyPath(path), wpath); + + HANDLE file = ::CreateFileW(wpath.c_str(), + GENERIC_READ,FILE_SHARE_READ,NULL,OPEN_EXISTING,FILE_ATTRIBUTE_NORMAL,NULL); + LARGE_INTEGER li; + if(::GetFileSizeEx(file,&li)){ + *file_size = (uint64_t)li.QuadPart; + }else + sRet = Status::IOError(path,"Could not get the file size."); + CloseHandle(file); + return sRet; +} + +Status Win32Env::RenameFile( const std::string& src, const std::string& target ) +{ + Status sRet; + std::string src_path = src; + std::wstring wsrc_path; + ToWidePath(ModifyPath(src_path), wsrc_path); + std::string target_path = target; + std::wstring wtarget_path; + ToWidePath(ModifyPath(target_path), wtarget_path); + + if(!MoveFileW(wsrc_path.c_str(), wtarget_path.c_str() ) ){ + DWORD err = GetLastError(); + if(err == 0x000000b7){ + if(!::DeleteFileW(wtarget_path.c_str() ) ) + sRet = Status::IOError(src, "Could not rename file."); + else if(!::MoveFileW(wsrc_path.c_str(), + wtarget_path.c_str() ) ) + sRet = Status::IOError(src, "Could not rename file."); + } + } + return sRet; +} + +Status Win32Env::LockFile( const std::string& fname, FileLock** lock ) +{ + Status sRet; + std::string path = fname; + ModifyPath(path); + Win32FileLock* _lock = new Win32FileLock(path); + if(!_lock->isEnable()){ + delete _lock; + *lock = NULL; + sRet = Status::IOError(path, "Could not lock file."); + } + else + *lock = _lock; + return sRet; +} + +Status Win32Env::UnlockFile( FileLock* lock ) +{ + Status sRet; + delete lock; + return sRet; +} + +void Win32Env::Schedule( void (*function)(void* arg), void* arg ) +{ + QueueUserWorkItem(Win32::WorkItemWrapperProc, + new Win32::WorkItemWrapper(function,arg), + WT_EXECUTEDEFAULT); +} + +void Win32Env::StartThread( void (*function)(void* arg), void* arg ) +{ + ::_beginthread(function,0,arg); +} + +Status Win32Env::GetTestDirectory( std::string* path ) +{ + Status sRet; + WCHAR TempPath[MAX_PATH]; + ::GetTempPathW(MAX_PATH,TempPath); + ToNarrowPath(TempPath, *path); + path->append("leveldb\\test\\"); + ModifyPath(*path); + return sRet; +} + +uint64_t Win32Env::NowMicros() +{ +#ifndef USE_VISTA_API +#define GetTickCount64 GetTickCount +#endif + return (uint64_t)(GetTickCount64()*1000); +} + +static Status CreateDirInner( const std::string& dirname ) +{ + Status sRet; + DWORD attr = ::GetFileAttributes(dirname.c_str()); + if (attr == INVALID_FILE_ATTRIBUTES) { // doesn't exist: + std::size_t slash = dirname.find_last_of("\\"); + if (slash != std::string::npos){ + sRet = CreateDirInner(dirname.substr(0, slash)); + if (!sRet.ok()) return sRet; + } + BOOL result = ::CreateDirectory(dirname.c_str(), NULL); + if (result == FALSE) { + sRet = Status::IOError(dirname, "Could not create directory."); + return sRet; + } + } + return sRet; +} + +Status Win32Env::CreateDir( const std::string& dirname ) +{ + std::string path = dirname; + if(path[path.length() - 1] != '\\'){ + path += '\\'; + } + ModifyPath(path); + + return CreateDirInner(path); +} + +Status Win32Env::DeleteDir( const std::string& dirname ) +{ + Status sRet; + std::wstring path; + ToWidePath(dirname, path); + ModifyPath(path); + if(!::RemoveDirectoryW( path.c_str() ) ){ + sRet = Status::IOError(dirname, "Could not delete directory."); + } + return sRet; +} + +Status Win32Env::NewSequentialFile( const std::string& fname, SequentialFile** result ) +{ + Status sRet; + std::string path = fname; + ModifyPath(path); + Win32SequentialFile* pFile = new Win32SequentialFile(path); + if(pFile->isEnable()){ + *result = pFile; + }else { + delete pFile; + sRet = Status::IOError(path, Win32::GetLastErrSz()); + } + return sRet; +} + +Status Win32Env::NewRandomAccessFile( const std::string& fname, RandomAccessFile** result ) +{ + Status sRet; + std::string path = fname; + Win32RandomAccessFile* pFile = new Win32RandomAccessFile(ModifyPath(path)); + if(!pFile->isEnable()){ + delete pFile; + *result = NULL; + sRet = Status::IOError(path, Win32::GetLastErrSz()); + }else + *result = pFile; + return sRet; +} + +Status Win32Env::NewLogger( const std::string& fname, Logger** result ) +{ + Status sRet; + std::string path = fname; + Win32MapFile* pMapFile = new Win32MapFile(ModifyPath(path)); + if(!pMapFile->isEnable()){ + delete pMapFile; + *result = NULL; + sRet = Status::IOError(path,"could not create a logger."); + }else + *result = new Win32Logger(pMapFile); + return sRet; +} + +Status Win32Env::NewWritableFile( const std::string& fname, WritableFile** result ) +{ + Status sRet; + std::string path = fname; + Win32MapFile* pFile = new Win32MapFile(ModifyPath(path)); + if(!pFile->isEnable()){ + *result = NULL; + sRet = Status::IOError(fname,Win32::GetLastErrSz()); + }else + *result = pFile; + return sRet; +} + +Win32Env::Win32Env() +{ + +} + +Win32Env::~Win32Env() +{ + +} + + +} // Win32 namespace + +static port::OnceType once = LEVELDB_ONCE_INIT; +static Env* default_env; +static void InitDefaultEnv() { default_env = new Win32::Win32Env(); } + +Env* Env::Default() { + port::InitOnce(&once, InitDefaultEnv); + return default_env; +} + +} // namespace leveldb + +#endif // defined(LEVELDB_PLATFORM_WINDOWS) diff --git a/src/leveldb/util/filter_policy.cc b/src/leveldb/util/filter_policy.cc new file mode 100644 index 00000000..7b045c8c --- /dev/null +++ b/src/leveldb/util/filter_policy.cc @@ -0,0 +1,11 @@ +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/filter_policy.h" + +namespace leveldb { + +FilterPolicy::~FilterPolicy() { } + +} // namespace leveldb diff --git a/src/leveldb/util/hash.cc b/src/leveldb/util/hash.cc new file mode 100644 index 00000000..ed439ce7 --- /dev/null +++ b/src/leveldb/util/hash.cc @@ -0,0 +1,52 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include "util/coding.h" +#include "util/hash.h" + +// The FALLTHROUGH_INTENDED macro can be used to annotate implicit fall-through +// between switch labels. The real definition should be provided externally. +// This one is a fallback version for unsupported compilers. +#ifndef FALLTHROUGH_INTENDED +#define FALLTHROUGH_INTENDED do { } while (0) +#endif + +namespace leveldb { + +uint32_t Hash(const char* data, size_t n, uint32_t seed) { + // Similar to murmur hash + const uint32_t m = 0xc6a4a793; + const uint32_t r = 24; + const char* limit = data + n; + uint32_t h = seed ^ (n * m); + + // Pick up four bytes at a time + while (data + 4 <= limit) { + uint32_t w = DecodeFixed32(data); + data += 4; + h += w; + h *= m; + h ^= (h >> 16); + } + + // Pick up remaining bytes + switch (limit - data) { + case 3: + h += static_cast(data[2]) << 16; + FALLTHROUGH_INTENDED; + case 2: + h += static_cast(data[1]) << 8; + FALLTHROUGH_INTENDED; + case 1: + h += static_cast(data[0]); + h *= m; + h ^= (h >> r); + break; + } + return h; +} + + +} // namespace leveldb diff --git a/src/leveldb/util/hash.h b/src/leveldb/util/hash.h new file mode 100644 index 00000000..8889d56b --- /dev/null +++ b/src/leveldb/util/hash.h @@ -0,0 +1,19 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// Simple hash function used for internal data structures + +#ifndef STORAGE_LEVELDB_UTIL_HASH_H_ +#define STORAGE_LEVELDB_UTIL_HASH_H_ + +#include +#include + +namespace leveldb { + +extern uint32_t Hash(const char* data, size_t n, uint32_t seed); + +} + +#endif // STORAGE_LEVELDB_UTIL_HASH_H_ diff --git a/src/leveldb/util/hash_test.cc b/src/leveldb/util/hash_test.cc new file mode 100644 index 00000000..eaa1c92c --- /dev/null +++ b/src/leveldb/util/hash_test.cc @@ -0,0 +1,54 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "util/hash.h" +#include "util/testharness.h" + +namespace leveldb { + +class HASH { }; + +TEST(HASH, SignedUnsignedIssue) { + const unsigned char data1[1] = {0x62}; + const unsigned char data2[2] = {0xc3, 0x97}; + const unsigned char data3[3] = {0xe2, 0x99, 0xa5}; + const unsigned char data4[4] = {0xe1, 0x80, 0xb9, 0x32}; + const unsigned char data5[48] = { + 0x01, 0xc0, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x14, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x04, 0x00, + 0x00, 0x00, 0x00, 0x14, + 0x00, 0x00, 0x00, 0x18, + 0x28, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x02, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + }; + + ASSERT_EQ(Hash(0, 0, 0xbc9f1d34), 0xbc9f1d34); + ASSERT_EQ( + Hash(reinterpret_cast(data1), sizeof(data1), 0xbc9f1d34), + 0xef1345c4); + ASSERT_EQ( + Hash(reinterpret_cast(data2), sizeof(data2), 0xbc9f1d34), + 0x5b663814); + ASSERT_EQ( + Hash(reinterpret_cast(data3), sizeof(data3), 0xbc9f1d34), + 0x323c078f); + ASSERT_EQ( + Hash(reinterpret_cast(data4), sizeof(data4), 0xbc9f1d34), + 0xed21633a); + ASSERT_EQ( + Hash(reinterpret_cast(data5), sizeof(data5), 0x12345678), + 0xf333dabb); +} + +} // namespace leveldb + +int main(int argc, char** argv) { + return leveldb::test::RunAllTests(); +} diff --git a/src/leveldb/util/histogram.cc b/src/leveldb/util/histogram.cc new file mode 100644 index 00000000..bb95f583 --- /dev/null +++ b/src/leveldb/util/histogram.cc @@ -0,0 +1,139 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include +#include "port/port.h" +#include "util/histogram.h" + +namespace leveldb { + +const double Histogram::kBucketLimit[kNumBuckets] = { + 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 16, 18, 20, 25, 30, 35, 40, 45, + 50, 60, 70, 80, 90, 100, 120, 140, 160, 180, 200, 250, 300, 350, 400, 450, + 500, 600, 700, 800, 900, 1000, 1200, 1400, 1600, 1800, 2000, 2500, 3000, + 3500, 4000, 4500, 5000, 6000, 7000, 8000, 9000, 10000, 12000, 14000, + 16000, 18000, 20000, 25000, 30000, 35000, 40000, 45000, 50000, 60000, + 70000, 80000, 90000, 100000, 120000, 140000, 160000, 180000, 200000, + 250000, 300000, 350000, 400000, 450000, 500000, 600000, 700000, 800000, + 900000, 1000000, 1200000, 1400000, 1600000, 1800000, 2000000, 2500000, + 3000000, 3500000, 4000000, 4500000, 5000000, 6000000, 7000000, 8000000, + 9000000, 10000000, 12000000, 14000000, 16000000, 18000000, 20000000, + 25000000, 30000000, 35000000, 40000000, 45000000, 50000000, 60000000, + 70000000, 80000000, 90000000, 100000000, 120000000, 140000000, 160000000, + 180000000, 200000000, 250000000, 300000000, 350000000, 400000000, + 450000000, 500000000, 600000000, 700000000, 800000000, 900000000, + 1000000000, 1200000000, 1400000000, 1600000000, 1800000000, 2000000000, + 2500000000.0, 3000000000.0, 3500000000.0, 4000000000.0, 4500000000.0, + 5000000000.0, 6000000000.0, 7000000000.0, 8000000000.0, 9000000000.0, + 1e200, +}; + +void Histogram::Clear() { + min_ = kBucketLimit[kNumBuckets-1]; + max_ = 0; + num_ = 0; + sum_ = 0; + sum_squares_ = 0; + for (int i = 0; i < kNumBuckets; i++) { + buckets_[i] = 0; + } +} + +void Histogram::Add(double value) { + // Linear search is fast enough for our usage in db_bench + int b = 0; + while (b < kNumBuckets - 1 && kBucketLimit[b] <= value) { + b++; + } + buckets_[b] += 1.0; + if (min_ > value) min_ = value; + if (max_ < value) max_ = value; + num_++; + sum_ += value; + sum_squares_ += (value * value); +} + +void Histogram::Merge(const Histogram& other) { + if (other.min_ < min_) min_ = other.min_; + if (other.max_ > max_) max_ = other.max_; + num_ += other.num_; + sum_ += other.sum_; + sum_squares_ += other.sum_squares_; + for (int b = 0; b < kNumBuckets; b++) { + buckets_[b] += other.buckets_[b]; + } +} + +double Histogram::Median() const { + return Percentile(50.0); +} + +double Histogram::Percentile(double p) const { + double threshold = num_ * (p / 100.0); + double sum = 0; + for (int b = 0; b < kNumBuckets; b++) { + sum += buckets_[b]; + if (sum >= threshold) { + // Scale linearly within this bucket + double left_point = (b == 0) ? 0 : kBucketLimit[b-1]; + double right_point = kBucketLimit[b]; + double left_sum = sum - buckets_[b]; + double right_sum = sum; + double pos = (threshold - left_sum) / (right_sum - left_sum); + double r = left_point + (right_point - left_point) * pos; + if (r < min_) r = min_; + if (r > max_) r = max_; + return r; + } + } + return max_; +} + +double Histogram::Average() const { + if (num_ == 0.0) return 0; + return sum_ / num_; +} + +double Histogram::StandardDeviation() const { + if (num_ == 0.0) return 0; + double variance = (sum_squares_ * num_ - sum_ * sum_) / (num_ * num_); + return sqrt(variance); +} + +std::string Histogram::ToString() const { + std::string r; + char buf[200]; + snprintf(buf, sizeof(buf), + "Count: %.0f Average: %.4f StdDev: %.2f\n", + num_, Average(), StandardDeviation()); + r.append(buf); + snprintf(buf, sizeof(buf), + "Min: %.4f Median: %.4f Max: %.4f\n", + (num_ == 0.0 ? 0.0 : min_), Median(), max_); + r.append(buf); + r.append("------------------------------------------------------\n"); + const double mult = 100.0 / num_; + double sum = 0; + for (int b = 0; b < kNumBuckets; b++) { + if (buckets_[b] <= 0.0) continue; + sum += buckets_[b]; + snprintf(buf, sizeof(buf), + "[ %7.0f, %7.0f ) %7.0f %7.3f%% %7.3f%% ", + ((b == 0) ? 0.0 : kBucketLimit[b-1]), // left + kBucketLimit[b], // right + buckets_[b], // count + mult * buckets_[b], // percentage + mult * sum); // cumulative percentage + r.append(buf); + + // Add hash marks based on percentage; 20 marks for 100%. + int marks = static_cast(20*(buckets_[b] / num_) + 0.5); + r.append(marks, '#'); + r.push_back('\n'); + } + return r; +} + +} // namespace leveldb diff --git a/src/leveldb/util/histogram.h b/src/leveldb/util/histogram.h new file mode 100644 index 00000000..1ef9f3c8 --- /dev/null +++ b/src/leveldb/util/histogram.h @@ -0,0 +1,42 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_UTIL_HISTOGRAM_H_ +#define STORAGE_LEVELDB_UTIL_HISTOGRAM_H_ + +#include + +namespace leveldb { + +class Histogram { + public: + Histogram() { } + ~Histogram() { } + + void Clear(); + void Add(double value); + void Merge(const Histogram& other); + + std::string ToString() const; + + private: + double min_; + double max_; + double num_; + double sum_; + double sum_squares_; + + enum { kNumBuckets = 154 }; + static const double kBucketLimit[kNumBuckets]; + double buckets_[kNumBuckets]; + + double Median() const; + double Percentile(double p) const; + double Average() const; + double StandardDeviation() const; +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_UTIL_HISTOGRAM_H_ diff --git a/src/leveldb/util/logging.cc b/src/leveldb/util/logging.cc new file mode 100644 index 00000000..ca6b3244 --- /dev/null +++ b/src/leveldb/util/logging.cc @@ -0,0 +1,72 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "util/logging.h" + +#include +#include +#include +#include +#include "leveldb/env.h" +#include "leveldb/slice.h" + +namespace leveldb { + +void AppendNumberTo(std::string* str, uint64_t num) { + char buf[30]; + snprintf(buf, sizeof(buf), "%llu", (unsigned long long) num); + str->append(buf); +} + +void AppendEscapedStringTo(std::string* str, const Slice& value) { + for (size_t i = 0; i < value.size(); i++) { + char c = value[i]; + if (c >= ' ' && c <= '~') { + str->push_back(c); + } else { + char buf[10]; + snprintf(buf, sizeof(buf), "\\x%02x", + static_cast(c) & 0xff); + str->append(buf); + } + } +} + +std::string NumberToString(uint64_t num) { + std::string r; + AppendNumberTo(&r, num); + return r; +} + +std::string EscapeString(const Slice& value) { + std::string r; + AppendEscapedStringTo(&r, value); + return r; +} + +bool ConsumeDecimalNumber(Slice* in, uint64_t* val) { + uint64_t v = 0; + int digits = 0; + while (!in->empty()) { + char c = (*in)[0]; + if (c >= '0' && c <= '9') { + ++digits; + const int delta = (c - '0'); + static const uint64_t kMaxUint64 = ~static_cast(0); + if (v > kMaxUint64/10 || + (v == kMaxUint64/10 && delta > kMaxUint64%10)) { + // Overflow + return false; + } + v = (v * 10) + delta; + in->remove_prefix(1); + } else { + break; + } + } + *val = v; + return (digits > 0); +} + +} // namespace leveldb diff --git a/src/leveldb/util/logging.h b/src/leveldb/util/logging.h new file mode 100644 index 00000000..1b450d24 --- /dev/null +++ b/src/leveldb/util/logging.h @@ -0,0 +1,43 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// Must not be included from any .h files to avoid polluting the namespace +// with macros. + +#ifndef STORAGE_LEVELDB_UTIL_LOGGING_H_ +#define STORAGE_LEVELDB_UTIL_LOGGING_H_ + +#include +#include +#include +#include "port/port.h" + +namespace leveldb { + +class Slice; +class WritableFile; + +// Append a human-readable printout of "num" to *str +extern void AppendNumberTo(std::string* str, uint64_t num); + +// Append a human-readable printout of "value" to *str. +// Escapes any non-printable characters found in "value". +extern void AppendEscapedStringTo(std::string* str, const Slice& value); + +// Return a human-readable printout of "num" +extern std::string NumberToString(uint64_t num); + +// Return a human-readable version of "value". +// Escapes any non-printable characters found in "value". +extern std::string EscapeString(const Slice& value); + +// Parse a human-readable number from "*in" into *value. On success, +// advances "*in" past the consumed number and sets "*val" to the +// numeric value. Otherwise, returns false and leaves *in in an +// unspecified state. +extern bool ConsumeDecimalNumber(Slice* in, uint64_t* val); + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_UTIL_LOGGING_H_ diff --git a/src/leveldb/util/mutexlock.h b/src/leveldb/util/mutexlock.h new file mode 100644 index 00000000..1ff5a9ef --- /dev/null +++ b/src/leveldb/util/mutexlock.h @@ -0,0 +1,41 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_UTIL_MUTEXLOCK_H_ +#define STORAGE_LEVELDB_UTIL_MUTEXLOCK_H_ + +#include "port/port.h" +#include "port/thread_annotations.h" + +namespace leveldb { + +// Helper class that locks a mutex on construction and unlocks the mutex when +// the destructor of the MutexLock object is invoked. +// +// Typical usage: +// +// void MyClass::MyMethod() { +// MutexLock l(&mu_); // mu_ is an instance variable +// ... some complex code, possibly with multiple return paths ... +// } + +class SCOPED_LOCKABLE MutexLock { + public: + explicit MutexLock(port::Mutex *mu) EXCLUSIVE_LOCK_FUNCTION(mu) + : mu_(mu) { + this->mu_->Lock(); + } + ~MutexLock() UNLOCK_FUNCTION() { this->mu_->Unlock(); } + + private: + port::Mutex *const mu_; + // No copying allowed + MutexLock(const MutexLock&); + void operator=(const MutexLock&); +}; + +} // namespace leveldb + + +#endif // STORAGE_LEVELDB_UTIL_MUTEXLOCK_H_ diff --git a/src/leveldb/util/options.cc b/src/leveldb/util/options.cc new file mode 100644 index 00000000..76af5b93 --- /dev/null +++ b/src/leveldb/util/options.cc @@ -0,0 +1,29 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "leveldb/options.h" + +#include "leveldb/comparator.h" +#include "leveldb/env.h" + +namespace leveldb { + +Options::Options() + : comparator(BytewiseComparator()), + create_if_missing(false), + error_if_exists(false), + paranoid_checks(false), + env(Env::Default()), + info_log(NULL), + write_buffer_size(4<<20), + max_open_files(1000), + block_cache(NULL), + block_size(4096), + block_restart_interval(16), + compression(kSnappyCompression), + filter_policy(NULL) { +} + + +} // namespace leveldb diff --git a/src/leveldb/util/posix_logger.h b/src/leveldb/util/posix_logger.h new file mode 100644 index 00000000..c063c2b7 --- /dev/null +++ b/src/leveldb/util/posix_logger.h @@ -0,0 +1,98 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// Logger implementation that can be shared by all environments +// where enough Posix functionality is available. + +#ifndef STORAGE_LEVELDB_UTIL_POSIX_LOGGER_H_ +#define STORAGE_LEVELDB_UTIL_POSIX_LOGGER_H_ + +#include +#include +#include +#include +#include "leveldb/env.h" + +namespace leveldb { + +class PosixLogger : public Logger { + private: + FILE* file_; + uint64_t (*gettid_)(); // Return the thread id for the current thread + public: + PosixLogger(FILE* f, uint64_t (*gettid)()) : file_(f), gettid_(gettid) { } + virtual ~PosixLogger() { + fclose(file_); + } + virtual void Logv(const char* format, va_list ap) { + const uint64_t thread_id = (*gettid_)(); + + // We try twice: the first time with a fixed-size stack allocated buffer, + // and the second time with a much larger dynamically allocated buffer. + char buffer[500]; + for (int iter = 0; iter < 2; iter++) { + char* base; + int bufsize; + if (iter == 0) { + bufsize = sizeof(buffer); + base = buffer; + } else { + bufsize = 30000; + base = new char[bufsize]; + } + char* p = base; + char* limit = base + bufsize; + + struct timeval now_tv; + gettimeofday(&now_tv, NULL); + const time_t seconds = now_tv.tv_sec; + struct tm t; + localtime_r(&seconds, &t); + p += snprintf(p, limit - p, + "%04d/%02d/%02d-%02d:%02d:%02d.%06d %llx ", + t.tm_year + 1900, + t.tm_mon + 1, + t.tm_mday, + t.tm_hour, + t.tm_min, + t.tm_sec, + static_cast(now_tv.tv_usec), + static_cast(thread_id)); + + // Print the message + if (p < limit) { + va_list backup_ap; + va_copy(backup_ap, ap); + p += vsnprintf(p, limit - p, format, backup_ap); + va_end(backup_ap); + } + + // Truncate to available space if necessary + if (p >= limit) { + if (iter == 0) { + continue; // Try again with larger buffer + } else { + p = limit - 1; + } + } + + // Add newline if necessary + if (p == base || p[-1] != '\n') { + *p++ = '\n'; + } + + assert(p <= limit); + fwrite(base, 1, p - base, file_); + fflush(file_); + if (base != buffer) { + delete[] base; + } + break; + } + } +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_UTIL_POSIX_LOGGER_H_ diff --git a/src/leveldb/util/random.h b/src/leveldb/util/random.h new file mode 100644 index 00000000..ddd51b1c --- /dev/null +++ b/src/leveldb/util/random.h @@ -0,0 +1,64 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_UTIL_RANDOM_H_ +#define STORAGE_LEVELDB_UTIL_RANDOM_H_ + +#include + +namespace leveldb { + +// A very simple random number generator. Not especially good at +// generating truly random bits, but good enough for our needs in this +// package. +class Random { + private: + uint32_t seed_; + public: + explicit Random(uint32_t s) : seed_(s & 0x7fffffffu) { + // Avoid bad seeds. + if (seed_ == 0 || seed_ == 2147483647L) { + seed_ = 1; + } + } + uint32_t Next() { + static const uint32_t M = 2147483647L; // 2^31-1 + static const uint64_t A = 16807; // bits 14, 8, 7, 5, 2, 1, 0 + // We are computing + // seed_ = (seed_ * A) % M, where M = 2^31-1 + // + // seed_ must not be zero or M, or else all subsequent computed values + // will be zero or M respectively. For all other values, seed_ will end + // up cycling through every number in [1,M-1] + uint64_t product = seed_ * A; + + // Compute (product % M) using the fact that ((x << 31) % M) == x. + seed_ = static_cast((product >> 31) + (product & M)); + // The first reduction may overflow by 1 bit, so we may need to + // repeat. mod == M is not possible; using > allows the faster + // sign-bit-based test. + if (seed_ > M) { + seed_ -= M; + } + return seed_; + } + // Returns a uniformly distributed value in the range [0..n-1] + // REQUIRES: n > 0 + uint32_t Uniform(int n) { return Next() % n; } + + // Randomly returns true ~"1/n" of the time, and false otherwise. + // REQUIRES: n > 0 + bool OneIn(int n) { return (Next() % n) == 0; } + + // Skewed: pick "base" uniformly from range [0,max_log] and then + // return "base" random bits. The effect is to pick a number in the + // range [0,2^max_log-1] with exponential bias towards smaller numbers. + uint32_t Skewed(int max_log) { + return Uniform(1 << Uniform(max_log + 1)); + } +}; + +} // namespace leveldb + +#endif // STORAGE_LEVELDB_UTIL_RANDOM_H_ diff --git a/src/leveldb/util/status.cc b/src/leveldb/util/status.cc new file mode 100644 index 00000000..a44f35b3 --- /dev/null +++ b/src/leveldb/util/status.cc @@ -0,0 +1,75 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include "port/port.h" +#include "leveldb/status.h" + +namespace leveldb { + +const char* Status::CopyState(const char* state) { + uint32_t size; + memcpy(&size, state, sizeof(size)); + char* result = new char[size + 5]; + memcpy(result, state, size + 5); + return result; +} + +Status::Status(Code code, const Slice& msg, const Slice& msg2) { + assert(code != kOk); + const uint32_t len1 = msg.size(); + const uint32_t len2 = msg2.size(); + const uint32_t size = len1 + (len2 ? (2 + len2) : 0); + char* result = new char[size + 5]; + memcpy(result, &size, sizeof(size)); + result[4] = static_cast(code); + memcpy(result + 5, msg.data(), len1); + if (len2) { + result[5 + len1] = ':'; + result[6 + len1] = ' '; + memcpy(result + 7 + len1, msg2.data(), len2); + } + state_ = result; +} + +std::string Status::ToString() const { + if (state_ == NULL) { + return "OK"; + } else { + char tmp[30]; + const char* type; + switch (code()) { + case kOk: + type = "OK"; + break; + case kNotFound: + type = "NotFound: "; + break; + case kCorruption: + type = "Corruption: "; + break; + case kNotSupported: + type = "Not implemented: "; + break; + case kInvalidArgument: + type = "Invalid argument: "; + break; + case kIOError: + type = "IO error: "; + break; + default: + snprintf(tmp, sizeof(tmp), "Unknown code(%d): ", + static_cast(code())); + type = tmp; + break; + } + std::string result(type); + uint32_t length; + memcpy(&length, state_, sizeof(length)); + result.append(state_ + 5, length); + return result; + } +} + +} // namespace leveldb diff --git a/src/leveldb/util/testharness.cc b/src/leveldb/util/testharness.cc new file mode 100644 index 00000000..402fab34 --- /dev/null +++ b/src/leveldb/util/testharness.cc @@ -0,0 +1,77 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "util/testharness.h" + +#include +#include +#include +#include + +namespace leveldb { +namespace test { + +namespace { +struct Test { + const char* base; + const char* name; + void (*func)(); +}; +std::vector* tests; +} + +bool RegisterTest(const char* base, const char* name, void (*func)()) { + if (tests == NULL) { + tests = new std::vector; + } + Test t; + t.base = base; + t.name = name; + t.func = func; + tests->push_back(t); + return true; +} + +int RunAllTests() { + const char* matcher = getenv("LEVELDB_TESTS"); + + int num = 0; + if (tests != NULL) { + for (size_t i = 0; i < tests->size(); i++) { + const Test& t = (*tests)[i]; + if (matcher != NULL) { + std::string name = t.base; + name.push_back('.'); + name.append(t.name); + if (strstr(name.c_str(), matcher) == NULL) { + continue; + } + } + fprintf(stderr, "==== Test %s.%s\n", t.base, t.name); + (*t.func)(); + ++num; + } + } + fprintf(stderr, "==== PASSED %d tests\n", num); + return 0; +} + +std::string TmpDir() { + std::string dir; + Status s = Env::Default()->GetTestDirectory(&dir); + ASSERT_TRUE(s.ok()) << s.ToString(); + return dir; +} + +int RandomSeed() { + const char* env = getenv("TEST_RANDOM_SEED"); + int result = (env != NULL ? atoi(env) : 301); + if (result <= 0) { + result = 301; + } + return result; +} + +} // namespace test +} // namespace leveldb diff --git a/src/leveldb/util/testharness.h b/src/leveldb/util/testharness.h new file mode 100644 index 00000000..da4fe68b --- /dev/null +++ b/src/leveldb/util/testharness.h @@ -0,0 +1,138 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_UTIL_TESTHARNESS_H_ +#define STORAGE_LEVELDB_UTIL_TESTHARNESS_H_ + +#include +#include +#include +#include "leveldb/env.h" +#include "leveldb/slice.h" +#include "util/random.h" + +namespace leveldb { +namespace test { + +// Run some of the tests registered by the TEST() macro. If the +// environment variable "LEVELDB_TESTS" is not set, runs all tests. +// Otherwise, runs only the tests whose name contains the value of +// "LEVELDB_TESTS" as a substring. E.g., suppose the tests are: +// TEST(Foo, Hello) { ... } +// TEST(Foo, World) { ... } +// LEVELDB_TESTS=Hello will run the first test +// LEVELDB_TESTS=o will run both tests +// LEVELDB_TESTS=Junk will run no tests +// +// Returns 0 if all tests pass. +// Dies or returns a non-zero value if some test fails. +extern int RunAllTests(); + +// Return the directory to use for temporary storage. +extern std::string TmpDir(); + +// Return a randomization seed for this run. Typically returns the +// same number on repeated invocations of this binary, but automated +// runs may be able to vary the seed. +extern int RandomSeed(); + +// An instance of Tester is allocated to hold temporary state during +// the execution of an assertion. +class Tester { + private: + bool ok_; + const char* fname_; + int line_; + std::stringstream ss_; + + public: + Tester(const char* f, int l) + : ok_(true), fname_(f), line_(l) { + } + + ~Tester() { + if (!ok_) { + fprintf(stderr, "%s:%d:%s\n", fname_, line_, ss_.str().c_str()); + exit(1); + } + } + + Tester& Is(bool b, const char* msg) { + if (!b) { + ss_ << " Assertion failure " << msg; + ok_ = false; + } + return *this; + } + + Tester& IsOk(const Status& s) { + if (!s.ok()) { + ss_ << " " << s.ToString(); + ok_ = false; + } + return *this; + } + +#define BINARY_OP(name,op) \ + template \ + Tester& name(const X& x, const Y& y) { \ + if (! (x op y)) { \ + ss_ << " failed: " << x << (" " #op " ") << y; \ + ok_ = false; \ + } \ + return *this; \ + } + + BINARY_OP(IsEq, ==) + BINARY_OP(IsNe, !=) + BINARY_OP(IsGe, >=) + BINARY_OP(IsGt, >) + BINARY_OP(IsLe, <=) + BINARY_OP(IsLt, <) +#undef BINARY_OP + + // Attach the specified value to the error message if an error has occurred + template + Tester& operator<<(const V& value) { + if (!ok_) { + ss_ << " " << value; + } + return *this; + } +}; + +#define ASSERT_TRUE(c) ::leveldb::test::Tester(__FILE__, __LINE__).Is((c), #c) +#define ASSERT_OK(s) ::leveldb::test::Tester(__FILE__, __LINE__).IsOk((s)) +#define ASSERT_EQ(a,b) ::leveldb::test::Tester(__FILE__, __LINE__).IsEq((a),(b)) +#define ASSERT_NE(a,b) ::leveldb::test::Tester(__FILE__, __LINE__).IsNe((a),(b)) +#define ASSERT_GE(a,b) ::leveldb::test::Tester(__FILE__, __LINE__).IsGe((a),(b)) +#define ASSERT_GT(a,b) ::leveldb::test::Tester(__FILE__, __LINE__).IsGt((a),(b)) +#define ASSERT_LE(a,b) ::leveldb::test::Tester(__FILE__, __LINE__).IsLe((a),(b)) +#define ASSERT_LT(a,b) ::leveldb::test::Tester(__FILE__, __LINE__).IsLt((a),(b)) + +#define TCONCAT(a,b) TCONCAT1(a,b) +#define TCONCAT1(a,b) a##b + +#define TEST(base,name) \ +class TCONCAT(_Test_,name) : public base { \ + public: \ + void _Run(); \ + static void _RunIt() { \ + TCONCAT(_Test_,name) t; \ + t._Run(); \ + } \ +}; \ +bool TCONCAT(_Test_ignored_,name) = \ + ::leveldb::test::RegisterTest(#base, #name, &TCONCAT(_Test_,name)::_RunIt); \ +void TCONCAT(_Test_,name)::_Run() + +// Register the specified test. Typically not used directly, but +// invoked via the macro expansion of TEST. +extern bool RegisterTest(const char* base, const char* name, void (*func)()); + + +} // namespace test +} // namespace leveldb + +#endif // STORAGE_LEVELDB_UTIL_TESTHARNESS_H_ diff --git a/src/leveldb/util/testutil.cc b/src/leveldb/util/testutil.cc new file mode 100644 index 00000000..bee56bf7 --- /dev/null +++ b/src/leveldb/util/testutil.cc @@ -0,0 +1,51 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "util/testutil.h" + +#include "util/random.h" + +namespace leveldb { +namespace test { + +Slice RandomString(Random* rnd, int len, std::string* dst) { + dst->resize(len); + for (int i = 0; i < len; i++) { + (*dst)[i] = static_cast(' ' + rnd->Uniform(95)); // ' ' .. '~' + } + return Slice(*dst); +} + +std::string RandomKey(Random* rnd, int len) { + // Make sure to generate a wide variety of characters so we + // test the boundary conditions for short-key optimizations. + static const char kTestChars[] = { + '\0', '\1', 'a', 'b', 'c', 'd', 'e', '\xfd', '\xfe', '\xff' + }; + std::string result; + for (int i = 0; i < len; i++) { + result += kTestChars[rnd->Uniform(sizeof(kTestChars))]; + } + return result; +} + + +extern Slice CompressibleString(Random* rnd, double compressed_fraction, + size_t len, std::string* dst) { + int raw = static_cast(len * compressed_fraction); + if (raw < 1) raw = 1; + std::string raw_data; + RandomString(rnd, raw, &raw_data); + + // Duplicate the random data until we have filled "len" bytes + dst->clear(); + while (dst->size() < len) { + dst->append(raw_data); + } + dst->resize(len); + return Slice(*dst); +} + +} // namespace test +} // namespace leveldb diff --git a/src/leveldb/util/testutil.h b/src/leveldb/util/testutil.h new file mode 100644 index 00000000..adad3fc1 --- /dev/null +++ b/src/leveldb/util/testutil.h @@ -0,0 +1,53 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_UTIL_TESTUTIL_H_ +#define STORAGE_LEVELDB_UTIL_TESTUTIL_H_ + +#include "leveldb/env.h" +#include "leveldb/slice.h" +#include "util/random.h" + +namespace leveldb { +namespace test { + +// Store in *dst a random string of length "len" and return a Slice that +// references the generated data. +extern Slice RandomString(Random* rnd, int len, std::string* dst); + +// Return a random key with the specified length that may contain interesting +// characters (e.g. \x00, \xff, etc.). +extern std::string RandomKey(Random* rnd, int len); + +// Store in *dst a string of length "len" that will compress to +// "N*compressed_fraction" bytes and return a Slice that references +// the generated data. +extern Slice CompressibleString(Random* rnd, double compressed_fraction, + size_t len, std::string* dst); + +// A wrapper that allows injection of errors. +class ErrorEnv : public EnvWrapper { + public: + bool writable_file_error_; + int num_writable_file_errors_; + + ErrorEnv() : EnvWrapper(Env::Default()), + writable_file_error_(false), + num_writable_file_errors_(0) { } + + virtual Status NewWritableFile(const std::string& fname, + WritableFile** result) { + if (writable_file_error_) { + ++num_writable_file_errors_; + *result = NULL; + return Status::IOError(fname, "fake error"); + } + return target()->NewWritableFile(fname, result); + } +}; + +} // namespace test +} // namespace leveldb + +#endif // STORAGE_LEVELDB_UTIL_TESTUTIL_H_ diff --git a/src/miner/miner.cpp b/src/miner/miner.cpp new file mode 100644 index 00000000..a9f4ea75 --- /dev/null +++ b/src/miner/miner.cpp @@ -0,0 +1,1202 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "miner/miner.h" + +#include "structs/amount.h" +#include "primitives/block.h" +#include "primitives/transaction.h" +#include "structs/hash.h" +#include "core/main.h" +#include "net/net.h" +#include "chain/pow.h" +#include "utils/timedata.h" +#include "utils/util.h" +#include "utils/utilmoneystr.h" +#ifdef ENABLE_WALLET +#include "wallet/wallet.h" +#endif + +#include "multichain/multichain.h" + +#include +#include + +using namespace std; + +////////////////////////////////////////////////////////////////////////////// +// +// BitcoinMiner +// + +// +// Unconfirmed transactions in the memory pool often depend on other +// transactions in the memory pool. When we select transactions from the +// pool, we select by highest priority or fee rate, so we might consider +// transactions that depend on transactions that aren't yet in the block. +// The COrphan class keeps track of these 'temporary orphans' while +// CreateBlock is figuring out which transactions to include. +// +class COrphan +{ +public: + const CTransaction* ptx; + set setDependsOn; + CFeeRate feeRate; + double dPriority; + + COrphan(const CTransaction* ptxIn) : ptx(ptxIn), feeRate(0), dPriority(0) + { + } +}; + +uint64_t nLastBlockTx = 0; +uint64_t nLastBlockSize = 0; + +// We want to sort transactions by priority and fee rate, so: +typedef boost::tuple TxPriority; +class TxPriorityCompare +{ + bool byFee; + +public: + TxPriorityCompare(bool _byFee) : byFee(_byFee) { } + + bool operator()(const TxPriority& a, const TxPriority& b) + { + if (byFee) + { + if (a.get<1>() == b.get<1>()) + return a.get<0>() < b.get<0>(); + return a.get<1>() < b.get<1>(); + } + else + { + if (a.get<0>() == b.get<0>()) + return a.get<1>() < b.get<1>(); + return a.get<0>() < b.get<0>(); + } + } +}; + +bool UpdateTime(CBlockHeader* pblock, const CBlockIndex* pindexPrev) +{ +/* MCHN START */ + uint32_t original_nTime=pblock->nTime; + uint32_t original_nBits=pblock->nBits; +/* MCHN END */ + pblock->nTime = std::max(pindexPrev->GetMedianTimePast()+1, GetAdjustedTime()); + + // Updating time can change work required on testnet: + if (Params().AllowMinDifficultyBlocks()) + pblock->nBits = GetNextWorkRequired(pindexPrev, pblock); +/* MCHN START */ + if((original_nTime != pblock->nTime) || (original_nBits != pblock->nBits)) + { + return true; + } + return false; +/* MCHN END */ +} + +/* MCHN START */ + +bool CreateBlockSignature(CBlock *block,uint32_t hash_type,CWallet *pwallet) +{ + int coinbase_tx,op_return_output; + uint256 hash_to_verify; + uint256 original_merkle_root; + std::vector vchSigOut; + std::vector vchPubKey; + + block->nMerkleTreeType=MERKLETREE_FULL; + block->nSigHashType=BLOCKSIGHASH_NONE; + + if(!mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + block->hashMerkleRoot=block->BuildMerkleTree(); + return true; + } + + if(block->vSigner[0] == 0) + { + return false; + } + + coinbase_tx=-1; + op_return_output=-1; + for (unsigned int i = 0; i < block->vtx.size(); i++) + { + if(coinbase_tx<0) + { + const CTransaction &tx = block->vtx[i]; + if (block->vtx[i].IsCoinBase()) + { + coinbase_tx=i; + for (unsigned int j = 0; j < tx.vout.size(); j++) + { + + const CScript& script1 = tx.vout[j].scriptPubKey; + if(script1.IsUnspendable()) + { + op_return_output=j; + } + } + } + } + } + + if(coinbase_tx<0) + { + block->nSigHashType=BLOCKSIGHASH_INVALID; + return false; + } + + if((hash_type == BLOCKSIGHASH_HEADER) && (op_return_output >= 0)) + { + block->nSigHashType=BLOCKSIGHASH_INVALID; + return false; + } +// if(op_return_output >= 0) + { + CMutableTransaction tx=block->vtx[coinbase_tx]; + tx.vout.clear(); + for(int i=0;i<(int)block->vtx[coinbase_tx].vout.size();i++) + { + if((i != op_return_output) && + ((block->vtx[coinbase_tx].vout[i].nValue != 0) || (mc_gState->m_Permissions->m_Block == 0))) + { + tx.vout.push_back(block->vtx[coinbase_tx].vout[i]); + } + } + block->vtx[coinbase_tx]=tx; + } + + switch(hash_type) + { + case BLOCKSIGHASH_HEADER: + block->nMerkleTreeType=MERKLETREE_NO_COINBASE_OP_RETURN; + block->nSigHashType=BLOCKSIGHASH_HEADER; + hash_to_verify=block->GetHash(); + break; + case BLOCKSIGHASH_NO_SIGNATURE_AND_NONCE: + block->nMerkleTreeType=MERKLETREE_NO_COINBASE_OP_RETURN; + block->hashMerkleRoot=block->BuildMerkleTree(); + block->nNonce=0; + hash_to_verify=block->GetHash(); + block->nMerkleTreeType=MERKLETREE_FULL; + break; + default: + block->nSigHashType=BLOCKSIGHASH_INVALID; + return false; + } + + CMutableTransaction tx=block->vtx[coinbase_tx]; + tx.vout.clear(); + for(int i=0;i<(int)block->vtx[coinbase_tx].vout.size();i++) + { + tx.vout.push_back(block->vtx[coinbase_tx].vout[i]); + } + + CTxOut txOut; + + txOut.nValue = 0; + txOut.scriptPubKey = CScript() << OP_RETURN; + + size_t elem_size; + const unsigned char *elem; + + vchPubKey=std::vector (block->vSigner+1, block->vSigner+1+block->vSigner[0]); + + CPubKey pubKeyOut(vchPubKey); + CKey key; + if(!pwallet->GetKey(pubKeyOut.GetID(), key)) + { + return false; + } + + vector vchSig; + key.Sign(hash_to_verify, vchSig); + + mc_Script *lpScript; + lpScript=new mc_Script; + + lpScript->SetBlockSignature(vchSig.data(),vchSig.size(),hash_type,block->vSigner+1,block->vSigner[0]); + + for(int element=0;element < lpScript->GetNumElements();element++) + { + elem = lpScript->GetData(element,&elem_size); + if(elem) + { + txOut.scriptPubKey << vector(elem, elem + elem_size); + } + } + delete lpScript; + + tx.vout.push_back(txOut); + + block->vtx[coinbase_tx]=tx; + + switch(hash_type) + { + case BLOCKSIGHASH_NO_SIGNATURE_AND_NONCE: + block->hashMerkleRoot=block->BuildMerkleTree(); + break; + } + + return true; +} + +/* MCHN END */ + + +/* MCHN START */ +//CBlockTemplate* CreateNewBlock(const CScript& scriptPubKeyIn) +CBlockTemplate* CreateNewBlock(const CScript& scriptPubKeyIn,CWallet *pwallet,CPubKey *ppubkey,int *canMine) +/* MCHN END */ +{ + // Create new block + auto_ptr pblocktemplate(new CBlockTemplate()); + if(!pblocktemplate.get()) + return NULL; + CBlock *pblock = &pblocktemplate->block; // pointer for convenience + // -regtest only: allow overriding block.nVersion with + // -blockversion=N to test forking scenarios + if (Params().MineBlocksOnDemand()) + pblock->nVersion = GetArg("-blockversion", pblock->nVersion); + + // Create coinbase tx + CMutableTransaction txNew; + txNew.vin.resize(1); + txNew.vin[0].prevout.SetNull(); + +/* MCHN START */ + + txNew.vout.resize(1); + + int prevCanMine=MC_PTP_MINE; + if(canMine) + { + prevCanMine=*canMine; + } + +/* MCHN END */ + + txNew.vout[0].scriptPubKey = scriptPubKeyIn; + + // Add dummy coinbase tx as first transaction + pblock->vtx.push_back(CTransaction()); + pblocktemplate->vTxFees.push_back(-1); // updated at end + pblocktemplate->vTxSigOps.push_back(-1); // updated at end + + // Largest block you're willing to create: + unsigned int nBlockMaxSize = GetArg("-blockmaxsize", DEFAULT_BLOCK_MAX_SIZE); + // Limit to betweeen 1K and MAX_BLOCK_SIZE-1K for sanity: + nBlockMaxSize = std::max((unsigned int)1000, std::min((unsigned int)(MAX_BLOCK_SIZE-1000), nBlockMaxSize)); + + // How much of the block should be dedicated to high-priority transactions, + // included regardless of the fees they pay + unsigned int nBlockPrioritySize = GetArg("-blockprioritysize", DEFAULT_BLOCK_PRIORITY_SIZE); + nBlockPrioritySize = std::min(nBlockMaxSize, nBlockPrioritySize); + + // Minimum block size you want to create; block will be filled with free transactions + // until there are no more or the block reaches this size: + unsigned int nBlockMinSize = GetArg("-blockminsize", DEFAULT_BLOCK_MIN_SIZE); + nBlockMinSize = std::min(nBlockMaxSize, nBlockMinSize); + + // Collect memory pool transactions into the block + CAmount nFees = 0; + + { + LOCK2(cs_main, mempool.cs); + + CBlockIndex* pindexPrev = chainActive.Tip(); + const int nHeight = pindexPrev->nHeight + 1; + CCoinsViewCache view(pcoinsTip); + + // Priority order to process transactions + list vOrphan; // list memory doesn't move + map > mapDependers; + bool fPrintPriority = GetBoolArg("-printpriority", false); + + // This vector will be sorted into a priority queue: + vector vecPriority; + vecPriority.reserve(mempool.mapTx.size()); +/* MCHN START */ +// mempool records are processed in the order they were accepted + + set setAdded; +/* + for (map::iterator mi = mempool.mapTx.begin(); + mi != mempool.mapTx.end(); ++mi) + { + const CTransaction& tx = mi->second.GetTx(); + */ + + double orderPriority=mempool.mapTx.size(); + + mempool.defragmentHashList(); + for(int pos=0;posm_Count;pos++) + { + uint256 hash; + hash=*(uint256*)mempool.hashList->GetRow(pos); + + if(!mempool.exists(hash)) + { + LogPrint("mchn","mchn: Miner: Tx not found in the mempool: %s\n",hash.GetHex().c_str()); + continue; + } + + const CTransaction& tx = mempool.mapTx[hash].GetTx(); +/* MCHN END */ + + if (tx.IsCoinBase() || !IsFinalTx(tx, nHeight)) + { + LogPrint("mchn","mchn: Miner: Coinbase or not final tx found: %s\n",tx.GetHash().GetHex().c_str()); + continue; + } + + COrphan* porphan = NULL; + double dPriority = 0; + CAmount nTotalIn = 0; + bool fMissingInputs = false; + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + // Read prev transaction + if (!view.HaveCoins(txin.prevout.hash)) + { + // This should never happen; all transactions in the memory + // pool should connect to either transactions in the chain + // or other transactions in the memory pool. + if (!mempool.mapTx.count(txin.prevout.hash)) + { + LogPrintf("ERROR: mempool transaction missing input\n"); + if (fDebug) assert("mempool transaction missing input" == 0); + fMissingInputs = true; + if (porphan) + vOrphan.pop_back(); + break; + } + + // Has to wait for dependencies +/* MCHN START */ + if(setAdded.count(txin.prevout.hash) == 0) + { +/* MCHN END */ + if (!porphan) + { + // Use list for automatic deletion + vOrphan.push_back(COrphan(&tx)); + porphan = &vOrphan.back(); + } + mapDependers[txin.prevout.hash].push_back(porphan); + porphan->setDependsOn.insert(txin.prevout.hash); +/* MCHN START */ + } +/* MCHN END */ + nTotalIn += mempool.mapTx[txin.prevout.hash].GetTx().vout[txin.prevout.n].nValue; + continue; + } + const CCoins* coins = view.AccessCoins(txin.prevout.hash); + assert(coins); + + CAmount nValueIn = coins->vout[txin.prevout.n].nValue; + nTotalIn += nValueIn; + + int nConf = nHeight - coins->nHeight; + + dPriority += (double)nValueIn * nConf; + } + if (fMissingInputs) + { + LogPrint("mchn","mchn: Miner: Missing inputs for %s\n",tx.GetHash().GetHex().c_str()); + continue; + } + + // Priority is sum(valuein * age) / modified_txsize + unsigned int nTxSize = ::GetSerializeSize(tx, SER_NETWORK, PROTOCOL_VERSION); + dPriority = tx.ComputePriority(dPriority, nTxSize); + +/* MCHN START */ +// Priority ignored - txs are processed in the order they were accepted + dPriority=orderPriority; + orderPriority-=1.; + +// uint256 hash = tx.GetHash(); +/* MCHN END */ + mempool.ApplyDeltas(hash, dPriority, nTotalIn); + + CFeeRate feeRate(nTotalIn-tx.GetValueOut(), nTxSize); + + if (porphan) + { + LogPrint("mchn","mchn: Miner: Orphan %s\n",tx.GetHash().GetHex().c_str()); + porphan->dPriority = dPriority; + porphan->feeRate = feeRate; + } + else +/* MCHN START */ + { + setAdded.insert(tx.GetHash()); + vecPriority.push_back(TxPriority(dPriority, feeRate, &tx)); + } +// vecPriority.push_back(TxPriority(dPriority, feeRate, &mi->second.GetTx())); +/* MCHN END */ + } + + // Collect transactions into block + uint64_t nBlockSize = 1000; + uint64_t nBlockTx = 0; + int nBlockSigOps = 100; +// bool fSortedByFee = (nBlockPrioritySize <= 0); + +/* MCHN START */ + TxPriorityCompare comparer(false); +// TxPriorityCompare comparer(fSortedByFee); + bool overblocksize_logged=false; +/* MCHN END */ + std::make_heap(vecPriority.begin(), vecPriority.end(), comparer); + + while (!vecPriority.empty()) + { + // Take highest priority transaction off the priority queue: + double dPriority = vecPriority.front().get<0>(); + CFeeRate feeRate = vecPriority.front().get<1>(); + const CTransaction& tx = *(vecPriority.front().get<2>()); + + std::pop_heap(vecPriority.begin(), vecPriority.end(), comparer); + vecPriority.pop_back(); + + // Size limits + unsigned int nTxSize = ::GetSerializeSize(tx, SER_NETWORK, PROTOCOL_VERSION); + if (nBlockSize + nTxSize >= nBlockMaxSize) + { + if(!overblocksize_logged) + { + overblocksize_logged=true; + LogPrint("mchn","mchn: Miner: Over block size: %s\n",tx.GetHash().GetHex().c_str()); + } + continue; + } + // Legacy limits on sigOps: + unsigned int nTxSigOps = GetLegacySigOpCount(tx); + if (nBlockSigOps + nTxSigOps >= MAX_BLOCK_SIGOPS) + { + LogPrint("mchn","mchn: Miner: Over sigop count 1: %s\n",tx.GetHash().GetHex().c_str()); + continue; + } + + // Skip free transactions if we're past the minimum block size: + const uint256& hash = tx.GetHash(); + double dPriorityDelta = 0; + CAmount nFeeDelta = 0; + mempool.ApplyDeltas(hash, dPriorityDelta, nFeeDelta); +/* MCHN + if (fSortedByFee && (dPriorityDelta <= 0) && (nFeeDelta <= 0) && (feeRate < ::minRelayTxFee) && (nBlockSize + nTxSize >= nBlockMinSize)) + continue; + + // Prioritise by fee once past the priority size or we run out of high-priority + // transactions: + if (!fSortedByFee && + ((nBlockSize + nTxSize >= nBlockPrioritySize) || !AllowFree(dPriority))) + { + fSortedByFee = true; + comparer = TxPriorityCompare(fSortedByFee); + std::make_heap(vecPriority.begin(), vecPriority.end(), comparer); + } +*/ + if (!view.HaveInputs(tx)) + { + LogPrint("mchn","mchn: Miner: No inputs for %s\n",tx.GetHash().GetHex().c_str()); + continue; + } + + CAmount nTxFees = view.GetValueIn(tx)-tx.GetValueOut(); + + nTxSigOps += GetP2SHSigOpCount(tx, view); + if (nBlockSigOps + nTxSigOps >= MAX_BLOCK_SIGOPS) + { + LogPrint("mchn","mchn: Miner: Over sigop count 2: %s\n",tx.GetHash().GetHex().c_str()); + continue; + } + + // Note that flags: we don't want to set mempool/IsStandard() + // policy here, but we still have to ensure that the block we + // create only contains transactions that are valid in new blocks. + CValidationState state; + +/* MCHN START */ +// if (!CheckInputs(tx, state, view, true, MANDATORY_SCRIPT_VERIFY_FLAGS, true))// May fail if send permission was lost + if (!CheckInputs(tx, state, view, false, 0, true)) +/* MCHN END */ + { + LogPrint("mchn","mchn: Miner: CheckInput failure %s\n",tx.GetHash().GetHex().c_str()); + continue; + } + + CTxUndo txundo; + UpdateCoins(tx, state, view, txundo, nHeight); + + // Added + pblock->vtx.push_back(tx); + pblocktemplate->vTxFees.push_back(nTxFees); + pblocktemplate->vTxSigOps.push_back(nTxSigOps); + nBlockSize += nTxSize; + ++nBlockTx; + nBlockSigOps += nTxSigOps; + nFees += nTxFees; + + if (fPrintPriority) + { + LogPrintf("priority %.1f fee %s txid %s\n", + dPriority, feeRate.ToString(), tx.GetHash().ToString()); + } + + // Add transactions that depend on this one to the priority queue + if (mapDependers.count(hash)) + { + BOOST_FOREACH(COrphan* porphan, mapDependers[hash]) + { + if (!porphan->setDependsOn.empty()) + { + porphan->setDependsOn.erase(hash); + if (porphan->setDependsOn.empty()) + { + vecPriority.push_back(TxPriority(porphan->dPriority, porphan->feeRate, porphan->ptx)); + std::push_heap(vecPriority.begin(), vecPriority.end(), comparer); + } + } + } + } + } + + nLastBlockTx = nBlockTx; + nLastBlockSize = nBlockSize; + +/* MCHN START */ +// If block was dropped, this happens too many times +// LogPrintf("CreateNewBlock(): total size %u\n", nBlockSize); +/* MCHN END */ + + // Compute final coinbase transaction. + txNew.vout[0].nValue = GetBlockValue(nHeight, nFees); + txNew.vin[0].scriptSig = CScript() << nHeight << OP_0; + + pblock->vSigner[0]=ppubkey->size(); + memcpy(pblock->vSigner+1,ppubkey->begin(),pblock->vSigner[0]); + + pblock->vtx[0] = txNew; + pblocktemplate->vTxFees[0] = -nFees; + + + // Fill in header + pblock->hashPrevBlock = pindexPrev->GetBlockHash(); + UpdateTime(pblock, pindexPrev); + pblock->nBits = GetNextWorkRequired(pindexPrev, pblock); + pblock->nNonce = 0; + pblocktemplate->vTxSigOps[0] = GetLegacySigOpCount(pblock->vtx[0]); + +/* MCHN START */ + bool testValidity=true; + +// If this node cannot mine for some reason (permission or diversity, block is not tested for validity to avoid exception + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(canMine) + { + const unsigned char *pubkey_hash=(unsigned char *)Hash160(ppubkey->begin(),ppubkey->end()).begin(); + *canMine=mc_gState->m_Permissions->CanMine(NULL,pubkey_hash); + if((*canMine & MC_PTP_MINE) == 0) + { + if(prevCanMine & MC_PTP_MINE) + { + LogPrintf("mchn: MultiChainMiner: cannot mine now, waiting...\n"); + } + testValidity=false; + } + else + { + if((prevCanMine & MC_PTP_MINE) == 0) + { + LogPrintf("CreateNewBlock(): total size %u\n", nBlockSize); + LogPrintf("mchn: MultiChainMiner: Starting mining...\n"); + } + } + } + } + else + { + LogPrintf("CreateNewBlock(): total size %u\n", nBlockSize); + } + + + if(testValidity) + { +/* MCHN END */ + + CValidationState state; + if (!TestBlockValidity(state, *pblock, pindexPrev, false, false)) + throw std::runtime_error("CreateNewBlock() : TestBlockValidity failed"); + +/* MCHN START */ + } +/* MCHN END */ + } + + return pblocktemplate.release(); +} + +/* MCHN START */ +CBlockTemplate* CreateNewBlock(const CScript& scriptPubKeyIn) +{ + return CreateNewBlock(scriptPubKeyIn,NULL,NULL,NULL); +} +/* MCHN END */ + + +void IncrementExtraNonce(CBlock* pblock, CBlockIndex* pindexPrev, unsigned int& nExtraNonce,CWallet *pwallet) +{ + // Update nExtraNonce + static uint256 hashPrevBlock; + if (hashPrevBlock != pblock->hashPrevBlock) + { + nExtraNonce = 0; + hashPrevBlock = pblock->hashPrevBlock; + } + ++nExtraNonce; + unsigned int nHeight = pindexPrev->nHeight+1; // Height first in coinbase required for block.version=2 + CMutableTransaction txCoinbase(pblock->vtx[0]); + txCoinbase.vin[0].scriptSig = (CScript() << nHeight << CScriptNum(nExtraNonce)) + COINBASE_FLAGS; + assert(txCoinbase.vin[0].scriptSig.size() <= 100); + + pblock->vtx[0] = txCoinbase; + +/* MCHN START */ + CreateBlockSignature(pblock,BLOCKSIGHASH_NO_SIGNATURE_AND_NONCE,pwallet); + pblock->hashMerkleRoot = pblock->BuildMerkleTree(); +/* MCHN END */ +} + +#ifdef ENABLE_WALLET +////////////////////////////////////////////////////////////////////////////// +// +// Internal miner +// +double dHashesPerSec = 0.0; +int64_t nHPSTimerStart = 0; + +// +// ScanHash scans nonces looking for a hash with at least some zero bits. +// The nonce is usually preserved between calls, but periodically or if the +// nonce is 0xffff0000 or above, the block is rebuilt and nNonce starts over at +// zero. +// +bool static ScanHash(const CBlockHeader *pblock, uint32_t& nNonce, uint256 *phash) +{ + // Write the first 76 bytes of the block header to a double-SHA256 state. + CHash256 hasher; + CDataStream ss(SER_NETWORK, PROTOCOL_VERSION); + ss << *pblock; + assert(ss.size() == 80); + hasher.Write((unsigned char*)&ss[0], 76); + + while (true) { + nNonce++; + + // Write the last 4 bytes of the block header (the nonce) to a copy of + // the double-SHA256 state, and compute the result. + CHash256(hasher).Write((unsigned char*)&nNonce, 4).Finalize((unsigned char*)phash); + + // Return the nonce if the hash has at least some zero bits, + // caller will check if it has enough to reach the target + if (((uint16_t*)phash)[15] == 0) + return true; + + // If nothing found after trying for a while, return -1 + if ((nNonce & 0xffff) == 0) +// if ((nNonce & 0xff) == 0) + return false; + if ((nNonce & 0xfff) == 0) + boost::this_thread::interruption_point(); + } +} + +CBlockTemplate* CreateNewBlockWithKey(CReserveKey& reservekey) +{ + CPubKey pubkey; + if (!reservekey.GetReservedKey(pubkey)) + return NULL; + + CScript scriptPubKey = CScript() << ToByteVector(pubkey) << OP_CHECKSIG; + return CreateNewBlock(scriptPubKey); +} + +/* MCHN START */ +// Block should be mined for specific keys, not just any from pool + + +CBlockTemplate* CreateNewBlockWithDefaultKey(CWallet *pwallet,int *canMine) +{ + CPubKey pubkey; + bool key_found; + + { + LOCK(cs_main); + key_found=pwallet->GetKeyFromAddressBook(pubkey,MC_PTP_MINE); + } + if(!key_found) + { + if(canMine) + { + if(*canMine & MC_PTP_MINE) + { + *canMine=0; + LogPrintf("mchn: Cannot find address having mining permission\n"); + } + } + return NULL; + } + + const unsigned char *pubkey_hash=(unsigned char *)Hash160(pubkey.begin(),pubkey.end()).begin(); + + CScript scriptPubKey = CScript() << OP_DUP << OP_HASH160 << vector(pubkey_hash, pubkey_hash + 20) << OP_EQUALVERIFY << OP_CHECKSIG; + + return CreateNewBlock(scriptPubKey,pwallet,&pubkey,canMine); +} + +/* MCHN END */ + +bool ProcessBlockFound(CBlock* pblock, CWallet& wallet, CReserveKey& reservekey) +{ + LogPrintf("%s\n", pblock->ToString()); + LogPrintf("generated %s\n", FormatMoney(pblock->vtx[0].vout[0].nValue)); + + // Found a solution + { + LOCK(cs_main); + if(mc_gState->m_NodePausedState & MC_NPS_MINING) + { + return error("MultiChainMiner : mining is paused, generated block is dropped"); + } + if (pblock->hashPrevBlock != chainActive.Tip()->GetBlockHash()) + { + return error("MultiChainMiner : generated block is stale"); + } + } + + // Remove key from key pool + reservekey.KeepKey(); + + // Track how many getdata requests this block gets + { + LOCK(wallet.cs_wallet); + wallet.mapRequestCount[pblock->GetHash()] = 0; + } + + // Process this block the same as if we had received it from another node + CValidationState state; + if (!ProcessNewBlock(state, NULL, pblock)) + return error("MultiChainMiner : ProcessNewBlock, block not accepted"); + + return true; +} + +void static BitcoinMiner(CWallet *pwallet) +{ + LogPrintf("MultiChainMiner started\n"); + SetThreadPriority(THREAD_PRIORITY_LOWEST); + RenameThread("bitcoin-miner"); + + // Each thread has its own key and counter + CReserveKey reservekey(pwallet); + unsigned int nExtraNonce = 0; + +/* MCHN START */ + int canMine; + int prevCanMine; + canMine=MC_PTP_MINE; + prevCanMine=canMine; + + double AverageTime=0; + double dHashesTarget=0; + double wHashesPerSec=0; + uint64_t ulTarget=0; + int wSize=10; + int wPos=0; + uint64_t wCount[10]; + uint64_t wTime[10]; + memset(wCount,0,wSize*sizeof(uint64_t)); + memset(wTime,0,wSize*sizeof(uint64_t)); +/* + uint32_t tExpectedLastBlock=0; + uint32_t tExpectedMedian=0; + uint32_t tExpected=0; + */ + uint32_t ExpectedInterval=mc_RandomInRange((int)(0.5*(Params().TargetSpacing())),(int)(1.5*(Params().TargetSpacing()))); + int ExpectedBlock=mc_gState->m_Permissions->m_Block; +/* MCHN END */ + + + try { + while (true) { +/* MCHN START */ + bool not_setup_period=true; + + if(mc_gState->m_NodePausedState & MC_NPS_MINING) + { + __US_Sleep(1000); + boost::this_thread::interruption_point(); + } + + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if((canMine & MC_PTP_MINE) == 0) + { + __US_Sleep(1000); + boost::this_thread::interruption_point(); + } + if(mc_gState->m_Permissions->IsSetupPeriod()) + { + not_setup_period=false; + } + } + if (Params().MiningRequiresPeers() + && not_setup_period + && ( (mc_gState->m_Permissions->GetMinerCount() > 1) + || (mc_gState->m_NetworkParams->GetInt64Param("anyonecanmine") != 0) + || (mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + ) + ) { + + // Busy-wait for the network to come online so we don't waste time mining + // on an obsolete chain. In regtest mode we expect to fly solo. + + bool wait_for_peers=true; + if(wait_for_peers) + { + int active_nodes=0; + + while ((active_nodes == 0) && (mc_gState->m_Permissions->GetMinerCount() > 1)) + { + vector vNodesCopy = vNodes; + BOOST_FOREACH(CNode* pnode, vNodesCopy) + { + if(pnode->fSuccessfullyConnected) + { + active_nodes++; + } + } + + if(active_nodes == 0) + MilliSleep(1000); + } + } + } + + // + // Create new block + // + unsigned int nTransactionsUpdatedLast = mempool.GetTransactionsUpdated(); + CBlockIndex* pindexPrev = chainActive.Tip(); +/* MCHN START */ + +// uint32_t ExpectedTime=Params().TargetSpacing();//mc_RandomInRange(0,Params().TargetSpacing()*2); + if(mc_gState->m_Permissions->m_Block != ExpectedBlock) + { + ExpectedInterval=Params().TargetSpacing(); + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + ExpectedInterval=4; +// ExpectedInterval/=2; + } + + ExpectedInterval=mc_RandomInRange((int)(0.5*ExpectedInterval),(int)(1.5*ExpectedInterval)); + ExpectedBlock=mc_gState->m_Permissions->m_Block; + } + uint32_t ExpectedTime=ExpectedInterval; +// auto_ptr pblocktemplate(CreateNewBlockWithKey(reservekey)); + canMine=prevCanMine; + auto_ptr pblocktemplate(CreateNewBlockWithDefaultKey(pwallet,&canMine)); + prevCanMine=canMine; + if (!pblocktemplate.get()) + { + canMine=0; + } + else + { +// if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + CBlock *pblock = &pblocktemplate->block; + if(pblock->GetBlockTime()-pindexPrev->GetBlockTime() < ExpectedTime) + { + ExpectedTime-=(pblock->GetBlockTime()-pindexPrev->GetBlockTime()); + } + else + { + ExpectedTime=0; + } + ExpectedTime+=mc_TimeNowAsUInt(); + +// tExpectedLastBlock=ExpectedTime; + + int past_blocks=24; + int found_blocks=0; + CBlockIndex* pindexPast=pindexPrev; + uint64_t avTime=0; + for(int i=0;iGetBlockTime(); + found_blocks++; + pindexPast=pindexPast->pprev; + } + } + if(found_blocks) + { + uint64_t exTime=avTime/found_blocks+(found_blocks-1)*Params().TargetSpacing()/2+ExpectedInterval; +// tExpectedMedian=exTime; + if(exTime0) + { + wTotalCount+=wCount[i]; + wTotalTime+=wTime[i]; + } + else + { + take_it=false; + } + } + if(wTotalTime>0) + { + wHashesPerSec=1000*(double)wTotalCount/(double)wTotalTime; + } + if(take_it) + { + uint256 hashRequired = uint256().SetCompact(pblock->nBits); + memcpy(&ulTarget,(unsigned char*)&hashRequired+26,6); + dHashesTarget=0xffffffffffff/(double)ulTarget; + AverageTime=dHashesTarget/wHashesPerSec; + } + AverageTime/=mc_gState->m_Permissions->GetActiveMinerCount(); + if(mc_TimeNowAsUInt()+AverageTimem_Permissions->m_Block > 1) + { + canMine=0; + } + } + } + } + + if(mc_gState->m_NodePausedState & MC_NPS_MINING) + { + canMine=0; + } + + if(fReindex) + { + canMine=0; + } + + if(canMine & MC_PTP_MINE) + { +/* MCHN END */ + if (!pblocktemplate.get()) + { + LogPrintf("Error in MultiChainMiner: Keypool ran out, please call keypoolrefill before restarting the mining thread\n"); + return; + } + CBlock *pblock = &pblocktemplate->block; + IncrementExtraNonce(pblock, pindexPrev, nExtraNonce,pwallet); + + LogPrintf("Running MultiChainMiner with %u transactions in block (%u bytes)\n", pblock->vtx.size(), + ::GetSerializeSize(*pblock, SER_NETWORK, PROTOCOL_VERSION)); + + // + // Search + // + int64_t nStart = GetTime(); + uint256 hashTarget = uint256().SetCompact(pblock->nBits); + uint256 hash; + uint32_t nNonce = 0; + uint32_t nOldNonce = 0; + + uint64_t wStartTime=GetTimeMillis(); + uint64_t wThisCount=0; + while (true) { + bool fFound = ScanHash(pblock, nNonce, &hash); + uint32_t nHashesDone = nNonce - nOldNonce; + nOldNonce = nNonce; + + wThisCount+=nHashesDone; + + // Check if something found + if (fFound) + { + if (hash <= hashTarget) + { + // Found a solution + pblock->nNonce = nNonce; + assert(hash == pblock->GetHash()); + + SetThreadPriority(THREAD_PRIORITY_NORMAL); + + LogPrint("mchn","mchn: MultiChainMiner: Block Found - %s, height: %d\n",hash.GetHex(),mc_gState->m_Permissions->m_Block+1); + LogPrintf("MultiChainMiner:\n"); + LogPrintf("proof-of-work found \n hash: %s \ntarget: %s\n", hash.GetHex(), hashTarget.GetHex()); + +/* MCHN START */ + if(!ProcessBlockFound(pblock, *pwallet, reservekey)) + { + __US_Sleep(1000); + } +/* MCHN END */ + + SetThreadPriority(THREAD_PRIORITY_LOWEST); + + // In regression test mode, stop mining after a block is found. + if (Params().MineBlocksOnDemand()) + throw boost::thread_interrupted(); + + break; + } + } + + // Meter hashes/sec + static int64_t nHashCounter; + if (nHPSTimerStart == 0) + { + nHPSTimerStart = GetTimeMillis(); + nHashCounter = 0; + } + else + nHashCounter += nHashesDone; + if (GetTimeMillis() - nHPSTimerStart > 4000) + { + static CCriticalSection cs; + { + LOCK(cs); + if (GetTimeMillis() - nHPSTimerStart > 4000) + { + dHashesPerSec = 1000.0 * nHashCounter / (GetTimeMillis() - nHPSTimerStart); + nHPSTimerStart = GetTimeMillis(); + nHashCounter = 0; + static int64_t nLogTime; + if (GetTime() - nLogTime > 30 * 60) + { + nLogTime = GetTime(); + LogPrintf("hashmeter %6.0f khash/s\n", dHashesPerSec/1000.0); + } + } + } + } + + // Check for stop or if block needs to be rebuilt + boost::this_thread::interruption_point(); + // Regtest mode doesn't require peers + if (vNodes.empty() && Params().MiningRequiresPeers() + && not_setup_period + && ( (mc_gState->m_Permissions->GetMinerCount() > 1) + || (mc_gState->m_NetworkParams->GetInt64Param("anyonecanmine") != 0) + || (mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + ) + ) + { +// if (vNodes.empty() && Params().MiningRequiresPeers() && not_setup_period) + break; + } + if (nNonce >= 0xffff0000) + break; + if (mempool.GetTransactionsUpdated() != nTransactionsUpdatedLast && GetTime() - nStart > 60) + break; + if (pindexPrev != chainActive.Tip()) + break; + + // Update nTime every few seconds + + if(UpdateTime(pblock, pindexPrev)) + { +/* MCHN START */ + CreateBlockSignature(pblock,BLOCKSIGHASH_NO_SIGNATURE_AND_NONCE,pwallet); +/* MCHN END */ + } + if (Params().AllowMinDifficultyBlocks()) + { + // Changing pblock->nTime can change work required on testnet: + hashTarget.SetCompact(pblock->nBits); + } + } +/* MCHN START */ + uint64_t wTimeNow=GetTimeMillis(); + if(wTimeNow>wStartTime+100) + { + wCount[wPos]=wThisCount; + wTime[wPos]=wTimeNow-wStartTime; + wPos=(wPos+1)%wSize; + } + + } + else + { + __US_Sleep(100); + } +/* MCHN END */ + } + } + catch (boost::thread_interrupted) + { + LogPrintf("MultiChainMiner terminated\n"); + throw; + } +} + +void GenerateBitcoins(bool fGenerate, CWallet* pwallet, int nThreads) +{ + static boost::thread_group* minerThreads = NULL; + + if (nThreads < 0) { + // In regtest threads defaults to 1 + if (Params().DefaultMinerThreads()) + nThreads = Params().DefaultMinerThreads(); + else + nThreads = boost::thread::hardware_concurrency(); + } + + if (minerThreads != NULL) + { + minerThreads->interrupt_all(); +/* MCHN START */ + minerThreads->join_all(); +/* MCHN END */ + + delete minerThreads; + minerThreads = NULL; + } + + if (nThreads == 0 || !fGenerate) + return; + + minerThreads = new boost::thread_group(); + for (int i = 0; i < nThreads; i++) + minerThreads->create_thread(boost::bind(&BitcoinMiner, pwallet)); +} + +#endif // ENABLE_WALLET diff --git a/src/miner/miner.h b/src/miner/miner.h new file mode 100644 index 00000000..db5406d8 --- /dev/null +++ b/src/miner/miner.h @@ -0,0 +1,38 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2013 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_MINER_H +#define BITCOIN_MINER_H + +#include + +class CBlock; +class CBlockHeader; +class CBlockIndex; +class CReserveKey; +class CScript; +class CWallet; + +struct CBlockTemplate; + +/** Run the miner threads */ +void GenerateBitcoins(bool fGenerate, CWallet* pwallet, int nThreads); +/** Generate a new block, without valid proof-of-work */ +CBlockTemplate* CreateNewBlock(const CScript& scriptPubKeyIn); +CBlockTemplate* CreateNewBlockWithKey(CReserveKey& reservekey); +/** Modify the extranonce in a block */ +/* MCHN START */ +void IncrementExtraNonce(CBlock* pblock, CBlockIndex* pindexPrev, unsigned int& nExtraNonce,CWallet *pwallet); +//void IncrementExtraNonce(CBlock* pblock, CBlockIndex* pindexPrev, unsigned int& nExtraNonce); +/* MCHN END */ +/** Check mined block */ +bool CheckWork(CBlock* pblock, CWallet& wallet, CReserveKey& reservekey); +bool UpdateTime(CBlockHeader* block, const CBlockIndex* pindexPrev); + +extern double dHashesPerSec; +extern int64_t nHPSTimerStart; + +#endif // BITCOIN_MINER_H diff --git a/src/multichain-cli-res.rc b/src/multichain-cli-res.rc new file mode 100644 index 00000000..173d3d13 --- /dev/null +++ b/src/multichain-cli-res.rc @@ -0,0 +1,35 @@ +#include // needed for VERSIONINFO +#include "version/clientversion.h" // holds the needed client version information + +#define VER_PRODUCTVERSION CLIENT_VERSION_MAJOR,CLIENT_VERSION_MINOR,CLIENT_VERSION_REVISION,CLIENT_VERSION_BUILD +#define VER_PRODUCTVERSION_STR STRINGIZE(CLIENT_VERSION_MAJOR) "." STRINGIZE(CLIENT_VERSION_MINOR) "." STRINGIZE(CLIENT_VERSION_REVISION) "." STRINGIZE(CLIENT_VERSION_BUILD) +#define VER_FILEVERSION VER_PRODUCTVERSION +#define VER_FILEVERSION_STR VER_PRODUCTVERSION_STR + +VS_VERSION_INFO VERSIONINFO +FILEVERSION VER_FILEVERSION +PRODUCTVERSION VER_PRODUCTVERSION +FILEOS VOS_NT_WINDOWS32 +FILETYPE VFT_APP +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "040904E4" // U.S. English - multilingual (hex) + BEGIN + VALUE "CompanyName", "Coin Sciences Ltd" + VALUE "FileDescription", "multichain-cli (OSS RPC client for MultiChain)" + VALUE "FileVersion", VER_FILEVERSION_STR + VALUE "InternalName", "multichain-cli" + VALUE "LegalCopyright", COPYRIGHT_STR + VALUE "LegalTrademarks1", "The software product under this license is provided free of charge. Full terms are shown at: http://www.multichain.com/terms-of-service/." + VALUE "OriginalFilename", "multichain-cli.exe" + VALUE "ProductName", "multichain-cli" + VALUE "ProductVersion", VER_PRODUCTVERSION_STR + END + END + + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x0, 1252 // language neutral - multilingual (decimal) + END +END diff --git a/src/multichain-util-res.rc b/src/multichain-util-res.rc new file mode 100644 index 00000000..09b285f0 --- /dev/null +++ b/src/multichain-util-res.rc @@ -0,0 +1,35 @@ +#include // needed for VERSIONINFO +#include "version/clientversion.h" // holds the needed client version information + +#define VER_PRODUCTVERSION CLIENT_VERSION_MAJOR,CLIENT_VERSION_MINOR,CLIENT_VERSION_REVISION,CLIENT_VERSION_BUILD +#define VER_PRODUCTVERSION_STR STRINGIZE(CLIENT_VERSION_MAJOR) "." STRINGIZE(CLIENT_VERSION_MINOR) "." STRINGIZE(CLIENT_VERSION_REVISION) "." STRINGIZE(CLIENT_VERSION_BUILD) +#define VER_FILEVERSION VER_PRODUCTVERSION +#define VER_FILEVERSION_STR VER_PRODUCTVERSION_STR + +VS_VERSION_INFO VERSIONINFO +FILEVERSION VER_FILEVERSION +PRODUCTVERSION VER_PRODUCTVERSION +FILEOS VOS_NT_WINDOWS32 +FILETYPE VFT_APP +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "040904E4" // U.S. English - multilingual (hex) + BEGIN + VALUE "CompanyName", "Coin Sciences Ltd" + VALUE "FileDescription", "multichain-util (Utilities for MultiChain)" + VALUE "FileVersion", VER_FILEVERSION_STR + VALUE "InternalName", "multichain-util" + VALUE "LegalCopyright", COPYRIGHT_STR + VALUE "LegalTrademarks1", "The software product under this license is provided free of charge. Full terms are shown at: http://www.multichain.com/terms-of-service/." + VALUE "OriginalFilename", "multichain-util.exe" + VALUE "ProductName", "multichain-util" + VALUE "ProductVersion", VER_PRODUCTVERSION_STR + END + END + + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x0, 1252 // language neutral - multilingual (decimal) + END +END diff --git a/src/multichain/multichain-cli.cpp b/src/multichain/multichain-cli.cpp new file mode 100644 index 00000000..bab8dd88 --- /dev/null +++ b/src/multichain/multichain-cli.cpp @@ -0,0 +1,497 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "chainparams/chainparamsbase.h" +#include "version/clientversion.h" +#include "rpc/rpcclient.h" +#include "rpc/rpcprotocol.h" +#include "utils/util.h" +#include "utils/utilstrencodings.h" + +/* MCHN START */ +#include "multichain/multichain.h" +#include "chainparams/globals.h" +/* MCHN END */ + +#include +//#include +#include + +#define _(x) std::string(x) /* Keep the _() around in case gettext or such will be used later to translate non-UI */ + +using namespace std; +using namespace boost; +using namespace boost::asio; +using namespace json_spirit; + +std::string HelpMessageCli() +{ + string strUsage; + strUsage += _("Options:") + "\n"; + strUsage += " -? " + _("This help message") + "\n"; + strUsage += " -conf= " + strprintf(_("Specify configuration file (default: %s)"), "multichain.conf") + "\n"; + strUsage += " -datadir= " + _("Specify data directory") + "\n"; +/* MCHN START */ + strUsage += " -saveclilog= " + _("If =0 multichain-cli history is not saved, default 1") + "\n"; +/* + strUsage += " -testnet " + _("Use the test network") + "\n"; + strUsage += " -regtest " + _("Enter regression test mode, which uses a special chain in which blocks can be " + "solved instantly. This is intended for regression testing tools and app development.") + "\n"; + */ +/* MCHN END */ + strUsage += " -rpcconnect= " + strprintf(_("Send commands to node running on (default: %s)"), "127.0.0.1") + "\n"; + strUsage += " -rpcport= " + _("Connect to JSON-RPC on ") + "\n"; + strUsage += " -rpcwait " + _("Wait for RPC server to start") + "\n"; + strUsage += " -rpcuser= " + _("Username for JSON-RPC connections") + "\n"; + strUsage += " -rpcpassword= " + _("Password for JSON-RPC connections") + "\n"; + + strUsage += "\n" + _("SSL options: ") + "\n"; + strUsage += " -rpcssl " + _("Use OpenSSL (https) for JSON-RPC connections") + "\n"; + + return strUsage; +} + +////////////////////////////////////////////////////////////////////////////// +// +// Start +// + +// +// Exception thrown on connection error. This error is used to determine +// when to wait if -rpcwait is given. +// +class CConnectionFailed : public std::runtime_error +{ +public: + + explicit inline CConnectionFailed(const std::string& msg) : + std::runtime_error(msg) + {} + +}; + +static bool AppInitRPC(int argc, char* argv[]) +{ + // + // Parameters + // + int err = MC_ERR_NOERROR; + int minargs=2; + +#ifndef WIN32 + minargs=1; +#endif + + // + // Parameters + // + // If Qt is used, parameters/bitcoin.conf are parsed in qt/bitcoin.cpp's main() + + mc_gState=new mc_State; + + mc_gState->m_Params->Parse(argc, argv); + + if(mc_gState->m_Params->NetworkName()) + { + if(strlen(mc_gState->m_Params->NetworkName()) > MC_PRM_NETWORK_NAME_MAX_SIZE) + { + fprintf(stderr, "ERROR: invalid chain name: %s\n",mc_gState->m_Params->NetworkName()); + return false; + } + } + +// ParseParameters(argc, argv); + if (mc_gState->m_Params->HasOption("-?") || + mc_gState->m_Params->HasOption("-help") || + mc_gState->m_Params->HasOption("-version") || + (mc_gState->m_Params->NetworkName() == NULL) || + mc_gState->m_Params->m_NumArgumentsGetFullVersion()); + + std::string strUsage = ""; + if (mc_gState->m_Params->HasOption("-version")) + { + } + else + { + strUsage += "\n" + _("Usage:") + "\n" + + " multichain-cli [options] [params] " + _("Send command to MultiChain Core") + "\n" + + " multichain-cli [options] help " + _("List commands") + "\n" + + " multichain-cli [options] help " + _("Get help for a command") + "\n"; + + strUsage += "\n" + HelpMessageCli(); // MCHN-TODO Edit help message + } + + fprintf(stdout, "%s", strUsage.c_str()); + return false; + } + +/* + if (!boost::filesystem::is_directory(GetDataDir(false))) { + fprintf(stderr, "Error: Specified data directory \"%s\" does not exist.\n", mapArgs["-datadir"].c_str()); + return false; + } +*/ + int RPCPort=MC_DEFAULT_RPC_PORT; + int read_err; + err=MC_ERR_NOERROR; + + boost::filesystem::path path=boost::filesystem::path(string(mc_gState->m_Params->DataDir(0,0))); + if(boost::filesystem::is_directory(path)) + { + path=boost::filesystem::path(string(mc_gState->m_Params->DataDir(1,0))); + + if (!boost::filesystem::is_directory(path)) + { + err=mc_gState->m_Params->ReadConfig(NULL); + } + else + { + read_err=mc_gState->m_NetworkParams->Read(mc_gState->m_Params->NetworkName()); + if(read_err) + { + err=mc_gState->m_Params->ReadConfig(NULL); + + if (mapArgs["-rpcuser"] == "" && mapArgs["-rpcpassword"] == "") + { + if(read_err != MC_ERR_FILE_READ_ERROR) + { + fprintf(stderr,"ERROR: Couldn't read configuration file for blockchain %s. Please try upgrading MultiChain. Exiting...\n",mc_gState->m_Params->NetworkName()); + return false; + } + } + } + else + { + err=mc_gState->m_Params->ReadConfig(mc_gState->m_Params->NetworkName()); + RPCPort=mc_gState->m_NetworkParams->GetInt64Param("defaultrpcport"); + } + } + } + + if(err) + { + fprintf(stderr,"ERROR: Couldn't read parameter file for blockchain %s. Exiting...\n",mc_gState->m_Params->NetworkName()); + return false; + } + + RPCPort=mc_gState->m_Params->GetOption("-rpcport",RPCPort); + + SelectMultiChainBaseParams(mc_gState->m_Params->NetworkName(),RPCPort); + + // Check for -testnet or -regtest parameter (BaseParams() calls are only valid after this clause) +/* + if (!SelectBaseParamsFromCommandLine()) { + fprintf(stderr, "Error: Invalid combination of -regtest and -testnet.\n"); + return false; + } + */ + + return true; +} + +Object CallRPC(const string& strMethod, const Array& params) +{ + if (mapArgs["-rpcuser"] == "" && mapArgs["-rpcpassword"] == "") + throw runtime_error(strprintf( + _("No credentials found for chain \"%s\"\n\n" + "You must set rpcpassword= in the configuration file:\n%s/multichain.conf\n" + "If the file does not exist, create it with owner-readable-only file permissions."), + mc_gState->m_Params->NetworkName(),mc_gState->m_Params->DataDir(1,0))); + + // Connect to localhost + bool fUseSSL = GetBoolArg("-rpcssl", false); + asio::io_service io_service; + ssl::context context(io_service, ssl::context::sslv23); + context.set_options(ssl::context::no_sslv2 | ssl::context::no_sslv3); + asio::ssl::stream sslStream(io_service, context); + SSLIOStreamDevice d(sslStream, fUseSSL); + iostreams::stream< SSLIOStreamDevice > stream(d); + + const bool fConnected = d.connect(GetArg("-rpcconnect", "127.0.0.1"), GetArg("-rpcport", itostr(BaseParams().RPCPort()))); + if (!fConnected) + throw CConnectionFailed("couldn't connect to server"); + + // HTTP basic authentication + + string strUserPass64 = EncodeBase64(mapArgs["-rpcuser"] + ":" + mapArgs["-rpcpassword"]); + map mapRequestHeaders; + mapRequestHeaders["Authorization"] = string("Basic ") + strUserPass64; + // Send request + string strRequest = JSONRPCRequest(strMethod, params, 1); + string strPost = HTTPPost(strRequest, mapRequestHeaders); + stream << strPost << std::flush; + + printf("%s\n",strRequest.c_str()); + + // Receive HTTP reply status + int nProto = 0; + int nStatus = ReadHTTPStatus(stream, nProto); + + // Receive HTTP reply message headers and body + map mapHeaders; + string strReply; + ReadHTTPMessage(stream, mapHeaders, strReply, nProto, std::numeric_limits::max()); + + if (nStatus == HTTP_UNAUTHORIZED) + throw runtime_error("incorrect rpcuser or rpcpassword (authorization failed)"); + else if (nStatus >= 400 && nStatus != HTTP_BAD_REQUEST && nStatus != HTTP_NOT_FOUND && nStatus != HTTP_INTERNAL_SERVER_ERROR) + throw runtime_error(strprintf("server returned HTTP error %d", nStatus)); + else if (strReply.empty()) + throw runtime_error("no response from server"); + + // Parse reply + Value valReply; + if (!read_string(strReply, valReply)) + throw runtime_error("couldn't parse reply from server"); + const Object& reply = valReply.get_obj(); + if (reply.empty()) + throw runtime_error("expected reply to have result, error and id properties"); + + return reply; +} + +int CommandLineRPC(int argc, char *argv[]) +{ + string strPrint; + int nRet = 0; + + bool fWaitForNetworkName=true; + try { + // Skip switches + while (argc > 1 && (IsSwitchChar(argv[1][0]) || fWaitForNetworkName)) { + if(!IsSwitchChar(argv[1][0])) + { + fWaitForNetworkName=false; + } + argc--; + argv++; + } + + // Method + if (argc < 2) + throw runtime_error("too few parameters"); + string strMethod = argv[1]; + + // Parameters default to strings + std::vector strParams(&argv[2], &argv[argc]); + Array params = RPCConvertValues(strMethod, strParams); + + // Execute and handle connection failures with -rpcwait + const bool fWait = GetBoolArg("-rpcwait", false); + do { + try { + const Object reply = CallRPC(strMethod, params); + + // Parse reply + const Value& result = find_value(reply, "result"); + const Value& error = find_value(reply, "error"); + + if (error.type() != null_type) { + // Error + const int code = find_value(error.get_obj(), "code").get_int(); + if (fWait && code == RPC_IN_WARMUP) + throw CConnectionFailed("server in warmup"); + strPrint = "error: " + write_string(error, false); + nRet = abs(code); + + if (error.type() == obj_type) + { + Value errCode = find_value(error.get_obj(), "code"); + Value errMsg = find_value(error.get_obj(), "message"); + strPrint = (errCode.type() == null_type) ? "" : "error code: "+strprintf("%s",errCode.get_int())+"\n"; + + if (errMsg.type() == str_type) + strPrint += "error message:\n"+errMsg.get_str(); + } + + } else { + // Result + if (result.type() == null_type) + strPrint = ""; + else if (result.type() == str_type) + strPrint = result.get_str(); + else + strPrint = write_string(result, true); + } + + // Connection succeeded, no need to retry. + break; + } + catch (const CConnectionFailed& e) { + if (fWait) + MilliSleep(1000); + else + throw; + } + } while (fWait); + } + catch (boost::thread_interrupted) { + throw; + } + catch (std::exception& e) { + strPrint = string("error: ") + e.what(); + nRet = EXIT_FAILURE; + } + catch (...) { + PrintExceptionContinue(NULL, "CommandLineRPC()"); + throw; + } + + if (strPrint != "") { + fprintf((nRet == 0 ? stdout : stderr), "%s\n", strPrint.c_str()); + } + return nRet; +} + +int main(int argc, char* argv[]) +{ + SetupEnvironment(); + try { + if(!AppInitRPC(argc, argv)) + return EXIT_FAILURE; + } + catch (std::exception& e) { + PrintExceptionContinue(&e, "AppInitRPC()"); + return EXIT_FAILURE; + } catch (...) { + PrintExceptionContinue(NULL, "AppInitRPC()"); + return EXIT_FAILURE; + } + + boost::filesystem::path path_cli_log; + if (mapArgs.count("-datadir")) + { + path_cli_log = boost::filesystem::system_complete(mapArgs["-datadir"]); + } + else + { + path_cli_log = string(mc_gState->m_Params->DataDir(0,0)); + } + path_cli_log /= string(".cli_history"); + boost::filesystem::create_directories(path_cli_log); + path_cli_log /= string(mc_gState->m_Params->NetworkName() + string(".log")); + + #ifndef WIN32 + if(mc_gState->m_Params->m_NumArguments == 1) // Interactive mode + { + fprintf(stdout,"\nMultiChain Core RPC client %s\n\n",mc_gState->GetFullVersion()); + if (mapArgs["-rpcuser"] == "" && mapArgs["-rpcpassword"] == "") + { + string str=strprintf( + _("No credentials found for chain \"%s\"\n\n" + "You must set rpcpassword= in the configuration file:\n%s/multichain.conf\n" + "If the file does not exist, create it with owner-readable-only file permissions."), + mc_gState->m_Params->NetworkName(),mc_gState->m_Params->DataDir(1,0)); + printf("error: %s\n",str.c_str()); + return EXIT_FAILURE; + } + + fprintf(stdout,"\nInteractive mode\n\n"); + + mc_TerminalInput *term=new mc_TerminalInput; + term->SetPrompt(mc_gState->m_Params->NetworkName()); + char *command; + char *commandEnd; + + char *dest; + dest=NULL; + dest=(char*)mc_New(MC_DCT_TERM_BUFFER_SIZE+16384); + char *argv_p[1024]; + int argc_p,offset,shift; + bool exitnow=false; + + term->LoadDataFromLog(path_cli_log.string().c_str()); + + term->Prompt(); + while(!exitnow && ((command=term->GetLine()) != NULL)) + { + if((strcmp(command,"exit")==0) || + (strcmp(command,"quit")==0) || + (strcmp(command,"bye")==0)) + { + printf("\nBye\n"); + exitnow=1; + } + else + { + printf("\n"); + commandEnd=command+strlen(command); + offset=0; + argc_p=0; + strcpy(dest+offset,"multichain-cli"); + argv_p[argc_p]=dest+offset; + argc_p++; + offset+=strlen(dest+offset)+1; + strcpy(dest+offset,mc_gState->m_Params->NetworkName()); + argv_p[argc_p]=dest+offset; + argc_p++; + offset+=strlen(dest+offset)+1; + shift=mc_StringToArg(command,dest+offset); + while(shift>0) + { + argv_p[argc_p]=dest+offset; + argc_p++; + offset+=strlen(dest+offset)+1; + command+=shift; + shift=mc_StringToArg(command,dest+offset); + if(argc>=1024) + { + shift=0; + } + if(command>=commandEnd) + { + shift=0; + } + } + + if(shift<0) + { + printf("\nParsing error: %s\n",command); + } + else + { + if(argc_p>2) + { + try { + if(GetBoolArg("-saveclilog",true)) + { + mc_SaveCliCommandToLog(path_cli_log.string().c_str(), argc_p, argv_p); + } + CommandLineRPC(argc_p, argv_p); + } + catch (std::exception& e) { + PrintExceptionContinue(&e, "CommandLineRPC()"); + } catch (...) { + PrintExceptionContinue(NULL, "CommandLineRPC()"); + } + } + } + + term->Prompt(); + } + } + + mc_Delete(dest); + delete term; + + return 0; + } +#endif + + int ret = EXIT_FAILURE; + try { + mc_SaveCliCommandToLog(path_cli_log.string().c_str(), argc, argv); + ret = CommandLineRPC(argc, argv); + } + catch (std::exception& e) { + PrintExceptionContinue(&e, "CommandLineRPC()"); + } catch (...) { + PrintExceptionContinue(NULL, "CommandLineRPC()"); + } + return ret; +} + diff --git a/src/multichain/multichain-util.cpp b/src/multichain/multichain-util.cpp new file mode 100644 index 00000000..523c527a --- /dev/null +++ b/src/multichain/multichain-util.cpp @@ -0,0 +1,218 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" +#include "chainparams/globals.h" + +int main(int argc, char* argv[]) +{ + int err; + int version,v; + char fileName[MC_DCT_DB_MAX_PATH]; + char DataDirArg[MC_DCT_DB_MAX_PATH]; + int isSetDataDirArg; + FILE *fHan; + + mc_MultichainParams* params; + mc_MultichainParams* paramsOld; + mc_gState=new mc_State; + + mc_gState->m_Params->Parse(argc, argv); + mc_gState->m_Params->ReadConfig(NULL); + + printf("MultiChain utilities %s\n\n",mc_gState->GetFullVersion()); + + err=MC_ERR_OPERATION_NOT_SUPPORTED; + + if(mc_gState->m_Params->Command()) + { + if(strcmp(mc_gState->m_Params->Command(),"create") == 0) + { + if(mc_gState->m_Params->m_NumArguments>1) + { + params=new mc_MultichainParams; + + err=MC_ERR_NOERROR; + + version=mc_gState->GetProtocolVersion(); + if(mc_gState->m_Params->m_NumArguments>2) + { + v=atoi(mc_gState->m_Params->m_Arguments[2]); + if( (v>=10002) && (v<=version) ) + { + version=v; + } + else + { + fprintf(stderr,"ERROR: Invalid value for protocol version. Valid range: 10002 - %d\n",mc_gState->GetProtocolVersion()); + err=MC_ERR_INVALID_PARAMETER_VALUE; + } + } + + if(err == MC_ERR_NOERROR) + { +// err=params->Create(mc_gState->m_Params->m_Arguments[1],version); + err=params->Read(mc_gState->m_Params->m_Arguments[1],argc, argv,version); + } + if(err == MC_ERR_NOERROR) + { + err=params->Validate(); + } + if(err == MC_ERR_NOERROR) + { + err=params->Write(0); + } + + if(err == MC_ERR_NOERROR) + { + mc_GenerateConfFiles(mc_gState->m_Params->m_Arguments[1]); + } + if(err == MC_ERR_NOERROR) + { + printf("Blockchain parameter set was successfully generated.\n"); + mc_GetFullFileName(mc_gState->m_Params->m_Arguments[1],"params", ".dat",MC_FOM_RELATIVE_TO_DATADIR,fileName); + printf("You can edit it in %s before running multichaind for the first time.\n\n",fileName); + printf("To generate blockchain please run \"multichaind %s -daemon\".\n",params->Name()); + } + else + { + fprintf(stderr,"ERROR: Blockchain parameter set was not generated.\n"); + } + delete params; + } + } + if(strcmp(mc_gState->m_Params->Command(),"clone") == 0) + { + if(mc_gState->m_Params->m_NumArguments>2) + { + params=new mc_MultichainParams; + paramsOld=new mc_MultichainParams; + + + isSetDataDirArg=mc_GetDataDirArg(DataDirArg); + if(isSetDataDirArg) + { + mc_UnsetDataDirArg(); + } + err=MC_ERR_NOERROR; + mc_GetFullFileName(mc_gState->m_Params->m_Arguments[1],"params", ".dat",MC_FOM_RELATIVE_TO_DATADIR,fileName); + if ((fHan = fopen(fileName, "r"))) + { + fclose(fHan); + } + else + { + fprintf(stderr,"Cannot create chain parameter set, file %s does not exist\n",fileName); + err=MC_ERR_FILE_READ_ERROR; + } + + if(err == MC_ERR_NOERROR) + { + err=paramsOld->Read(mc_gState->m_Params->m_Arguments[1]); + } + if(isSetDataDirArg) + { + mc_SetDataDirArg(DataDirArg); + } + + mc_gState->m_Params->m_Arguments[1]=mc_gState->m_Params->m_Arguments[2]; + if(err == MC_ERR_NOERROR) + { + err=params->Clone(mc_gState->m_Params->m_Arguments[2],paramsOld); + } + if(err == MC_ERR_NOERROR) + { + err=params->Validate(); + } + if(err == MC_ERR_NOERROR) + { + err=params->Write(0); + } + if(err == MC_ERR_NOERROR) + { + + printf("Blockchain parameter set was successfully cloned.\n"); + mc_GetFullFileName(mc_gState->m_Params->m_Arguments[2],"params", ".dat",MC_FOM_RELATIVE_TO_DATADIR,fileName); + printf("You can edit it in %s before running multichaind for the first time.\n\n",fileName); + printf("To generate blockchain please run \"multichaind %s -daemon\".\n",params->Name()); + } + else + { + fprintf(stderr,"ERROR: Blockchain parameter set was not generated.\n"); + } + delete paramsOld; + delete params; + } + err=MC_ERR_NOERROR; + } +/* + if(strcmp(mc_gState->m_Params->Command(),"test") == 0) + { + if(mc_gState->m_Params->m_NumArguments>1) + { + printf("\n>>>>> Test %s started\n\n",mc_gState->m_Params->m_Arguments[1]); + if(strcmp(mc_gState->m_Params->m_Arguments[1],"scenario") == 0) + { + if(mc_gState->m_Params->m_NumArguments>2) + { + err=mc_TestScenario(mc_gState->m_Params->m_Arguments[2]); + } + } + } + + if(err == MC_ERR_NOERROR) + { + printf("\n>>>>> Test completed\n\n"); + } + else + { + if(err == MC_ERR_OPERATION_NOT_SUPPORTED) + { + printf("\n>>>>> ERROR: Test not found\n\n"); + } + else + { + printf("\n>>>>> ERROR: Test exited with error code %d\n\n",err); + } + } + + } +*/ + + } + + if(err == MC_ERR_OPERATION_NOT_SUPPORTED) + { + mc_GetFullFileName("","params", ".dat",MC_FOM_RELATIVE_TO_DATADIR,fileName); + printf("Usage:\n"); + printf(" multichain-util create ( = %d ) [options] Creates new multichain configuration file %s with default parameters\n", + mc_gState->GetProtocolVersion(),fileName); + mc_GetFullFileName("","params", ".dat",MC_FOM_RELATIVE_TO_DATADIR,fileName); + printf(" multichain-util clone [options] Creates new multichain configuration file %s copying parameters\n",fileName); + + isSetDataDirArg=mc_GetDataDirArg(DataDirArg); + if(isSetDataDirArg) + { + mc_UnsetDataDirArg(); + } + mc_GetFullFileName("","params", ".dat",MC_FOM_RELATIVE_TO_DATADIR,fileName); + if(isSetDataDirArg) + { + mc_SetDataDirArg(DataDirArg); + } + printf(" from %s\n",fileName); + printf("\n"); + printf("Options:\n"); + printf(" -datadir= Specify data directory\n"); + printf(" -= Specify blockchain parameter value, e.g. -anyone-can-connect=true\n\n"); + } + + delete mc_gState; + + if(err) + { + return 1; + } + + return 0; +} diff --git a/src/multichain/multichain.h b/src/multichain/multichain.h new file mode 100644 index 00000000..79380bcc --- /dev/null +++ b/src/multichain/multichain.h @@ -0,0 +1,22 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAIN_H +#define MULTICHAIN_H + + +#include "utils/define.h" +#include "utils/declare.h" + +#include "chainparams/chainparams.h" +#include "protocol/multichainscript.h" +#include "permissions/permission.h" +#include "entities/asset.h" +#include "chainparams/state.h" + + +extern unsigned int MIN_RELAY_TX_FEE; + + +#endif /* MULTICHAIN_H */ + diff --git a/src/multichain/multichaind.cpp b/src/multichain/multichaind.cpp new file mode 100644 index 00000000..abad505e --- /dev/null +++ b/src/multichain/multichaind.cpp @@ -0,0 +1,388 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "version/clientversion.h" +#include "rpc/rpcserver.h" +#include "core/init.h" +#include "core/main.h" +#include "ui/noui.h" +#include "ui/ui_interface.h" +#include "utils/util.h" + + +#include +#include +#include + +#include "multichain/multichain.h" +#include "chainparams/globals.h" +static bool fDaemon; + +void DebugPrintClose(); + +void DetectShutdownThread(boost::thread_group* threadGroup) +{ + bool fShutdown = ShutdownRequested(); + // Tell the main threads to shutdown. + while (!fShutdown) + { + MilliSleep(200); + fShutdown = ShutdownRequested(); + } + if (threadGroup) + { + threadGroup->interrupt_all(); + threadGroup->join_all(); + } +} + +bool mc_DoesParentDataDirExist() +{ + if (mapArgs.count("-datadir")) + { + boost::filesystem::path path=boost::filesystem::system_complete(mapArgs["-datadir"]); + if (!boost::filesystem::is_directory(path)) + { + return false; + } + } + return true; +} + +////////////////////////////////////////////////////////////////////////////// +// +// Start +// +bool AppInit(int argc, char* argv[]) +{ + boost::thread_group threadGroup; + boost::thread* detectShutdownThread = NULL; + + bool fRet = false; + + int size; + int err = MC_ERR_NOERROR; + int pipes[2]; + int bytes_read; + int bytes_written; + char bufOutput[4096]; + bool is_daemon; + // + // Parameters + // + // If Qt is used, parameters/bitcoin.conf are parsed in qt/bitcoin.cpp's main() + + + mc_gState=new mc_State; + + mc_gState->m_Params->Parse(argc, argv); + + if(mc_gState->m_Params->NetworkName()) + { + if(strlen(mc_gState->m_Params->NetworkName()) > MC_PRM_NETWORK_NAME_MAX_SIZE) + { + fprintf(stderr, "Error: invalid chain name: %s\n",mc_gState->m_Params->NetworkName()); + return false; + } + } + + if(!mc_DoesParentDataDirExist()) + { + fprintf(stderr,"\nError: Data directory %s needs to exist before calling multichaind. Exiting...\n\n",mapArgs["-datadir"].c_str()); + return false; + } + + + mc_gState->m_Params->HasOption("-?"); + + // Process help and version before taking care about datadir + if (mc_gState->m_Params->HasOption("-?") || + mc_gState->m_Params->HasOption("-help") || + mc_gState->m_Params->HasOption("-version") || + (mc_gState->m_Params->NetworkName() == NULL)) + { + fprintf(stdout,"\nMultiChain Core Daemon %s\n\n",mc_gState->GetFullVersion()); + std::string strUsage = ""; + if (mc_gState->m_Params->HasOption("-version")) + { + strUsage += LicenseInfo(); + } + else + { + strUsage += "\n" + _("Usage:") + "\n" + + " multichaind [options] " + _("Start MultiChain Core Daemon") + "\n"; + + strUsage += "\n" + HelpMessage(HMM_BITCOIND); // MCHN-TODO edit help message + } + + fprintf(stdout, "%s", strUsage.c_str()); + + delete mc_gState; + return false; + } + + if(!GetBoolArg("-shortoutput", false)) + { + fprintf(stdout,"\nMultiChain Core Daemon %s\n\n",mc_gState->GetFullVersion()); + } + + pipes[1]=STDOUT_FILENO; + is_daemon=false; +#ifndef WIN32 + fDaemon = GetBoolArg("-daemon", false); + + if (fDaemon) + { + delete mc_gState; + + if(!GetBoolArg("-shortoutput", false)) + { + fprintf(stdout, "MultiChain server starting\n"); + } + + if (pipe(pipes)) + { + fprintf(stderr, "Error: couldn't create pipe between parent and child processes\n"); + return false; + } + + pid_t pid = fork(); + if (pid < 0) + { + fprintf(stderr, "Error: fork() returned %d errno %d\n", pid, errno); +// delete mc_gState; + return false; + } + + if(pid == 0) + { + is_daemon=true; + close(pipes[0]); + } + + if (pid > 0) // Parent process, pid is child process id + { +// delete mc_gState; + close(pipes[1]); + bytes_read=1; + while(bytes_read>0) + { + bytes_read=read(pipes[0],bufOutput,4096); + if(bytes_read <= 0) + { + return true; + } + bytes_written=write(STDOUT_FILENO,bufOutput,bytes_read); + if(bytes_written != bytes_read) + { + return true; + } + } + return true; + } + + pid_t sid = setsid(); + if (sid < 0) + fprintf(stderr, "ERROR: setsid() returned %d errno %d\n", sid, errno); + + mc_gState=new mc_State; + + mc_gState->m_Params->Parse(argc, argv); + } +#endif + + mc_GenerateConfFiles(mc_gState->m_Params->NetworkName()); + + err=mc_gState->m_Params->ReadConfig(mc_gState->m_Params->NetworkName()); + + if(err) + { + fprintf(stderr,"ERROR: Couldn't read parameter file for blockchain %s. Exiting...\n",mc_gState->m_Params->NetworkName()); + delete mc_gState; + return false; + } + + err=mc_gState->m_NetworkParams->Read(mc_gState->m_Params->NetworkName()); + if(err) + { + fprintf(stderr,"ERROR: Couldn't read configuration file for blockchain %s. Please try upgrading MultiChain. Exiting...\n",mc_gState->m_Params->NetworkName()); + delete mc_gState; + return false; + } + + err=mc_gState->m_NetworkParams->Validate(); + if(err) + { + fprintf(stderr,"ERROR: Couldn't validate parameter set for blockchain %s. Exiting...\n",mc_gState->m_Params->NetworkName()); + delete mc_gState; + return false; + } + + if(GetBoolArg("-reindex", false)) + { + mc_RemoveDir(mc_gState->m_Params->NetworkName(),"permissions.db"); + mc_RemoveFile(mc_gState->m_Params->NetworkName(),"permissions",".dat",MC_FOM_RELATIVE_TO_DATADIR); + mc_RemoveFile(mc_gState->m_Params->NetworkName(),"permissions",".log",MC_FOM_RELATIVE_TO_DATADIR); + } + + mc_gState->m_Permissions= new mc_Permissions; + err=mc_gState->m_Permissions->Initialize(mc_gState->m_Params->NetworkName(),0); + if(err) + { + fprintf(stderr,"\nERROR: Couldn't initialize permission database for blockchain %s. Probably multichaind for this blockchain is already running. Exiting...\n",mc_gState->m_Params->NetworkName()); + delete mc_gState; + return false; + } + + if( (mc_gState->m_NetworkParams->GetParam("protocolversion",&size) != NULL) && + (mc_gState->GetProtocolVersion() < (int)mc_gState->m_NetworkParams->GetInt64Param("protocolversion")) ) + { + fprintf(stderr,"ERROR: Parameter set for blockchain %s was generated by MultiChain running newer protocol version (%d)\n\n", + mc_gState->m_Params->NetworkName(),(int)mc_gState->m_NetworkParams->GetInt64Param("protocolversion")); + fprintf(stderr,"Please upgrade MultiChain\n\n"); + delete mc_gState; + return false; + } + + switch(mc_gState->m_NetworkParams->m_Status) + { + case MC_PRM_STATUS_EMPTY: + case MC_PRM_STATUS_MINIMAL: + if(mc_gState->GetSeedNode() == NULL) + { + fprintf(stderr,"ERROR: Parameter set for blockchain %s is not complete. \n\n\n",mc_gState->m_Params->NetworkName()); + fprintf(stderr,"If you want to create new blockchain please run one of the following:\n\n"); + fprintf(stderr," multichain-util create %s\n",mc_gState->m_Params->NetworkName()); + fprintf(stderr," multichain-util clone %s\n",mc_gState->m_Params->NetworkName()); + fprintf(stderr,"\nAnd rerun multichaind %s\n\n\n",mc_gState->m_Params->NetworkName()); + fprintf(stderr,"If you want to connect to existing blockchain please specify seed node:\n\n"); + fprintf(stderr," multichaind %s@\n",mc_gState->m_Params->NetworkName()); + fprintf(stderr," multichaind %s@:\n\n\n",mc_gState->m_Params->NetworkName()); + delete mc_gState; + return false; + } + break; + case MC_PRM_STATUS_ERROR: + fprintf(stderr,"ERROR: Parameter set for blockchain %s has errors. Please run one of the following:\n\n",mc_gState->m_Params->NetworkName()); + fprintf(stderr," multichain-util create %s\n",mc_gState->m_Params->NetworkName()); + fprintf(stderr," multichain-util clone %s\n",mc_gState->m_Params->NetworkName()); + fprintf(stderr,"\nAnd rerun multichaind %s\n",mc_gState->m_Params->NetworkName()); + delete mc_gState; + return false; + case MC_PRM_STATUS_INVALID: + fprintf(stderr,"ERROR: Parameter set for blockchain %s is invalid. You may generate new network using these parameters by running:\n\n",mc_gState->m_Params->NetworkName()); + fprintf(stderr," multichain-util clone %s \n",mc_gState->m_Params->NetworkName()); + delete mc_gState; + return false; + case MC_PRM_STATUS_GENERATED: + case MC_PRM_STATUS_VALID: + break; + default: + fprintf(stderr,"INTERNAL ERROR: Unknown parameter set status %d\n",mc_gState->m_NetworkParams->m_Status); + delete mc_gState; + return false; + break; + } + + SelectMultiChainParams(mc_gState->m_Params->NetworkName()); + + try + { +/* +#ifndef WIN32 + fDaemon = GetBoolArg("-daemon", false); + + if (fDaemon) + { + fprintf(stdout, "MultiChain server starting\n"); + + pid_t pid = fork(); + if (pid < 0) + { + fprintf(stderr, "Error: fork() returned %d errno %d\n", pid, errno); + delete mc_gState; + return false; + } + if (pid > 0) // Parent process, pid is child process id + { + delete mc_gState; + return true; + } + + pid_t sid = setsid(); + if (sid < 0) + fprintf(stderr, "ERROR: setsid() returned %d errno %d\n", sid, errno); + + } +#endif +*/ + SoftSetBoolArg("-server", true); + detectShutdownThread = new boost::thread(boost::bind(&DetectShutdownThread, &threadGroup)); + fRet = AppInit2(threadGroup,pipes[1]); + if(is_daemon) + { + close(pipes[1]); + } + } + catch (std::exception& e) { + PrintExceptionContinue(&e, "AppInit()"); + } catch (...) { + PrintExceptionContinue(NULL, "AppInit()"); + } + + if (!fRet) + { + if (detectShutdownThread) + detectShutdownThread->interrupt(); + + threadGroup.interrupt_all(); + // threadGroup.join_all(); was left out intentionally here, because we didn't re-test all of + // the startup-failure cases to make sure they don't result in a hang due to some + // thread-blocking-waiting-for-another-thread-during-startup case + } + if (detectShutdownThread) + { + detectShutdownThread->join(); + delete detectShutdownThread; + detectShutdownThread = NULL; + } + + Shutdown(); + DebugPrintClose(); + + std::string datadir_to_delete=""; + + if(mc_gState->m_NetworkParams->Validate() == 0) + { + if(mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_EMPTY) + { + datadir_to_delete=strprintf("%s",mc_gState->m_Params->NetworkName()); + } + } + + delete mc_gState; + + if(datadir_to_delete.size()) + { + mc_RemoveDataDir(datadir_to_delete.c_str()); + } + + return fRet; +} + + +int main(int argc, char* argv[]) +{ + SetupEnvironment(); + + + + // Connect bitcoind signal handlers + noui_connect(); + + return (AppInit(argc, argv) ? 0 : 1); + +} + diff --git a/src/multichaind-res.rc b/src/multichaind-res.rc new file mode 100644 index 00000000..9e4b7dc2 --- /dev/null +++ b/src/multichaind-res.rc @@ -0,0 +1,35 @@ +#include // needed for VERSIONINFO +#include "version/clientversion.h" // holds the needed client version information + +#define VER_PRODUCTVERSION CLIENT_VERSION_MAJOR,CLIENT_VERSION_MINOR,CLIENT_VERSION_REVISION,CLIENT_VERSION_BUILD +#define VER_PRODUCTVERSION_STR STRINGIZE(CLIENT_VERSION_MAJOR) "." STRINGIZE(CLIENT_VERSION_MINOR) "." STRINGIZE(CLIENT_VERSION_REVISION) "." STRINGIZE(CLIENT_VERSION_BUILD) +#define VER_FILEVERSION VER_PRODUCTVERSION +#define VER_FILEVERSION_STR VER_PRODUCTVERSION_STR + +VS_VERSION_INFO VERSIONINFO +FILEVERSION VER_FILEVERSION +PRODUCTVERSION VER_PRODUCTVERSION +FILEOS VOS_NT_WINDOWS32 +FILETYPE VFT_APP +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "040904E4" // U.S. English - multilingual (hex) + BEGIN + VALUE "CompanyName", "Coin Sciences Ltd" + VALUE "FileDescription", "multichaind (OSS daemon/client for MultiChain)" + VALUE "FileVersion", VER_FILEVERSION_STR + VALUE "InternalName", "multichaind" + VALUE "LegalCopyright", COPYRIGHT_STR + VALUE "LegalTrademarks1", "The software product under this license is provided free of charge. Full terms are shown at: http://www.multichain.com/terms-of-service/." + VALUE "OriginalFilename", "multichaind.exe" + VALUE "ProductName", "multichaind" + VALUE "ProductVersion", VER_PRODUCTVERSION_STR + END + END + + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x0, 1252 // language neutral - multilingual (decimal) + END +END diff --git a/src/net/net.cpp b/src/net/net.cpp new file mode 100644 index 00000000..9fe1fd84 --- /dev/null +++ b/src/net/net.cpp @@ -0,0 +1,2380 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#if defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" +#endif + +#include "net/net.h" + +#include "storage/addrman.h" +#include "chainparams/chainparams.h" +#include "version/clientversion.h" +#include "primitives/transaction.h" +#include "ui/ui_interface.h" + +#include "keys/key.h" +#include "keys/pubkey.h" +#include "wallet/wallet.h" +extern CWallet* pwalletMain; +#include "multichain/multichain.h" + +#ifdef WIN32 +#include +#else +#include +#endif + +#ifdef USE_UPNP +#include +#include +#include +#include +#endif + +#include +#include + +// Dump addresses to peers.dat every 15 minutes (900s) +#define DUMP_ADDRESSES_INTERVAL 900 + +#if !defined(HAVE_MSG_NOSIGNAL) && !defined(MSG_NOSIGNAL) +#define MSG_NOSIGNAL 0 +#endif + +// Fix for ancient MinGW versions, that don't have defined these in ws2tcpip.h. +// Todo: Can be removed when our pull-tester is upgraded to a modern MinGW version. +#ifdef WIN32 +#ifndef PROTECTION_LEVEL_UNRESTRICTED +#define PROTECTION_LEVEL_UNRESTRICTED 10 +#endif +#ifndef IPV6_PROTECTION_LEVEL +#define IPV6_PROTECTION_LEVEL 23 +#endif +#endif + +using namespace boost; +using namespace std; + +namespace { + const int MAX_OUTBOUND_CONNECTIONS = 8; + + struct ListenSocket { + SOCKET socket; + bool whitelisted; + + ListenSocket(SOCKET socket, bool whitelisted) : socket(socket), whitelisted(whitelisted) {} + }; +} + +// +// Global state variables +// +bool fDiscover = true; +bool fListen = true; +uint64_t nLocalServices = NODE_NETWORK; +CCriticalSection cs_mapLocalHost; +map mapLocalHost; +static bool vfReachable[NET_MAX] = {}; +static bool vfLimited[NET_MAX] = {}; +static CNode* pnodeLocalHost = NULL; +uint64_t nLocalHostNonce = 0; +static std::vector vhListenSocket; +CAddrMan addrman; +int nMaxConnections = 125; +bool fAddressesInitialized = false; + +vector vNodes; +CCriticalSection cs_vNodes; +map mapRelay; +deque > vRelayExpiration; +CCriticalSection cs_mapRelay; +limitedmap mapAlreadyAskedFor(MAX_INV_SZ); + +static deque vOneShots; +CCriticalSection cs_vOneShots; + +set setservAddNodeAddresses; +CCriticalSection cs_setservAddNodeAddresses; + +vector vAddedNodes; +CCriticalSection cs_vAddedNodes; + +NodeId nLastNodeId = 0; +CCriticalSection cs_nLastNodeId; + +static CSemaphore *semOutbound = NULL; + +// Signals for message handling +static CNodeSignals g_signals; +CNodeSignals& GetNodeSignals() { return g_signals; } + +void AddOneShot(string strDest) +{ + LOCK(cs_vOneShots); + vOneShots.push_back(strDest); +} + +unsigned short GetListenPort() +{ + return (unsigned short)(GetArg("-port", Params().GetDefaultPort())); +} + +// find 'best' local address for a particular peer +bool GetLocal(CService& addr, const CNetAddr *paddrPeer) +{ + if (!fListen) + return false; + + int nBestScore = -1; + int nBestReachability = -1; + { + LOCK(cs_mapLocalHost); + for (map::iterator it = mapLocalHost.begin(); it != mapLocalHost.end(); it++) + { + int nScore = (*it).second.nScore; + int nReachability = (*it).first.GetReachabilityFrom(paddrPeer); + if (nReachability > nBestReachability || (nReachability == nBestReachability && nScore > nBestScore)) + { + addr = CService((*it).first, (*it).second.nPort); + nBestReachability = nReachability; + nBestScore = nScore; + } + } + } + return nBestScore >= 0; +} + +// get best local address for a particular peer as a CAddress +// Otherwise, return the unroutable 0.0.0.0 but filled in with +// the normal parameters, since the IP may be changed to a useful +// one by discovery. +CAddress GetLocalAddress(const CNetAddr *paddrPeer) +{ + CAddress ret(CService("0.0.0.0",GetListenPort()),0); + CService addr; + if (GetLocal(addr, paddrPeer)) + { + ret = CAddress(addr); + } + ret.nServices = nLocalServices; + ret.nTime = GetAdjustedTime(); + return ret; +} + +bool RecvLine(SOCKET hSocket, string& strLine) +{ + strLine = ""; + while (true) + { + char c; + int nBytes = recv(hSocket, &c, 1, 0); + if (nBytes > 0) + { + if (c == '\n') + continue; + if (c == '\r') + return true; + strLine += c; + if (strLine.size() >= 9000) + return true; + } + else if (nBytes <= 0) + { + boost::this_thread::interruption_point(); + if (nBytes < 0) + { + int nErr = WSAGetLastError(); + if (nErr == WSAEMSGSIZE) + continue; + if (nErr == WSAEWOULDBLOCK || nErr == WSAEINTR || nErr == WSAEINPROGRESS) + { + MilliSleep(10); + continue; + } + } + if (!strLine.empty()) + return true; + if (nBytes == 0) + { + // socket closed + LogPrint("net", "socket closed\n"); + return false; + } + else + { + // socket error + int nErr = WSAGetLastError(); + LogPrint("net", "recv failed: %s\n", NetworkErrorString(nErr)); + return false; + } + } + } +} + +int GetnScore(const CService& addr) +{ + LOCK(cs_mapLocalHost); + if (mapLocalHost.count(addr) == LOCAL_NONE) + return 0; + return mapLocalHost[addr].nScore; +} + +// Is our peer's addrLocal potentially useful as an external IP source? +bool IsPeerAddrLocalGood(CNode *pnode) +{ + return fDiscover && pnode->addr.IsRoutable() && pnode->addrLocal.IsRoutable() && + !IsLimited(pnode->addrLocal.GetNetwork()); +} + +// pushes our own address to a peer +void AdvertizeLocal(CNode *pnode) +{ + if (fListen && pnode->fSuccessfullyConnected) + { + CAddress addrLocal = GetLocalAddress(&pnode->addr); + // If discovery is enabled, sometimes give our peer the address it + // tells us that it sees us as in case it has a better idea of our + // address than we do. + if (IsPeerAddrLocalGood(pnode) && (!addrLocal.IsRoutable() || + GetRand((GetnScore(addrLocal) > LOCAL_MANUAL) ? 8:2) == 0)) + { + addrLocal.SetIP(pnode->addrLocal); + } + if (addrLocal.IsRoutable()) + { + pnode->PushAddress(addrLocal); + } + } +} + +void SetReachable(enum Network net, bool fFlag) +{ + LOCK(cs_mapLocalHost); + vfReachable[net] = fFlag; + if (net == NET_IPV6 && fFlag) + vfReachable[NET_IPV4] = true; +} + +// learn a new local address +bool AddLocal(const CService& addr, int nScore) +{ + if (!addr.IsRoutable()) + return false; + + if (!fDiscover && nScore < LOCAL_MANUAL) + return false; + + if (IsLimited(addr)) + return false; + + LogPrintf("AddLocal(%s,%i)\n", addr.ToString(), nScore); + + { + LOCK(cs_mapLocalHost); + bool fAlready = mapLocalHost.count(addr) > 0; + LocalServiceInfo &info = mapLocalHost[addr]; + if (!fAlready || nScore >= info.nScore) { + info.nScore = nScore + (fAlready ? 1 : 0); + info.nPort = addr.GetPort(); + } + SetReachable(addr.GetNetwork()); + } + + return true; +} + +bool AddLocal(const CNetAddr &addr, int nScore) +{ + return AddLocal(CService(addr, GetListenPort()), nScore); +} + +/** Make a particular network entirely off-limits (no automatic connects to it) */ +void SetLimited(enum Network net, bool fLimited) +{ + if (net == NET_UNROUTABLE) + return; + LOCK(cs_mapLocalHost); + vfLimited[net] = fLimited; +} + +bool IsLimited(enum Network net) +{ + LOCK(cs_mapLocalHost); + return vfLimited[net]; +} + +bool IsLimited(const CNetAddr &addr) +{ + return IsLimited(addr.GetNetwork()); +} + +/** vote for a local address */ +bool SeenLocal(const CService& addr) +{ + { + LOCK(cs_mapLocalHost); + if (mapLocalHost.count(addr) == 0) + return false; + mapLocalHost[addr].nScore++; + } + return true; +} + + +/** check whether a given address is potentially local */ +bool IsLocal(const CService& addr) +{ + LOCK(cs_mapLocalHost); + return mapLocalHost.count(addr) > 0; +} + +/** check whether a given network is one we can probably connect to */ +bool IsReachable(enum Network net) +{ + LOCK(cs_mapLocalHost); + return vfReachable[net] && !vfLimited[net]; +} + +/** check whether a given address is in a network we can probably connect to */ +bool IsReachable(const CNetAddr& addr) +{ + enum Network net = addr.GetNetwork(); + return IsReachable(net); +} + +void AddressCurrentlyConnected(const CService& addr) +{ + addrman.Connected(addr); +} + + +uint64_t CNode::nTotalBytesRecv = 0; +uint64_t CNode::nTotalBytesSent = 0; +CCriticalSection CNode::cs_totalBytesRecv; +CCriticalSection CNode::cs_totalBytesSent; + +CNode* FindNode(const CNetAddr& ip) +{ + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + if ((CNetAddr)pnode->addr == ip) + return (pnode); + return NULL; +} + +CNode* FindNode(const std::string& addrName) +{ + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + if (pnode->addrName == addrName) + return (pnode); + return NULL; +} + +CNode* FindNode(const CService& addr) +{ + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + if ((CService)pnode->addr == addr) + return (pnode); + return NULL; +} + +CNode* ConnectNode(CAddress addrConnect, const char *pszDest) +{ + if (pszDest == NULL) { +// if (IsLocal(addrConnect)) + if (IsLocal(addrConnect) && (addrConnect.GetPort() == GetListenPort())) + return NULL; + + // Look for an existing connection + CNode* pnode = FindNode((CService)addrConnect); + if (pnode) + { + pnode->AddRef(); + return pnode; + } + } + + /// debug print + LogPrint("net", "net: trying connection %s lastseen=%.1fhrs\n", + pszDest ? pszDest : addrConnect.ToString(), + pszDest ? 0.0 : (double)(GetAdjustedTime() - addrConnect.nTime)/3600.0); + + // Connect + SOCKET hSocket; + bool proxyConnectionFailed = false; + if (pszDest ? ConnectSocketByName(addrConnect, hSocket, pszDest, Params().GetDefaultPort(), nConnectTimeout, &proxyConnectionFailed) : + ConnectSocket(addrConnect, hSocket, nConnectTimeout, &proxyConnectionFailed)) + { +/* MCHN START */ + SetReachable(addrConnect.GetNetwork()); +/* MCHN END */ + addrman.Attempt(addrConnect); + + // Add node + CNode* pnode = new CNode(hSocket, addrConnect, pszDest ? pszDest : "", false); + pnode->AddRef(); + +/* MCHN START */ + if(pszDest) + { + if(mc_gState->GetSeedNode()) + { + if(strcmp(pszDest,mc_gState->GetSeedNode()) == 0) + { + mc_gState->m_pSeedNode=pnode; + LogPrint("mchn","mchn: Connected to seed node %s\n",pszDest); + } + } + } + +/* MCHN END */ + { + LOCK(cs_vNodes); + vNodes.push_back(pnode); + } + + pnode->nTimeConnected = GetTime(); + + return pnode; + } else if (!proxyConnectionFailed) { + // If connecting to the node failed, and failure is not caused by a problem connecting to + // the proxy, mark this as an attempt. + addrman.Attempt(addrConnect); + } + + return NULL; +} + +void CNode::CloseSocketDisconnect() +{ + fDisconnect = true; + if (hSocket != INVALID_SOCKET) + { + LogPrint("net", "disconnecting peer=%d\n", id); + CloseSocket(hSocket); + } + + // in case this fails, we'll empty the recv buffer when the CNode is deleted + TRY_LOCK(cs_vRecvMsg, lockRecv); + if (lockRecv) + vRecvMsg.clear(); +} + +void CNode::PushVersion() +{ + int nBestHeight = g_signals.GetHeight().get_value_or(0); + + /// when NTP implemented, change to just nTime = GetAdjustedTime() + int64_t nTime = (fInbound ? GetAdjustedTime() : GetTime()); + CAddress addrYou = (addr.IsRoutable() && !IsProxy(addr) ? addr : CAddress(CService("0.0.0.0",0))); + CAddress addrMe = GetLocalAddress(&addr); + GetRandBytes((unsigned char*)&nLocalHostNonce, sizeof(nLocalHostNonce)); + nVersionNonceSent=nLocalHostNonce; // MCHN + if (fLogIPs) + LogPrint("net", "send version message: version %d, blocks=%d, us=%s, them=%s, peer=%d\n", PROTOCOL_VERSION, nBestHeight, addrMe.ToString(), addrYou.ToString(), id); + else + LogPrint("net", "send version message: version %d, blocks=%d, us=%s, peer=%d\n", PROTOCOL_VERSION, nBestHeight, addrMe.ToString(), id); + +/* MCHN START */ + std::string subver=FormatSubVersion(CLIENT_NAME, CLIENT_VERSION, std::vector()); + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + subver=FormatSubVersion("MultiChain", mc_gState->GetProtocolVersion(), std::vector()); + } + PushMessage("version", PROTOCOL_VERSION, nLocalServices, nTime, addrYou, addrMe, + nLocalHostNonce, subver, nBestHeight, true); +/* MCHN END */ +} + + + + + +std::map CNode::setBanned; +CCriticalSection CNode::cs_setBanned; + +void CNode::ClearBanned() +{ + setBanned.clear(); +} + +bool CNode::IsBanned(CNetAddr ip) +{ + bool fResult = false; + { + LOCK(cs_setBanned); + std::map::iterator i = setBanned.find(ip); + if (i != setBanned.end()) + { + int64_t t = (*i).second; + if (GetTime() < t) + fResult = true; + } + } + return fResult; +} + +bool CNode::Ban(const CNetAddr &addr) { + int64_t banTime = GetTime()+GetArg("-bantime", 60*60*24); // Default 24-hour ban + { + LOCK(cs_setBanned); + if (setBanned[addr] < banTime) + setBanned[addr] = banTime; + } + return true; +} + + +std::vector CNode::vWhitelistedRange; +CCriticalSection CNode::cs_vWhitelistedRange; + +bool CNode::IsWhitelistedRange(const CNetAddr &addr) { + LOCK(cs_vWhitelistedRange); + BOOST_FOREACH(const CSubNet& subnet, vWhitelistedRange) { + if (subnet.Match(addr)) + return true; + } + return false; +} + +void CNode::AddWhitelistedRange(const CSubNet &subnet) { + LOCK(cs_vWhitelistedRange); + vWhitelistedRange.push_back(subnet); +} + +#undef X +#define X(name) stats.name = name +void CNode::copyStats(CNodeStats &stats) +{ + stats.nodeid = this->GetId(); + X(nServices); + X(nLastSend); + X(nLastRecv); + X(nTimeConnected); + X(addrName); + X(nVersion); + X(cleanSubVer); + X(fInbound); + X(nStartingHeight); + X(nSendBytes); + X(nRecvBytes); + X(fWhitelisted); + + // It is common for nodes with good ping times to suddenly become lagged, + // due to a new block arriving or other large transfer. + // Merely reporting pingtime might fool the caller into thinking the node was still responsive, + // since pingtime does not update until the ping is complete, which might take a while. + // So, if a ping is taking an unusually long time in flight, + // the caller can immediately detect that this is happening. + int64_t nPingUsecWait = 0; + if ((0 != nPingNonceSent) && (0 != nPingUsecStart)) { + nPingUsecWait = GetTimeMicros() - nPingUsecStart; + } + + // Raw ping time is in microseconds, but show it to user as whole seconds (Bitcoin users should be well used to small numbers with many decimal places by now :) + stats.dPingTime = (((double)nPingUsecTime) / 1e6); + stats.dPingWait = (((double)nPingUsecWait) / 1e6); + + // Leave string empty if addrLocal invalid (not filled in yet) + stats.addrLocal = addrLocal.IsValid() ? addrLocal.ToString() : ""; + +/* MCHN START */ + stats.kAddrLocal=kAddrLocal; + stats.kAddrRemote=kAddrRemote; + stats.fSuccessfullyConnected=fSuccessfullyConnected; +/* MCHN END */ + +} +#undef X + +// requires LOCK(cs_vRecvMsg) +bool CNode::ReceiveMsgBytes(const char *pch, unsigned int nBytes) +{ + while (nBytes > 0) { + + // get current incomplete message, or create a new one + if (vRecvMsg.empty() || + vRecvMsg.back().complete()) + vRecvMsg.push_back(CNetMessage(SER_NETWORK, nRecvVersion)); + + CNetMessage& msg = vRecvMsg.back(); + + // absorb network data + int handled; + if (!msg.in_data) + handled = msg.readHeader(pch, nBytes); + else + handled = msg.readData(pch, nBytes); + + if (handled < 0) + return false; + + pch += handled; + nBytes -= handled; + + if (msg.complete()) + msg.nTime = GetTimeMicros(); + } + + return true; +} + +int CNetMessage::readHeader(const char *pch, unsigned int nBytes) +{ + // copy data to temporary parsing buffer + unsigned int nRemaining = 24 - nHdrPos; + unsigned int nCopy = std::min(nRemaining, nBytes); + + memcpy(&hdrbuf[nHdrPos], pch, nCopy); + nHdrPos += nCopy; + +// mc_Dump("HEAD",pch,nCopy); + + // if header incomplete, exit + if (nHdrPos < 24) + return nCopy; + + // deserialize to CMessageHeader + try { + hdrbuf >> hdr; + } + catch (const std::exception &) { + return -1; + } + + // reject messages larger than MAX_SIZE + if (hdr.nMessageSize > MAX_SIZE) + return -1; + + // switch state to reading message data + in_data = true; + + return nCopy; +} + +int CNetMessage::readData(const char *pch, unsigned int nBytes) +{ + unsigned int nRemaining = hdr.nMessageSize - nDataPos; + unsigned int nCopy = std::min(nRemaining, nBytes); + + if (vRecv.size() < nDataPos + nCopy) { + // Allocate up to 256 KiB ahead, but never more than the total message size. + vRecv.resize(std::min(hdr.nMessageSize, nDataPos + nCopy + 256 * 1024)); + } + + memcpy(&vRecv[nDataPos], pch, nCopy); + +// mc_Dump("RECV",pch,nCopy); + + nDataPos += nCopy; + + return nCopy; +} + + + + + + + + + +// requires LOCK(cs_vSend) +void SocketSendData(CNode *pnode) +{ + std::deque::iterator it = pnode->vSendMsg.begin(); + + while (it != pnode->vSendMsg.end()) { + const CSerializeData &data = *it; + assert(data.size() > pnode->nSendOffset); + +// mc_Dump("SEND",(const void*)(&data[pnode->nSendOffset]),data.size()- pnode->nSendOffset); + + int nBytes = send(pnode->hSocket, &data[pnode->nSendOffset], data.size() - pnode->nSendOffset, MSG_NOSIGNAL | MSG_DONTWAIT); + if (nBytes > 0) { + pnode->nLastSend = GetTime(); + pnode->nSendBytes += nBytes; + pnode->nSendOffset += nBytes; + pnode->RecordBytesSent(nBytes); + if (pnode->nSendOffset == data.size()) { + pnode->nSendOffset = 0; + pnode->nSendSize -= data.size(); + it++; + } else { + // could not send full message; stop sending more + break; + } + } else { + if (nBytes < 0) { + // error + int nErr = WSAGetLastError(); + if (nErr != WSAEWOULDBLOCK && nErr != WSAEMSGSIZE && nErr != WSAEINTR && nErr != WSAEINPROGRESS) + { + LogPrintf("socket send error %s\n", NetworkErrorString(nErr)); + pnode->CloseSocketDisconnect(); + } + } + // couldn't send anything at all + break; + } + } + + if (it == pnode->vSendMsg.end()) { + assert(pnode->nSendOffset == 0); + assert(pnode->nSendSize == 0); + } + pnode->vSendMsg.erase(pnode->vSendMsg.begin(), it); +} + +static list vNodesDisconnected; + +void ThreadSocketHandler() +{ + unsigned int nPrevNodeCount = 0; + while (true) + { + // + // Disconnect nodes + // + { + LOCK(cs_vNodes); + // Disconnect unused nodes + vector vNodesCopy = vNodes; + BOOST_FOREACH(CNode* pnode, vNodesCopy) + { + if (pnode->fDisconnect || + (pnode->GetRefCount() <= 0 && pnode->vRecvMsg.empty() && pnode->nSendSize == 0 && pnode->ssSend.empty())) + { + // remove from vNodes + vNodes.erase(remove(vNodes.begin(), vNodes.end(), pnode), vNodes.end()); + + // release outbound grant (if any) + pnode->grantOutbound.Release(); + + // close socket and cleanup +/* MCHN START */ + LogPrint("net","disconnect flag set\n"); +/* MCHN END */ + pnode->CloseSocketDisconnect(); + + // hold in disconnected pool until all refs are released + if (pnode->fNetworkNode || pnode->fInbound) + pnode->Release(); + vNodesDisconnected.push_back(pnode); + } + } + } + { + // Delete disconnected nodes + list vNodesDisconnectedCopy = vNodesDisconnected; + BOOST_FOREACH(CNode* pnode, vNodesDisconnectedCopy) + { + // wait until threads are done using it + if (pnode->GetRefCount() <= 0) + { + bool fDelete = false; + { + TRY_LOCK(pnode->cs_vSend, lockSend); + if (lockSend) + { + TRY_LOCK(pnode->cs_vRecvMsg, lockRecv); + if (lockRecv) + { + TRY_LOCK(pnode->cs_inventory, lockInv); + if (lockInv) + fDelete = true; + } + } + } + if (fDelete) + { + vNodesDisconnected.remove(pnode); + delete pnode; + } + } + } + } + if(vNodes.size() != nPrevNodeCount) { + nPrevNodeCount = vNodes.size(); + uiInterface.NotifyNumConnectionsChanged(nPrevNodeCount); + } + + // + // Find which sockets have data to receive + // + struct timeval timeout; + timeout.tv_sec = 0; + timeout.tv_usec = 50000; // frequency to poll pnode->vSend + + fd_set fdsetRecv; + fd_set fdsetSend; + fd_set fdsetError; + FD_ZERO(&fdsetRecv); + FD_ZERO(&fdsetSend); + FD_ZERO(&fdsetError); + SOCKET hSocketMax = 0; + bool have_fds = false; + + BOOST_FOREACH(const ListenSocket& hListenSocket, vhListenSocket) { + FD_SET(hListenSocket.socket, &fdsetRecv); + hSocketMax = max(hSocketMax, hListenSocket.socket); + have_fds = true; + } + + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + { + if (pnode->hSocket == INVALID_SOCKET) + continue; + FD_SET(pnode->hSocket, &fdsetError); + hSocketMax = max(hSocketMax, pnode->hSocket); + have_fds = true; + + // Implement the following logic: + // * If there is data to send, select() for sending data. As this only + // happens when optimistic write failed, we choose to first drain the + // write buffer in this case before receiving more. This avoids + // needlessly queueing received data, if the remote peer is not themselves + // receiving data. This means properly utilizing TCP flow control signalling. + // * Otherwise, if there is no (complete) message in the receive buffer, + // or there is space left in the buffer, select() for receiving data. + // * (if neither of the above applies, there is certainly one message + // in the receiver buffer ready to be processed). + // Together, that means that at least one of the following is always possible, + // so we don't deadlock: + // * We send some data. + // * We wait for data to be received (and disconnect after timeout). + // * We process a message in the buffer (message handler thread). + { + TRY_LOCK(pnode->cs_vSend, lockSend); + if (lockSend && !pnode->vSendMsg.empty()) { + FD_SET(pnode->hSocket, &fdsetSend); + continue; + } + } + { + TRY_LOCK(pnode->cs_vRecvMsg, lockRecv); + if (lockRecv && ( + pnode->vRecvMsg.empty() || !pnode->vRecvMsg.front().complete() || + pnode->GetTotalRecvSize() <= ReceiveFloodSize())) + FD_SET(pnode->hSocket, &fdsetRecv); + } + } + } + + int nSelect = select(have_fds ? hSocketMax + 1 : 0, + &fdsetRecv, &fdsetSend, &fdsetError, &timeout); + boost::this_thread::interruption_point(); + + if (nSelect == SOCKET_ERROR) + { + if (have_fds) + { + int nErr = WSAGetLastError(); + LogPrintf("socket select error %s\n", NetworkErrorString(nErr)); + for (unsigned int i = 0; i <= hSocketMax; i++) + FD_SET(i, &fdsetRecv); + } + FD_ZERO(&fdsetSend); + FD_ZERO(&fdsetError); + MilliSleep(timeout.tv_usec/1000); + } + + // + // Accept new connections + // + BOOST_FOREACH(const ListenSocket& hListenSocket, vhListenSocket) + { + if (hListenSocket.socket != INVALID_SOCKET && FD_ISSET(hListenSocket.socket, &fdsetRecv)) + { + struct sockaddr_storage sockaddr; + socklen_t len = sizeof(sockaddr); + SOCKET hSocket = accept(hListenSocket.socket, (struct sockaddr*)&sockaddr, &len); + CAddress addr; + int nInbound = 0; + + if (hSocket != INVALID_SOCKET) + if (!addr.SetSockAddr((const struct sockaddr*)&sockaddr)) + LogPrintf("Warning: Unknown socket family\n"); + + LogPrint("net","Connection attempt from: %s\n", addr.ToString().c_str()); + + bool whitelisted = hListenSocket.whitelisted || CNode::IsWhitelistedRange(addr); + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + if (pnode->fInbound) + nInbound++; + } + + if (hSocket == INVALID_SOCKET) + { + int nErr = WSAGetLastError(); + if (nErr != WSAEWOULDBLOCK) + LogPrintf("socket error accept failed: %s\n", NetworkErrorString(nErr)); + } + else if (nInbound >= nMaxConnections - MAX_OUTBOUND_CONNECTIONS) + { + CloseSocket(hSocket); + } + else if (CNode::IsBanned(addr) && !whitelisted) + { + LogPrintf("connection from %s dropped (banned)\n", addr.ToString()); + CloseSocket(hSocket); + } + else + { + CNode* pnode = new CNode(hSocket, addr, "", true); + pnode->AddRef(); + pnode->fWhitelisted = whitelisted; + + { + LOCK(cs_vNodes); + vNodes.push_back(pnode); + } + } + } + } + + // + // Service each socket + // + vector vNodesCopy; + { + LOCK(cs_vNodes); + vNodesCopy = vNodes; + BOOST_FOREACH(CNode* pnode, vNodesCopy) + pnode->AddRef(); + } + BOOST_FOREACH(CNode* pnode, vNodesCopy) + { + boost::this_thread::interruption_point(); + + // + // Receive + // + if (pnode->hSocket == INVALID_SOCKET) + continue; + if (FD_ISSET(pnode->hSocket, &fdsetRecv) || FD_ISSET(pnode->hSocket, &fdsetError)) + { + TRY_LOCK(pnode->cs_vRecvMsg, lockRecv); + if (lockRecv) + { + { + // typical socket buffer is 8K-64K + char pchBuf[0x10000]; + int nBytes = recv(pnode->hSocket, pchBuf, sizeof(pchBuf), MSG_DONTWAIT); + if (nBytes > 0) + { + if (!pnode->ReceiveMsgBytes(pchBuf, nBytes)) + { +/* MCHN START */ + LogPrint("net","receive error\n"); +/* MCHN END */ + pnode->CloseSocketDisconnect(); + } + pnode->nLastRecv = GetTime(); + pnode->nRecvBytes += nBytes; + pnode->RecordBytesRecv(nBytes); + } + else if (nBytes == 0) + { + // socket closed gracefully + if (!pnode->fDisconnect) + LogPrint("net", "socket closed\n"); +/* MCHN START */ + else + LogPrint("net","socket closed, disconnect flag is set\n"); +/* MCHN END */ + pnode->CloseSocketDisconnect(); + } + else if (nBytes < 0) + { + // error + int nErr = WSAGetLastError(); + if (nErr != WSAEWOULDBLOCK && nErr != WSAEMSGSIZE && nErr != WSAEINTR && nErr != WSAEINPROGRESS) + { + if (!pnode->fDisconnect) + LogPrintf("socket recv error %s\n", NetworkErrorString(nErr)); +/* MCHN START */ + else + LogPrintf("socket recv error %s, disconnect flag is set\n", NetworkErrorString(nErr)); +/* MCHN END */ + pnode->CloseSocketDisconnect(); + } + } + } + } + } + + // + // Send + // + if (pnode->hSocket == INVALID_SOCKET) + continue; + if (FD_ISSET(pnode->hSocket, &fdsetSend)) + { + TRY_LOCK(pnode->cs_vSend, lockSend); + if (lockSend) + SocketSendData(pnode); + } + + // + // Inactivity checking + // + int64_t nTime = GetTime(); + if (nTime - pnode->nTimeConnected > 60) + { + if (pnode->nLastRecv == 0 || pnode->nLastSend == 0) + { + LogPrint("net", "socket no message in first 60 seconds, %d %d from %d\n", pnode->nLastRecv != 0, pnode->nLastSend != 0, pnode->id); + pnode->fDisconnect = true; + } + else if (nTime - pnode->nLastSend > TIMEOUT_INTERVAL) + { + LogPrintf("socket sending timeout: %is\n", nTime - pnode->nLastSend); + pnode->fDisconnect = true; + } + else if (nTime - pnode->nLastRecv > (pnode->nVersion > BIP0031_VERSION ? TIMEOUT_INTERVAL : 90*60)) + { + LogPrintf("socket receive timeout: %is\n", nTime - pnode->nLastRecv); + pnode->fDisconnect = true; + } + else if (pnode->nPingNonceSent && pnode->nPingUsecStart + TIMEOUT_INTERVAL * 1000000 < GetTimeMicros()) + { + LogPrintf("ping timeout: %fs\n", 0.000001 * (GetTimeMicros() - pnode->nPingUsecStart)); + pnode->fDisconnect = true; + } + } + } + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodesCopy) + pnode->Release(); + } + } +} + + + + + + + + + +#ifdef USE_UPNP +void ThreadMapPort() +{ + std::string port = strprintf("%u", GetListenPort()); + const char * multicastif = 0; + const char * minissdpdpath = 0; + struct UPNPDev * devlist = 0; + char lanaddr[64]; + +#ifndef UPNPDISCOVER_SUCCESS + /* miniupnpc 1.5 */ + devlist = upnpDiscover(2000, multicastif, minissdpdpath, 0); +#else + /* miniupnpc 1.6 */ + int error = 0; + devlist = upnpDiscover(2000, multicastif, minissdpdpath, 0, 0, &error); +#endif + + struct UPNPUrls urls; + struct IGDdatas data; + int r; + + r = UPNP_GetValidIGD(devlist, &urls, &data, lanaddr, sizeof(lanaddr)); + if (r == 1) + { + if (fDiscover) { + char externalIPAddress[40]; + r = UPNP_GetExternalIPAddress(urls.controlURL, data.first.servicetype, externalIPAddress); + if(r != UPNPCOMMAND_SUCCESS) + LogPrintf("UPnP: GetExternalIPAddress() returned %d\n", r); + else + { + if(externalIPAddress[0]) + { + LogPrintf("UPnP: ExternalIPAddress = %s\n", externalIPAddress); + AddLocal(CNetAddr(externalIPAddress), LOCAL_UPNP); + } + else + LogPrintf("UPnP: GetExternalIPAddress failed.\n"); + } + } + + string strDesc = "MultiChain " + FormatFullVersion(); + + try { + while (true) { +#ifndef UPNPDISCOVER_SUCCESS + /* miniupnpc 1.5 */ + r = UPNP_AddPortMapping(urls.controlURL, data.first.servicetype, + port.c_str(), port.c_str(), lanaddr, strDesc.c_str(), "TCP", 0); +#else + /* miniupnpc 1.6 */ + r = UPNP_AddPortMapping(urls.controlURL, data.first.servicetype, + port.c_str(), port.c_str(), lanaddr, strDesc.c_str(), "TCP", 0, "0"); +#endif + + if(r!=UPNPCOMMAND_SUCCESS) + LogPrintf("AddPortMapping(%s, %s, %s) failed with code %d (%s)\n", + port, port, lanaddr, r, strupnperror(r)); + else + LogPrintf("UPnP Port Mapping successful.\n");; + + MilliSleep(20*60*1000); // Refresh every 20 minutes + } + } + catch (boost::thread_interrupted) + { + r = UPNP_DeletePortMapping(urls.controlURL, data.first.servicetype, port.c_str(), "TCP", 0); + LogPrintf("UPNP_DeletePortMapping() returned : %d\n", r); + freeUPNPDevlist(devlist); devlist = 0; + FreeUPNPUrls(&urls); + throw; + } + } else { + LogPrintf("No valid UPnP IGDs found\n"); + freeUPNPDevlist(devlist); devlist = 0; + if (r != 0) + FreeUPNPUrls(&urls); + } +} + +void MapPort(bool fUseUPnP) +{ + static boost::thread* upnp_thread = NULL; + + if (fUseUPnP) + { + if (upnp_thread) { + upnp_thread->interrupt(); + upnp_thread->join(); + delete upnp_thread; + } + upnp_thread = new boost::thread(boost::bind(&TraceThread, "upnp", &ThreadMapPort)); + } + else if (upnp_thread) { + upnp_thread->interrupt(); + upnp_thread->join(); + delete upnp_thread; + upnp_thread = NULL; + } +} + +#else +void MapPort(bool) +{ + // Intentionally left blank. +} +#endif + + + + + + +void ThreadDNSAddressSeed() +{ + // goal: only query DNS seeds if address need is acute + if ((addrman.size() > 0) && + (!GetBoolArg("-forcednsseed", false))) { + MilliSleep(11 * 1000); + + LOCK(cs_vNodes); + if (vNodes.size() >= 2) { + LogPrintf("P2P peers available. Skipped DNS seeding.\n"); + return; + } + } + + const vector &vSeeds = Params().DNSSeeds(); + int found = 0; + + LogPrintf("Loading addresses from DNS seeds (could take a while)\n"); + + BOOST_FOREACH(const CDNSSeedData &seed, vSeeds) { + if (HaveNameProxy()) { + AddOneShot(seed.host); + } else { + vector vIPs; + vector vAdd; + if (LookupHost(seed.host.c_str(), vIPs)) + { + BOOST_FOREACH(CNetAddr& ip, vIPs) + { + int nOneDay = 24*3600; + CAddress addr = CAddress(CService(ip, Params().GetDefaultPort())); + addr.nTime = GetTime() - 3*nOneDay - GetRand(4*nOneDay); // use a random age between 3 and 7 days old + vAdd.push_back(addr); + found++; + } + } + addrman.Add(vAdd, CNetAddr(seed.name, true)); + } + } + + LogPrintf("%d addresses found from DNS seeds\n", found); +} + + + + + + + + + + + + +void DumpAddresses() +{ + int64_t nStart = GetTimeMillis(); + +/* MCHN START */ + if(mc_gState->m_NetworkParams->m_Status != MC_PRM_STATUS_EMPTY) + { +/* MCHN END */ + + CAddrDB adb; + adb.Write(addrman); + + LogPrint("net", "Flushed %d addresses to peers.dat %dms\n", + addrman.size(), GetTimeMillis() - nStart); +/* MCHN START */ + } +/* MCHN END */ +} + +void static ProcessOneShot() +{ + string strDest; + { + LOCK(cs_vOneShots); + if (vOneShots.empty()) + return; + strDest = vOneShots.front(); + vOneShots.pop_front(); + } + CAddress addr; + CSemaphoreGrant grant(*semOutbound, true); + if (grant) { + if (!OpenNetworkConnection(addr, &grant, strDest.c_str(), true)) + AddOneShot(strDest); + } +} + +void ThreadOpenConnections() +{ + // Connect to specific addresses + if (mapArgs.count("-connect") && mapMultiArgs["-connect"].size() > 0) + { + for (int64_t nLoop = 0;; nLoop++) + { + ProcessOneShot(); + BOOST_FOREACH(string strAddr, mapMultiArgs["-connect"]) + { + CAddress addr; + OpenNetworkConnection(addr, NULL, strAddr.c_str()); + for (int i = 0; i < 10 && i < nLoop; i++) + { + MilliSleep(500); + } + } + MilliSleep(500); + } + } + +/* MCHN START */ + + if(mc_gState->GetSeedNode()) // MCHN-TODO. Connection to seed node, find later how to disconnect + { + CAddress addr; + OpenNetworkConnection(addr, NULL, mc_gState->GetSeedNode()); + } + +/* MCHN END */ + + // Initiate network connections + int64_t nStart = GetTime(); + while (true) + { + ProcessOneShot(); + + MilliSleep(500); + + CSemaphoreGrant grant(*semOutbound); + boost::this_thread::interruption_point(); + + + + // Add seed nodes if DNS seeds are all down (an infrastructure attack?). + if (addrman.size() == 0 && (GetTime() - nStart > 60)) { + static bool done = false; + if (!done) { + LogPrintf("Adding fixed seed nodes as DNS doesn't seem to be available.\n"); + addrman.Add(Params().FixedSeeds(), CNetAddr("127.0.0.1")); + done = true; + } + } + + // + // Choose an address to connect to based on most recently seen + // + CAddress addrConnect; + + // Only connect out to one peer per network group (/16 for IPv4). + // Do this here so we don't have to critsect vNodes inside mapAddresses critsect. + int nOutbound = 0; + set > setConnected; + set setConnectedVerifiedAddresses; + set setConnectedFromAddresses; + set setConnectedToAddresses; + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) { + if (!pnode->fInbound) { + setConnected.insert(pnode->addr.GetGroup()); + nOutbound++; + } +/* MCHN START */ + if (pnode->fInbound) + { + if (((CNetAddr)pnode->addr) == (CNetAddr)pnode->addrFromVersion) + { + setConnectedVerifiedAddresses.insert(pnode->addrFromVersion.ToStringIPPort()); + } + setConnectedFromAddresses.insert(pnode->addrFromVersion.ToStringIPPort()); + } + else + { + setConnectedToAddresses.insert(pnode->addr.ToStringIPPort()); + } +/* MCHN END */ + } + } + + int64_t nANow = GetAdjustedTime(); + + int nTries = 0; + while (true) + { + // use an nUnkBias between 10 (no outgoing connections) and 90 (8 outgoing connections) + CAddress addr = addrman.Select(10 + min(nOutbound,8)*10); + + // if we selected an invalid address, restart +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if (!addr.IsValid() || (IsLocal(addr) && (addr.GetPort() == GetListenPort()))) + { + break; + } + MilliSleep(100); + } + else + { +/* MCHN END */ + if (!addr.IsValid() || setConnected.count(addr.GetGroup()) || IsLocal(addr)) + break; +/* MCHN START */ + } +/* MCHN END */ + + // If we didn't find an appropriate destination after trying 100 addresses fetched from addrman, + // stop this loop, and let the outer loop run again (which sleeps, adds seed nodes, recalculates + // already-connected network ranges, ...) before trying new addrman addresses. + nTries++; + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if (nTries > 20) + break; + } + else + { + if (nTries > 100) + break; + } + + if (IsLimited(addr)) + continue; + + // only consider very recently tried nodes after 30 failed attempts +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { +/* MCHN END */ + if (nANow - addr.nLastTry < 600 && nTries < 30) + continue; +/* MCHN START */ + } +/* MCHN END */ + + + // do not allow non-default ports, unless after 50 invalid addresses selected already +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { +/* MCHN END */ + if (addr.GetPort() != Params().GetDefaultPort() && nTries < 50) + { + continue; + } +/* MCHN START */ + } + + + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + + if(setConnectedVerifiedAddresses.count(addr.ToStringIPPort())) + { + continue; + } + if(setConnectedToAddresses.count(addr.ToStringIPPort())) + { + continue; + } + if (nANow - addr.nLastTry < 600) + { + if(setConnectedFromAddresses.count(addr.ToStringIPPort())) + { + continue; + } + } + } + +/* MCHN END */ + + addrConnect = addr; + break; + } + +/* MCHN START */ + if (addrConnect.IsValid()) + { + OpenNetworkConnection(addrConnect, &grant); + } +/* MCHN END */ + } +} + +void ThreadOpenAddedConnections() +{ + { + LOCK(cs_vAddedNodes); + vAddedNodes = mapMultiArgs["-addnode"]; + } + + if (HaveNameProxy()) { + while(true) { + list lAddresses(0); + { + LOCK(cs_vAddedNodes); + BOOST_FOREACH(string& strAddNode, vAddedNodes) + lAddresses.push_back(strAddNode); + } + BOOST_FOREACH(string& strAddNode, lAddresses) { + CAddress addr; + CSemaphoreGrant grant(*semOutbound); + OpenNetworkConnection(addr, &grant, strAddNode.c_str()); + MilliSleep(500); + } + MilliSleep(120000); // Retry every 2 minutes + } + } + + for (unsigned int i = 0; true; i++) + { + list lAddresses(0); + { + LOCK(cs_vAddedNodes); + BOOST_FOREACH(string& strAddNode, vAddedNodes) + lAddresses.push_back(strAddNode); + } + + list > lservAddressesToAdd(0); + BOOST_FOREACH(string& strAddNode, lAddresses) + { + vector vservNode(0); + if(Lookup(strAddNode.c_str(), vservNode, Params().GetDefaultPort(), fNameLookup, 0)) + { + lservAddressesToAdd.push_back(vservNode); + { + LOCK(cs_setservAddNodeAddresses); + BOOST_FOREACH(CService& serv, vservNode) + setservAddNodeAddresses.insert(serv); + } + } + } + // Attempt to connect to each IP for each addnode entry until at least one is successful per addnode entry + // (keeping in mind that addnode entries can have many IPs if fNameLookup) + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + for (list >::iterator it = lservAddressesToAdd.begin(); it != lservAddressesToAdd.end(); it++) + BOOST_FOREACH(CService& addrNode, *(it)) + if (pnode->addr == addrNode) + { + it = lservAddressesToAdd.erase(it); + it--; + break; + } + } + BOOST_FOREACH(vector& vserv, lservAddressesToAdd) + { + CSemaphoreGrant grant(*semOutbound); + OpenNetworkConnection(CAddress(vserv[i % vserv.size()]), &grant); + MilliSleep(500); + } + MilliSleep(120000); // Retry every 2 minutes + } +} + +// if successful, this moves the passed grant to the constructed node +bool OpenNetworkConnection(const CAddress& addrConnect, CSemaphoreGrant *grantOutbound, const char *pszDest, bool fOneShot) +{ + // + // Initiate outbound network connection + // + boost::this_thread::interruption_point(); + if (!pszDest) { +// if (IsLocal(addrConnect) || + if ((IsLocal(addrConnect) && (addrConnect.GetPort() == GetListenPort())) || +// FindNode((CNetAddr)addrConnect) || CNode::IsBanned(addrConnect) || + (FindNode((CNetAddr)addrConnect) && !IsLocal(addrConnect)) || CNode::IsBanned(addrConnect) || + FindNode(addrConnect.ToStringIPPort())) + { + LogPrint("net","net: Node found: %s\n",addrConnect.ToStringIPPort().c_str()); + return false; + } + } else if (FindNode(pszDest)) + { + LogPrint("net","net: Node found: %s\n",pszDest); + return false; + } +/* MCHN START */ + if(pszDest) + { + LogPrint("net","net: Trying to connect to %s (by address)\n",pszDest); + } + else + { + LogPrint("net","net: Trying to connect to %s\n",addrConnect.ToStringIPPort().c_str()); + } +/* MCHN END */ + + CNode* pnode = ConnectNode(addrConnect, pszDest); + boost::this_thread::interruption_point(); + + if (!pnode) + { +/* MCHN START */ + LogPrint("net","net: Connection not established\n"); +/* MCHN END */ + return false; + } + if (grantOutbound) + grantOutbound->MoveTo(pnode->grantOutbound); + pnode->fNetworkNode = true; + if (fOneShot) + pnode->fOneShot = true; + + LogPrint("net","net: Connection to %s established\n",addrConnect.ToStringIPPort().c_str()); + return true; +} + + +void ThreadMessageHandler() +{ + SetThreadPriority(THREAD_PRIORITY_BELOW_NORMAL); + while (true) + { + vector vNodesCopy; + { + LOCK(cs_vNodes); + vNodesCopy = vNodes; + BOOST_FOREACH(CNode* pnode, vNodesCopy) { + pnode->AddRef(); + } + } + + // Poll the connected nodes for messages + CNode* pnodeTrickle = NULL; + if (!vNodesCopy.empty()) + pnodeTrickle = vNodesCopy[GetRand(vNodesCopy.size())]; + + bool fSleep = true; + + BOOST_FOREACH(CNode* pnode, vNodesCopy) + { + if (pnode->fDisconnect) + continue; + + // Receive messages + { + TRY_LOCK(pnode->cs_vRecvMsg, lockRecv); + if (lockRecv) + { + if (!g_signals.ProcessMessages(pnode)) + { +/* MCHN START */ + LogPrint("net","socket closed because of error in message processing\n"); +/* MCHN END */ + pnode->CloseSocketDisconnect(); + } + if (pnode->nSendSize < SendBufferSize()) + { + if (!pnode->vRecvGetData.empty() || (!pnode->vRecvMsg.empty() && pnode->vRecvMsg[0].complete())) + { + fSleep = false; + } + } + } + } + boost::this_thread::interruption_point(); + +/* MCHN START */ +/* Avoid communication with the node before connection is properly established with verackack and the network paramset is valid*/ + if (pnode->fDisconnect) + continue; + + if((mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_VALID) && pnode->fParameterSetVerified) + { +/* MCHN END */ + // Send messages + { + TRY_LOCK(pnode->cs_vSend, lockSend); + if (lockSend) + g_signals.SendMessages(pnode, pnode == pnodeTrickle); + } + boost::this_thread::interruption_point(); +/* MCHN START */ + } + +/* MCHN END */ + } + + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodesCopy) + pnode->Release(); + } + + if (fSleep) + MilliSleep(100); + } +} + + + + + + +bool BindListenPort(const CService &addrBind, string& strError, bool fWhitelisted) +{ + strError = ""; + int nOne = 1; + + // Create socket for listening for incoming connections + struct sockaddr_storage sockaddr; + socklen_t len = sizeof(sockaddr); + if (!addrBind.GetSockAddr((struct sockaddr*)&sockaddr, &len)) + { + strError = strprintf("Error: Bind address family for %s not supported", addrBind.ToString()); + LogPrintf("%s\n", strError); + return false; + } + + SOCKET hListenSocket = socket(((struct sockaddr*)&sockaddr)->sa_family, SOCK_STREAM, IPPROTO_TCP); + if (hListenSocket == INVALID_SOCKET) + { + strError = strprintf("Error: Couldn't open socket for incoming connections (socket returned error %s)", NetworkErrorString(WSAGetLastError())); + LogPrintf("%s\n", strError); + return false; + } + +#ifndef WIN32 +#ifdef SO_NOSIGPIPE + // Different way of disabling SIGPIPE on BSD + setsockopt(hListenSocket, SOL_SOCKET, SO_NOSIGPIPE, (void*)&nOne, sizeof(int)); +#endif + // Allow binding if the port is still in TIME_WAIT state after + // the program was closed and restarted. Not an issue on windows! + setsockopt(hListenSocket, SOL_SOCKET, SO_REUSEADDR, (void*)&nOne, sizeof(int)); +#endif + + // Set to non-blocking, incoming connections will also inherit this + if (!SetSocketNonBlocking(hListenSocket, true)) { + strError = strprintf("BindListenPort: Setting listening socket to non-blocking failed, error %s\n", NetworkErrorString(WSAGetLastError())); + LogPrintf("%s\n", strError); + return false; + } + + // some systems don't have IPV6_V6ONLY but are always v6only; others do have the option + // and enable it by default or not. Try to enable it, if possible. + if (addrBind.IsIPv6()) { +#ifdef IPV6_V6ONLY +#ifdef WIN32 + setsockopt(hListenSocket, IPPROTO_IPV6, IPV6_V6ONLY, (const char*)&nOne, sizeof(int)); +#else + setsockopt(hListenSocket, IPPROTO_IPV6, IPV6_V6ONLY, (void*)&nOne, sizeof(int)); +#endif +#endif +#ifdef WIN32 + int nProtLevel = PROTECTION_LEVEL_UNRESTRICTED; + setsockopt(hListenSocket, IPPROTO_IPV6, IPV6_PROTECTION_LEVEL, (const char*)&nProtLevel, sizeof(int)); +#endif + } + + if (::bind(hListenSocket, (struct sockaddr*)&sockaddr, len) == SOCKET_ERROR) + { + int nErr = WSAGetLastError(); + if (nErr == WSAEADDRINUSE) + strError = strprintf(_("Unable to bind to %s on this computer. MultiChain Core is probably already running."), addrBind.ToString()); + else + strError = strprintf(_("Unable to bind to %s on this computer (bind returned error %s)"), addrBind.ToString(), NetworkErrorString(nErr)); + LogPrintf("%s\n", strError); + CloseSocket(hListenSocket); + return false; + } + LogPrintf("Bound to %s\n", addrBind.ToString()); + + // Listen for incoming connections + if (listen(hListenSocket, SOMAXCONN) == SOCKET_ERROR) + { + strError = strprintf(_("Error: Listening for incoming connections failed (listen returned error %s)"), NetworkErrorString(WSAGetLastError())); + LogPrintf("%s\n", strError); + CloseSocket(hListenSocket); + return false; + } + + vhListenSocket.push_back(ListenSocket(hListenSocket, fWhitelisted)); + + if (addrBind.IsRoutable() && fDiscover && !fWhitelisted) + AddLocal(addrBind, LOCAL_BIND); + + return true; +} + +void static Discover(boost::thread_group& threadGroup) +{ + if (!fDiscover) + return; + +#ifdef WIN32 + // Get local host IP + char pszHostName[256] = ""; + if (gethostname(pszHostName, sizeof(pszHostName)) != SOCKET_ERROR) + { + vector vaddr; + if (LookupHost(pszHostName, vaddr)) + { + BOOST_FOREACH (const CNetAddr &addr, vaddr) + { + if (AddLocal(addr, LOCAL_IF)) + LogPrintf("%s: %s - %s\n", __func__, pszHostName, addr.ToString()); + } + } + } +#else + // Get local host ip + struct ifaddrs* myaddrs; + if (getifaddrs(&myaddrs) == 0) + { + for (struct ifaddrs* ifa = myaddrs; ifa != NULL; ifa = ifa->ifa_next) + { + if (ifa->ifa_addr == NULL) continue; + if ((ifa->ifa_flags & IFF_UP) == 0) continue; + if (strcmp(ifa->ifa_name, "lo") == 0) continue; + if (strcmp(ifa->ifa_name, "lo0") == 0) continue; + if (ifa->ifa_addr->sa_family == AF_INET) + { + struct sockaddr_in* s4 = (struct sockaddr_in*)(ifa->ifa_addr); + CNetAddr addr(s4->sin_addr); + if (AddLocal(addr, LOCAL_IF)) + LogPrintf("%s: IPv4 %s: %s\n", __func__, ifa->ifa_name, addr.ToString()); + } + else if (ifa->ifa_addr->sa_family == AF_INET6) + { + struct sockaddr_in6* s6 = (struct sockaddr_in6*)(ifa->ifa_addr); + CNetAddr addr(s6->sin6_addr); + if (AddLocal(addr, LOCAL_IF)) + LogPrintf("%s: IPv6 %s: %s\n", __func__, ifa->ifa_name, addr.ToString()); + } + } + freeifaddrs(myaddrs); + } +#endif +} + + +void StartNode(boost::thread_group& threadGroup) +{ + uiInterface.InitMessage(_("Loading addresses...")); + // Load addresses for peers.dat + int64_t nStart = GetTimeMillis(); + { +/* MCHN START */ + if(mc_gState->m_NetworkParams->m_Status != MC_PRM_STATUS_EMPTY) + { +/* MCHN END */ + CAddrDB adb; + if (!adb.Read(addrman)) + LogPrintf("Invalid or missing peers.dat; recreating\n"); +/* MCHN START */ + } +/* MCHN END */ + } + LogPrintf("Loaded %i addresses from peers.dat %dms\n", + addrman.size(), GetTimeMillis() - nStart); + fAddressesInitialized = true; + + if (semOutbound == NULL) { + // initialize semaphore + int nMaxOutbound = min(MAX_OUTBOUND_CONNECTIONS, nMaxConnections); + semOutbound = new CSemaphore(nMaxOutbound); + } + + if (pnodeLocalHost == NULL) + pnodeLocalHost = new CNode(INVALID_SOCKET, CAddress(CService("127.0.0.1", 0), nLocalServices)); + + Discover(threadGroup); + + // + // Start threads + // + + if (!GetBoolArg("-dnsseed", true)) + LogPrintf("DNS seeding disabled\n"); + else + threadGroup.create_thread(boost::bind(&TraceThread, "dnsseed", &ThreadDNSAddressSeed)); + + // Map ports with UPnP + MapPort(GetBoolArg("-upnp", DEFAULT_UPNP)); + + // Send and receive from sockets, accept connections + threadGroup.create_thread(boost::bind(&TraceThread, "net", &ThreadSocketHandler)); + + // Initiate outbound connections from -addnode + threadGroup.create_thread(boost::bind(&TraceThread, "addcon", &ThreadOpenAddedConnections)); + + // Initiate outbound connections + threadGroup.create_thread(boost::bind(&TraceThread, "opencon", &ThreadOpenConnections)); + + // Process messages + threadGroup.create_thread(boost::bind(&TraceThread, "msghand", &ThreadMessageHandler)); + + // Dump network addresses + threadGroup.create_thread(boost::bind(&LoopForever, "dumpaddr", &DumpAddresses, DUMP_ADDRESSES_INTERVAL * 1000)); +} + +bool StopNode() +{ + LogPrintf("StopNode()\n"); + MapPort(false); + if (semOutbound) + for (int i=0; ipost(); + + if (fAddressesInitialized) + { + DumpAddresses(); + fAddressesInitialized = false; + } + +/* MCHN START */ + LogPrintf("Node stopped\n"); +/* MCHN END */ + return true; +} + +class CNetCleanup +{ +public: + CNetCleanup() {} + + ~CNetCleanup() + { + // Close sockets + BOOST_FOREACH(CNode* pnode, vNodes) + if (pnode->hSocket != INVALID_SOCKET) + CloseSocket(pnode->hSocket); + BOOST_FOREACH(ListenSocket& hListenSocket, vhListenSocket) + if (hListenSocket.socket != INVALID_SOCKET) + if (!CloseSocket(hListenSocket.socket)) + LogPrintf("CloseSocket(hListenSocket) failed with error %s\n", NetworkErrorString(WSAGetLastError())); + + // clean up some globals (to help leak detection) + BOOST_FOREACH(CNode *pnode, vNodes) + delete pnode; + BOOST_FOREACH(CNode *pnode, vNodesDisconnected) + delete pnode; + vNodes.clear(); + vNodesDisconnected.clear(); + vhListenSocket.clear(); + delete semOutbound; + semOutbound = NULL; + delete pnodeLocalHost; + pnodeLocalHost = NULL; + +#ifdef WIN32 + // Shutdown Windows Sockets + WSACleanup(); +#endif + } +} +instance_of_cnetcleanup; + + + + + + + +void RelayTransaction(const CTransaction& tx) +{ + CDataStream ss(SER_NETWORK, PROTOCOL_VERSION); + ss.reserve(10000); + ss << tx; + RelayTransaction(tx, ss); +} + +void RelayTransaction(const CTransaction& tx, const CDataStream& ss) +{ + CInv inv(MSG_TX, tx.GetHash()); + { + LOCK(cs_mapRelay); + // Expire old relay messages + while (!vRelayExpiration.empty() && vRelayExpiration.front().first < GetTime()) + { + mapRelay.erase(vRelayExpiration.front().second); + vRelayExpiration.pop_front(); + } + + // Save original serialized message so newer versions are preserved + mapRelay.insert(std::make_pair(inv, ss)); + vRelayExpiration.push_back(std::make_pair(GetTime() + 15 * 60, inv)); + } + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + { + if(!pnode->fRelayTxes) + continue; + LOCK(pnode->cs_filter); + if (pnode->pfilter) + { + if (pnode->pfilter->IsRelevantAndUpdate(tx)) + pnode->PushInventory(inv); + } else + pnode->PushInventory(inv); + } +} + +void CNode::RecordBytesRecv(uint64_t bytes) +{ + LOCK(cs_totalBytesRecv); + nTotalBytesRecv += bytes; +} + +void CNode::RecordBytesSent(uint64_t bytes) +{ + LOCK(cs_totalBytesSent); + nTotalBytesSent += bytes; +} + +uint64_t CNode::GetTotalBytesRecv() +{ + LOCK(cs_totalBytesRecv); + return nTotalBytesRecv; +} + +uint64_t CNode::GetTotalBytesSent() +{ + LOCK(cs_totalBytesSent); + return nTotalBytesSent; +} + +void CNode::Fuzz(int nChance) +{ + if (!fSuccessfullyConnected) return; // Don't fuzz initial handshake + if (GetRand(nChance) != 0) return; // Fuzz 1 of every nChance messages + + switch (GetRand(3)) + { + case 0: + // xor a random byte with a random value: + if (!ssSend.empty()) { + CDataStream::size_type pos = GetRand(ssSend.size()); + ssSend[pos] ^= (unsigned char)(GetRand(256)); + } + break; + case 1: + // delete a random byte: + if (!ssSend.empty()) { + CDataStream::size_type pos = GetRand(ssSend.size()); + ssSend.erase(ssSend.begin()+pos); + } + break; + case 2: + // insert a random byte at a random position + { + CDataStream::size_type pos = GetRand(ssSend.size()); + char ch = (char)GetRand(256); + ssSend.insert(ssSend.begin()+pos, ch); + } + break; + } + // Chance of more than one change half the time: + // (more changes exponentially less likely): + Fuzz(2); +} + +// +// CAddrDB +// + +CAddrDB::CAddrDB() +{ + pathAddr = GetDataDir() / "peers.dat"; +} + +bool CAddrDB::Write(const CAddrMan& addr) +{ + // Generate random temporary filename + unsigned short randv = 0; + GetRandBytes((unsigned char*)&randv, sizeof(randv)); + std::string tmpfn = strprintf("peers.dat.%04x", randv); + + // serialize addresses, checksum data up to that point, then append csum + CDataStream ssPeers(SER_DISK, CLIENT_VERSION); + ssPeers << FLATDATA(Params().MessageStart()); + ssPeers << addr; + uint256 hash = Hash(ssPeers.begin(), ssPeers.end()); + ssPeers << hash; + + // open temp output file, and associate with CAutoFile + boost::filesystem::path pathTmp = GetDataDir() / tmpfn; + FILE *file = fopen(pathTmp.string().c_str(), "wb"); + CAutoFile fileout(file, SER_DISK, CLIENT_VERSION); + if (fileout.IsNull()) + return error("%s : Failed to open file %s", __func__, pathTmp.string()); + + // Write and commit header, data + try { + fileout << ssPeers; + } + catch (std::exception &e) { + return error("%s : Serialize or I/O error - %s", __func__, e.what()); + } + FileCommit(fileout.Get()); + fileout.fclose(); + + // replace existing peers.dat, if any, with new peers.dat.XXXX + if (!RenameOver(pathTmp, pathAddr)) + return error("%s : Rename-into-place failed", __func__); + + return true; +} + +bool CAddrDB::Read(CAddrMan& addr) +{ + // open input file, and associate with CAutoFile + FILE *file = fopen(pathAddr.string().c_str(), "rb"); + CAutoFile filein(file, SER_DISK, CLIENT_VERSION); + if (filein.IsNull()) + return error("%s : Failed to open file %s", __func__, pathAddr.string()); + + // use file size to size memory buffer + int fileSize = boost::filesystem::file_size(pathAddr); + int dataSize = fileSize - sizeof(uint256); + // Don't try to resize to a negative number if file is small + if (dataSize < 0) + dataSize = 0; + vector vchData; + vchData.resize(dataSize); + uint256 hashIn; + + // read data and checksum from file + try { + filein.read((char *)&vchData[0], dataSize); + filein >> hashIn; + } + catch (std::exception &e) { + return error("%s : Deserialize or I/O error - %s", __func__, e.what()); + } + filein.fclose(); + + CDataStream ssPeers(vchData, SER_DISK, CLIENT_VERSION); + + // verify stored checksum matches input data + uint256 hashTmp = Hash(ssPeers.begin(), ssPeers.end()); + if (hashIn != hashTmp) + return error("%s : Checksum mismatch, data corrupted", __func__); + + unsigned char pchMsgTmp[4]; + try { + // de-serialize file header (network specific magic number) and .. + ssPeers >> FLATDATA(pchMsgTmp); + + // ... verify the network matches ours + if (memcmp(pchMsgTmp, Params().MessageStart(), sizeof(pchMsgTmp))) + return error("%s : Invalid network magic number", __func__); + + // de-serialize address data into one CAddrMan object + ssPeers >> addr; + } + catch (std::exception &e) { + return error("%s : Deserialize or I/O error - %s", __func__, e.what()); + } + + return true; +} + +unsigned int ReceiveFloodSize() { return 1000*GetArg("-maxreceivebuffer", 5*1000); } +unsigned int SendBufferSize() { return 1000*GetArg("-maxsendbuffer", 1*1000); } + +CNode::CNode(SOCKET hSocketIn, CAddress addrIn, std::string addrNameIn, bool fInboundIn) : ssSend(SER_NETWORK, INIT_PROTO_VERSION), setAddrKnown(5000) +{ + nServices = 0; + hSocket = hSocketIn; + nRecvVersion = INIT_PROTO_VERSION; + nLastSend = 0; + nLastRecv = 0; + nSendBytes = 0; + nRecvBytes = 0; + nTimeConnected = GetTime(); + addr = addrIn; + addrName = addrNameIn == "" ? addr.ToStringIPPort() : addrNameIn; + nVersion = 0; + strSubVer = ""; + fWhitelisted = false; + fOneShot = false; + fClient = false; // set by version message + fInbound = fInboundIn; + fNetworkNode = false; + fSuccessfullyConnected = false; + fDisconnect = false; + nRefCount = 0; + nSendSize = 0; + nSendOffset = 0; + hashContinue = 0; + nStartingHeight = -1; + fGetAddr = false; + fRelayTxes = false; + setInventoryKnown.max_size(SendBufferSize() / 1000); + pfilter = new CBloomFilter(); + nPingNonceSent = 0; + nPingUsecStart = 0; + nPingUsecTime = 0; + fPingQueued = false; + +/* MCHN START */ + fDefaultMessageStart=false; + fVerackackReceived=false; + fVerackackSent=false; + fParameterSetVerified=false; + fSyncedOnce=false; + fCanConnectLocal=false; + fCanConnectRemote=false; +/* MCHN END */ + + { + LOCK(cs_nLastNodeId); + id = nLastNodeId++; + } + + if (fLogIPs) + LogPrint("net", "Added connection to %s peer=%d\n", addrName, id); + else + LogPrint("net", "Added connection peer=%d\n", id); + + // Be shy and don't send version until we hear + if (hSocket != INVALID_SOCKET && !fInbound) + PushVersion(); + + GetNodeSignals().InitializeNode(GetId(), this); +} + +CNode::~CNode() +{ + CloseSocket(hSocket); + + if (pfilter) + delete pfilter; + + GetNodeSignals().FinalizeNode(GetId()); +} + +void CNode::AskFor(const CInv& inv) +{ + if (mapAskFor.size() > MAPASKFOR_MAX_SZ) + return; +/* MCHN START */ + if(mc_gState->m_NodePausedState & MC_NPS_INCOMING) + { + if(inv.type != MSG_BLOCK) + { + return; + } + } +/* MCHN END */ + // We're using mapAskFor as a priority queue, + // the key is the earliest time the request can be sent + int64_t nRequestTime; + limitedmap::const_iterator it = mapAlreadyAskedFor.find(inv); + if (it != mapAlreadyAskedFor.end()) + nRequestTime = it->second; + else + nRequestTime = 0; + LogPrint("net", "askfor %s %d (%s) peer=%d\n", inv.ToString(), nRequestTime, DateTimeStrFormat("%H:%M:%S", nRequestTime/1000000), id); + + // Make sure not to reuse time indexes to keep things in the same order + int64_t nNow = GetTimeMicros() - 1000000; + static int64_t nLastTime; + ++nLastTime; + nNow = std::max(nNow, nLastTime); + nLastTime = nNow; + // Each retry is 2 minutes after the last + nRequestTime = std::max(nRequestTime + 2 * 60 * 1000000, nNow); + if (it != mapAlreadyAskedFor.end()) + mapAlreadyAskedFor.update(it, nRequestTime); + else + mapAlreadyAskedFor.insert(std::make_pair(inv, nRequestTime)); + mapAskFor.insert(std::make_pair(nRequestTime, inv)); +} + +void CNode::BeginMessage(const char* pszCommand) EXCLUSIVE_LOCK_FUNCTION(cs_vSend) +{ + ENTER_CRITICAL_SECTION(cs_vSend); + assert(ssSend.size() == 0); + ssSend << CMessageHeader(pszCommand, 0); + LogPrint("net", "sending: %s ", pszCommand); + LogPrint("mchnminor","mchn: SEND: %s\n",pszCommand); +} + +void CNode::AbortMessage() UNLOCK_FUNCTION(cs_vSend) +{ + ssSend.clear(); + + LEAVE_CRITICAL_SECTION(cs_vSend); + + LogPrint("net", "(aborted)\n"); +} + +void CNode::EndMessage() UNLOCK_FUNCTION(cs_vSend) +{ + // The -*messagestest options are intentionally not documented in the help message, + // since they are only used during development to debug the networking code and are + // not intended for end-users. + if (mapArgs.count("-dropmessagestest") && GetRand(GetArg("-dropmessagestest", 2)) == 0) + { + LogPrint("net", "dropmessages DROPPING SEND MESSAGE\n"); + AbortMessage(); + return; + } + if (mapArgs.count("-fuzzmessagestest")) + Fuzz(GetArg("-fuzzmessagestest", 10)); + + if (ssSend.size() == 0) + return; + + // Set the size + unsigned int nSize = ssSend.size() - CMessageHeader::HEADER_SIZE; + memcpy((char*)&ssSend[CMessageHeader::MESSAGE_SIZE_OFFSET], &nSize, sizeof(nSize)); + + // Set the checksum + uint256 hash = Hash(ssSend.begin() + CMessageHeader::HEADER_SIZE, ssSend.end()); + unsigned int nChecksum = 0; + memcpy(&nChecksum, &hash, sizeof(nChecksum)); + assert(ssSend.size () >= CMessageHeader::CHECKSUM_OFFSET + sizeof(nChecksum)); + memcpy((char*)&ssSend[CMessageHeader::CHECKSUM_OFFSET], &nChecksum, sizeof(nChecksum)); + + LogPrint("net", "(%d bytes) peer=%d\n", nSize, id); + + std::deque::iterator it = vSendMsg.insert(vSendMsg.end(), CSerializeData()); + ssSend.GetAndClear(*it); + nSendSize += (*it).size(); + + // If write queue empty, attempt "optimistic write" + if (it == vSendMsg.begin()) + SocketSendData(this); + + LEAVE_CRITICAL_SECTION(cs_vSend); +} + +/* MCHN START */ + +int mc_QuerySeed(boost::thread_group& threadGroup,const char *seedAddr) +{ + int err; +// AddOneShot(seedAddr); + StartNode(threadGroup); + + unsigned int StartTime=mc_TimeNowAsUInt(); + + LogPrint("mchn","mchn: Sending query to seed node %s\n",seedAddr); + + err=MC_ERR_NOERROR; + + while( (mc_gState->m_NetworkState != MC_NTS_SEED_READY) && (mc_gState->m_NetworkState != MC_NTS_SEED_NO_PARAMS)) // MCHN-TODO add timeout + { + __US_Sleep(1); + if(mc_TimeNowAsUInt() - StartTime > 10) + { + LogPrintf("mchn: Could not establish connection with seed node - timeout\n"); + mc_gState->m_NetworkState=MC_NTS_SEED_READY; + err=MC_ERR_CONNECTION_ERROR; + } + } + + if(err) + { + mc_gState->m_NetworkState=MC_NTS_NOT_READY; + } + + LogPrint("mchn","mchn: Query completed, waiting for seed node to disconnect\n"); + + StartTime=mc_TimeNowAsUInt(); + bool fCheckDisconnect=true; + while(fCheckDisconnect) + { + { + LOCK(cs_vNodes); + if(vNodes.size() == 0) + { + LogPrint("mchn","mchn: Successfully disconnected from seed node\n"); + fCheckDisconnect=false; + } + if(fCheckDisconnect) + { + __US_Sleep(1); + if(mc_TimeNowAsUInt() - StartTime > 10) + { + LogPrint("mchn","mchn: Could not disconnect from seed node - timeout\n"); + fCheckDisconnect=false; + err=MC_ERR_CONNECTION_ERROR; + } + + } + } + } + + threadGroup.interrupt_all(); + threadGroup.join_all(); + + return err; +} + +/* MCHN END */ diff --git a/src/net/net.h b/src/net/net.h new file mode 100644 index 00000000..af4d1f90 --- /dev/null +++ b/src/net/net.h @@ -0,0 +1,704 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_NET_H +#define BITCOIN_NET_H + +#include "structs/bloom.h" +#include "utils/compat.h" +#include "structs/hash.h" +#include "structs/limitedmap.h" +#include "utils/mruset.h" +#include "net/netbase.h" +#include "protocol/netprotocol.h" +#include "utils/random.h" +#include "utils/streams.h" +#include "utils/sync.h" +#include "structs/uint256.h" +#include "utils/utilstrencodings.h" + +/* MCHN START */ +#include "keys/pubkey.h" +/* MCHN END */ + +#include +#include + +#ifndef WIN32 +#include +#endif + +#include +#include +#include + +class CAddrMan; +class CBlockIndex; +class CNode; + +namespace boost { + class thread_group; +} // namespace boost + +/** Time between pings automatically sent out for latency probing and keepalive (in seconds). */ +static const int PING_INTERVAL = 2 * 60; +/** Time after which to disconnect, after waiting for a ping response (or inactivity). */ +static const int TIMEOUT_INTERVAL = 20 * 60; +/** The maximum number of entries in an 'inv' protocol message */ +static const unsigned int MAX_INV_SZ = 50000; +/** The maximum number of new addresses to accumulate before announcing. */ +static const unsigned int MAX_ADDR_TO_SEND = 1000; +/** -listen default */ +static const bool DEFAULT_LISTEN = true; +/** -upnp default */ +#ifdef USE_UPNP +static const bool DEFAULT_UPNP = USE_UPNP; +#else +static const bool DEFAULT_UPNP = false; +#endif +/** The maximum number of entries in mapAskFor */ +static const size_t MAPASKFOR_MAX_SZ = MAX_INV_SZ; + +unsigned int ReceiveFloodSize(); +unsigned int SendBufferSize(); + +void AddOneShot(std::string strDest); +bool RecvLine(SOCKET hSocket, std::string& strLine); +void AddressCurrentlyConnected(const CService& addr); +CNode* FindNode(const CNetAddr& ip); +CNode* FindNode(const std::string& addrName); +CNode* FindNode(const CService& ip); +CNode* ConnectNode(CAddress addrConnect, const char *pszDest = NULL); +bool OpenNetworkConnection(const CAddress& addrConnect, CSemaphoreGrant *grantOutbound = NULL, const char *strDest = NULL, bool fOneShot = false); +void MapPort(bool fUseUPnP); +unsigned short GetListenPort(); +bool BindListenPort(const CService &bindAddr, std::string& strError, bool fWhitelisted = false); +void StartNode(boost::thread_group& threadGroup); +bool StopNode(); +void SocketSendData(CNode *pnode); +int mc_QuerySeed(boost::thread_group& threadGroup,const char *seedAddr); + + +typedef int NodeId; + +// Signals for message handling +struct CNodeSignals +{ + boost::signals2::signal GetHeight; + boost::signals2::signal ProcessMessages; + boost::signals2::signal SendMessages; + boost::signals2::signal InitializeNode; + boost::signals2::signal FinalizeNode; +}; + + +CNodeSignals& GetNodeSignals(); + + +enum +{ + LOCAL_NONE, // unknown + LOCAL_IF, // address a local interface listens on + LOCAL_BIND, // address explicit bound to + LOCAL_UPNP, // address reported by UPnP + LOCAL_MANUAL, // address explicitly specified (-externalip=) + + LOCAL_MAX +}; + +bool IsPeerAddrLocalGood(CNode *pnode); +void AdvertizeLocal(CNode *pnode); +void SetLimited(enum Network net, bool fLimited = true); +bool IsLimited(enum Network net); +bool IsLimited(const CNetAddr& addr); +bool AddLocal(const CService& addr, int nScore = LOCAL_NONE); +bool AddLocal(const CNetAddr& addr, int nScore = LOCAL_NONE); +bool SeenLocal(const CService& addr); +bool IsLocal(const CService& addr); +bool GetLocal(CService &addr, const CNetAddr *paddrPeer = NULL); +bool IsReachable(enum Network net); +bool IsReachable(const CNetAddr &addr); +void SetReachable(enum Network net, bool fFlag = true); +CAddress GetLocalAddress(const CNetAddr *paddrPeer = NULL); + + +extern bool fDiscover; +extern bool fListen; +extern uint64_t nLocalServices; +extern uint64_t nLocalHostNonce; +extern CAddrMan addrman; +extern int nMaxConnections; + +extern std::vector vNodes; +extern CCriticalSection cs_vNodes; +extern std::map mapRelay; +extern std::deque > vRelayExpiration; +extern CCriticalSection cs_mapRelay; +extern limitedmap mapAlreadyAskedFor; + +extern std::vector vAddedNodes; +extern CCriticalSection cs_vAddedNodes; + +extern NodeId nLastNodeId; +extern CCriticalSection cs_nLastNodeId; + +struct LocalServiceInfo { + int nScore; + int nPort; +}; + +extern CCriticalSection cs_mapLocalHost; +extern std::map mapLocalHost; + +class CNodeStats +{ +public: + NodeId nodeid; + uint64_t nServices; + int64_t nLastSend; + int64_t nLastRecv; + int64_t nTimeConnected; + std::string addrName; + int nVersion; + std::string cleanSubVer; + bool fInbound; + int nStartingHeight; + uint64_t nSendBytes; + uint64_t nRecvBytes; + bool fWhitelisted; + double dPingTime; + double dPingWait; + std::string addrLocal; +/* MCHN START */ + CKeyID kAddrRemote; + CKeyID kAddrLocal; + bool fSuccessfullyConnected; +/* MCHN END */ +}; + + + + +class CNetMessage { +public: + bool in_data; // parsing header (false) or data (true) + + CDataStream hdrbuf; // partially received header + CMessageHeader hdr; // complete header + unsigned int nHdrPos; + + CDataStream vRecv; // received message data + unsigned int nDataPos; + + int64_t nTime; // time (in microseconds) of message receipt. + + CNetMessage(int nTypeIn, int nVersionIn) : hdrbuf(nTypeIn, nVersionIn), vRecv(nTypeIn, nVersionIn) { + hdrbuf.resize(24); + in_data = false; + nHdrPos = 0; + nDataPos = 0; + nTime = 0; + } + + bool complete() const + { + if (!in_data) + return false; + return (hdr.nMessageSize == nDataPos); + } + + void SetVersion(int nVersionIn) + { + hdrbuf.SetVersion(nVersionIn); + vRecv.SetVersion(nVersionIn); + } + + int readHeader(const char *pch, unsigned int nBytes); + int readData(const char *pch, unsigned int nBytes); +}; + + + + + +/** Information about a peer */ +class CNode +{ +public: + // socket + uint64_t nServices; + SOCKET hSocket; + CDataStream ssSend; + size_t nSendSize; // total size of all vSendMsg entries + size_t nSendOffset; // offset inside the first vSendMsg already sent + uint64_t nSendBytes; + std::deque vSendMsg; + CCriticalSection cs_vSend; + + std::deque vRecvGetData; + std::deque vRecvMsg; + CCriticalSection cs_vRecvMsg; + uint64_t nRecvBytes; + int nRecvVersion; + + int64_t nLastSend; + int64_t nLastRecv; + int64_t nTimeConnected; + CAddress addr; + std::string addrName; + CService addrLocal; + int nVersion; + // strSubVer is whatever byte array we read from the wire. However, this field is intended + // to be printed out, displayed to humans in various forms and so on. So we sanitize it and + // store the sanitized version in cleanSubVer. The original should be used when dealing with + // the network or wire types and the cleaned string used when displayed or logged. + std::string strSubVer, cleanSubVer; + bool fWhitelisted; // This peer can bypass DoS banning. + bool fOneShot; + bool fClient; + bool fInbound; + bool fNetworkNode; + bool fSuccessfullyConnected; + bool fDisconnect; + // We use fRelayTxes for two purposes - + // a) it allows us to not relay tx invs before receiving the peer's version message + // b) the peer may tell us in their version message that we should not relay tx invs + // until they have initialized their bloom filter. + bool fRelayTxes; + CSemaphoreGrant grantOutbound; + CCriticalSection cs_filter; + CBloomFilter* pfilter; + int nRefCount; + NodeId id; + +/* MCHN START*/ + + uint64_t nVersionNonceReceived; + uint64_t nVersionNonceSent; + uint64_t nVerackNonceReceived; + uint64_t nVerackNonceSent; + bool fDefaultMessageStart; + bool fVerackackReceived; + bool fVerackackSent; + bool fParameterSetVerified; + bool fSyncedOnce; + bool fCanConnectRemote; + bool fCanConnectLocal; + CKeyID kAddrRemote; + CKeyID kAddrLocal; + + CAddress addrFromVersion; + +/* MCHN END*/ + +protected: + + // Denial-of-service detection/prevention + // Key is IP address, value is banned-until-time + static std::map setBanned; + static CCriticalSection cs_setBanned; + + // Whitelisted ranges. Any node connecting from these is automatically + // whitelisted (as well as those connecting to whitelisted binds). + static std::vector vWhitelistedRange; + static CCriticalSection cs_vWhitelistedRange; + + // Basic fuzz-testing + void Fuzz(int nChance); // modifies ssSend + +public: + uint256 hashContinue; + int nStartingHeight; + + // flood relay + std::vector vAddrToSend; + mruset setAddrKnown; + bool fGetAddr; + std::set setKnown; + + // inventory based relay + mruset setInventoryKnown; + std::vector vInventoryToSend; + CCriticalSection cs_inventory; + std::multimap mapAskFor; + + // Ping time measurement: + // The pong reply we're expecting, or 0 if no pong expected. + uint64_t nPingNonceSent; + // Time (in usec) the last ping was sent, or 0 if no ping was ever sent. + int64_t nPingUsecStart; + // Last measured round-trip time. + int64_t nPingUsecTime; + // Whether a ping is requested. + bool fPingQueued; + + CNode(SOCKET hSocketIn, CAddress addrIn, std::string addrNameIn = "", bool fInboundIn=false); + ~CNode(); + +private: + // Network usage totals + static CCriticalSection cs_totalBytesRecv; + static CCriticalSection cs_totalBytesSent; + static uint64_t nTotalBytesRecv; + static uint64_t nTotalBytesSent; + + CNode(const CNode&); + void operator=(const CNode&); + +public: + + NodeId GetId() const { + return id; + } + + int GetRefCount() + { + assert(nRefCount >= 0); + return nRefCount; + } + + // requires LOCK(cs_vRecvMsg) + unsigned int GetTotalRecvSize() + { + unsigned int total = 0; + BOOST_FOREACH(const CNetMessage &msg, vRecvMsg) + total += msg.vRecv.size() + 24; + return total; + } + + // requires LOCK(cs_vRecvMsg) + bool ReceiveMsgBytes(const char *pch, unsigned int nBytes); + + // requires LOCK(cs_vRecvMsg) + void SetRecvVersion(int nVersionIn) + { + nRecvVersion = nVersionIn; + BOOST_FOREACH(CNetMessage &msg, vRecvMsg) + msg.SetVersion(nVersionIn); + } + + CNode* AddRef() + { + nRefCount++; + return this; + } + + void Release() + { + nRefCount--; + } + + + + void AddAddressKnown(const CAddress& addr) + { + setAddrKnown.insert(addr); + } + + void PushAddress(const CAddress& addr) + { + // Known checking here is only to save space from duplicates. + // SendMessages will filter it again for knowns that were added + // after addresses were pushed. + if (addr.IsValid() && !setAddrKnown.count(addr)) { + if (vAddrToSend.size() >= MAX_ADDR_TO_SEND) { + vAddrToSend[insecure_rand() % vAddrToSend.size()] = addr; + } else { + vAddrToSend.push_back(addr); + } + } + } + + + void AddInventoryKnown(const CInv& inv) + { + { + LOCK(cs_inventory); + setInventoryKnown.insert(inv); + } + } + + void PushInventory(const CInv& inv) + { + { + LOCK(cs_inventory); + if (!setInventoryKnown.count(inv)) + vInventoryToSend.push_back(inv); + } + } + + void AskFor(const CInv& inv); + + // TODO: Document the postcondition of this function. Is cs_vSend locked? + void BeginMessage(const char* pszCommand) EXCLUSIVE_LOCK_FUNCTION(cs_vSend); + + // TODO: Document the precondition of this function. Is cs_vSend locked? + void AbortMessage() UNLOCK_FUNCTION(cs_vSend); + + // TODO: Document the precondition of this function. Is cs_vSend locked? + void EndMessage() UNLOCK_FUNCTION(cs_vSend); + + void PushVersion(); + + void PushMessage(const char* pszCommand) + { + try + { + BeginMessage(pszCommand); + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1) + { + try + { + BeginMessage(pszCommand); + ssSend << a1; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1, const T2& a2) + { + try + { + BeginMessage(pszCommand); + ssSend << a1 << a2; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1, const T2& a2, const T3& a3) + { + try + { + BeginMessage(pszCommand); + ssSend << a1 << a2 << a3; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1, const T2& a2, const T3& a3, const T4& a4) + { + try + { + BeginMessage(pszCommand); + ssSend << a1 << a2 << a3 << a4; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1, const T2& a2, const T3& a3, const T4& a4, const T5& a5) + { + try + { + BeginMessage(pszCommand); + ssSend << a1 << a2 << a3 << a4 << a5; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1, const T2& a2, const T3& a3, const T4& a4, const T5& a5, const T6& a6) + { + try + { + BeginMessage(pszCommand); + ssSend << a1 << a2 << a3 << a4 << a5 << a6; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1, const T2& a2, const T3& a3, const T4& a4, const T5& a5, const T6& a6, const T7& a7) + { + try + { + BeginMessage(pszCommand); + ssSend << a1 << a2 << a3 << a4 << a5 << a6 << a7; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1, const T2& a2, const T3& a3, const T4& a4, const T5& a5, const T6& a6, const T7& a7, const T8& a8) + { + try + { + BeginMessage(pszCommand); + ssSend << a1 << a2 << a3 << a4 << a5 << a6 << a7 << a8; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1, const T2& a2, const T3& a3, const T4& a4, const T5& a5, const T6& a6, const T7& a7, const T8& a8, const T9& a9) + { + try + { + BeginMessage(pszCommand); + ssSend << a1 << a2 << a3 << a4 << a5 << a6 << a7 << a8 << a9; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1, const T2& a2, const T3& a3, const T4& a4, const T5& a5, const T6& a6, const T7& a7, const T8& a8, const T9& a9, const T10& a10) + { + try + { + BeginMessage(pszCommand); + ssSend << a1 << a2 << a3 << a4 << a5 << a6 << a7 << a8 << a9 << a10; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1, const T2& a2, const T3& a3, const T4& a4, const T5& a5, const T6& a6, const T7& a7, const T8& a8, const T9& a9, const T10& a10, const T11& a11) + { + try + { + BeginMessage(pszCommand); + ssSend << a1 << a2 << a3 << a4 << a5 << a6 << a7 << a8 << a9 << a10 << a11; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + template + void PushMessage(const char* pszCommand, const T1& a1, const T2& a2, const T3& a3, const T4& a4, const T5& a5, const T6& a6, const T7& a7, const T8& a8, const T9& a9, const T10& a10, const T11& a11, const T12& a12) + { + try + { + BeginMessage(pszCommand); + ssSend << a1 << a2 << a3 << a4 << a5 << a6 << a7 << a8 << a9 << a10 << a11 << a12; + EndMessage(); + } + catch (...) + { + AbortMessage(); + throw; + } + } + + bool IsSubscribed(unsigned int nChannel); + void Subscribe(unsigned int nChannel, unsigned int nHops=0); + void CancelSubscribe(unsigned int nChannel); + void CloseSocketDisconnect(); + + // Denial-of-service detection/prevention + // The idea is to detect peers that are behaving + // badly and disconnect/ban them, but do it in a + // one-coding-mistake-won't-shatter-the-entire-network + // way. + // IMPORTANT: There should be nothing I can give a + // node that it will forward on that will make that + // node's peers drop it. If there is, an attacker + // can isolate a node and/or try to split the network. + // Dropping a node for sending stuff that is invalid + // now but might be valid in a later version is also + // dangerous, because it can cause a network split + // between nodes running old code and nodes running + // new code. + static void ClearBanned(); // needed for unit testing + static bool IsBanned(CNetAddr ip); + static bool Ban(const CNetAddr &ip); + void copyStats(CNodeStats &stats); + + static bool IsWhitelistedRange(const CNetAddr &ip); + static void AddWhitelistedRange(const CSubNet &subnet); + + // Network stats + static void RecordBytesRecv(uint64_t bytes); + static void RecordBytesSent(uint64_t bytes); + + static uint64_t GetTotalBytesRecv(); + static uint64_t GetTotalBytesSent(); +}; + + + +class CTransaction; +void RelayTransaction(const CTransaction& tx); +void RelayTransaction(const CTransaction& tx, const CDataStream& ss); + +/** Access to the (IP) address database (peers.dat) */ +class CAddrDB +{ +private: + boost::filesystem::path pathAddr; +public: + CAddrDB(); + bool Write(const CAddrMan& addr); + bool Read(CAddrMan& addr); +}; + +#endif // BITCOIN_NET_H diff --git a/src/net/netbase.cpp b/src/net/netbase.cpp new file mode 100644 index 00000000..88bae23b --- /dev/null +++ b/src/net/netbase.cpp @@ -0,0 +1,1373 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifdef HAVE_CONFIG_H +#include "config/bitcoin-config.h" +#endif + +#include "net/netbase.h" + +#include "structs/hash.h" +#include "utils/sync.h" +#include "structs/uint256.h" +#include "utils/util.h" +#include "utils/utilstrencodings.h" + +#ifdef HAVE_GETADDRINFO_A +#include +#endif + +#ifndef WIN32 +#if HAVE_INET_PTON +#include +#endif +#include +#endif + +#include // for to_lower() +#include // for startswith() and endswith() +#include + +#if !defined(HAVE_MSG_NOSIGNAL) && !defined(MSG_NOSIGNAL) +#define MSG_NOSIGNAL 0 +#endif + +using namespace std; + +// Settings +static proxyType proxyInfo[NET_MAX]; +static CService nameProxy; +static CCriticalSection cs_proxyInfos; +int nConnectTimeout = DEFAULT_CONNECT_TIMEOUT; +bool fNameLookup = false; + +static const unsigned char pchIPv4[12] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0xff, 0xff }; + +// Need ample time for negotiation for very slow proxies such as Tor (milliseconds) +static const int SOCKS5_RECV_TIMEOUT = 20 * 1000; + +enum Network ParseNetwork(std::string net) { + boost::to_lower(net); + if (net == "ipv4") return NET_IPV4; + if (net == "ipv6") return NET_IPV6; + if (net == "tor" || net == "onion") return NET_TOR; + return NET_UNROUTABLE; +} + +std::string GetNetworkName(enum Network net) { + switch(net) + { + case NET_IPV4: return "ipv4"; + case NET_IPV6: return "ipv6"; + case NET_TOR: return "onion"; + default: return ""; + } +} + +void SplitHostPort(std::string in, int &portOut, std::string &hostOut) { + size_t colon = in.find_last_of(':'); + // if a : is found, and it either follows a [...], or no other : is in the string, treat it as port separator + bool fHaveColon = colon != in.npos; + bool fBracketed = fHaveColon && (in[0]=='[' && in[colon-1]==']'); // if there is a colon, and in[0]=='[', colon is not 0, so in[colon-1] is safe + bool fMultiColon = fHaveColon && (in.find_last_of(':',colon-1) != in.npos); + if (fHaveColon && (colon==0 || fBracketed || !fMultiColon)) { + int32_t n; + if (ParseInt32(in.substr(colon + 1), &n) && n > 0 && n < 0x10000) { + in = in.substr(0, colon); + portOut = n; + } + } + if (in.size()>0 && in[0] == '[' && in[in.size()-1] == ']') + hostOut = in.substr(1, in.size()-2); + else + hostOut = in; +} + +bool static LookupIntern(const char *pszName, std::vector& vIP, unsigned int nMaxSolutions, bool fAllowLookup) +{ + vIP.clear(); + + { + CNetAddr addr; + if (addr.SetSpecial(std::string(pszName))) { + vIP.push_back(addr); + return true; + } + } + +#ifdef HAVE_GETADDRINFO_A + struct in_addr ipv4_addr; +#ifdef HAVE_INET_PTON + if (inet_pton(AF_INET, pszName, &ipv4_addr) > 0) { + vIP.push_back(CNetAddr(ipv4_addr)); + return true; + } + + struct in6_addr ipv6_addr; + if (inet_pton(AF_INET6, pszName, &ipv6_addr) > 0) { + vIP.push_back(CNetAddr(ipv6_addr)); + return true; + } +#else + ipv4_addr.s_addr = inet_addr(pszName); + if (ipv4_addr.s_addr != INADDR_NONE) { + vIP.push_back(CNetAddr(ipv4_addr)); + return true; + } +#endif +#endif + + struct addrinfo aiHint; + memset(&aiHint, 0, sizeof(struct addrinfo)); + aiHint.ai_socktype = SOCK_STREAM; + aiHint.ai_protocol = IPPROTO_TCP; + aiHint.ai_family = AF_UNSPEC; +#ifdef WIN32 + aiHint.ai_flags = fAllowLookup ? 0 : AI_NUMERICHOST; +#else + aiHint.ai_flags = fAllowLookup ? AI_ADDRCONFIG : AI_NUMERICHOST; +#endif + + struct addrinfo *aiRes = NULL; +#ifdef HAVE_GETADDRINFO_A + struct gaicb gcb, *query = &gcb; + memset(query, 0, sizeof(struct gaicb)); + gcb.ar_name = pszName; + gcb.ar_request = &aiHint; + int nErr = getaddrinfo_a(GAI_NOWAIT, &query, 1, NULL); + if (nErr) + return false; + + do { + // Should set the timeout limit to a resonable value to avoid + // generating unnecessary checking call during the polling loop, + // while it can still response to stop request quick enough. + // 2 seconds looks fine in our situation. + struct timespec ts = { 2, 0 }; + gai_suspend(&query, 1, &ts); + boost::this_thread::interruption_point(); + + nErr = gai_error(query); + if (0 == nErr) + aiRes = query->ar_result; + } while (nErr == EAI_INPROGRESS); +#else + int nErr = getaddrinfo(pszName, NULL, &aiHint, &aiRes); +#endif + if (nErr) + return false; + + struct addrinfo *aiTrav = aiRes; + while (aiTrav != NULL && (nMaxSolutions == 0 || vIP.size() < nMaxSolutions)) + { + if (aiTrav->ai_family == AF_INET) + { + assert(aiTrav->ai_addrlen >= sizeof(sockaddr_in)); + vIP.push_back(CNetAddr(((struct sockaddr_in*)(aiTrav->ai_addr))->sin_addr)); + } + + if (aiTrav->ai_family == AF_INET6) + { + assert(aiTrav->ai_addrlen >= sizeof(sockaddr_in6)); + vIP.push_back(CNetAddr(((struct sockaddr_in6*)(aiTrav->ai_addr))->sin6_addr)); + } + + aiTrav = aiTrav->ai_next; + } + + freeaddrinfo(aiRes); + + return (vIP.size() > 0); +} + +bool LookupHost(const char *pszName, std::vector& vIP, unsigned int nMaxSolutions, bool fAllowLookup) +{ + std::string strHost(pszName); + if (strHost.empty()) + return false; + if (boost::algorithm::starts_with(strHost, "[") && boost::algorithm::ends_with(strHost, "]")) + { + strHost = strHost.substr(1, strHost.size() - 2); + } + + return LookupIntern(strHost.c_str(), vIP, nMaxSolutions, fAllowLookup); +} + +bool Lookup(const char *pszName, std::vector& vAddr, int portDefault, bool fAllowLookup, unsigned int nMaxSolutions) +{ + if (pszName[0] == 0) + return false; + int port = portDefault; + std::string hostname = ""; + SplitHostPort(std::string(pszName), port, hostname); + + std::vector vIP; + bool fRet = LookupIntern(hostname.c_str(), vIP, nMaxSolutions, fAllowLookup); + if (!fRet) + return false; + vAddr.resize(vIP.size()); + for (unsigned int i = 0; i < vIP.size(); i++) + vAddr[i] = CService(vIP[i], port); + return true; +} + +bool Lookup(const char *pszName, CService& addr, int portDefault, bool fAllowLookup) +{ + std::vector vService; + bool fRet = Lookup(pszName, vService, portDefault, fAllowLookup, 1); + if (!fRet) + return false; + addr = vService[0]; + return true; +} + +bool LookupNumeric(const char *pszName, CService& addr, int portDefault) +{ + return Lookup(pszName, addr, portDefault, false); +} + +/** + * Convert milliseconds to a struct timeval for select. + */ +struct timeval static MillisToTimeval(int64_t nTimeout) +{ + struct timeval timeout; + timeout.tv_sec = nTimeout / 1000; + timeout.tv_usec = (nTimeout % 1000) * 1000; + return timeout; +} + +/** + * Read bytes from socket. This will either read the full number of bytes requested + * or return False on error or timeout. + * This function can be interrupted by boost thread interrupt. + * + * @param data Buffer to receive into + * @param len Length of data to receive + * @param timeout Timeout in milliseconds for receive operation + * + * @note This function requires that hSocket is in non-blocking mode. + */ +bool static InterruptibleRecv(char* data, size_t len, int timeout, SOCKET& hSocket) +{ + int64_t curTime = GetTimeMillis(); + int64_t endTime = curTime + timeout; + // Maximum time to wait in one select call. It will take up until this time (in millis) + // to break off in case of an interruption. + const int64_t maxWait = 1000; + while (len > 0 && curTime < endTime) { + ssize_t ret = recv(hSocket, data, len, 0); // Optimistically try the recv first + if (ret > 0) { + len -= ret; + data += ret; + } else if (ret == 0) { // Unexpected disconnection + return false; + } else { // Other error or blocking + int nErr = WSAGetLastError(); + if (nErr == WSAEINPROGRESS || nErr == WSAEWOULDBLOCK || nErr == WSAEINVAL) { + struct timeval tval = MillisToTimeval(std::min(endTime - curTime, maxWait)); + fd_set fdset; + FD_ZERO(&fdset); + FD_SET(hSocket, &fdset); + int nRet = select(hSocket + 1, &fdset, NULL, NULL, &tval); + if (nRet == SOCKET_ERROR) { + return false; + } + } else { + return false; + } + } + boost::this_thread::interruption_point(); + curTime = GetTimeMillis(); + } + return len == 0; +} + +bool static Socks5(string strDest, int port, SOCKET& hSocket) +{ + LogPrintf("SOCKS5 connecting %s\n", strDest); + if (strDest.size() > 255) + { + CloseSocket(hSocket); + return error("Hostname too long"); + } + char pszSocks5Init[] = "\5\1\0"; + ssize_t nSize = sizeof(pszSocks5Init) - 1; + + ssize_t ret = send(hSocket, pszSocks5Init, nSize, MSG_NOSIGNAL); + if (ret != nSize) + { + CloseSocket(hSocket); + return error("Error sending to proxy"); + } + char pchRet1[2]; + if (!InterruptibleRecv(pchRet1, 2, SOCKS5_RECV_TIMEOUT, hSocket)) + { + CloseSocket(hSocket); + return error("Error reading proxy response"); + } + if (pchRet1[0] != 0x05 || pchRet1[1] != 0x00) + { + CloseSocket(hSocket); + return error("Proxy failed to initialize"); + } + string strSocks5("\5\1"); + strSocks5 += '\000'; strSocks5 += '\003'; + strSocks5 += static_cast(std::min((int)strDest.size(), 255)); + strSocks5 += strDest; + strSocks5 += static_cast((port >> 8) & 0xFF); + strSocks5 += static_cast((port >> 0) & 0xFF); + ret = send(hSocket, strSocks5.data(), strSocks5.size(), MSG_NOSIGNAL); + if (ret != (ssize_t)strSocks5.size()) + { + CloseSocket(hSocket); + return error("Error sending to proxy"); + } + char pchRet2[4]; + if (!InterruptibleRecv(pchRet2, 4, SOCKS5_RECV_TIMEOUT, hSocket)) + { + CloseSocket(hSocket); + return error("Error reading proxy response"); + } + if (pchRet2[0] != 0x05) + { + CloseSocket(hSocket); + return error("Proxy failed to accept request"); + } + if (pchRet2[1] != 0x00) + { + CloseSocket(hSocket); + switch (pchRet2[1]) + { + case 0x01: return error("Proxy error: general failure"); + case 0x02: return error("Proxy error: connection not allowed"); + case 0x03: return error("Proxy error: network unreachable"); + case 0x04: return error("Proxy error: host unreachable"); + case 0x05: return error("Proxy error: connection refused"); + case 0x06: return error("Proxy error: TTL expired"); + case 0x07: return error("Proxy error: protocol error"); + case 0x08: return error("Proxy error: address type not supported"); + default: return error("Proxy error: unknown"); + } + } + if (pchRet2[2] != 0x00) + { + CloseSocket(hSocket); + return error("Error: malformed proxy response"); + } + char pchRet3[256]; + switch (pchRet2[3]) + { + case 0x01: ret = InterruptibleRecv(pchRet3, 4, SOCKS5_RECV_TIMEOUT, hSocket); break; + case 0x04: ret = InterruptibleRecv(pchRet3, 16, SOCKS5_RECV_TIMEOUT, hSocket); break; + case 0x03: + { + ret = InterruptibleRecv(pchRet3, 1, SOCKS5_RECV_TIMEOUT, hSocket); + if (!ret) { + CloseSocket(hSocket); + return error("Error reading from proxy"); + } + int nRecv = pchRet3[0]; + ret = InterruptibleRecv(pchRet3, nRecv, SOCKS5_RECV_TIMEOUT, hSocket); + break; + } + default: CloseSocket(hSocket); return error("Error: malformed proxy response"); + } + if (!ret) + { + CloseSocket(hSocket); + return error("Error reading from proxy"); + } + if (!InterruptibleRecv(pchRet3, 2, SOCKS5_RECV_TIMEOUT, hSocket)) + { + CloseSocket(hSocket); + return error("Error reading from proxy"); + } + LogPrintf("SOCKS5 connected %s\n", strDest); + return true; +} + +bool static ConnectSocketDirectly(const CService &addrConnect, SOCKET& hSocketRet, int nTimeout) +{ + hSocketRet = INVALID_SOCKET; + + struct sockaddr_storage sockaddr; + socklen_t len = sizeof(sockaddr); + if (!addrConnect.GetSockAddr((struct sockaddr*)&sockaddr, &len)) { + LogPrintf("Cannot connect to %s: unsupported network\n", addrConnect.ToString()); + return false; + } + + SOCKET hSocket = socket(((struct sockaddr*)&sockaddr)->sa_family, SOCK_STREAM, IPPROTO_TCP); + if (hSocket == INVALID_SOCKET) + return false; + +#ifdef SO_NOSIGPIPE + int set = 1; + // Different way of disabling SIGPIPE on BSD + setsockopt(hSocket, SOL_SOCKET, SO_NOSIGPIPE, (void*)&set, sizeof(int)); +#endif + + // Set to non-blocking + if (!SetSocketNonBlocking(hSocket, true)) + return error("ConnectSocketDirectly: Setting socket to non-blocking failed, error %s\n", NetworkErrorString(WSAGetLastError())); + + if (connect(hSocket, (struct sockaddr*)&sockaddr, len) == SOCKET_ERROR) + { + int nErr = WSAGetLastError(); + // WSAEINVAL is here because some legacy version of winsock uses it + if (nErr == WSAEINPROGRESS || nErr == WSAEWOULDBLOCK || nErr == WSAEINVAL) + { + struct timeval timeout = MillisToTimeval(nTimeout); + fd_set fdset; + FD_ZERO(&fdset); + FD_SET(hSocket, &fdset); + int nRet = select(hSocket + 1, NULL, &fdset, NULL, &timeout); + if (nRet == 0) + { + LogPrint("net", "net: connection to %s timeout\n", addrConnect.ToString()); + CloseSocket(hSocket); + return false; + } + if (nRet == SOCKET_ERROR) + { + LogPrint("net","net: select() for %s failed: %s\n", addrConnect.ToString(), NetworkErrorString(WSAGetLastError())); + CloseSocket(hSocket); + return false; + } + socklen_t nRetSize = sizeof(nRet); +#ifdef WIN32 + if (getsockopt(hSocket, SOL_SOCKET, SO_ERROR, (char*)(&nRet), &nRetSize) == SOCKET_ERROR) +#else + if (getsockopt(hSocket, SOL_SOCKET, SO_ERROR, &nRet, &nRetSize) == SOCKET_ERROR) +#endif + { + LogPrint("net","net: getsockopt() for %s failed: %s\n", addrConnect.ToString(), NetworkErrorString(WSAGetLastError())); + CloseSocket(hSocket); + return false; + } + if (nRet != 0) + { + LogPrint("net","net: connect() to %s failed after select(): %s\n", addrConnect.ToString(), NetworkErrorString(nRet)); + CloseSocket(hSocket); + return false; + } + } +#ifdef WIN32 + else if (WSAGetLastError() != WSAEISCONN) +#else + else +#endif + { + LogPrint("net","net: connect() to %s failed: %s\n", addrConnect.ToString(), NetworkErrorString(WSAGetLastError())); + CloseSocket(hSocket); + return false; + } + } + + hSocketRet = hSocket; + return true; +} + +bool SetProxy(enum Network net, CService addrProxy) { + assert(net >= 0 && net < NET_MAX); + if (!addrProxy.IsValid()) + return false; + LOCK(cs_proxyInfos); + proxyInfo[net] = addrProxy; + return true; +} + +bool GetProxy(enum Network net, proxyType &proxyInfoOut) { + assert(net >= 0 && net < NET_MAX); + LOCK(cs_proxyInfos); + if (!proxyInfo[net].IsValid()) + return false; + proxyInfoOut = proxyInfo[net]; + return true; +} + +bool SetNameProxy(CService addrProxy) { + if (!addrProxy.IsValid()) + return false; + LOCK(cs_proxyInfos); + nameProxy = addrProxy; + return true; +} + +bool GetNameProxy(CService &nameProxyOut) { + LOCK(cs_proxyInfos); + if(!nameProxy.IsValid()) + return false; + nameProxyOut = nameProxy; + return true; +} + +bool HaveNameProxy() { + LOCK(cs_proxyInfos); + return nameProxy.IsValid(); +} + +bool IsProxy(const CNetAddr &addr) { + LOCK(cs_proxyInfos); + for (int i = 0; i < NET_MAX; i++) { + if (addr == (CNetAddr)proxyInfo[i]) + return true; + } + return false; +} + +bool ConnectSocket(const CService &addrDest, SOCKET& hSocketRet, int nTimeout, bool *outProxyConnectionFailed) +{ + proxyType proxy; + if (outProxyConnectionFailed) + *outProxyConnectionFailed = false; + // no proxy needed (none set for target network) + if (!GetProxy(addrDest.GetNetwork(), proxy)) + return ConnectSocketDirectly(addrDest, hSocketRet, nTimeout); + + SOCKET hSocket = INVALID_SOCKET; + + // first connect to proxy server + if (!ConnectSocketDirectly(proxy, hSocket, nTimeout)) { + if (outProxyConnectionFailed) + *outProxyConnectionFailed = true; + return false; + } + // do socks negotiation + if (!Socks5(addrDest.ToStringIP(), addrDest.GetPort(), hSocket)) + return false; + + hSocketRet = hSocket; + return true; +} + +bool ConnectSocketByName(CService &addr, SOCKET& hSocketRet, const char *pszDest, int portDefault, int nTimeout, bool *outProxyConnectionFailed) +{ + string strDest; + int port = portDefault; + + if (outProxyConnectionFailed) + *outProxyConnectionFailed = false; + + SplitHostPort(string(pszDest), port, strDest); + + SOCKET hSocket = INVALID_SOCKET; + + CService nameProxy; + GetNameProxy(nameProxy); + + CService addrResolved(CNetAddr(strDest, fNameLookup && !HaveNameProxy()), port); + if (addrResolved.IsValid()) { + addr = addrResolved; + return ConnectSocket(addr, hSocketRet, nTimeout); + } + + addr = CService("0.0.0.0:0"); + + if (!HaveNameProxy()) + return false; + // first connect to name proxy server + if (!ConnectSocketDirectly(nameProxy, hSocket, nTimeout)) { + if (outProxyConnectionFailed) + *outProxyConnectionFailed = true; + return false; + } + // do socks negotiation + if (!Socks5(strDest, (unsigned short)port, hSocket)) + return false; + + hSocketRet = hSocket; + return true; +} + +void CNetAddr::Init() +{ + memset(ip, 0, sizeof(ip)); +} + +void CNetAddr::SetIP(const CNetAddr& ipIn) +{ + memcpy(ip, ipIn.ip, sizeof(ip)); +} + +void CNetAddr::SetRaw(Network network, const uint8_t *ip_in) +{ + switch(network) + { + case NET_IPV4: + memcpy(ip, pchIPv4, 12); + memcpy(ip+12, ip_in, 4); + break; + case NET_IPV6: + memcpy(ip, ip_in, 16); + break; + default: + assert(!"invalid network"); + } +} + +static const unsigned char pchOnionCat[] = {0xFD,0x87,0xD8,0x7E,0xEB,0x43}; + +bool CNetAddr::SetSpecial(const std::string &strName) +{ + if (strName.size()>6 && strName.substr(strName.size() - 6, 6) == ".onion") { + std::vector vchAddr = DecodeBase32(strName.substr(0, strName.size() - 6).c_str()); + if (vchAddr.size() != 16-sizeof(pchOnionCat)) + return false; + memcpy(ip, pchOnionCat, sizeof(pchOnionCat)); + for (unsigned int i=0; i<16-sizeof(pchOnionCat); i++) + ip[i + sizeof(pchOnionCat)] = vchAddr[i]; + return true; + } + return false; +} + +CNetAddr::CNetAddr() +{ + Init(); +} + +CNetAddr::CNetAddr(const struct in_addr& ipv4Addr) +{ + SetRaw(NET_IPV4, (const uint8_t*)&ipv4Addr); +} + +CNetAddr::CNetAddr(const struct in6_addr& ipv6Addr) +{ + SetRaw(NET_IPV6, (const uint8_t*)&ipv6Addr); +} + +CNetAddr::CNetAddr(const char *pszIp, bool fAllowLookup) +{ + Init(); + std::vector vIP; + if (LookupHost(pszIp, vIP, 1, fAllowLookup)) + *this = vIP[0]; +} + +CNetAddr::CNetAddr(const std::string &strIp, bool fAllowLookup) +{ + Init(); + std::vector vIP; + if (LookupHost(strIp.c_str(), vIP, 1, fAllowLookup)) + *this = vIP[0]; +} + +unsigned int CNetAddr::GetByte(int n) const +{ + return ip[15-n]; +} + +bool CNetAddr::IsIPv4() const +{ + return (memcmp(ip, pchIPv4, sizeof(pchIPv4)) == 0); +} + +bool CNetAddr::IsIPv6() const +{ + return (!IsIPv4() && !IsTor()); +} + +bool CNetAddr::IsRFC1918() const +{ + return IsIPv4() && ( + GetByte(3) == 10 || + (GetByte(3) == 192 && GetByte(2) == 168) || + (GetByte(3) == 172 && (GetByte(2) >= 16 && GetByte(2) <= 31))); +} + +bool CNetAddr::IsRFC2544() const +{ + return IsIPv4() && GetByte(3) == 198 && (GetByte(2) == 18 || GetByte(2) == 19); +} + +bool CNetAddr::IsRFC3927() const +{ + return IsIPv4() && (GetByte(3) == 169 && GetByte(2) == 254); +} + +bool CNetAddr::IsRFC6598() const +{ + return IsIPv4() && GetByte(3) == 100 && GetByte(2) >= 64 && GetByte(2) <= 127; +} + +bool CNetAddr::IsRFC5737() const +{ + return IsIPv4() && ((GetByte(3) == 192 && GetByte(2) == 0 && GetByte(1) == 2) || + (GetByte(3) == 198 && GetByte(2) == 51 && GetByte(1) == 100) || + (GetByte(3) == 203 && GetByte(2) == 0 && GetByte(1) == 113)); +} + +bool CNetAddr::IsRFC3849() const +{ + return GetByte(15) == 0x20 && GetByte(14) == 0x01 && GetByte(13) == 0x0D && GetByte(12) == 0xB8; +} + +bool CNetAddr::IsRFC3964() const +{ + return (GetByte(15) == 0x20 && GetByte(14) == 0x02); +} + +bool CNetAddr::IsRFC6052() const +{ + static const unsigned char pchRFC6052[] = {0,0x64,0xFF,0x9B,0,0,0,0,0,0,0,0}; + return (memcmp(ip, pchRFC6052, sizeof(pchRFC6052)) == 0); +} + +bool CNetAddr::IsRFC4380() const +{ + return (GetByte(15) == 0x20 && GetByte(14) == 0x01 && GetByte(13) == 0 && GetByte(12) == 0); +} + +bool CNetAddr::IsRFC4862() const +{ + static const unsigned char pchRFC4862[] = {0xFE,0x80,0,0,0,0,0,0}; + return (memcmp(ip, pchRFC4862, sizeof(pchRFC4862)) == 0); +} + +bool CNetAddr::IsRFC4193() const +{ + return ((GetByte(15) & 0xFE) == 0xFC); +} + +bool CNetAddr::IsRFC6145() const +{ + static const unsigned char pchRFC6145[] = {0,0,0,0,0,0,0,0,0xFF,0xFF,0,0}; + return (memcmp(ip, pchRFC6145, sizeof(pchRFC6145)) == 0); +} + +bool CNetAddr::IsRFC4843() const +{ + return (GetByte(15) == 0x20 && GetByte(14) == 0x01 && GetByte(13) == 0x00 && (GetByte(12) & 0xF0) == 0x10); +} + +bool CNetAddr::IsTor() const +{ + return (memcmp(ip, pchOnionCat, sizeof(pchOnionCat)) == 0); +} + +bool CNetAddr::IsLocal() const +{ + // IPv4 loopback + if (IsIPv4() && (GetByte(3) == 127 || GetByte(3) == 0)) + return true; + + // IPv6 loopback (::1/128) + static const unsigned char pchLocal[16] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}; + if (memcmp(ip, pchLocal, 16) == 0) + return true; + + return false; +} + +bool CNetAddr::IsMulticast() const +{ + return (IsIPv4() && (GetByte(3) & 0xF0) == 0xE0) + || (GetByte(15) == 0xFF); +} + +bool CNetAddr::IsValid() const +{ + // Cleanup 3-byte shifted addresses caused by garbage in size field + // of addr messages from versions before 0.2.9 checksum. + // Two consecutive addr messages look like this: + // header20 vectorlen3 addr26 addr26 addr26 header20 vectorlen3 addr26 addr26 addr26... + // so if the first length field is garbled, it reads the second batch + // of addr misaligned by 3 bytes. + if (memcmp(ip, pchIPv4+3, sizeof(pchIPv4)-3) == 0) + return false; + + // unspecified IPv6 address (::/128) + unsigned char ipNone[16] = {}; + if (memcmp(ip, ipNone, 16) == 0) + return false; + + // documentation IPv6 address + if (IsRFC3849()) + return false; + + if (IsIPv4()) + { + // INADDR_NONE + uint32_t ipNone = INADDR_NONE; + if (memcmp(ip+12, &ipNone, 4) == 0) + return false; + + // 0 + ipNone = 0; + if (memcmp(ip+12, &ipNone, 4) == 0) + return false; + } + + return true; +} + +bool CNetAddr::IsRoutable() const +{ +/* MCHN START*/ +// return IsValid() && !(IsRFC1918() || IsRFC2544() || IsRFC3927() || IsRFC4862() || IsRFC6598() || IsRFC5737() || (IsRFC4193() && !IsTor()) || IsRFC4843() || IsLocal()); + return IsValid(); +/* MCHN END*/ +} + +enum Network CNetAddr::GetNetwork() const +{ + if (!IsRoutable()) + return NET_UNROUTABLE; + + if (IsIPv4()) + return NET_IPV4; + + if (IsTor()) + return NET_TOR; + + return NET_IPV6; +} + +std::string CNetAddr::ToStringIP() const +{ + if (IsTor()) + return EncodeBase32(&ip[6], 10) + ".onion"; + CService serv(*this, 0); + struct sockaddr_storage sockaddr; + socklen_t socklen = sizeof(sockaddr); + if (serv.GetSockAddr((struct sockaddr*)&sockaddr, &socklen)) { + char name[1025] = ""; + if (!getnameinfo((const struct sockaddr*)&sockaddr, socklen, name, sizeof(name), NULL, 0, NI_NUMERICHOST)) + return std::string(name); + } + if (IsIPv4()) + return strprintf("%u.%u.%u.%u", GetByte(3), GetByte(2), GetByte(1), GetByte(0)); + else + return strprintf("%x:%x:%x:%x:%x:%x:%x:%x", + GetByte(15) << 8 | GetByte(14), GetByte(13) << 8 | GetByte(12), + GetByte(11) << 8 | GetByte(10), GetByte(9) << 8 | GetByte(8), + GetByte(7) << 8 | GetByte(6), GetByte(5) << 8 | GetByte(4), + GetByte(3) << 8 | GetByte(2), GetByte(1) << 8 | GetByte(0)); +} + +std::string CNetAddr::ToString() const +{ + return ToStringIP(); +} + +bool operator==(const CNetAddr& a, const CNetAddr& b) +{ + return (memcmp(a.ip, b.ip, 16) == 0); +} + +bool operator!=(const CNetAddr& a, const CNetAddr& b) +{ + return (memcmp(a.ip, b.ip, 16) != 0); +} + +bool operator<(const CNetAddr& a, const CNetAddr& b) +{ + return (memcmp(a.ip, b.ip, 16) < 0); +} + +bool CNetAddr::GetInAddr(struct in_addr* pipv4Addr) const +{ + if (!IsIPv4()) + return false; + memcpy(pipv4Addr, ip+12, 4); + return true; +} + +bool CNetAddr::GetIn6Addr(struct in6_addr* pipv6Addr) const +{ + memcpy(pipv6Addr, ip, 16); + return true; +} + +// get canonical identifier of an address' group +// no two connections will be attempted to addresses with the same group +std::vector CNetAddr::GetGroup() const +{ + std::vector vchRet; + int nClass = NET_IPV6; + int nStartByte = 0; + int nBits = 16; + + // all local addresses belong to the same group + if (IsLocal()) + { + nClass = 255; + nBits = 0; + } + + // all unroutable addresses belong to the same group + if (!IsRoutable()) + { + nClass = NET_UNROUTABLE; + nBits = 0; + } + // for IPv4 addresses, '1' + the 16 higher-order bits of the IP + // includes mapped IPv4, SIIT translated IPv4, and the well-known prefix + else if (IsIPv4() || IsRFC6145() || IsRFC6052()) + { + nClass = NET_IPV4; + nStartByte = 12; + } + // for 6to4 tunnelled addresses, use the encapsulated IPv4 address + else if (IsRFC3964()) + { + nClass = NET_IPV4; + nStartByte = 2; + } + // for Teredo-tunnelled IPv6 addresses, use the encapsulated IPv4 address + else if (IsRFC4380()) + { + vchRet.push_back(NET_IPV4); + vchRet.push_back(GetByte(3) ^ 0xFF); + vchRet.push_back(GetByte(2) ^ 0xFF); + return vchRet; + } + else if (IsTor()) + { + nClass = NET_TOR; + nStartByte = 6; + nBits = 4; + } + // for he.net, use /36 groups + else if (GetByte(15) == 0x20 && GetByte(14) == 0x01 && GetByte(13) == 0x04 && GetByte(12) == 0x70) + nBits = 36; + // for the rest of the IPv6 network, use /32 groups + else + nBits = 32; + + vchRet.push_back(nClass); + while (nBits >= 8) + { + vchRet.push_back(GetByte(15 - nStartByte)); + nStartByte++; + nBits -= 8; + } + if (nBits > 0) + vchRet.push_back(GetByte(15 - nStartByte) | ((1 << nBits) - 1)); + + return vchRet; +} + +uint64_t CNetAddr::GetHash() const +{ + uint256 hash = Hash(&ip[0], &ip[16]); + uint64_t nRet; + memcpy(&nRet, &hash, sizeof(nRet)); + return nRet; +} + +// private extensions to enum Network, only returned by GetExtNetwork, +// and only used in GetReachabilityFrom +static const int NET_UNKNOWN = NET_MAX + 0; +static const int NET_TEREDO = NET_MAX + 1; +int static GetExtNetwork(const CNetAddr *addr) +{ + if (addr == NULL) + return NET_UNKNOWN; + if (addr->IsRFC4380()) + return NET_TEREDO; + return addr->GetNetwork(); +} + +/** Calculates a metric for how reachable (*this) is from a given partner */ +int CNetAddr::GetReachabilityFrom(const CNetAddr *paddrPartner) const +{ + enum Reachability { + REACH_UNREACHABLE, + REACH_DEFAULT, + REACH_TEREDO, + REACH_IPV6_WEAK, + REACH_IPV4, + REACH_IPV6_STRONG, + REACH_PRIVATE + }; + + if (!IsRoutable()) + return REACH_UNREACHABLE; + + int ourNet = GetExtNetwork(this); + int theirNet = GetExtNetwork(paddrPartner); + bool fTunnel = IsRFC3964() || IsRFC6052() || IsRFC6145(); + + switch(theirNet) { + case NET_IPV4: + switch(ourNet) { + default: return REACH_DEFAULT; + case NET_IPV4: return REACH_IPV4; + } + case NET_IPV6: + switch(ourNet) { + default: return REACH_DEFAULT; + case NET_TEREDO: return REACH_TEREDO; + case NET_IPV4: return REACH_IPV4; + case NET_IPV6: return fTunnel ? REACH_IPV6_WEAK : REACH_IPV6_STRONG; // only prefer giving our IPv6 address if it's not tunnelled + } + case NET_TOR: + switch(ourNet) { + default: return REACH_DEFAULT; + case NET_IPV4: return REACH_IPV4; // Tor users can connect to IPv4 as well + case NET_TOR: return REACH_PRIVATE; + } + case NET_TEREDO: + switch(ourNet) { + default: return REACH_DEFAULT; + case NET_TEREDO: return REACH_TEREDO; + case NET_IPV6: return REACH_IPV6_WEAK; + case NET_IPV4: return REACH_IPV4; + } + case NET_UNKNOWN: + case NET_UNROUTABLE: + default: + switch(ourNet) { + default: return REACH_DEFAULT; + case NET_TEREDO: return REACH_TEREDO; + case NET_IPV6: return REACH_IPV6_WEAK; + case NET_IPV4: return REACH_IPV4; + case NET_TOR: return REACH_PRIVATE; // either from Tor, or don't care about our address + } + } +} + +void CService::Init() +{ + port = 0; +} + +CService::CService() +{ + Init(); +} + +CService::CService(const CNetAddr& cip, unsigned short portIn) : CNetAddr(cip), port(portIn) +{ +} + +CService::CService(const struct in_addr& ipv4Addr, unsigned short portIn) : CNetAddr(ipv4Addr), port(portIn) +{ +} + +CService::CService(const struct in6_addr& ipv6Addr, unsigned short portIn) : CNetAddr(ipv6Addr), port(portIn) +{ +} + +CService::CService(const struct sockaddr_in& addr) : CNetAddr(addr.sin_addr), port(ntohs(addr.sin_port)) +{ + assert(addr.sin_family == AF_INET); +} + +CService::CService(const struct sockaddr_in6 &addr) : CNetAddr(addr.sin6_addr), port(ntohs(addr.sin6_port)) +{ + assert(addr.sin6_family == AF_INET6); +} + +bool CService::SetSockAddr(const struct sockaddr *paddr) +{ + switch (paddr->sa_family) { + case AF_INET: + *this = CService(*(const struct sockaddr_in*)paddr); + return true; + case AF_INET6: + *this = CService(*(const struct sockaddr_in6*)paddr); + return true; + default: + return false; + } +} + +CService::CService(const char *pszIpPort, bool fAllowLookup) +{ + Init(); + CService ip; + if (Lookup(pszIpPort, ip, 0, fAllowLookup)) + *this = ip; +} + +CService::CService(const char *pszIpPort, int portDefault, bool fAllowLookup) +{ + Init(); + CService ip; + if (Lookup(pszIpPort, ip, portDefault, fAllowLookup)) + *this = ip; +} + +CService::CService(const std::string &strIpPort, bool fAllowLookup) +{ + Init(); + CService ip; + if (Lookup(strIpPort.c_str(), ip, 0, fAllowLookup)) + *this = ip; +} + +CService::CService(const std::string &strIpPort, int portDefault, bool fAllowLookup) +{ + Init(); + CService ip; + if (Lookup(strIpPort.c_str(), ip, portDefault, fAllowLookup)) + *this = ip; +} + +unsigned short CService::GetPort() const +{ + return port; +} + +bool operator==(const CService& a, const CService& b) +{ + return (CNetAddr)a == (CNetAddr)b && a.port == b.port; +} + +bool operator!=(const CService& a, const CService& b) +{ + return (CNetAddr)a != (CNetAddr)b || a.port != b.port; +} + +bool operator<(const CService& a, const CService& b) +{ + return (CNetAddr)a < (CNetAddr)b || ((CNetAddr)a == (CNetAddr)b && a.port < b.port); +} + +bool CService::GetSockAddr(struct sockaddr* paddr, socklen_t *addrlen) const +{ + if (IsIPv4()) { + if (*addrlen < (socklen_t)sizeof(struct sockaddr_in)) + return false; + *addrlen = sizeof(struct sockaddr_in); + struct sockaddr_in *paddrin = (struct sockaddr_in*)paddr; + memset(paddrin, 0, *addrlen); + if (!GetInAddr(&paddrin->sin_addr)) + return false; + paddrin->sin_family = AF_INET; + paddrin->sin_port = htons(port); + return true; + } + if (IsIPv6()) { + if (*addrlen < (socklen_t)sizeof(struct sockaddr_in6)) + return false; + *addrlen = sizeof(struct sockaddr_in6); + struct sockaddr_in6 *paddrin6 = (struct sockaddr_in6*)paddr; + memset(paddrin6, 0, *addrlen); + if (!GetIn6Addr(&paddrin6->sin6_addr)) + return false; + paddrin6->sin6_family = AF_INET6; + paddrin6->sin6_port = htons(port); + return true; + } + return false; +} + +std::vector CService::GetKey() const +{ + std::vector vKey; + vKey.resize(18); + memcpy(&vKey[0], ip, 16); + vKey[16] = port / 0x100; + vKey[17] = port & 0x0FF; + return vKey; +} + +std::string CService::ToStringPort() const +{ + return strprintf("%u", port); +} + +std::string CService::ToStringIPPort() const +{ + if (IsIPv4() || IsTor()) { + return ToStringIP() + ":" + ToStringPort(); + } else { + return "[" + ToStringIP() + "]:" + ToStringPort(); + } +} + +std::string CService::ToString() const +{ + return ToStringIPPort(); +} + +void CService::SetPort(unsigned short portIn) +{ + port = portIn; +} + +CSubNet::CSubNet(): + valid(false) +{ + memset(netmask, 0, sizeof(netmask)); +} + +CSubNet::CSubNet(const std::string &strSubnet, bool fAllowLookup) +{ + size_t slash = strSubnet.find_last_of('/'); + std::vector vIP; + + valid = true; + // Default to /32 (IPv4) or /128 (IPv6), i.e. match single address + memset(netmask, 255, sizeof(netmask)); + + std::string strAddress = strSubnet.substr(0, slash); + if (LookupHost(strAddress.c_str(), vIP, 1, fAllowLookup)) + { + network = vIP[0]; + if (slash != strSubnet.npos) + { + std::string strNetmask = strSubnet.substr(slash + 1); + int32_t n; + // IPv4 addresses start at offset 12, and first 12 bytes must match, so just offset n + int noffset = network.IsIPv4() ? (12 * 8) : 0; + if (ParseInt32(strNetmask, &n)) // If valid number, assume /24 symtex + { + if(n >= 0 && n <= (128 - noffset)) // Only valid if in range of bits of address + { + n += noffset; + // Clear bits [n..127] + for (; n < 128; ++n) + netmask[n>>3] &= ~(1<<(n&7)); + } + else + { + valid = false; + } + } + else // If not a valid number, try full netmask syntax + { + if (LookupHost(strNetmask.c_str(), vIP, 1, false)) // Never allow lookup for netmask + { + // Remember: GetByte returns bytes in reversed order + // Copy only the *last* four bytes in case of IPv4, the rest of the mask should stay 1's as + // we don't want pchIPv4 to be part of the mask. + int asize = network.IsIPv4() ? 4 : 16; + for(int x=0; x +#include +#include + +extern int nConnectTimeout; +extern bool fNameLookup; + +/** -timeout default */ +static const int DEFAULT_CONNECT_TIMEOUT = 5000; + +#ifdef WIN32 +// In MSVC, this is defined as a macro, undefine it to prevent a compile and link error +#undef SetPort +#endif + +enum Network +{ + NET_UNROUTABLE = 0, + NET_IPV4, + NET_IPV6, + NET_TOR, + + NET_MAX, +}; + +/** IP address (IPv6, or IPv4 using mapped IPv6 range (::FFFF:0:0/96)) */ +class CNetAddr +{ + protected: + unsigned char ip[16]; // in network byte order + + public: + CNetAddr(); + CNetAddr(const struct in_addr& ipv4Addr); + explicit CNetAddr(const char *pszIp, bool fAllowLookup = false); + explicit CNetAddr(const std::string &strIp, bool fAllowLookup = false); + void Init(); + void SetIP(const CNetAddr& ip); + + /** + * Set raw IPv4 or IPv6 address (in network byte order) + * @note Only NET_IPV4 and NET_IPV6 are allowed for network. + */ + void SetRaw(Network network, const uint8_t *data); + + bool SetSpecial(const std::string &strName); // for Tor addresses + bool IsIPv4() const; // IPv4 mapped address (::FFFF:0:0/96, 0.0.0.0/0) + bool IsIPv6() const; // IPv6 address (not mapped IPv4, not Tor) + bool IsRFC1918() const; // IPv4 private networks (10.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12) + bool IsRFC2544() const; // IPv4 inter-network communcations (192.18.0.0/15) + bool IsRFC6598() const; // IPv4 ISP-level NAT (100.64.0.0/10) + bool IsRFC5737() const; // IPv4 documentation addresses (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24) + bool IsRFC3849() const; // IPv6 documentation address (2001:0DB8::/32) + bool IsRFC3927() const; // IPv4 autoconfig (169.254.0.0/16) + bool IsRFC3964() const; // IPv6 6to4 tunnelling (2002::/16) + bool IsRFC4193() const; // IPv6 unique local (FC00::/7) + bool IsRFC4380() const; // IPv6 Teredo tunnelling (2001::/32) + bool IsRFC4843() const; // IPv6 ORCHID (2001:10::/28) + bool IsRFC4862() const; // IPv6 autoconfig (FE80::/64) + bool IsRFC6052() const; // IPv6 well-known prefix (64:FF9B::/96) + bool IsRFC6145() const; // IPv6 IPv4-translated address (::FFFF:0:0:0/96) + bool IsTor() const; + bool IsLocal() const; + bool IsRoutable() const; + bool IsValid() const; + bool IsMulticast() const; + enum Network GetNetwork() const; + std::string ToString() const; + std::string ToStringIP() const; + unsigned int GetByte(int n) const; + uint64_t GetHash() const; + bool GetInAddr(struct in_addr* pipv4Addr) const; + std::vector GetGroup() const; + int GetReachabilityFrom(const CNetAddr *paddrPartner = NULL) const; + + CNetAddr(const struct in6_addr& pipv6Addr); + bool GetIn6Addr(struct in6_addr* pipv6Addr) const; + + friend bool operator==(const CNetAddr& a, const CNetAddr& b); + friend bool operator!=(const CNetAddr& a, const CNetAddr& b); + friend bool operator<(const CNetAddr& a, const CNetAddr& b); + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(FLATDATA(ip)); + } +}; + +class CSubNet +{ + protected: + /// Network (base) address + CNetAddr network; + /// Netmask, in network byte order + uint8_t netmask[16]; + /// Is this value valid? (only used to signal parse errors) + bool valid; + + public: + CSubNet(); + explicit CSubNet(const std::string &strSubnet, bool fAllowLookup = false); + + bool Match(const CNetAddr &addr) const; + + std::string ToString() const; + bool IsValid() const; + + friend bool operator==(const CSubNet& a, const CSubNet& b); + friend bool operator!=(const CSubNet& a, const CSubNet& b); +}; + +/** A combination of a network address (CNetAddr) and a (TCP) port */ +class CService : public CNetAddr +{ + protected: + unsigned short port; // host order + + public: + CService(); + CService(const CNetAddr& ip, unsigned short port); + CService(const struct in_addr& ipv4Addr, unsigned short port); + CService(const struct sockaddr_in& addr); + explicit CService(const char *pszIpPort, int portDefault, bool fAllowLookup = false); + explicit CService(const char *pszIpPort, bool fAllowLookup = false); + explicit CService(const std::string& strIpPort, int portDefault, bool fAllowLookup = false); + explicit CService(const std::string& strIpPort, bool fAllowLookup = false); + void Init(); + void SetPort(unsigned short portIn); + unsigned short GetPort() const; + bool GetSockAddr(struct sockaddr* paddr, socklen_t *addrlen) const; + bool SetSockAddr(const struct sockaddr* paddr); + friend bool operator==(const CService& a, const CService& b); + friend bool operator!=(const CService& a, const CService& b); + friend bool operator<(const CService& a, const CService& b); + std::vector GetKey() const; + std::string ToString() const; + std::string ToStringPort() const; + std::string ToStringIPPort() const; + + CService(const struct in6_addr& ipv6Addr, unsigned short port); + CService(const struct sockaddr_in6& addr); + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(FLATDATA(ip)); + unsigned short portN = htons(port); + READWRITE(portN); + if (ser_action.ForRead()) + port = ntohs(portN); + } +}; + +typedef CService proxyType; + +enum Network ParseNetwork(std::string net); +std::string GetNetworkName(enum Network net); +void SplitHostPort(std::string in, int &portOut, std::string &hostOut); +bool SetProxy(enum Network net, CService addrProxy); +bool GetProxy(enum Network net, proxyType &proxyInfoOut); +bool IsProxy(const CNetAddr &addr); +bool SetNameProxy(CService addrProxy); +bool HaveNameProxy(); +bool LookupHost(const char *pszName, std::vector& vIP, unsigned int nMaxSolutions = 0, bool fAllowLookup = true); +bool Lookup(const char *pszName, CService& addr, int portDefault = 0, bool fAllowLookup = true); +bool Lookup(const char *pszName, std::vector& vAddr, int portDefault = 0, bool fAllowLookup = true, unsigned int nMaxSolutions = 0); +bool LookupNumeric(const char *pszName, CService& addr, int portDefault = 0); +bool ConnectSocket(const CService &addr, SOCKET& hSocketRet, int nTimeout, bool *outProxyConnectionFailed = 0); +bool ConnectSocketByName(CService &addr, SOCKET& hSocketRet, const char *pszDest, int portDefault, int nTimeout, bool *outProxyConnectionFailed = 0); +/** Return readable error string for a network error code */ +std::string NetworkErrorString(int err); +/** Close socket and set hSocket to INVALID_SOCKET */ +bool CloseSocket(SOCKET& hSocket); +/** Disable or enable blocking-mode for a socket */ +bool SetSocketNonBlocking(SOCKET& hSocket, bool fNonBlocking); + +#endif // BITCOIN_NETBASE_H diff --git a/src/net/rest.cpp b/src/net/rest.cpp new file mode 100644 index 00000000..0ccef9aa --- /dev/null +++ b/src/net/rest.cpp @@ -0,0 +1,254 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "primitives/block.h" +#include "primitives/transaction.h" +#include "core/main.h" +#include "rpc/rpcserver.h" +#include "utils/streams.h" +#include "utils/sync.h" +#include "utils/utilstrencodings.h" +#include "version/bcversion.h" + +#include + +using namespace std; +using namespace json_spirit; + +enum RetFormat { + RF_UNDEF, + RF_BINARY, + RF_HEX, + RF_JSON, +}; + +static const struct { + enum RetFormat rf; + const char* name; +} rf_names[] = { + {RF_UNDEF, ""}, + {RF_BINARY, "bin"}, + {RF_HEX, "hex"}, + {RF_JSON, "json"}, +}; + +class RestErr +{ +public: + enum HTTPStatusCode status; + string message; +}; + +extern void TxToJSON(const CTransaction& tx, const uint256 hashBlock, Object& entry); +extern Object blockToJSON(const CBlock& block, const CBlockIndex* blockindex, bool txDetails = false, int verbose_level = 1); + +static RestErr RESTERR(enum HTTPStatusCode status, string message) +{ + RestErr re; + re.status = status; + re.message = message; + return re; +} + +static enum RetFormat ParseDataFormat(vector& params, const string strReq) +{ + boost::split(params, strReq, boost::is_any_of(".")); + if (params.size() > 1) { + for (unsigned int i = 0; i < ARRAYLEN(rf_names); i++) + if (params[1] == rf_names[i].name) + return rf_names[i].rf; + } + + return rf_names[0].rf; +} + +static string AvailableDataFormatsString() +{ + string formats = ""; + for (unsigned int i = 0; i < ARRAYLEN(rf_names); i++) + if (strlen(rf_names[i].name) > 0) { + formats.append("."); + formats.append(rf_names[i].name); + formats.append(", "); + } + + if (formats.length() > 0) + return formats.substr(0, formats.length() - 2); + + return formats; +} + +static bool ParseHashStr(const string& strReq, uint256& v) +{ + if (!IsHex(strReq) || (strReq.size() != 64)) + return false; + + v.SetHex(strReq); + return true; +} + +static bool rest_block(AcceptedConnection* conn, + string& strReq, + map& mapHeaders, + bool fRun, + bool showTxDetails) +{ + vector params; + enum RetFormat rf = ParseDataFormat(params, strReq); + + string hashStr = params[0]; + uint256 hash; + if (!ParseHashStr(hashStr, hash)) + throw RESTERR(HTTP_BAD_REQUEST, "Invalid hash: " + hashStr); + + CBlock block; + CBlockIndex* pblockindex = NULL; + { + LOCK(cs_main); + if (mapBlockIndex.count(hash) == 0) + throw RESTERR(HTTP_NOT_FOUND, hashStr + " not found"); + + pblockindex = mapBlockIndex[hash]; + if (!ReadBlockFromDisk(block, pblockindex)) + throw RESTERR(HTTP_NOT_FOUND, hashStr + " not found"); + } + + CDataStream ssBlock(SER_NETWORK, PROTOCOL_VERSION); + ssBlock << block; + + switch (rf) { + case RF_BINARY: { + string binaryBlock = ssBlock.str(); + conn->stream() << HTTPReplyHeader(HTTP_OK, fRun, binaryBlock.size(), "application/octet-stream") << binaryBlock << std::flush; + return true; + } + + case RF_HEX: { + string strHex = HexStr(ssBlock.begin(), ssBlock.end()) + "\n"; + conn->stream() << HTTPReply(HTTP_OK, strHex, fRun, false, "text/plain") << std::flush; + return true; + } + + case RF_JSON: { + Object objBlock = blockToJSON(block, pblockindex, showTxDetails); + string strJSON = write_string(Value(objBlock), false) + "\n"; + conn->stream() << HTTPReply(HTTP_OK, strJSON, fRun) << std::flush; + return true; + } + + default: { + throw RESTERR(HTTP_NOT_FOUND, "output format not found (available: " + AvailableDataFormatsString() + ")"); + } + } + + // not reached + return true; // continue to process further HTTP reqs on this cxn +} + +static bool rest_block_extended(AcceptedConnection* conn, + string& strReq, + map& mapHeaders, + bool fRun) +{ + return rest_block(conn, strReq, mapHeaders, fRun, true); +} + +static bool rest_block_notxdetails(AcceptedConnection* conn, + string& strReq, + map& mapHeaders, + bool fRun) +{ + return rest_block(conn, strReq, mapHeaders, fRun, false); +} + +static bool rest_tx(AcceptedConnection* conn, + string& strReq, + map& mapHeaders, + bool fRun) +{ + vector params; + enum RetFormat rf = ParseDataFormat(params, strReq); + + string hashStr = params[0]; + uint256 hash; + if (!ParseHashStr(hashStr, hash)) + throw RESTERR(HTTP_BAD_REQUEST, "Invalid hash: " + hashStr); + + CTransaction tx; + uint256 hashBlock = 0; + if (!GetTransaction(hash, tx, hashBlock, true)) + throw RESTERR(HTTP_NOT_FOUND, hashStr + " not found"); + + CDataStream ssTx(SER_NETWORK, PROTOCOL_VERSION); + ssTx << tx; + + switch (rf) { + case RF_BINARY: { + string binaryTx = ssTx.str(); + conn->stream() << HTTPReplyHeader(HTTP_OK, fRun, binaryTx.size(), "application/octet-stream") << binaryTx << std::flush; + return true; + } + + case RF_HEX: { + string strHex = HexStr(ssTx.begin(), ssTx.end()) + "\n"; + conn->stream() << HTTPReply(HTTP_OK, strHex, fRun, false, "text/plain") << std::flush; + return true; + } + + case RF_JSON: { + Object objTx; + TxToJSON(tx, hashBlock, objTx); + string strJSON = write_string(Value(objTx), false) + "\n"; + conn->stream() << HTTPReply(HTTP_OK, strJSON, fRun) << std::flush; + return true; + } + + default: { + throw RESTERR(HTTP_NOT_FOUND, "output format not found (available: " + AvailableDataFormatsString() + ")"); + } + } + + // not reached + return true; // continue to process further HTTP reqs on this cxn +} + +static const struct { + const char* prefix; + bool (*handler)(AcceptedConnection* conn, + string& strURI, + map& mapHeaders, + bool fRun); +} uri_prefixes[] = { + {"/rest/tx/", rest_tx}, + {"/rest/block/notxdetails/", rest_block_notxdetails}, + {"/rest/block/", rest_block_extended}, +}; + +bool HTTPReq_REST(AcceptedConnection* conn, + string& strURI, + map& mapHeaders, + bool fRun) +{ + try { + std::string statusmessage; + if (RPCIsInWarmup(&statusmessage)) + throw RESTERR(HTTP_SERVICE_UNAVAILABLE, "Service temporarily unavailable: " + statusmessage); + + for (unsigned int i = 0; i < ARRAYLEN(uri_prefixes); i++) { + unsigned int plen = strlen(uri_prefixes[i].prefix); + if (strURI.substr(0, plen) == uri_prefixes[i].prefix) { + string strReq = strURI.substr(plen); + return uri_prefixes[i].handler(conn, strReq, mapHeaders, fRun); + } + } + } catch (RestErr& re) { + conn->stream() << HTTPReply(re.status, re.message + "\r\n", false, false, "text/plain") << std::flush; + return false; + } + + conn->stream() << HTTPError(HTTP_NOT_FOUND, false) << std::flush; + return false; +} diff --git a/src/obj-test/.gitignore b/src/obj-test/.gitignore new file mode 100644 index 00000000..d6b7ef32 --- /dev/null +++ b/src/obj-test/.gitignore @@ -0,0 +1,2 @@ +* +!.gitignore diff --git a/src/obj/.gitignore b/src/obj/.gitignore new file mode 100644 index 00000000..d6b7ef32 --- /dev/null +++ b/src/obj/.gitignore @@ -0,0 +1,2 @@ +* +!.gitignore diff --git a/src/permissions/permission.cpp b/src/permissions/permission.cpp new file mode 100644 index 00000000..fe850df2 --- /dev/null +++ b/src/permissions/permission.cpp @@ -0,0 +1,2965 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" + +unsigned char null_entity[MC_PLS_SIZE_ENTITY]; + +int mc_IsNullEntity(const void* lpEntity) +{ + if(lpEntity == NULL) + { + return 1; + } + if(memcmp(lpEntity,null_entity,MC_PLS_SIZE_ENTITY) == 0) + { + return 1; + } + return 0; +} + + +void mc_PermissionDBRow::Zero() +{ + memset(this,0,sizeof(mc_PermissionDBRow)); +} + +void mc_PermissionLedgerRow::Zero() +{ + memset(this,0,sizeof(mc_PermissionLedgerRow)); +} + +void mc_BlockLedgerRow::Zero() +{ + memset(this,0,sizeof(mc_BlockLedgerRow)); +} + +void mc_PermissionDetails::Zero() +{ + memset(this,0,sizeof(mc_PermissionDetails)); +} + + +/** Set initial database object values */ + +void mc_PermissionDB::Zero() +{ + m_FileName[0]=0; + m_DB=0; + if(mc_gState->m_Features->PerEntityPermissions()) + { + m_KeyOffset=0; + m_KeySize=MC_PLS_SIZE_ENTITY+24; // Entity,address,type + } + else + { + m_KeyOffset=MC_PLS_SIZE_ENTITY; + m_KeySize=24; + } + m_ValueOffset=56; + m_ValueSize=24; + m_TotalSize=m_KeySize+m_ValueSize; +} + +/** Set database file name */ + +void mc_PermissionDB::SetName(const char* name) +{ + mc_GetFullFileName(name,"permissions",".db",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR,m_FileName); +} + +/** Open database */ + +int mc_PermissionDB::Open() +{ + + m_DB=new mc_Database; + + m_DB->SetOption("KeySize",0,m_KeySize); + m_DB->SetOption("ValueSize",0,m_ValueSize); + + return m_DB->Open(m_FileName,MC_OPT_DB_DATABASE_CREATE_IF_MISSING | MC_OPT_DB_DATABASE_TRANSACTIONAL | MC_OPT_DB_DATABASE_LEVELDB); +} + +/** Close database */ + +int mc_PermissionDB::Close() +{ + if(m_DB) + { + m_DB->Close(); + delete m_DB; + m_DB=NULL; + } + return 0; +} + +/** Set initial ledger values */ + +void mc_PermissionLedger::Zero() +{ + m_FileName[0]=0; + m_FileHan=0; + if(mc_gState->m_Features->PerEntityPermissions()) + { + m_KeyOffset=0; + m_KeySize=MC_PLS_SIZE_ENTITY+32; // Entity,address,type,prevrow + } + else + { + m_KeyOffset=MC_PLS_SIZE_ENTITY; + m_KeySize=32; + } + m_ValueOffset=MC_PLS_SIZE_ENTITY+32; + m_ValueSize=64; + m_TotalSize=m_KeySize+m_ValueSize; +} + +/** Set ledger file name */ + +void mc_PermissionLedger::SetName(const char* name) +{ + mc_GetFullFileName(name,"permissions",".dat",MC_FOM_RELATIVE_TO_DATADIR,m_FileName); +} + +/** Open ledger file */ + +int mc_PermissionLedger::Open() +{ + if(m_FileHan>0) + { + return m_FileHan; + } + + m_FileHan=open(m_FileName,_O_BINARY | O_RDWR | O_CREAT, S_IRUSR | S_IWUSR); + return m_FileHan; +} + +/** Close ledger file */ + +int mc_PermissionLedger::Close() +{ + if(m_FileHan>0) + { + close(m_FileHan); + } + m_FileHan=0; + return 0; +} + +/** Returns ledger row */ + +int mc_PermissionLedger::GetRow(uint64_t RowID, mc_PermissionLedgerRow* row) +{ + int64_t off; + off=RowID*m_TotalSize; + + if(m_FileHan<=0) + { + return MC_ERR_INTERNAL_ERROR; + } + if(lseek64(m_FileHan,off,SEEK_SET) != off) + { + return MC_ERR_NOT_FOUND; + } + + row->Zero(); + + if(read(m_FileHan,(unsigned char*)row+m_KeyOffset,m_TotalSize) != m_TotalSize) + { + return MC_ERR_FILE_READ_ERROR; + } + + return MC_ERR_NOERROR; + +} + +/** Returns ledger size*/ + +uint64_t mc_PermissionLedger::GetSize() +{ + if(m_FileHan<=0) + { + return 0; + } + + return lseek64(m_FileHan,0,SEEK_END); +} + +/** Writes row into ledger without specifying position */ + +int mc_PermissionLedger::WriteRow(mc_PermissionLedgerRow* row) +{ + if(m_FileHan<=0) + { + return MC_ERR_INTERNAL_ERROR; + } + + if(write(m_FileHan,(unsigned char*)row+m_KeyOffset,m_TotalSize) != m_TotalSize) + { + return MC_ERR_INTERNAL_ERROR; + } + + return MC_ERR_NOERROR; +} + +/** Writes row into ledger to specified position */ + +int mc_PermissionLedger::SetRow(uint64_t RowID, mc_PermissionLedgerRow* row) +{ + if(m_FileHan<=0) + { + return MC_ERR_INTERNAL_ERROR; + } + + int64_t off; + off=RowID*m_TotalSize; + + if(lseek64(m_FileHan,off,SEEK_SET) != off) + { + return MC_ERR_INTERNAL_ERROR; + } + + return WriteRow(row); +} + +/** Setting initial values */ + +int mc_Permissions::Zero() +{ + m_Database=NULL; + m_Ledger=NULL; + m_MemPool = NULL; + m_TmpPool = NULL; + m_CopiedMemPool=NULL; + m_Name[0]=0x00; + m_LogFileName[0]=0x00; + m_Block=-1; + m_Row=0; + m_AdminCount=0; + m_MinerCount=0; + m_DBRowCount=0; + m_CheckPointRow=0; + m_CheckPointAdminCount=0; + m_CheckPointMinerCount=0; + m_CheckPointMemPoolSize=0; + m_CopiedAdminCount=0; + m_CopiedMinerCount=0; + m_ClearedAdminCount=0; + m_ClearedMinerCount=0; + + m_Semaphore=NULL; + m_LockedBy=0; + + return MC_ERR_NOERROR; +} + +/** Initialization */ + +int mc_Permissions::Initialize(const char *name,int mode) +{ + int err,value_len; + int32_t pdbBlock,pldBlock; + uint64_t pdbLastRow,pldLastRow; + uint64_t ledger_size; + char msg[256]; + + unsigned char *ptr; + + mc_PermissionDBRow pdbRow; + mc_PermissionLedgerRow pldRow; + + strcpy(m_Name,name); + memset(null_entity,0,MC_PLS_SIZE_ENTITY); + + err=MC_ERR_NOERROR; + + m_Ledger=new mc_PermissionLedger; + m_Database=new mc_PermissionDB; + + m_Ledger->SetName(name); + m_Database->SetName(name); + mc_GetFullFileName(name,"permissions",".log",MC_FOM_RELATIVE_TO_DATADIR,m_LogFileName); + + err=m_Database->Open(); + + if(err) + { + LogString("Initialize: Cannot open database"); + return err; + } + + pdbBlock=-1; + pdbLastRow=1; + + pdbRow.Zero(); + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(err) + { + LogString("Initialize: Cannot read from database"); + return err; + } + + if(ptr) + { + memcpy((char*)&pdbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + pdbBlock=pdbRow.m_BlockTo; + pdbLastRow=pdbRow.m_LedgerRow; + } + else + { + pdbRow.Zero(); + pdbRow.m_BlockTo=(uint32_t)pdbBlock; + pdbRow.m_LedgerRow=pdbLastRow; + + err=m_Database->m_DB->Write((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&pdbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,0); + if(err) + { + return err; + } + + err=m_Database->m_DB->Commit(0);//MC_OPT_DB_DATABASE_TRANSACTIONAL + if(err) + { + return err; + } + } + + m_TmpPool=new mc_Buffer; + m_TmpPool->Initialize(m_Database->m_ValueOffset,sizeof(mc_PermissionLedgerRow),MC_BUF_MODE_MAP); + + m_MemPool=new mc_Buffer; + + err=m_MemPool->Initialize(m_Ledger->m_KeySize,m_Ledger->m_TotalSize,MC_BUF_MODE_MAP); + + m_CopiedMemPool=new mc_Buffer; + + err=m_CopiedMemPool->Initialize(m_Ledger->m_KeySize,m_Ledger->m_TotalSize,0); + + pldBlock=-1; + pldLastRow=1; + + if(m_Ledger->Open() <= 0) + { + return MC_ERR_DBOPEN_ERROR; + } + + if(m_Ledger->GetRow(0,&pldRow) == 0) + { + pldBlock=pldRow.m_BlockTo; + pldLastRow=pldRow.m_PrevRow; + } + else + { + pldRow.Zero(); + pldRow.m_BlockTo=(uint32_t)pldBlock; + pldRow.m_PrevRow=pldLastRow; + m_Ledger->SetRow(0,&pldRow); + } + + ledger_size=m_Ledger->GetSize()/m_Ledger->m_TotalSize; + m_Row=ledger_size; + + m_Ledger->Close(); + if(pdbBlock != pldBlock) + { + LogString("Initialize: Database corrupted 1"); + return MC_ERR_CORRUPTED; + } + + if(pdbLastRow != pldLastRow) + { + LogString("Initialize: Database corrupted 2"); + return MC_ERR_CORRUPTED; + } + + if(pldLastRow > ledger_size) + { + LogString("Initialize: Database corrupted 3"); + return MC_ERR_CORRUPTED; + } + + m_Block=pdbBlock; + m_Row=pdbLastRow; + + UpdateCounts(); + m_ClearedAdminCount=m_AdminCount; + m_ClearedMinerCount=m_MinerCount; + + m_Semaphore=__US_SemCreate(); + if(m_Semaphore == NULL) + { + LogString("Initialize: Cannot initialize semaphore"); + return MC_ERR_INTERNAL_ERROR; + } + + sprintf(msg,"Initialized: Admin count: %d, Miner count: %d, DB rows: %d, ledger rows: %ld",m_AdminCount,m_MinerCount,m_DBRowCount,m_Row); + LogString(msg); + return MC_ERR_NOERROR; +} + +/** Logging message */ + +void mc_Permissions::LogString(const char *message) +{ + FILE *fHan; + + fHan=fopen(m_LogFileName,"a"); + if(fHan == NULL) + { + return; + } + mc_LogString(fHan,message); + + fclose(fHan); +} + +/** Freeing objects */ + +int mc_Permissions::Destroy() +{ + int removefiles; + mc_PermissionDBRow pdbRow; + + removefiles=0; + if(m_Ledger) + { + m_Ledger->Open(); + if(m_Ledger->GetSize() == m_Ledger->m_TotalSize) + { + removefiles=1; + } + m_Ledger->Close(); + } + + if(m_Semaphore) + { + __US_SemDestroy(m_Semaphore); + } + + if(m_Database) + { + if(removefiles) + { + pdbRow.Zero(); + m_Database->m_DB->Delete((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,0); + m_Database->m_DB->Commit(0); + } + m_Database->Close(); + delete m_Database; + } + + if(m_Ledger) + { + if(removefiles) + { + m_Ledger->Close(); + remove(m_Ledger->m_FileName); + } + delete m_Ledger; + } + + if(m_MemPool) + { + delete m_MemPool; + } + if(m_TmpPool) + { + delete m_TmpPool; + } + if(m_CopiedMemPool) + { + delete m_CopiedMemPool; + } + + Zero(); + + return MC_ERR_NOERROR; +} + +/** Locking permissions object */ + +void mc_Permissions::Lock(int write_mode) +{ + uint64_t this_thread; + this_thread=__US_ThreadID(); + + if(this_thread == m_LockedBy) + { + LogString("Secondary lock!!!"); + return; + } + + __US_SemWait(m_Semaphore); + m_LockedBy=this_thread; +} + +/** Unlocking permissions object */ + +void mc_Permissions::UnLock() +{ + m_LockedBy=0; + __US_SemPost(m_Semaphore); +} + +int mc_MemcmpCheckSize(const void *s1,const char *s2,size_t s1_size) +{ + if(strlen(s2) != s1_size) + { + return 1; + } + return memcmp(s1,s2,s1_size); +} + +uint32_t mc_Permissions::GetPossiblePermissionTypes(uint32_t entity_type) +{ + uint32_t full_type; + + full_type=0; + switch(entity_type) + { + case MC_ENT_TYPE_ASSET: + full_type = MC_PTP_ISSUE | MC_PTP_ADMIN; + break; + case MC_ENT_TYPE_STREAM: + full_type = MC_PTP_WRITE | MC_PTP_ACTIVATE | MC_PTP_ADMIN; + break; + case MC_ENT_TYPE_NONE: + full_type = MC_PTP_GLOBAL_ALL; + if(mc_gState->m_Features->Streams() == 0) + { + full_type-=MC_PTP_CREATE; + } + break; + default: + if(mc_gState->m_Features->FixedIn10007()) + { + if(entity_type <= MC_ENT_TYPE_MAX) + { + full_type = MC_PTP_WRITE | MC_PTP_ACTIVATE | MC_PTP_ADMIN; + } + } + break; + } + if(mc_gState->m_Features->ActivatePermission() == 0) + { + if(full_type & MC_PTP_ACTIVATE) + { + full_type-=MC_PTP_ACTIVATE; + } + } + return full_type; +} + + +/** Return ORed MC_PTP_ constants by textual value */ + +uint32_t mc_Permissions::GetPermissionType(const char *str,int entity_type) +{ + uint32_t result,perm_type,full_type; + char* ptr; + char* start; + char* ptrEnd; + char c; + + full_type=GetPossiblePermissionTypes(entity_type); + + ptr=(char*)str; + ptrEnd=ptr+strlen(ptr); + start=ptr; + + result=0; + while(ptr<=ptrEnd) + { + c=*ptr; + if( (c == ',') || (c ==0x00)) + { + if(ptr > start) + { + perm_type=0; + if((mc_MemcmpCheckSize(start,"all", ptr-start) == 0) || + (mc_MemcmpCheckSize(start,"*", ptr-start) == 0)) + { + perm_type=full_type; + } + if(mc_MemcmpCheckSize(start,"connect", ptr-start) == 0)perm_type = MC_PTP_CONNECT; + if(mc_MemcmpCheckSize(start,"send", ptr-start) == 0)perm_type = MC_PTP_SEND; + if(mc_MemcmpCheckSize(start,"receive", ptr-start) == 0)perm_type = MC_PTP_RECEIVE; + if(mc_MemcmpCheckSize(start,"issue", ptr-start) == 0)perm_type = MC_PTP_ISSUE; + if(mc_MemcmpCheckSize(start,"mine", ptr-start) == 0)perm_type = MC_PTP_MINE; + if(mc_MemcmpCheckSize(start,"admin", ptr-start) == 0)perm_type = MC_PTP_ADMIN; + if(mc_gState->m_Features->ActivatePermission()) + { + if(mc_MemcmpCheckSize(start,"activate", ptr-start) == 0)perm_type = MC_PTP_ACTIVATE; + } + if(mc_gState->m_Features->Streams()) + { + if(mc_MemcmpCheckSize(start,"create", ptr-start) == 0)perm_type = MC_PTP_CREATE; + if(mc_MemcmpCheckSize(start,"write", ptr-start) == 0)perm_type = MC_PTP_WRITE; + } + + if(perm_type == 0) + { + return 0; + } + result |= perm_type; + start=ptr+1; + } + } + ptr++; + } + + if(result & ~full_type) + { + result=0; + } + + return result; +} + +/** Returns permission value and details for key (entity,address,type) */ + +uint32_t mc_Permissions::GetPermission(const void* lpEntity,const void* lpAddress,uint32_t type,mc_PermissionLedgerRow *row,int checkmempool) +{ + if(lpEntity == NULL) + { + return GetPermission(null_entity,lpAddress,type,row,checkmempool); + } + + int err,value_len,mprow,found_in_db; + uint32_t result; + mc_PermissionLedgerRow pldRow; + mc_PermissionDBRow pdbRow; + unsigned char *ptr; + + pdbRow.Zero(); + memcpy(&pdbRow.m_Entity,lpEntity,MC_PLS_SIZE_ENTITY); + memcpy(&pdbRow.m_Address,lpAddress,MC_PLS_SIZE_ADDRESS); + pdbRow.m_Type=type; + + pldRow.Zero(); + memcpy(&pldRow.m_Entity,lpEntity,MC_PLS_SIZE_ENTITY); + memcpy(&pldRow.m_Address,lpAddress,MC_PLS_SIZE_ADDRESS); + pldRow.m_Type=type; + + memcpy(row,&pldRow,sizeof(mc_PermissionLedgerRow)); + + if(m_Database->m_DB == NULL) + { + LogString("GetPermission: Database not opened"); + return 0; + } + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(err) + { + LogString("GetPermission: Cannot read from database"); + return 0; + } + + result=0; + found_in_db=0; + if(ptr) + { + memcpy((char*)&pdbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + + pldRow.m_PrevRow=pdbRow.m_LedgerRow; + pldRow.m_BlockFrom=pdbRow.m_BlockFrom; + pldRow.m_BlockTo=pdbRow.m_BlockTo; + pldRow.m_Flags=pdbRow.m_Flags; + row->m_BlockFrom=pdbRow.m_BlockFrom; + row->m_BlockTo=pdbRow.m_BlockTo; + row->m_ThisRow=pdbRow.m_LedgerRow; + row->m_Flags=pdbRow.m_Flags; + found_in_db=1; + row->m_FoundInDB=found_in_db; + } + + if(checkmempool) + { + mprow=0; + while(mprow>=0) + { + mprow=m_MemPool->Seek((unsigned char*)&pldRow+m_Ledger->m_KeyOffset); + if(mprow>=0) + { + memcpy((unsigned char*)row+m_Ledger->m_KeyOffset,m_MemPool->GetRow(mprow),m_Ledger->m_TotalSize); + row->m_FoundInDB=found_in_db; + pldRow.m_PrevRow=row->m_ThisRow; + } + } + } + + result=0; + if((uint32_t)(m_Block+1) >= row->m_BlockFrom) + { + if((uint32_t)(m_Block+1) < row->m_BlockTo) + { + result=type; + } + } + + return result; +} + +/** Returns permission value and details for NULL entity */ + +uint32_t mc_Permissions::GetPermission(const void* lpAddress,uint32_t type,mc_PermissionLedgerRow *row) +{ + return GetPermission(NULL,lpAddress,type,row,1); +} + +/** Returns permission value for key (entity,address,type) */ + +uint32_t mc_Permissions::GetPermission(const void* lpEntity,const void* lpAddress,uint32_t type) +{ + mc_PermissionLedgerRow row; + return GetPermission(lpEntity,lpAddress,type,&row,1); +} + +/** Returns permission value for NULL entity */ + +uint32_t mc_Permissions::GetPermission(const void* lpAddress,uint32_t type) +{ + mc_PermissionLedgerRow row; + return GetPermission(lpAddress,type,&row); +} + +/** Returns all permissions (entity,address) ANDed with type */ + +uint32_t mc_Permissions::GetAllPermissions(const void* lpEntity,const void* lpAddress,uint32_t type) +{ + uint32_t result; + + result=0; + + if(type & MC_PTP_CONNECT) result |= CanConnect(lpEntity,lpAddress); + if(type & MC_PTP_SEND) result |= CanSend(lpEntity,lpAddress); + if(type & MC_PTP_RECEIVE) result |= CanReceive(lpEntity,lpAddress); + if(type & MC_PTP_WRITE) result |= CanWrite(lpEntity,lpAddress); + if(type & MC_PTP_CREATE) result |= CanCreate(lpEntity,lpAddress); + if(type & MC_PTP_ISSUE) result |= CanIssue(lpEntity,lpAddress); + if(type & MC_PTP_ADMIN) result |= CanAdmin(lpEntity,lpAddress); + if(type & MC_PTP_MINE) result |= CanMine(lpEntity,lpAddress); + if(type & MC_PTP_ACTIVATE) result |= CanActivate(lpEntity,lpAddress); + + return result; +} +/** Returns non-zero value if (entity,address) can connect */ + +int mc_Permissions::CanConnect(const void* lpEntity,const void* lpAddress) +{ + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return MC_PTP_CONNECT; + } + +// if(lpEntity == NULL) + if(mc_IsNullEntity(lpEntity)) + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanconnect")) + { + return MC_PTP_CONNECT; + } + } + + int result; + Lock(0); + + result=GetPermission(lpEntity,lpAddress,MC_PTP_CONNECT); + + UnLock(); + + return result; +} + +/** Returns non-zero value if (entity,address) can send */ + +int mc_Permissions::CanSend(const void* lpEntity,const void* lpAddress) +{ + int result; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return MC_PTP_SEND; + } + + if(mc_IsNullEntity(lpEntity)) + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecansend")) + { + return MC_PTP_SEND; + } + } + + Lock(0); + + result = GetPermission(lpEntity,lpAddress,MC_PTP_SEND); + + if(result == 0) + { + if(mc_gState->m_Features->Streams()) + { + result |= GetPermission(lpEntity,lpAddress,MC_PTP_ISSUE); + } + } + + if(result == 0) + { + if(mc_gState->m_Features->Streams()) + { + result |= GetPermission(lpEntity,lpAddress,MC_PTP_CREATE); + } + } + + if(result == 0) + { + result |= GetPermission(lpEntity,lpAddress,MC_PTP_ADMIN); + } + + if(result == 0) + { + if(mc_gState->m_Features->ActivatePermission()) + { + result |= GetPermission(lpEntity,lpAddress,MC_PTP_ACTIVATE); + } + } + + if(result) + { + result = MC_PTP_SEND; + } + + UnLock(); + + return result; +} + +/** Returns non-zero value if (entity,address) can receive */ + +int mc_Permissions::CanReceive(const void* lpEntity,const void* lpAddress) +{ + int result; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return MC_PTP_RECEIVE; + } + + if(mc_IsNullEntity(lpEntity)) + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanreceive")) + { + return MC_PTP_RECEIVE; + } + } + + Lock(0); + + result = GetPermission(lpEntity,lpAddress,MC_PTP_RECEIVE); + + if(result == 0) + { + result |= GetPermission(lpEntity,lpAddress,MC_PTP_ADMIN); + } + + if(result == 0) + { + if(mc_gState->m_Features->ActivatePermission()) + { + result |= GetPermission(lpEntity,lpAddress,MC_PTP_ACTIVATE); + } + } + + if(result) + { + result = MC_PTP_RECEIVE; + } + + UnLock(); + + return result; +} + +/** Returns non-zero value if (entity,address) can write */ + +int mc_Permissions::CanWrite(const void* lpEntity,const void* lpAddress) +{ + int result; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return 0; + } + + Lock(0); + + result = GetPermission(lpEntity,lpAddress,MC_PTP_WRITE); + + if(result) + { + result = MC_PTP_WRITE; + } + + UnLock(); + + return result; +} + +/** Returns non-zero value if (entity,address) can write */ + +int mc_Permissions::CanCreate(const void* lpEntity,const void* lpAddress) +{ + int result; + +// if(lpEntity == NULL) + if(mc_IsNullEntity(lpEntity)) + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecancreate")) + { + return MC_PTP_CREATE; + } + } + + Lock(0); + + result = GetPermission(lpEntity,lpAddress,MC_PTP_CREATE); + + if(result) + { + result = MC_PTP_CREATE; + } + + UnLock(); + + return result; +} + +/** Returns non-zero value if (entity,address) can issue */ + +int mc_Permissions::CanIssue(const void* lpEntity,const void* lpAddress) +{ +// if(lpEntity == NULL) + if(mc_IsNullEntity(lpEntity)) + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanissue")) + { + return MC_PTP_ISSUE; + } + } + + int result; + Lock(0); + + result=GetPermission(lpEntity,lpAddress,MC_PTP_ISSUE); + + UnLock(); + + return result; +} + +/** Returns 1 if we are still in setup period (NULL entity only) */ + +int mc_Permissions::IsSetupPeriod() +{ + if(m_Block+1m_NetworkParams->GetInt64Param("setupfirstblocks")) + { + return 1; + } + return 0; +} + +/** Returns number of active miners (NULL entity only) */ + +int mc_Permissions::GetActiveMinerCount() +{ + Lock(0); + + int miner_count=m_MinerCount; + int diversity; + if(!IsSetupPeriod()) + { + diversity=(int)mc_gState->m_NetworkParams->GetInt64Param("miningdiversity"); + if(diversity > 0) + { + diversity=(int)((miner_count*diversity-1)/MC_PRM_DECIMAL_GRANULARITY); + } + diversity++; +// diversity=(int)((m_MinerCount*(uint32_t)mc_gState->m_NetworkParams->GetInt64Param("miningdiversity")-1)/MC_PRM_DECIMAL_GRANULARITY)+1; + miner_count-=diversity-1; + if(miner_count<1) + { + miner_count=1; + } + if(miner_count > m_MinerCount) + { + miner_count=m_MinerCount; + } + } + if(m_MinerCount <= 0) + { + miner_count=1; + } + + UnLock(); + + return miner_count; +} + +/** Returns non-zero value if (entity,address) can mine */ + +int mc_Permissions::CanMine(const void* lpEntity,const void* lpAddress) +{ + mc_PermissionLedgerRow row; + + int result; + int diversity; + int32_t last; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return MC_PTP_MINE; + } + + if(mc_IsNullEntity(lpEntity)) + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanmine")) + { + return MC_PTP_MINE; + } + } + + Lock(0); + + int miner_count=m_MinerCount; + int check_mempool=1; + if(mc_gState->m_Features->UnconfirmedMinersCannotMine()) + { + miner_count=m_ClearedMinerCount; + check_mempool=0; + } + + result = GetPermission(lpEntity,lpAddress,MC_PTP_MINE,&row,check_mempool); + + if(result) + { + if(mc_IsNullEntity(lpEntity)) // Mining diversity is checked only for NULL object + { + GetPermission(lpEntity,lpAddress,MC_PTP_BLOCK_MINER,&row,0); + last=row.m_BlockFrom; + if(last) + { + if(!IsSetupPeriod()) + { + diversity=(int)mc_gState->m_NetworkParams->GetInt64Param("miningdiversity"); + if(diversity > 0) + { + diversity=(int)((miner_count*diversity-1)/MC_PRM_DECIMAL_GRANULARITY); + } + diversity++; + if(diversity<1) + { + diversity=1; + } + if(diversity > miner_count) + { + diversity=miner_count; + } + if(m_Block+1-last <= diversity-1) + { + result=0; + } + } + } + } + } + + UnLock(); + + return result; +} + +/** Returns non-zero value if address can mine block with soecific height */ + +int mc_Permissions::CanMineBlock(const void* lpAddress,uint32_t block) +{ + mc_PermissionLedgerRow row; + mc_BlockLedgerRow block_row; + int result; + int miner_count; + int diversity; + uint32_t last; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return MC_PTP_MINE; + } + + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanmine")) + { + return MC_PTP_MINE; + } + + if(block == 0) + { + return 0; + } + + Lock(0); + + result=1; + if(m_Ledger->Open() <= 0) + { + LogString("Error: CanMineBlock: couldn't open ledger"); + result=0; + } + + if(result) + { + GetPermission(NULL,lpAddress,MC_PTP_MINE,&row,0); + + if(row.m_ThisRow) + { + m_Ledger->GetRow(row.m_ThisRow,&row); + while( (row.m_PrevRow > 0 ) && (row.m_BlockReceived >= block) ) + { + m_Ledger->GetRow(row.m_PrevRow,&row); + } + if(row.m_BlockReceived >= block) + { + result=0; + } + } + else + { + result=0; + } + + } + + if(result) + { + if( (block < row.m_BlockFrom) || (block >= row.m_BlockTo) ) + { + result=0; + } + } + + if(result>0) + { + GetPermission(NULL,lpAddress,MC_PTP_BLOCK_MINER,&row,0); + + if(row.m_ThisRow) + { + m_Ledger->GetRow(row.m_ThisRow,&row); + while( (row.m_PrevRow > 0 ) && (row.m_BlockReceived >= block) ) + { + m_Ledger->GetRow(row.m_PrevRow,&row); + } + if(row.m_BlockReceived < block) + { + last=row.m_BlockFrom; + if(last) + { + block_row.Zero(); + sprintf((char*)block_row.m_Address,"Block %08X row",block-1); + GetPermission(block_row.m_Address,MC_PTP_BLOCK_INDEX,(mc_PermissionLedgerRow*)&block_row); + m_Ledger->GetRow(block_row.m_ThisRow,(mc_PermissionLedgerRow*)&block_row); + + miner_count=block_row.m_MinerCount; + + if(miner_count) + { + if(block >= mc_gState->m_NetworkParams->GetInt64Param("setupfirstblocks")) + { + diversity=(int)mc_gState->m_NetworkParams->GetInt64Param("miningdiversity"); + if(diversity > 0) + { + diversity=(int)((miner_count*diversity-1)/MC_PRM_DECIMAL_GRANULARITY); + } + diversity++; + if(diversity<1) + { + diversity=1; + } + if(diversity > miner_count) + { + diversity=miner_count; + } + if((int)(block-last) <= diversity-1) + { + result=0; + } + } + } + } + } + } + } + + m_Ledger->Close(); + + UnLock(); + + return result; +} + +/** Returns non-zero value if (entity,address) can admin */ + +int mc_Permissions::CanAdmin(const void* lpEntity,const void* lpAddress) +{ + if(mc_gState->m_Features->Streams()) + { + if(m_Block == -1) + { + return MC_PTP_ADMIN; + } + } + else + { + if(m_AdminCount == 0) + { + return MC_PTP_ADMIN; + } + } + + if(mc_IsNullEntity(lpEntity)) + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanadmin")) + { + return MC_PTP_ADMIN; + } + } + + int result; + Lock(0); + + result=GetPermission(lpEntity,lpAddress,MC_PTP_ADMIN); + + UnLock(); + + return result; +} + +/** Returns non-zero value if (entity,address) can activate */ + +int mc_Permissions::CanActivate(const void* lpEntity,const void* lpAddress) +{ + if(mc_gState->m_Features->ActivatePermission() == 0) + { + if(CanAdmin(lpEntity,lpAddress)) + { + return MC_PTP_ACTIVATE; + } + else + { + return 0; + } + } + + if(mc_IsNullEntity(lpEntity)) + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanactivate")) + { + return MC_PTP_ACTIVATE; + } + } + + int result; + Lock(0); + + result=GetPermission(lpEntity,lpAddress,MC_PTP_ACTIVATE); + + UnLock(); + + if(result == 0) + { + result = CanAdmin(lpEntity,lpAddress); + } + + if(result) + { + result = MC_PTP_ACTIVATE; + } + + return result; +} + +/** Updates admin and miner counts (NULL entity only) */ + +int mc_Permissions::UpdateCounts() +{ + mc_PermissionDBRow pdbRow; + unsigned char *ptr; + int dbvalue_len,err; + uint32_t type; + err=MC_ERR_NOERROR; + + pdbRow.Zero(); + + m_AdminCount=0; + m_MinerCount=0; + + m_DBRowCount=0; + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&dbvalue_len,MC_OPT_DB_DATABASE_SEEK_ON_READ,&err); + if(err) + { + return err; + } + if(ptr == NULL) + { + return MC_ERR_NOERROR; + } + + memcpy((char*)&pdbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + + while(ptr) + { + m_DBRowCount++; + type=pdbRow.m_Type; + if( (type == MC_PTP_ADMIN) || (type == MC_PTP_MINE)) + { + if(memcmp(pdbRow.m_Entity,null_entity,MC_PLS_SIZE_ENTITY) == 0) + { + if((uint32_t)(m_Block+1) >= pdbRow.m_BlockFrom) + { + if((uint32_t)(m_Block+1) < pdbRow.m_BlockTo) + { + if(type == MC_PTP_ADMIN) + { + m_AdminCount++; + } + if(type == MC_PTP_MINE) + { + m_MinerCount++; + } + } + } + } + } + ptr=(unsigned char*)m_Database->m_DB->MoveNext(&err); + if(err) + { + LogString("Error on MoveNext"); + } + if(ptr) + { + memcpy((char*)&pdbRow+m_Database->m_KeyOffset,ptr,m_Database->m_TotalSize); + } + } + + return err; +} + +/** Dumps database and ledger */ + +void mc_Permissions::Dump() +{ + mc_PermissionDBRow pdbRow; + mc_PermissionLedgerRow pldRow; + + unsigned char *ptr; + int dbvalue_len,err,i; + + pdbRow.Zero(); + + printf("\nDB\n"); + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&dbvalue_len,MC_OPT_DB_DATABASE_SEEK_ON_READ,&err); + if(err) + { + return; + } + + if(ptr) + { + memcpy((char*)&pdbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + while(ptr) + { + mc_MemoryDumpCharSize((char*)&pdbRow+m_Database->m_KeyOffset,0,m_Database->m_TotalSize,m_Database->m_TotalSize); + ptr=(unsigned char*)m_Database->m_DB->MoveNext(&err); + if(ptr) + { + memcpy((char*)&pdbRow+m_Database->m_KeyOffset,ptr,m_Database->m_TotalSize); + } + } + } + + printf("Ledger\n"); + + m_Ledger->Open(); + + for(i=0;i<(int)m_Row-m_MemPool->GetCount();i++) + { + m_Ledger->GetRow(i,&pldRow); + mc_MemoryDumpCharSize((unsigned char*)&pldRow+m_Ledger->m_KeyOffset,0,m_Ledger->m_TotalSize,m_Ledger->m_TotalSize/2); + } + + m_Ledger->Close(); + + mc_DumpSize("MemPool",m_MemPool->GetRow(0),m_MemPool->GetCount()*m_Ledger->m_TotalSize,m_Ledger->m_TotalSize/2); + +} + +/** Returns number of admins (NULL entity only) */ + +int mc_Permissions::GetAdminCount() +{ + return m_AdminCount; +} + +/** Returns number of miners (NULL entity only) */ + +int mc_Permissions::GetMinerCount() +{ + return m_MinerCount; +} + +/** Clears memory pool, external, with lock */ + +int mc_Permissions::ClearMemPool() +{ + int result; + Lock(1); + result=ClearMemPoolInternal(); + UnLock(); + return result; +} + +/** Clears memory pool, internal, without lock */ + +int mc_Permissions::ClearMemPoolInternal() +{ + char msg[256]; + if(m_MemPool->GetCount()) + { + m_Row-=m_MemPool->GetCount(); + m_MemPool->Clear(); + m_AdminCount=m_ClearedAdminCount; + m_MinerCount=m_ClearedMinerCount; + UpdateCounts(); + sprintf(msg,"Mempool clr : %9d, Admin count: %d, Miner count: %d, DB rows: %d, Ledger Rows: %ld",m_Block,m_AdminCount,m_MinerCount,m_DBRowCount,m_Row); + LogString(msg); + } + m_ClearedAdminCount=m_AdminCount; + m_ClearedMinerCount=m_MinerCount; + + return MC_ERR_NOERROR; +} + +/** Copies memory pool */ + +int mc_Permissions::CopyMemPool() +{ + int i,err; + unsigned char *ptr; + char msg[256]; + + Lock(1); + err=MC_ERR_NOERROR; + + m_CopiedMemPool->Clear(); + for(i=0;iGetCount();i++) + { + ptr=m_MemPool->GetRow(i); + err=m_CopiedMemPool->Add(ptr,ptr+m_MemPool->m_KeySize); + if(err) + { + LogString("Error while copying mempool"); + goto exitlbl; + } + } + + m_CopiedAdminCount=m_AdminCount; + m_CopiedMinerCount=m_MinerCount; + + if(m_MemPool->GetCount()) + { +// UpdateCounts(); + sprintf(msg,"Mempool copy: %9d, Admin count: %d, Miner count: %d, DB rows: %d, Ledger Rows: %ld",m_Block,m_AdminCount,m_MinerCount,m_DBRowCount,m_Row); + LogString(msg); + } + +exitlbl: + + UnLock(); + return err; +} + +/** Restores memory pool */ + +int mc_Permissions::RestoreMemPool() +{ + int i,err; + unsigned char *ptr; + char msg[256]; + + err=MC_ERR_NOERROR; + + Lock(1); + + ClearMemPoolInternal(); + + for(i=0;iGetCount();i++) + { + ptr=m_CopiedMemPool->GetRow(i); + err=m_MemPool->Add(ptr,ptr+m_MemPool->m_KeySize); + if(err) + { + LogString("Error while restoring mempool"); + goto exitlbl; + } + } + + m_Row+=m_CopiedMemPool->GetCount(); + m_AdminCount=m_CopiedAdminCount; + m_MinerCount=m_CopiedMinerCount; + + if(m_MemPool->GetCount()) + { +// UpdateCounts(); + sprintf(msg,"Mempool rstr: %9d, Admin count: %d, Miner count: %d, DB rows: %d, Ledger Rows: %ld",m_Block,m_AdminCount,m_MinerCount,m_DBRowCount,m_Row); + LogString(msg); + } + +exitlbl: + + UnLock(); + + return err; +} + + +/** Returns number of admins required for consensus for specific permission type (NULL entity only) */ + +int mc_Permissions::AdminConsensus(uint32_t type) +{ + int consensus; + + if(GetAdminCount()==0) + { + return 1; + } + + switch(type) + { + case MC_PTP_ADMIN: + case MC_PTP_MINE: + case MC_PTP_ACTIVATE: + case MC_PTP_ISSUE: + case MC_PTP_CREATE: + if(IsSetupPeriod()) + { + return 1; + } + + consensus=0; + if(type == MC_PTP_ADMIN) + { + consensus=mc_gState->m_NetworkParams->GetInt64Param("adminconsensusadmin"); + } + if(type == MC_PTP_MINE) + { + consensus=mc_gState->m_NetworkParams->GetInt64Param("adminconsensusmine"); + } + if(type == MC_PTP_ACTIVATE) + { + if(mc_gState->m_Features->ActivatePermission() == 0) + { + return GetAdminCount()+1; + } + consensus=mc_gState->m_NetworkParams->GetInt64Param("adminconsensusactivate"); + } + if(type == MC_PTP_ISSUE) + { + consensus=mc_gState->m_NetworkParams->GetInt64Param("adminconsensusissue"); + } + if(type == MC_PTP_CREATE) + { + consensus=mc_gState->m_NetworkParams->GetInt64Param("adminconsensuscreate"); + } + + if(consensus==0) + { + return 1; + } + + return (int)((GetAdminCount()*(uint32_t)consensus-1)/MC_PRM_DECIMAL_GRANULARITY)+1; + } + + return 1; +} + +/** Verifies whether consensus is reached. Update blockFrom/To fields if needed. */ + +int mc_Permissions::VerifyConsensus(mc_PermissionLedgerRow *newRow,mc_PermissionLedgerRow *lastRow,int *remaining) +{ + int consensus,required,found,exit_now; + uint64_t prevRow,countLedgerRows; + mc_PermissionLedgerRow pldRow; + mc_PermissionLedgerRow *ptr; + + consensus=AdminConsensus(newRow->m_Type); + required=consensus; + + if(remaining) + { + *remaining=0; + } + + newRow->m_Consensus=consensus; + + if(required <= 1) + { + return MC_ERR_NOERROR; + } + + countLedgerRows=m_Row-m_MemPool->GetCount(); + + m_TmpPool->Clear(); + + exit_now=0; + memcpy(&pldRow,newRow,sizeof(mc_PermissionLedgerRow)); + while(required) + { + memcpy(pldRow.m_Address,pldRow.m_Admin,MC_PLS_SIZE_ADDRESS); // Looking for row from the same admin + + ptr=&pldRow; + + found=m_TmpPool->Seek((unsigned char*)ptr); + if(found >= 0) + { + ptr=(mc_PermissionLedgerRow *)m_TmpPool->GetRow(found); + if(ptr->m_Timestamp < pldRow.m_Timestamp) + { + if(ptr->m_GrantFrom == newRow->m_GrantFrom) + { + if(ptr->m_GrantTo == newRow->m_GrantTo) + { + required++; // Duplicate record, compensate for decrementing below + } + } + m_TmpPool->PutRow(found,&pldRow,(unsigned char*)&pldRow+m_Database->m_ValueOffset); + ptr=&pldRow; + } + else + { + ptr=NULL; // Ignore this record, we already have newer + } + } + else + { + m_TmpPool->Add(&pldRow,(unsigned char*)&pldRow+m_Database->m_ValueOffset); + } + + if(ptr) + { + if(ptr->m_GrantFrom == newRow->m_GrantFrom) + { + if(ptr->m_GrantTo == newRow->m_GrantTo) + { + if((remaining == NULL) || (pldRow.m_BlockReceived > 0)) // Avoid decrementing when checking for RequiredForConsensus + { + required--; + } + } + } + } + + if(required) + { + prevRow=pldRow.m_PrevRow; + + if(prevRow == 0) + { + exit_now=1; + } + else + { + if(prevRow >= countLedgerRows) // Previous row in mempool + { + pldRow.Zero(); + memcpy((unsigned char*)&pldRow+m_Ledger->m_KeyOffset,m_MemPool->GetRow(prevRow-countLedgerRows),m_Ledger->m_TotalSize); + } + else // Previous row in ledger + { + if(m_Ledger->Open() <= 0) + { + LogString("Error: VerifyConsensus: couldn't open ledger"); + return MC_ERR_DBOPEN_ERROR; + } + + if(m_Ledger->GetRow(prevRow,&pldRow)) + { + LogString("Error: VerifyConsensus: couldn't get row"); + m_Ledger->Close(); + return MC_ERR_DBOPEN_ERROR; + } + + if(memcmp(&pldRow,newRow,m_Database->m_ValueOffset) != 0) + { + LogString("Error: VerifyConsensus: row key mismatch"); + m_Ledger->Close(); + return MC_ERR_DBOPEN_ERROR; + } + + if(prevRow != pldRow.m_ThisRow) + { + LogString("Error: VerifyConsensus: row id mismatch"); + m_Ledger->Close(); + return MC_ERR_DBOPEN_ERROR; + } + } + } + + if(pldRow.m_Consensus > 0) // Reached previous consensus row + { + exit_now=1; + } + + if(exit_now) // We didn't reach consensus, revert blockFrom/To to previous values + { + if(remaining) + { + *remaining=required; + } + newRow->m_Consensus=0; + newRow->m_BlockFrom=lastRow->m_BlockFrom; + newRow->m_BlockTo=lastRow->m_BlockTo; + return MC_ERR_NOERROR; + } + + } + + } + + m_Ledger->Close(); + + return MC_ERR_NOERROR; +} + +/** Fills details buffer for permission row */ + +int mc_Permissions::FillPermissionDetails(mc_PermissionDetails *plsRow,mc_Buffer *plsDetailsBuffer) +{ + int consensus,required,found,exit_now,phase,in_consensus,i; + uint64_t prevRow,countLedgerRows; + mc_PermissionLedgerRow pldRow; + mc_PermissionLedgerRow pldAdmin; + mc_PermissionDetails plsDetails; + mc_PermissionLedgerRow *ptr; + + countLedgerRows=m_Row-m_MemPool->GetCount(); + consensus=AdminConsensus(plsRow->m_Type); + required=consensus; + + plsRow->m_RequiredAdmins=consensus; + + prevRow=plsRow->m_LastRow; + + phase=0; + exit_now=0; + + int path; + + while(exit_now == 0) + { + if(prevRow == 0) + { + LogString("Internal error: FillPermissionDetails: prevRow=0 "); + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(prevRow >= countLedgerRows) // in mempool + { + pldRow.Zero(); + memcpy((unsigned char*)&pldRow+m_Ledger->m_KeyOffset,m_MemPool->GetRow(prevRow-countLedgerRows),m_Ledger->m_TotalSize); + } + else + { + if(m_Ledger->GetRow(prevRow,&pldRow)) + { + m_Ledger->Close(); + LogString("Error: FillPermissionDetails: couldn't get row"); + return MC_ERR_DBOPEN_ERROR; + } + if(memcmp(&pldRow,plsRow,m_Database->m_ValueOffset) != 0) // m_Database->m_ValueOffset are common for all structures + { + m_Ledger->Close(); + LogString("Error: FillPermissionDetails: row key mismatch"); + return MC_ERR_DBOPEN_ERROR; + } + + if(prevRow != pldRow.m_ThisRow) + { + m_Ledger->Close(); + LogString("Error: FillPermissionDetails: row id mismatch"); + return MC_ERR_DBOPEN_ERROR; + } + } + + path=0; + + if(phase == 0) + { + path+=1; + if(pldRow.m_Consensus > 0) // The last row is in consensus + { + memcpy(plsRow->m_LastAdmin,pldRow.m_Admin,MC_PLS_SIZE_ADDRESS); + if(required <= 1) // No pending records, this is the only admin, no details required + { + return MC_ERR_NOERROR; + } + phase=2; // We are in consensus, but want to find all admins + } + else + { + plsRow->m_Flags |= MC_PFL_HAVE_PENDING; + phase=1; // There are pending records + } + if(plsDetailsBuffer == NULL) // We have details, but they are not required + { + return MC_ERR_NOERROR; + } + else + { + m_TmpPool->Clear(); + } + } + else + { + if(pldRow.m_Consensus > 0) // Consensus record + { + path+=128; + if(phase == 1) // We were waiting for consensus record, now we are just looking to fill list of admins + { + path+=256; + phase=2; + } + else // This is previous consensus, we should not go beyond this point + { + path+=512; + exit_now=1; + } + } + } + + if(exit_now == 0) + { + path+=2; + in_consensus=0; // This row match consensus + if(pldRow.m_GrantFrom == plsRow->m_BlockFrom) + { + if(pldRow.m_GrantTo == plsRow->m_BlockTo) + { + if(phase == 2) + { + in_consensus=1; + } + } + } + + ptr=NULL; + if((phase == 1) || (in_consensus>0)) // If we just fill consensus admin list (phase 2) we ignore non-consensus records + { + path+=4; + memcpy(&pldAdmin,&pldRow,sizeof(mc_PermissionLedgerRow)); + memcpy(pldAdmin.m_Address,pldRow.m_Admin,MC_PLS_SIZE_ADDRESS); + pldAdmin.m_Type=in_consensus; + + ptr=&pldAdmin; // Row with address replaced by admin + + found=m_TmpPool->Seek((unsigned char*)ptr); + if(found >= 0) + { + path+=8; + ptr=(mc_PermissionLedgerRow *)m_TmpPool->GetRow(found); + if(ptr->m_Timestamp < pldAdmin.m_Timestamp) + { + if(ptr->m_GrantFrom == plsRow->m_BlockFrom) + { + if(ptr->m_GrantTo == plsRow->m_BlockTo) + { + required++; // Duplicate record, compensate for decrementing below + } + } + m_TmpPool->PutRow(found,&pldAdmin,(unsigned char*)&pldAdmin+m_Database->m_ValueOffset); + ptr=&pldAdmin; + } + else + { + ptr=NULL; // Ignore this record, we already have newer + } + } + else + { + m_TmpPool->Add(&pldAdmin,(unsigned char*)&pldAdmin+m_Database->m_ValueOffset); + } + } + + if(ptr) + { + path+=16; + if(in_consensus) + { + path+=32; + required--; + if(required <= 0) + { + exit_now=1; + } + } + } + } + + if(required > 0) + { + prevRow=pldRow.m_PrevRow; + + if(prevRow == 0) + { + exit_now=1; + } + } + else + { + exit_now=1; + } + } + + for(i=0;im_Count;i++) + { + ptr=(mc_PermissionLedgerRow*)m_TmpPool->GetRow(i); + + memcpy(plsDetails.m_Entity,plsRow->m_Entity,MC_PLS_SIZE_ENTITY); + memcpy(plsDetails.m_Address,plsRow->m_Address,MC_PLS_SIZE_ADDRESS); + plsDetails.m_Type=plsRow->m_Type; + plsDetails.m_BlockFrom=ptr->m_GrantFrom; + plsDetails.m_BlockTo=ptr->m_GrantTo; + plsDetails.m_Flags=ptr->m_Flags; + plsDetails.m_LastRow=ptr->m_ThisRow; + memcpy(plsDetails.m_LastAdmin,ptr->m_Address,MC_PLS_SIZE_ADDRESS); + plsDetails.m_RequiredAdmins=ptr->m_Type; + + plsDetailsBuffer->Add(&plsDetails,(unsigned char*)&plsDetails+m_Ledger->m_ValueOffset); + } + + return MC_ERR_NOERROR; +} + +/** Returns number of admins required for consensus */ + +int mc_Permissions::RequiredForConsensus(const void* lpEntity,const void* lpAddress,uint32_t type,uint32_t from,uint32_t to) +{ + mc_PermissionLedgerRow pldRow; + mc_PermissionLedgerRow pldLast; + int required; + + if(mc_IsNullEntity(lpEntity) == 0) + { + return 1; + } + + GetPermission(lpEntity,lpAddress,type,&pldLast,1); + + switch(type) + { + case MC_PTP_ADMIN: + case MC_PTP_MINE: + case MC_PTP_CONNECT: + case MC_PTP_RECEIVE: + case MC_PTP_SEND: + case MC_PTP_ISSUE: + case MC_PTP_CREATE: + case MC_PTP_ACTIVATE: + case MC_PTP_WRITE: + if(pldLast.m_ThisRow) + { + if(pldLast.m_BlockFrom == from) + { + if(pldLast.m_BlockTo == to) + { + return 0; + } + } + + pldRow.Zero(); + + if(lpEntity) + { + memcpy(pldRow.m_Entity,lpEntity,MC_PLS_SIZE_ENTITY); + } + memcpy(pldRow.m_Address,lpAddress,MC_PLS_SIZE_ADDRESS); + pldRow.m_Type=type; + pldRow.m_PrevRow=pldLast.m_ThisRow; + pldRow.m_BlockFrom=from; + pldRow.m_BlockTo=to; + pldRow.m_BlockReceived=0; + pldRow.m_GrantFrom=from; + pldRow.m_GrantTo=to; + pldRow.m_ThisRow=m_Row; + pldRow.m_Timestamp=mc_TimeNowAsUInt(); + pldRow.m_Flags=0; + pldRow.m_FoundInDB=pldLast.m_FoundInDB; + + if(VerifyConsensus(&pldRow,&pldLast,&required)) + { + return -2; + } + } + else + { + required=AdminConsensus(type); + } + break; + default: + return -1; + } + + return required; +} + +/** Returns permission details for specific record, external, locks */ + +mc_Buffer *mc_Permissions::GetPermissionDetails(mc_PermissionDetails *plsRow) +{ + mc_Buffer *result; + + Lock(0); + + result=new mc_Buffer; + + result->Initialize(m_Ledger->m_ValueOffset,sizeof(mc_PermissionDetails),MC_BUF_MODE_DEFAULT); + + if(m_Ledger->Open() <= 0) + { + LogString("Error: FillPermissionDetails: couldn't open ledger"); + goto exitlbl; + } + + if(FillPermissionDetails(plsRow,result)) + { + delete result; + result=NULL; + } + + m_Ledger->Close(); + +exitlbl: + UnLock(); + + return result; +} + +/** Returns list of permission states */ + +mc_Buffer *mc_Permissions::GetPermissionList(const void* lpEntity,const void* lpAddress,uint32_t type,mc_Buffer *old_buffer) +{ + mc_PermissionDBRow pdbRow; + mc_PermissionLedgerRow pldRow; + mc_PermissionDetails plsRow; + unsigned char *ptr; + mc_PermissionDetails *ptrFound; + + int dbvalue_len,err,i,cur_type,found,first,take_it; + unsigned char null_entity[32]; + memset(null_entity,0,MC_PLS_SIZE_ENTITY); + + Lock(0); + + mc_Buffer *result; + if(old_buffer == NULL) + { + result=new mc_Buffer; + // m_Database->m_ValueOffset bytes are common for all structures + result->Initialize(m_Database->m_ValueOffset,sizeof(mc_PermissionDetails),MC_BUF_MODE_MAP); + } + else + { + result=old_buffer; + } + + if(lpAddress) + { + cur_type=1; + plsRow.Zero(); + + if(lpEntity) + { + memcpy(plsRow.m_Entity,lpEntity,MC_PLS_SIZE_ENTITY); + } + memcpy(plsRow.m_Address,lpAddress,MC_PLS_SIZE_ADDRESS); + for(i=0;i<32;i++) + { + if(cur_type & type) + { + GetPermission(lpEntity,lpAddress,cur_type,&pldRow,1); + if(pldRow.m_ThisRow) + { + plsRow.m_Type=pldRow.m_Type; + plsRow.m_BlockFrom=pldRow.m_BlockFrom; + plsRow.m_BlockTo=pldRow.m_BlockTo; + plsRow.m_Flags=pldRow.m_Flags; + plsRow.m_LastRow=pldRow.m_ThisRow; + result->Add(&plsRow,(unsigned char*)&plsRow+m_Database->m_ValueOffset); + } + } + cur_type=cur_type<<1; + } + } + else + { + pdbRow.Zero(); + + if(lpEntity) + { + memcpy(pdbRow.m_Entity,lpEntity,MC_PLS_SIZE_ENTITY); + pdbRow.m_Type=MC_PTP_CONNECT; + } + + first=1; + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&dbvalue_len,MC_OPT_DB_DATABASE_SEEK_ON_READ,&err); + if(err) + { + LogString("Error: GetPermissionList: db read"); + delete result; + result=NULL; + goto exitlbl; + } + + if(ptr) + { + memcpy((char*)&pdbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + while(ptr) + { + if(first == 0) + { + if(type & pdbRow.m_Type) + { + plsRow.Zero(); + memcpy(plsRow.m_Entity,pdbRow.m_Entity,MC_PLS_SIZE_ENTITY); + memcpy(plsRow.m_Address,pdbRow.m_Address,MC_PLS_SIZE_ADDRESS); + plsRow.m_Type=pdbRow.m_Type; + plsRow.m_BlockFrom=pdbRow.m_BlockFrom; + plsRow.m_BlockTo=pdbRow.m_BlockTo; + plsRow.m_Flags=pdbRow.m_Flags; + plsRow.m_LastRow=pdbRow.m_LedgerRow; + result->Add(&plsRow,(unsigned char*)&plsRow+m_Database->m_ValueOffset); + } + } + first=0; + ptr=(unsigned char*)m_Database->m_DB->MoveNext(&err); + if(ptr) + { + memcpy((char*)&pdbRow+m_Database->m_KeyOffset,ptr,m_Database->m_TotalSize); + if(mc_IsNullEntity(lpEntity)) + { + if(memcmp(pdbRow.m_Entity,null_entity,MC_PLS_SIZE_ENTITY)) + { + ptr=NULL; + } + } + else + { + if(memcmp(pdbRow.m_Entity,lpEntity,MC_PLS_SIZE_ENTITY)) + { + ptr=NULL; + } + } + } + } + } + + for(i=0;iGetCount();i++) + { + ptr=m_MemPool->GetRow(i); + pldRow.Zero(); + memcpy((unsigned char*)&pldRow+m_Ledger->m_KeyOffset,ptr,m_Ledger->m_TotalSize); + take_it=0; + if(mc_IsNullEntity(lpEntity)) + { + if(memcmp(pldRow.m_Entity,null_entity,MC_PLS_SIZE_ENTITY) == 0) + { + take_it=1; + } + } + else + { + if(memcmp(pldRow.m_Entity,lpEntity,MC_PLS_SIZE_ENTITY) == 0) + { + take_it=1; + } + } + if(take_it) + { + if(type & pldRow.m_Type) + { + found=result->Seek(&pldRow); + if(found >= 0) + { + ptrFound=(mc_PermissionDetails*)(result->GetRow(found)); + ptrFound->m_BlockFrom=pldRow.m_BlockFrom; + ptrFound->m_BlockTo=pldRow.m_BlockTo; + ptrFound->m_Flags=pldRow.m_Flags; + ptrFound->m_LastRow=pldRow.m_ThisRow; + } + else + { + if(first == 0) + { + plsRow.Zero(); + memcpy(plsRow.m_Entity,pldRow.m_Entity,MC_PLS_SIZE_ENTITY); + memcpy(plsRow.m_Address,pldRow.m_Address,MC_PLS_SIZE_ADDRESS); + plsRow.m_Type=pldRow.m_Type; + plsRow.m_BlockFrom=pldRow.m_BlockFrom; + plsRow.m_BlockTo=pldRow.m_BlockTo; + plsRow.m_Flags=pldRow.m_Flags; + plsRow.m_LastRow=pldRow.m_ThisRow; + result->Add(&plsRow,(unsigned char*)&plsRow+m_Database->m_ValueOffset); + } + first=0; + } + } + else + { + first=0; + } + } + } + } + + if(m_Ledger->Open() <= 0) + { + LogString("Error: GetPermissionList: couldn't open ledger"); + goto exitlbl; + } + + for(i=0;im_Count;i++) + { + FillPermissionDetails((mc_PermissionDetails*)result->GetRow(i),NULL); + } + + m_Ledger->Close(); + +exitlbl: + UnLock(); + + return result; +} + +/** Frees buffer returned by GetPermissionList */ + +void mc_Permissions::FreePermissionList(mc_Buffer *permissions) +{ + if(permissions) + { + delete permissions; + } +} + +/** Returns 1 if Activate permission is ebough for granting/revoking permission of specified type */ + + +int mc_Permissions::IsActivateEnough(uint32_t type) +{ + if(mc_gState->m_Features->ActivatePermission() == 0) + { + return 0; + } + if(type & ( MC_PTP_ADMIN | MC_PTP_ISSUE | MC_PTP_MINE | MC_PTP_ACTIVATE | MC_PTP_CREATE)) + { + return 0; + } + return 1; +} + +/** Sets permission record, external, locks */ + +int mc_Permissions::SetPermission(const void* lpEntity,const void* lpAddress,uint32_t type,const void* lpAdmin,uint32_t from,uint32_t to,uint32_t timestamp,uint32_t flags,int update_mempool) +{ + int result; + + if(mc_IsNullEntity(lpEntity) || ((flags & MC_PFL_ENTITY_GENESIS) == 0)) + { + if(!CanAdmin(lpEntity,lpAdmin)) + { + if(IsActivateEnough(type)) + { + if(!CanActivate(lpEntity,lpAdmin)) + { + return MC_ERR_NOT_ALLOWED; + } + } + else + { + return MC_ERR_NOT_ALLOWED; + } + } + } + + Lock(1); + result=SetPermissionInternal(lpEntity,lpAddress,type,lpAdmin,from,to,timestamp,flags,update_mempool); + UnLock(); + return result; +} + +/** Sets permission record, internal */ + +int mc_Permissions::SetPermissionInternal(const void* lpEntity,const void* lpAddress,uint32_t type,const void* lpAdmin,uint32_t from,uint32_t to,uint32_t timestamp,uint32_t flags,int update_mempool) +{ + mc_PermissionLedgerRow pldRow; + mc_PermissionLedgerRow pldLast; + int err,i,num_types,thisBlock,lastAllowed,thisAllowed; + char msg[256]; + uint32_t types[9]; + uint32_t pr_entity,pr_address,pr_admin; + num_types=0; + types[num_types]=MC_PTP_CONNECT;num_types++; + types[num_types]=MC_PTP_SEND;num_types++; + types[num_types]=MC_PTP_RECEIVE;num_types++; + if(mc_gState->m_Features->Streams()) + { + types[num_types]=MC_PTP_WRITE;num_types++; + types[num_types]=MC_PTP_CREATE;num_types++; + } + types[num_types]=MC_PTP_ISSUE;num_types++; + types[num_types]=MC_PTP_MINE;num_types++; + if(mc_gState->m_Features->ActivatePermission()) + { + if(mc_gState->m_Features->FixedGrantsInTheSameTx()) + { + types[num_types]=MC_PTP_ACTIVATE;num_types++; + types[num_types]=MC_PTP_ADMIN;num_types++; + } + else + { + types[num_types]=MC_PTP_ADMIN;num_types++; + types[num_types]=MC_PTP_ACTIVATE;num_types++; + } + } + else + { + types[num_types]=MC_PTP_ADMIN;num_types++; + } + + err=MC_ERR_NOERROR; + + + for(i=0;iGetCount() == 0) + { + m_ClearedAdminCount=m_AdminCount; + m_ClearedMinerCount=m_MinerCount; + } + m_MemPool->Add((unsigned char*)&pldRow+m_Ledger->m_KeyOffset,(unsigned char*)&pldRow+m_Ledger->m_ValueOffset); + m_Row++; + + if( (type == MC_PTP_ADMIN) || (type == MC_PTP_MINE)) + { + if(mc_IsNullEntity(lpEntity)) + { + thisAllowed=GetPermission(lpEntity,lpAddress,type); + if(lastAllowed != thisAllowed) + { + if(lastAllowed) + { + thisAllowed=-1; + } + else + { + thisAllowed=1; + } + if(type == MC_PTP_ADMIN) + { + m_AdminCount+=thisAllowed; + } + if(type == MC_PTP_MINE) + { + m_MinerCount+=thisAllowed; + } + } + } + +/* + if(type == MC_PTP_ADMIN) + { + sprintf(msg,"Admin grant: (%08x-%08x-%08x) (%ld-%ld, %ld), In consensus: %d, Admin count: %d, Miner count: %d", + *(uint32_t*)((void*)pldRow.m_Entity),*(uint32_t*)((void*)pldRow.m_Address),*(uint32_t*)((void*)pldRow.m_Admin), + (int64_t)from,(int64_t)to,(int64_t)timestamp,pldRow.m_Consensus,m_AdminCount,m_MinerCount); + } + if(type == MC_PTP_MINE) + { + sprintf(msg,"Miner grant: (%08x-%08x-%08x) (%ld-%ld, %ld), In consensus: %d, Admin count: %d, Miner count: %d", + *(uint32_t*)((void*)pldRow.m_Entity),*(uint32_t*)((void*)pldRow.m_Address),*(uint32_t*)((void*)pldRow.m_Admin), + (int64_t)from,(int64_t)to,(int64_t)timestamp,pldRow.m_Consensus,m_AdminCount,m_MinerCount); + } + */ + } + memcpy(&pr_entity,pldRow.m_Entity,sizeof(uint32_t)); + memcpy(&pr_address,pldRow.m_Address,sizeof(uint32_t)); + memcpy(&pr_admin,pldRow.m_Admin,sizeof(uint32_t)); + sprintf(msg,"Grant: (%08x-%08x-%08x-%08x) (%ld-%ld, %ld), In consensus: %d, Admin count: %d, Miner count: %d", + pr_entity,pr_address,pldRow.m_Type,pr_admin, + (int64_t)from,(int64_t)to,(int64_t)timestamp,pldRow.m_Consensus,m_AdminCount,m_MinerCount); + LogString(msg); + } + } + else + { + err=SetPermissionInternal(lpEntity,lpAddress,types[i],lpAdmin,from,to,timestamp,flags,update_mempool); + if(err) + { + return err; + } + } + } + } + return err; +} + +/** Sets mempool checkpoint */ + + +int mc_Permissions::SetCheckPoint() +{ + Lock(1); + + m_CheckPointRow=m_Row; + m_CheckPointAdminCount=m_AdminCount; + m_CheckPointMinerCount=m_MinerCount; + m_CheckPointMemPoolSize=m_MemPool->GetCount(); + + UnLock(); + + return MC_ERR_NOERROR; +} + +/** Rolls back to mempool checkpoint */ + +int mc_Permissions::RollBackToCheckPoint() +{ + Lock(1); + + m_Row=m_CheckPointRow; + m_AdminCount=m_CheckPointAdminCount; + m_MinerCount=m_CheckPointMinerCount; + m_MemPool->SetCount(m_CheckPointMemPoolSize); + + UnLock(); + + return MC_ERR_NOERROR; +} + +/** Returns address of the specific block miner */ + +int mc_Permissions::GetBlockMiner(uint32_t block,unsigned char* lpMiner) +{ + int err; + int64_t last_block_row; + mc_BlockLedgerRow pldRow; + mc_BlockLedgerRow pldLast; + uint32_t type; + + Lock(0); + + type=MC_PTP_BLOCK_INDEX; + + pldLast.Zero(); + pldRow.Zero(); + + sprintf((char*)pldRow.m_Address,"Block %08X row",block); + + GetPermission(pldRow.m_Address,type,(mc_PermissionLedgerRow*)&pldLast); + + last_block_row=pldLast.m_ThisRow; + + if(last_block_row <= 1) + { + LogString("Error: GetBlockMiner: block row not found"); + err=MC_ERR_NOT_FOUND; + goto exitlbl; + } + + last_block_row--; + err=MC_ERR_NOERROR; + + if(m_Ledger->Open() <= 0) + { + LogString("Error: GetBlockMiner: couldn't open ledger"); + err=MC_ERR_FILE_READ_ERROR; + goto exitlbl; + } + + m_Ledger->GetRow(last_block_row,(mc_PermissionLedgerRow*)&pldRow); + + if(pldRow.m_Type == MC_PTP_BLOCK_MINER) + { + memcpy(lpMiner,pldRow.m_Address,MC_PLS_SIZE_ADDRESS); + } + else + { + LogString("Error: GetBlockMiner: row type mismatch"); + err=MC_ERR_CORRUPTED; + } + + m_Ledger->Close(); + +exitlbl: + UnLock(); + + return err; +} + + +/** Verifies if block of specific height has specified hash */ + +int mc_Permissions::VerifyBlockHash(int32_t block,const void* lpHash) +{ + int type; + int64_t last_block_row; + int result; + + mc_BlockLedgerRow pldRow; + mc_BlockLedgerRow pldLast; + + type=MC_PTP_BLOCK_INDEX; + + pldRow.Zero(); + pldLast.Zero(); + sprintf((char*)(pldRow.m_Address),"Block %08X row",block); + + GetPermission(pldRow.m_Address,type,(mc_PermissionLedgerRow*)&pldLast); + + last_block_row=pldLast.m_ThisRow; + + if(last_block_row == 0) + { + return 0; + } + + if(m_Ledger->Open() <= 0) + { + LogString("Error: VerifyBlockHash: couldn't open ledger"); + return 0; + } + + m_Ledger->GetRow(last_block_row,(mc_PermissionLedgerRow*)&pldRow); + + result=1; + if(memcmp(pldRow.m_CommitHash,lpHash,MC_PLS_SIZE_HASH)) + { + result=0; + } + + m_Ledger->Close(); + + return result; +} + +/** Commits block, external, locks */ + +int mc_Permissions::Commit(const void* lpMiner,const void* lpHash) +{ + int result; + Lock(1); + result=CommitInternal(lpMiner,lpHash); + UnLock(); + return result; +} + +/** Commits block, external, no lock */ + +int mc_Permissions::CommitInternal(const void* lpMiner,const void* lpHash) +{ + int i,err,thisBlock,value_len,pld_items; + char msg[256]; + + mc_PermissionDBRow pdbRow; + mc_PermissionLedgerRow pldRow; + mc_BlockLedgerRow pldBlockRow; + mc_BlockLedgerRow pldBlockLast; + + unsigned char *ptr; + + err=MC_ERR_NOERROR; + + pld_items=m_MemPool->GetCount(); + + if(lpMiner) + { + thisBlock=m_Block+1; + + GetPermission(lpMiner,MC_PTP_BLOCK_MINER,(mc_PermissionLedgerRow*)&pldBlockLast); + + pldBlockRow.Zero(); + memcpy(pldBlockRow.m_Address,lpMiner,MC_PLS_SIZE_ADDRESS); + pldBlockRow.m_Type=MC_PTP_BLOCK_MINER; + pldBlockRow.m_PrevRow=pldBlockLast.m_ThisRow; + pldBlockRow.m_BlockFrom=thisBlock; + pldBlockRow.m_BlockTo=thisBlock; + pldBlockRow.m_BlockReceived=thisBlock; + memcpy(pldBlockRow.m_CommitHash,lpHash,MC_PLS_SIZE_HASH); + pldBlockRow.m_ThisRow=m_Row; + pldBlockRow.m_FoundInDB=pldBlockLast.m_FoundInDB; + m_MemPool->Add((unsigned char*)&pldBlockRow+m_Ledger->m_KeyOffset,(unsigned char*)&pldBlockRow+m_Ledger->m_ValueOffset); + m_Row++; + + pldBlockRow.Zero(); + sprintf((char*)(pldBlockRow.m_Address),"Block %08X row",thisBlock); + pldBlockRow.m_Type=MC_PTP_BLOCK_INDEX; + pldBlockRow.m_PrevRow=m_Row-m_MemPool->GetCount()-1; + pldBlockRow.m_BlockFrom=thisBlock; + pldBlockRow.m_BlockTo=thisBlock; + pldBlockRow.m_BlockReceived=thisBlock; + memcpy(pldBlockRow.m_CommitHash,lpHash,MC_PLS_SIZE_HASH); + pldBlockRow.m_ThisRow=m_Row; + pldBlockRow.m_FoundInDB=pldBlockLast.m_FoundInDB; + m_MemPool->Add((unsigned char*)&pldBlockRow+m_Ledger->m_KeyOffset,(unsigned char*)&pldBlockRow+m_Ledger->m_ValueOffset); + m_Row++; + } + + if(m_Ledger->Open() <= 0) + { + LogString("Error: Commit: couldn't open ledger"); + return MC_ERR_DBOPEN_ERROR; + } + + thisBlock=m_Block+1; + if(m_MemPool->GetCount()) + { + for(i=0;im_KeyOffset,m_MemPool->GetRow(i),m_Ledger->m_TotalSize); + if(i) + { + m_Ledger->WriteRow(&pldRow); + } + else + { + m_Ledger->SetRow(m_Row-m_MemPool->GetCount(),&pldRow); + } + } + + if(err == MC_ERR_NOERROR) + { + for(i=0;iGetCount();i++) + { + if(err == MC_ERR_NOERROR) + { + memcpy((unsigned char*)&pldRow+m_Ledger->m_KeyOffset,m_MemPool->GetRow(i),m_Ledger->m_TotalSize); + + pdbRow.Zero(); + memcpy(pdbRow.m_Entity,pldRow.m_Entity,MC_PLS_SIZE_ENTITY); + memcpy(pdbRow.m_Address,pldRow.m_Address,MC_PLS_SIZE_ADDRESS); + pdbRow.m_Type=pldRow.m_Type; + pdbRow.m_BlockFrom=pldRow.m_BlockFrom; + pdbRow.m_BlockTo=pldRow.m_BlockTo; + pdbRow.m_LedgerRow=pldRow.m_ThisRow; + pdbRow.m_Flags=pldRow.m_Flags; + + if(pldRow.m_FoundInDB == 0 ) + { + m_DBRowCount++; + } + err=m_Database->m_DB->Write((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&pdbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + LogString("Error: Commit: DB write error"); + } + } + } + } + } + + if(err == MC_ERR_NOERROR) + { + pdbRow.Zero(); + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(ptr) + { + memcpy((char*)&pdbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + } + else + { + m_DBRowCount++; + } + + pdbRow.m_BlockTo=thisBlock; + pdbRow.m_LedgerRow=m_Row; + err=m_Database->m_DB->Write((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&pdbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + LogString("Error: Commit: DB write error (0)"); + } + } + + if(err == MC_ERR_NOERROR) + { + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + LogString("Error: Commit: DB commit error"); + } + } + + if(err == MC_ERR_NOERROR) + { + m_Block++; + UpdateCounts(); + m_Block--; + m_ClearedAdminCount=m_AdminCount; + m_ClearedMinerCount=m_MinerCount; + for(i=pld_items;iGetCount();i++) + { + memcpy((unsigned char*)&pldBlockRow+m_Ledger->m_KeyOffset,m_MemPool->GetRow(i),m_Ledger->m_TotalSize); + pldBlockRow.m_AdminCount=m_AdminCount; + pldBlockRow.m_MinerCount=m_MinerCount; + if(i) + { + m_Ledger->WriteRow((mc_PermissionLedgerRow*)&pldBlockRow); + } + else + { + m_Ledger->SetRow(m_Row-m_MemPool->GetCount(),(mc_PermissionLedgerRow*)&pldBlockRow); + } + } + + } + + if(err == MC_ERR_NOERROR) + { + m_Ledger->GetRow(0,&pldRow); + + thisBlock=m_Block+1; + pldRow.m_BlockTo=thisBlock; + pldRow.m_PrevRow=m_Row; + m_Ledger->SetRow(0,&pldRow); + } + + m_Ledger->Close(); + + + if(err) + { + RollBackInternal(m_Block); + } + else + { + m_MemPool->Clear(); + m_Block++; +/* + UpdateCounts(); + m_ClearedAdminCount=m_AdminCount; + m_ClearedMinerCount=m_MinerCount; + */ + } + + + sprintf(msg,"Block commit: %9d (Hash: %08x, Miner: %08x), Admin count: %d, Miner count: %d, DB rows: %d, Ledger Rows: %ld", + m_Block,*(uint32_t*)lpHash,*(uint32_t*)lpMiner,m_AdminCount,m_MinerCount,m_DBRowCount,m_Row); + LogString(msg); + return err; +} + +/** Rolls one block back */ + +int mc_Permissions::RollBack() +{ + return RollBack(m_Block-1); +} + +/** Rolls back to specific height, external, locks */ + +int mc_Permissions::RollBack(int block) +{ + int result; + Lock(1); + result=RollBackInternal(block); + UnLock(); + return result; +} + +/** Rolls back to specific height, internal, no lock */ + +int mc_Permissions::RollBackInternal(int block) +{ + int err; + int take_it,value_len; + uint64_t this_row,prev_row; + mc_PermissionDBRow pdbRow; + mc_PermissionLedgerRow pldRow; + unsigned char *ptr; + char msg[256]; + + err=MC_ERR_NOERROR; + + if(m_Ledger->Open() <= 0) + { + LogString("Error: Rollback: couldn't open ledger"); + return MC_ERR_DBOPEN_ERROR; + } + + ClearMemPoolInternal(); + + this_row=m_Row-1; + take_it=1; + if(this_row == 0) + { + take_it=0; + } + + while(take_it && (this_row>0)) + { + m_Ledger->GetRow(this_row,&pldRow); + + if((int32_t)pldRow.m_BlockReceived > block) + { + pdbRow.Zero(); + memcpy(pdbRow.m_Entity,pldRow.m_Entity,MC_PLS_SIZE_ENTITY); + memcpy(pdbRow.m_Address,pldRow.m_Address,MC_PLS_SIZE_ADDRESS); + pdbRow.m_Type=pldRow.m_Type; + prev_row=pldRow.m_PrevRow; + if(prev_row) + { + m_Ledger->GetRow(prev_row,&pldRow); + pdbRow.m_BlockFrom=pldRow.m_BlockFrom; + pdbRow.m_BlockTo=pldRow.m_BlockTo; + pdbRow.m_LedgerRow=prev_row; + pdbRow.m_Flags=pldRow.m_Flags; + err=m_Database->m_DB->Write((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&pdbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + LogString("Error: Rollback: db write error"); + } + } + else + { + m_DBRowCount--; + err=m_Database->m_DB->Delete((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + LogString("Error: Rollback: db delete error"); + } + } + } + else + { + take_it=0; + } + if(err) + { + take_it=0; + } + if(take_it) + { + this_row--; + } + } + + this_row++; + + if(err == MC_ERR_NOERROR) + { + m_Ledger->GetRow(0,&pldRow); + + pldRow.m_BlockTo=block; + pldRow.m_PrevRow=this_row; + m_Ledger->SetRow(0,&pldRow); + } + + m_Ledger->Close(); + + if(err == MC_ERR_NOERROR) + { + pdbRow.Zero(); + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(ptr) + { + memcpy((char*)&pdbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + } + else + { + m_DBRowCount++; + } + + pdbRow.m_BlockTo=block; + pdbRow.m_LedgerRow=this_row; + err=m_Database->m_DB->Write((char*)&pdbRow+m_Database->m_KeyOffset,m_Database->m_KeySize, + (char*)&pdbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + LogString("Error: Rollback: db write error (0)"); + } + } + + if(err == MC_ERR_NOERROR) + { + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + + if(err == MC_ERR_NOERROR) + { + m_Block=block; + m_Row=this_row; + UpdateCounts(); + m_ClearedAdminCount=m_AdminCount; + m_ClearedMinerCount=m_MinerCount; + } + + sprintf(msg,"Block rollback: %9d, Admin count: %d, Miner count: %d, DB rows: %d, Ledger Rows: %ld",m_Block,m_AdminCount,m_MinerCount,m_DBRowCount,m_Row); + LogString(msg); + return err; +} + + + diff --git a/src/permissions/permission.h b/src/permissions/permission.h new file mode 100644 index 00000000..849b41a0 --- /dev/null +++ b/src/permissions/permission.h @@ -0,0 +1,298 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAIN_PERMISSION_H +#define MULTICHAIN_PERMISSION_H + +#include "utils/declare.h" +#include "utils/dbwrapper.h" + +#define MC_PTP_CONNECT 0x00000001 +#define MC_PTP_SEND 0x00000002 +#define MC_PTP_RECEIVE 0x00000004 +#define MC_PTP_WRITE 0x00000008 +#define MC_PTP_ISSUE 0x00000010 +#define MC_PTP_CREATE 0x00000020 +#define MC_PTP_MINE 0x00000100 +#define MC_PTP_ADMIN 0x00001000 +#define MC_PTP_ACTIVATE 0x00002000 +#define MC_PTP_BLOCK_MINER 0x01000000 +#define MC_PTP_BLOCK_INDEX 0x02000000 +#define MC_PTP_ALL 0x00FFFFFF +#define MC_PTP_GLOBAL_ALL 0x00003137 + +#define MC_PTP_CACHED_SCRIPT_REQUIRED 0x02000000 // Special temporary value for coin selection + + +#define MC_PFL_NONE 0x00000000 +#define MC_PFL_IS_SCRIPTHASH 0x00000001 +#define MC_PFL_ENTITY_GENESIS 0x00000002 +#define MC_PFL_HAVE_PENDING 0x01000000 + +#define MC_PLS_SIZE_ENTITY 32 +#define MC_PLS_SIZE_ADDRESS 20 +#define MC_PLS_SIZE_HASH 32 + + + +/** Database record structure */ + +typedef struct mc_PermissionDBRow +{ + unsigned char m_Entity[MC_PLS_SIZE_ENTITY]; // Entity genesis transaction TxID, all 0s for master permissions + unsigned char m_Address[MC_PLS_SIZE_ADDRESS]; // Address + uint32_t m_Type; // Permission type MC_PTP_ constants + uint32_t m_BlockFrom; // Permission block-from + uint32_t m_BlockTo; // Permission block-to + uint64_t m_LedgerRow; // Row in the ledger corresponding to the last change for this key + uint32_t m_Flags; // Flags MC_PFL_ constants + uint32_t m_Reserved1; // Reserved to align to 80 bytes + void Zero(); +} mc_PermissionDBRow; + +/** Database */ + +typedef struct mc_PermissionDB +{ + char m_FileName[MC_DCT_DB_MAX_PATH]; // Full file name + mc_Database *m_DB; // Database object + uint32_t m_KeyOffset; // Offset of the key in mc_PermissionDBRow structure, 32 for protocol<=10003, 0 otherwise + uint32_t m_KeySize; // Size of the database key, 24 for protocol<=10003, 56 otherwise + uint32_t m_ValueOffset; // Offset of the value in mc_PermissionDBRow structure, 56 + uint32_t m_ValueSize; // Size of the database value, 24 + uint32_t m_TotalSize; // Totals size of the database row + mc_PermissionDB() + { + Zero(); + } + + ~mc_PermissionDB() + { + Close(); + } + void Zero(); + int Open(); + int Close(); + void SetName(const char *name); +} mc_PermissionDB; + +/** Ledger and mempool record structure */ + +typedef struct mc_PermissionLedgerRow +{ + unsigned char m_Entity[MC_PLS_SIZE_ENTITY]; // Entity genesis transaction TxID, all 0s for master permissions + unsigned char m_Address[MC_PLS_SIZE_ADDRESS]; // Address + uint32_t m_Type; // Permission type MC_PTP_ constants + uint64_t m_PrevRow; // Previous row in the ledger corresponding to this key + uint32_t m_BlockFrom; // Permission block-from after processing this row + uint32_t m_BlockTo; // Permission block-to after processing this row + uint32_t m_Consensus; // If consensus is reached returns number of admins, 0 otherwise + uint32_t m_Flags; // Flags MC_PFL_ constants + unsigned char m_Admin[MC_PLS_SIZE_ADDRESS]; // Admin address + uint32_t m_GrantFrom; // Permission block-from specified in this row + uint32_t m_GrantTo; // Permission block-to specified in this row + uint32_t m_Timestamp; // Timestamp of this row + uint32_t m_FoundInDB; // Row is found in database + uint32_t m_BlockReceived; // Block this transaction was confirmed + uint64_t m_ThisRow; // Row in the ledger + + void Zero(); +} mc_PermissionLedgerRow; + +/** Block details row stored in permission ledger */ + +typedef struct mc_BlockLedgerRow +{ + unsigned char m_Entity[MC_PLS_SIZE_ENTITY]; // Placeholder, always zeroes + unsigned char m_Address[MC_PLS_SIZE_ADDRESS]; // Miner Address or sprintf(...,"Block %08X row",) + uint32_t m_Type; // MC_PTP_BLOCK_MINER or MC_PTP_BLOCK_INDEX + uint64_t m_PrevRow; // Previous row in the ledger corresponding to this key + uint32_t m_BlockFrom; // BlockHeight + uint32_t m_BlockTo; // BlockHeight + uint32_t m_AdminCount; // AdminCount + uint32_t m_MinerCount; // MinerCount + unsigned char m_CommitHash[MC_PLS_SIZE_HASH]; // BlockHash + uint32_t m_FoundInDB; // Row is found in database + uint32_t m_BlockReceived; // BlockHeight + uint64_t m_ThisRow; // Row in the ledger + + void Zero(); +} mc_BlockLedgerRow; + + +/** Ledger */ + +typedef struct mc_PermissionLedger +{ + char m_FileName[MC_DCT_DB_MAX_PATH]; // Full file name + int m_FileHan; // File handle + uint32_t m_KeyOffset; // Offset of the key in mc_PermissionLedgerRow structure, 32 for protocol<=10003, 0 otherwise + uint32_t m_KeySize; // Size of the ledger key, 24 for protocol<=10003, 56 otherwise + uint32_t m_ValueOffset; // Offset of the value in mc_PermissionLedgerRow structure, 56 + uint32_t m_ValueSize; // Size of the ledger value 72 + uint32_t m_TotalSize; // Totals size of the ledger row + + mc_PermissionLedger() + { + Zero(); + } + + ~mc_PermissionLedger() + { + Close(); + } + + void Zero(); + int Open(); + int Close(); + void SetName(const char *name); + int GetRow(uint64_t RowID,mc_PermissionLedgerRow *row); + uint64_t GetSize(); + int WriteRow(mc_PermissionLedgerRow *row); + int SetRow(uint64_t RowID,mc_PermissionLedgerRow *row); + +} mc_PermissionLedger; + +/** Permission details structure */ + +typedef struct mc_PermissionDetails +{ + unsigned char m_Entity[MC_PLS_SIZE_ENTITY]; // Entity genesis transaction TxID, all 0s for master permissions + unsigned char m_Address[MC_PLS_SIZE_ADDRESS]; // Address + uint32_t m_Type; // Permission type MC_PTP_ constants + uint32_t m_BlockFrom; // Permission block-from after processing this row + uint32_t m_BlockTo; // Permission block-to after processing this row + uint32_t m_Flags; // Flags MC_PFL_ constants + int32_t m_RequiredAdmins; // Number of admins required for consensus + unsigned char m_LastAdmin[MC_PLS_SIZE_ADDRESS]; // Last admin address + uint32_t m_Reserved1; // Reserved for alignment + uint64_t m_LastRow; // Last row in the ledger + void Zero(); +} mc_PermissionDetails; + + + + +typedef struct mc_Permissions +{ + mc_PermissionDB *m_Database; + mc_PermissionLedger *m_Ledger; + mc_Buffer *m_MemPool; + mc_Buffer *m_TmpPool; + char m_Name[MC_PRM_NETWORK_NAME_MAX_SIZE+1]; + char m_LogFileName[MC_DCT_DB_MAX_PATH+1]; + int m_Block; + uint64_t m_Row; + int m_AdminCount; + int m_MinerCount; + int m_DBRowCount; + + uint64_t m_CheckPointRow; + int m_CheckPointAdminCount; + int m_CheckPointMinerCount; + uint64_t m_CheckPointMemPoolSize; + + int m_CopiedAdminCount; + int m_CopiedMinerCount; + int m_ClearedAdminCount; + int m_ClearedMinerCount; + + mc_Buffer *m_CopiedMemPool; + + void *m_Semaphore; + uint64_t m_LockedBy; + + mc_Permissions() + { + Zero(); + } + + ~mc_Permissions() + { + Destroy(); + } + +// External functions + int Initialize(const char *name,int mode); + + int SetPermission(const void* lpEntity,const void* lpAddress,uint32_t type,const void* lpAdmin,uint32_t from,uint32_t to,uint32_t timestamp,uint32_t flags,int update_mempool); + int Commit(const void* lpMiner,const void* lpHash); + int RollBack(int block); + int RollBack(); + + int ClearMemPool(); + int CopyMemPool(); + int RestoreMemPool(); + + int SetCheckPoint(); + int RollBackToCheckPoint(); + + uint32_t GetAllPermissions(const void* lpEntity,const void* lpAddress,uint32_t type); + uint32_t GetPermissionType(const char *str,int entity_type); + uint32_t GetPossiblePermissionTypes(uint32_t entity_type); + + int GetAdminCount(); + int GetMinerCount(); + int GetActiveMinerCount(); + + int GetBlockMiner(uint32_t block,unsigned char* lpMiner); + + int CanConnect(const void* lpEntity,const void* lpAddress); + int CanSend(const void* lpEntity,const void* lpAddress); + int CanReceive(const void* lpEntity,const void* lpAddress); + int CanWrite(const void* lpEntity,const void* lpAddress); + int CanCreate(const void* lpEntity,const void* lpAddress); + int CanIssue(const void* lpEntity,const void* lpAddress); + int CanMine(const void* lpEntity,const void* lpAddress); + int CanAdmin(const void* lpEntity,const void* lpAddress); + int CanActivate(const void* lpEntity,const void* lpAddress); + + int CanMineBlock(const void* lpAddress,uint32_t block); + + int IsActivateEnough(uint32_t type); + + + mc_Buffer *GetPermissionList(const void* lpEntity,const void* lpAddress,uint32_t type,mc_Buffer *old_buffer); + mc_Buffer *GetPermissionDetails(mc_PermissionDetails *plsRow); + void FreePermissionList(mc_Buffer *permissions); + +// Internal functions + int Zero(); + int Destroy(); + + void Lock(int write_mode); + void UnLock(); + + int SetPermissionInternal(const void* lpEntity,const void* lpAddress,uint32_t type,const void* lpAdmin,uint32_t from,uint32_t to,uint32_t timestamp,uint32_t flags,int update_mempool); + int CommitInternal(const void* lpMiner,const void* lpHash); + int RollBackInternal(int block); + + int UpdateCounts(); + int AdminConsensus(uint32_t type); + + int ClearMemPoolInternal(); + + + + int RequiredForConsensus(const void* lpEntity,const void* lpAddress,uint32_t type,uint32_t from,uint32_t to); + int VerifyConsensus(mc_PermissionLedgerRow *newRow,mc_PermissionLedgerRow *lastRow,int *remaining); + int VerifyBlockHash(int32_t block,const void* lpHash); + int FillPermissionDetails(mc_PermissionDetails *plsRow,mc_Buffer *plsDetailsBuffer); + + uint32_t GetPermission(const void* lpAddress,uint32_t type); + uint32_t GetPermission(const void* lpEntity,const void* lpAddress,uint32_t type); + uint32_t GetPermission(const void* lpAddress,uint32_t type,mc_PermissionLedgerRow *row); + uint32_t GetPermission(const void* lpEntity,const void* lpAddress,uint32_t type,mc_PermissionLedgerRow *row,int checkmempool); + int IsSetupPeriod(); + + + void LogString(const char *message); + void Dump(); + + +} mc_Permissions; + + + +#endif /* MULTICHAIN_PERMISSION_H */ + diff --git a/src/primitives/block.cpp b/src/primitives/block.cpp new file mode 100644 index 00000000..fe5519c3 --- /dev/null +++ b/src/primitives/block.cpp @@ -0,0 +1,152 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "primitives/block.h" + +#include "structs/hash.h" +#include "utils/tinyformat.h" +#include "utils/utilstrencodings.h" + +uint256 CBlockHeader::GetHash() const +{ + return Hash(BEGIN(nVersion), END(nNonce)); +} + +uint256 CBlock::BuildMerkleTree(bool* fMutated) const +{ + /* WARNING! If you're reading this because you're learning about crypto + and/or designing a new system that will use merkle trees, keep in mind + that the following merkle tree algorithm has a serious flaw related to + duplicate txids, resulting in a vulnerability (CVE-2012-2459). + + The reason is that if the number of hashes in the list at a given time + is odd, the last one is duplicated before computing the next level (which + is unusual in Merkle trees). This results in certain sequences of + transactions leading to the same merkle root. For example, these two + trees: + + A A + / \ / \ + B C B C + / \ | / \ / \ + D E F D E F F + / \ / \ / \ / \ / \ / \ / \ + 1 2 3 4 5 6 1 2 3 4 5 6 5 6 + + for transaction lists [1,2,3,4,5,6] and [1,2,3,4,5,6,5,6] (where 5 and + 6 are repeated) result in the same root hash A (because the hash of both + of (F) and (F,F) is C). + + The vulnerability results from being able to send a block with such a + transaction list, with the same merkle root, and the same block hash as + the original without duplication, resulting in failed validation. If the + receiving node proceeds to mark that block as permanently invalid + however, it will fail to accept further unmodified (and thus potentially + valid) versions of the same block. We defend against this by detecting + the case where we would hash two identical hashes at the end of the list + together, and treating that identically to the block having an invalid + merkle root. Assuming no double-SHA256 collisions, this will detect all + known ways of changing the transactions without affecting the merkle + root. + */ + vMerkleTree.clear(); + vMerkleTree.reserve(vtx.size() * 2 + 16); // Safe upper bound for the number of total nodes. + for (std::vector::const_iterator it(vtx.begin()); it != vtx.end(); ++it) + { +/* MCHN START */ + if((nMerkleTreeType == MERKLETREE_NO_COINBASE_OP_RETURN) && it->IsCoinBase()) + { + CMutableTransaction tx=*it; + tx.vout.clear(); + for(int i=0;i<(int)it->vout.size();i++) + { +// if(it->vout[i].scriptPubKey.begin()[0] != 0x6a) + if(!it->vout[i].scriptPubKey.IsUnspendable()) + { + tx.vout.push_back(it->vout[i]); + } + } + vMerkleTree.push_back(tx.GetHash()); + } + else + { + vMerkleTree.push_back(it->GetHash()); + } +/* MCHN END */ + } + int j = 0; + bool mutated = false; + for (int nSize = vtx.size(); nSize > 1; nSize = (nSize + 1) / 2) + { + for (int i = 0; i < nSize; i += 2) + { + int i2 = std::min(i+1, nSize-1); + if (i2 == i + 1 && i2 + 1 == nSize && vMerkleTree[j+i] == vMerkleTree[j+i2]) { + // Two identical hashes at the end of the list at a particular level. + mutated = true; + } + vMerkleTree.push_back(Hash(BEGIN(vMerkleTree[j+i]), END(vMerkleTree[j+i]), + BEGIN(vMerkleTree[j+i2]), END(vMerkleTree[j+i2]))); + } + j += nSize; + } + if (fMutated) { + *fMutated = mutated; + } + return (vMerkleTree.empty() ? 0 : vMerkleTree.back()); +} + +std::vector CBlock::GetMerkleBranch(int nIndex) const +{ + if (vMerkleTree.empty()) + BuildMerkleTree(); + std::vector vMerkleBranch; + int j = 0; + for (int nSize = vtx.size(); nSize > 1; nSize = (nSize + 1) / 2) + { + int i = std::min(nIndex^1, nSize-1); + vMerkleBranch.push_back(vMerkleTree[j+i]); + nIndex >>= 1; + j += nSize; + } + return vMerkleBranch; +} + +uint256 CBlock::CheckMerkleBranch(uint256 hash, const std::vector& vMerkleBranch, int nIndex) +{ + if (nIndex == -1) + return 0; + for (std::vector::const_iterator it(vMerkleBranch.begin()); it != vMerkleBranch.end(); ++it) + { + if (nIndex & 1) + hash = Hash(BEGIN(*it), END(*it), BEGIN(hash), END(hash)); + else + hash = Hash(BEGIN(hash), END(hash), BEGIN(*it), END(*it)); + nIndex >>= 1; + } + return hash; +} + +std::string CBlock::ToString() const +{ + std::stringstream s; + s << strprintf("CBlock(hash=%s, ver=%d, hashPrevBlock=%s, hashMerkleRoot=%s, nTime=%u, nBits=%08x, nNonce=%u, vtx=%u)\n", + GetHash().ToString(), + nVersion, + hashPrevBlock.ToString(), + hashMerkleRoot.ToString(), + nTime, nBits, nNonce, + vtx.size()); + for (unsigned int i = 0; i < vtx.size(); i++) + { + s << " " << vtx[i].ToString() << "\n"; + } + s << " vMerkleTree: "; + for (unsigned int i = 0; i < vMerkleTree.size(); i++) + s << " " << vMerkleTree[i].ToString(); + s << "\n"; + return s.str(); +} diff --git a/src/primitives/block.h b/src/primitives/block.h new file mode 100644 index 00000000..99fe3796 --- /dev/null +++ b/src/primitives/block.h @@ -0,0 +1,202 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_PRIMITIVES_BLOCK_H +#define BITCOIN_PRIMITIVES_BLOCK_H + +#include "primitives/transaction.h" +#include "utils/serialize.h" +#include "structs/uint256.h" + +/** The maximum allowed size for a serialized block, in bytes (network rule) */ +//static const unsigned int MAX_BLOCK_SIZE = 1000000; +extern unsigned int MAX_BLOCK_SIZE; // MCHN global + +/* MCHN START */ +/** Block signature hash types/flags */ +enum +{ + BLOCKSIGHASH_NONE = 0, + BLOCKSIGHASH_HEADER = 1, + BLOCKSIGHASH_NO_SIGNATURE_AND_NONCE = 2, + BLOCKSIGHASH_INVALID = 0xfe, + BLOCKSIGHASH_UNKNOWN = 0xff, +}; +enum +{ + MERKLETREE_FULL = 0, + MERKLETREE_NO_COINBASE_OP_RETURN = 1, + MERKLETREE_UNKNOWN = 0xff, +}; +/* MCHN END */ + + +/** Nodes collect new transactions into a block, hash them into a hash tree, + * and scan through nonce values to make the block's hash satisfy proof-of-work + * requirements. When they solve the proof-of-work, they broadcast the block + * to everyone and the block is added to the block chain. The first transaction + * in the block is a special one that creates a new coin owned by the creator + * of the block. + */ +class CBlockHeader +{ +public: + // header + static const int32_t CURRENT_VERSION=3; + int32_t nVersion; + uint256 hashPrevBlock; + uint256 hashMerkleRoot; + uint32_t nTime; + uint32_t nBits; + uint32_t nNonce; + + CBlockHeader() + { + SetNull(); + } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(this->nVersion); + nVersion = this->nVersion; + READWRITE(hashPrevBlock); + READWRITE(hashMerkleRoot); + READWRITE(nTime); + READWRITE(nBits); + READWRITE(nNonce); + } + + void SetNull() + { + nVersion = CBlockHeader::CURRENT_VERSION; + hashPrevBlock = 0; + hashMerkleRoot = 0; + nTime = 0; + nBits = 0; + nNonce = 0; + } + + bool IsNull() const + { + return (nBits == 0); + } + + uint256 GetHash() const; + + int64_t GetBlockTime() const + { + return (int64_t)nTime; + } +}; + + +class CBlock : public CBlockHeader +{ +public: + // network and disk + std::vector vtx; + + // memory only + mutable std::vector vMerkleTree; + +/* MCHN START */ + uint32_t nSigHashType; + uint32_t nMerkleTreeType; + unsigned char vSigner[256]; +/* MCHN END */ + + CBlock() + { + SetNull(); + } + + CBlock(const CBlockHeader &header) + { + SetNull(); + *((CBlockHeader*)this) = header; + } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(*(CBlockHeader*)this); + READWRITE(vtx); + } + + void SetNull() + { + CBlockHeader::SetNull(); + vtx.clear(); + vMerkleTree.clear(); +/* MCHN START */ + nSigHashType=BLOCKSIGHASH_UNKNOWN; + nMerkleTreeType=MERKLETREE_UNKNOWN; +/* MCHN END */ + } + + CBlockHeader GetBlockHeader() const + { + CBlockHeader block; + block.nVersion = nVersion; + block.hashPrevBlock = hashPrevBlock; + block.hashMerkleRoot = hashMerkleRoot; + block.nTime = nTime; + block.nBits = nBits; + block.nNonce = nNonce; + return block; + } + + // Build the in-memory merkle tree for this block and return the merkle root. + // If non-NULL, *mutated is set to whether mutation was detected in the merkle + // tree (a duplication of transactions in the block leading to an identical + // merkle root). + uint256 BuildMerkleTree(bool* mutated = NULL) const; + + std::vector GetMerkleBranch(int nIndex) const; + static uint256 CheckMerkleBranch(uint256 hash, const std::vector& vMerkleBranch, int nIndex); + std::string ToString() const; +}; + + +/** Describes a place in the block chain to another node such that if the + * other node doesn't have the same branch, it can find a recent common trunk. + * The further back it is, the further before the fork it may be. + */ +struct CBlockLocator +{ + std::vector vHave; + + CBlockLocator() {} + + CBlockLocator(const std::vector& vHaveIn) + { + vHave = vHaveIn; + } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + if (!(nType & SER_GETHASH)) + READWRITE(nVersion); + READWRITE(vHave); + } + + void SetNull() + { + vHave.clear(); + } + + bool IsNull() + { + return vHave.empty(); + } +}; + +#endif // BITCOIN_PRIMITIVES_BLOCK_H diff --git a/src/primitives/transaction.cpp b/src/primitives/transaction.cpp new file mode 100644 index 00000000..d4817855 --- /dev/null +++ b/src/primitives/transaction.cpp @@ -0,0 +1,149 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "primitives/transaction.h" + +#include "structs/hash.h" +#include "utils/tinyformat.h" +#include "utils/utilstrencodings.h" + +std::string COutPoint::ToString() const +{ + return strprintf("COutPoint(%s, %u)", hash.ToString().substr(0,10), n); +} + +CTxIn::CTxIn(COutPoint prevoutIn, CScript scriptSigIn, uint32_t nSequenceIn) +{ + prevout = prevoutIn; + scriptSig = scriptSigIn; + nSequence = nSequenceIn; +} + +CTxIn::CTxIn(uint256 hashPrevTx, uint32_t nOut, CScript scriptSigIn, uint32_t nSequenceIn) +{ + prevout = COutPoint(hashPrevTx, nOut); + scriptSig = scriptSigIn; + nSequence = nSequenceIn; +} + +std::string CTxIn::ToString() const +{ + std::string str; + str += "CTxIn("; + str += prevout.ToString(); + if (prevout.IsNull()) + str += strprintf(", coinbase %s", HexStr(scriptSig)); + else + str += strprintf(", scriptSig=%s", scriptSig.ToString().substr(0,24)); + if (nSequence != std::numeric_limits::max()) + str += strprintf(", nSequence=%u", nSequence); + str += ")"; + return str; +} + +CTxOut::CTxOut(const CAmount& nValueIn, CScript scriptPubKeyIn) +{ + nValue = nValueIn; + scriptPubKey = scriptPubKeyIn; +} + +uint256 CTxOut::GetHash() const +{ + return SerializeHash(*this); +} + +std::string CTxOut::ToString() const +{ +/* MCHN START */ + if(COIN == 0) + { + return strprintf("CTxOut(nValue=%d.%08d, scriptPubKey=%s)", nValue, nValue, scriptPubKey.ToString().substr(0,30)); + } +/* MCHN END */ + return strprintf("CTxOut(nValue=%d.%08d, scriptPubKey=%s)", nValue / COIN, nValue % COIN, scriptPubKey.ToString().substr(0,30)); +} + +CMutableTransaction::CMutableTransaction() : nVersion(CTransaction::CURRENT_VERSION), nLockTime(0) {} +CMutableTransaction::CMutableTransaction(const CTransaction& tx) : nVersion(tx.nVersion), vin(tx.vin), vout(tx.vout), nLockTime(tx.nLockTime) {} + +uint256 CMutableTransaction::GetHash() const +{ + return SerializeHash(*this); +} + +void CTransaction::UpdateHash() const +{ + *const_cast(&hash) = SerializeHash(*this); +} + +CTransaction::CTransaction() : hash(0), nVersion(CTransaction::CURRENT_VERSION), vin(), vout(), nLockTime(0) { } + +CTransaction::CTransaction(const CMutableTransaction &tx) : nVersion(tx.nVersion), vin(tx.vin), vout(tx.vout), nLockTime(tx.nLockTime) { + UpdateHash(); +} + +CTransaction& CTransaction::operator=(const CTransaction &tx) { + *const_cast(&nVersion) = tx.nVersion; + *const_cast*>(&vin) = tx.vin; + *const_cast*>(&vout) = tx.vout; + *const_cast(&nLockTime) = tx.nLockTime; + *const_cast(&hash) = tx.hash; + return *this; +} + +CAmount CTransaction::GetValueOut() const +{ + CAmount nValueOut = 0; + for (std::vector::const_iterator it(vout.begin()); it != vout.end(); ++it) + { + nValueOut += it->nValue; + if (!MoneyRange(it->nValue) || !MoneyRange(nValueOut)) + throw std::runtime_error("CTransaction::GetValueOut() : value out of range"); + } + return nValueOut; +} + +double CTransaction::ComputePriority(double dPriorityInputs, unsigned int nTxSize) const +{ + nTxSize = CalculateModifiedSize(nTxSize); + if (nTxSize == 0) return 0.0; + + return dPriorityInputs / nTxSize; +} + +unsigned int CTransaction::CalculateModifiedSize(unsigned int nTxSize) const +{ + // In order to avoid disincentivizing cleaning up the UTXO set we don't count + // the constant overhead for each txin and up to 110 bytes of scriptSig (which + // is enough to cover a compressed pubkey p2sh redemption) for priority. + // Providing any more cleanup incentive than making additional inputs free would + // risk encouraging people to create junk outputs to redeem later. + if (nTxSize == 0) + nTxSize = ::GetSerializeSize(*this, SER_NETWORK, PROTOCOL_VERSION); + for (std::vector::const_iterator it(vin.begin()); it != vin.end(); ++it) + { + unsigned int offset = 41U + std::min(110U, (unsigned int)it->scriptSig.size()); + if (nTxSize > offset) + nTxSize -= offset; + } + return nTxSize; +} + +std::string CTransaction::ToString() const +{ + std::string str; + str += strprintf("CTransaction(hash=%s, ver=%d, vin.size=%u, vout.size=%u, nLockTime=%u)\n", + GetHash().ToString().substr(0,10), + nVersion, + vin.size(), + vout.size(), + nLockTime); + for (unsigned int i = 0; i < vin.size(); i++) + str += " " + vin[i].ToString() + "\n"; + for (unsigned int i = 0; i < vout.size(); i++) + str += " " + vout[i].ToString() + "\n"; + return str; +} diff --git a/src/primitives/transaction.h b/src/primitives/transaction.h new file mode 100644 index 00000000..58de1450 --- /dev/null +++ b/src/primitives/transaction.h @@ -0,0 +1,309 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_PRIMITIVES_TRANSACTION_H +#define BITCOIN_PRIMITIVES_TRANSACTION_H + +#include "structs/amount.h" +#include "script/script.h" +#include "utils/serialize.h" +#include "structs/uint256.h" + +//#include "multichain/multichain.h" +#include "chainparams/state.h" + +/** An outpoint - a combination of a transaction hash and an index n into its vout */ +class COutPoint +{ +public: + uint256 hash; + uint32_t n; + + COutPoint() { SetNull(); } + COutPoint(uint256 hashIn, uint32_t nIn) { hash = hashIn; n = nIn; } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(FLATDATA(*this)); + } + + void SetNull() { hash = 0; n = (uint32_t) -1; } + bool IsNull() const { return (hash == 0 && n == (uint32_t) -1); } + + friend bool operator<(const COutPoint& a, const COutPoint& b) + { + return (a.hash < b.hash || (a.hash == b.hash && a.n < b.n)); + } + + friend bool operator==(const COutPoint& a, const COutPoint& b) + { + return (a.hash == b.hash && a.n == b.n); + } + + friend bool operator!=(const COutPoint& a, const COutPoint& b) + { + return !(a == b); + } + + std::string ToString() const; +}; + +/** An input of a transaction. It contains the location of the previous + * transaction's output that it claims and a signature that matches the + * output's public key. + */ +class CTxIn +{ +public: + COutPoint prevout; + CScript scriptSig; + uint32_t nSequence; + + CTxIn() + { + nSequence = std::numeric_limits::max(); + } + + explicit CTxIn(COutPoint prevoutIn, CScript scriptSigIn=CScript(), uint32_t nSequenceIn=std::numeric_limits::max()); + CTxIn(uint256 hashPrevTx, uint32_t nOut, CScript scriptSigIn=CScript(), uint32_t nSequenceIn=std::numeric_limits::max()); + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(prevout); + READWRITE(scriptSig); + READWRITE(nSequence); + } + + bool IsFinal() const + { + return (nSequence == std::numeric_limits::max()); + } + + friend bool operator==(const CTxIn& a, const CTxIn& b) + { + return (a.prevout == b.prevout && + a.scriptSig == b.scriptSig && + a.nSequence == b.nSequence); + } + + friend bool operator!=(const CTxIn& a, const CTxIn& b) + { + return !(a == b); + } + + std::string ToString() const; +}; + +/** An output of a transaction. It contains the public key that the next input + * must be able to sign with to claim it. + */ +class CTxOut +{ +public: + CAmount nValue; + CScript scriptPubKey; + + CTxOut() + { + SetNull(); + } + + CTxOut(const CAmount& nValueIn, CScript scriptPubKeyIn); + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(nValue); + READWRITE(scriptPubKey); + } + + void SetNull() + { + nValue = -1; + scriptPubKey.clear(); + } + + bool IsNull() const + { + return (nValue == -1); + } + + uint256 GetHash() const; + +/* MCHN START */ + CAmount GetDustThreshold(const CFeeRate &minRelayTxFee) const + { + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + int64_t minOutput=mc_gState->m_NetworkParams->GetInt64Param("minimumperoutput"); + if(minOutput >= 0) + { + return (nValue < minOutput); + } + } + size_t nSize = GetSerializeSize(SER_DISK,0)+148u; + return 3*minRelayTxFee.GetFee(nSize); + } +/* MCHN END */ + + bool IsDust(CFeeRate minRelayTxFee) const + { + // "Dust" is defined in terms of CTransaction::minRelayTxFee, + // which has units satoshis-per-kilobyte. + // If you'd pay more than 1/3 in fees + // to spend something, then we consider it dust. + // A typical txout is 34 bytes big, and will + // need a CTxIn of at least 148 bytes to spend: + // so dust is a txout less than 546 satoshis + // with default minRelayTxFee. +/* MCHN START */ +/* + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + int64_t minOutput=mc_gState->m_NetworkParams->GetInt64Param("minimumperoutput"); + if(minOutput >= 0) + { + return (nValue < minOutput); + } + } + size_t nSize = GetSerializeSize(SER_DISK,0)+148u; + return (nValue < 3*minRelayTxFee.GetFee(nSize)); +*/ + return (nValue < GetDustThreshold(minRelayTxFee)); +/* MCHN END */ + } + + friend bool operator==(const CTxOut& a, const CTxOut& b) + { + return (a.nValue == b.nValue && + a.scriptPubKey == b.scriptPubKey); + } + + friend bool operator!=(const CTxOut& a, const CTxOut& b) + { + return !(a == b); + } + + std::string ToString() const; +}; + +struct CMutableTransaction; + +/** The basic transaction that is broadcasted on the network and contained in + * blocks. A transaction can contain multiple inputs and outputs. + */ +class CTransaction +{ +private: + /** Memory only. */ + const uint256 hash; + void UpdateHash() const; + +public: + static const int32_t CURRENT_VERSION=1; + + // The local variables are made const to prevent unintended modification + // without updating the cached hash value. However, CTransaction is not + // actually immutable; deserialization and assignment are implemented, + // and bypass the constness. This is safe, as they update the entire + // structure, including the hash. + const int32_t nVersion; + const std::vector vin; + const std::vector vout; + const uint32_t nLockTime; + + /** Construct a CTransaction that qualifies as IsNull() */ + CTransaction(); + + /** Convert a CMutableTransaction into a CTransaction. */ + CTransaction(const CMutableTransaction &tx); + + CTransaction& operator=(const CTransaction& tx); + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(*const_cast(&this->nVersion)); + nVersion = this->nVersion; + READWRITE(*const_cast*>(&vin)); + READWRITE(*const_cast*>(&vout)); + READWRITE(*const_cast(&nLockTime)); + if (ser_action.ForRead()) + UpdateHash(); + } + + bool IsNull() const { + return vin.empty() && vout.empty(); + } + + const uint256& GetHash() const { + return hash; + } + + // Return sum of txouts. + CAmount GetValueOut() const; + // GetValueIn() is a method on CCoinsViewCache, because + // inputs must be known to compute value in. + + // Compute priority, given priority of inputs and (optionally) tx size + double ComputePriority(double dPriorityInputs, unsigned int nTxSize=0) const; + + // Compute modified tx size for priority calculation (optionally given tx size) + unsigned int CalculateModifiedSize(unsigned int nTxSize=0) const; + + bool IsCoinBase() const + { + return (vin.size() == 1 && vin[0].prevout.IsNull()); + } + + friend bool operator==(const CTransaction& a, const CTransaction& b) + { + return a.hash == b.hash; + } + + friend bool operator!=(const CTransaction& a, const CTransaction& b) + { + return a.hash != b.hash; + } + + std::string ToString() const; +}; + +/** A mutable version of CTransaction. */ +struct CMutableTransaction +{ + int32_t nVersion; + std::vector vin; + std::vector vout; + uint32_t nLockTime; + + CMutableTransaction(); + CMutableTransaction(const CTransaction& tx); + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(this->nVersion); + nVersion = this->nVersion; + READWRITE(vin); + READWRITE(vout); + READWRITE(nLockTime); + } + + /** Compute the hash of this CMutableTransaction. This is computed on the + * fly, as opposed to GetHash() in CTransaction, which uses a cached result. + */ + uint256 GetHash() const; +}; + +#endif // BITCOIN_PRIMITIVES_TRANSACTION_H diff --git a/src/protocol/handshake.cpp b/src/protocol/handshake.cpp new file mode 100644 index 00000000..9761cf0a --- /dev/null +++ b/src/protocol/handshake.cpp @@ -0,0 +1,529 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2016 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "core/init.h" +#include "core/main.h" +#include "utils/util.h" +#include "wallet/wallet.h" +#include "multichain/multichain.h" +#include "keys/pubkey.h" +#include "keys/key.h" +#include "structs/base58.h" +#include "net/net.h" + +using namespace std; + +bool MultichainNode_CanConnect(CNode *pnode) +{ + bool ret=true; + + if(!mc_gState->m_Permissions->CanConnect(NULL,pnode->kAddrRemote.begin())) + { + if(pnode->fInbound) + { + ret=false; + } + if(mc_gState->m_pSeedNode != (void*)pnode) + { + ret=false; + } + } + + pnode->fCanConnectRemote=ret; + + return ret; +} + +bool MultichainNode_DisconnectRemote(CNode *pnode) +{ + if(pnode->fDisconnect) + { + return true; + } + + if((mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_VALID) && pnode->fSuccessfullyConnected && pnode->fParameterSetVerified) + { + return !MultichainNode_CanConnect(pnode); + } + + return false; +} + +bool MultichainNode_DisconnectLocal(CNode *pnode) +{ + if(pnode->fDisconnect) + { + return true; + } + + bool ret=true; + + if((mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_VALID) && pnode->fSuccessfullyConnected && pnode->fParameterSetVerified) + { + if(!mc_gState->m_Permissions->CanConnect(NULL,pnode->kAddrLocal.begin())) + { + if(pnode->fInbound) + { + ret=false; + } + if(mc_gState->m_pSeedNode != (void*)pnode) + { + ret=false; + } + } + } + + pnode->fCanConnectLocal=ret; + + return !ret; +} + + +bool MultichainNode_RespondToGetData(CNode *pnode) +{ + return !(pnode->fDisconnect) & mc_gState->m_Permissions->CanConnect(NULL,pnode->kAddrRemote.begin()); +} + +bool MultichainNode_SendInv(CNode *pnode) +{ + return !(pnode->fDisconnect) & mc_gState->m_Permissions->CanConnect(NULL,pnode->kAddrRemote.begin()); +} + +bool MultichainNode_AcceptData(CNode *pnode) +{ + if(mc_gState->m_pSeedNode == (void*)pnode) + { + return true; + } + + return !MultichainNode_DisconnectRemote(pnode); +} + +bool MultichainNode_IgnoreIncoming(CNode *pnode) +{ + if(mc_gState->m_NodePausedState & MC_NPS_INCOMING) + { + return true; + } + return false; +} + +bool MultichainNode_IsLocal(CNode *pnode) +{ + return (IsLocal(pnode->addr) || pnode->addr.IsRFC1918()) && (pnode->addr.GetPort() == GetListenPort()); +} + + +bool VerifyMultichainVerackHash(string sParameterSetHash,uint64_t nNonce) +{ + unsigned char* stored_hash=(unsigned char*)mc_gState->m_NetworkParams->GetParam("chainparamshash",NULL); + CHashWriter ss(SER_GETHASH, 0); + ss << vector(stored_hash, stored_hash+32); + ss << nNonce; + uint256 expected_hash=ss.GetHash(); + if(memcmp(&expected_hash,sParameterSetHash.c_str(),32)) + { + return false; + } + return true; +} + +bool ProcessMultichainVerack(CNode* pfrom, CDataStream& vRecv,bool fIsVerackack,bool *disconnect_flag) +{ + string sParameterSet=""; + string sParameterSetHash=""; + string sSigScript=""; + bool only_check_signature; + + uint64_t nNonce; + + only_check_signature=false; + if(!fIsVerackack) + { + if(pfrom->nVersionNonceSent == 1) + { + LogPrintf("mchn: We don't expect verack from peer=%d\n", pfrom->id); + return false; + } + vRecv >> pfrom->nVerackNonceReceived >> sParameterSet; + nNonce=pfrom->nVersionNonceSent; + } + else + { + if(pfrom->nVerackNonceSent == 1) + { + LogPrintf("mchn: We don't expect verackack from peer=%d\n", pfrom->id); + return false; + } + nNonce=pfrom->nVerackNonceSent; + pfrom->fVerackackReceived=true; + } + + vRecv >> sParameterSetHash >> sSigScript; + + if(sParameterSetHash.size() != 32) + { + LogPrintf("mchn: Wrong parameter set hash size (%d) from peer=%d\n", sParameterSetHash.size(), pfrom->id); + return false; + } + + if(mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_VALID) + { + if(!VerifyMultichainVerackHash(sParameterSetHash,nNonce)) + { + if(!fIsVerackack) + { + LogPrintf("mchn: Parameter set hash mismatch from peer=%d\n", pfrom->id); + return false; + } + } + else + { + if(fIsVerackack) + { + only_check_signature=true; +// return false; + } + } + } + + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanconnect") == 0) + { + CScript scriptSig((unsigned char*)sSigScript.c_str(),(unsigned char*)sSigScript.c_str()+sSigScript.size()); + + vector vchSigOut; + vector vchPubKey; + + opcodetype opcode; + + CScript::const_iterator pc = scriptSig.begin(); + + if (!scriptSig.GetOp(pc, opcode, vchSigOut)) + { + LogPrintf("mchn: Cannot extract signature from sigScript from peer=%d\n", pfrom->id); + return false; + } + + vchSigOut.resize(vchSigOut.size()-1); + if (!scriptSig.GetOp(pc, opcode, vchPubKey)) + { + LogPrintf("mchn: Cannot extract pubkey from sigScript from peer=%d\n", pfrom->id); + return false; + } + + CPubKey pubKeyOut(vchPubKey); + if (!pubKeyOut.IsValid()) + { + LogPrintf("mchn: Invalid pubkey received from peer=%d\n", pfrom->id); + return false; + } + + + CKeyID pubKeyHash=pubKeyOut.GetID(); + pfrom->kAddrRemote=pubKeyHash; + if(fIsVerackack) + { + LogPrintf("mchn: Connection from %s received on peer=%d in verackack (%s)\n",CBitcoinAddress(pubKeyHash).ToString().c_str(), pfrom->id,pfrom->addr.ToString()); + if(!MultichainNode_CanConnect(pfrom)) + { + LogPrintf("mchn: Permission denied for address %s received from peer=%d\n",CBitcoinAddress(pubKeyHash).ToString().c_str(), pfrom->id); + pfrom->fVerackackReceived=false; // Will resend minimal parameter set + if(only_check_signature) + { + return false; + } +// return false; + } +/* + { + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pnode, vNodes) + { + if(pnode->id != pfrom->id) + { + if(pnode->kAddrRemote == pubKeyHash) + { + LogPrintf("mchn: Already connected to address %s received from peer=%d\n",CBitcoinAddress(pubKeyHash).ToString().c_str(), pfrom->id); + pfrom->fVerackackReceived=false; // Will resend minimal parameter set + if(only_check_signature) + { + return false; + } + } + } + } + } + */ + } + else + { + LogPrintf("mchn: Connection from %s received on peer=%d in verack\n",CBitcoinAddress(pubKeyHash).ToString().c_str(), pfrom->id); + } + + CHashWriter ss(SER_GETHASH, 0); + ss << vector((unsigned char*)sParameterSetHash.c_str(), (unsigned char*)sParameterSetHash.c_str()+32); + ss << vector((unsigned char*)&nNonce, (unsigned char*)&nNonce+sizeof(nNonce)); + uint256 signed_hash=ss.GetHash(); + + if(!pubKeyOut.Verify(signed_hash,vchSigOut)) + { + LogPrintf("mchn: Wrong signature received from peer=%d\n", pfrom->id); + return false; + } + if(only_check_signature) + { + *disconnect_flag=false; + return false; + } + } + else + { + if(only_check_signature) + { + *disconnect_flag=false; + return false; + } + } + + int current_status=mc_gState->m_NetworkParams->m_Status; + + if((current_status == MC_PRM_STATUS_EMPTY) || + ((current_status == MC_PRM_STATUS_MINIMAL) && pfrom->fVerackackSent)) + { + pfrom->fDisconnect=true; + + if(mc_gState->m_NetworkParams->Set(mc_gState->m_Params->NetworkName(),sParameterSet.c_str(),sParameterSet.size()) == MC_ERR_NOERROR) + { + mc_gState->m_NetworkParams->Validate(); + + switch(current_status) + { + case MC_PRM_STATUS_EMPTY: + if((mc_gState->m_NetworkParams->m_Status != MC_PRM_STATUS_MINIMAL) && + (mc_gState->m_NetworkParams->m_Status != MC_PRM_STATUS_VALID)) + { + LogPrintf("mchn: Invalid parameter set received from %s\n", pfrom->addr.ToString()); + return false; + } + break; + case MC_PRM_STATUS_MINIMAL: + if((mc_gState->m_NetworkParams->m_Status != MC_PRM_STATUS_MINIMAL) && + (mc_gState->m_NetworkParams->m_Status != MC_PRM_STATUS_VALID)) + { + LogPrintf("mchn: Invalid parameter set received from %s\n", pfrom->addr.ToString()); + return false; + } + break; + default: + LogPrintf("mchn: Invalid current parameter set status\n", pfrom->addr.ToString()); + return false; + } + if(strcmp(mc_gState->m_NetworkParams->Name(),mc_gState->m_Params->NetworkName())) + { + LogPrintf("mchn: Parameter set received from %s has different name\n", pfrom->addr.ToString()); + mc_gState->m_NetworkParams->Read(mc_gState->m_Params->NetworkName()); + mc_gState->m_NetworkParams->Validate(); + return false; + } + + if(mc_gState->m_NetworkParams->m_Status == MC_PRM_STATUS_VALID) + { + if(!VerifyMultichainVerackHash(sParameterSetHash,nNonce)) + { + LogPrintf("mchn: Parameter set received from peer %d doesn't match received hash\n", pfrom->id); + return false; + } + } + + if(mc_gState->m_NetworkParams->Write(1)) + { + LogPrintf("mchn: Cannot store parameter set received from %s\n", pfrom->addr.ToString()); + mc_gState->m_NetworkParams->m_Status=MC_PRM_STATUS_ERROR; + return false; + } + else + { + LogPrintf("mchn: Successfully stored parameter set received from %s\n", pfrom->addr.ToString()); + } + } + else + { + LogPrintf("mchn: Cannot parse parameter set received from %s\n", pfrom->addr.ToString()); + return false; + } + + } + + return true; +} + +bool PushMultiChainVerack(CNode* pfrom, bool fIsVerackack) +{ + vectorvParameterSet; + vectorvParameterSetHash; + vectorvSigScript; + uint64_t nNonce; + + if(!fIsVerackack) + { + if(mc_gState->m_NetworkParams->m_Status != MC_PRM_STATUS_VALID) // Ignoring version message from seed node in initial handshake + { + return false; + } + } + + if(!fIsVerackack) + { + if((pfrom->fDefaultMessageStart && (mc_gState->m_NetworkParams->GetInt64Param("anyonecanconnect")!=0) ) || + pfrom->fVerackackReceived) + { + vParameterSet=vector(mc_gState->m_NetworkParams->m_lpData,mc_gState->m_NetworkParams->m_lpData+mc_gState->m_NetworkParams->m_Size); + LogPrintf("mchn: Sending full parameter set to %s\n", pfrom->addr.ToString()); + } + else + { + string fields_to_send [] ={"protocolversion","chainname","defaultrpcport","networkmessagestart","addresspubkeyhashversion","addresschecksumvalue"}; + int NumFieldsToSend=1; + unsigned char *ptr; + int size; +// if(pfrom->fDefaultMessageStart) + { + NumFieldsToSend=6; + } + LogPrintf("mchn: Sending minimal parameter set to %s\n", pfrom->addr.ToString()); + for(int f=0;fm_NetworkParams->GetParam(fields_to_send[f].c_str(),&size); + if(ptr == NULL) + { + LogPrintf("mchn: Internal error: Invalid parameter set\n"); + return false; + } + vParameterSet.insert(vParameterSet.end(),ptr-MC_PRM_PARAM_SIZE_BYTES,ptr+size); + } + } + nNonce=pfrom->nVersionNonceReceived; + } + else + { + if(pfrom->fVerackackSent) + { + return true; + } + nNonce=pfrom->nVerackNonceReceived; + pfrom->fVerackackSent=true; + } + + unsigned char* stored_hash=(unsigned char*)mc_gState->m_NetworkParams->GetParam("chainparamshash",NULL); + CHashWriter ssHash(SER_GETHASH, 0); + if(stored_hash) + { + ssHash << vector(stored_hash, stored_hash+32); + } + ssHash << nNonce; + uint256 hash_to_send=ssHash.GetHash(); + + vParameterSetHash=vector((unsigned char*)&hash_to_send, (unsigned char*)&hash_to_send+32); + + CHashWriter ssSig(SER_GETHASH, 0); + ssSig << vParameterSetHash; + ssSig << vector((unsigned char*)&nNonce, (unsigned char*)&nNonce+sizeof(nNonce)); + uint256 signed_hash=ssSig.GetHash(); + + CKey key; + CKeyID keyID; + CPubKey pkey; + bool key_found=false; + + + if(mapArgs.count("-handshakelocal")) + { + CBitcoinAddress address(mapArgs["-handshakelocal"]); + LogPrintf("mchn: Using handshake address %d\n",mapArgs["-handshakelocal"].c_str()); + if (address.IsValid()) + { + CTxDestination dst=address.Get(); + CKeyID *lpKeyID=boost::get (&dst); + if(lpKeyID) + { + if(pwalletMain->GetKey(*lpKeyID, key)) + { + keyID=*lpKeyID; + key_found=true; + } + else + { + LogPrintf("mchn: handshakelocal address %s doesn't belong to this wallet, using default address for connection\n",mapArgs["-handshakelocal"].c_str()); + printf("\nWarning: handshakelocal address %s doesn't belong to this wallet, using default address for connection\n\n",mapArgs["-handshakelocal"].c_str()); + } + } + else + { + LogPrintf("mchn: handshakelocal address %s is invalid, using default address for connection\n",mapArgs["-handshakelocal"].c_str()); + printf("\nWarning: handshakelocal address %s is invalid, using default address for connection\n\n",mapArgs["-handshakelocal"].c_str()); + } + } + else + { + LogPrintf("mchn: handshakelocal address %s is invalid, using default address for connection\n",mapArgs["-handshakelocal"].c_str()); + printf("\nWarning: handshakelocal address %s is invalid, using default address for connection\n\n",mapArgs["-handshakelocal"].c_str()); + } + } + + if(!key_found) + { + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_CONNECT)) + { + LogPrintf("mchn: Cannot find address having connect permission, trying default key\n"); + pkey=pwalletMain->vchDefaultKey; + } + keyID=pkey.GetID(); + } + + + pfrom->kAddrLocal=keyID; + + if(!pwalletMain->GetKey(keyID, key)) + { + LogPrintf("mchn: Internal error: Connection address not found in the wallet\n"); + return false; + } + + if(key_found) + { + pkey=key.GetPubKey(); + } + + CScript scriptSig; + vector vchSig; + if (!key.Sign(signed_hash, vchSig)) + { + LogPrintf("mchn: Internal error: Cannot sign parameter set hash\n"); + return false; + } + + vchSig.push_back(0x00); + scriptSig << vchSig; + scriptSig << ToByteVector(pkey); + + vSigScript=scriptSig; + + if(!fIsVerackack) + { + GetRandBytes((unsigned char*)&(pfrom->nVerackNonceSent), sizeof(pfrom->nVerackNonceSent)); + pfrom->PushMessage("verack", pfrom->nVerackNonceSent, vParameterSet, vParameterSetHash, vSigScript); + } + else + { + pfrom->PushMessage("verackack", vParameterSetHash, vSigScript); + } + + return true; + +} + + diff --git a/src/protocol/multichainblock.cpp b/src/protocol/multichainblock.cpp new file mode 100644 index 00000000..aadd1ba2 --- /dev/null +++ b/src/protocol/multichainblock.cpp @@ -0,0 +1,335 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2016 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "core/main.h" +#include "utils/util.h" +#include "multichain/multichain.h" +#include "wallet/wallettxs.h" + +extern mc_WalletTxs* pwalletTxsMain; + + +using namespace std; + +bool AcceptMultiChainTransaction(const CTransaction& tx, + const CCoinsViewCache &inputs, + int offset, + bool accept, + string& reason); +bool AcceptAssetTransfers(const CTransaction& tx, const CCoinsViewCache &inputs, string& reason); +bool AcceptAssetGenesis(const CTransaction &tx,int offset,bool accept,string& reason); +bool AcceptPermissionsAndCheckForDust(const CTransaction &tx,bool accept,string& reason); + + +bool ReplayMemPool(CTxMemPool& pool, int from,bool accept) +{ + int pos; + uint256 hash; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return true; + } + + int total_txs=pool.hashList->m_Count; + + LogPrint("mchn", "mchn: Replaying memory pool (%d new transactions, total %d)\n",total_txs-from,total_txs); + + for(pos=from;posm_Count;pos++) + { + hash=*(uint256*)pool.hashList->GetRow(pos); + if(pool.exists(hash)) + { + const CTransaction& tx = pool.mapTx[hash].GetTx(); + string reason; + string removed_type=""; + list removed; + + if(mc_gState->m_Features->Streams()) + { + LOCK(pool.cs); + CCoinsView dummy; + CCoinsViewCache view(&dummy); + CCoinsViewMemPool viewMemPool(pcoinsTip, pool); + view.SetBackend(viewMemPool); + if(!AcceptMultiChainTransaction(tx,view,-1,accept,reason)) + { + removed_type="rejected"; + } + } + else + { + if(removed_type.size() == 0) + { + if(!AcceptPermissionsAndCheckForDust(tx,accept,reason)) + { + removed_type="permissions"; + } + } + if(removed_type.size() == 0) + { + if(!AcceptAssetGenesis(tx,-1,true,reason)) + { + removed_type="issue"; + } + } + if(removed_type.size() == 0) + { + LOCK(pool.cs); + CCoinsView dummy; + CCoinsViewCache view(&dummy); + CCoinsViewMemPool viewMemPool(pcoinsTip, pool); + view.SetBackend(viewMemPool); + if(!AcceptAssetTransfers(tx, view, reason)) + { + removed_type="transfer"; + } + } + } + + if(removed_type.size()) + { + LogPrintf("mchn: Tx %s removed from the mempool (%s), reason: %s\n",tx.GetHash().ToString().c_str(),removed_type.c_str(),reason.c_str()); + pool.remove(tx, removed, true, "replay"); + } + else + { + pwalletTxsMain->AddTx(NULL,tx,-1,NULL,-1); + } + } + } + + return true; +} + + + +bool VerifyBlockSignature(CBlock *block,bool force) +{ + unsigned char sig[255]; + int sig_size,key_size; + uint32_t hash_type; + uint256 hash_to_verify; + uint256 original_merkle_root; + uint32_t original_nonce; + std::vector vchSigOut; + std::vector vchPubKey; + + if(!force) + { + if(block->nMerkleTreeType != MERKLETREE_UNKNOWN) + { + if(block->nSigHashType == BLOCKSIGHASH_INVALID) + { + return false; + } + return true; + } + } + + block->nMerkleTreeType=MERKLETREE_FULL; + block->nSigHashType=BLOCKSIGHASH_NONE; + block->vSigner[0]=0; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + for (unsigned int i = 0; i < block->vtx.size(); i++) + { + const CTransaction &tx = block->vtx[i]; + if (tx.IsCoinBase()) + { + for (unsigned int j = 0; j < tx.vout.size(); j++) + { + mc_gState->m_TmpScript->Clear(); + + const CScript& script1 = tx.vout[j].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + if(block->vSigner[0] == 0) + { + mc_gState->m_TmpScript->SetElement(e); + sig_size=255; + key_size=255; + if(mc_gState->m_TmpScript->GetBlockSignature(sig,&sig_size,&hash_type,block->vSigner+1,&key_size) == 0) + { + block->vSigner[0]=(unsigned char)key_size; + } + } + } + } + } + } + } + + if(block->vSigner[0]) + { + switch(hash_type) + { + case BLOCKSIGHASH_HEADER: + block->nMerkleTreeType=MERKLETREE_NO_COINBASE_OP_RETURN; + block->nSigHashType=BLOCKSIGHASH_HEADER; + hash_to_verify=block->GetHash(); + break; + case BLOCKSIGHASH_NO_SIGNATURE_AND_NONCE: + + original_merkle_root=block->hashMerkleRoot; + original_nonce=block->nNonce; + + block->nMerkleTreeType=MERKLETREE_NO_COINBASE_OP_RETURN; + block->hashMerkleRoot=block->BuildMerkleTree(); + block->nNonce=0; + hash_to_verify=block->GetHash(); + + block->hashMerkleRoot=original_merkle_root; + block->nNonce=original_nonce; + + block->nMerkleTreeType=MERKLETREE_FULL; + block->BuildMerkleTree(); + break; + default: + LogPrintf("mchn: Invalid hash type received in block signature\n"); + block->nSigHashType=BLOCKSIGHASH_INVALID; + return false; + } + +// printf("V: %s -> %s\n",hash_to_verify.GetHex().c_str(),block->GetHash().GetHex().c_str()); + + vchSigOut=std::vector (sig, sig+sig_size); + vchPubKey=std::vector (block->vSigner+1, block->vSigner+1+block->vSigner[0]); + + CPubKey pubKeyOut(vchPubKey); + if (!pubKeyOut.IsValid()) + { + LogPrintf("mchn: Invalid pubkey received in block signature\n"); + block->nSigHashType=BLOCKSIGHASH_INVALID; + return false; + } + if(!pubKeyOut.Verify(hash_to_verify,vchSigOut)) + { + LogPrintf("mchn: Wrong block signature\n"); + block->nSigHashType=BLOCKSIGHASH_INVALID; + return false; + } + } + else + { + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(block->hashPrevBlock != uint256(0)) + { + LogPrintf("mchn: Block signature not found\n"); + block->nSigHashType=BLOCKSIGHASH_INVALID; + return false; + } + } + } + + return true; +} + +/* MCHN END */ + + + +bool CheckBlockPermissions(const CBlock& block,CBlockIndex* prev_block,unsigned char *lpMinerAddress) +{ + bool checked=true; + const unsigned char *genesis_key; + int key_size; + vector vchSigOut; + vector vchPubKey; + + LogPrint("mchn","mchn: Checking block signature and miner permissions...\n"); + + key_size=255; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return true; + } + + mc_gState->m_Permissions->CopyMemPool(); + mc_gState->m_Permissions->ClearMemPool(); + + if(mc_gState->m_Features->UnconfirmedMinersCannotMine() == 0) + { + + for (unsigned int i = 0; i < block.vtx.size(); i++) + { + if(checked) + { + const CTransaction &tx = block.vtx[i]; + if (!tx.IsCoinBase()) + { + string reason; + if(!AcceptPermissionsAndCheckForDust(tx,true,reason)) + { + checked=false; + } + } + } + } + } + + if(checked) + { + if(prev_block) + { + if((block.nSigHashType == BLOCKSIGHASH_UNKNOWN) || (block.nSigHashType == BLOCKSIGHASH_INVALID)) + { + checked=false; + } + else + { + vchPubKey=vector (block.vSigner+1, block.vSigner+1+block.vSigner[0]); + + CPubKey pubKeyOut(vchPubKey); + if (!pubKeyOut.IsValid()) + { + LogPrintf("mchn: Invalid pubkey received in block signature\n"); + checked = false; + } + if(checked) + { + CKeyID pubKeyHash=pubKeyOut.GetID(); + memcpy(lpMinerAddress,pubKeyHash.begin(),20); + if(!mc_gState->m_Permissions->CanMine(NULL,pubKeyHash.begin())) + { + // mc_DumpSize("Connection address",pubKeyHash.begin(),20,20); + LogPrintf("mchn: Permission denied for miner %s received in block signature\n",CBitcoinAddress(pubKeyHash).ToString().c_str()); + checked = false; + } + } + } + } + else + { + genesis_key=(unsigned char*)mc_gState->m_NetworkParams->GetParam("genesispubkey",&key_size); + vchPubKey=vector (genesis_key, genesis_key+key_size); + + CPubKey pubKeyOut(vchPubKey); + if (!pubKeyOut.IsValid()) + { + LogPrintf("mchn: Invalid pubkey received in block signature\n"); + checked = false; + } + if(checked) + { + CKeyID pubKeyHash=pubKeyOut.GetID(); + memcpy(lpMinerAddress,pubKeyHash.begin(),20); + } + } + } + + mc_gState->m_Permissions->RestoreMemPool(); + + return checked; +} + diff --git a/src/protocol/multichainscript.cpp b/src/protocol/multichainscript.cpp new file mode 100644 index 00000000..ae2c08e2 --- /dev/null +++ b/src/protocol/multichainscript.cpp @@ -0,0 +1,2069 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" + +#define MC_DCT_SCRIPT_ALLOC_BUFFER_CHUNK 4096 +#define MC_DCT_SCRIPT_ALLOC_INDEX_CHUNK 16 + +#define MC_DCT_SCRIPT_OP_PUSHDATA1 0x4c +#define MC_DCT_SCRIPT_OP_PUSHDATA2 0x4d +#define MC_DCT_SCRIPT_OP_PUSHDATA4 0x4e +#define MC_DCT_SCRIPT_OP_16 0x60 +#define MC_DCT_SCRIPT_OP_RETURN 0x6a +#define MC_DCT_SCRIPT_OP_DROP 0x75 + +#define MC_DCT_SCRIPT_COINSPARK_IDENTIFIER "SPK" +#define MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER "spk" +#define MC_DCT_SCRIPT_IDENTIFIER_LEN 3 + +#define MC_DCT_SCRIPT_MULTICHAIN_ENTITY_PREFIX 'e' +#define MC_DCT_SCRIPT_MULTICHAIN_CASHED_SCRIPT_PREFIX 'i' +#define MC_DCT_SCRIPT_MULTICHAIN_KEY_PREFIX 'k' +#define MC_DCT_SCRIPT_MULTICHAIN_NEW_ENTITY_PREFIX 'n' +#define MC_DCT_SCRIPT_MULTICHAIN_UPDATE_ENTITY_PREFIX 'u' +#define MC_DCT_SCRIPT_MULTICHAIN_PERMISSIONS_PREFIX 'p' +#define MC_DCT_SCRIPT_MULTICHAIN_BLOCK_SIGNATURE_PREFIX 'b' +#define MC_DCT_SCRIPT_MULTICHAIN_ASSET_GENESIS_PREFIX 'g' +#define MC_DCT_SCRIPT_MULTICHAIN_ASSET_QUANTITY_PREFIX 'q' +#define MC_DCT_SCRIPT_MULTICHAIN_ASSET_DETAILS_PREFIX 'a' +#define MC_DCT_SCRIPT_MULTICHAIN_ASSET_FOLLOWON_PREFIX 'o' +#define MC_DCT_SCRIPT_MULTICHAIN_GENERAL_DETAILS_PREFIX 'c' + +#define MC_DCT_SCRIPT_TYPE_REGULAR 0x00 +#define MC_DCT_SCRIPT_TYPE_OP_RETURN 0x01 +#define MC_DCT_SCRIPT_TYPE_OP_DROP 0x02 +#define MC_DCT_SCRIPT_TYPE_DIRTY_OP_RETURN 0x04 + + + +int mc_Script::Zero() +{ + m_Size=0; + m_NumElements=0; + m_CurrentElement=-1; + m_lpData=NULL; + m_lpCoord=NULL; + m_AllocElements=0; + m_AllocSize=0; + m_ScriptType=MC_DCT_SCRIPT_TYPE_REGULAR; + + return MC_ERR_NOERROR; +} + +int mc_Script::Destroy() +{ + if(m_lpData) + { + mc_Delete(m_lpData); + } + if(m_lpCoord) + { + mc_Delete(m_lpCoord); + } + + return Zero(); +} + +int mc_Script::Resize(size_t bytes,int elements) +{ + int NewSize; + unsigned char *lpNewBuffer; + int *lpNewCoord; + + NewSize=m_AllocSize; + while(m_Size+(int)bytes > NewSize) + { + NewSize+=MC_DCT_SCRIPT_ALLOC_BUFFER_CHUNK; + } + + if(NewSize > m_AllocSize) + { + lpNewBuffer=(unsigned char *)mc_New(NewSize); + if(lpNewBuffer == NULL) + { + return MC_ERR_ALLOCATION; + } + if(m_lpData) + { + if(m_Size) + { + memcpy(lpNewBuffer,m_lpData,m_Size); + } + mc_Delete(m_lpData); + } + m_lpData=lpNewBuffer; + m_AllocSize=NewSize; + } + + NewSize=m_AllocElements; + while(m_NumElements+elements > NewSize) + { + NewSize+=MC_DCT_SCRIPT_ALLOC_INDEX_CHUNK; + } + + if(NewSize > m_AllocElements) + { + lpNewCoord=(int *)mc_New(NewSize*2*sizeof(int)); + if(lpNewCoord == NULL) + { + return MC_ERR_ALLOCATION; + } + if(m_lpCoord) + { + if(m_NumElements) + { + memcpy(lpNewCoord,m_lpCoord,m_NumElements*2*sizeof(int)); + } + mc_Delete(m_lpCoord); + } + m_lpCoord=lpNewCoord; + m_AllocElements=NewSize; + } + + return MC_ERR_NOERROR; +} + +int mc_Script::Clear() +{ + m_Size=0; + m_NumElements=0; + m_CurrentElement=-1; + m_ScriptType=MC_DCT_SCRIPT_TYPE_REGULAR; + + return MC_ERR_NOERROR; +} + +int mc_Script::GetNumElements() +{ + return m_NumElements; +} + +int mc_Script::AddElement() +{ + int err; + + err=Resize(0,1); + if(err) + { + return err; + } + + m_NumElements++; + m_CurrentElement++; + m_lpCoord[2*m_CurrentElement + 0]=m_Size; + m_lpCoord[2*m_CurrentElement + 1]=0; + + return MC_ERR_NOERROR; +} + +int mc_Script::GetElement() +{ + return m_CurrentElement; +} + +int mc_Script::SetElement(int element) +{ + if(element >= m_NumElements) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + m_CurrentElement=element; + return MC_ERR_NOERROR; +} + +const unsigned char* mc_Script::GetData(int element, size_t* bytes) +{ + if(element >= m_NumElements) + { + return NULL; + } + + m_CurrentElement=element; + if(bytes) + { + *bytes=m_lpCoord[2*m_CurrentElement + 1]; + } + + return m_lpData+m_lpCoord[2*m_CurrentElement + 0]; +} + +int mc_Script::SetData(const unsigned char* src, const size_t bytes) +{ + int err; + + err=Resize(bytes,0); + if(err) + { + return err; + } + + if(bytes) + { + memcpy(m_lpData+m_lpCoord[2*m_CurrentElement + 0]+m_lpCoord[2*m_CurrentElement + 1],src,bytes); + m_lpCoord[2*m_CurrentElement + 1]+=bytes; + m_Size+=bytes; + } + + return MC_ERR_NOERROR; +} + +int mc_Script::SetSpecialParamValue(unsigned char param, const unsigned char* param_value, const size_t param_value_size) +{ + unsigned char buf[16]; + int size,err; + + err=MC_ERR_NOERROR; + + buf[0]=0x00; + buf[1]=param; + err=SetData(buf,2); + if(err) + { + return err; + } + + size=mc_PutVarInt(buf,16,param_value_size); + err=SetData(buf,size); + if(err) + { + return err; + } + + err=SetData(param_value,param_value_size); + if(err) + { + return err; + } + + return MC_ERR_NOERROR; +} + +int mc_Script::SetParamValue(const char *param_name,const size_t param_name_size,const unsigned char* param_value,const size_t param_value_size) +{ + unsigned char buf[16]; + int size,err; + + err=MC_ERR_NOERROR; + + err=SetData((unsigned char*)param_name,param_name_size); + if(err) + { + return err; + } + + buf[0]=0x00; + err=SetData(buf,1); + if(err) + { + return err; + } + + size=mc_PutVarInt(buf,16,param_value_size); + err=SetData(buf,size); + if(err) + { + return err; + } + + err=SetData(param_value,param_value_size); + if(err) + { + return err; + } + + return MC_ERR_NOERROR; +} + +uint32_t mc_GetParamFromDetailsScript(const unsigned char *ptr,uint32_t total,uint32_t offset,uint32_t* param_value_start,size_t *bytes,int *err) +{ + int shift,name_size,value_size,size; + + *param_value_start=0; + *err=MC_ERR_NOERROR; + + if(offset>=total) + { + if(offset > total) + { + *err=MC_ERR_ERROR_IN_SCRIPT; + } + return total; + } + + name_size=0; + if(mc_gState->m_Features->SpecialParamsInDetailsScript()) + { + if(ptr[offset] == 0x00) + { + name_size=2; + } + } + + if(name_size == 0) + { + name_size=strlen((char*)ptr+offset)+1; + } + + if(offset+name_size >= total) + { + *err=MC_ERR_ERROR_IN_SCRIPT; + return total; + } + + value_size=mc_GetVarInt(ptr+offset+name_size,total-offset-name_size,-1,&shift); + if(value_size<0) + { + *err=MC_ERR_ERROR_IN_SCRIPT; + return total; + } + + size=name_size+shift+value_size; + if(offset+size>total) + { + *err=MC_ERR_ERROR_IN_SCRIPT; + return total; + } + + *bytes=value_size; + *param_value_start=offset+name_size+shift; + + return offset+size; +} + +uint32_t mc_GetParamFromDetailsScript(const unsigned char *ptr,uint32_t total,uint32_t offset,uint32_t* param_value_start,size_t *bytes) +{ + int err; + return mc_GetParamFromDetailsScript(ptr,total,offset,param_value_start,bytes,&err); +} + +uint32_t mc_FindSpecialParamInDetailsScript(const unsigned char *ptr,uint32_t total,uint32_t param,size_t *bytes) +{ + uint32_t offset,new_offset; + uint32_t value_offset; + size_t value_size; + + if(mc_gState->m_Features->SpecialParamsInDetailsScript() == 0) + { + return total; + } + + offset=0; + while(offset 0) + { + if(ptr[offset] == 0x00) + { + if(ptr[offset+1] == (unsigned char)param) + { + *bytes=value_size; + return value_offset; + } + } + } + offset=new_offset; + } + + return total; +} + +uint32_t mc_FindNamedParamInDetailsScript(const unsigned char *ptr,uint32_t total,const char *param,size_t *bytes) +{ + uint32_t offset,new_offset; + uint32_t value_offset; + size_t value_size; + + offset=0; + while(offset 0) + { + if((ptr[offset] != 0x00) || (mc_gState->m_Features->SpecialParamsInDetailsScript() == 0) ) + { + if( strlen(param) == strlen((char*)ptr+offset) ) + { + if(mc_StringCompareCaseInsensitive(param,(char*)ptr+offset,strlen(param)) == 0) + { + *bytes=value_size; + return value_offset; + } + } + } + } + offset=new_offset; + } + + return total; +} + +int mc_VerifyDetailsScript(const unsigned char *script,uint32_t script_size) +{ + uint32_t offset=0; + uint32_t new_offset,param_value_start; + size_t bytes; + int err; + + while(offset=total) + { + return total; + } + + name_size=strlen((char*)ptr+offset)+1; + if(offset+name_size >= total) + { + return total; + } + + value_size=mc_GetVarInt(ptr+offset+name_size,total-offset-name_size,-1,&shift); + if(value_size<0) + { + return total; + } + + size=name_size+shift+value_size; + if(offset+size>total) + { + return total; + } + + *bytes=value_size; + *param_value_start=offset+name_size+shift; + + return offset+size; +} + + +int mc_Script::IsOpReturnScript() +{ + if(m_ScriptType & MC_DCT_SCRIPT_TYPE_OP_RETURN) + { + return 1; + } + + return 0; +} + +int mc_Script::IsDirtyOpReturnScript() +{ + if(m_ScriptType & MC_DCT_SCRIPT_TYPE_DIRTY_OP_RETURN) + { + return 1; + } + + return 0; +} + +int mc_GetPushDataElement(unsigned char *src,int size,int *op_drop_offset,int *op_drop_size) +{ + unsigned char opcode; + unsigned char *ptr; + unsigned char *ptrEnd; + int nSize; + + ptr=(unsigned char*)src; + ptrEnd=ptr+size; + + if(ptr >= ptrEnd) + { + return MC_ERR_WRONG_SCRIPT; + } + + nSize=0; + opcode=*ptr++; + + if (opcode <= MC_DCT_SCRIPT_OP_PUSHDATA4) + { + nSize = 0; + if (opcode < MC_DCT_SCRIPT_OP_PUSHDATA1) + { + nSize = (int)opcode; + } + else if (opcode == MC_DCT_SCRIPT_OP_PUSHDATA1) + { + if (ptrEnd - ptr < 1) + { + return MC_ERR_WRONG_SCRIPT; + } + nSize = mc_GetLE(ptr,1); + ptr++; + } + else if (opcode == MC_DCT_SCRIPT_OP_PUSHDATA2) + { + if (ptrEnd - ptr < 2) + { + return MC_ERR_WRONG_SCRIPT; + } + nSize = mc_GetLE(ptr,2); + ptr+=2; + } + else if (opcode == MC_DCT_SCRIPT_OP_PUSHDATA4) + { + if (ptrEnd - ptr < 4) + { + return MC_ERR_WRONG_SCRIPT; + } + nSize = mc_GetLE(ptr,4); + ptr+=4; + } + + if(ptrEnd MC_DCT_SCRIPT_OP_16) + { + return MC_ERR_WRONG_SCRIPT; + } + } + + *op_drop_offset=ptr-src; + *op_drop_size=nSize; + + return MC_ERR_NOERROR; +} + +const unsigned char *mc_ParseOpDropOpReturnScript(const unsigned char *src,int size,int *op_drop_offset,int *op_drop_size,int op_drop_count,int *op_return_offset,int *op_return_size) +{ + unsigned char *ptr; + unsigned char *ptrEnd; + int d; + int elem_offset, elem_size; + + ptr=(unsigned char*)src; + ptrEnd=ptr+size; + + d=0; + while( (d= ptrEnd) + { + return NULL; + } + + ptr++; + if(mc_GetPushDataElement(ptr,ptrEnd-ptr,op_return_offset,op_return_size) == MC_ERR_WRONG_SCRIPT) + { + *op_return_offset=ptr-src; + *op_return_size=0; + } + else + { + *op_return_offset+=ptr-src; + } + + return src; +} + +const unsigned char *mc_ExtractAddressFromInputScript(const unsigned char *src,int size,int *op_addr_offset,int *op_addr_size,int* is_redeem_script,int* sighash_type,int check_last) +{ + unsigned char *ptr; + unsigned char *ptrEnd; + int off,len; + + ptr=(unsigned char*)src; + ptrEnd=ptr+size; + + *is_redeem_script=0; + *sighash_type=-1; + while(ptr 0) ) + { + if(len) + { + if( (ptr[off+len-1] & 0x1f) != 1 ) + { + if( (ptr[off+len-1] & 0x1f) != 3 ) + { + *sighash_type=2; // SIGHASH_NONE + } + else + { + if(*sighash_type != 2) + { + *sighash_type=3; // SIGHASH_SINGLE + } + } + } + } + } + if(ptr == src) + { + if(len == 0) // Multisig, either bare or P2SH + { + *is_redeem_script=1; + } + else + { + if(ptr+off+len == ptrEnd) // Single element, pay-to-pubkey + { + return NULL; + } + } + } + ptr+=off+len; + } + else + { + *sighash_type=2; // SIGHASH_NONE + return NULL; // Not push-only script + } + } + + if(*sighash_type < 0) // Empty script + { + *sighash_type=2; // SIGHASH_NONE + return NULL; + } + + *op_addr_offset=ptr-src-len; + *op_addr_size=len; + + if(check_last) + { + return NULL; // pay-to-pubkey or bare multisig + } + + return src; +} + +int mc_Script::SetScript(const unsigned char* src,const size_t bytes,int type) +{ + unsigned char opcode; + unsigned char *ptr; + unsigned char *ptrEnd; + unsigned char *ptrPrev; + int nSize,lastSize; + int is_op_return,take_it,is_redeem_script; + + Clear(); + + if(bytes == 0) + { + return MC_ERR_NOERROR; + } + + is_op_return=0; + ptr=(unsigned char*)src; + ptrEnd=ptr+bytes; + ptrPrev=ptr; + lastSize=-1; + + while(ptr= 0) // Push data before OP_RETURN + { + m_ScriptType |= MC_DCT_SCRIPT_TYPE_DIRTY_OP_RETURN; + } + } + else + { + if(opcode == MC_DCT_SCRIPT_OP_DROP) + { + m_ScriptType |= MC_DCT_SCRIPT_TYPE_OP_DROP; + if(lastSize >= 0) // OP_DROP after push data + { + take_it=1; + } + else // Duplicate OP_DROP or OP_DROP without push data + { + m_ScriptType |= MC_DCT_SCRIPT_TYPE_DIRTY_OP_RETURN; + } + } + else + { + if(lastSize >= 0) // Duplicate push data + { + m_ScriptType |= MC_DCT_SCRIPT_TYPE_DIRTY_OP_RETURN; + } + } + } + if(take_it) + { + AddElement(); + SetData(ptrPrev,lastSize); + } + } + else + { + is_redeem_script=0; + if( (type == MC_SCR_TYPE_SCRIPTSIG) && (nSize>1) ) + { + if( (*ptr>0x50) && (*ptr<=0x60)) + { + if( m_NumElements > (*ptr-0x50) ) // OP_0 with at least required number of signatures + { + is_redeem_script=1; + } + } + } + + if(is_redeem_script) + { + nSize=0; + } + else + { + AddElement(); + if(nSize) + { + SetData(ptr,nSize); + } + else + { + if(is_op_return) // Empty OP_RETURN + { + SetData(ptr,0); + } + else + { + SetData(ptr-1,1); + } + } + if(is_op_return) // Everything after element after OP_RETURN is ignored + { + ptr=ptrEnd; + } + is_op_return=0; // Reset flag to avoid adding two empty elements + } + } + + ptrPrev=ptr; + lastSize=nSize; + if(is_op_return == 0) + { + if(lastSize == 0) + { + if(mc_gState->m_Features->FixedIn10007()) + { + if(opcode > MC_DCT_SCRIPT_OP_16) // Not push data + { + lastSize=-1; + } + } + else + { + lastSize=-1; + } + } + } + + if(lastSize<0) + { + if(opcode != MC_DCT_SCRIPT_OP_RETURN) + { + if(opcode != MC_DCT_SCRIPT_OP_DROP) + { + m_ScriptType |= MC_DCT_SCRIPT_TYPE_DIRTY_OP_RETURN; + } + } + } + + + ptr+=nSize; + } + + if(is_op_return) // Empty OP_RETURN + { + AddElement(); + SetData(ptr,0); + } + + if(m_ScriptType & MC_DCT_SCRIPT_TYPE_DIRTY_OP_RETURN) + { + if( (m_ScriptType & MC_DCT_SCRIPT_TYPE_OP_RETURN ) == 0) + { + m_ScriptType -= MC_DCT_SCRIPT_TYPE_DIRTY_OP_RETURN; + } + } + + return MC_ERR_NOERROR; +} + +int mc_Script::GetPermission(uint32_t *type,uint32_t *from,uint32_t *to,uint32_t *timestamp) +{ + unsigned char *ptr; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(m_lpCoord[m_CurrentElement*2+1] != MC_DCT_SCRIPT_IDENTIFIER_LEN+1+16) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + + if(memcmp(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] != MC_DCT_SCRIPT_MULTICHAIN_PERMISSIONS_PREFIX) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + *type= (uint32_t)mc_GetLE(ptr+ 0,4); + *from= (uint32_t)mc_GetLE(ptr+ 4,4); + *to =(uint32_t)mc_GetLE(ptr+ 8,4); + *timestamp=(uint32_t)mc_GetLE(ptr+12,4); + + return MC_ERR_NOERROR; +} + +int mc_Script::SetPermission(uint32_t type,uint32_t from,uint32_t to,uint32_t timestamp) +{ + unsigned char buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1+16]; + unsigned char *ptr; + int err; + + err=AddElement(); + if(err) + { + return err; + } + + ptr=buf; + memcpy(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN); + ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_PERMISSIONS_PREFIX; + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + mc_PutLE(ptr+ 0,&type,4); + mc_PutLE(ptr+ 4,&from,4); + mc_PutLE(ptr+ 8,&to,4); + mc_PutLE(ptr+12,×tamp,4); + + return SetData(buf,MC_DCT_SCRIPT_IDENTIFIER_LEN+1+16); +} + +int mc_Script::GetBlockSignature(unsigned char* sig,int *sig_size,uint32_t* hash_type,unsigned char* key,int *key_size) +{ + unsigned char *ptr; + int sig_len,key_len; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(m_lpCoord[m_CurrentElement*2+1] < MC_DCT_SCRIPT_IDENTIFIER_LEN+1+3) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + + if(memcmp(ptr,MC_DCT_SCRIPT_COINSPARK_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] != MC_DCT_SCRIPT_MULTICHAIN_BLOCK_SIGNATURE_PREFIX) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + + sig_len=mc_GetLE(ptr,1); + ptr++; + + if(sig_len>*sig_size) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(m_lpCoord[m_CurrentElement*2+1] < MC_DCT_SCRIPT_IDENTIFIER_LEN+1+3+sig_len) + { + return MC_ERR_WRONG_SCRIPT; + } + + memcpy(sig,ptr,sig_len); + *sig_size=sig_len; + ptr+=sig_len; + + *hash_type=(uint32_t)mc_GetLE(ptr,1); + ptr++; + + key_len=mc_GetLE(ptr,1); + ptr++; + + if(key_len>*key_size) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(m_lpCoord[m_CurrentElement*2+1] != MC_DCT_SCRIPT_IDENTIFIER_LEN+1+3+sig_len+key_len) + { + return MC_ERR_WRONG_SCRIPT; + } + + memcpy(key,ptr,key_len); + *key_size=key_len; + ptr+=key_len; + + return MC_ERR_NOERROR; +} + +int mc_Script::SetBlockSignature(const unsigned char* sig,int sig_size,uint32_t hash_type,const unsigned char* key,int key_size) +{ + int err; + unsigned char buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1]; + + if((sig_size>0xff) || (key_size>0xff)) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + err=AddElement(); + if(err) + { + return err; + } + + memcpy(buf,MC_DCT_SCRIPT_COINSPARK_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN); + buf[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_BLOCK_SIGNATURE_PREFIX; + + err=SetData(buf,MC_DCT_SCRIPT_IDENTIFIER_LEN+1); + if(err) + { + return err; + } + + mc_PutLE(buf,&sig_size,1); + err=SetData(buf,1); + if(err) + { + return err; + } + + err=SetData(sig,sig_size); + if(err) + { + return err; + } + + mc_PutLE(buf,&hash_type,1); + err=SetData(buf,1); + if(err) + { + return err; + } + + mc_PutLE(buf,&key_size,1); + err=SetData(buf,1); + if(err) + { + return err; + } + + err=SetData(key,key_size); + if(err) + { + return err; + } + + return MC_ERR_NOERROR; +} + +int mc_Script::GetAssetGenesis(int64_t *quantity) +{ + unsigned char *ptr; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(m_lpCoord[m_CurrentElement*2+1] != MC_DCT_SCRIPT_IDENTIFIER_LEN+1+8) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + + if(memcmp(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] != MC_DCT_SCRIPT_MULTICHAIN_ASSET_GENESIS_PREFIX) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + *quantity=(uint64_t)mc_GetLE(ptr+ 0,8); + + if(*quantity < 0) + { + return MC_ERR_NOT_ALLOWED; + } + + return MC_ERR_NOERROR; +} + +int mc_Script::SetAssetGenesis(int64_t quantity) +{ + unsigned char buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1+8]; + unsigned char *ptr; + int err; + + if(quantity < 0) + { + return MC_ERR_NOT_ALLOWED; + } + + err=AddElement(); + if(err) + { + return err; + } + + ptr=buf; + memcpy(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN); + ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_ASSET_GENESIS_PREFIX; + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + mc_PutLE(ptr+ 0,&quantity,8); + + return SetData(buf,MC_DCT_SCRIPT_IDENTIFIER_LEN+1+8); +} + +int mc_Script::GetAssetDetails(char* name,int* multiple,unsigned char* script,int *script_size) +{ + unsigned char *ptr; + unsigned char *ptrEnd; + unsigned char *ptrStart; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(mc_gState->m_Features->SpecialParamsInDetailsScript()) + { + if(GetGeneralDetails(script,script_size) == MC_ERR_NOERROR) + { + name[0]=0; + *multiple=1; + return MC_ERR_NOERROR; + } + } + + if(m_lpCoord[m_CurrentElement*2+1] < MC_DCT_SCRIPT_IDENTIFIER_LEN+1 + 4 + 1) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(m_lpCoord[m_CurrentElement*2+1] > MC_DCT_SCRIPT_IDENTIFIER_LEN+1+MC_ENT_MAX_SCRIPT_SIZE + 4 + MC_ENT_MAX_NAME_SIZE + 1) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + ptrEnd=ptr+m_lpCoord[m_CurrentElement*2+1]; + + if(memcmp(ptr,MC_DCT_SCRIPT_COINSPARK_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] != MC_DCT_SCRIPT_MULTICHAIN_ASSET_DETAILS_PREFIX) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + + *multiple=(int)mc_GetLE(ptr+ 0,4); + ptr+=4; + + ptrStart=ptr; + while(*ptr) + { + if(ptr >= ptrStart+MC_ENT_MAX_NAME_SIZE) + { + return MC_ERR_WRONG_SCRIPT; + } + ptr++; + if(ptr >= ptrEnd) + { + return MC_ERR_WRONG_SCRIPT; + } + } + + strcpy(name,(char*)ptrStart); + ptr++; + + *script_size=ptrEnd-ptr; + + if(*script_size) + { + memcpy(script,ptr,*script_size); + } + + return MC_ERR_NOERROR; +} + +int mc_Script::SetAssetDetails(const char*name,int multiple,const unsigned char* script,int script_size) +{ + int err; + unsigned char buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1+4]; + + if((script_size<0) || (script_size>MC_ENT_MAX_SCRIPT_SIZE)) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + err=AddElement(); + if(err) + { + return err; + } + + memcpy(buf,MC_DCT_SCRIPT_COINSPARK_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN); + buf[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_ASSET_DETAILS_PREFIX; + + err=SetData(buf,MC_DCT_SCRIPT_IDENTIFIER_LEN+1); + if(err) + { + return err; + } + + mc_PutLE(buf,&multiple,4); + err=SetData(buf,4); + if(err) + { + return err; + } + + err=SetData((unsigned char*)name,strlen(name)+1); + if(err) + { + return err; + } + + if(script_size) + { + err=SetData(script,script_size); + if(err) + { + return err; + } + } + + return MC_ERR_NOERROR; +} + +int mc_Script::GetGeneralDetails(unsigned char* script,int *script_size) +{ + unsigned char *ptr; + unsigned char *ptrEnd; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(m_lpCoord[m_CurrentElement*2+1] < MC_DCT_SCRIPT_IDENTIFIER_LEN+1) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(m_lpCoord[m_CurrentElement*2+1] > MC_DCT_SCRIPT_IDENTIFIER_LEN+1+MC_ENT_MAX_SCRIPT_SIZE) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + ptrEnd=ptr+m_lpCoord[m_CurrentElement*2+1]; + + if(memcmp(ptr,MC_DCT_SCRIPT_COINSPARK_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] != MC_DCT_SCRIPT_MULTICHAIN_GENERAL_DETAILS_PREFIX) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + + *script_size=ptrEnd-ptr; + + if(*script_size) + { + memcpy(script,ptr,*script_size); + } + + return MC_ERR_NOERROR; +} + +int mc_Script::SetGeneralDetails(const unsigned char* script,int script_size) +{ + int err; + unsigned char buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1+4]; + + if((script_size<0) || (script_size>MC_ENT_MAX_SCRIPT_SIZE)) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + err=AddElement(); + if(err) + { + return err; + } + + memcpy(buf,MC_DCT_SCRIPT_COINSPARK_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN); + buf[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_GENERAL_DETAILS_PREFIX; + + err=SetData(buf,MC_DCT_SCRIPT_IDENTIFIER_LEN+1); + if(err) + { + return err; + } + + if(script_size) + { + err=SetData(script,script_size); + if(err) + { + return err; + } + } + + return MC_ERR_NOERROR; +} + +int mc_Script::GetNewEntityType(uint32_t *type,int *update,unsigned char* script,int *script_size) +{ + unsigned char *ptr; + unsigned char *ptrEnd; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(m_lpCoord[m_CurrentElement*2+1] < MC_DCT_SCRIPT_IDENTIFIER_LEN+1+1) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(m_lpCoord[m_CurrentElement*2+1] > MC_DCT_SCRIPT_IDENTIFIER_LEN+1+1+MC_ENT_MAX_SCRIPT_SIZE) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + ptrEnd=ptr+m_lpCoord[m_CurrentElement*2+1]; + + if(memcmp(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] == MC_DCT_SCRIPT_MULTICHAIN_NEW_ENTITY_PREFIX) + { + *update=0; + } + else + { + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] == MC_DCT_SCRIPT_MULTICHAIN_UPDATE_ENTITY_PREFIX) + { + *update=1; + } + else + { + return MC_ERR_WRONG_SCRIPT; + } + } + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + + *type=(uint32_t)(*ptr); + ptr++; + + *script_size=ptrEnd-ptr; + + if(*script_size) + { + memcpy(script,ptr,*script_size); + } + + return mc_VerifyDetailsScript(script,*script_size); +} + +int mc_Script::SetNewEntityType(const uint32_t type,const int update,const unsigned char* script,int script_size) +{ + int err; + unsigned char buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1+1]; + + if((script_size<0) || (script_size>MC_ENT_MAX_SCRIPT_SIZE)) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + err=mc_VerifyDetailsScript(script,script_size); + if(err) + { + return err; + } + + err=AddElement(); + if(err) + { + return err; + } + + memcpy(buf,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN); + if(update) + { + buf[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_UPDATE_ENTITY_PREFIX; + } + else + { + buf[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_NEW_ENTITY_PREFIX; + } + buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1]=(unsigned char)(type & 0xff); + + err=SetData(buf,MC_DCT_SCRIPT_IDENTIFIER_LEN+1+1); + if(err) + { + return err; + } + + if(script_size) + { + err=SetData(script,script_size); + if(err) + { + return err; + } + } + + return MC_ERR_NOERROR; +} + +int mc_Script::GetItemKey(unsigned char *key,int *key_size) +{ + unsigned char *ptr; + unsigned char *ptrEnd; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(m_lpCoord[m_CurrentElement*2+1] < MC_DCT_SCRIPT_IDENTIFIER_LEN+1) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(m_lpCoord[m_CurrentElement*2+1] > MC_DCT_SCRIPT_IDENTIFIER_LEN+1+MC_ENT_MAX_ITEM_KEY_SIZE) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + ptrEnd=ptr+m_lpCoord[m_CurrentElement*2+1]; + + if(memcmp(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] != MC_DCT_SCRIPT_MULTICHAIN_KEY_PREFIX) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + + *key_size=ptrEnd-ptr; + + if(*key_size) + { + memcpy(key,ptr,*key_size); + } + + return MC_ERR_NOERROR; +} + +int mc_Script::SetItemKey(const unsigned char* key,int key_size) +{ + int err; + unsigned char buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1+4]; + + if((key_size<0) || (key_size>MC_ENT_MAX_ITEM_KEY_SIZE)) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + err=AddElement(); + if(err) + { + return err; + } + + memcpy(buf,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN); + buf[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_KEY_PREFIX; + + err=SetData(buf,MC_DCT_SCRIPT_IDENTIFIER_LEN+1); + if(err) + { + return err; + } + + if(key_size) + { + err=SetData(key,key_size); + if(err) + { + return err; + } + } + + return MC_ERR_NOERROR; +} + + +int mc_Script::GetEntity(unsigned char *short_txid) +{ + unsigned char *ptr; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(m_lpCoord[m_CurrentElement*2+1] != MC_DCT_SCRIPT_IDENTIFIER_LEN+1+MC_AST_SHORT_TXID_SIZE) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + + if(memcmp(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] != MC_DCT_SCRIPT_MULTICHAIN_ENTITY_PREFIX) + { + return MC_ERR_WRONG_SCRIPT; + } + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + + memcpy(short_txid,ptr,MC_AST_SHORT_TXID_SIZE); + + return MC_ERR_NOERROR; +} + +int mc_Script::SetEntity(const unsigned char *short_txid) +{ + int err; + unsigned char buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1+MC_AST_SHORT_TXID_SIZE]; + + err=AddElement(); + if(err) + { + return err; + } + + memcpy(buf,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN); + buf[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_ENTITY_PREFIX; + memcpy(buf+MC_DCT_SCRIPT_IDENTIFIER_LEN+1,short_txid,MC_AST_SHORT_TXID_SIZE); + + err=SetData(buf,MC_DCT_SCRIPT_IDENTIFIER_LEN+1+MC_AST_SHORT_TXID_SIZE); + if(err) + { + return err; + } + + return MC_ERR_NOERROR; +} + +int mc_Script::GetNewEntityType(uint32_t *type) +{ + unsigned char *ptr; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + if(m_lpCoord[m_CurrentElement*2+1] < MC_DCT_SCRIPT_IDENTIFIER_LEN+1+1) + { + return MC_ERR_WRONG_SCRIPT; + } + } + else + { + if(m_lpCoord[m_CurrentElement*2+1] != MC_DCT_SCRIPT_IDENTIFIER_LEN+1+1) + { + return MC_ERR_WRONG_SCRIPT; + } + } + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + + if(memcmp(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] != MC_DCT_SCRIPT_MULTICHAIN_NEW_ENTITY_PREFIX) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + + *type=(uint32_t)(*ptr); + + return MC_ERR_NOERROR; +} + +int mc_Script::SetNewEntityType(const uint32_t type) +{ + int err; + unsigned char buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1+1]; + + err=AddElement(); + if(err) + { + return err; + } + + memcpy(buf,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN); + buf[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_NEW_ENTITY_PREFIX; + buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1]=(unsigned char)(type & 0xff); + + err=SetData(buf,MC_DCT_SCRIPT_IDENTIFIER_LEN+1+1); + if(err) + { + return err; + } + + return MC_ERR_NOERROR; +} + + +int mc_Script::GetFullRef(unsigned char *ref,uint32_t *script_type) +{ + unsigned char *ptr; + int items,i,new_ref,shift,ref_type; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(m_lpCoord[m_CurrentElement*2+1] < MC_DCT_SCRIPT_IDENTIFIER_LEN+1) + { + return MC_ERR_WRONG_SCRIPT; + } + + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + + if(memcmp(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + *script_type=0; + + if(mc_gState->m_Features->FollowOnIssues()) + { + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] == MC_DCT_SCRIPT_MULTICHAIN_ASSET_FOLLOWON_PREFIX) + { + *script_type=MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON; + } + } + + if(*script_type == 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + + items=(m_lpCoord[m_CurrentElement*2+1] - (MC_DCT_SCRIPT_IDENTIFIER_LEN+1)) / (mc_gState->m_NetworkParams->m_AssetRefSize + MC_AST_ASSET_QUANTITY_SIZE); + + if(m_lpCoord[m_CurrentElement*2+1] != MC_DCT_SCRIPT_IDENTIFIER_LEN + 1 + items * (mc_gState->m_NetworkParams->m_AssetRefSize + MC_AST_ASSET_QUANTITY_SIZE)) + { + return MC_ERR_WRONG_SCRIPT; + } + + new_ref=1; + shift=0; + ref_type=MC_AST_ASSET_REF_TYPE_REF; + if(mc_gState->m_Features->ShortTxIDAsAssetRef()) + { + shift=MC_AST_SHORT_TXID_OFFSET; + ref_type=MC_AST_ASSET_REF_TYPE_SHORT_TXID; + } + memset(ref,0,MC_AST_ASSET_FULLREF_SIZE); + + for(i=0;im_NetworkParams->m_AssetRefSize)) + { + return MC_ERR_WRONG_SCRIPT; + } + } + else + { + memcpy(ref+shift,ptr,mc_gState->m_NetworkParams->m_AssetRefSize); + new_ref=0; + } + ptr+=mc_gState->m_NetworkParams->m_AssetRefSize + MC_AST_ASSET_QUANTITY_SIZE; + } + + if(new_ref == 0) + { + mc_SetABRefType(ref,ref_type); + } + + return MC_ERR_NOERROR; +} + +int mc_Script::GetAssetQuantities(mc_Buffer *amounts,uint32_t script_type) +{ + unsigned char *ptr; + int items,i,row,shift,ref_type; + int64_t last,quantity; + uint32_t found_script_type,last_script_type; + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + int valid_identitfier; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(m_lpCoord[m_CurrentElement*2+1] < MC_DCT_SCRIPT_IDENTIFIER_LEN+1) + { + return MC_ERR_WRONG_SCRIPT; + } + + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + + if(memcmp(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + valid_identitfier=0; + if(script_type & MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER) + { + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] == MC_DCT_SCRIPT_MULTICHAIN_ASSET_QUANTITY_PREFIX) + { + valid_identitfier=1; + found_script_type=MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER; + } + } + + if(mc_gState->m_Features->FollowOnIssues()) + { + if(script_type & MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON) + { + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] == MC_DCT_SCRIPT_MULTICHAIN_ASSET_FOLLOWON_PREFIX) + { + valid_identitfier=1; + found_script_type=MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON; + } + } + } + + if(valid_identitfier == 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr+=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + + shift=0; + ref_type=MC_AST_ASSET_REF_TYPE_REF; + if(mc_gState->m_Features->ShortTxIDAsAssetRef()) + { + shift=MC_AST_SHORT_TXID_OFFSET; + ref_type=MC_AST_ASSET_REF_TYPE_SHORT_TXID; + } + + items=(m_lpCoord[m_CurrentElement*2+1] - (MC_DCT_SCRIPT_IDENTIFIER_LEN+1)) / (mc_gState->m_NetworkParams->m_AssetRefSize + MC_AST_ASSET_QUANTITY_SIZE); + + if(m_lpCoord[m_CurrentElement*2+1] != MC_DCT_SCRIPT_IDENTIFIER_LEN + 1 + items * (mc_gState->m_NetworkParams->m_AssetRefSize + MC_AST_ASSET_QUANTITY_SIZE)) + { + return MC_ERR_WRONG_SCRIPT; + } + + for(i=0;im_NetworkParams->m_AssetRefSize,MC_AST_ASSET_QUANTITY_SIZE); + if(quantity < 0) + { + return MC_ERR_NOT_ALLOWED; + } + memset(buf,0, MC_AST_ASSET_FULLREF_BUF_SIZE); + memcpy(buf+shift,ptr,mc_gState->m_NetworkParams->m_AssetRefSize); + mc_SetABRefType(buf,ref_type); + + row=amounts->Seek(buf); + last=0; + if(row >= 0) + { + last=mc_GetABQuantity(amounts->GetRow(row)); + last_script_type=(uint32_t)mc_GetABScriptType(amounts->GetRow(row)); +/* + if(quantity+last < 0) + { + return MC_ERR_NOT_ALLOWED; + } + */ + quantity+=last; + if(last >= 0) + { + if(quantity < 0) // Protection from overflow + { + return MC_ERR_NOT_ALLOWED; + } + } + found_script_type|=last_script_type; + mc_SetABQuantity(amounts->GetRow(row),quantity); + mc_SetABScriptType(amounts->GetRow(row),found_script_type); + } + else + { + mc_SetABQuantity(buf,quantity); + mc_SetABScriptType(buf,found_script_type); + amounts->Add(buf); + } + + ptr+=mc_gState->m_NetworkParams->m_AssetRefSize + MC_AST_ASSET_QUANTITY_SIZE; + } + + return MC_ERR_NOERROR; +} + +int mc_Script::SetAssetQuantities(mc_Buffer *amounts,uint32_t script_type) +{ + unsigned char buf[MC_DCT_SCRIPT_IDENTIFIER_LEN+1]; + unsigned char *ptr; + int err,i,shift; + + err=AddElement(); + if(err) + { + return err; + } + + ptr=buf; + memcpy(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN); + + switch(script_type) + { + case MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER: + ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_ASSET_QUANTITY_PREFIX; + break; + case MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON: + if(mc_gState->m_Features->FollowOnIssues()) + { + ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_ASSET_FOLLOWON_PREFIX; + } + else + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + break; + default: + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + err=SetData(buf,MC_DCT_SCRIPT_IDENTIFIER_LEN+1); + if(err) + { + return err; + } + + shift=0; + if(mc_gState->m_Features->ShortTxIDAsAssetRef()) + { + shift=MC_AST_SHORT_TXID_OFFSET; + } + + + for(i=0;iGetCount();i++) + { + err=SetData(amounts->GetRow(i)+shift,mc_gState->m_NetworkParams->m_AssetRefSize); + if(err) + { + return err; + } + err=SetData(amounts->GetRow(i)+MC_AST_ASSET_QUANTITY_OFFSET,MC_AST_ASSET_QUANTITY_SIZE); + if(err) + { + return err; + } + } + + return MC_ERR_NOERROR; +} + +int mc_Script::GetCachedScript(int offset, int *next_offset, int* vin, unsigned char** script, int *script_size) +{ + unsigned char *ptr; + unsigned char *ptrEnd; + int shift; + + if(m_CurrentElement<0) + { + return MC_ERR_INVALID_PARAMETER_VALUE; + } + + if(m_lpCoord[m_CurrentElement*2+1] < MC_DCT_SCRIPT_IDENTIFIER_LEN+1) + { + return MC_ERR_WRONG_SCRIPT; + } + + ptr=m_lpData+m_lpCoord[m_CurrentElement*2+0]; + + if(memcmp(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN) != 0) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN] != MC_DCT_SCRIPT_MULTICHAIN_CASHED_SCRIPT_PREFIX) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(offset == 0) + { + *next_offset=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + return MC_ERR_NOERROR; + } + + if(m_lpCoord[m_CurrentElement*2+1] == offset) + { + return MC_ERR_WRONG_SCRIPT; + } + + if(m_lpCoord[m_CurrentElement*2+1] < offset + 4) + { + return MC_ERR_ERROR_IN_SCRIPT; + } + + ptrEnd=ptr+m_lpCoord[m_CurrentElement*2+1]; + ptr+=offset; + + *vin=mc_GetLE(ptr,4); + if(*vin<0) + { + return MC_ERR_ERROR_IN_SCRIPT; + } + ptr+=4; + + *script_size=(int)mc_GetVarInt(ptr,ptrEnd-ptr,-1,&shift); + if(*script_size<0) + { + return MC_ERR_ERROR_IN_SCRIPT; + } + + ptr+=shift; + + if(ptr+*script_size > ptrEnd) + { + return MC_ERR_ERROR_IN_SCRIPT; + } + + *script=ptr; + *next_offset=offset+4+shift+*script_size; + + return MC_ERR_NOERROR; +} + +int mc_Script::SetCachedScript(int offset, int *next_offset, int vin, unsigned char* script, int script_size) +{ + unsigned char buf[16]; + unsigned char *ptr; + int err,shift; + + if(offset == 0) + { + err=AddElement(); + if(err) + { + return err; + } + + ptr=buf; + memcpy(ptr,MC_DCT_SCRIPT_MULTICHAIN_IDENTIFIER,MC_DCT_SCRIPT_IDENTIFIER_LEN); + + ptr[MC_DCT_SCRIPT_IDENTIFIER_LEN]=MC_DCT_SCRIPT_MULTICHAIN_CASHED_SCRIPT_PREFIX; + + err=SetData(buf,MC_DCT_SCRIPT_IDENTIFIER_LEN+1); + if(err) + { + return err; + } + *next_offset=MC_DCT_SCRIPT_IDENTIFIER_LEN+1; + return MC_ERR_NOERROR; + } + + mc_PutLE(buf,&vin,4); + err=SetData(buf,4); + if(err) + { + return err; + } + + shift=mc_PutVarInt(buf,16,script_size); + err=SetData(buf,shift); + if(err) + { + return err; + } + + err=SetData(script,script_size); + if(err) + { + return err; + } + + *next_offset=offset+4+shift+script_size; + + return MC_ERR_NOERROR; +} diff --git a/src/protocol/multichainscript.h b/src/protocol/multichainscript.h new file mode 100644 index 00000000..088e6d25 --- /dev/null +++ b/src/protocol/multichainscript.h @@ -0,0 +1,97 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAINSCRIPT_H +#define MULTICHAINSCRIPT_H + +#include "utils/declare.h" + +#define MC_SCR_TYPE_SCRIPTPUBKEY 0 +#define MC_SCR_TYPE_SCRIPTSIG 1 +#define MC_SCR_TYPE_SCRIPTSIGRAW 2 + +#define MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER 0x00000001 +#define MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON 0x00000002 + + +typedef struct mc_Script +{ + int m_Size; + int m_NumElements; + int m_CurrentElement; + unsigned char* m_lpData; + int *m_lpCoord; + int m_AllocElements; + int m_AllocSize; + int m_ScriptType; + + mc_Script() + { + Zero(); + } + + ~mc_Script() + { + Destroy(); + } + + int Zero(); + int Destroy(); + int Resize(size_t bytes,int elements); + + int SetScript(const unsigned char* src,const size_t bytes,int type); + int IsOpReturnScript(); + int IsDirtyOpReturnScript(); + int Clear(); + + int GetNumElements(); + int AddElement(); + int SetSpecialParamValue(unsigned char param,const unsigned char* param_value,const size_t param_value_size); + int SetParamValue(const char *param_name,const size_t param_name_size,const unsigned char* param_value,const size_t param_value_size); + size_t GetParamValue(const unsigned char *ptr,size_t total,size_t offset,size_t* param_value_start,size_t *bytes); + int SetData(const unsigned char* src,const size_t bytes); + const unsigned char* GetData(int element,size_t *bytes); + + int GetElement(); + int SetElement(int element); + + int GetEntity(unsigned char *short_txid); + int SetEntity(const unsigned char *short_txid); + + int GetNewEntityType(uint32_t *type); + int SetNewEntityType(const uint32_t type); + + int GetNewEntityType(uint32_t *type,int *update,unsigned char* script,int *script_size); + int SetNewEntityType(const uint32_t type,const int update,const unsigned char* script,int script_size); + + int GetItemKey(unsigned char *key,int *key_size); + int SetItemKey(const unsigned char* key,int key_size); + + int GetPermission(uint32_t *type,uint32_t *from,uint32_t *to,uint32_t *timestamp); + int SetPermission(uint32_t type,uint32_t from,uint32_t to,uint32_t timestamp); + + int GetBlockSignature(unsigned char* sig,int *sig_size,uint32_t *hash_type,unsigned char* key,int *key_size); + int SetBlockSignature(const unsigned char* sig,int sig_size,uint32_t hash_type,const unsigned char* key,int key_size); + + int GetAssetGenesis(int64_t *quantity); + int SetAssetGenesis(int64_t quantity); + + int GetAssetDetails(char* name,int* multiple,unsigned char* script,int *script_size); + int SetAssetDetails(const char*name,int multiple,const unsigned char* script,int script_size); + + int GetGeneralDetails(unsigned char* script,int *script_size); + int SetGeneralDetails(const unsigned char* script,int script_size); + + int GetAssetQuantities(mc_Buffer *amounts,uint32_t script_type); + int SetAssetQuantities(mc_Buffer *amounts,uint32_t script_type); + + int GetFullRef(unsigned char *ref,uint32_t *script_type); + + int GetCachedScript(int offset, int *next_offset, int* vin, unsigned char** script, int *script_size); + int SetCachedScript(int offset, int *next_offset, int vin, unsigned char* script, int script_size); + +} mc_Script; + + +#endif /* MULTICHAINSCRIPT_H */ + diff --git a/src/protocol/multichaintx.cpp b/src/protocol/multichaintx.cpp new file mode 100644 index 00000000..e058048c --- /dev/null +++ b/src/protocol/multichaintx.cpp @@ -0,0 +1,2743 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2016 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "core/main.h" +#include "utils/util.h" +#include "multichain/multichain.h" +#include "structs/base58.h" + +using namespace std; + +string EncodeHexTx(const CTransaction& tx); + +uint160 mc_GenesisAdmin(const CTransaction& tx) +{ + uint32_t type,from,to,timestamp; + for (unsigned int j = 0; j < tx.vout.size(); j++) + { + mc_gState->m_TmpScript->Clear(); + const CScript& script1 = tx.vout[j].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + CTxDestination addressRet; + + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + mc_gState->m_TmpScript->SetElement(e); + if(mc_gState->m_TmpScript->GetPermission(&type,&from,&to,×tamp) == 0) + { + if(type == MC_PTP_GLOBAL_ALL) + { + CTxDestination addressRet; + if(ExtractDestination(script1, addressRet)) + { + CKeyID *lpKeyID=boost::get (&addressRet); + if(lpKeyID) + { + return *(uint160*)lpKeyID; + } + } + } + } + } + } + return 0; +} + +bool mc_ExtractInputAssetQuantities(const CScript& script1, uint256 hash, string& reason) +{ + int err; + int64_t quantity; + CScript::const_iterator pc1 = script1.begin(); + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + mc_gState->m_TmpScript->SetElement(e); + err=mc_gState->m_TmpScript->GetAssetQuantities(mc_gState->m_TmpAssetsIn,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER | MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON); + if((err != MC_ERR_NOERROR) && (err != MC_ERR_WRONG_SCRIPT)) + { + reason="Asset transfer script rejected - error in script"; + return false; + } + err=mc_gState->m_TmpScript->GetAssetGenesis(&quantity); + if(err == 0) + { + mc_EntityDetails entity; + unsigned char buf_amounts[MC_AST_ASSET_FULLREF_BUF_SIZE]; + memset(buf_amounts,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + if(mc_gState->m_Assets->FindEntityByTxID(&entity,(unsigned char*)&hash)) + { + if(mc_gState->m_Features->ShortTxIDAsAssetRef() == 0) + { + if(entity.IsUnconfirmedGenesis()) // Not confirmed genesis has -1 in offset field + { + reason="Asset transfer script rejected - using unconfirmed issue"; + return false; + } + } + memcpy(buf_amounts,entity.GetFullRef(),MC_AST_ASSET_FULLREF_SIZE); + int row=mc_gState->m_TmpAssetsIn->Seek(buf_amounts); + if(row>=0) + { + int64_t last=mc_GetABQuantity(mc_gState->m_TmpAssetsIn->GetRow(row)); + quantity+=last; + mc_SetABQuantity(mc_gState->m_TmpAssetsIn->GetRow(row),quantity); + } + else + { + mc_SetABQuantity(buf_amounts,quantity); + mc_gState->m_TmpAssetsIn->Add(buf_amounts); + } + } + else + { + reason="Asset transfer script rejected - issue tx not found"; + return false; + } + } + else + { + if(err != MC_ERR_WRONG_SCRIPT) + { + reason="Asset transfer script rejected - error in input issue script"; + return false; + } + } + } + + return true; +} + +bool mc_ExtractOutputAssetQuantities(string& reason) +{ + int err; + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + mc_gState->m_TmpScript->SetElement(e); + err=mc_gState->m_TmpScript->GetAssetQuantities(mc_gState->m_TmpAssetsOut,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER); + if((err != MC_ERR_NOERROR) && (err != MC_ERR_WRONG_SCRIPT)) + { + reason="Asset transfer script rejected - error in output transfer script"; + return false; + } + } + + return true; +} + +bool mc_CompareAssetQuantities(string& reason) +{ + unsigned char *ptrIn; + unsigned char *ptrOut; + int64_t quantity; + mc_EntityDetails entity; + + for(int i=0;im_TmpAssetsIn->GetCount();i++) + { + ptrIn=mc_gState->m_TmpAssetsIn->GetRow(i); + int row=mc_gState->m_TmpAssetsOut->Seek(ptrIn); + quantity=mc_GetABQuantity(ptrIn); + if(quantity>0) + { + if(row>=0) + { + ptrOut=mc_gState->m_TmpAssetsOut->GetRow(row); + if(memcmp(ptrIn,ptrOut,MC_AST_ASSET_QUANTITY_OFFSET+MC_AST_ASSET_QUANTITY_SIZE)) + { + reason="Asset transfer script rejected - mismatch in input/output quantities"; + return false; + } + } + else + { + reason="Asset transfer script rejected - mismatch in input/output quantities"; + return false; + } + } + } + + for(int i=0;im_TmpAssetsOut->GetCount();i++) + { + ptrOut=mc_gState->m_TmpAssetsOut->GetRow(i); + int row=mc_gState->m_TmpAssetsIn->Seek(ptrOut); + quantity=mc_GetABQuantity(ptrOut); + + if(mc_gState->m_Features->ShortTxIDAsAssetRef()) + { + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ptrOut) == 0) + { + reason="Asset transfer script rejected - asset not found"; + return false; + } + } + + if(quantity>0) + { + if(row>=0) + { + ptrIn=mc_gState->m_TmpAssetsIn->GetRow(row); + if(memcmp(ptrIn,ptrOut,MC_AST_ASSET_QUANTITY_OFFSET+MC_AST_ASSET_QUANTITY_SIZE)) + { + reason="Asset transfer script rejected - mismatch in input/output quantities"; + return false; + } + } + else + { + reason="Asset transfer script rejected - mismatch in input/output quantities"; + return false; + } + } + } + + return true; +} + +bool AcceptAssetGenesisFromPredefinedIssuers(const CTransaction &tx, + int offset,bool accept, + vector vInputDestinations, + vector vInputHashTypes, + vector vInputScriptTypes, + bool fReturnFalseIfFound, + string& reason) +{ + int update_mempool=0; + mc_EntityDetails entity; + mc_EntityDetails this_entity; + unsigned char details_script[MC_ENT_MAX_SCRIPT_SIZE]; + char asset_name[MC_ENT_MAX_NAME_SIZE+1]; + int multiple; + int details_script_size; + int err; + int64_t quantity,total; + uint256 txid; + bool new_issue,follow_on; + int details_script_type; + unsigned char *ptrOut; + vector issuers; + vector issuer_flags; + uint32_t flags; + uint32_t value_offset; + size_t value_size; + unsigned char short_txid[MC_AST_SHORT_TXID_SIZE]; + + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return true; + } + if(tx.IsCoinBase()) + { + return true; + } + + if(accept) + { + update_mempool=1; + } + + total=0; + + asset_name[0]=0; + multiple=1; + new_issue=false; + follow_on=false; + details_script_type=-1; + mc_gState->m_TmpAssetsOut->Clear(); + + details_script_size=0; + for (unsigned int j = 0; j < tx.vout.size(); j++) + { + mc_gState->m_TmpScript->Clear(); + + const CScript& script1 = tx.vout[j].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + uint32_t new_entity_type; + int entity_update; + + CTxDestination addressRet; + + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + if(mc_gState->m_TmpScript->IsOpReturnScript()) + { + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + if(mc_gState->m_TmpScript->GetNumElements() == 2 ) // One OP_DROP + OP_RETRUN - new entity + { + mc_gState->m_TmpScript->SetElement(0); + err=mc_gState->m_TmpScript->GetNewEntityType(&new_entity_type,&entity_update,details_script,&details_script_size); + if(err == 0) // New entity element + { + if(new_entity_type == MC_ENT_TYPE_ASSET) + { + if(details_script_type >= 0) + { + reason="Metadata script rejected - too many new entities"; + return false; + } + if(entity_update == 0) + { + details_script_type=entity_update; + *asset_name=0x00; + multiple=1; + value_offset=mc_FindSpecialParamInDetailsScript(details_script,details_script_size,MC_ENT_SPRM_NAME,&value_size); + if(value_offset<(uint32_t)details_script_size) + { + memcpy(asset_name,details_script+value_offset,value_size); + asset_name[value_size]=0x00; + } + value_offset=mc_FindSpecialParamInDetailsScript(details_script,details_script_size,MC_ENT_SPRM_ASSET_MULTIPLE,&value_size); + if(value_offset<(uint32_t)details_script_size) + { + multiple=mc_GetLE(details_script+value_offset,value_size); + } + } + } + } + else + { + if(err != MC_ERR_WRONG_SCRIPT) + { + reason="Asset details script rejected - error in script"; + return false; + } + } + + } + if(mc_gState->m_TmpScript->GetNumElements()) // 2 OP_DROPs + OP_RETURN - entity update + { + mc_gState->m_TmpScript->SetElement(1); + err=mc_gState->m_TmpScript->GetNewEntityType(&new_entity_type,&entity_update,details_script,&details_script_size); + if(err == 0) // New entity element + { + if(entity_update == 0) + { + reason="Metadata script rejected - wrong element, should be entity update"; + return false; + } + if(details_script_type >= 0) + { + reason="Metadata script rejected - too many new new entities/entity updates"; + return false; + } + if(new_entity_type == MC_ENT_TYPE_ASSET) + { + details_script_type=entity_update; + mc_gState->m_TmpScript->SetElement(0); + // Should be spke + if(mc_gState->m_TmpScript->GetEntity(short_txid)) // Entity element + { + reason="Metadata script rejected - wrong element, should be entityref"; + return false; + } + if(mc_gState->m_Assets->FindEntityByShortTxID(&entity,short_txid) == 0) + { + reason="Metadata script rejected - entity not found"; + return false; + } + } + else + { + reason="Metadata script rejected - wrong entity type, should be asset"; + return false; + } + } + } + } + else + { + if(details_script_size == 0) + { + int e=mc_gState->m_TmpScript->GetNumElements()-1; + if(e == 0) + { + mc_gState->m_TmpScript->SetElement(e); + err=mc_gState->m_TmpScript->GetAssetDetails(asset_name,&multiple,details_script,&details_script_size); + if(err) + { + if(err != MC_ERR_WRONG_SCRIPT) + { + reason="Asset details script rejected - error in script"; + return false; + } + details_script_size=0; + } + } + } + } + } + else + { + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + mc_gState->m_TmpScript->SetElement(e); + err=mc_gState->m_TmpScript->GetAssetGenesis(&quantity); + if(err == 0) + { + new_issue=true; + if(quantity+total<0) + { + reason="Asset issue script rejected - overflow"; + return false; + } + + total+=quantity; + + if(!ExtractDestination(script1, addressRet)) + { + reason="Asset issue script rejected - wrong destination type"; + return false; + } + + CKeyID *lpKeyID=boost::get (&addressRet); + CScriptID *lpScriptID=boost::get (&addressRet); + if((lpKeyID == NULL) && (lpScriptID == NULL)) + { + reason="Asset issue script rejected - wrong destination type"; + return false; + } + CBitcoinAddress address; + if(lpKeyID != NULL) + { + address=CBitcoinAddress(*lpKeyID); + } + else + { + address=CBitcoinAddress(*lpScriptID); + } + if(update_mempool) + { + LogPrint("mchn","Found asset issue script in tx %s for %s - (%ld)\n", + tx.GetHash().GetHex().c_str(), + address.ToString().c_str(),quantity); + } + } + else + { + if(err != MC_ERR_WRONG_SCRIPT) + { + reason="Asset issue script rejected - error in script"; + return false; + } + if(mc_gState->m_Features->FollowOnIssues()) + { + err=mc_gState->m_TmpScript->GetAssetQuantities(mc_gState->m_TmpAssetsOut,MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON); + if((err != MC_ERR_NOERROR) && (err != MC_ERR_WRONG_SCRIPT)) + { + reason="Asset follow-on script rejected - error in follow-on script"; + return false; + } + if(err == MC_ERR_NOERROR) + { + if(update_mempool) + { + LogPrint("mchn","Found asset follow-on script in tx %s\n", + tx.GetHash().GetHex().c_str()); + } + } + } + } + } + } + } + + if(details_script_type >= 0) + { + if(details_script_type) + { + follow_on=true; + } + else + { + new_issue=true; + } + } + + if(mc_gState->m_TmpAssetsOut->GetCount()) + { + follow_on=true; + } + + if(new_issue) + { + if(total == 0) + { + if(mc_gState->m_Features->FollowOnIssues() == 0) + { + reason="Asset follow-on script rejected - issue quantity should be positive"; + return false; + } + } + } + + if(follow_on) + { + total=0; + if(mc_gState->m_TmpAssetsOut->GetCount() > 1) + { + reason="Asset follow-on script rejected - follow-on for several assets"; + return false; + } + if(new_issue) + { + reason="Asset follow-on script rejected - follow-on and issue in one transaction"; + return false; + } + ptrOut=NULL; + if(mc_gState->m_TmpAssetsOut->GetCount() == 0) + { + if(mc_gState->m_Assets->FindEntityByShortTxID(&entity,short_txid) == 0) + { + reason="Details script rejected - entity not found"; + return false; + } + } + else + { + ptrOut=mc_gState->m_TmpAssetsOut->GetRow(0); + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ptrOut) == 0) + { + reason="Asset follow-on script rejected - asset not found"; + return false; + } + } + if(mc_gState->m_Features->ShortTxIDAsAssetRef() == 0) + { + if(entity.IsUnconfirmedGenesis()) + { + reason="Asset follow-on script rejected - unconfirmed asset"; + return false; + } + } + if(entity.AllowedFollowOns() == 0) + { + reason="Asset follow-on script rejected - follow-ons not allowed for this asset"; + return false; + } + if(details_script_type >= 0) + { + if(memcmp(entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,short_txid,MC_AST_SHORT_TXID_SIZE)) + { + reason="Asset follow-on script rejected - mismatch in follow-on quantity asset and details script"; + return false; + } + } + if(ptrOut) + { + total=mc_GetABQuantity(ptrOut); + if(total+mc_gState->m_Assets->GetTotalQuantity(&entity) < 0) + { + reason="Asset follow-on script rejected - exceeds maximal value for asset"; + return false; + } + } + } + + if(!new_issue && !follow_on) + { + return true; + } + + if(fReturnFalseIfFound) + { + { + reason="Asset issue script rejected - not allowed in this transaction, conflicts with other entities"; + return false; + } + } + + issuers.clear(); + if(vInputDestinations.size()) + { + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + if(vInputHashTypes[i] == SIGHASH_ALL) + { + if(vInputDestinations[i] != 0) + { + bool can_issue=false; + if(new_issue) + { + if(mc_gState->m_Permissions->CanIssue(NULL,(unsigned char*)&vInputDestinations[i])) + { + can_issue=true; + } + } + else + { + if(mc_gState->m_Permissions->CanIssue(entity.GetTxID(),(unsigned char*)&vInputDestinations[i])) + { + can_issue=true; + } + } + if(can_issue) + { + issuers.push_back(vInputDestinations[i]); + flags=MC_PFL_NONE; + if(vInputScriptTypes[i] == TX_SCRIPTHASH) + { + flags |= MC_PFL_IS_SCRIPTHASH; + } + issuer_flags.push_back(flags); + } + } + } + } + } + else + { + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + const CScript& script2 = tx.vin[i].scriptSig; + CScript::const_iterator pc2 = script2.begin(); + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc2[0]),(size_t)(script2.end()-pc2),MC_SCR_TYPE_SCRIPTSIG); + + if(mc_gState->m_TmpScript->GetNumElements() == 2) + { + size_t elem_size; + const unsigned char *elem; + + elem = mc_gState->m_TmpScript1->GetData(0,&elem_size); + + if(elem_size > 1) // If this is multisig with one signature it should be OP_0 + { + unsigned char hash_type=elem[elem_size-1] & 0x1f; + + if(hash_type == SIGHASH_ALL) + { + elem = mc_gState->m_TmpScript->GetData(1,&elem_size); + const unsigned char *pubkey_hash=(unsigned char *)Hash160(elem,elem+elem_size).begin(); + + if(new_issue) + { + if(mc_gState->m_Permissions->CanIssue(NULL,pubkey_hash)) + { + issuers.push_back(Hash160(elem,elem+elem_size)); + } + } + else + { + if(mc_gState->m_Permissions->CanIssue(entity.GetTxID(),pubkey_hash)) + { + issuers.push_back(Hash160(elem,elem+elem_size)); + } + } + } + } + } + } + } + + if(issuers.size() == 0) + { + reason="Inputs don't belong to valid issuer"; + return false; + } + + err=MC_ERR_NOERROR; + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->AddElement(); + if(mc_gState->m_Features->FollowOnIssues()) + { + unsigned char issuer_buf[24]; + memset(issuer_buf,0,sizeof(issuer_buf)); + uint32_t flags=MC_PFL_NONE; + uint32_t timestamp=0; + set stored_issuers; + + if(new_issue) + { + mc_gState->m_Permissions->SetCheckPoint(); + err=MC_ERR_NOERROR; + + txid=tx.GetHash(); + err=mc_gState->m_Permissions->SetPermission(&txid,issuer_buf,MC_PTP_CONNECT, + (unsigned char*)issuers[0].begin(),0,(uint32_t)(-1),timestamp,flags | MC_PFL_ENTITY_GENESIS ,update_mempool); + } + + for (unsigned int i = 0; i < issuers.size(); i++) + { + if(err == MC_ERR_NOERROR) + { + if(stored_issuers.count(issuers[i]) == 0) + { + memcpy(issuer_buf,issuers[i].begin(),sizeof(uint160)); + mc_PutLE(issuer_buf+sizeof(uint160),&issuer_flags[i],4); + mc_gState->m_TmpScript->SetSpecialParamValue(MC_ENT_SPRM_ISSUER,issuer_buf,sizeof(issuer_buf)); + if(new_issue) + { + err=mc_gState->m_Permissions->SetPermission(&txid,issuer_buf,MC_PTP_ADMIN | MC_PTP_ISSUE, + (unsigned char*)issuers[0].begin(),0,(uint32_t)(-1),timestamp,flags | MC_PFL_ENTITY_GENESIS ,update_mempool); + } + stored_issuers.insert(issuers[i]); + } + } + } + + memset(issuer_buf,0,sizeof(issuer_buf)); + mc_gState->m_TmpScript->SetSpecialParamValue(MC_ENT_SPRM_ISSUER,issuer_buf,1); + if(new_issue) + { + if((err != MC_ERR_NOERROR) || !accept) + { + mc_gState->m_Permissions->RollBackToCheckPoint(); + } + } + if(err) + { + reason="Cannot update permission database for issued asset"; + return false; + } + } + + const unsigned char *special_script; + size_t special_script_size=0; + special_script=mc_gState->m_TmpScript->GetData(0,&special_script_size); + txid=tx.GetHash(); + if(new_issue) + { + err=mc_gState->m_Assets->InsertAsset(&txid,offset,total,asset_name,multiple,details_script,details_script_size,special_script,special_script_size,update_mempool); + } + else + { + err=mc_gState->m_Assets->InsertAssetFollowOn(&txid,offset,total,details_script,details_script_size,special_script,special_script_size,entity.GetTxID(),update_mempool); + } + + if(err) + { + reason="Asset issue script rejected - could not insert new asset to database"; + if(err == MC_ERR_FOUND) + { + reason="Asset issue script rejected - entity with this name/asset-ref/txid already exists"; + if(mc_gState->m_Assets->FindEntityByTxID(&entity,(unsigned char*)&txid) == 0) + { + if(strlen(asset_name) == 0) + { + mc_gState->m_Assets->FindEntityByName(&entity,asset_name); + } + } + + LogPrint("mchn","Asset already exists. TxID: %s, AssetRef: %d-%d-%d, Name: %s\n", + tx.GetHash().GetHex().c_str(), + mc_gState->m_Assets->m_Block+1,offset,(int)(*((unsigned char*)&txid+31))+256*(int)(*((unsigned char*)&txid+30)), + entity.GetName()); + } + return false; + } + + + if(update_mempool) + { + if(offset>=0) + { + if(mc_gState->m_Assets->FindEntityByTxID(&this_entity,(unsigned char*)&txid)) + { + if(new_issue) + { + LogPrint("mchn","New asset. TxID: %s, AssetRef: %d-%d-%d, Name: %s\n", + tx.GetHash().GetHex().c_str(), + mc_gState->m_Assets->m_Block+1,offset,(int)(*((unsigned char*)&txid+0))+256*(int)(*((unsigned char*)&txid+1)), + this_entity.GetName()); + } + else + { + uint256 otxid; + memcpy(&otxid,entity.GetTxID(),32); + LogPrint("mchn","Follow-on issue. TxID: %s, Original issue txid: %s\n", + tx.GetHash().GetHex().c_str(),otxid.GetHex().c_str()); + } + } + else + { + reason="Asset issue script rejected - could not insert new asset to database"; + return false; + } + } + } + + return true; +} + + +bool AcceptMultiChainTransaction(const CTransaction& tx, + const CCoinsViewCache &inputs, + int offset, + bool accept, + string& reason) +{ + bool fScriptHashAllFound; + bool fSeedNodeInvolved; + bool fReject; + bool fShouldHaveDestination; + bool fRejectIfOpDropOpReturn; + bool fCheckCachedScript; + int nNewEntityOutput; + unsigned char details_script[MC_ENT_MAX_SCRIPT_SIZE]; + int details_script_size; + uint32_t new_entity_type; + int err; + + vector vInputScriptTypes; + vector vInputDestinations; + vector vInputHashTypes; + vector vInputCanGrantAdminMine; + set vAllowedAdmins; + set vAllowedActivators; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return true; + } + + details_script_size=0; + new_entity_type=MC_ENT_TYPE_NONE; + + fCheckCachedScript=false; + if(mc_gState->m_Features->CachedInputScript()) + { + if(mc_gState->m_NetworkParams->GetInt64Param("supportminerprecheck")) + { + fCheckCachedScript=true; + } + } + + fScriptHashAllFound=false; + fRejectIfOpDropOpReturn=false; + mc_gState->m_TmpAssetsIn->Clear(); + for (unsigned int i = 0; i < tx.vin.size(); i++) // Processing input scripts + { + if(tx.IsCoinBase()) + { + if(i == 0) + { + if(mc_gState->m_Permissions->m_Block == -1) + { + vInputScriptTypes.push_back(TX_PUBKEYHASH); + vInputDestinations.push_back(mc_GenesisAdmin(tx)); // Genesis admin is considered to be admin/opener of everything in genesis block + fCheckCachedScript=false; + } + else + { + vInputScriptTypes.push_back(TX_NONSTANDARD); + vInputDestinations.push_back(0); + } + fScriptHashAllFound=true; // Allows to transfer OP_RETURN metadata in this tx. + vInputHashTypes.push_back(SIGHASH_ALL); + } + else + { + vInputScriptTypes.push_back(TX_NONSTANDARD); + vInputDestinations.push_back(0); + vInputHashTypes.push_back(SIGHASH_NONE); + } + } + else + { + const CScript& script2 = tx.vin[i].scriptSig; + CScript::const_iterator pc2 = script2.begin(); + if(mc_gState->m_Features->FixedIn10007()) + { + if (!script2.IsPushOnly()) + { + reason="sigScript should be push-only"; + return false; + } + } + + const COutPoint &prevout = tx.vin[i].prevout; + const CCoins *coins = inputs.AccessCoins(prevout.hash); + assert(coins); + + const CScript& script1 = coins->vout[prevout.n].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + txnouttype typeRet; + int nRequiredRet; + vector addressRets; + int op_addr_offset,op_addr_size,is_redeem_script,sighash_type,check_last; + + sighash_type=SIGHASH_NONE; + if(ExtractDestinations(script1,typeRet,addressRets,nRequiredRet)) // Is Standard transaction + { + if (typeRet != TX_NULL_DATA) // Single-address destinations + { + CKeyID *lpKeyID=boost::get (&addressRets[0]); + CScriptID *lpScriptID=boost::get (&addressRets[0]); + if( (lpKeyID == NULL) && (lpScriptID == NULL) ) + { + reason="Internal error: cannot extract address from input scriptPubKey"; + return false; + } + else + { + if(typeRet != TX_MULTISIG) + { + if(lpKeyID) + { + vInputDestinations.push_back(*(uint160*)lpKeyID); + } + if(lpScriptID) + { + vInputDestinations.push_back(*(uint160*)lpScriptID); + } + } + else + { + vInputDestinations.push_back(0); + } + } + + check_last=0; + if( (typeRet == TX_PUBKEY) || (typeRet == TX_MULTISIG) ) + { + check_last=1; + } + + // Find sighash_type + mc_ExtractAddressFromInputScript((unsigned char*)(&pc2[0]),(int)(script2.end()-pc2),&op_addr_offset,&op_addr_size,&is_redeem_script,&sighash_type,check_last); + if(sighash_type == SIGHASH_ALL) + { + fScriptHashAllFound=true; + } + if(check_last) + { + fRejectIfOpDropOpReturn=true; // pay-to-pubkey and bare multisig script cannot be considered "publisher" for the stream, because we cannot + // extract it from the input script. Though we can still accept this transaction it is rejected + // for consistency with principle "each input which signs the stream item must contain a permitted writer" + } + } + else + { + fRejectIfOpDropOpReturn=true; // Null data scripts cannot be used in txs with OP_DROP+OP_RETURN + // We should not be there at all as null data scripts cannot be signed generally speaking + vInputDestinations.push_back(0); + } + vInputScriptTypes.push_back(typeRet); + } + else + { + fRejectIfOpDropOpReturn=true; // Non-standard inputs cannot be used in txs with OP_DROP+OP_RETURN + // Otherwise we cannot be sure where are the signatures in input script + vInputScriptTypes.push_back(TX_NONSTANDARD); + vInputDestinations.push_back(0); + } + + vInputHashTypes.push_back(sighash_type); + + if(!mc_ExtractInputAssetQuantities(script1,prevout.hash,reason)) // Filling input asset quantity list + { + return false; + } + } + + vInputCanGrantAdminMine.push_back(!fCheckCachedScript); + } + + fReject=false; + nNewEntityOutput=-1; + fSeedNodeInvolved=false; + fShouldHaveDestination=false; + fShouldHaveDestination |= (mc_gState->m_NetworkParams->GetInt64Param("anyonecanreceive") == 0); + fShouldHaveDestination |= (mc_gState->m_NetworkParams->GetInt64Param("allowmultisigoutputs") == 0); + fShouldHaveDestination |= (mc_gState->m_NetworkParams->GetInt64Param("allowp2shoutputs") == 0); + + mc_gState->m_Permissions->SetCheckPoint(); + mc_gState->m_TmpAssetsOut->Clear(); + + // Processing output scripts + // pass 0 - all permissions, except mine, admin and activate + script type checks + // pass 1 - mine permissions (to retrieve cached scripts in pass=0) + // pass 2 - admin and activate permissions (to retrive list of admins in pass=0) + // pass 3 - everything except permissions - balance, assets, streams + for(int pass=0;pass<4;pass++) + { + for (unsigned int j = 0; j < tx.vout.size(); j++) + { + bool fNoDestinationInOutput; + bool fIsPurePermission; + bool fAdminRequired; + bool fCheckAdminList; + bool fAllValidPublishers; + bool fScriptParsed; + unsigned char short_txid[MC_AST_SHORT_TXID_SIZE]; + mc_EntityDetails entity; + int receive_required; + uint32_t type,from,to,timestamp,flags; + unsigned char item_key[MC_ENT_MAX_ITEM_KEY_SIZE]; + int item_key_size,entity_update; + int cs_offset,cs_new_offset,cs_size,cs_vin; + unsigned char *cs_script; + + const CScript& script1 = tx.vout[j].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + if(mc_gState->m_TmpScript->IsOpReturnScript()) // OP_RETURN + { + if( (pass == 0) && !fScriptHashAllFound) + { + if(mc_gState->m_Features->FixedIn10007()) + { + if( (vInputHashTypes.size() <= j) || (vInputHashTypes[j] != SIGHASH_SINGLE) ) + { + reason="Output with metadata should be properly signed"; + fReject=true; + goto exitlbl; + } + } + else + { + reason="Tx with metadata should have at least one SIGHASH_ALL output"; + fReject=true; + goto exitlbl; + } + } + + if( (pass == 0) && (mc_gState->m_TmpScript->GetNumElements() > 1) ) // We have at least one OP_DROP element + { + if(fRejectIfOpDropOpReturn) // We cannot extract sighash_type properly from the input script + // as we don't know where are the signatures + { + reason="Non-standard, P2PK or bare multisig inputs cannot be used in this tx"; + fReject=true; + goto exitlbl; + } + + if(mc_gState->m_TmpScript->IsDirtyOpReturnScript()) + { + reason="Non-standard, Only OP_DROP elements are allowed in metadata outputs with OP_DROP"; + fReject=true; + goto exitlbl; + } + } + + if( (pass == 0) && (mc_gState->m_TmpScript->GetNumElements() == 2) ) // One OP_DROP + OP_RETURN - new entity or cached script + { + mc_gState->m_TmpScript->SetElement(0); + fScriptParsed=false; + + if(mc_gState->m_Features->CachedInputScript()) + { + cs_offset=0; + while( (err=mc_gState->m_TmpScript->GetCachedScript(cs_offset,&cs_new_offset,&cs_vin,&cs_script,&cs_size)) != MC_ERR_WRONG_SCRIPT ) + { + fScriptParsed=true; + if(err != MC_ERR_NOERROR) + { + reason="Metadata script rejected - error in cached script"; + fReject=true; + goto exitlbl; + } + if(cs_offset) + { + if( cs_vin >= (int)tx.vin.size() ) + { + reason="Metadata script rejected - invalid input in cached script"; + fReject=true; + goto exitlbl; + } + const COutPoint &prevout = tx.vin[cs_vin].prevout; + const CCoins *coins = inputs.AccessCoins(prevout.hash); + + const CScript& script3 = coins->vout[prevout.n].scriptPubKey; + CScript::const_iterator pc3 = script3.begin(); + + if(cs_size != (int)script3.size()) + { + reason="Metadata script rejected - cached script mismatch"; + fReject=true; + goto exitlbl; + } + if(memcmp(cs_script,(unsigned char*)&pc3[0],cs_size)) + { + reason="Metadata script rejected - cached script mismatch"; + fReject=true; + goto exitlbl; + } + if(fCheckCachedScript) + { + if(vInputHashTypes[cs_vin] == SIGHASH_ALL) + { + vInputCanGrantAdminMine[cs_vin]=true; + } + } + } + cs_offset=cs_new_offset; + } + } + + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + err=mc_gState->m_TmpScript->GetNewEntityType(&new_entity_type,&entity_update,details_script,&details_script_size); + if(err == 0) // New entity element + { + fScriptParsed=true; + if(nNewEntityOutput >= 0) + { + reason="Metadata script rejected - too many new entities"; + fReject=true; + goto exitlbl; + } + if(entity_update) + { + reason="Metadata script rejected - entity update script should be preceded by entityref"; + fReject=true; + goto exitlbl; + } + if(new_entity_type <= MC_ENT_TYPE_MAX) + { + if(new_entity_type != MC_ENT_TYPE_ASSET) + { + nNewEntityOutput=j; + } + } + else + { + reason="Metadata script rejected - unsupported new entity type"; + fReject=true; + goto exitlbl; + } + } + else + { + if(err != MC_ERR_WRONG_SCRIPT) + { + reason="Entity details script rejected - error in script"; + fReject=true; + goto exitlbl; + } + } + } + else + { + if(mc_gState->m_TmpScript->GetNewEntityType(&new_entity_type) == 0) // New entity element + { + fScriptParsed=true; + if(nNewEntityOutput >= 0) + { + reason="Metadata script rejected - too many new entities"; + fReject=true; + goto exitlbl; + } + if(new_entity_type != MC_ENT_TYPE_STREAM) + { + reason="Metadata script rejected - unsupported new entity type"; + fReject=true; + goto exitlbl; + } + nNewEntityOutput=j; + // Should be SPKc + mc_gState->m_TmpScript->SetElement(1); + err=mc_gState->m_TmpScript->GetGeneralDetails(details_script,&details_script_size); + if(err) + { + if(err != MC_ERR_WRONG_SCRIPT) + { + reason="Entity details script rejected - error in script"; + fReject=true; + goto exitlbl; + } + details_script_size=0; + } + } + } + if(!fScriptParsed) + { + reason="Metadata script rejected - Unrecognized script, should be new entity"; + if(mc_gState->m_Features->CachedInputScript()) + { + reason+=" or input script cache"; + } + fReject=true; + goto exitlbl; + } + } + + if( (pass == 0) && (mc_gState->m_TmpScript->GetNumElements() > 3) ) // More than 2 OP_DROPs + { + reason="Metadata script rejected - too many elements"; + fReject=true; + goto exitlbl; + } + + if( (pass == 3) && (mc_gState->m_TmpScript->GetNumElements() == 3) ) // 2 OP_DROPs + OP_RETURN - item key or entity update + { + mc_gState->m_TmpScript->SetElement(0); + // Should be spke + if(mc_gState->m_TmpScript->GetEntity(short_txid)) // Entity element + { + reason="Metadata script rejected - wrong element, should be entityref"; + fReject=true; + goto exitlbl; + } + if(mc_gState->m_Assets->FindEntityByShortTxID(&entity,short_txid) == 0) + { + reason="Metadata script rejected - entity not found"; + fReject=true; + goto exitlbl; + } + + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + if(entity.GetEntityType() == MC_ENT_TYPE_ASSET) + { + mc_gState->m_TmpScript->SetElement(1); + err=mc_gState->m_TmpScript->GetNewEntityType(&new_entity_type,&entity_update,details_script,&details_script_size); + if(err == 0) // New entity element + { + if(entity_update == 0) + { + reason="Metadata script rejected - wrong element, should be entity update"; + fReject=true; + goto exitlbl; + } + if(new_entity_type != MC_ENT_TYPE_ASSET) + { + reason="Metadata script rejected - entity type mismatch in update script"; + fReject=true; + goto exitlbl; + } + } + else + { + reason="Metadata script rejected - wrong element, should be entity update"; + fReject=true; + goto exitlbl; + } + } + else // (pseudo)stream + { + mc_gState->m_TmpScript->SetElement(1); + // Should be spkk + if(mc_gState->m_TmpScript->GetItemKey(item_key,&item_key_size)) // Item key + { + reason="Metadata script rejected - wrong element, should be item key"; + fReject=true; + goto exitlbl; + } + } + } + else + { + if(entity.GetEntityType() != MC_ENT_TYPE_STREAM) + { + reason="Metadata script rejected - not stream entity"; + fReject=true; + goto exitlbl; + } + mc_gState->m_TmpScript->SetElement(1); + // Should be spkk + if(mc_gState->m_TmpScript->GetItemKey(item_key,&item_key_size)) // Item key + { + reason="Metadata script rejected - wrong element, should be item key"; + fReject=true; + goto exitlbl; + } + } + + if(entity.GetEntityType() != MC_ENT_TYPE_ASSET) + { + fAllValidPublishers=true; + if(entity.AnyoneCanWrite() == 0) + { + if(mc_gState->m_Features->FixedIn10007()) + { + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + if( (vInputHashTypes[i] == SIGHASH_ALL) || ( (vInputHashTypes[i] == SIGHASH_SINGLE) && (i == j) ) ) + { + if(fAllValidPublishers) + { + if(mc_gState->m_Permissions->CanWrite(entity.GetTxID(),(unsigned char*)&vInputDestinations[i]) == 0) + { + fAllValidPublishers=false; + } + } + } + } + } + else + { + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + if(fAllValidPublishers) + { + fAllValidPublishers=false; + if( (vInputHashTypes[i] == SIGHASH_ALL) || ( (vInputHashTypes[i] == SIGHASH_SINGLE) && (i == j) ) ) + { + // Only in these two cases we can extract address from the input script + // When subscribing to the stream we don't have output scripts + if( (vInputScriptTypes[i] == TX_PUBKEYHASH) || (vInputScriptTypes[i] == TX_SCRIPTHASH) ) + { + if(mc_gState->m_Permissions->CanWrite(entity.GetTxID(),(unsigned char*)&vInputDestinations[i])) + { + fAllValidPublishers=true; + } + } + } + } + } + } + } + if(!fAllValidPublishers) + { + reason="Metadata script rejected - Inputs don't belong to valid publisher"; + fReject=true; + goto exitlbl; + } + } + } + } + else // Not OP_RETURN + { + txnouttype typeRet; + int nRequiredRet; + vector addressRets; + + fNoDestinationInOutput=false; + + if(!ExtractDestinations(script1,typeRet,addressRets,nRequiredRet)) + { + fNoDestinationInOutput=true; + } + + if( (pass == 0) && fShouldHaveDestination ) // Some setting in the protocol require address can be extracted + { + if(fNoDestinationInOutput && (mc_gState->m_NetworkParams->GetInt64Param("anyonecanreceive") == 0)) + { + reason="Script rejected - destination required "; + fReject=true; + goto exitlbl; + } + + if((typeRet == TX_MULTISIG) && (mc_gState->m_NetworkParams->GetInt64Param("allowmultisigoutputs") == 0)) + { + reason="Script rejected - multisig is not allowed"; + fReject=true; + goto exitlbl; + } + + if((typeRet == TX_SCRIPTHASH) && (mc_gState->m_NetworkParams->GetInt64Param("allowp2shoutputs") == 0)) + { + reason="Script rejected - P2SH is not allowed"; + fReject=true; + goto exitlbl; + } + } + + receive_required=0; // Number of required receive permissions + if( (pass == 3) && fShouldHaveDestination ) + { + receive_required=addressRets.size(); + if(typeRet == TX_MULTISIG) + { + receive_required-=(pc1[0]-0x50); + receive_required+=1; + if(receive_required>(int)addressRets.size()) + { + receive_required=addressRets.size(); + } + } + } + + fIsPurePermission=false; + if(mc_gState->m_TmpScript->GetNumElements()) + { + fIsPurePermission=true; + } + + entity.Zero(); // Permission processing + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + mc_gState->m_TmpScript->SetElement(e); + if(mc_gState->m_TmpScript->GetEntity(short_txid) == 0) // Entity element + { + if(entity.GetEntityType()) + { + reason="Script rejected - duplicate entity script"; + fReject=true; + goto exitlbl; + } + if(mc_gState->m_Assets->FindEntityByShortTxID(&entity,short_txid) == 0) + { + reason="Script rejected - entity not found"; + fReject=true; + goto exitlbl; + } + } + else // Not entity element + { + if(mc_gState->m_TmpScript->GetPermission(&type,&from,&to,×tamp) == 0)// Permission script + { + if(fNoDestinationInOutput) + { + reason="Script rejected - wrong destination type in output with permission script"; + fReject=true; + goto exitlbl; + } + fCheckAdminList=false; + if(mc_gState->m_Features->CachedInputScript()) + { + fCheckAdminList=true; + } + else + { + if(type & ( MC_PTP_ACTIVATE | MC_PTP_ADMIN )) + { + fCheckAdminList=true; + } + } + fAdminRequired=false; + switch(pass) + { + case 0: // Not admin or activate +// if(type & ( MC_PTP_ACTIVATE | MC_PTP_ADMIN )) + { + fAdminRequired=fCheckAdminList; + } + if(mc_gState->m_Features->CachedInputScript()) + { +// type &= ~( MC_PTP_ACTIVATE | MC_PTP_ADMIN | MC_PTP_MINE); + type &= ( MC_PTP_CONNECT | MC_PTP_SEND | MC_PTP_RECEIVE | MC_PTP_WRITE); + } + else + { + type &= ~( MC_PTP_ACTIVATE | MC_PTP_ADMIN); + } + break; + case 1: // Admin or activate + if(mc_gState->m_Features->CachedInputScript()) + { + type &= ( MC_PTP_CREATE | MC_PTP_ISSUE | MC_PTP_ACTIVATE ); +// type &= ( MC_PTP_MINE ); + } + else + { + type = 0; + } + break; + case 2: // Admin or activate + if(mc_gState->m_Features->CachedInputScript()) + { + type &= ( MC_PTP_MINE | MC_PTP_ADMIN ); + } + else + { + type &= ( MC_PTP_ACTIVATE | MC_PTP_ADMIN ); + } + break; + case 3: // This pass is not about permissions + type=0; + break; + } + if(fAdminRequired) // this is done for pass=2, calculating list of valid admins before this tx has effect + { + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + if( (vInputHashTypes[i] == SIGHASH_ALL) || ( (vInputHashTypes[i] == SIGHASH_SINGLE) && (i == j) ) ) + { + if(mc_gState->m_Permissions->CanAdmin(entity.GetTxID(),(unsigned char*)&vInputDestinations[i])) + { + vAllowedAdmins.insert(strprintf("%d-%d-%d",i,j,e)); + } + if(mc_gState->m_Permissions->CanActivate(entity.GetTxID(),(unsigned char*)&vInputDestinations[i])) + { + vAllowedActivators.insert(strprintf("%d-%d-%d",i,j,e)); + } + } + } + } + + if(type) + { + CKeyID *lpKeyID=boost::get (&addressRets[0]); + CScriptID *lpScriptID=boost::get (&addressRets[0]); + // Permissions cannot be granted to nonstandard outputs or bare multisigs + if(((lpKeyID == NULL) && (lpScriptID == NULL)) || (addressRets.size() > 1)) + { + reason="Permission script rejected - wrong destination type"; + fReject=true; + goto exitlbl; + } + CBitcoinAddress address; + unsigned char* ptr=NULL; + flags=MC_PFL_NONE; + if(lpKeyID != NULL) + { + address=CBitcoinAddress(*lpKeyID); + ptr=(unsigned char*)(lpKeyID); + if(type & MC_PTP_CONNECT) + { + if(mc_gState->m_pSeedNode) + { + CNode* seed_node; + seed_node=(CNode*)(mc_gState->m_pSeedNode); + + // If connect permission of seed node was involved, we may want to disconnect from it + if(memcmp(ptr,seed_node->kAddrRemote.begin(),20) == 0) + { + fSeedNodeInvolved=true; + } + } + } + } + else + { + flags=MC_PFL_IS_SCRIPTHASH; + address=CBitcoinAddress(*lpScriptID); + ptr=(unsigned char*)(lpScriptID); + } + + LogPrint("mchn","Found permission script in tx %s for %s - (%08x: %d - %d)\n", + tx.GetHash().GetHex().c_str(), + address.ToString().c_str(), + type, from, to); + + bool fAdminFound=false; + bool fAdminFoundWithoutCachedScript=false; + bool fActivateIsEnough=mc_gState->m_Permissions->IsActivateEnough(type); + for (unsigned int i = 0; i < tx.vin.size(); i++) + { +// if( ((pass < 2) && ( (vInputHashTypes[i] == SIGHASH_ALL) || ( (vInputHashTypes[i] == SIGHASH_SINGLE) && (i == j) ) )) || +// ((pass == 2) && (vAllowedAdmins.count(strprintf("%d-%d-%d",i,j,e)) > 0)) ) // Only inputs having admin permission before this tx are tried + if( ( fCheckAdminList && !fActivateIsEnough && (vAllowedAdmins.count(strprintf("%d-%d-%d",i,j,e)) > 0)) || + ( fCheckAdminList && fActivateIsEnough && (vAllowedActivators.count(strprintf("%d-%d-%d",i,j,e)) > 0)) || + (!fCheckAdminList && ( (vInputHashTypes[i] == SIGHASH_ALL) || ( (vInputHashTypes[i] == SIGHASH_SINGLE) && (i == j) ) ))) + { + if(vInputDestinations[i] != 0) + { + if( ( (type & (MC_PTP_ADMIN | MC_PTP_MINE)) == 0) || vInputCanGrantAdminMine[i] || (entity.GetEntityType() > 0) ) + { + if(mc_gState->m_Permissions->SetPermission(entity.GetTxID(),ptr,type,(unsigned char*)&vInputDestinations[i],from,to,timestamp,flags,1) == 0) + { + fAdminFound=true; + } + } + else + { + fAdminFoundWithoutCachedScript=true; + } + } + } + } + if(!fAdminFound) + { + reason="Inputs don't belong to valid admin"; + if(fAdminFoundWithoutCachedScript) + { + reason="Inputs require scriptPubKey cache to support miner precheck"; + } + fReject=true; + goto exitlbl; + } + } + entity.Zero(); // Entity element in the non-op-return output should be followed by permission element + // So only permission can nullify it + } + else // Not permission script + { + if(entity.GetEntityType()) + { + reason="Script rejected - entity script should be followed by permission"; + fReject=true; + goto exitlbl; + } + fIsPurePermission=false; + } + } + } // End of permission script processing + + if(entity.GetEntityType()) + { + reason="Script rejected - incomplete entity script"; + fReject=true; + goto exitlbl; + } + + if(tx.vout[j].nValue > 0) + { + fIsPurePermission=false; + } + + if(pass == 3) + { + if(!mc_ExtractOutputAssetQuantities(reason)) // Filling output asset quantity list + { + fReject=true; + goto exitlbl; + } + } + + if( (pass == 3) && !fIsPurePermission ) // Check for dust and receive permission + { + if((offset < 0) && Params().RequireStandard()) // If not in block - part of IsStandard check + { + if (tx.vout[j].IsDust(::minRelayTxFee)) + { + if(!tx.IsCoinBase()) + { + reason="Transaction amount too small"; + fReject=true; + goto exitlbl; + } + } + } + for(int a=0;a<(int)addressRets.size();a++) + { + CKeyID *lpKeyID=boost::get (&addressRets[a]); + CScriptID *lpScriptID=boost::get (&addressRets[a]); + if((lpKeyID == NULL) && (lpScriptID == NULL)) + { + reason="Script rejected - wrong destination type"; + fReject=true; + goto exitlbl; + } + unsigned char* ptr=NULL; + if(lpKeyID != NULL) + { + ptr=(unsigned char*)(lpKeyID); + } + else + { + ptr=(unsigned char*)(lpScriptID); + } + + bool fCanReceive=mc_gState->m_Permissions->CanReceive(NULL,ptr); + + if(tx.IsCoinBase()) // Miner can send funds to himself in coinbase even without receive permission + { + fCanReceive |= mc_gState->m_Permissions->CanMine(NULL,ptr); + } + if(fCanReceive) + { + receive_required--; + } + } + if(receive_required>0) + { + if( (tx.vout[j].nValue > 0) || + (mc_gState->m_TmpScript->GetNumElements() > 0) || + (mc_gState->m_Features->AnyoneCanReceiveEmpty() == 0) ) + { + reason="One of the outputs doesn't have receive permission"; + fReject=true; + goto exitlbl; + } + } + } + } // End of non-OP_RETURN output + } // End of output processing + } // End of pass loop + + if(!mc_CompareAssetQuantities(reason)) // Comparing input/output asset quantities + { + fReject=true; + goto exitlbl; + } + + // Checking asset geneses and follow-ons + // If new entity is found - asset genesis is forbidden, otherwise they will have the same reference + // If we fail here only permission db was updated. it will be rolled back + if(!AcceptAssetGenesisFromPredefinedIssuers(tx,offset,accept,vInputDestinations,vInputHashTypes,vInputScriptTypes,(nNewEntityOutput >= 0),reason)) + { + fReject=true; + goto exitlbl; + } + + if(nNewEntityOutput >= 0) + { + vector openers; + vector opener_flags; + unsigned char opener_buf[24]; + + uint32_t flags=MC_PFL_NONE; + uint32_t timestamp=0; + set stored_openers; + int update_mempool; + uint256 txid; + mc_EntityDetails entity; + + update_mempool=0; + if(accept) + { + update_mempool=1; + } + + openers.clear(); // List of openers + opener_flags.clear(); + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + if( (vInputHashTypes[i] == SIGHASH_ALL) || ( (vInputHashTypes[i] == SIGHASH_SINGLE) && ((int)i == nNewEntityOutput) ) ) + { + if(mc_gState->m_Permissions->CanCreate(NULL,(unsigned char*)&vInputDestinations[i])) + { + openers.push_back(vInputDestinations[i]); + flags=MC_PFL_NONE; + if(vInputScriptTypes[i] == TX_SCRIPTHASH) + { + flags |= MC_PFL_IS_SCRIPTHASH; + } + opener_flags.push_back(flags); + } + } + } + + if(openers.size() == 0) + { + reason="Metadata script rejected - Inputs don't belong to valid creator"; + fReject=true; + goto exitlbl; + } + + err=MC_ERR_NOERROR; + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->AddElement(); + txid=tx.GetHash(); // Setting first record in teh per-entity permissions list + memset(opener_buf,0,sizeof(opener_buf)); + err=mc_gState->m_Permissions->SetPermission(&txid,opener_buf,MC_PTP_CONNECT, + (unsigned char*)openers[0].begin(),0,(uint32_t)(-1),timestamp, MC_PFL_ENTITY_GENESIS ,update_mempool); + + for (unsigned int i = 0; i < openers.size(); i++) + { + if(err == MC_ERR_NOERROR) + { + if(stored_openers.count(openers[i]) == 0) + { + memcpy(opener_buf,openers[i].begin(),sizeof(uint160)); + mc_PutLE(opener_buf+sizeof(uint160),&opener_flags[i],4); + mc_gState->m_TmpScript->SetSpecialParamValue(MC_ENT_SPRM_ISSUER,opener_buf,sizeof(opener_buf)); + // Granting default permissions to openers + err=mc_gState->m_Permissions->SetPermission(&txid,opener_buf,MC_PTP_ADMIN | MC_PTP_ACTIVATE | MC_PTP_WRITE, + (unsigned char*)openers[i].begin(),0,(uint32_t)(-1),timestamp,opener_flags[i] | MC_PFL_ENTITY_GENESIS ,update_mempool); + stored_openers.insert(openers[i]); + } + } + } + + memset(opener_buf,0,sizeof(opener_buf)); // Storing opener list in entity metadata + mc_gState->m_TmpScript->SetSpecialParamValue(MC_ENT_SPRM_ISSUER,opener_buf,1); + if(err) + { + reason=" Cannot update permission database for new entity"; + fReject=true; + goto exitlbl; + } + + const unsigned char *special_script; + size_t special_script_size=0; + special_script=mc_gState->m_TmpScript->GetData(0,&special_script_size); + err=mc_gState->m_Assets->InsertStream(&txid,offset,new_entity_type,details_script,details_script_size,special_script,special_script_size,update_mempool); + if(err) + { + reason="New entity script rejected - could not insert new entity to database"; + if(err == MC_ERR_FOUND) + { + reason="New entity script rejected - entity with this name already exists"; + } + fReject=true; + goto exitlbl; + } + + if(update_mempool) + { + if(mc_gState->m_Assets->FindEntityByTxID(&entity,(unsigned char*)&txid)) + { + if(offset>=0) + { + LogPrint("mchn","New stream. TxID: %s, StreamRef: %d-%d-%d, Name: %s\n", + tx.GetHash().GetHex().c_str(), + mc_gState->m_Assets->m_Block+1,offset,(int)(*((unsigned char*)&txid+0))+256*(int)(*((unsigned char*)&txid+1)), + entity.GetName()); + } + else + { + LogPrint("mchn","New stream. TxID: %s, unconfirmed, Name: %s\n", + tx.GetHash().GetHex().c_str(),entity.GetName()); + } + } + else + { + reason="New entity script rejected - could not insert new entity to database"; + fReject=true; + goto exitlbl; + } + } + } + + +exitlbl: + + if(accept) + { + if(fSeedNodeInvolved) // Checking if we should disconnect from seed node + { + CNode* seed_node; + seed_node=(CNode*)(mc_gState->m_pSeedNode); + + if(!mc_gState->m_Permissions->CanConnect(NULL,seed_node->kAddrRemote.begin())) + { + LogPrintf("mchn: Seed node lost connect permission \n"); + mc_gState->m_pSeedNode=NULL; + } + } + } + + if(!accept || fReject) // Rolling back permission database if we were just checking or error occurred + { + mc_gState->m_Permissions->RollBackToCheckPoint(); + } + + if(fReject) + { + LogPrint("mchn","mchn: Tx rejected: %s\n",EncodeHexTx(tx)); + } + + return !fReject; +} + +/** + * Used only for protocols < 10006 + */ + +bool AcceptAssetTransfers(const CTransaction& tx, const CCoinsViewCache &inputs, string& reason) +{ + unsigned char *ptrIn; + unsigned char *ptrOut; + int err; + int64_t quantity; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return true; + } + if(tx.IsCoinBase()) + { + return true; + } + + mc_gState->m_TmpAssetsIn->Clear(); + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + const COutPoint &prevout = tx.vin[i].prevout; + const CCoins *coins = inputs.AccessCoins(prevout.hash); + assert(coins); + + mc_gState->m_TmpScript->Clear(); + + const CScript& script1 = coins->vout[prevout.n].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + mc_gState->m_TmpScript->SetElement(e); + err=mc_gState->m_TmpScript->GetAssetQuantities(mc_gState->m_TmpAssetsIn,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER | MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON); + if((err != MC_ERR_NOERROR) && (err != MC_ERR_WRONG_SCRIPT)) + { + reason="Asset transfer script rejected - error in script"; + return false; + } + err=mc_gState->m_TmpScript->GetAssetGenesis(&quantity); + if(err == 0) + { + uint256 hash=prevout.hash; + mc_EntityDetails entity; + unsigned char buf_amounts[MC_AST_ASSET_FULLREF_BUF_SIZE]; + memset(buf_amounts,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + if(mc_gState->m_Assets->FindEntityByTxID(&entity,(unsigned char*)&hash)) + { + if(entity.IsUnconfirmedGenesis()) // Not confirmed genesis has -1 in offset field + { + reason="Asset transfer script rejected - using unconfirmed issue"; + return false; + } + memcpy(buf_amounts,entity.GetFullRef(),MC_AST_ASSET_FULLREF_SIZE); + int row=mc_gState->m_TmpAssetsIn->Seek(buf_amounts); + if(row>=0) + { + int64_t last=mc_GetABQuantity(mc_gState->m_TmpAssetsIn->GetRow(row)); + quantity+=last; + mc_SetABQuantity(mc_gState->m_TmpAssetsIn->GetRow(row),quantity); + } + else + { + mc_SetABQuantity(buf_amounts,quantity); + mc_gState->m_TmpAssetsIn->Add(buf_amounts); + } + } + else + { + reason="Asset transfer script rejected - issue tx not found"; + return false; + } + } + else + { + if(err != MC_ERR_WRONG_SCRIPT) + { + reason="Asset transfer script rejected - error in input issue script"; + return false; + } + } + } + } + + mc_gState->m_TmpAssetsOut->Clear(); + for (unsigned int j = 0; j < tx.vout.size(); j++) + { + mc_gState->m_TmpScript->Clear(); + + const CScript& script1 = tx.vout[j].scriptPubKey; + + CScript::const_iterator pc1 = script1.begin(); + + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + mc_gState->m_TmpScript->SetElement(e); + err=mc_gState->m_TmpScript->GetAssetQuantities(mc_gState->m_TmpAssetsOut,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER); + if((err != MC_ERR_NOERROR) && (err != MC_ERR_WRONG_SCRIPT)) + { + reason="Asset transfer script rejected - error in output transfer script"; + return false; + } + if(err == MC_ERR_NOERROR) + { + } + } + } + + LogPrint("mchnminor","Found asset transfer script in tx %s, %d assets\n", + tx.GetHash().GetHex().c_str(),mc_gState->m_TmpAssetsOut->GetCount()); + for(int i=0;im_TmpAssetsIn->GetCount();i++) + { + ptrIn=mc_gState->m_TmpAssetsIn->GetRow(i); + int row=mc_gState->m_TmpAssetsOut->Seek(ptrIn); + quantity=mc_GetABQuantity(ptrIn); + if(quantity>0) + { + if(row>=0) + { + ptrOut=mc_gState->m_TmpAssetsOut->GetRow(row); + if(memcmp(ptrIn,ptrOut,MC_AST_ASSET_QUANTITY_OFFSET+MC_AST_ASSET_QUANTITY_SIZE)) + { + reason="Asset transfer script rejected - mismatch in input/output quantities"; + return false; + } + } + else + { + reason="Asset transfer script rejected - mismatch in input/output quantities"; + return false; + } + } + } + + for(int i=0;im_TmpAssetsOut->GetCount();i++) + { + ptrOut=mc_gState->m_TmpAssetsOut->GetRow(i); + int row=mc_gState->m_TmpAssetsIn->Seek(ptrOut); + quantity=mc_GetABQuantity(ptrOut); + if(quantity>0) + { + if(row>=0) + { + ptrIn=mc_gState->m_TmpAssetsIn->GetRow(row); + if(memcmp(ptrIn,ptrOut,MC_AST_ASSET_QUANTITY_OFFSET+MC_AST_ASSET_QUANTITY_SIZE)) + { + reason="Asset transfer script rejected - mismatch in input/output quantities"; + return false; + } + } + else + { + reason="Asset transfer script rejected - mismatch in input/output quantities"; + return false; + } + } + } + + return true; +} + +/** + * Used only for protocols < 10006 + */ + + +bool AcceptAssetGenesis(const CTransaction &tx,int offset,bool accept,string& reason) +{ + int update_mempool=0; + mc_EntityDetails entity; + mc_EntityDetails this_entity; + unsigned char details_script[MC_ENT_MAX_SCRIPT_SIZE]; + char asset_name[MC_ENT_MAX_NAME_SIZE+1]; + int multiple; + int details_script_size; + int err; + int64_t quantity,total; + uint256 txid; + bool new_issue; + unsigned char *ptrOut; + vector issuers; + + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return true; + } + if(tx.IsCoinBase()) + { + return true; + } + + if(accept) + { + update_mempool=1; + } + + total=0; + + asset_name[0]=0; + multiple=1; + new_issue=false; + mc_gState->m_TmpAssetsOut->Clear(); + + details_script_size=0; + for (unsigned int j = 0; j < tx.vout.size(); j++) + { + mc_gState->m_TmpScript->Clear(); + + const CScript& script1 = tx.vout[j].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + CTxDestination addressRet; + + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + if(mc_gState->m_TmpScript->IsOpReturnScript()) + { + int e=mc_gState->m_TmpScript->GetNumElements()-1; + if(e != 0) + { + reason="Asset details script rejected - error in script, too many script elements"; + return false; + } + { + mc_gState->m_TmpScript->SetElement(e); + err=mc_gState->m_TmpScript->GetAssetDetails(asset_name,&multiple,details_script,&details_script_size); + if(err) + { + if(err != MC_ERR_WRONG_SCRIPT) + { + reason="Asset details script rejected - error in script"; + return false; + } + details_script_size=0; + } + + } + } + else + { + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + mc_gState->m_TmpScript->SetElement(e); + err=mc_gState->m_TmpScript->GetAssetGenesis(&quantity); + if(err == 0) + { + new_issue=true; + if(quantity+total<0) + { + reason="Asset issue script rejected - negative total amount"; +// printf("%s\n",reason.c_str()); + return false; + } + + total+=quantity; + + if(!ExtractDestination(script1, addressRet)) + { + reason="Asset issue script rejected - wrong destination type"; + return false; + } + + CKeyID *lpKeyID=boost::get (&addressRet); + CScriptID *lpScriptID=boost::get (&addressRet); + if((lpKeyID == NULL) && (lpScriptID == NULL)) + { + reason="Asset issue script rejected - wrong destination type"; + return false; + } + CBitcoinAddress address; + if(lpKeyID != NULL) + { + address=CBitcoinAddress(*lpKeyID); + } + else + { + address=CBitcoinAddress(*lpScriptID); + } + if(update_mempool) + { + LogPrint("mchn","Found asset issue script in tx %s for %s - (%ld)\n", + tx.GetHash().GetHex().c_str(), + address.ToString().c_str(),quantity); + } + } + else + { + if(err != MC_ERR_WRONG_SCRIPT) + { + reason="Asset issue script rejected - error in script"; + return false; + } + if(mc_gState->m_Features->FollowOnIssues()) + { + err=mc_gState->m_TmpScript->GetAssetQuantities(mc_gState->m_TmpAssetsOut,MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON); + if((err != MC_ERR_NOERROR) && (err != MC_ERR_WRONG_SCRIPT)) + { + reason="Asset follow-on script rejected - error in follow-on script"; + return false; + } + if(err == MC_ERR_NOERROR) + { + if(update_mempool) + { + LogPrint("mchn","Found asset follow-on script in tx %s\n", + tx.GetHash().GetHex().c_str()); + } + } + } + } + } + + } + } + + if(new_issue) + { + if(total == 0) + { + if(mc_gState->m_Features->FollowOnIssues() == 0) + { + reason="Asset follow-on script rejected - issue quantity should be positive"; + return false; + } + } + } + + if(mc_gState->m_TmpAssetsOut->GetCount()) + { + if(mc_gState->m_TmpAssetsOut->GetCount() > 1) + { + reason="Asset follow-on script rejected - follow-on for several assets"; + return false; + } + if(new_issue) + { + reason="Asset follow-on script rejected - follow-on and issue in one transaction"; + return false; + } + ptrOut=mc_gState->m_TmpAssetsOut->GetRow(0); + if(mc_gState->m_Assets->FindEntityByRef(&entity,ptrOut) == 0) + { + reason="Asset follow-on script rejected - asset not found"; + return false; + } + if(entity.AllowedFollowOns() == 0) + { + reason="Asset follow-on script rejected - follow-ons not allowed for this asset"; + return false; + } + total=mc_GetABQuantity(ptrOut); + } + + if(!new_issue && (mc_gState->m_TmpAssetsOut->GetCount() == 0)) + { + return true; + } + + issuers.clear(); + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + const CScript& script2 = tx.vin[i].scriptSig; + CScript::const_iterator pc2 = script2.begin(); + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc2[0]),(size_t)(script2.end()-pc2),MC_SCR_TYPE_SCRIPTSIG); + + if(mc_gState->m_TmpScript->GetNumElements() == 2) + { + size_t elem_size; + const unsigned char *elem; + + elem = mc_gState->m_TmpScript1->GetData(0,&elem_size); + + if(elem_size > 1) // If this is multisig with one signature it should be OP_0 + { + unsigned char hash_type=elem[elem_size-1] & 0x1f; + + if(hash_type == SIGHASH_ALL) + { + elem = mc_gState->m_TmpScript->GetData(1,&elem_size); + const unsigned char *pubkey_hash=(unsigned char *)Hash160(elem,elem+elem_size).begin(); + + if(new_issue) + { + if(mc_gState->m_Permissions->CanIssue(NULL,pubkey_hash)) + { + issuers.push_back(Hash160(elem,elem+elem_size)); + } + } + else + { + if(mc_gState->m_Permissions->CanIssue(entity.GetTxID(),pubkey_hash)) + { + issuers.push_back(Hash160(elem,elem+elem_size)); + } + } + } + } + } + } + + if(issuers.size() == 0) + { + reason="Inputs don't belong to valid issuer"; + return false; + } + + err=MC_ERR_NOERROR; + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->AddElement(); + if(mc_gState->m_Features->FollowOnIssues()) + { + unsigned char issuer_buf[20]; + memset(issuer_buf,0,sizeof(issuer_buf)); + uint32_t flags=MC_PFL_NONE; + uint32_t timestamp=0; + set stored_issuers; + + if(new_issue) + { + mc_gState->m_Permissions->SetCheckPoint(); + err=MC_ERR_NOERROR; + + txid=tx.GetHash(); + err=mc_gState->m_Permissions->SetPermission(&txid,issuer_buf,MC_PTP_CONNECT, + (unsigned char*)issuers[0].begin(),0,(uint32_t)(-1),timestamp,flags | MC_PFL_ENTITY_GENESIS ,update_mempool); + } + + for (unsigned int i = 0; i < issuers.size(); i++) + { + if(err == MC_ERR_NOERROR) + { + if(stored_issuers.count(issuers[i]) == 0) + { + memcpy(issuer_buf,issuers[i].begin(),sizeof(uint160)); + mc_gState->m_TmpScript->SetSpecialParamValue(MC_ENT_SPRM_ISSUER,issuer_buf,sizeof(issuer_buf)); + if(new_issue) + { + err=mc_gState->m_Permissions->SetPermission(&txid,issuer_buf,MC_PTP_ADMIN | MC_PTP_ISSUE, + (unsigned char*)issuers[0].begin(),0,(uint32_t)(-1),timestamp,flags | MC_PFL_ENTITY_GENESIS ,update_mempool); + } + stored_issuers.insert(issuers[i]); + } + } + } + + memset(issuer_buf,0,sizeof(issuer_buf)); + mc_gState->m_TmpScript->SetSpecialParamValue(MC_ENT_SPRM_ISSUER,issuer_buf,1); + if(new_issue) + { + if((err != MC_ERR_NOERROR) || !accept) + { + mc_gState->m_Permissions->RollBackToCheckPoint(); + } + } + if(err) + { + reason="Cannot update permission database for issued asset"; + return false; + } + } + + const unsigned char *special_script; + size_t special_script_size=0; + special_script=mc_gState->m_TmpScript->GetData(0,&special_script_size); + txid=tx.GetHash(); + if(new_issue) + { + err=mc_gState->m_Assets->InsertAsset(&txid,offset,total,asset_name,multiple,details_script,details_script_size,special_script,special_script_size,update_mempool); + } + else + { + err=mc_gState->m_Assets->InsertAssetFollowOn(&txid,offset,total,details_script,details_script_size,special_script,special_script_size,entity.GetTxID(),update_mempool); + } + + if(err) + { + reason="Asset issue script rejected - could not insert new asset to database"; + if(err == MC_ERR_FOUND) + { + reason="Asset issue script rejected - asset with such name/asset-ref/txid already exists"; + if(mc_gState->m_Assets->FindEntityByTxID(&entity,(unsigned char*)&txid) == 0) + { + if(strlen(asset_name) == 0) + { + mc_gState->m_Assets->FindEntityByName(&entity,asset_name); + } + } + + LogPrint("mchn","Asset already exists. TxID: %s, AssetRef: %d-%d-%d, Name: %s\n", + tx.GetHash().GetHex().c_str(), + mc_gState->m_Assets->m_Block+1,offset,(int)(*((unsigned char*)&txid+31))+256*(int)(*((unsigned char*)&txid+30)), + entity.GetName()); + } + return false; + } + + + if(update_mempool) + { + if(offset>=0) + { + if(mc_gState->m_Assets->FindEntityByTxID(&this_entity,(unsigned char*)&txid)) + { + if(new_issue) + { + LogPrint("mchn","New asset. TxID: %s, AssetRef: %d-%d-%d, Name: %s\n", + tx.GetHash().GetHex().c_str(), + mc_gState->m_Assets->m_Block+1,offset,(int)(*((unsigned char*)&txid+0))+256*(int)(*((unsigned char*)&txid+1)), + this_entity.GetName()); + } + else + { + uint256 otxid; + memcpy(&otxid,entity.GetTxID(),32); + LogPrint("mchn","Follow-on issue. TxID: %s, Original issue txid: %s\n", + tx.GetHash().GetHex().c_str(),otxid.GetHex().c_str()); + } + } + else + { + reason="Asset issue script rejected - could not insert new asset to database"; + return false; + } + } + } + + return true; +} + +/** + * Used only for protocols < 10006 + */ + +bool AcceptPermissionsAndCheckForDust(const CTransaction &tx,bool accept,string& reason) +{ + uint32_t type,from,to,timestamp,flags; + int receive_required; + bool is_op_return; + bool is_pure_permission; + bool seed_node_involved; + bool reject; + bool sighashall_found; + int pass; + + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return true; + } + + sighashall_found=false; + + std::vector admin_outputs; + admin_outputs.resize(tx.vin.size()); + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + const CScript& script2 = tx.vin[i].scriptSig; + CScript::const_iterator pc2 = script2.begin(); + + mc_gState->m_TmpScript1->Clear(); + mc_gState->m_TmpScript1->SetScript((unsigned char*)(&pc2[0]),(size_t)(script2.end()-pc2),MC_SCR_TYPE_SCRIPTSIG); + + admin_outputs[i]=tx.vout.size(); + if(tx.IsCoinBase()) + { + admin_outputs[i]=-1; + } + else + { + bool is_pay_to_pubkeyhash=false; + if(mc_gState->m_TmpScript1->GetNumElements() == 2) + { + size_t elem_size; + const unsigned char *elem; + unsigned char hash_type; + + elem = mc_gState->m_TmpScript1->GetData(0,&elem_size); + + if(elem_size > 1) + { + is_pay_to_pubkeyhash=true; + hash_type=elem[elem_size-1] & 0x1f; + + if(hash_type == SIGHASH_ALL) + { + admin_outputs[i]=-1; + sighashall_found=true; + } + else + { + if(hash_type == SIGHASH_SINGLE) + { + admin_outputs[i]=i; + } + else + { + reason="Permission script rejected - invalid signature hash type"; + // printf("%s\n",reason.c_str()); + return false; + } + } + } + } + if(!is_pay_to_pubkeyhash) + { + int first_sig=1; + int this_sig; + if(mc_gState->m_TmpScript1->GetNumElements() == 1) // pay to pubkey + { + first_sig=0; + } + this_sig=first_sig; + while(this_sigm_TmpScript1->GetNumElements()) + { + size_t elem_size; + const unsigned char *elem; + unsigned char hash_type; + + elem = mc_gState->m_TmpScript1->GetData(this_sig,&elem_size); + if(elem_size > 1) + { + hash_type=elem[elem_size-1] & 0x1f; + + if(hash_type == SIGHASH_ALL) + { + sighashall_found=true; + } + this_sig++; + } + else // First element of redeemScript + { + this_sig=mc_gState->m_TmpScript1->GetNumElements(); + } + } + } + } + } + + reject=false; + seed_node_involved=false; + mc_gState->m_Permissions->SetCheckPoint(); + + for(pass=0;pass<3;pass++) + { + for (unsigned int j = 0; j < tx.vout.size(); j++) + { + mc_gState->m_TmpScript->Clear(); + + const CScript& script1 = tx.vout[j].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + CTxDestination addressRet; + + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + is_op_return=false; + if(mc_gState->m_TmpScript->IsOpReturnScript()) + { + is_op_return=true; + if(!sighashall_found) + { + if(!tx.IsCoinBase()) + { + reason="Tx with metadata should have at least one SIGHASH_ALL output"; + reject=true; + goto exitlbl; + } + } + } + else + { + txnouttype typeRet; + int nRequiredRet; + vector addressRets; + + if(!ExtractDestinations(script1,typeRet,addressRets,nRequiredRet)) + { + reason="Script rejected - wrong destination type"; + reject=true; + goto exitlbl; + } + + if((typeRet == TX_MULTISIG) && (mc_gState->m_NetworkParams->GetInt64Param("allowmultisigoutputs") == 0)) + { + reason="Script rejected - multisig is not allowed"; + reject=true; + goto exitlbl; + } + + if((typeRet == TX_SCRIPTHASH) && (mc_gState->m_NetworkParams->GetInt64Param("allowp2shoutputs") == 0)) + { + reason="Script rejected - P2SH is not allowed"; + reject=true; + goto exitlbl; + } + + receive_required=addressRets.size(); + if(typeRet == TX_MULTISIG) + { + receive_required-=(pc1[0]-0x50); + receive_required+=1; + if(receive_required>(int)addressRets.size()) + { + receive_required=addressRets.size(); + } + } + + is_pure_permission=false; + if(mc_gState->m_TmpScript->GetNumElements()) + { + is_pure_permission=true; + } + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + mc_gState->m_TmpScript->SetElement(e); + if(mc_gState->m_TmpScript->GetPermission(&type,&from,&to,×tamp) == 0) + { + if(mc_gState->m_Features->FixedGrantsInTheSameTx()) + { + switch(pass) + { + case 0: + type &= ~( MC_PTP_ACTIVATE | MC_PTP_ADMIN ); + break; + case 1: + type &= ( MC_PTP_ACTIVATE | MC_PTP_ADMIN ); + break; + case 2: + type=0; + break; + } + } + else + { + if(pass) + { + type=0; + } + } + if(type) + { + CKeyID *lpKeyID=boost::get (&addressRets[0]); + CScriptID *lpScriptID=boost::get (&addressRets[0]); + if(((lpKeyID == NULL) && (lpScriptID == NULL)) || (addressRets.size() > 1)) + { + reason="Permission script rejected - wrong destination type"; + reject=true; + goto exitlbl; + } + if(lpScriptID) + { + if(type != MC_PTP_RECEIVE) + { + reason="Permission script rejected - only receive permission can be set for P2SH"; + reject=true; + goto exitlbl; + } + } + + CBitcoinAddress address; + unsigned char* ptr=NULL; + flags=MC_PFL_NONE; + if(lpKeyID != NULL) + { + address=CBitcoinAddress(*lpKeyID); + ptr=(unsigned char*)(lpKeyID); + if(type & MC_PTP_CONNECT) + { + if(mc_gState->m_pSeedNode) + { + CNode* seed_node; + seed_node=(CNode*)(mc_gState->m_pSeedNode); + + if(memcmp(ptr,seed_node->kAddrRemote.begin(),20) == 0) + { + seed_node_involved=true; + } + } + } + } + else + { + flags=MC_PFL_IS_SCRIPTHASH; + address=CBitcoinAddress(*lpScriptID); + ptr=(unsigned char*)(lpScriptID); + } + LogPrint("mchn","Found permission script in tx %s for %s - (%08x: %d - %d)\n", + tx.GetHash().GetHex().c_str(), + address.ToString().c_str(), + type, from, to); + + bool admin_found=false; + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + if((admin_outputs[i] < 0) || (admin_outputs[i] == (int)j)) + { + const CScript& script2 = tx.vin[i].scriptSig; + CScript::const_iterator pc2 = script2.begin(); + + mc_gState->m_TmpScript1->Clear(); + mc_gState->m_TmpScript1->SetScript((unsigned char*)(&pc2[0]),(size_t)(script2.end()-pc2),MC_SCR_TYPE_SCRIPTSIG); + + if(mc_gState->m_TmpScript1->GetNumElements() > 1) + { + size_t elem_size; + const unsigned char *elem; + + elem = mc_gState->m_TmpScript1->GetData(0,&elem_size); + if(elem_size > 1) // it is not multisig with one signature + { + elem = mc_gState->m_TmpScript1->GetData(1,&elem_size); + const unsigned char *pubkey_hash=(unsigned char *)Hash160(elem,elem+elem_size).begin(); + if(mc_gState->m_Permissions->SetPermission(NULL,ptr,type,pubkey_hash,from,to,timestamp,flags,1) == 0) + { + admin_found=true; + } + } + } + } + } + if(!admin_found) + { + reason="Inputs don't belong to valid admin"; + reject=true; + goto exitlbl; + } + } + } + else + { + is_pure_permission=false; + } + } + + if(tx.vout[j].nValue > 0) + { + is_pure_permission=false; + } + + + if( (pass == 2) && !is_op_return && !is_pure_permission ) + { + if (tx.vout[j].IsDust(::minRelayTxFee)) + { + if(!tx.IsCoinBase()) + { + reason="Transaction amount too small"; + reject=true; + goto exitlbl; + } + } + + for(int a=0;a<(int)addressRets.size();a++) + { + CKeyID *lpKeyID=boost::get (&addressRets[a]); + CScriptID *lpScriptID=boost::get (&addressRets[a]); + if((lpKeyID == NULL) && (lpScriptID == NULL)) + { + reason="Script rejected - wrong destination type"; +// printf("%s\n",reason.c_str()); + reject=true; + goto exitlbl; + } + unsigned char* ptr=NULL; + if(lpKeyID != NULL) + { + ptr=(unsigned char*)(lpKeyID); + } + else + { + ptr=(unsigned char*)(lpScriptID); + } + + bool can_receive=mc_gState->m_Permissions->CanReceive(NULL,ptr); + + if(tx.IsCoinBase()) + { + can_receive |= mc_gState->m_Permissions->CanMine(NULL,ptr); + } + + if(can_receive) + { + receive_required--; + } + } + + if(receive_required>0) + { + reason="One of the outputs doesn't have receive permission"; + reject=true; + goto exitlbl; + } + } + } + } + } + +exitlbl: + + if(accept) + { + if(seed_node_involved) + { + CNode* seed_node; + seed_node=(CNode*)(mc_gState->m_pSeedNode); + + if(!mc_gState->m_Permissions->CanConnect(NULL,seed_node->kAddrRemote.begin())) + { + LogPrintf("mchn: Seed node lost connect permission \n"); + mc_gState->m_pSeedNode=NULL; + } + } + } + + if(!accept || reject) + { + mc_gState->m_Permissions->RollBackToCheckPoint(); + } + + return !reject; +} + + diff --git a/src/protocol/netprotocol.cpp b/src/protocol/netprotocol.cpp new file mode 100644 index 00000000..0474c134 --- /dev/null +++ b/src/protocol/netprotocol.cpp @@ -0,0 +1,148 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "protocol/netprotocol.h" + +#include "chainparams/chainparams.h" +#include "utils/util.h" +#include "utils/utilstrencodings.h" + +#ifndef WIN32 +# include +#endif + +static const char* ppszTypeName[] = +{ + "ERROR", + "tx", + "block", + "filtered block" +}; + +CMessageHeader::CMessageHeader() +{ + memcpy(pchMessageStart, Params().MessageStart(), MESSAGE_START_SIZE); + memset(pchCommand, 0, sizeof(pchCommand)); + nMessageSize = -1; + nChecksum = 0; +} + +CMessageHeader::CMessageHeader(const char* pszCommand, unsigned int nMessageSizeIn) +{ + memcpy(pchMessageStart, Params().MessageStart(), MESSAGE_START_SIZE); + memset(pchCommand, 0, sizeof(pchCommand)); + strncpy(pchCommand, pszCommand, COMMAND_SIZE); + nMessageSize = nMessageSizeIn; + nChecksum = 0; +} + +std::string CMessageHeader::GetCommand() const +{ + return std::string(pchCommand, pchCommand + strnlen(pchCommand, COMMAND_SIZE)); +} +/* MCHN START */ +bool CMessageHeader::IsValid(bool SkipMessageStartCheck) const +{ + // Check start string + if(!SkipMessageStartCheck) + { + if (memcmp(pchMessageStart, Params().MessageStart(), MESSAGE_START_SIZE) != 0) + return false; + } +/* MCHN END */ + // Check the command string for errors + for (const char* p1 = pchCommand; p1 < pchCommand + COMMAND_SIZE; p1++) + { + if (*p1 == 0) + { + // Must be all zeros after the first zero + for (; p1 < pchCommand + COMMAND_SIZE; p1++) + if (*p1 != 0) + return false; + } + else if (*p1 < ' ' || *p1 > 0x7E) + return false; + } + + // Message size + if (nMessageSize > MAX_SIZE) + { + LogPrintf("CMessageHeader::IsValid() : (%s, %u bytes) nMessageSize > MAX_SIZE\n", GetCommand(), nMessageSize); + return false; + } + + return true; +} + + + +CAddress::CAddress() : CService() +{ + Init(); +} + +CAddress::CAddress(CService ipIn, uint64_t nServicesIn) : CService(ipIn) +{ + Init(); + nServices = nServicesIn; +} + +void CAddress::Init() +{ + nServices = NODE_NETWORK; + nTime = 100000000; + nLastTry = 0; +} + +CInv::CInv() +{ + type = 0; + hash = 0; +} + +CInv::CInv(int typeIn, const uint256& hashIn) +{ + type = typeIn; + hash = hashIn; +} + +CInv::CInv(const std::string& strType, const uint256& hashIn) +{ + unsigned int i; + for (i = 1; i < ARRAYLEN(ppszTypeName); i++) + { + if (strType == ppszTypeName[i]) + { + type = i; + break; + } + } + if (i == ARRAYLEN(ppszTypeName)) + throw std::out_of_range(strprintf("CInv::CInv(string, uint256) : unknown type '%s'", strType)); + hash = hashIn; +} + +bool operator<(const CInv& a, const CInv& b) +{ + return (a.type < b.type || (a.type == b.type && a.hash < b.hash)); +} + +bool CInv::IsKnownType() const +{ + return (type >= 1 && type < (int)ARRAYLEN(ppszTypeName)); +} + +const char* CInv::GetCommand() const +{ + if (!IsKnownType()) + throw std::out_of_range(strprintf("CInv::GetCommand() : type=%d unknown type", type)); + return ppszTypeName[type]; +} + +std::string CInv::ToString() const +{ + return strprintf("%s %s", GetCommand(), hash.ToString()); +} diff --git a/src/protocol/netprotocol.h b/src/protocol/netprotocol.h new file mode 100644 index 00000000..a123a994 --- /dev/null +++ b/src/protocol/netprotocol.h @@ -0,0 +1,153 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2013 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef __cplusplus +#error This header can only be compiled as C++. +#endif + +#ifndef BITCOIN_PROTOCOL_H +#define BITCOIN_PROTOCOL_H + +#include "net/netbase.h" +#include "utils/serialize.h" +#include "structs/uint256.h" +#include "version/bcversion.h" + +#include +#include + +#define MESSAGE_START_SIZE 4 + +/** Message header. + * (4) message start. + * (12) command. + * (4) size. + * (4) checksum. + */ +class CMessageHeader +{ +public: + CMessageHeader(); + CMessageHeader(const char* pszCommand, unsigned int nMessageSizeIn); + + std::string GetCommand() const; + bool IsValid(bool SkipMessageStartCheck) const; // MCHN + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) + { + READWRITE(FLATDATA(pchMessageStart)); + READWRITE(FLATDATA(pchCommand)); + READWRITE(nMessageSize); + READWRITE(nChecksum); + } + + // TODO: make private (improves encapsulation) +public: + enum { + COMMAND_SIZE = 12, + MESSAGE_SIZE_SIZE = sizeof(int), + CHECKSUM_SIZE = sizeof(int), + + MESSAGE_SIZE_OFFSET = MESSAGE_START_SIZE + COMMAND_SIZE, + CHECKSUM_OFFSET = MESSAGE_SIZE_OFFSET + MESSAGE_SIZE_SIZE, + HEADER_SIZE = MESSAGE_START_SIZE + COMMAND_SIZE + MESSAGE_SIZE_SIZE + CHECKSUM_SIZE + }; + char pchMessageStart[MESSAGE_START_SIZE]; + char pchCommand[COMMAND_SIZE]; + unsigned int nMessageSize; + unsigned int nChecksum; +}; + +/** nServices flags */ +enum { + NODE_NETWORK = (1 << 0), + + // Bits 24-31 are reserved for temporary experiments. Just pick a bit that + // isn't getting used, or one not being used much, and notify the + // bitcoin-development mailing list. Remember that service bits are just + // unauthenticated advertisements, so your code must be robust against + // collisions and other cases where nodes may be advertising a service they + // do not actually support. Other service bits should be allocated via the + // BIP process. +}; + +/** A CService with information about it as peer */ +class CAddress : public CService +{ +public: + CAddress(); + explicit CAddress(CService ipIn, uint64_t nServicesIn = NODE_NETWORK); + + void Init(); + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) + { + if (ser_action.ForRead()) + Init(); + if (nType & SER_DISK) + READWRITE(nVersion); + if ((nType & SER_DISK) || + (nVersion >= CADDR_TIME_VERSION && !(nType & SER_GETHASH))) + READWRITE(nTime); + READWRITE(nServices); + READWRITE(*(CService*)this); + } + + // TODO: make private (improves encapsulation) +public: + uint64_t nServices; + + // disk and network only + unsigned int nTime; + + // memory only + int64_t nLastTry; +}; + +/** inv message data */ +class CInv +{ +public: + CInv(); + CInv(int typeIn, const uint256& hashIn); + CInv(const std::string& strType, const uint256& hashIn); + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) + { + READWRITE(type); + READWRITE(hash); + } + + friend bool operator<(const CInv& a, const CInv& b); + + bool IsKnownType() const; + const char* GetCommand() const; + std::string ToString() const; + + // TODO: make private (improves encapsulation) +public: + int type; + uint256 hash; +}; + +enum { + MSG_TX = 1, + MSG_BLOCK, + // Nodes may always request a MSG_FILTERED_BLOCK in a getdata, however, + // MSG_FILTERED_BLOCK should not appear in any invs except as a part of getdata. + MSG_FILTERED_BLOCK, +}; + +#endif // BITCOIN_PROTOCOL_H diff --git a/src/rpc/rpcassets.cpp b/src/rpc/rpcassets.cpp new file mode 100644 index 00000000..bfaed050 --- /dev/null +++ b/src/rpc/rpcassets.cpp @@ -0,0 +1,2090 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + + +#include "rpc/rpcwallet.h" + +void mergeGenesisWithAssets(mc_Buffer *genesis_amounts, mc_Buffer *asset_amounts) +{ + if(mc_gState->m_Features->ShortTxIDAsAssetRef() == 0) + { + return; + } + + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + int64_t quantity; + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + + for(int a=0;aGetCount();a++) + { + memcpy(buf+MC_AST_SHORT_TXID_OFFSET,(unsigned char*)genesis_amounts->GetRow(a)+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + mc_SetABRefType(buf,MC_AST_ASSET_REF_TYPE_SHORT_TXID); + quantity=mc_GetABQuantity(genesis_amounts->GetRow(a)); + int row=asset_amounts->Seek(buf); + if(row >= 0) + { + int64_t last=mc_GetABQuantity(asset_amounts->GetRow(row)); + quantity+=last; + mc_SetABQuantity(asset_amounts->GetRow(row),quantity); + } + else + { + mc_SetABQuantity(buf,quantity); + asset_amounts->Add(buf); + } + } + + genesis_amounts->Clear(); +} + +Value issuefromcmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 4) + throw runtime_error("Help message not found\n"); + + CBitcoinAddress address(params[1].get_str()); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + + // Amount + CAmount nAmount = mc_gState->m_NetworkParams->GetInt64Param("minimumperoutput"); + if (params.size() > 5 && params[5].type() != null_type) + { + nAmount = AmountFromValue(params[5]); + if(nAmount < 0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid amount"); + } + // Wallet comments + CWalletTx wtx; + + mc_Script *lpScript; + lpScript=new mc_Script; + + int64_t quantity; + int multiple; + double dQuantity; + double dUnit; + + dQuantity=0.; + dUnit=1.; + if (params.size() > 3 && params[3].type() != null_type) + { + dQuantity=params[3].get_real(); + } + + if (params.size() > 4 && params[4].type() != null_type) + { + dUnit=params[4].get_real(); + } + + if(dQuantity<0.) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid quantity. Should be positive."); + if(dUnit<=0.) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid smallest unit. Valid Range [0.00000001 - 1]."); + if(dUnit>1.) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid smallest unit. Valid Range [0.00000001 - 1]."); + + multiple=(int)((1 + 0.1*dUnit)/dUnit); + if(fabs((double)multiple*dUnit-1)>0.0001) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid smallest unit. 1 should be divisible by this number."); + + quantity=(int64_t)(dQuantity*multiple+0.1); + if(quantity<0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid quantity or smallest unit. "); + if( (quantity == 0) && (mc_gState->m_Features->FollowOnIssues() == 0) ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid asset quantity"); + } + if(multiple<=0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid quantity or smallest unit."); + + int64_t quantity_to_check=(int64_t)((dQuantity+0.1*dUnit)/dUnit); + double dDelta; + + dDelta=fabs(dQuantity/dUnit-quantity); + quantity_to_check=quantity; + while(quantity_to_check > 1) + { + quantity_to_check /= 2; + dDelta /= 2.; + } + + if(dDelta>1.e-14) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Quantity should be divisible by smallest unit."); + + } + + + + if(!AddressCanReceive(address.Get())) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Destination address doesn't have receive permission"); + } + lpScript->SetAssetGenesis(quantity); + + mc_Script *lpDetailsScript; + lpDetailsScript=NULL; + + mc_Script *lpDetails; + lpDetails=new mc_Script; + lpDetails->AddElement(); + + int ret,type; + string asset_name=""; + bool is_open=false; + bool name_is_found=false; + + if (params.size() > 2 && params[2].type() != null_type)// && !params[2].get_str().empty()) + { + if(params[2].type() == obj_type) + { + if(mc_gState->m_Features->FollowOnIssues()) + { + Object objSpecialParams = params[2].get_obj(); + BOOST_FOREACH(const Pair& s, objSpecialParams) + { + if(s.name_ == "name") + { + if(!name_is_found) + { + asset_name=s.value_.get_str().c_str(); + name_is_found=true; + } + } + if(s.name_ == "open") + { + if(s.value_.type() == bool_type) + { + is_open=s.value_.get_bool(); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid value for 'open' field, should be boolean"); + } + } + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Asset name should be string"); + } + } + else + { + if(params[2].get_str().size()) + { + asset_name=params[2].get_str(); + } + } + } + + if(mc_gState->m_Features->Streams()) + { + if(asset_name == "*") + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid asset name"); + } + } + + unsigned char buf_a[MC_AST_ASSET_REF_SIZE]; + if(CoinSparkAssetRefDecode(buf_a,asset_name.c_str(),asset_name.size())) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid asset name"); + } + + if(asset_name.size()) + { + ret=ParseAssetKey(asset_name.c_str(),NULL,NULL,NULL,NULL,&type,MC_ENT_TYPE_ANY); + if(ret != -3) + { + if(type == 3) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Asset or stream with this name already exists"); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid asset name"); + } + } + } + + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + if(asset_name.size()) + { + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_NAME,(const unsigned char*)(asset_name.c_str()),asset_name.size());//+1); + } + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_ASSET_MULTIPLE,(unsigned char*)&multiple,4); + } + + if(is_open) + { + unsigned char b=1; + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_FOLLOW_ONS,&b,1); + } + + if (params.size() > 6) + { + if(params[6].type() == obj_type) + { + Object objParams = params[6].get_obj(); + BOOST_FOREACH(const Pair& s, objParams) + { + lpDetails->SetParamValue(s.name_.c_str(),s.name_.size(),(unsigned char*)s.value_.get_str().c_str(),s.value_.get_str().size()); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid custom fields, expecting object"); + } + } + int err; + size_t bytes; + const unsigned char *script; + size_t elem_size; + const unsigned char *elem; + CScript scriptOpReturn=CScript(); + + script=lpDetails->GetData(0,&bytes); +// if(bytes > 0) + { + lpDetailsScript=new mc_Script; + + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_ASSET,0,script,bytes); + + elem = lpDetailsScript->GetData(0,&elem_size); + scriptOpReturn << vector(elem, elem + elem_size) << OP_DROP << OP_RETURN; + } + else + { + err=lpDetailsScript->SetAssetDetails(asset_name.c_str(),multiple,script,bytes); + if(err) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid custom fields or asset name, too long"); + } + + elem = lpDetailsScript->GetData(0,&elem_size); + scriptOpReturn << OP_RETURN << vector(elem, elem + elem_size); + } + } + + + + vector addresses; + addresses.push_back(address.Get()); + + vector fromaddresses; + + if(params[0].get_str() != "*") + { + fromaddresses=ParseAddresses(params[0].get_str(),false,false); + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + if( (IsMine(*pwalletMain, fromaddresses[0]) & ISMINE_SPENDABLE) != ISMINE_SPENDABLE ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key for from-address is not found in this wallet"); + } + + set thisFromAddresses; + + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + thisFromAddresses.insert(fromaddress); + } + + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_ISSUE,&thisFromAddresses)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "from-address doesn't have issue permission"); + } + } + else + { + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_ISSUE)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "This wallet doesn't have keys with issue permission"); + } + } + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + SendMoneyToSeveralAddresses(addresses, nAmount, wtx, lpScript, scriptOpReturn,fromaddresses); + + if(lpDetailsScript) + { + delete lpDetailsScript; + } + delete lpDetails; + delete lpScript; + + return wtx.GetHash().GetHex(); +} + + + +Value issuecmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 3) + throw runtime_error("Help message not found\n"); + + Array ext_params; + ext_params.push_back("*"); + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + } + + return issuefromcmd(ext_params,fHelp); +} + +Value issuemorefromcmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 4 || params.size() > 6) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_Features->FollowOnIssues() == 0 ) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + + CBitcoinAddress address(params[1].get_str()); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + + // Amount + CAmount nAmount = mc_gState->m_NetworkParams->GetInt64Param("minimumperoutput"); + if (params.size() > 4 && params[4].type() != null_type) + { + nAmount = AmountFromValue(params[4]); + if(nAmount < 0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid amount"); + } + + mc_Script *lpScript; + lpScript=new mc_Script; + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + int multiple=1; + mc_EntityDetails entity; + + if (params.size() > 2 && params[2].type() != null_type && !params[2].get_str().empty()) + { + ParseEntityIdentifier(params[2],&entity, MC_ENT_TYPE_ASSET); + memcpy(buf,entity.GetFullRef(),MC_AST_ASSET_FULLREF_SIZE); + if(mc_gState->m_Features->ShortTxIDAsAssetRef() == 0) + { + if(entity.IsUnconfirmedGenesis()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Unconfirmed asset: ")+params[2].get_str()); + } + } + multiple=entity.GetAssetMultiple(); + } + else + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid asset reference"); + } + + Value raw_qty=params[3]; + + int64_t quantity = (int64_t)(raw_qty.get_real() * multiple + 0.499999); + if(quantity<0) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid asset quantity"); + } + if( (quantity == 0) && (mc_gState->m_Features->FollowOnIssues() == 0) ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid asset quantity"); + } + + + mc_SetABQuantity(buf,quantity); + + mc_Buffer *lpBuffer; + lpBuffer=new mc_Buffer; + + mc_InitABufferDefault(lpBuffer); + + lpBuffer->Add(buf); + + lpScript->SetAssetQuantities(lpBuffer,MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON); + + delete lpBuffer; + + + + // Wallet comments + CWalletTx wtx; + + + + if(!AddressCanReceive(address.Get())) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Destination address doesn't have receive permission"); + } + + + mc_Script *lpDetailsScript; + lpDetailsScript=NULL; + + mc_Script *lpDetails; + lpDetails=new mc_Script; + lpDetails->AddElement(); + + if (params.size() > 5) + { + if(params[5].type() == obj_type) + { + Object objParams = params[5].get_obj(); + BOOST_FOREACH(const Pair& s, objParams) + { + lpDetails->SetParamValue(s.name_.c_str(),s.name_.size(),(unsigned char*)s.value_.get_str().c_str(),s.value_.get_str().size()); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid extra-params, expecting object"); + } + } + + size_t bytes; + const unsigned char *script; + size_t elem_size; + const unsigned char *elem; + CScript scriptOpReturn=CScript(); + + script=lpDetails->GetData(0,&bytes); + if(bytes > 0) + { +// mc_DumpSize("script",script,bytes,bytes); + lpDetailsScript=new mc_Script; + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + lpDetailsScript->SetEntity(entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET); + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_ASSET,1,script,bytes); + + elem = lpDetailsScript->GetData(0,&elem_size); + scriptOpReturn << vector(elem, elem + elem_size) << OP_DROP; + elem = lpDetailsScript->GetData(1,&elem_size); + scriptOpReturn << vector(elem, elem + elem_size) << OP_DROP << OP_RETURN; + } + else + { + lpDetailsScript->SetGeneralDetails(script,bytes); + elem = lpDetailsScript->GetData(0,&elem_size); + scriptOpReturn << OP_RETURN << vector(elem, elem + elem_size); + } + } + + + + vector addresses; + addresses.push_back(address.Get()); + + vector fromaddresses; + + if(params[0].get_str() != "*") + { + fromaddresses=ParseAddresses(params[0].get_str(),false,false); + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + if( (IsMine(*pwalletMain, fromaddresses[0]) & ISMINE_SPENDABLE) != ISMINE_SPENDABLE ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key for from-address is not found in this wallet"); + } + } + else + { +/* + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_ISSUE)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "This wallet doesn't have keys with issue permission"); + } + */ + } + + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,buf)) + { + if(entity.AllowedFollowOns()) + { + if(fromaddresses.size() == 1) + { + CKeyID *lpKeyID=boost::get (&fromaddresses[0]); + if(lpKeyID != NULL) + { + if(mc_gState->m_Permissions->CanIssue(entity.GetTxID(),(unsigned char*)(lpKeyID)) == 0) + { + throw JSONRPCError(RPC_INTERNAL_ERROR, "Issuing more units for this asset is not allowed from this address"); + } + } + else + { + throw JSONRPCError(RPC_INTERNAL_ERROR, "Issuing more units is allowed only from P2PKH addresses"); + } + } + else + { + bool issuer_found=false; + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, pwalletMain->mapAddressBook) + { + const CBitcoinAddress& address = item.first; + CKeyID keyID; + + if(address.GetKeyID(keyID)) + { + if(mc_gState->m_Permissions->CanIssue(entity.GetTxID(),(unsigned char*)(&keyID))) + { + issuer_found=true; + } + } + } + if(!issuer_found) + { + throw JSONRPCError(RPC_INTERNAL_ERROR, "Issuing more units for this asset is not allowed from this wallet"); + } + } + } + else + { + throw JSONRPCError(RPC_INTERNAL_ERROR, "Issuing more units not allowed for this asset: "+params[2].get_str()); + } + } + else + { + throw JSONRPCError(RPC_INTERNAL_ERROR, "Asset not found"); + } + + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + SendMoneyToSeveralAddresses(addresses, nAmount, wtx, lpScript, scriptOpReturn,fromaddresses); + + if(lpDetailsScript) + { + delete lpDetailsScript; + } + delete lpDetails; + delete lpScript; + + return wtx.GetHash().GetHex(); +} + +Value issuemorecmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 3) + throw runtime_error("Help message not found\n"); + + Array ext_params; + ext_params.push_back("*"); + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + } + + return issuemorefromcmd(ext_params,fHelp); +} + +Value getmultibalances(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 5) + throw runtime_error("Help message not found\n"); + + isminefilter filter = ISMINE_SPENDABLE; + + Object balances; + + bool fUnlockedOnly=true; + bool fIncludeWatchOnly=false; + int nMinDepth = 1; + + if(params.size() > 2) + { + nMinDepth = params[2].get_int(); + } + + if(params.size() > 3) + { + if(params[3].get_bool()) + { + fIncludeWatchOnly=true; + } + } + + if(params.size() > 4) + { + if(params[4].get_bool()) + { + fUnlockedOnly=false; + } + } + + if(fIncludeWatchOnly) + { + filter = filter | ISMINE_WATCH_ONLY; + } + + set setAddresses; + set setAddressesWithBalances; + if(params.size() > 0) + { + if( (params[0].type() != str_type) || (params[0].get_str() != "*") ) + { + setAddresses=ParseAddresses(params[0],filter); + if(setAddresses.size() == 0) + { + return balances; + } + } + } + + vector inputStrings; + + if (params.size() > 1 && params[1].type() != null_type && ((params[1].type() != str_type) || (params[1].get_str() !="*" ) ) ) + { + if(params[1].type() == str_type) + { + inputStrings.push_back(params[1].get_str()); + if(params[1].get_str() == "") + { + return balances; + } + } + else + { + inputStrings=ParseStringList(params[1]); + if(inputStrings.size() == 0) + { + return balances; + } + } + } + + set setAssets; + if(inputStrings.size()) + { + for(int is=0;is<(int)inputStrings.size();is++) + { + mc_EntityDetails entity; + ParseEntityIdentifier(inputStrings[is],&entity, MC_ENT_TYPE_ASSET); + uint256 hash=*(uint256*)entity.GetTxID(); + if (setAssets.count(hash)) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid parameter, duplicate asset: ")+inputStrings[is]); + } + setAssets.insert(hash); + } + } + + mc_Buffer *asset_amounts; + mc_Buffer *addresstxid_amounts; + unsigned char buf[80+MC_AST_ASSET_QUANTITY_SIZE]; + unsigned char totbuf[80+MC_AST_ASSET_QUANTITY_SIZE]; + int64_t quantity; + int row; + unsigned char *ptr; + + assert(pwalletMain != NULL); + + + { + LOCK(cs_main); + + mc_Script *lpScript; + lpScript=new mc_Script; + + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + asset_amounts->Clear(); + + addresstxid_amounts=new mc_Buffer; + addresstxid_amounts->Initialize(80,80+MC_AST_ASSET_QUANTITY_SIZE,MC_BUF_MODE_MAP); + addresstxid_amounts->Clear(); + + memset(totbuf,0,80+MC_AST_ASSET_QUANTITY_SIZE); + addresstxid_amounts->Add(totbuf,totbuf+80); + + + vector vecOutputs; + pwalletMain->AvailableCoins(vecOutputs, false, NULL, fUnlockedOnly,true); + BOOST_FOREACH(const COutput& out, vecOutputs) + { + if(!out.IsTrusted()) + { + if (out.nDepth < nMinDepth) + { + continue; + } + } + + CTxOut txout; + uint256 hash=out.GetHashAndTxOut(txout); + + string str_addr; + CTxDestination address; + if (!ExtractDestination(txout.scriptPubKey, address)) + { + continue; + } + + str_addr=CBitcoinAddress(address).ToString(); + if ( (setAddresses.size()>0) && (setAddresses.count(str_addr) == 0) ) + { + continue; + } + isminetype fIsMine=pwalletMain->IsMine(txout); + + if (!(fIsMine & filter)) + { + continue; + } + + memset(totbuf,0,80+MC_AST_ASSET_QUANTITY_SIZE); + memset(buf,0,80+MC_AST_ASSET_QUANTITY_SIZE); + memcpy(buf,str_addr.c_str(),str_addr.size()); + + quantity=txout.nValue; + if(quantity > 0) + { + quantity+=mc_GetLE(addresstxid_amounts->GetRow(0)+80,MC_AST_ASSET_QUANTITY_SIZE); + mc_PutLE(addresstxid_amounts->GetRow(0)+80,&quantity,MC_AST_ASSET_QUANTITY_SIZE); + row=addresstxid_amounts->Seek(buf); + quantity=txout.nValue; + if(row >= 0) + { + quantity+=mc_GetLE(addresstxid_amounts->GetRow(row)+80,MC_AST_ASSET_QUANTITY_SIZE); + mc_PutLE(addresstxid_amounts->GetRow(row)+80,&quantity,MC_AST_ASSET_QUANTITY_SIZE); + } + else + { + mc_PutLE(buf+80,&quantity,MC_AST_ASSET_QUANTITY_SIZE); + addresstxid_amounts->Add(buf,buf+80); + } + } + asset_amounts->Clear(); + if(CreateAssetBalanceList(txout,asset_amounts,lpScript)) + { + for(int a=0;aGetCount();a++) + { + const unsigned char *txid; + txid=NULL; + ptr=(unsigned char *)asset_amounts->GetRow(a); + if(mc_GetABRefType(ptr) == MC_AST_ASSET_REF_TYPE_GENESIS) +// if(mc_GetLE(ptr,4) == 0) + { +// hash=out.tx->GetHash(); + txid=(unsigned char*)&hash; + } + else + { + mc_EntityDetails entity; + + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ptr)) + { + txid=entity.GetTxID(); + } + } + if(txid) + { + if(setAssets.size()) + { +// hash=*(uint256*)txid; + if(setAssets.count(*(uint256*)txid) == 0) + { + txid=NULL; + } + } + } + + if(txid) + { + quantity=mc_GetABQuantity(ptr); + memcpy(totbuf+48,txid,32); + row=addresstxid_amounts->Seek(totbuf); + if(row >= 0) + { + quantity+=mc_GetLE(addresstxid_amounts->GetRow(row)+80,MC_AST_ASSET_QUANTITY_SIZE); + mc_PutLE(addresstxid_amounts->GetRow(row)+80,&quantity,MC_AST_ASSET_QUANTITY_SIZE); + } + else + { + mc_PutLE(totbuf+80,&quantity,MC_AST_ASSET_QUANTITY_SIZE); + addresstxid_amounts->Add(totbuf,totbuf+80); + } + + quantity=mc_GetABQuantity(ptr); + memcpy(buf+48,txid,32); + row=addresstxid_amounts->Seek(buf); + if(row >= 0) + { + quantity+=mc_GetLE(addresstxid_amounts->GetRow(row)+80,MC_AST_ASSET_QUANTITY_SIZE); + mc_PutLE(addresstxid_amounts->GetRow(row)+80,&quantity,MC_AST_ASSET_QUANTITY_SIZE); + } + else + { + mc_PutLE(buf+80,&quantity,MC_AST_ASSET_QUANTITY_SIZE); + addresstxid_amounts->Add(buf,buf+80); + } + } + } + } + } + + bool take_total=false; + bool have_addr=true; + + memset(totbuf,0,80+MC_AST_ASSET_QUANTITY_SIZE); + while(have_addr) + { + memset(buf,0,80+MC_AST_ASSET_QUANTITY_SIZE); + Array addr_balances; + set setAssetsWithBalances; + int64_t btc=0; + for(int i=0;iGetCount();i++) + { + bool take_it=false; + ptr=addresstxid_amounts->GetRow(i); + if(ptr[47] == 0) + { + if(take_total || (memcmp(ptr,totbuf,48) != 0)) + { + if(take_total || (memcmp(buf,totbuf,48) == 0)) + { + take_it=true; + if(!take_total) + { + memcpy(buf,ptr,48); + } + } + else + { + if(memcmp(ptr,buf,48) == 0) + { + take_it=true; + } + } + } + } + if(take_it) + { + quantity=mc_GetLE(ptr+80,MC_AST_ASSET_QUANTITY_SIZE); + if(memcmp(ptr+48,totbuf+48,32) == 0) + { + btc=quantity; + } + else + { + Object asset_entry; + asset_entry=AssetEntry(ptr+48,quantity,1); + addr_balances.push_back(asset_entry); + if(setAssets.size()) + { + uint256 issue_txid=*(uint256*)(ptr+48); + setAssetsWithBalances.insert(issue_txid); + } + } + ptr[47]=0x01; + } + } + if(setAssets.size()) + { + BOOST_FOREACH(const uint256& rem_asset, setAssets) + { + if(setAssetsWithBalances.count(rem_asset) == 0) + { + Object asset_entry; + asset_entry=AssetEntry((unsigned char*)&rem_asset,0,1); + addr_balances.push_back(asset_entry); + } + } + } + if((mc_gState->m_NetworkParams->GetInt64Param("initialblockreward") != 0) || (mc_gState->m_NetworkParams->GetInt64Param("firstblockreward") > 0)) + { + Object asset_entry; + asset_entry=AssetEntry(NULL,btc,1); + addr_balances.push_back(asset_entry); + } + + string out_addr=""; + if(take_total) + { + have_addr=false; + out_addr="total"; + if(setAddresses.size()) + { + BOOST_FOREACH(const string& rem_addr, setAddresses) + { + if(setAddressesWithBalances.count(rem_addr) == 0) + { + Array empty_balances; + if(setAssets.size()) + { + BOOST_FOREACH(const uint256& rem_asset, setAssets) + { + Object asset_entry; + asset_entry=AssetEntry((unsigned char*)&rem_asset,0,1); + empty_balances.push_back(asset_entry); + } + } + balances.push_back(Pair(rem_addr, empty_balances)); + } + } + } + } + else + { + if(memcmp(buf,totbuf,48) == 0) + { + take_total=true; + } + else + { + out_addr=string((char*)buf); + } + } + if(out_addr.size()) + { + setAddressesWithBalances.insert(out_addr); + balances.push_back(Pair(out_addr, addr_balances)); + } + } + + + delete addresstxid_amounts; + delete asset_amounts; + delete lpScript; + } + + return balances; +} + +Value getaddressbalances(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 3) + throw runtime_error("Help message not found\n"); + + vector fromaddresses; + fromaddresses=ParseAddresses(params[0].get_str(),false,true); + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + isminefilter filter = ISMINE_SPENDABLE; + + filter = filter | ISMINE_WATCH_ONLY; + + bool fUnlockedOnly=true; + + if(params.size() > 2) + if(params[2].get_bool()) + fUnlockedOnly=false; + + + + set setAddress; + + + bool check_account=true; + + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + setAddress.insert(fromaddress); + } + + int nMinDepth = 1; + if (params.size() > 1) + nMinDepth = params[1].get_int(); + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + asset_amounts->Clear(); + + mc_Buffer *genesis_amounts; + genesis_amounts=new mc_Buffer; + genesis_amounts->Initialize(32,32+MC_AST_ASSET_QUANTITY_SIZE,MC_BUF_MODE_MAP); + genesis_amounts->Clear(); + + mc_Script *lpScript; + lpScript=new mc_Script; + + int last_size=0; + Array assets; + CAmount totalBTC=0; + vector vecOutputs; + assert(pwalletMain != NULL); + + uint160 addr=0; + + if(fromaddresses.size() == 1) + { + CTxDestination addressRet=fromaddresses[0]; + const CKeyID *lpKeyID=boost::get (&addressRet); + const CScriptID *lpScriptID=boost::get (&addressRet); + if(lpKeyID) + { + addr=*(uint160*)lpKeyID; + } + if(lpScriptID) + { + addr=*(uint160*)lpScriptID; + } + } + + pwalletMain->AvailableCoins(vecOutputs, false, NULL, fUnlockedOnly,true,addr); + BOOST_FOREACH(const COutput& out, vecOutputs) { + +/* + if(out.nDepth == 0) + { + if(!out.tx->IsTrusted(out.nDepth)) + { + continue; + } + } + else + { + */ + if(!out.IsTrusted()) + { + if (out.nDepth < nMinDepth) + { + continue; + } + } +/* + } +*/ + CTxOut txout; + uint256 hash=out.GetHashAndTxOut(txout); + if(check_account) + { + CTxDestination address; + if (!ExtractDestination(txout.scriptPubKey, address)) + continue; + + if (!setAddress.count(address)) + continue; + } + + isminetype fIsMine=pwalletMain->IsMine(txout); + if (!(fIsMine & filter)) + { + continue; + } + + + + CAmount nValue = txout.nValue; + totalBTC+=nValue; + if(CreateAssetBalanceList(txout,asset_amounts,lpScript)) + { + unsigned char *ptr; + string assetref=""; + int64_t quantity; + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + int n; + bool is_genesis; + + n=asset_amounts->GetCount(); + + for(int a=last_size;aGetRow(a); + is_genesis=false; + if(mc_GetABRefType(ptr) == MC_AST_ASSET_REF_TYPE_GENESIS) + { + mc_EntityDetails entity; + quantity=mc_GetABQuantity(asset_amounts->GetRow(a)); + if(mc_gState->m_Assets->FindEntityByTxID(&entity,(unsigned char*)&hash)) + { + if((entity.IsUnconfirmedGenesis() != 0) && (mc_gState->m_Features->ShortTxIDAsAssetRef() == 0) ) + { + is_genesis=true; + } + else + { + ptr=(unsigned char *)entity.GetFullRef(); + memcpy(buf,ptr,MC_AST_ASSET_FULLREF_SIZE); + int row=asset_amounts->Seek(buf); + if(row >= 0) + { + int64_t last=mc_GetABQuantity(asset_amounts->GetRow(row)); + quantity+=last; + mc_SetABQuantity(asset_amounts->GetRow(row),quantity); + } + else + { + mc_SetABQuantity(buf,quantity); + asset_amounts->Add(buf); + } + } + } + + if(is_genesis) + { + int row=genesis_amounts->Seek(&hash); + if(row >= 0) + { + int64_t last=mc_GetLE(genesis_amounts->GetRow(row)+32,MC_AST_ASSET_QUANTITY_SIZE); + quantity+=last; + mc_PutLE(genesis_amounts->GetRow(row)+32,&quantity,MC_AST_ASSET_QUANTITY_SIZE); + } + else + { + mc_SetABQuantity(buf,quantity); + genesis_amounts->Add(&hash,buf+MC_AST_ASSET_QUANTITY_OFFSET); + } + } + } + } + last_size=asset_amounts->GetCount(); + } + } + + + unsigned char *ptr; + string assetref=""; + + mergeGenesisWithAssets(genesis_amounts, asset_amounts); + + for(int a=0;aGetCount();a++) + { + Object asset_entry; + ptr=(unsigned char *)asset_amounts->GetRow(a); + + mc_EntityDetails entity; + const unsigned char *txid; + + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ptr)) + { + txid=entity.GetTxID(); + asset_entry=AssetEntry(txid,mc_GetABQuantity(ptr),1); + assets.push_back(asset_entry); + } + } + + for(int a=0;aGetCount();a++) + { + Object asset_entry; + ptr=(unsigned char *)genesis_amounts->GetRow(a); + + asset_entry=AssetEntry(ptr,mc_GetLE(ptr+32,MC_AST_ASSET_QUANTITY_SIZE),1); + assets.push_back(asset_entry); + } + + if((mc_gState->m_NetworkParams->GetInt64Param("initialblockreward") != 0) || (mc_gState->m_NetworkParams->GetInt64Param("firstblockreward") > 0)) + { + Object asset_entry; + asset_entry=AssetEntry(NULL,totalBTC,1); + assets.push_back(asset_entry); + } + + + delete lpScript; + delete asset_amounts; + delete genesis_amounts; + +/* MCHN END */ + return assets; +} + + + +Value getassetbalances(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 4) + throw runtime_error("Help message not found\n"); + + bool check_account=false; + + isminefilter filter = ISMINE_SPENDABLE; + if(params.size() > 2) + if(params[2].get_bool()) + filter = filter | ISMINE_WATCH_ONLY; + + bool fUnlockedOnly=true; + + if(params.size() > 3) + if(params[3].get_bool()) + fUnlockedOnly=false; + + set setAddress; + + if (params.size() > 0) + { + if (params[0].get_str() != "*") + { + if (params[0].get_str() != "") + { + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need getassetbalances, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + } + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, pwalletMain->mapAddressBook) + { + const CBitcoinAddress& address = item.first; + const string& strName = item.second.name; + if (strName == params[0].get_str()) + { + setAddress.insert(address); + } + } + check_account=true; + } + } + + + int nMinDepth = 1; + if (params.size() > 1) + nMinDepth = params[1].get_int(); + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + asset_amounts->Clear(); + + mc_Buffer *genesis_amounts; + genesis_amounts=new mc_Buffer; + genesis_amounts->Initialize(32,32+MC_AST_ASSET_QUANTITY_SIZE,MC_BUF_MODE_MAP); + genesis_amounts->Clear(); + + mc_Script *lpScript; + lpScript=new mc_Script; + + int last_size=0; + Array assets; + CAmount totalBTC=0; + vector vecOutputs; + assert(pwalletMain != NULL); + pwalletMain->AvailableCoins(vecOutputs, false, NULL, fUnlockedOnly,true); + BOOST_FOREACH(const COutput& out, vecOutputs) { + +/* + if(out.nDepth == 0) + { + if(!out.tx->IsTrusted(out.nDepth)) + { + continue; + } + } + else + { + */ + if(!out.IsTrusted()) + { + if (out.nDepth < nMinDepth) + { + continue; + } + } +/* + } +*/ + CTxOut txout; + uint256 hash=out.GetHashAndTxOut(txout); + + if(check_account) + { + CTxDestination address; + if (!ExtractDestination(txout.scriptPubKey, address)) + continue; + + if (!setAddress.count(address)) + continue; + } + + isminetype fIsMine=pwalletMain->IsMine(txout); + + if (!(fIsMine & filter)) + { + continue; + } + + CAmount nValue = txout.nValue; + totalBTC+=nValue; + + if(CreateAssetBalanceList(txout,asset_amounts,lpScript)) + { + unsigned char *ptr; + string assetref=""; + int64_t quantity; + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + int n; + bool is_genesis; + + n=asset_amounts->GetCount(); + + for(int a=last_size;aGetRow(a); + is_genesis=false; + if(mc_GetABRefType(ptr) == MC_AST_ASSET_REF_TYPE_GENESIS) + { + mc_EntityDetails entity; + quantity=mc_GetABQuantity(asset_amounts->GetRow(a)); + if(mc_gState->m_Assets->FindEntityByTxID(&entity,(unsigned char*)&hash)) + { + if((entity.IsUnconfirmedGenesis() != 0) && (mc_gState->m_Features->ShortTxIDAsAssetRef() == 0) ) + { + is_genesis=true; + } + else + { + ptr=(unsigned char *)entity.GetFullRef(); + memcpy(buf,ptr,MC_AST_ASSET_FULLREF_SIZE); + int row=asset_amounts->Seek(buf); + if(row >= 0) + { + int64_t last=mc_GetABQuantity(asset_amounts->GetRow(row)); + quantity+=last; + mc_SetABQuantity(asset_amounts->GetRow(row),quantity); + } + else + { + mc_SetABQuantity(buf,quantity); + asset_amounts->Add(buf); + } + } + } + + if(is_genesis) + { + int row=genesis_amounts->Seek(&hash); + if(row >= 0) + { + int64_t last=mc_GetLE(genesis_amounts->GetRow(row)+32,MC_AST_ASSET_QUANTITY_SIZE); + quantity+=last; + mc_PutLE(genesis_amounts->GetRow(row)+32,&quantity,MC_AST_ASSET_QUANTITY_SIZE); + } + else + { + mc_SetABQuantity(buf,quantity); + genesis_amounts->Add(&hash,buf+MC_AST_ASSET_QUANTITY_OFFSET); + } + } + } + } + last_size=asset_amounts->GetCount(); + } + } + + + unsigned char *ptr; + string assetref=""; + + mergeGenesisWithAssets(genesis_amounts, asset_amounts); + + for(int a=0;aGetCount();a++) + { + Object asset_entry; + ptr=(unsigned char *)asset_amounts->GetRow(a); + + mc_EntityDetails entity; + const unsigned char *txid; + + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ptr)) + { + txid=entity.GetTxID(); + asset_entry=AssetEntry(txid,mc_GetABQuantity(ptr),1); + assets.push_back(asset_entry); + } + } + + for(int a=0;aGetCount();a++) + { + Object asset_entry; + ptr=(unsigned char *)genesis_amounts->GetRow(a); + + asset_entry=AssetEntry(ptr,mc_GetLE(ptr+32,MC_AST_ASSET_QUANTITY_SIZE),1); + assets.push_back(asset_entry); + } + + delete lpScript; + delete asset_amounts; + delete genesis_amounts; + +/* MCHN END */ + return assets; +} + + +Value gettotalbalances(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 3) + throw runtime_error("Help message not found\n"); + + Array new_params; + + new_params.push_back("*"); + + for(int i=0;i<(int)params.size();i++) + { + new_params.push_back(params[i]); + } + + return getassetbalances(new_params,fHelp); +} + + +Value listassets(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 4) + throw runtime_error("Help message not found\n"); + + mc_Buffer *assets; + unsigned char *txid; + int output_level; + Array results; + + int count,start; + count=2147483647; + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + count=params[2].get_int(); + if(count < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + start=-count; + if (params.size() > 3) + { + if(params[3].type() == int_type) + { + start=params[3].get_int(); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid start")); + } + } + + assets=NULL; + vector inputStrings; + if (params.size() > 0 && params[0].type() != null_type && ((params[0].type() != str_type) || (params[0].get_str() !="*" ) ) ) + { + if(params[0].type() == str_type) + { + inputStrings.push_back(params[0].get_str()); + if(params[0].get_str() == "") + { + return results; + } + } + else + { + inputStrings=ParseStringList(params[0]); + if(inputStrings.size() == 0) + { + return results; + } + } + } + if(inputStrings.size()) + { + { + LOCK(cs_main); + for(int is=0;is<(int)inputStrings.size();is++) + { + mc_EntityDetails entity; + ParseEntityIdentifier(inputStrings[is],&entity, MC_ENT_TYPE_ASSET); + uint256 hash=*(uint256*)entity.GetTxID(); + + assets=mc_gState->m_Assets->GetEntityList(assets,(unsigned char*)&hash,MC_ENT_TYPE_ASSET); + } + } + } + else + { + { + LOCK(cs_main); + assets=mc_gState->m_Assets->GetEntityList(assets,NULL,MC_ENT_TYPE_ASSET); + } + } + + if(assets == NULL) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Cannot open asset database"); + + output_level=8; + + if (params.size() > 1) + { + if(params[1].type() == int_type) + { + if(params[1].get_int()) + { + output_level=9; + } + } + if(params[1].type() == bool_type) + { + if(params[1].get_bool()) + { + output_level=9; + } + } + } + + mc_AdjustStartAndCount(&count,&start,assets->GetCount()); + + Array partial_results; + int unconfirmed_count=0; + if(count > 0) + { + for(int i=0;iGetCount();i++) + { + Object entry; + + txid=assets->GetRow(i); + entry=AssetEntry(txid,-1,output_level); + if(entry.size()>0) + { + BOOST_FOREACH(const Pair& p, entry) + { + if(p.name_ == "assetref") + { + if(p.value_.type() == str_type) + { + results.push_back(entry); + } + else + { + unconfirmed_count++; + } + } + } + } + } + + sort(results.begin(), results.end(), AssetCompareByRef); + + for(int i=0;iGetCount();i++) + { + Object entry; + + txid=assets->GetRow(i); + + entry=AssetEntry(txid,-1,output_level); + if(entry.size()>0) + { + BOOST_FOREACH(const Pair& p, entry) + { + if(p.name_ == "assetref") + { + if(p.value_.type() != str_type) + { + results.push_back(entry); + } + } + } + } + } + } + + bool return_partial=false; + if(count != assets->GetCount()) + { + return_partial=true; + } + mc_gState->m_Assets->FreeEntityList(assets); + if(return_partial) + { + for(int i=start;i streams_already_seen; + Array aMetaData; + Array aItems; + + double units=1.; + units= 1./(double)(entity->GetAssetMultiple()); + + uint256 hash=wtx.GetHash(); + + map mAddresses; + + memset(bufEmptyAssetRef,0,MC_AST_ASSET_QUANTITY_OFFSET); + + BOOST_FOREACH(const CTxIn& txin, wtx.vin) + { + CTxOut prevout; + + int err; + const CWalletTx& prev=pwalletTxsMain->GetWalletTx(txin.prevout.hash,NULL,&err); + if(err == MC_ERR_NOERROR) + { + if (txin.prevout.n < prev.vout.size()) + { + prevout=prev.vout[txin.prevout.n]; + CTxDestination address; + int required=0; + + ExtractDestination(prevout.scriptPubKey, address); + + string strFailReason; + amounts->Clear(); + if(CreateAssetBalanceList(prevout,amounts,lpScript,&required)) + { + for(int i=0;iGetCount();i++) + { + int64_t quantity=-1; + if(memcmp(entity->GetFullRef(),amounts->GetRow(i),MC_AST_ASSET_FULLREF_SIZE) == 0) + { + quantity=mc_GetABQuantity(amounts->GetRow(i)); + } + else + { + if(memcmp(entity->GetTxID(),&(txin.prevout.hash),sizeof(uint256)) == 0) + { + if(memcmp(bufEmptyAssetRef,amounts->GetRow(i),MC_AST_ASSET_QUANTITY_OFFSET) == 0) + { + quantity=mc_GetABQuantity(amounts->GetRow(i)); + } + } + } + + if(quantity >= 0) + { + map::iterator itold = mAddresses.find(address); + if (itold == mAddresses.end()) + { + mAddresses.insert(make_pair(address, -quantity)); + } + else + { + itold->second-=quantity; + } + } + } + } + } + } + } + + for (int i = 0; i < (int)wtx.vout.size(); ++i) + { + const CTxOut& txout = wtx.vout[i]; + if(!txout.scriptPubKey.IsUnspendable()) + { + string strFailReason; + CTxDestination address; + int required=0; + + ExtractDestination(txout.scriptPubKey, address); + + amounts->Clear(); + if(CreateAssetBalanceList(txout,amounts,lpScript,&required)) + { + for(int i=0;iGetCount();i++) + { + int64_t quantity=-1; + if(memcmp(entity->GetFullRef(),amounts->GetRow(i),MC_AST_ASSET_FULLREF_SIZE) == 0) + { + quantity=mc_GetABQuantity(amounts->GetRow(i)); + } + else + { + if(memcmp(entity->GetTxID(),&hash,sizeof(uint256)) == 0) + { + if(memcmp(bufEmptyAssetRef,amounts->GetRow(i),MC_AST_ASSET_QUANTITY_OFFSET) == 0) + { + quantity=mc_GetABQuantity(amounts->GetRow(i)); + } + } + } + + if(quantity >= 0) + { + map::iterator itold = mAddresses.find(address); + if (itold == mAddresses.end()) + { + mAddresses.insert(make_pair(address, +quantity)); + } + else + { + itold->second+=quantity; + } + } + } + } + } + else + { + const CScript& script2 = wtx.vout[i].scriptPubKey; + CScript::const_iterator pc2 = script2.begin(); + + lpScript->Clear(); + lpScript->SetScript((unsigned char*)(&pc2[0]),(size_t)(script2.end()-pc2),MC_SCR_TYPE_SCRIPTPUBKEY); + + size_t elem_size; + const unsigned char *elem; + + if(lpScript->GetNumElements()<=1) + { + if(lpScript->GetNumElements()==1) + { + elem = lpScript->GetData(lpScript->GetNumElements()-1,&elem_size); + aMetaData.push_back(OpReturnEntry(elem,elem_size,wtx.GetHash(),i)); + } + } + else + { + elem = lpScript->GetData(lpScript->GetNumElements()-1,&elem_size); + if(elem_size) + { + aMetaData.push_back(OpReturnEntry(elem,elem_size,wtx.GetHash(),i)); + } + + lpScript->SetElement(0); + lpScript->GetNewEntityType(&new_entity_type); + } + } + } + + if(mAddresses.size() == 0) + { + return entry; + } + + Array vin; + Array vout; + + if(fLong) + { + BOOST_FOREACH(const CTxIn& txin, wtx.vin) + { + CTxOut TxOutIn; + vin.push_back(TxOutEntry(TxOutIn,-1,txin,txin.prevout.hash,amounts,lpScript)); + } + } + for (int i = 0; i < (int)wtx.vout.size(); ++i) + { + CTxIn TxIn; + Value data_item_entry=DataItemEntry(wtx,i,streams_already_seen,0x01); + if(!data_item_entry.is_null()) + { + aItems.push_back(data_item_entry); + } + if(fLong) + { + Array aTxOutItems; + if(!data_item_entry.is_null()) + { + aTxOutItems.push_back(data_item_entry); + } + Object txout_entry=TxOutEntry(wtx.vout[i],i,TxIn,wtx.GetHash(),amounts,lpScript); + txout_entry.push_back(Pair("items", aTxOutItems)); + vout.push_back(txout_entry); + } + } + + int64_t other_amount=0; + + Object oBalance; + BOOST_FOREACH(const PAIRTYPE(CTxDestination, int64_t)& item, mAddresses) + { + const CTxDestination dest=item.first; + const CKeyID *lpKeyID=boost::get (&dest); + const CScriptID *lpScript=boost::get (&dest); + if( (lpKeyID == NULL) && (lpScript == NULL) ) + { + other_amount=item.second; + } + else + { + oBalance.push_back(Pair(CBitcoinAddress(item.first).ToString(), units*item.second)); + } + } + if(other_amount != 0) + { + oBalance.push_back(Pair("", other_amount)); + } + + entry.push_back(Pair("addresses", oBalance)); + entry.push_back(Pair("items", aItems)); + entry.push_back(Pair("data", aMetaData)); + + WalletTxToJSON(wtx, entry, true); + + if(fLong) + { + entry.push_back(Pair("vin", vin)); + entry.push_back(Pair("vout", vout)); + string strHex = EncodeHexTx(static_cast(wtx)); + entry.push_back(Pair("hex", strHex)); + } + + return entry; +} + + +Value getassettransaction(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 3) + throw runtime_error("Help message not found\n"); + + if((mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "API is not supported with this wallet version. To get this functionality, run \"multichaind -walletdbversion=2 -rescan\" "); + } + + mc_EntityDetails asset_entity; + ParseEntityIdentifier(params[0],&asset_entity,MC_ENT_TYPE_ASSET); + + mc_TxEntityStat entStat; + entStat.Zero(); + memcpy(&entStat,asset_entity.GetShortRef(),mc_gState->m_NetworkParams->m_AssetRefSize); + entStat.m_Entity.m_EntityType=MC_TET_ASSET; + entStat.m_Entity.m_EntityType |= MC_TET_CHAINPOS; + + if(!pwalletTxsMain->FindEntity(&entStat)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Not subscribed to this asset"); + } + + uint256 hash = ParseHashV(params[1], "parameter 2"); + + bool verbose=false; + + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + if(params[2].get_int()) + { + verbose=true; + } + } + if(params[2].type() == bool_type) + { + if(params[2].get_bool()) + { + verbose=true; + } + } + } + + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; + + Object entry=ListAssetTransactions(wtx, &asset_entity, verbose, asset_amounts, lpScript); + + if(entry.size() == 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Specified asset was not transferred in this transaction"); + } + + delete asset_amounts; + delete lpScript; + + return entry; +} + + +Value listassettransactions(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 5) + throw runtime_error("Help message not found\n"); + + if((mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "API is not supported with this wallet version. To get this functionality, run \"multichaind -walletdbversion=2 -rescan\" "); + } + + Array retArray; + + mc_TxEntityStat entStat; + mc_TxEntityRow *lpEntTx; + + mc_EntityDetails asset_entity; + ParseEntityIdentifier(params[0],&asset_entity,MC_ENT_TYPE_ASSET); + + int count,start,shift; + bool verbose=false; + bool fGenesis; + + if (params.size() > 1) + { + if(params[1].type() == int_type) + { + if(params[1].get_int()) + { + verbose=true; + } + } + if(params[1].type() == bool_type) + { + if(params[1].get_bool()) + { + verbose=true; + } + } + } + + count=10; + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + count=params[2].get_int(); + if(count < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + start=-count; + if (params.size() > 3) + { + if(params[3].type() == int_type) + { + start=params[3].get_int(); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid start")); + } + } + + bool fLocalOrdering = false; + if (params.size() > 4) + fLocalOrdering = params[4].get_bool(); + + entStat.Zero(); + memcpy(&entStat,asset_entity.GetShortRef(),mc_gState->m_NetworkParams->m_AssetRefSize); + entStat.m_Entity.m_EntityType=MC_TET_ASSET; + if(fLocalOrdering) + { + entStat.m_Entity.m_EntityType |= MC_TET_TIMERECEIVED; + } + else + { + entStat.m_Entity.m_EntityType |= MC_TET_CHAINPOS; + } + if(!pwalletTxsMain->FindEntity(&entStat)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Not subscribed to this asset"); + } + + mc_Buffer *entity_rows; + entity_rows=new mc_Buffer; + entity_rows->Initialize(MC_TDB_ENTITY_KEY_SIZE,MC_TDB_ROW_SIZE,MC_BUF_MODE_DEFAULT); + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; + + pwalletTxsMain->GetList(&entStat.m_Entity,1,1,entity_rows); + shift=1; + if(entity_rows->GetCount()) + { + lpEntTx=(mc_TxEntityRow*)entity_rows->GetRow(0); + if(memcmp(lpEntTx->m_TxId,asset_entity.GetTxID(),MC_TDB_TXID_SIZE) == 0) + { + shift=0; + } + } + + mc_AdjustStartAndCount(&count,&start,entStat.m_LastPos+shift); + + fGenesis=false; + if(shift) + { + if(start) + { + start-=shift; + shift=0; + } + else + { + count-=shift; + fGenesis=true; + } + } + + pwalletTxsMain->GetList(&entStat.m_Entity,start+1,count,entity_rows); + + + for(int i=0;iGetCount()+shift;i++) + { + uint256 hash; + if(fGenesis && (i == 0)) + { + memcpy(&hash,asset_entity.GetTxID(),MC_TDB_TXID_SIZE); + } + else + { + lpEntTx=(mc_TxEntityRow*)entity_rows->GetRow(i-shift); + memcpy(&hash,lpEntTx->m_TxId,MC_TDB_TXID_SIZE); + } + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + Object entry=ListAssetTransactions(wtx, &asset_entity, verbose, asset_amounts, lpScript); + + + if(entry.size()) + { + retArray.push_back(entry); + } + } + + delete entity_rows; + delete asset_amounts; + delete lpScript; + + return retArray; +} diff --git a/src/rpc/rpcblockchain.cpp b/src/rpc/rpcblockchain.cpp new file mode 100644 index 00000000..32d17b87 --- /dev/null +++ b/src/rpc/rpcblockchain.cpp @@ -0,0 +1,659 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "chain/checkpoints.h" +#include "core/main.h" +#include "rpc/rpcserver.h" +#include "utils/sync.h" +#include "utils/util.h" + +#include + +/* MCHN START */ +#include "structs/base58.h" +/* MCHN END */ + +#include "json/json_spirit_value.h" + +using namespace json_spirit; +using namespace std; + +extern void TxToJSON(const CTransaction& tx, const uint256 hashBlock, Object& entry); + +/* MCHN START */ +bool ParseMultichainTxOutToBuffer(uint256 hash,const CTxOut& txout,mc_Buffer *amounts,mc_Script *lpScript,int *allowed,int *required,string& strFailReason); +bool CreateAssetBalanceList(const CTxOut& out,mc_Buffer *amounts,mc_Script *lpScript); +Object AssetEntry(const unsigned char *txid,int64_t quantity,int output_level); +Array PermissionEntries(const CTxOut& txout,mc_Script *lpScript,bool fLong); +string EncodeHexTx(const CTransaction& tx); +/* MCHN END */ + +void ScriptPubKeyToJSON(const CScript& scriptPubKey, Object& out, bool fIncludeHex); + +double GetDifficulty(const CBlockIndex* blockindex) +{ + // Floating point number that is a multiple of the minimum difficulty, + // minimum difficulty = 1.0. + if (blockindex == NULL) + { + if (chainActive.Tip() == NULL) + return 1.0; + else + blockindex = chainActive.Tip(); + } + + int nShift = (blockindex->nBits >> 24) & 0xff; + + double dDiff = + (double)0x0000ffff / (double)(blockindex->nBits & 0x00ffffff); + + while (nShift < 29) + { + dDiff *= 256.0; + nShift++; + } + while (nShift > 29) + { + dDiff /= 256.0; + nShift--; + } + + return dDiff; +} + + +Object blockToJSON(const CBlock& block, const CBlockIndex* blockindex, bool txDetails = false, int verbose_level = 1) +{ + Object result; + result.push_back(Pair("hash", block.GetHash().GetHex())); +/* MCHN START */ + CKeyID keyID; + Value miner; + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(mc_gState->m_Permissions->GetBlockMiner(blockindex->nHeight,(unsigned char*)&keyID) == MC_ERR_NOERROR) + { + miner=CBitcoinAddress(keyID).ToString(); + } + } + result.push_back(Pair("miner", miner)); +/* MCHN END */ + int confirmations = -1; + // Only report confirmations if the block is on the main chain + if (chainActive.Contains(blockindex)) + confirmations = chainActive.Height() - blockindex->nHeight + 1; + result.push_back(Pair("confirmations", confirmations)); + result.push_back(Pair("size", (int)::GetSerializeSize(block, SER_NETWORK, PROTOCOL_VERSION))); + result.push_back(Pair("height", blockindex->nHeight)); + result.push_back(Pair("version", block.nVersion)); + result.push_back(Pair("merkleroot", block.hashMerkleRoot.GetHex())); + Array txs; + Object objTx; + BOOST_FOREACH(const CTransaction&tx, block.vtx) + { + if(txDetails) + { + objTx.clear(); + switch(verbose_level) + { + case 1: + txs.push_back(tx.GetHash().GetHex()); + break; + case 2: + objTx.push_back(Pair("txid", tx.GetHash().GetHex())); + objTx.push_back(Pair("hex", EncodeHexTx(tx))); + txs.push_back(objTx); + break; + case 3: + objTx.push_back(Pair("txid", tx.GetHash().GetHex())); + objTx.push_back(Pair("version", tx.nVersion)); + objTx.push_back(Pair("locktime", (int64_t)tx.nLockTime)); + objTx.push_back(Pair("hex", EncodeHexTx(tx))); + txs.push_back(objTx); + break; + case 4: + TxToJSON(tx, uint256(0), objTx); + objTx.push_back(Pair("hex", EncodeHexTx(tx))); + txs.push_back(objTx); + break; + } + } + else + txs.push_back(tx.GetHash().GetHex()); + } + result.push_back(Pair("tx", txs)); + result.push_back(Pair("time", block.GetBlockTime())); + result.push_back(Pair("nonce", (uint64_t)block.nNonce)); + result.push_back(Pair("bits", strprintf("%08x", block.nBits))); + result.push_back(Pair("difficulty", GetDifficulty(blockindex))); + result.push_back(Pair("chainwork", blockindex->nChainWork.GetHex())); + + if (blockindex->pprev) + result.push_back(Pair("previousblockhash", blockindex->pprev->GetBlockHash().GetHex())); + CBlockIndex *pnext = chainActive.Next(blockindex); + if (pnext) + result.push_back(Pair("nextblockhash", pnext->GetBlockHash().GetHex())); + return result; +} + + +Value getblockcount(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + return chainActive.Height(); +} + +Value getbestblockhash(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + return chainActive.Tip()->GetBlockHash().GetHex(); +} + +Value getdifficulty(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + return GetDifficulty(); +} + + +Value getrawmempool(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 1) // MCHN + throw runtime_error("Help message not found\n"); + + bool fVerbose = false; + if (params.size() > 0) + fVerbose = params[0].get_bool(); + + if (fVerbose) + { + LOCK(mempool.cs); + Object o; + BOOST_FOREACH(const PAIRTYPE(uint256, CTxMemPoolEntry)& entry, mempool.mapTx) + { + const uint256& hash = entry.first; + const CTxMemPoolEntry& e = entry.second; + Object info; + info.push_back(Pair("size", (int)e.GetTxSize())); + info.push_back(Pair("fee", ValueFromAmount(e.GetFee()))); + info.push_back(Pair("time", e.GetTime())); + info.push_back(Pair("height", (int)e.GetHeight())); + info.push_back(Pair("startingpriority", e.GetPriority(e.GetHeight()))); + info.push_back(Pair("currentpriority", e.GetPriority(chainActive.Height()))); + const CTransaction& tx = e.GetTx(); + set setDepends; + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + if (mempool.exists(txin.prevout.hash)) + setDepends.insert(txin.prevout.hash.ToString()); + } + Array depends(setDepends.begin(), setDepends.end()); + info.push_back(Pair("depends", depends)); + o.push_back(Pair(hash.ToString(), info)); + } + return o; + } + else + { + vector vtxid; + mempool.queryHashes(vtxid); + + Array a; + BOOST_FOREACH(const uint256& hash, vtxid) + a.push_back(hash.ToString()); + + return a; + } +} + +Value getblockhash(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + int64_t nHeight = params[0].get_int64(); // MCHN - was int + if (nHeight < 0 || nHeight > chainActive.Height()) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Block height out of range"); + + CBlockIndex* pblockindex = chainActive[nHeight]; + return pblockindex->GetBlockHash().GetHex(); +} + +/* MCHN START */ +Value clearmempool(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 0) + throw runtime_error("Help message not found\n"); + + uint32_t required_paused_state=MC_NPS_INCOMING | MC_NPS_MINING; + if((mc_gState->m_NodePausedState & required_paused_state) != required_paused_state) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "Local mining and the processing of incoming transactions and blocks should be paused."); + } + + ClearMemPools(); + + return "Mempool cleared"; +} + +Value setlastblock(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 1) // MCHN + throw runtime_error("Help message not found\n"); + + uint32_t required_paused_state=MC_NPS_INCOMING | MC_NPS_MINING; + if((mc_gState->m_NodePausedState & required_paused_state) != required_paused_state) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "Local mining and the processing of incoming transactions and blocks should be paused."); + } + + + if(params.size() == 0) + { + SetLastBlock(0); + } + else + { + std::string strHash = params[0].get_str(); + if(strHash.size() < 64) + { + int nHeight = atoi(params[0].get_str().c_str()); + if (nHeight <= 0 || nHeight > chainActive.Height()) + { + nHeight+=chainActive.Height(); + if (nHeight <= 0 || nHeight > chainActive.Height()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Block height out of range"); + } + } + + strHash=chainActive[nHeight]->GetBlockHash().GetHex(); + } + + uint256 hash(strHash); + + string result=SetLastBlock(hash); + + if(result.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, result); + } + } + + return chainActive.Tip()->GetBlockHash().GetHex(); +} +/* MCHN END */ + + +Value getblock(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) // MCHN + throw runtime_error("Help message not found\n"); + + std::string strHash = params[0].get_str(); + if(strHash.size() < 64) + { + int nHeight = atoi(params[0].get_str().c_str()); + if (nHeight < 0 || nHeight > chainActive.Height()) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Block height out of range"); + + strHash=chainActive[nHeight]->GetBlockHash().GetHex(); + } + uint256 hash(strHash); +/* + bool fVerbose = true; + if (params.size() > 1) + fVerbose = params[1].get_bool(); +*/ + int verbose_level=1; + if (params.size() > 1) + { + if (params[1].type() == bool_type) + { + if(!params[1].get_bool()) + { + verbose_level=0; + } + } + else + { + verbose_level=params[1].get_int(); + if((verbose_level < 0) || (verbose_level >4)) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "verbose out of range"); + } + } + } + + if (mapBlockIndex.count(hash) == 0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Block not found"); + + CBlock block; + CBlockIndex* pblockindex = mapBlockIndex[hash]; + + if(!ReadBlockFromDisk(block, pblockindex)) + throw JSONRPCError(RPC_INTERNAL_ERROR, "Can't read block from disk"); + +// if (!fVerbose) + if(verbose_level == 0) + { + CDataStream ssBlock(SER_NETWORK, PROTOCOL_VERSION); + ssBlock << block; +// std::string strHex = HexStr(ssBlock.begin(), ssBlock.end()); + return HexStr(ssBlock.begin(), ssBlock.end()); + } + + return blockToJSON(block, pblockindex, true, verbose_level); +} + +Value gettxoutsetinfo(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + Object ret; + + CCoinsStats stats; + FlushStateToDisk(); + if (pcoinsTip->GetStats(stats)) { + ret.push_back(Pair("height", (int64_t)stats.nHeight)); + ret.push_back(Pair("bestblock", stats.hashBlock.GetHex())); + ret.push_back(Pair("transactions", (int64_t)stats.nTransactions)); + ret.push_back(Pair("txouts", (int64_t)stats.nTransactionOutputs)); + ret.push_back(Pair("bytes_serialized", (int64_t)stats.nSerializedSize)); + ret.push_back(Pair("hash_serialized", stats.hashSerialized.GetHex())); + ret.push_back(Pair("total_amount", ValueFromAmount(stats.nTotalAmount))); + } + return ret; +} + +Value gettxout(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 3) // MCHN + throw runtime_error("Help message not found\n"); + + Object ret; + + std::string strHash = params[0].get_str(); + uint256 hash(strHash); + int64_t n = params[1].get_int64(); // MCHN was int + bool fMempool = true; + if (params.size() > 2) + fMempool = params[2].get_bool(); + + CCoins coins; + if (fMempool) { + LOCK(mempool.cs); + CCoinsViewMemPool view(pcoinsTip, mempool); + if (!view.GetCoins(hash, coins)) + return Value::null; + mempool.pruneSpent(hash, coins); // TODO: this should be done by the CCoinsViewMemPool + } else { + if (!pcoinsTip->GetCoins(hash, coins)) + return Value::null; + } + if (n<0 || (unsigned int)n>=coins.vout.size() || coins.vout[n].IsNull()) + return Value::null; + + BlockMap::iterator it = mapBlockIndex.find(pcoinsTip->GetBestBlock()); + CBlockIndex *pindex = it->second; + ret.push_back(Pair("bestblock", pindex->GetBlockHash().GetHex())); + if ((unsigned int)coins.nHeight == MEMPOOL_HEIGHT) + ret.push_back(Pair("confirmations", 0)); + else + ret.push_back(Pair("confirmations", pindex->nHeight - coins.nHeight + 1)); + ret.push_back(Pair("value", ValueFromAmount(coins.vout[n].nValue))); + Object o; + ScriptPubKeyToJSON(coins.vout[n].scriptPubKey, o, true); + ret.push_back(Pair("scriptPubKey", o)); + ret.push_back(Pair("version", coins.nVersion)); + ret.push_back(Pair("coinbase", coins.fCoinBase)); + +/* MCHN START */ + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; + + asset_amounts->Clear(); + CTxOut txout=coins.vout[n]; + if(CreateAssetBalanceList(txout,asset_amounts,lpScript)) + { + Array assets; + unsigned char *ptr; + + for(int a=0;aGetCount();a++) + { + Object asset_entry; + ptr=(unsigned char *)asset_amounts->GetRow(a); + const unsigned char *txid; + + txid=(unsigned char*)&hash; + if( (mc_GetABRefType(ptr) != MC_AST_ASSET_REF_TYPE_SPECIAL) && + (mc_GetABRefType(ptr) != MC_AST_ASSET_REF_TYPE_GENESIS) ) +// if(mc_GetLE(ptr,4) > 0) + { + mc_EntityDetails entity; + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ptr)) + { + txid=entity.GetTxID(); + } + } + + asset_entry=AssetEntry(txid,mc_GetABQuantity(ptr),3); + + if(mc_GetABRefType(ptr) == MC_AST_ASSET_REF_TYPE_GENESIS) +// if(mc_GetLE(ptr,4) == 0) + { + asset_entry.push_back(Pair("issue", true)); + } + assets.push_back(asset_entry); + } + + ret.push_back(Pair("assets", assets)); + } + Array permissions=PermissionEntries(txout,lpScript,false); + ret.push_back(Pair("permissions", permissions)); + +/* MCHN END */ + + return ret; +} + +Value verifychain(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 2) + throw runtime_error("Help message not found\n"); + + int nCheckLevel = GetArg("-checklevel", 3); + int nCheckDepth = GetArg("-checkblocks", 288); + if (params.size() > 0) + nCheckLevel = params[0].get_int(); + if (params.size() > 1) + nCheckDepth = params[1].get_int(); + + return CVerifyDB().VerifyDB(pcoinsTip, nCheckLevel, nCheckDepth); +} + +Value getblockchaininfo(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + Object obj; +/* MCHN START*/ +// obj.push_back(Pair("chain", Params().NetworkIDString())); + obj.push_back(Pair("chain",Params().TestnetToBeDeprecatedFieldRPC() ? "test" : "main")); + obj.push_back(Pair("chainname", string(mc_gState->m_NetworkParams->Name()))); + obj.push_back(Pair("description", string((char*)mc_gState->m_NetworkParams->GetParam("chaindescription",NULL)))); + obj.push_back(Pair("protocol", string((char*)mc_gState->m_NetworkParams->GetParam("chainprotocol",NULL)))); + obj.push_back(Pair("setupblocks", mc_gState->m_NetworkParams->GetInt64Param("setupfirstblocks"))); + obj.push_back(Pair("reindex", fReindex)); +/* MCHN END*/ + obj.push_back(Pair("blocks", (int)chainActive.Height())); + obj.push_back(Pair("headers", pindexBestHeader ? pindexBestHeader->nHeight : -1)); + obj.push_back(Pair("bestblockhash", chainActive.Tip()->GetBlockHash().GetHex())); + obj.push_back(Pair("difficulty", (double)GetDifficulty())); + obj.push_back(Pair("verificationprogress", Checkpoints::GuessVerificationProgress(chainActive.Tip()))); + obj.push_back(Pair("chainwork", chainActive.Tip()->nChainWork.GetHex())); + return obj; +} + +/** Comparison function for sorting the getchaintips heads. */ +struct CompareBlocksByHeight +{ + bool operator()(const CBlockIndex* a, const CBlockIndex* b) const + { + /* Make sure that unequal blocks with the same height do not compare + equal. Use the pointers themselves to make a distinction. */ + + if (a->nHeight != b->nHeight) + return (a->nHeight > b->nHeight); + + return a < b; + } +}; + +Value getchaintips(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + /* Build up a list of chain tips. We start with the list of all + known blocks, and successively remove blocks that appear as pprev + of another block. */ + std::set setTips; + BOOST_FOREACH(const PAIRTYPE(const uint256, CBlockIndex*)& item, mapBlockIndex) + setTips.insert(item.second); + BOOST_FOREACH(const PAIRTYPE(const uint256, CBlockIndex*)& item, mapBlockIndex) + { + const CBlockIndex* pprev = item.second->pprev; + if (pprev) + setTips.erase(pprev); + } + + // Always report the currently active tip. + setTips.insert(chainActive.Tip()); + + /* Construct the output array. */ + Array res; + BOOST_FOREACH(const CBlockIndex* block, setTips) + { + Object obj; + obj.push_back(Pair("height", block->nHeight)); + obj.push_back(Pair("hash", block->phashBlock->GetHex())); + + const int branchLen = block->nHeight - chainActive.FindFork(block)->nHeight; + obj.push_back(Pair("branchlen", branchLen)); + + string status; + if (chainActive.Contains(block)) { + // This block is part of the currently active chain. + status = "active"; + } else if (block->nStatus & BLOCK_FAILED_MASK) { + // This block or one of its ancestors is invalid. + status = "invalid"; + } else if (block->nChainTx == 0) { + // This block cannot be connected because full block data for it or one of its parents is missing. + status = "headers-only"; + } else if (block->IsValid(BLOCK_VALID_SCRIPTS)) { + // This block is fully validated, but no longer part of the active chain. It was probably the active block once, but was reorganized. + status = "valid-fork"; + } else if (block->IsValid(BLOCK_VALID_TREE)) { + // The headers for this block are valid, but it has not been validated. It was probably never part of the most-work chain. + status = "valid-headers"; + } else { + // No clue. + status = "unknown"; + } + obj.push_back(Pair("status", status)); + + res.push_back(obj); + } + + return res; +} + +Value getmempoolinfo(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + Object ret; + ret.push_back(Pair("size", (int64_t) mempool.size())); + ret.push_back(Pair("bytes", (int64_t) mempool.GetTotalTxSize())); + + return ret; +} + +Value invalidateblock(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + std::string strHash = params[0].get_str(); + uint256 hash(strHash); + CValidationState state; + + { + LOCK(cs_main); + if (mapBlockIndex.count(hash) == 0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Block not found"); + + CBlockIndex* pblockindex = mapBlockIndex[hash]; + InvalidateBlock(state, pblockindex); + } + + if (state.IsValid()) { + ActivateBestChain(state); + } + + if (!state.IsValid()) { + throw JSONRPCError(RPC_DATABASE_ERROR, state.GetRejectReason()); + } + + return Value::null; +} + +Value reconsiderblock(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + std::string strHash = params[0].get_str(); + uint256 hash(strHash); + CValidationState state; + + { + LOCK(cs_main); + if (mapBlockIndex.count(hash) == 0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Block not found"); + + CBlockIndex* pblockindex = mapBlockIndex[hash]; + ReconsiderBlock(state, pblockindex); + } + + if (state.IsValid()) { + ActivateBestChain(state); + } + + if (!state.IsValid()) { + throw JSONRPCError(RPC_DATABASE_ERROR, state.GetRejectReason()); + } + + return Value::null; +} + + diff --git a/src/rpc/rpcclient.cpp b/src/rpc/rpcclient.cpp new file mode 100644 index 00000000..b87323ea --- /dev/null +++ b/src/rpc/rpcclient.cpp @@ -0,0 +1,405 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "rpc/rpcclient.h" + +#include "rpc/rpcprotocol.h" +#include "utils/util.h" +#include "ui/ui_interface.h" + +#include +#include + +using namespace std; +using namespace json_spirit; + +class CRPCConvertParam +{ +public: + std::string methodName; //! method whose params want conversion + int paramIdx; //! 0-based idx of param to convert +}; + +static const CRPCConvertParam vRPCConvertParams[] = +{ + { "stop", 0 }, + { "setmocktime", 0 }, + { "getaddednodeinfo", 0 }, + { "setgenerate", 0 }, + { "setgenerate", 1 }, + { "getnetworkhashps", 0 }, + { "getnetworkhashps", 1 }, + { "sendtoaddress", 1 }, + { "send", 1 }, +/* MCHN START */ + { "createkeypairs", 0 }, + { "combineunspent", 1 }, + { "combineunspent", 2 }, + { "combineunspent", 3 }, + { "combineunspent", 4 }, + { "combineunspent", 5 }, + { "grant", 2 }, + { "grant", 3 }, + { "grant", 4 }, + { "grantwithmetadata", 2 }, + { "grantwithmetadata", 3 }, + { "grantwithmetadata", 4 }, + { "grantwithmetadata", 5 }, + { "grantwithdata", 2 }, + { "grantwithdata", 3 }, + { "grantwithdata", 4 }, + { "grantwithdata", 5 }, + { "revoke", 2 }, + { "issue", 1 }, + { "issue", 2 }, + { "issue", 3 }, + { "issue", 4 }, + { "issue", 5 }, + { "issuemore", 2 }, + { "issuemore", 3 }, + { "issuemore", 4 }, + { "listassets", 0 }, + { "listassets", 1 }, + { "listassets", 2 }, + { "listassets", 3 }, + { "liststreams", 0 }, + { "liststreams", 1 }, + { "liststreams", 2 }, + { "liststreams", 3 }, + { "getassetbalances", 1 }, + { "getassetbalances", 2 }, + { "getassetbalances", 3 }, + { "gettotalbalances", 0 }, + { "gettotalbalances", 1 }, + { "gettotalbalances", 2 }, + { "sendassettoaddress", 2 }, + { "sendassettoaddress", 3 }, + { "sendasset", 2 }, + { "sendasset", 3 }, + { "getblockchainparams", 0 }, + { "preparelockunspent", 0 }, + { "preparelockunspent", 1 }, + { "createrawexchange", 1 }, + { "createrawexchange", 2 }, + { "appendrawexchange", 2 }, + { "appendrawexchange", 3 }, + { "decoderawexchange", 1 }, + { "appendrawmetadata", 1 }, + { "appendrawdata", 1 }, + { "sendfromaddress", 2 }, + { "sendfrom", 2 }, + { "sendassetfrom", 3 }, + { "sendassetfrom", 4 }, + { "grantwithmetadatafrom", 3 }, + { "grantwithmetadatafrom", 4 }, + { "grantwithmetadatafrom", 5 }, + { "grantwithmetadatafrom", 6 }, + { "grantwithdatafrom", 3 }, + { "grantwithdatafrom", 4 }, + { "grantwithdatafrom", 5 }, + { "grantwithdatafrom", 6 }, + { "grantfrom", 3 }, + { "grantfrom", 4 }, + { "grantfrom", 5 }, + { "revokefrom", 3 }, + { "issuefrom", 2 }, + { "issuefrom", 3 }, + { "issuefrom", 4 }, + { "issuefrom", 5 }, + { "issuefrom", 6 }, + { "issuemorefrom", 3 }, + { "issuemorefrom", 4 }, + { "issuemorefrom", 5 }, + { "preparelockunspentfrom", 1 }, + { "preparelockunspentfrom", 2 }, + { "getaddressbalances", 1 }, + { "getaddressbalances", 2 }, + { "getmultibalances", 0 }, + { "getmultibalances", 1 }, + { "getmultibalances", 2 }, + { "getmultibalances", 3 }, + { "getmultibalances", 4 }, + { "listaddresses", 0 }, + { "listaddresses", 1 }, + { "listaddresses", 2 }, + { "listaddresses", 3 }, + { "listpermissions", 1 }, + { "listpermissions", 2 }, + { "sendwithmetadata", 1 }, + { "sendwithmetadata", 2 }, + { "sendwithdata", 1 }, + { "sendwithdata", 2 }, + { "sendwithmetadatafrom", 2 }, + { "sendwithmetadatafrom", 3 }, + { "sendwithdatafrom", 2 }, + { "sendwithdatafrom", 3 }, + { "getaddresses", 0 }, + { "listwallettransactions", 0 }, + { "listwallettransactions", 1 }, + { "listwallettransactions", 2 }, + { "listwallettransactions", 3 }, + { "listaddresstransactions", 1 }, + { "listaddresstransactions", 2 }, + { "listaddresstransactions", 3 }, + { "getwallettransaction", 1 }, + { "getwallettransaction", 2 }, + { "getaddresstransaction",2 }, + { "appendrawchange",2 }, + { "createfrom", 3 }, + { "createfrom", 4 }, + { "create", 2 }, + { "create", 3 }, + { "subscribe", 0 }, + { "subscribe", 1 }, + { "unsubscribe", 0 }, + { "listassettransactions", 1 }, + { "listassettransactions", 2 }, + { "listassettransactions", 3 }, + { "listassettransactions", 4 }, + { "getassettransaction", 2 }, + { "getstreamitem", 2 }, + { "liststreamitems", 1 }, + { "liststreamitems", 2 }, + { "liststreamitems", 3 }, + { "liststreamitems", 4 }, + { "gettxoutdata", 1 }, + { "gettxoutdata", 2 }, + { "gettxoutdata", 3 }, + { "liststreamkeys", 1 }, + { "liststreamkeys", 2 }, + { "liststreamkeys", 3 }, + { "liststreamkeys", 4 }, + { "liststreamkeys", 5 }, + { "liststreampublishers", 1 }, + { "liststreampublishers", 2 }, + { "liststreampublishers", 3 }, + { "liststreampublishers", 4 }, + { "liststreampublishers", 5 }, + { "liststreamkeyitems", 2 }, + { "liststreamkeyitems", 3 }, + { "liststreamkeyitems", 4 }, + { "liststreamkeyitems", 5 }, + { "liststreampublisheritems", 2 }, + { "liststreampublisheritems", 3 }, + { "liststreampublisheritems", 4 }, + { "liststreampublisheritems", 5 }, +/* MCHN END */ + { "settxfee", 0 }, + { "getreceivedbyaddress", 1 }, + { "getreceivedbyaccount", 1 }, + { "listreceivedbyaddress", 0 }, + { "listreceivedbyaddress", 1 }, + { "listreceivedbyaddress", 2 }, + { "listreceivedbyaccount", 0 }, + { "listreceivedbyaccount", 1 }, + { "listreceivedbyaccount", 2 }, + { "getbalance", 1 }, + { "getbalance", 2 }, + { "getblockhash", 0 }, + { "move", 2 }, + { "move", 3 }, +// { "sendfrom", 2 }, +// { "sendfrom", 3 }, + { "sendfromaccount", 2 }, + { "sendfromaccount", 3 }, + { "listtransactions", 1 }, + { "listtransactions", 2 }, + { "listtransactions", 3 }, + { "listaccounts", 0 }, + { "listaccounts", 1 }, + { "walletpassphrase", 1 }, + { "getblocktemplate", 0 }, + { "listsinceblock", 1 }, + { "listsinceblock", 2 }, + { "sendmany", 1 }, + { "sendmany", 2 }, + { "addmultisigaddress", 0 }, + { "addmultisigaddress", 1 }, + { "createmultisig", 0 }, + { "createmultisig", 1 }, + { "listunspent", 0 }, + { "listunspent", 1 }, + { "listunspent", 2 }, + { "getblock", 1 }, + { "gettransaction", 1 }, + { "getrawtransaction", 1 }, + { "createrawtransaction", 0 }, + { "createrawtransaction", 1 }, + { "createrawtransaction", 2 }, + { "createrawsendfrom", 1 }, + { "createrawsendfrom", 2 }, + { "signrawtransaction", 1 }, + { "signrawtransaction", 2 }, + { "sendrawtransaction", 1 }, + { "gettxout", 1 }, + { "gettxout", 2 }, + { "lockunspent", 0 }, + { "lockunspent", 1 }, + { "importprivkey", 0 }, + { "importprivkey", 2 }, + { "importaddress", 0 }, + { "importaddress", 2 }, + { "verifychain", 0 }, + { "verifychain", 1 }, + { "keypoolrefill", 0 }, + { "getrawmempool", 0 }, + { "estimatefee", 0 }, + { "estimatepriority", 0 }, + { "prioritisetransaction", 1 }, + { "prioritisetransaction", 2 }, +}; + +class CRPCConvertTable +{ +private: + std::set > members; + +public: + CRPCConvertTable(); + + bool convert(const std::string& method, int idx) { + return (members.count(std::make_pair(method, idx)) > 0); + } +}; + +CRPCConvertTable::CRPCConvertTable() +{ + const unsigned int n_elem = + (sizeof(vRPCConvertParams) / sizeof(vRPCConvertParams[0])); + + for (unsigned int i = 0; i < n_elem; i++) { + members.insert(std::make_pair(vRPCConvertParams[i].methodName, + vRPCConvertParams[i].paramIdx)); + } +} + +static CRPCConvertTable rpcCvtTable; + +/* MCHN START */ + +class CRPCConvertParamMayBeString +{ +public: + std::string methodName; //! method whose params want conversion + int paramIdx; //! 0-based idx of param to convert +}; + +static const CRPCConvertParamMayBeString vRPCConvertParamsMayBeString[] = +{ + { "issue", 1 }, + { "issuefrom", 2 }, + { "appendrawmetadata", 1 }, + { "appendrawdata", 1 }, + { "getmultibalances", 0 }, + { "getmultibalances", 1 }, + { "listaddresses", 0 }, + { "grantwithmetadata", 2 }, + { "grantwithdata", 2 }, + { "grantwithmetadatafrom", 3 }, + { "grantwithdatafrom", 3 }, + { "sendwithmetadata", 2 }, + { "sendwithdata", 2 }, + { "sendwithmetadatafrom", 3 }, + { "sendwithdatafrom", 3 }, + { "importaddress", 0 }, + { "importprivkey", 0 }, + { "subscribe", 0 }, + { "unsubscribe", 0 }, + { "liststreamkeys", 1 }, + { "liststreampublishers", 1 }, + { "listassets", 0 }, + { "liststreams", 0 }, + { "listpermissions", 1 }, +}; + +class CRPCConvertTableMayBeString +{ +private: + std::set > members; + +public: + CRPCConvertTableMayBeString(); + + bool maybestring(const std::string& method, int idx) { + return (members.count(std::make_pair(method, idx)) > 0); + } +}; + +CRPCConvertTableMayBeString::CRPCConvertTableMayBeString() +{ + const unsigned int n_elem = + (sizeof(vRPCConvertParamsMayBeString) / sizeof(vRPCConvertParamsMayBeString[0])); + + for (unsigned int i = 0; i < n_elem; i++) { + members.insert(std::make_pair(vRPCConvertParamsMayBeString[i].methodName, + vRPCConvertParamsMayBeString[i].paramIdx)); + } +} + +static CRPCConvertTableMayBeString rpcCvtTableMayBeString; + +/* MCHN END */ + +/** Convert strings to command-specific RPC representation */ +Array RPCConvertValues(const std::string &strMethod, const std::vector &strParams) +{ + Array params; + + for (unsigned int idx = 0; idx < strParams.size(); idx++) { + const std::string& strVal = strParams[idx]; + + // insert string value directly + if (!rpcCvtTable.convert(strMethod, idx)) { + params.push_back(strVal); + } + + // parse string as JSON, insert bool/number/object/etc. value + else { + Value jVal; + if (!read_string(strVal, jVal)) +/* MCHN START */ + { + if (!rpcCvtTableMayBeString.maybestring(strMethod, idx)) + { + throw runtime_error(string("Error parsing JSON:")+strVal); + } + else + { + params.push_back(strVal); + } + } + else + { + if (!rpcCvtTableMayBeString.maybestring(strMethod, idx)) + { + params.push_back(jVal); + } + else + { + if(jVal.type() == obj_type) + { + params.push_back(jVal); + } + else + { + if(jVal.type() == array_type) + { + params.push_back(jVal); + } + else + { + params.push_back(strVal); + } + } + } + } +/* MCHN END */ + } + } + + return params; +} + diff --git a/src/rpc/rpcclient.h b/src/rpc/rpcclient.h new file mode 100644 index 00000000..88e16e53 --- /dev/null +++ b/src/rpc/rpcclient.h @@ -0,0 +1,16 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_RPCCLIENT_H +#define BITCOIN_RPCCLIENT_H + +#include "json/json_spirit_reader_template.h" +#include "json/json_spirit_utils.h" +#include "json/json_spirit_writer_template.h" + +json_spirit::Array RPCConvertValues(const std::string& strMethod, const std::vector& strParams); + +#endif // BITCOIN_RPCCLIENT_H diff --git a/src/rpc/rpcdump.cpp b/src/rpc/rpcdump.cpp new file mode 100644 index 00000000..4527077f --- /dev/null +++ b/src/rpc/rpcdump.cpp @@ -0,0 +1,557 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "structs/base58.h" +#include "rpc/rpcserver.h" +#include "core/init.h" +#include "core/main.h" +#include "script/script.h" +#include "script/standard.h" +#include "utils/sync.h" +#include "utils/util.h" +#include "utils/utiltime.h" +#include "wallet/wallet.h" +/* MCHN START */ +#include "wallet/wallettxs.h" +/* MCHN END */ + +#include +#include + +#include +#include + +#include "json/json_spirit_value.h" + +using namespace json_spirit; +using namespace std; + +void EnsureWalletIsUnlocked(); +/* MCHN START */ +vector ParseStringList(Value param); +/* MCHN END */ + +std::string static EncodeDumpTime(int64_t nTime) { + return DateTimeStrFormat("%Y-%m-%dT%H:%M:%SZ", nTime); +} + +int64_t static DecodeDumpTime(const std::string &str) { + static const boost::posix_time::ptime epoch = boost::posix_time::from_time_t(0); + static const std::locale loc(std::locale::classic(), + new boost::posix_time::time_input_facet("%Y-%m-%dT%H:%M:%SZ")); + std::istringstream iss(str); + iss.imbue(loc); + boost::posix_time::ptime ptime(boost::date_time::not_a_date_time); + iss >> ptime; + if (ptime.is_not_a_date_time()) + return 0; + return (ptime - epoch).total_seconds(); +} + +std::string static EncodeDumpString(const std::string &str) { + std::stringstream ret; + BOOST_FOREACH(unsigned char c, str) { + if (c <= 32 || c >= 128 || c == '%') { + ret << '%' << HexStr(&c, &c + 1); + } else { + ret << c; + } + } + return ret.str(); +} + +std::string DecodeDumpString(const std::string &str) { + std::stringstream ret; + for (unsigned int pos = 0; pos < str.length(); pos++) { + unsigned char c = str[pos]; + if (c == '%' && pos+2 < str.length()) { + c = (((str[pos+1]>>6)*9+((str[pos+1]-'0')&15)) << 4) | + ((str[pos+2]>>6)*9+((str[pos+2]-'0')&15)); + pos += 2; + } + ret << c; + } + return ret.str(); +} + +Value importprivkey(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 3) + throw runtime_error("Help message not found\n"); + + EnsureWalletIsUnlocked(); + + string strLabel = ""; + if (params.size() > 1) + strLabel = params[1].get_str(); + + // Whether to perform rescan after import + bool fRescan = true; + if (params.size() > 2) + fRescan = params[2].get_bool(); + + + bool fNewFound=false; + vector inputStrings=ParseStringList(params[0]); + vector inputKeys; + vector inputPubKeys; + vector inputKeyIDs; + + for(int is=0;is<(int)inputStrings.size();is++) + { + string strSecret=inputStrings[is]; + + CBitcoinSecret vchSecret; + bool fGood = vchSecret.SetString(strSecret); + + if (!fGood) throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid private key encoding"); + + CKey key = vchSecret.GetKey(); + if (!key.IsValid()) throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key outside allowed range"); + + CPubKey pubkey = key.GetPubKey(); + assert(key.VerifyPubKey(pubkey)); + CKeyID vchAddress = pubkey.GetID(); + + inputKeys.push_back(key); + inputPubKeys.push_back(pubkey); + inputKeyIDs.push_back(vchAddress); + + if (!pwalletMain->HaveKey(vchAddress)) + { + fNewFound=true; + } + } + + if(!fNewFound) + { + return Value::null; + } + + pwalletMain->MarkDirty(); + + for(int is=0;is<(int)inputStrings.size();is++) + { + CKey key = inputKeys[is]; + CPubKey pubkey = inputPubKeys[is]; + CKeyID vchAddress = inputKeyIDs[is]; + + pwalletMain->SetAddressBook(vchAddress, strLabel, "receive"); + + // Don't throw error in case a key is already there + if (!pwalletMain->HaveKey(vchAddress)) + { + pwalletMain->mapKeyMetadata[vchAddress].nCreateTime = 1; + + if (!pwalletMain->AddKeyPubKey(key, pubkey)) + throw JSONRPCError(RPC_WALLET_ERROR, "Error adding key to wallet"); + + // whenever a key is imported, we need to scan the whole chain + pwalletMain->nTimeFirstKey = 1; // 0 would be considered 'no value' + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxEntity entity; + const CKeyID& KeyID=pubkey.GetID(); + + memcpy(entity.m_EntityID,&KeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + } + } + } + + if (fRescan) { + pwalletMain->ScanForWalletTransactions(chainActive.Genesis(), true, true); + } + +/* + string strSecret = params[0].get_str(); + CBitcoinSecret vchSecret; + bool fGood = vchSecret.SetString(strSecret); + + if (!fGood) throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid private key encoding"); + + CKey key = vchSecret.GetKey(); + if (!key.IsValid()) throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key outside allowed range"); + + CPubKey pubkey = key.GetPubKey(); + assert(key.VerifyPubKey(pubkey)); + CKeyID vchAddress = pubkey.GetID(); + { + pwalletMain->MarkDirty(); + pwalletMain->SetAddressBook(vchAddress, strLabel, "receive"); + + // Don't throw error in case a key is already there + if (pwalletMain->HaveKey(vchAddress)) + return Value::null; + + pwalletMain->mapKeyMetadata[vchAddress].nCreateTime = 1; + + if (!pwalletMain->AddKeyPubKey(key, pubkey)) + throw JSONRPCError(RPC_WALLET_ERROR, "Error adding key to wallet"); + + // whenever a key is imported, we need to scan the whole chain + pwalletMain->nTimeFirstKey = 1; // 0 would be considered 'no value' + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxEntity entity; + const CKeyID& KeyID=pubkey.GetID(); + + memcpy(entity.m_EntityID,&KeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + } + + if (fRescan) { + pwalletMain->ScanForWalletTransactions(chainActive.Genesis(), true, true); + } + } +*/ + return Value::null; +} + +Value importaddress(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 3) + throw runtime_error("Help message not found\n"); + + CScript script; + + string strLabel = ""; + if (params.size() > 1) + strLabel = params[1].get_str(); + + // Whether to perform rescan after import + bool fRescan = true; + if (params.size() > 2) + fRescan = params[2].get_bool(); + + bool fNewFound=false; + vector inputStrings=ParseStringList(params[0]); + vector inputAddresses; + vector inputScripts; + + for(int is=0;is<(int)inputStrings.size();is++) + { + string param=inputStrings[is]; + + CBitcoinAddress address(param); + if (address.IsValid()) { + script = GetScriptForDestination(address.Get()); + } else if (IsHex(param)) { + std::vector data(ParseHex(param)); + script = CScript(data.begin(), data.end()); + } else { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid MultiChain address or script"); + } + + inputAddresses.push_back(address); + inputScripts.push_back(script); + + if (::IsMine(*pwalletMain, script) == ISMINE_SPENDABLE) + throw JSONRPCError(RPC_WALLET_ERROR, "The wallet already contains the private key for this address or script"); + + if (!pwalletMain->HaveWatchOnly(script)) + { + fNewFound=true; + } + } + + if(!fNewFound) + { + return Value::null; + } + + pwalletMain->MarkDirty(); + + for(int is=0;is<(int)inputStrings.size();is++) + { + CBitcoinAddress address=inputAddresses[is]; + script=inputScripts[is]; + + // add to address book or update label + if (address.IsValid()) + { + pwalletMain->SetAddressBook(address.Get(), strLabel, "receive"); + } + else + { + CScriptID innerID(script); + address=CBitcoinAddress(innerID); + } + + // Don't throw error in case an address is already there + if (!pwalletMain->HaveWatchOnly(script)) + { + if (!pwalletMain->AddWatchOnly(script)) + throw JSONRPCError(RPC_WALLET_ERROR, "Error adding address to wallet"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxEntity entity; + CTxDestination addressRet=address.Get(); + const CKeyID *lpKeyID=boost::get (&addressRet); + const CScriptID *lpScriptID=boost::get (&addressRet); + + if(lpKeyID) + { + memcpy(entity.m_EntityID,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + } + if(lpScriptID) + { + memcpy(entity.m_EntityID,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + } + } + } + } + + { +/* + // add to address book or update label + if (address.IsValid()) + pwalletMain->SetAddressBook(address.Get(), strLabel, "receive"); + + // Don't throw error in case an address is already there + if (pwalletMain->HaveWatchOnly(script)) + return Value::null; + + pwalletMain->MarkDirty(); + + if (!pwalletMain->AddWatchOnly(script)) + throw JSONRPCError(RPC_WALLET_ERROR, "Error adding address to wallet"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxEntity entity; + CTxDestination addressRet=address.Get(); + const CKeyID *lpKeyID=boost::get (&addressRet); + const CScriptID *lpScriptID=boost::get (&addressRet); + + if(lpKeyID) + { + memcpy(entity.m_EntityID,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + } + if(lpScriptID) + { + memcpy(entity.m_EntityID,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + } + } +*/ + if (fRescan) + { + pwalletMain->ScanForWalletTransactions(chainActive.Genesis(), true, true); + pwalletMain->ReacceptWalletTransactions(); + } + } + + return Value::null; +} + +Value importwallet(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + EnsureWalletIsUnlocked(); + + ifstream file; + file.open(params[0].get_str().c_str(), std::ios::in | std::ios::ate); + if (!file.is_open()) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Cannot open wallet dump file"); + + int64_t nTimeBegin = chainActive.Tip()->GetBlockTime(); + + bool fGood = true; + + int64_t nFilesize = std::max((int64_t)1, (int64_t)file.tellg()); + file.seekg(0, file.beg); + + pwalletMain->ShowProgress(_("Importing..."), 0); // show progress dialog in GUI + while (file.good()) { + pwalletMain->ShowProgress("", std::max(1, std::min(99, (int)(((double)file.tellg() / (double)nFilesize) * 100)))); + std::string line; + std::getline(file, line); + if (line.empty() || line[0] == '#') + continue; + + std::vector vstr; + boost::split(vstr, line, boost::is_any_of(" ")); + if (vstr.size() < 2) + continue; + CBitcoinSecret vchSecret; + if (!vchSecret.SetString(vstr[0])) + continue; + CKey key = vchSecret.GetKey(); + CPubKey pubkey = key.GetPubKey(); + assert(key.VerifyPubKey(pubkey)); + CKeyID keyid = pubkey.GetID(); + if (pwalletMain->HaveKey(keyid)) { + LogPrintf("Skipping import of %s (key already present)\n", CBitcoinAddress(keyid).ToString()); + continue; + } + int64_t nTime = DecodeDumpTime(vstr[1]); + std::string strLabel; + bool fLabel = true; + for (unsigned int nStr = 2; nStr < vstr.size(); nStr++) { + if (boost::algorithm::starts_with(vstr[nStr], "#")) + break; + if (vstr[nStr] == "change=1") + fLabel = false; + if (vstr[nStr] == "reserve=1") + fLabel = false; + if (boost::algorithm::starts_with(vstr[nStr], "label=")) { + strLabel = DecodeDumpString(vstr[nStr].substr(6)); + fLabel = true; + } + } + LogPrintf("Importing %s...\n", CBitcoinAddress(keyid).ToString()); + if (!pwalletMain->AddKeyPubKey(key, pubkey)) { + fGood = false; + continue; + } + pwalletMain->mapKeyMetadata[keyid].nCreateTime = nTime; + if (fLabel) + pwalletMain->SetAddressBook(keyid, strLabel, "receive"); + +/* MCHN START */ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxEntity entity; + const CKeyID& KeyID=pubkey.GetID(); + + memcpy(entity.m_EntityID,&KeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + } +/* MCHN END */ + + nTimeBegin = std::min(nTimeBegin, nTime); + } + file.close(); + pwalletMain->ShowProgress("", 100); // hide progress dialog in GUI + + CBlockIndex *pindex = chainActive.Tip(); + while (pindex && pindex->pprev && pindex->GetBlockTime() > nTimeBegin - 12 * Params().TargetSpacing()) // MCHN + pindex = pindex->pprev; + + if (!pwalletMain->nTimeFirstKey || nTimeBegin < pwalletMain->nTimeFirstKey) + pwalletMain->nTimeFirstKey = nTimeBegin; + +/* MCHN START */ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + LogPrintf("Rescanning all %i blocks\n", chainActive.Height()); + pwalletMain->ScanForWalletTransactions(chainActive.Genesis(),false,true); + pwalletMain->MarkDirty(); + } + else + { + LogPrintf("Rescanning last %i blocks\n", chainActive.Height() - pindex->nHeight + 1); + pwalletMain->ScanForWalletTransactions(pindex,false,true); + pwalletMain->MarkDirty(); + } + + if (!fGood) + throw JSONRPCError(RPC_WALLET_ERROR, "Error adding some keys to wallet"); + + return Value::null; +} + +Value dumpprivkey(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + EnsureWalletIsUnlocked(); + + string strAddress = params[0].get_str(); + CBitcoinAddress address; + if (!address.SetString(strAddress)) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid MultiChain address"); + CKeyID keyID; + if (!address.GetKeyID(keyID)) + throw JSONRPCError(RPC_TYPE_ERROR, "Address does not refer to a key"); + CKey vchSecret; + if (!pwalletMain->GetKey(keyID, vchSecret)) + throw JSONRPCError(RPC_WALLET_ERROR, "Private key for address " + strAddress + " is not known"); + return CBitcoinSecret(vchSecret).ToString(); +} + + +Value dumpwallet(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + EnsureWalletIsUnlocked(); + + ofstream file; + file.open(params[0].get_str().c_str()); + if (!file.is_open()) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Cannot open wallet dump file"); + + std::map mapKeyBirth; + std::set setKeyPool; + pwalletMain->GetKeyBirthTimes(mapKeyBirth); + pwalletMain->GetAllReserveKeys(setKeyPool); + + // sort time/key pairs + std::vector > vKeyBirth; + for (std::map::const_iterator it = mapKeyBirth.begin(); it != mapKeyBirth.end(); it++) { + vKeyBirth.push_back(std::make_pair(it->second, it->first)); + } + mapKeyBirth.clear(); + std::sort(vKeyBirth.begin(), vKeyBirth.end()); + + // produce output + file << strprintf("# Wallet dump created by MultiChain %s (%s)\n", CLIENT_BUILD, CLIENT_DATE); + file << strprintf("# * Created on %s\n", EncodeDumpTime(GetTime())); + file << strprintf("# * Best block at time of backup was %i (%s),\n", chainActive.Height(), chainActive.Tip()->GetBlockHash().ToString()); + file << strprintf("# mined on %s\n", EncodeDumpTime(chainActive.Tip()->GetBlockTime())); + file << "\n"; + for (std::vector >::const_iterator it = vKeyBirth.begin(); it != vKeyBirth.end(); it++) { + const CKeyID &keyid = it->second; + std::string strTime = EncodeDumpTime(it->first); + std::string strAddr = CBitcoinAddress(keyid).ToString(); + CKey key; + if (pwalletMain->GetKey(keyid, key)) { + if (pwalletMain->mapAddressBook.count(keyid)) { + file << strprintf("%s %s label=%s # addr=%s\n", CBitcoinSecret(key).ToString(), strTime, EncodeDumpString(pwalletMain->mapAddressBook[keyid].name), strAddr); + } else if (setKeyPool.count(keyid)) { + file << strprintf("%s %s reserve=1 # addr=%s\n", CBitcoinSecret(key).ToString(), strTime, strAddr); + } else { + file << strprintf("%s %s change=1 # addr=%s\n", CBitcoinSecret(key).ToString(), strTime, strAddr); + } + } + } + file << "\n"; + file << "# End of dump\n"; + file.close(); + return Value::null; +} diff --git a/src/rpc/rpcexchange.cpp b/src/rpc/rpcexchange.cpp new file mode 100644 index 00000000..3bc61769 --- /dev/null +++ b/src/rpc/rpcexchange.cpp @@ -0,0 +1,830 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + + +#include "rpc/rpcwallet.h" +#include "script/sign.h" + +bool AcceptExchange(const CTransaction& tx, vector & inputs, vector & errors,string& reason) +{ + if(tx.vout.size() < tx.vin.size()) + { + reason="Too few outputs"; + return false; + } + + return GetTxInputsAsTxOuts(tx, inputs, errors, reason); +} + +Object ExchangeAssetEntry(uint256 hash,const CTxOut txout,mc_Script *lpScript,mc_Buffer *asset_amounts,mc_Buffer *total_amounts,bool& can_disable,string& strError) +{ + Object result; + CBitcoinAddress address; + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + bool is_input=false; + bool address_found=false; + + if(lpScript->m_Size) + { + is_input=true; + } + + if(is_input) + { + if(lpScript->GetNumElements() == 2) + { + size_t elem_size; + const unsigned char *elem; + unsigned char hash_type; + + elem = lpScript->GetData(0,&elem_size); + + if(elem_size>1) + { + hash_type=elem[elem_size-1]; + + if(hash_type != (SIGHASH_SINGLE | SIGHASH_ANYONECANPAY)) + { + strError="Signature hash type should be SINGLE | SIGHASH_ANYONECANPAY"; + } + + elem = lpScript->GetData(1,&elem_size); + uint160 addr=Hash160(elem,elem+elem_size); + const unsigned char *pubkey_hash=(unsigned char *)&addr; + address=CBitcoinAddress(*(CKeyID*)pubkey_hash); + address_found=true; + + if( (pwalletMain->IsMine(txout) & ISMINE_SPENDABLE) != ISMINE_NO ) + { + can_disable=true; + } + + if(mc_gState->m_Permissions->CanSend(NULL,pubkey_hash) == 0) + { + if(strError.size() == 0) + { + strError="Address doesn't have send permission"; + } + } + } + else // Multisig with one signature + { + if(strError.size() == 0) + { + strError="Only pay-to-keyhash addresses are supported in exchange"; + } + } + } + else + { + if(strError.size() == 0) + { + strError="Only pay-to-keyhash addresses are supported in exchange"; + } + } + } + else + { + CTxDestination addressRet; + + if(!ExtractDestination(txout.scriptPubKey, addressRet)) + { + if(strError.size() == 0) + { + strError="Only pay-to-keyhash addresses are supported in exchange"; + } + } + else + { + CKeyID *lpKeyID=boost::get (&addressRet); + if(lpKeyID != NULL) + { + address=CBitcoinAddress(*lpKeyID); + address_found=true; + } + else + { + if(strError.size() == 0) + { + strError="Only pay-to-keyhash addresses are supported in exchange"; + } + } + if(mc_gState->m_Permissions->CanReceive(NULL,(unsigned char*)(lpKeyID)) == 0) + { + if(strError.size() == 0) + { + strError="Address doesn't have receive permission"; + } + } + } + } + + Array assets; + + asset_amounts->Clear(); + if(!CreateAssetBalanceList(txout,asset_amounts,lpScript)) + { + if(strError.size() == 0) + { + strError="Wrong asset transfer script"; + } + } + else + { + for(int a=0;aGetCount();a++) + { + Object asset_entry; + unsigned char *ptr=(unsigned char *)asset_amounts->GetRow(a); + + mc_EntityDetails entity; + const unsigned char *txid; + + int64_t quantity,total; + + quantity=(int64_t)mc_GetABQuantity(ptr); + + total=quantity; + if(!is_input) + { + total=-quantity; + } + + if(total_amounts) + { + int row=total_amounts->Seek(ptr); + if(row >= 0) + { + total+=(int64_t)mc_GetABQuantity(total_amounts->GetRow(row)); + mc_SetABQuantity(total_amounts->GetRow(row),total); + } + else + { + memcpy(buf,ptr,MC_AST_ASSET_QUANTITY_OFFSET); + mc_SetABQuantity(buf,total); + total_amounts->Add(buf); + } + } + + + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ptr)) + { + txid=entity.GetTxID(); + asset_entry=AssetEntry(txid,quantity,1); + assets.push_back(asset_entry); + } + } + } + + result.push_back(Pair("amount", ValueFromAmount(txout.nValue))); + result.push_back(Pair("assets", assets)); + if(address_found) + { + result.push_back(Pair("address", address.ToString())); + } + else + { + result.push_back(Pair("address", "")); + } + +// result.push_back(Pair("error", strError)); + return result; +} + +Object DecodeExchangeTransaction(const CTransaction tx,int verbose,int64_t& native_balance,mc_Buffer *lpAssets,bool& is_complete,string& strError) +{ + Object result; + strError=""; + native_balance=0; + bool can_disable=false; + + vector input_txouts; + vector input_errors; + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + asset_amounts->Clear(); + + mc_Script *lpScript; + lpScript=new mc_Script; + + AcceptExchange(tx, input_txouts, input_errors,strError); + + if(strError.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + + native_balance=0; + Array exchanges; + + for(int i=0;i<(int)input_txouts.size();i++) + { + Object exchange; + string entry_error; + + + const CScript& script2 = tx.vin[i].scriptSig; + CScript::const_iterator pc2 = script2.begin(); + + lpScript->Clear(); + lpScript->SetScript((unsigned char*)(&pc2[0]),(size_t)(script2.end()-pc2),MC_SCR_TYPE_SCRIPTSIG); + + if((input_errors[i].size() > 0) && (input_txouts[i].nValue < 0) ) + { + input_txouts[i].nValue=0; + } + + native_balance+=(int64_t)input_txouts[i].nValue-(int64_t)tx.vout[i].nValue; + + entry_error=input_errors[i]; + + Object offer=ExchangeAssetEntry(tx.vin[i].prevout.hash,input_txouts[i],lpScript,asset_amounts,lpAssets,can_disable,entry_error); + + offer.push_back(Pair("txid", tx.vin[i].prevout.hash.ToString())); + offer.push_back(Pair("vout", (uint64_t)(tx.vin[i].prevout.n))); + + exchange.push_back(Pair("offer", offer)); + if(entry_error.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, entry_error + strprintf("; Input: %d, txid: %s, vout: %d",i,tx.vin[i].prevout.hash.GetHex().c_str(),tx.vin[i].prevout.n)); + } + + entry_error=""; + lpScript->Clear(); + exchange.push_back(Pair("ask", ExchangeAssetEntry(0,tx.vout[i],lpScript,asset_amounts,lpAssets,can_disable,entry_error))); + if(entry_error.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, entry_error+ strprintf("; Output: %d,",i)); + } + +// exchange.push_back(Pair("amount",ValueFromAmount((int64_t)input_txouts[i].nValue-(int64_t)tx.vout[i].nValue))); + exchanges.push_back(exchange); + } + + + + delete lpScript; + delete asset_amounts; + + is_complete=true; + if(lpAssets != NULL) + { + Object tocomplete; + Array offer; + Array ask; + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + int multiple=1; + + for(int a=0;aGetCount();a++) + { + Object asset_entry; + unsigned char *ptr=(unsigned char *)lpAssets->GetRow(a); + + mc_EntityDetails entity; + const unsigned char *txid; + + int64_t quantity; + + quantity=(int64_t)mc_GetABQuantity(ptr); + + if(quantity != 0) + { + is_complete=false; + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ptr)) + { + txid=entity.GetTxID(); + if(quantity > 0) + { + asset_entry=AssetEntry(txid,quantity,1); + offer.push_back(asset_entry); + } + else + { + quantity=-quantity; + asset_entry=AssetEntry(txid,quantity,1); + ask.push_back(asset_entry); + uint256 hash; + hash=*(uint256*)txid; + +// ParseAssetKey(hash.GetHex().c_str(),NULL,buf,NULL,&multiple,NULL, MC_ENT_TYPE_ASSET); + ParseAssetKeyToFullAssetRef(hash.GetHex().c_str(),buf,&multiple,NULL, MC_ENT_TYPE_ASSET); + tocomplete.push_back(Pair(hash.GetHex(),(double)quantity/double(multiple))); + } + } + } + } + +/* + tocomplete.push_back(Pair("offer", offer)); + tocomplete.push_back(Pair("ask", ask)); + tocomplete.push_back(Pair("amount", ValueFromAmount(-native_balance))); +*/ + Object obj_offer; + Object obj_ask; + + CAmount required_fee=0; + int64_t dust=0; + + unsigned int nBytes = ::GetSerializeSize(tx, SER_NETWORK, PROTOCOL_VERSION); + + CWallet::minTxFee = CFeeRate(MIN_RELAY_TX_FEE); + ::minRelayTxFee = CFeeRate(MIN_RELAY_TX_FEE); + + required_fee=pwalletMain->GetMinimumFee(nBytes,1,mempool); + + if(!is_complete || (native_balancem_NetworkParams->m_AssetRefSize+MC_AST_ASSET_QUANTITY_SIZE)+1+32; + required_fee=pwalletMain->GetMinimumFee(nBytes,1,mempool); + dust=-1; + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + dust=mc_gState->m_NetworkParams->GetInt64Param("minimumperoutput"); + } + if(dust<0) + { + size_t nSize = GetSerializeSize(SER_DISK,0)+148u; + dust=3*minRelayTxFee.GetFee(nSize); + } + } + +// native_balance-=required_fee; + + if(native_balance >= 0) + { + obj_offer.push_back(Pair("amount", ValueFromAmount(native_balance))); + obj_ask.push_back(Pair("amount", ValueFromAmount(0))); + } + else + { + is_complete=false; + obj_offer.push_back(Pair("amount", ValueFromAmount(0))); + obj_ask.push_back(Pair("amount", ValueFromAmount(-native_balance))); + } + + obj_offer.push_back(Pair("assets", offer)); + obj_ask.push_back(Pair("assets", ask)); + + result.push_back(Pair("offer", obj_offer)); + result.push_back(Pair("ask", obj_ask)); + + + if(is_complete) + { + result.push_back(Pair("completedfee", ValueFromAmount(native_balance))); + } + result.push_back(Pair("requiredfee", ValueFromAmount(required_fee))); + +// result.push_back(Pair("tocomplete", tocomplete)); + + result.push_back(Pair("candisable", can_disable)); + +// result.push_back(Pair("tocomplete", tocomplete)); + if(is_complete) + { + if(native_balanceGetBalance()>=required_fee-native_balance) + { + result.push_back(Pair("cancomplete", true)); + } + else + { + result.push_back(Pair("cancomplete", false)); + } + } + else + { + result.push_back(Pair("cancomplete", true)); + } + } + else + { + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_SEND | MC_PTP_RECEIVE)) + { + result.push_back(Pair("cancomplete", false)); + } + else + { + CTxDestination address=CTxDestination(pkey.GetID()); + + mc_Script *lpScript; + lpScript=new mc_Script; + + Value param=tocomplete; + uint256 offer_hash; + CAmount nAmount; + string strError=ParseRawOutputObject(param,nAmount,lpScript); + if(strError.size()) + { + throw JSONRPCError(RPC_INTERNAL_ERROR, strError); + } + + nAmount=-native_balance+dust+required_fee; + if(nAmount<2*dust) + { + nAmount=2*dust; + } + + CScript scriptPubKey = GetScriptForDestination(address); + + size_t elem_size; + const unsigned char *elem; + + for(int element=0;element < lpScript->GetNumElements();element++) + { + elem = lpScript->GetData(element,&elem_size); + if(elem) + { + scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + } + } + + CScript scriptOpReturn=CScript(); + + CReserveKey reservekey(pwalletMain); + CAmount nFeeRequired; + CWalletTx wtxNew; + + { + LOCK (pwalletMain->cs_wallet_send); + + if (!pwalletMain->CreateTransaction(scriptPubKey, nAmount, scriptOpReturn, wtxNew, reservekey, nFeeRequired, strError)) + { + result.push_back(Pair("cancomplete", false)); + } + else + { + result.push_back(Pair("cancomplete", true)); + } + } + + delete lpScript; + } + } + + result.push_back(Pair("complete", is_complete)); + } + + if(verbose) + { + result.push_back(Pair("exchanges", exchanges)); + } + +// result.push_back(Pair("error", strError)); +// result.push_back(Pair("valid", is_valid)); + return result; +} + + + +string FindExchangeOutPoint(const json_spirit::Array& params,int first_param,COutPoint& offer_input,CAmount& nAmount,mc_Script *lpScript) +{ + string strError=""; + + uint256 offer_hash; + +// bool offer_input_found=false; + +// uint32_t ts=0; + uint256 given_hash=0; + int given_vout=-1; +// if(params.size() >= 4) + { + given_hash.SetHex(params[first_param+0].get_str()); + given_vout=params[first_param+1].get_int(); + } + + offer_input=COutPoint(given_hash,given_vout); + + uint256 ask_hash; + nAmount=0; + lpScript->Clear(); + + if (params[first_param+2].type() != obj_type) + { + strError= "Invalid ask assets object"; + return strError; + } + else + { + string strError=ParseRawOutputObject(params[first_param+2],nAmount,lpScript); + if(strError.size()) + { + return strError; + } + CTxOut tmpTxOut=CTxOut(nAmount,CScript()); + if (tmpTxOut.IsDust(::minRelayTxFee)) + { + strError="Ask amount too small"; + return strError; + } + + } + + return strError; +} + +Value createrawexchange(const json_spirit::Array& params, bool fHelp) +{ + if (fHelp || params.size() != 3) + throw runtime_error("Help message not found\n"); + + + COutPoint offer_input; + mc_Script *lpScript; + lpScript=new mc_Script; + CAmount nAmount=0; + + string strError=FindExchangeOutPoint(params,0,offer_input,nAmount,lpScript); + if(strError.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + + CTxOut preparedTxOut; + if(!FindPreparedTxOut(preparedTxOut,offer_input,strError)) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + + const CScript& script1 = preparedTxOut.scriptPubKey; + CTxDestination addressRet; + if(!ExtractDestination(script1, addressRet)) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Cannot extract address from prepared output"); + } + + CKeyID *lpKeyID=boost::get (&addressRet); + if(lpKeyID == NULL) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Prepared output should be pay-to-pubkeyhash"); + } + + if(mc_gState->m_Permissions->CanSend(NULL,(unsigned char*)(lpKeyID)) == 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Address of prepared output doesn't have send permission"); + } + + if(mc_gState->m_Permissions->CanReceive(NULL,(unsigned char*)(lpKeyID)) == 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Address of prepared output doesn't have receive permission"); + } + + vector addresses; + addresses.push_back(addressRet); + + + + CScript scriptPubKey = GetScriptForDestination(addresses[0]); + + for(int element=0;element < lpScript->GetNumElements();element++) + { + size_t elem_size; + const unsigned char *elem; + elem = lpScript->GetData(element,&elem_size); + if(elem) + { + scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + } + else + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid script"); + } + + CMutableTransaction tx; + + CTxOut txout(nAmount, scriptPubKey); + + tx.vout.push_back(txout); + tx.vin.push_back(CTxIn(offer_input)); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + int err; + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(offer_input.hash,NULL,&err); + if(err) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Output not found in wallet"); + } + if(!SignSignature(*pwalletMain, wtx.vout[offer_input.n].scriptPubKey, tx, tx.vin.size()-1, SIGHASH_SINGLE | SIGHASH_ANYONECANPAY )) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Signing transaction failed"); + } + } + else + { + std::map::const_iterator it = pwalletMain->mapWallet.find(offer_input.hash); + + if (it == pwalletMain->mapWallet.end()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Output not found in wallet"); + } + + const CWalletTx* pcoin = &(*it).second; + + + if(!SignSignature(*pwalletMain, pcoin->vout[offer_input.n].scriptPubKey, tx, tx.vin.size()-1, SIGHASH_SINGLE | SIGHASH_ANYONECANPAY )) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Signing transaction failed"); + } + } + + delete lpScript; + + + return EncodeHexTx(tx); +} + + +Value appendrawexchange(const json_spirit::Array& params, bool fHelp) +{ + if (fHelp || params.size() != 4 ) + throw runtime_error("Help message not found\n"); + + COutPoint offer_input; + mc_Script *lpScript; + lpScript=new mc_Script; + CAmount nAmount=0; + + + string strError=FindExchangeOutPoint(params,1,offer_input,nAmount,lpScript); + if(strError.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + + CTxOut preparedTxOut; + if(!FindPreparedTxOut(preparedTxOut,offer_input,strError)) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + + const CScript& script1 = preparedTxOut.scriptPubKey; + CTxDestination addressRet; + if(!ExtractDestination(script1, addressRet)) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Cannot extract address from prepared output"); + } + + CKeyID *lpKeyID=boost::get (&addressRet); + if(lpKeyID == NULL) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Prepared output should be pay-to-pubkeyhash"); + } + + if(mc_gState->m_Permissions->CanSend(NULL,(unsigned char*)(lpKeyID)) == 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Address of prepared output doesn't have send permission"); + } + + if(mc_gState->m_Permissions->CanReceive(NULL,(unsigned char*)(lpKeyID)) == 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Address of prepared output doesn't have receive permission"); + } + + vector addresses; + addresses.push_back(addressRet); + + CScript scriptPubKey = GetScriptForDestination(addresses[0]); + + for(int element=0;element < lpScript->GetNumElements();element++) + { + size_t elem_size; + const unsigned char *elem; + elem = lpScript->GetData(element,&elem_size); + if(elem) + { + scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + } + else + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid script"); + } + + CMutableTransaction tx; + + vector txData(ParseHex(params[0].get_str())); + + CDataStream ssData(txData, SER_NETWORK, PROTOCOL_VERSION); + ssData >> tx; + + CTxOut txout(nAmount, scriptPubKey); + + tx.vout.push_back(txout); + tx.vin.push_back(CTxIn(offer_input)); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + int err; + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(offer_input.hash,NULL,&err); + if(err) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Output not found in wallet"); + } + if(!SignSignature(*pwalletMain, wtx.vout[offer_input.n].scriptPubKey, tx, tx.vin.size()-1, SIGHASH_SINGLE | SIGHASH_ANYONECANPAY )) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Signing transaction failed"); + } + } + else + { + std::map::const_iterator it = pwalletMain->mapWallet.find(offer_input.hash); + + if (it == pwalletMain->mapWallet.end()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Output not found in wallet"); + } + + const CWalletTx* pcoin = &(*it).second; + + + if(!SignSignature(*pwalletMain, pcoin->vout[offer_input.n].scriptPubKey, tx, tx.vin.size()-1, SIGHASH_SINGLE | SIGHASH_ANYONECANPAY )) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Signing transaction failed"); + } + } + delete lpScript; + + bool is_complete=true; + int64_t native_balance; + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + asset_amounts->Clear(); + + { + LOCK(cs_main); + Object decode_result; + + decode_result=DecodeExchangeTransaction(tx,0,native_balance,asset_amounts,is_complete,strError); + } + + delete asset_amounts; + + Object result; + result.push_back(Pair("hex", EncodeHexTx(tx))); + result.push_back(Pair("complete", is_complete)); + + return result; +} + +Value completeexchange(const json_spirit::Array& params, bool fHelp) +{ + return ""; +} + +Value decoderawexchange(const json_spirit::Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1) + throw runtime_error("Help message not found\n"); + + CTransaction tx; + + if (!DecodeHexTx(tx, params[0].get_str())) + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "TX decode failed"); + + int verbose=0; + if (params.size() > 1) + { + if(params[1].type() == int_type) + { + verbose=params[1].get_int(); + } + if(params[1].type() == bool_type) + { + if(params[1].get_bool()) + { + verbose=1; + } + } + } + + + Object result; + bool is_complete=true; + int64_t native_balance; + string strError; + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + asset_amounts->Clear(); + + { + LOCK(cs_main); + + result=DecodeExchangeTransaction(tx,verbose,native_balance,asset_amounts,is_complete,strError); + } + + delete asset_amounts; + + return result; +} + + diff --git a/src/rpc/rpchelp.cpp b/src/rpc/rpchelp.cpp new file mode 100644 index 00000000..671e1cf6 --- /dev/null +++ b/src/rpc/rpchelp.cpp @@ -0,0 +1,3473 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + + +#include "core/main.h" +#include "rpc/rpcserver.h" +#include "rpc/rpcutils.h" + +std::string HelpRequiringPassphraseWrapper() +{ +#ifdef ENABLE_WALLET + return HelpRequiringPassphrase() + "\n"; +#else + return ""; +#endif +} + +std::string mc_RPCHelpString(std::string strMethod) +{ + string strHelp=""; + map::iterator it = mapHelpStrings.find(strMethod); + if (it != mapHelpStrings.end()) + { + strHelp=it->second; + } + else + { + throw runtime_error("Help message not found\n"); + } + + return strHelp; +} + + +void mc_InitRPCHelpMap01() +{ + mapHelpStrings.insert(std::make_pair("getbestblockhash", + "getbestblockhash\n" + "\nReturns the hash of the best (tip) block in the longest block chain.\n" + "\nResult\n" + "\"hex\" (string) the block hash hex encoded\n" + "\nExamples\n" + + HelpExampleCli("getbestblockhash", "") + + HelpExampleRpc("getbestblockhash", "") + )); + + mapHelpStrings.insert(std::make_pair("getblock", + "getblock hash|height ( verbose )\n" + "\nReturns hex-encoded data or json object for block.\n" + "\nArguments:\n" + "1. hash|height (string, required) The block hash or block height in active chain\n" + "2. verbose (numeric or boolean, optional, default=1) 0(or false) - encoded data, 1(or true) - json object, 2 - with tx encoded data, 4 - with tx json object\n" + "\nResult (for verbose = 1, see help getrawtransaction for details about transactions - verbose = 4):\n" + "{\n" + " \"hash\" : \"hash\", (string) the block hash (same as provided)\n" + " \"miner\" : \"miner\", (string) the address of the miner\n" + " \"confirmations\" : n, (numeric) The number of confirmations, or -1 if the block is not on the main chain\n" + " \"size\" : n, (numeric) The block size\n" + " \"height\" : n, (numeric) The block height or index\n" + " \"version\" : n, (numeric) The block version\n" + " \"merkleroot\" : \"xxxx\", (string) The merkle root\n" + " \"tx\" : [ (array of string) The transaction ids\n" + " \"transactionid\" (string) The transaction id\n" + " ,...\n" + " ],\n" + " \"time\" : ttt, (numeric) The block time in seconds since epoch (Jan 1 1970 GMT)\n" + " \"nonce\" : n, (numeric) The nonce\n" + " \"bits\" : \"1d00ffff\", (string) The bits\n" + " \"difficulty\" : x.xxx, (numeric) The difficulty\n" + " \"previousblockhash\" : \"hash\", (string) The hash of the previous block\n" + " \"nextblockhash\" : \"hash\" (string) The hash of the next block\n" + "}\n" + "\nResult (for verbose=false):\n" + "\"data\" (string) A string that is serialized, hex-encoded data for block 'hash'.\n" + "\nExamples:\n" + + HelpExampleCli("getblock", "\"00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09\"") + + HelpExampleRpc("getblock", "\"00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09\"") + )); + + mapHelpStrings.insert(std::make_pair("getblockchaininfo", + "getblockchaininfo\n" + "Returns an object containing various state info regarding block chain processing.\n" + "\nResult:\n" + "{\n" + " \"chain\": \"xxxx\", (string) current network name as defined in BIP70 (main, test, regtest)\n" + " \"chainname\": \"xxxx\", (string) multichain network name\n" + " \"description\": \"xxxx\", (string) network desctription\n" + " \"protocol\": \"xxxx\", (string) protocol - multichain or bitcoin\n" + " \"setupblocks\": \"xxxx\", (string) number of network setup blocks\n" + " \"blocks\": xxxxxx, (numeric) the current number of blocks processed in the server\n" + " \"headers\": xxxxxx, (numeric) the current number of headers we have validated\n" + " \"bestblockhash\": \"...\", (string) the hash of the currently best block\n" + " \"difficulty\": xxxxxx, (numeric) the current difficulty\n" + " \"verificationprogress\": xxxx, (numeric) estimate of verification progress [0..1]\n" + " \"chainwork\": \"xxxx\" (string) total amount of work in active chain, in hexadecimal\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("getblockchaininfo", "") + + HelpExampleRpc("getblockchaininfo", "") + )); + + mapHelpStrings.insert(std::make_pair("getblockcount", + "getblockcount\n" + "\nReturns the number of blocks in the longest block chain.\n" + "\nResult:\n" + "n (numeric) The current block count\n" + "\nExamples:\n" + + HelpExampleCli("getblockcount", "") + + HelpExampleRpc("getblockcount", "") + )); + + mapHelpStrings.insert(std::make_pair("getblockhash", + "getblockhash index\n" + "\nReturns hash of block in best-block-chain at index provided.\n" + "\nArguments:\n" + "1. index (numeric, required) The block index\n" + "\nResult:\n" + "\"hash\" (string) The block hash\n" + "\nExamples:\n" + + HelpExampleCli("getblockhash", "1000") + + HelpExampleRpc("getblockhash", "1000") + )); + + mapHelpStrings.insert(std::make_pair("getchaintips", + "getchaintips\n" + "Return information about all known tips in the block tree," + " including the main chain as well as orphaned branches.\n" + "\nResult:\n" + "[\n" + " {\n" + " \"height\": xxxx, (numeric) height of the chain tip\n" + " \"hash\": \"xxxx\", (string) block hash of the tip\n" + " \"branchlen\": 0 (numeric) zero for main chain\n" + " \"status\": \"active\" (string) \"active\" for the main chain\n" + " },\n" + " {\n" + " \"height\": xxxx,\n" + " \"hash\": \"xxxx\",\n" + " \"branchlen\": 1 (numeric) length of branch connecting the tip to the main chain\n" + " \"status\": \"xxxx\" (string) status of the chain (active, valid-fork, valid-headers, headers-only, invalid)\n" + " }\n" + "]\n" + "Possible values for status:\n" + "1. \"invalid\" This branch contains at least one invalid block\n" + "2. \"headers-only\" Not all blocks for this branch are available, but the headers are valid\n" + "3. \"valid-headers\" All blocks are available for this branch, but they were never fully validated\n" + "4. \"valid-fork\" This branch is not part of the active chain, but is fully validated\n" + "5. \"active\" This is the tip of the active main chain, which is certainly valid\n" + "\nExamples:\n" + + HelpExampleCli("getchaintips", "") + + HelpExampleRpc("getchaintips", "") + )); + + mapHelpStrings.insert(std::make_pair("getdifficulty", + "getdifficulty\n" + "\nReturns the proof-of-work difficulty as a multiple of the minimum difficulty.\n" + "\nResult:\n" + "n.nnn (numeric) the proof-of-work difficulty as a multiple of the minimum difficulty.\n" + "\nExamples:\n" + + HelpExampleCli("getdifficulty", "") + + HelpExampleRpc("getdifficulty", "") + )); + + mapHelpStrings.insert(std::make_pair("getmempoolinfo", + "getmempoolinfo\n" + "\nReturns details on the active state of the TX memory pool.\n" + "\nResult:\n" + "{\n" + " \"size\": xxxxx (numeric) Current tx count\n" + " \"bytes\": xxxxx (numeric) Sum of all tx sizes\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("getmempoolinfo", "") + + HelpExampleRpc("getmempoolinfo", "") + )); + + mapHelpStrings.insert(std::make_pair("getrawmempool", + "getrawmempool ( verbose )\n" + "\nReturns all transaction ids in memory pool as a json array of string transaction ids.\n" + "\nArguments:\n" + "1. verbose (boolean, optional, default=false) true for a json object, false for array of transaction ids\n" + "\nResult: (for verbose = false):\n" + "[ (json array of string)\n" + " \"transactionid\" (string) The transaction id\n" + " ,...\n" + "]\n" + "\nResult: (for verbose = true):\n" + "{ (json object)\n" + " \"transactionid\" : { (json object)\n" + " \"size\" : n, (numeric) transaction size in bytes\n" + " \"fee\" : n, (numeric) transaction fee in native currency units\n" + " \"time\" : n, (numeric) local time transaction entered pool in seconds since 1 Jan 1970 GMT\n" + " \"height\" : n, (numeric) block height when transaction entered pool\n" + " \"startingpriority\" : n, (numeric) priority when transaction entered pool\n" + " \"currentpriority\" : n, (numeric) transaction priority now\n" + " \"depends\" : [ (array) unconfirmed transactions used as inputs for this transaction\n" + " \"transactionid\", (string) parent transaction id\n" + " ... ]\n" + " }, ...\n" + "]\n" + "\nExamples\n" + + HelpExampleCli("getrawmempool", "true") + + HelpExampleRpc("getrawmempool", "true") + )); + + mapHelpStrings.insert(std::make_pair("gettxout", + "gettxout txid n ( includemempool )\n" + "\nReturns details about an unspent transaction output.\n" + "\nArguments:\n" + "1. txid (string, required) The transaction id\n" + "2. n (numeric, required) vout value\n" + "3. includemempool (boolean, optional) Whether to included the mem pool\n" + "\nResult:\n" + "{\n" + " \"bestblock\" : \"hash\", (string) the block hash\n" + " \"confirmations\" : n, (numeric) The number of confirmations\n" + " \"value\" : x.xxx, (numeric) The transaction value in btc\n" + " \"scriptPubKey\" : { (json object)\n" + " \"asm\" : \"code\", (string) \n" + " \"hex\" : \"hex\", (string) \n" + " \"reqSigs\" : n, (numeric) Number of required signatures\n" + " \"type\" : \"pubkeyhash\", (string) The type, eg pubkeyhash\n" + " \"addresses\" : [ (array of string) array of addresses\n" + " \"address\" (string) address\n" + " ,...\n" + " ]\n" + " },\n" + " \"version\" : n, (numeric) The version\n" + " \"coinbase\" : true|false (boolean) Coinbase or not\n" + "}\n" + + "\nExamples:\n" + "\nGet unspent transactions\n" + + HelpExampleCli("listunspent", "") + + "\nView the details\n" + + HelpExampleCli("gettxout", "\"txid\" 1") + + "\nAs a json rpc call\n" + + HelpExampleRpc("gettxout", "\"txid\", 1") + )); +} + +void mc_InitRPCHelpMap02() +{ + mapHelpStrings.insert(std::make_pair("gettxoutsetinfo", + "gettxoutsetinfo\n" + "\nReturns statistics about the unspent transaction output set.\n" + "Note this call may take some time.\n" + "\nResult:\n" + "{\n" + " \"height\":n, (numeric) The current block height (index)\n" + " \"bestblock\": \"hex\", (string) the best block hash hex\n" + " \"transactions\": n, (numeric) The number of transactions\n" + " \"txouts\": n, (numeric) The number of output transactions\n" + " \"bytes_serialized\": n, (numeric) The serialized size\n" + " \"hash_serialized\": \"hash\", (string) The serialized hash\n" + " \"total_amount\": x.xxx (numeric) The total amount\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("gettxoutsetinfo", "") + + HelpExampleRpc("gettxoutsetinfo", "") + )); + + mapHelpStrings.insert(std::make_pair("listassets", + "listassets (asset-identifier(s) verbose count start )\n" + "1. \"asset-identifier\" (string, optional) Asset identifier - one of the following: issue txid, asset reference, asset name.\n" + "or\n" + "1. asset-identifier(s) (array, optional) A json array of asset identifiers \n" + "2. verbose (boolean, optional, default=false) If true, returns list of all issue transactions, including follow-ons \n" + "3. count (number, optional, default=INT_MAX - all) The number of assets to display\n" + "4. start (number, optional, default=-count - last) Start from specific asset, 0 based, if negative - from the end\n" + "\nReturns list of defined assets\n" + "\nExamples:\n" + + HelpExampleCli("listassets", "") + + HelpExampleRpc("listassets", "") + )); + + mapHelpStrings.insert(std::make_pair("listpermissions", + "listpermissions (\"permission(s)\" \"address(es)\" verbose )\n" + "\nReturns list of addresses having one of the specified permissions\n" + "\nArguments:\n" + "1. \"permission(s)\" (string, optional) Permission strings, comma delimited. Possible values: " + AllowedPermissions() + ". Default: all. \n" + "2. \"address(es)\" (string, optional, default \"*\") The addresses to retrieve permissions for. \"*\" for all addresses\n" + "or\n" + "2. \"address(es)\" (array, optional) A json array of addresses to return permissions for\n" + "3. verbose (boolean, optional, default=false) If true, returns list of pending grants \n" + "\nExamples:\n" + + HelpExampleCli("listpermissions", "connect,send,receive") + + HelpExampleCli("listpermissions", "all \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\"" ) + + HelpExampleRpc("listpermissions", "connect,send,receive") + )); + + mapHelpStrings.insert(std::make_pair("liststreams", + "liststreams (stream-identifier(s) verbose count start )\n" + "1. \"stream-identifier(s)\" (string, optional, default=*, all streams) Stream identifier - one of the following: issue txid, stream reference, stream name.\n" + "or\n" + "1. stream-identifier(s) (array, optional) A json array of stream identifiers \n" + "2. verbose (boolean, optional, default=false) If true, returns stream list of creators \n" + "3. count (number, optional, default=INT_MAX - all) The number of streams to display\n" + "4. start (number, optional, default=-count - last) Start from specific stream, 0 based, if negative - from the end\n" + "\nReturns list of defined streams\n" + "\nExamples:\n" + + HelpExampleCli("liststreams", "") + + HelpExampleRpc("liststreams", "") + )); + + mapHelpStrings.insert(std::make_pair("verifychain", + "verifychain ( checklevel numblocks )\n" + "\nVerifies blockchain database.\n" + "\nArguments:\n" + "1. checklevel (numeric, optional, 0-4, default=3) How thorough the block verification is.\n" + "2. numblocks (numeric, optional, default=288, 0=all) The number of blocks to check.\n" + "\nResult:\n" + "true|false (boolean) Verified or not\n" + "\nExamples:\n" + + HelpExampleCli("verifychain", "") + + HelpExampleRpc("verifychain", "") + )); + + mapHelpStrings.insert(std::make_pair("clearmempool", + "clearmempool \n" + "\nRemoves all transactions from the TX memory pool.\n" + "\nExamples:\n" + + HelpExampleCli("clearmempool", "") + + HelpExampleRpc("clearmempool", "") + )); + + mapHelpStrings.insert(std::make_pair("getblockchainparams", + "getblockchainparams ( displaynames )\n" + "1. displaynames (boolean, optional, default=true) use display names instead of internal\n" + "Returns an object containing various blockchain parameters.\n" + "\nExamples:\n" + + HelpExampleCli("getblockchainparams", "") + + HelpExampleRpc("getblockchainparams", "") + )); + + mapHelpStrings.insert(std::make_pair("getinfo", + "getinfo\n" + "Returns an object containing various state info.\n" + "\nResult:\n" + "{\n" + " \"version\": xxxxx, (numeric) the server version\n" + " \"protocolversion\": xxxxx, (numeric) the protocol version\n" + " \"chainname\": \"xxxx\", (string) multichain network name\n" + " \"description\": \"xxxx\", (string) network desctription\n" + " \"protocol\": \"xxxx\", (string) protocol - multichain or bitcoin\n" + " \"port\": xxxx, (numeric) network port\n" + " \"setupblocks\": \"xxxx\", (string) number of network setup blocks\n" + " \"walletversion\": xxxxx, (numeric) the wallet version\n" + " \"balance\": xxxxxxx, (numeric) the total native currency balance of the wallet\n" + " \"walletdbversion\": xxxxx, (numeric) the wallet database version\n" + " \"blocks\": xxxxxx, (numeric) the current number of blocks processed in the server\n" + " \"timeoffset\": xxxxx, (numeric) the time offset\n" + " \"connections\": xxxxx, (numeric) the number of connections\n" + " \"proxy\": \"host:port\", (string, optional) the proxy used by the server\n" + " \"difficulty\": xxxxxx, (numeric) the current difficulty\n" + " \"testnet\": true|false, (boolean) if the server is using testnet or not\n" + " \"keypoololdest\": xxxxxx, (numeric) the timestamp (seconds since GMT epoch) of the oldest pre-generated key in the key pool\n" + " \"keypoolsize\": xxxx, (numeric) how many new keys are pre-generated\n" + " \"unlocked_until\": ttt, (numeric) the timestamp in seconds since epoch (midnight Jan 1 1970 GMT) that the wallet is unlocked for transfers, or 0 if the wallet is locked\n" + " \"paytxfee\": x.xxxx, (numeric) the transaction fee set in btc/kb\n" + " \"relayfee\": x.xxxx, (numeric) minimum relay fee for non-free transactions in btc/kb\n" + " \"errors\": \"...\" (string) any error messages\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("getinfo", "") + + HelpExampleRpc("getinfo", "") + )); + + mapHelpStrings.insert(std::make_pair("help", + "help ( command )\n" + "\nList all commands, or get help for a specified command.\n" + "\nArguments:\n" + "1. \"command\" (string, optional) The command to get help on\n" + "\nResult:\n" + "\"text\" (string) The help text\n" + )); + + mapHelpStrings.insert(std::make_pair("pause", + "pause \"task(s)\" \n" + "\nPauses local mining or the processing of incoming transactions and blocks.\n" + "\nArguments:\n" + "1. \"task(s)\" (string, required) Task(s) to be paused. Possible values: " + AllowedPausedServices() + " \n" + "\nExamples:\n" + + HelpExampleCli("pause", "incoming,mining") + + HelpExampleRpc("pause", "incoming") + )); + +} + +void mc_InitRPCHelpMap03() +{ + mapHelpStrings.insert(std::make_pair("resume", + "resume \"task(s)\" \n" + "\nResumes local mining or the processing of incoming transactions and blocks\n" + "\nArguments:\n" + "1. \"task(s)\" (string, required) Task(s) to be resumed. Possible values: " + AllowedPausedServices() + " \n" + "\nExamples:\n" + + HelpExampleCli("resume", "incoming,mining") + + HelpExampleRpc("resume", "mining") + )); + + mapHelpStrings.insert(std::make_pair("setlastblock", + "setlastblock hash|height \n" + "\nSets last block in the chain.\n" + "\nArguments:\n" + "1. hash|height (string, optional) The block hash or block height in active chain or height before current tip (if negative), if omitted - best chain is activated\n" + "\nResult:\n" + "\"hash\" (string) The block hash of the chain tip\n" + "\nExamples:\n" + + HelpExampleCli("setlastblock", "\"00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09\"") + + HelpExampleRpc("setlastblock", "\"00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09\"") + )); + + mapHelpStrings.insert(std::make_pair("stop", + "stop\n" + "\nStop MultiChain server." + )); + + mapHelpStrings.insert(std::make_pair("getgenerate", + "getgenerate\n" + "\nReturn if the server is set to generate coins or not. The default is false.\n" + "It is set with the command line argument -gen (or bitcoin.conf setting gen)\n" + "It can also be set with the setgenerate call.\n" + "\nResult\n" + "true|false (boolean) If the server is set to generate coins or not\n" + "\nExamples:\n" + + HelpExampleCli("getgenerate", "") + + HelpExampleRpc("getgenerate", "") + )); + + mapHelpStrings.insert(std::make_pair("gethashespersec", + "gethashespersec\n" + "\nReturns a recent hashes per second performance measurement while generating.\n" + "See the getgenerate and setgenerate calls to turn generation on and off.\n" + "\nResult:\n" + "n (numeric) The recent hashes per second when generation is on (will return 0 if generation is off)\n" + "\nExamples:\n" + + HelpExampleCli("gethashespersec", "") + + HelpExampleRpc("gethashespersec", "") + )); + + mapHelpStrings.insert(std::make_pair("setgenerate", + "setgenerate generate ( genproclimit )\n" + "\nSet 'generate' true or false to turn generation on or off.\n" + "Generation is limited to 'genproclimit' processors, -1 is unlimited.\n" + "See the getgenerate call for the current setting.\n" + "\nArguments:\n" + "1. generate (boolean, required) Set to true to turn on generation, off to turn off.\n" + "2. genproclimit (numeric, optional) Set the processor limit for when generation is on. Can be -1 for unlimited.\n" + " Note: in -regtest mode, genproclimit controls how many blocks are generated immediately.\n" + "\nResult\n" + "[ blockhashes ] (array, -regtest only) hashes of blocks generated\n" + "\nExamples:\n" + "\nSet the generation on with a limit of one processor\n" + + HelpExampleCli("setgenerate", "true 1") + + "\nCheck the setting\n" + + HelpExampleCli("getgenerate", "") + + "\nTurn off generation\n" + + HelpExampleCli("setgenerate", "false") + + "\nUsing json rpc\n" + + HelpExampleRpc("setgenerate", "true, 1") + )); + + mapHelpStrings.insert(std::make_pair("getblocktemplate", + "getblocktemplate ( \"jsonrequestobject\" )\n" + "\nIf the request parameters include a 'mode' key, that is used to explicitly select between the default 'template' request or a 'proposal'.\n" + "It returns data needed to construct a block to work on.\n" + "See https://en.bitcoin.it/wiki/BIP_0022 for full specification.\n" + + "\nArguments:\n" + "1. \"jsonrequestobject\" (string, optional) A json object in the following spec\n" + " {\n" + " \"mode\":\"template\" (string, optional) This must be set to \"template\" or omitted\n" + " \"capabilities\":[ (array, optional) A list of strings\n" + " \"support\" (string) client side supported feature, 'longpoll', 'coinbasetxn', 'coinbasevalue', 'proposal', 'serverlist', 'workid'\n" + " ,...\n" + " ]\n" + " }\n" + "\n" + + "\nResult:\n" + "{\n" + " \"version\" : n, (numeric) The block version\n" + " \"previousblockhash\" : \"xxxx\", (string) The hash of current highest block\n" + " \"transactions\" : [ (array) contents of non-coinbase transactions that should be included in the next block\n" + " {\n" + " \"data\" : \"xxxx\", (string) transaction data encoded in hexadecimal (byte-for-byte)\n" + " \"hash\" : \"xxxx\", (string) hash/id encoded in little-endian hexadecimal\n" + " \"depends\" : [ (array) array of numbers \n" + " n (numeric) transactions before this one (by 1-based index in 'transactions' list) that must be present in the final block if this one is\n" + " ,...\n" + " ],\n" + " \"fee\": n, (numeric) difference in value between transaction inputs and outputs (in Satoshis); for coinbase transactions, this is a negative Number of the total collected block fees (ie, not including the block subsidy); if key is not present, fee is unknown and clients MUST NOT assume there isn't one\n" + " \"sigops\" : n, (numeric) total number of SigOps, as counted for purposes of block limits; if key is not present, sigop count is unknown and clients MUST NOT assume there aren't any\n" + " \"required\" : true|false (boolean) if provided and true, this transaction must be in the final block\n" + " }\n" + " ,...\n" + " ],\n" + " \"coinbaseaux\" : { (json object) data that should be included in the coinbase's scriptSig content\n" + " \"flags\" : \"flags\" (string) \n" + " },\n" + " \"coinbasevalue\" : n, (numeric) maximum allowable input to coinbase transaction, including the generation award and transaction fees (in Satoshis)\n" + " \"coinbasetxn\" : { ... }, (json object) information for coinbase transaction\n" + " \"target\" : \"xxxx\", (string) The hash target\n" + " \"mintime\" : xxx, (numeric) The minimum timestamp appropriate for next block time in seconds since epoch (Jan 1 1970 GMT)\n" + " \"mutable\" : [ (array of string) list of ways the block template may be changed \n" + " \"value\" (string) A way the block template may be changed, e.g. 'time', 'transactions', 'prevblock'\n" + " ,...\n" + " ],\n" + " \"noncerange\" : \"00000000ffffffff\", (string) A range of valid nonces\n" + " \"sigoplimit\" : n, (numeric) limit of sigops in blocks\n" + " \"sizelimit\" : n, (numeric) limit of block size\n" + " \"curtime\" : ttt, (numeric) current timestamp in seconds since epoch (Jan 1 1970 GMT)\n" + " \"bits\" : \"xxx\", (string) compressed target of next block\n" + " \"height\" : n (numeric) The height of the next block\n" + "}\n" + + "\nExamples:\n" + + HelpExampleCli("getblocktemplate", "") + + HelpExampleRpc("getblocktemplate", "") + )); + + mapHelpStrings.insert(std::make_pair("getmininginfo", + "getmininginfo\n" + "\nReturns a json object containing mining-related information." + "\nResult:\n" + "{\n" + " \"blocks\": nnn, (numeric) The current block\n" + " \"currentblocksize\": nnn, (numeric) The last block size\n" + " \"currentblocktx\": nnn, (numeric) The last block transaction\n" + " \"difficulty\": xxx.xxxxx (numeric) The current difficulty\n" + " \"errors\": \"...\" (string) Current errors\n" + " \"generate\": true|false (boolean) If the generation is on or off (see getgenerate or setgenerate calls)\n" + " \"genproclimit\": n (numeric) The processor limit for generation. -1 if no generation. (see getgenerate or setgenerate calls)\n" + " \"hashespersec\": n (numeric) The hashes per second of the generation, or 0 if no generation.\n" + " \"pooledtx\": n (numeric) The size of the mem pool\n" + " \"testnet\": true|false (boolean) If using testnet or not\n" + " \"chain\": \"xxxx\", (string) current network name as defined in BIP70 (main, test, regtest)\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("getmininginfo", "") + + HelpExampleRpc("getmininginfo", "") + )); + + mapHelpStrings.insert(std::make_pair("getnetworkhashps", + "getnetworkhashps ( blocks height )\n" + "\nReturns the estimated network hashes per second based on the last n blocks.\n" + "Pass in [blocks] to override # of blocks, -1 specifies since last difficulty change.\n" + "Pass in [height] to estimate the network speed at the time when a certain block was found.\n" + "\nArguments:\n" + "1. blocks (numeric, optional, default=120) The number of blocks, or -1 for blocks since last difficulty change.\n" + "2. height (numeric, optional, default=-1) To estimate at the time of the given height.\n" + "\nResult:\n" + "x (numeric) Hashes per second estimated\n" + "\nExamples:\n" + + HelpExampleCli("getnetworkhashps", "") + + HelpExampleRpc("getnetworkhashps", "") + )); + + mapHelpStrings.insert(std::make_pair("prioritisetransaction", + "prioritisetransaction txid priority-delta fee-delta\n" + "Accepts the transaction into mined blocks at a higher (or lower) priority\n" + "\nArguments:\n" + "1. txid (string, required) The transaction id.\n" + "2. priority-delta (numeric, required) The priority to add or subtract.\n" + " The transaction selection algorithm considers the tx as it would have a higher priority.\n" + " (priority of a transaction is calculated: coinage * value_in_satoshis / txsize) \n" + "3. fee-delta (numeric, required) The fee value (in satoshis) to add (or subtract, if negative).\n" + " The fee is not actually paid, only the algorithm for selecting transactions into a block\n" + " considers the transaction as it would have paid a higher (or lower) fee.\n" + "\nResult\n" + "true (boolean) Returns true\n" + "\nExamples:\n" + + HelpExampleCli("prioritisetransaction", "\"txid\" 0.0 10000") + + HelpExampleRpc("prioritisetransaction", "\"txid\", 0.0, 10000") + )); + +} + +void mc_InitRPCHelpMap04() +{ + mapHelpStrings.insert(std::make_pair("submitblock", + "submitblock hexdata ( \"jsonparametersobject\" )\n" + "\nAttempts to submit new block to network.\n" + "The 'jsonparametersobject' parameter is currently ignored.\n" + "See https://en.bitcoin.it/wiki/BIP_0022 for full specification.\n" + + "\nArguments\n" + "1. hexdata (string, required) the hex-encoded block data to submit\n" + "2. \"jsonparametersobject\" (string, optional) object of optional parameters\n" + " {\n" + " \"workid\" : \"id\" (string, optional) if the server provided a workid, it MUST be included with submissions\n" + " }\n" + "\nResult:\n" + "\nExamples:\n" + + HelpExampleCli("submitblock", "\"mydata\"") + + HelpExampleRpc("submitblock", "\"mydata\"") + )); + + mapHelpStrings.insert(std::make_pair("addnode", + "addnode \"node\" add|remove|onetry\n" + "\nAttempts add or remove a node from the addnode list.\n" + "Or try a connection to a node once.\n" + "\nArguments:\n" + "1. \"node\" (string, required) The node (see getpeerinfo for nodes)\n" + "2. \"command\" (string, required) 'add' to add a node to the list, 'remove' to remove a node from the list, 'onetry' to try a connection to the node once\n" + "\nExamples:\n" + + HelpExampleCli("addnode", "\"192.168.0.6:8333\" \"onetry\"") + + HelpExampleRpc("addnode", "\"192.168.0.6:8333\", \"onetry\"") + )); + + mapHelpStrings.insert(std::make_pair("getaddednodeinfo", + "getaddednodeinfo dns ( \"node\" )\n" + "\nReturns information about the given added node, or all added nodes\n" + "(note that onetry addnodes are not listed here)\n" + "If dns is false, only a list of added nodes will be provided,\n" + "otherwise connected information will also be available.\n" + "\nArguments:\n" + "1. dns (boolean, required) If false, only a list of added nodes will be provided, otherwise connected information will also be available.\n" + "2. \"node\" (string, optional) If provided, return information about this specific node, otherwise all nodes are returned.\n" + "\nResult:\n" + "[\n" + " {\n" + " \"addednode\" : \"192.168.0.201\", (string) The node ip address\n" + " \"connected\" : true|false, (boolean) If connected\n" + " \"addresses\" : [\n" + " {\n" + " \"address\" : \"192.168.0.201:8333\", (string) The MultiChain server host and port\n" + " \"connected\" : \"outbound\" (string) connection, inbound or outbound\n" + " }\n" + " ,...\n" + " ]\n" + " }\n" + " ,...\n" + "]\n" + "\nExamples:\n" + + HelpExampleCli("getaddednodeinfo", "true") + + HelpExampleCli("getaddednodeinfo", "true \"192.168.0.201\"") + + HelpExampleRpc("getaddednodeinfo", "true, \"192.168.0.201\"") + )); + + mapHelpStrings.insert(std::make_pair("getconnectioncount", + "getconnectioncount\n" + "\nReturns the number of connections to other nodes.\n" + "\nbResult:\n" + "n (numeric) The connection count\n" + "\nExamples:\n" + + HelpExampleCli("getconnectioncount", "") + + HelpExampleRpc("getconnectioncount", "") + )); + + mapHelpStrings.insert(std::make_pair("getnettotals", + "getnettotals\n" + "\nReturns information about network traffic, including bytes in, bytes out,\n" + "and current time.\n" + "\nResult:\n" + "{\n" + " \"totalbytesrecv\": n, (numeric) Total bytes received\n" + " \"totalbytessent\": n, (numeric) Total bytes sent\n" + " \"timemillis\": t (numeric) Total cpu time\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("getnettotals", "") + + HelpExampleRpc("getnettotals", "") + )); + + mapHelpStrings.insert(std::make_pair("getnetworkinfo", + "getnetworkinfo\n" + "Returns an object containing various state info regarding P2P networking.\n" + "\nResult:\n" + "{\n" + " \"version\": xxxxx, (numeric) the server version\n" + " \"subversion\": \"/Satoshi:x.x.x/\", (string) the server subversion string\n" + " \"protocolversion\": xxxxx, (numeric) the protocol version\n" + " \"localservices\": \"xxxxxxxxxxxxxxxx\", (string) the services we offer to the network\n" + " \"timeoffset\": xxxxx, (numeric) the time offset\n" + " \"connections\": xxxxx, (numeric) the number of connections\n" + " \"networks\": [ (array) information per network\n" + " {\n" + " \"name\": \"xxx\", (string) network (ipv4, ipv6 or onion)\n" + " \"limited\": true|false, (boolean) is the network limited using -onlynet?\n" + " \"reachable\": true|false, (boolean) is the network reachable?\n" + " \"proxy\": \"host:port\" (string) the proxy that is used for this network, or empty if none\n" + " }\n" + " ,...\n" + " ],\n" + " \"relayfee\": x.xxxxxxxx, (numeric) minimum relay fee for non-free transactions in btc/kb\n" + " \"localaddresses\": [ (array) list of local addresses\n" + " {\n" + " \"address\": \"xxxx\", (string) network address\n" + " \"port\": xxx, (numeric) network port\n" + " \"score\": xxx (numeric) relative score\n" + " }\n" + " ,...\n" + " ]\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("getnetworkinfo", "") + + HelpExampleRpc("getnetworkinfo", "") + )); + + mapHelpStrings.insert(std::make_pair("getpeerinfo", + "getpeerinfo\n" + "\nReturns data about each connected network node as a json array of objects.\n" + "\nbResult:\n" + "[\n" + " {\n" + " \"id\": n, (numeric) Peer index\n" + " \"addr\":\"host:port\", (string) The ip address and port of the peer\n" + " \"addrlocal\":\"ip:port\", (string) local address\n" + " \"services\":\"xxxxxxxxxxxxxxxx\", (string) The services offered\n" + " \"lastsend\": ttt, (numeric) The time in seconds since epoch (Jan 1 1970 GMT) of the last send\n" + " \"lastrecv\": ttt, (numeric) The time in seconds since epoch (Jan 1 1970 GMT) of the last receive\n" + " \"bytessent\": n, (numeric) The total bytes sent\n" + " \"bytesrecv\": n, (numeric) The total bytes received\n" + " \"conntime\": ttt, (numeric) The connection time in seconds since epoch (Jan 1 1970 GMT)\n" + " \"pingtime\": n, (numeric) ping time\n" + " \"pingwait\": n, (numeric) ping wait\n" + " \"version\": v, (numeric) The peer version, such as 7001\n" + " \"subver\": \"/Satoshi:0.8.5/\", (string) The string version\n" + " \"handshakelocal\": n, (string) If protocol is Multichain. Address used by local node for handshake.\n" + " \"handshake\": n, (string) If protocol is Multichain. Address used by remote node for handshake.\n" + " \"inbound\": true|false, (boolean) Inbound (true) or Outbound (false)\n" + " \"startingheight\": n, (numeric) The starting height (block) of the peer\n" + " \"banscore\": n, (numeric) The ban score\n" + " \"synced_headers\": n, (numeric) The last header we have in common with this peer\n" + " \"synced_blocks\": n, (numeric) The last block we have in common with this peer\n" + " \"inflight\": [\n" + " n, (numeric) The heights of blocks we're currently asking from this peer\n" + " ...\n" + " ]\n" + " }\n" + " ,...\n" + "]\n" + "\nExamples:\n" + + HelpExampleCli("getpeerinfo", "") + + HelpExampleRpc("getpeerinfo", "") + )); + + mapHelpStrings.insert(std::make_pair("ping", + "ping\n" + "\nRequests that a ping be sent to all other nodes, to measure ping time.\n" + "Results provided in getpeerinfo, pingtime and pingwait fields are decimal seconds.\n" + "Ping command is handled in queue with all other commands, so it measures processing backlog, not just network ping.\n" + "\nExamples:\n" + + HelpExampleCli("ping", "") + + HelpExampleRpc("ping", "") + )); + + mapHelpStrings.insert(std::make_pair("appendrawchange", + "appendrawchange tx-hex address ( native-fee )\n" + "\nAppends change output to raw transaction, containing any remaining assets / native currency in the inputs that are not already sent to other outputs. \n" + "\nArguments:\n" + "1. tx-hex (string, required) The hex string of the raw transaction)\n" + "2. address (string, required) The address to send the change to.\n" + "3. native-fee (numeric, optional) Native currency value deducted from that amount so it becomes a transaction fee. Default - calculated automatically\n" + "\nResult:\n" + "\"transaction\" (string) hex string of the transaction\n" + "\nExamples:\n" + + HelpExampleCli("appendrawchange", "\"hex\"" "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" ") + + HelpExampleCli("appendrawchange", "\"hex\"" "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.01") + + HelpExampleRpc("appendrawchange", "\"hex\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\"") + )); + + mapHelpStrings.insert(std::make_pair("appendrawmetadata", + "appendrawmetadata tx-hex data-hex|object \n" + "\nAppends new OP_RETURN output to existing raw transaction\n" + "Returns hex-encoded raw transaction.\n" + "\nArguments:\n" + "1. \"tx-hex\" (string, required) The transaction hex string\n" + "2. \"data-hex\" (string, required) Data hex string\n" + "or\n" + "2. \"issue-details\" (object, required) A json object with issue metadata\n" + " {\n" + " \"create\" : asset (string,required) asset\n" + " \"name\" : asset-name (string,optional) Asset name\n" + " \"multiple\" : n (numeric,optional, default 1) Number of raw units in one displayed unit\n" + " \"open\" : true|false (boolean, optional, default false) True if follow-on issues are allowed\n" + " \"details\" : (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + "or\n" + "2. \"issuemore-details\" (object, required) A json object with issuemore metadata\n" + " {\n" + " \"update\" : asset-identifier (string,required) Stream identifier - one of the following: asset txid, asset reference, asset name.\n" + " \"details\" : (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + " }\n" + "or\n" + "2. \"create-new-stream\" (object, required) A json object with new stream details\n" + " {\n" + " \"create\" : stream (string,required) stream\n" + " \"name\" : stream-name (string,optional) Stream name\n" + " \"open\" : true|false (string,optional, default: false) If true, anyone can publish\n" + " \"details\" : (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + " }\n" + "or\n" + "2. publish-new-stream-item (object, required) A json object with stream item\n" + " {\n" + " \"for\" : stream-identifier (string,required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + " \"key\" : key (string,optional, default: \"\") Item key\n" + " \"data\" : data-hex (string,optional, default: \"\") Data hex string\n" + " }\n" + "\nResult:\n" + "{\n" + " \"hex\": \"value\", (string) The raw transaction with appended data output (hex-encoded string)\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("appendrawmetadata", "\"tx-hexstring\" 48656C6C6F20576F726C64210A" ) + + HelpExampleRpc("appendrawmetadata", "\"tx-hexstring\",\"48656C6C6F20576F726C64210A\"") + )); + +} + +void mc_InitRPCHelpMap05() +{ + mapHelpStrings.insert(std::make_pair("appendrawdata", + "appendrawdata tx-hex data-hex|object \n" + "\nAppends new OP_RETURN output to existing raw transaction\n" + "Returns hex-encoded raw transaction.\n" + "\nArguments:\n" + "1. \"tx-hex\" (string, required) The transaction hex string\n" + "2. \"data-hex\" (string, required) Data hex string\n" + "or\n" + "2. \"issue-details\" (object, required) A json object with issue metadata\n" + " {\n" + " \"create\" : asset (string,required) asset\n" + " \"name\" : asset-name (string,optional) Asset name\n" + " \"multiple\" : n (numeric,optional, default 1) Number of raw units in one displayed unit\n" + " \"open\" : true|false (boolean, optional, default false) True if follow-on issues are allowed\n" + " \"details\" : (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + "or\n" + "2. \"issuemore-details\" (object, required) A json object with issuemore metadata\n" + " {\n" + " \"update\" : asset-identifier (string,required) Stream identifier - one of the following: asset txid, asset reference, asset name.\n" + " \"details\" : (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + " }\n" + "or\n" + "2. \"create-new-stream\" (object, required) A json object with new stream details\n" + " {\n" + " \"create\" : stream (string,required) stream\n" + " \"name\" : stream-name (string,optional) Stream name\n" + " \"open\" : true|false (string,optional, default: false) If true, anyone can publish\n" + " \"details\" : (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + " }\n" + "or\n" + "2. publish-new-stream-item (object, required) A json object with stream item\n" + " {\n" + " \"for\" : stream-identifier (string,required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + " \"key\" : key (string,optional, default: \"\") Item key\n" + " \"data\" : data-hex (string,optional, default: \"\") Data hex string\n" + " }\n" + "\nResult:\n" + "{\n" + " \"hex\": \"value\", (string) The raw transaction with appended data output (hex-encoded string)\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("appendrawdata", "\"tx-hexstring\" 48656C6C6F20576F726C64210A" ) + + HelpExampleRpc("appendrawdata", "\"tx-hexstring\",\"48656C6C6F20576F726C64210A\"") + )); + + mapHelpStrings.insert(std::make_pair("createrawtransaction", + "createrawtransaction [{\"txid\":\"id\",\"vout\":n},...] {\"address\":amount,...} ( [data] action ) \n" + "\nCreate a transaction spending the given inputs.\n" + + "\nArguments:\n" + "1. \"transactions\" (string, required) A json array of json objects\n" + " [\n" + " {\n" + " \"txid\":\"id\", (string, required) The transaction id\n" + " \"vout\":n (numeric, required) The output number\n" + " \"scriptPubKey\": \"hex\", (string, optional) script key, used if cache=true or action=sign\n" + " \"redeemScript\": \"hex\" (string, optional) redeem script, used if action=sign\n" + " \"cache\":true|false (boolean, optional) If true - add cached script to tx, if omitted - add automatically if needed\n" + " }\n" + " ,...\n" + " ]\n" + "2. \"addresses\" (string, required) a json object with addresses as keys and amounts as values\n" + " {\n" + " \"address\": \n" + " x.xxx (numeric, required) The key is the address, the value is the native currency amount\n" + " or \n" + " { (object) A json object of assets to send\n" + " \"asset-identifier\" : asset-quantity \n" + " ,...\n" + " }\n" + " or \n" + " { (object) A json object describing new asset issue\n" + " \"issue\" : \n" + " {\n" + " \"raw\" : n (numeric, required) The asset total amount in raw units \n" + " ,...\n" + " }\n" + " ,...\n" + " }\n" + " or \n" + " { (object) A json object describing follow-on asset issue\n" + " \"issuemore\" : \n" + " {\n" + " \"asset\" : \"asset-identifier\" (string, required) Asset identifier - one of the following: issue txid. asset reference, asset name.\n" + " \"raw\" : n (numeric, required) The asset total amount in raw units \n" + " ,...\n" + " }\n" + " ,...\n" + " }\n" + " or \n" + " { (object) A json object describing permission change\n" + " \"permissions\" : \n" + " {\n" + " \"type\" : \"permission(s)\" (string,required) Permission strings, comma delimited. Possible values: " + AllowedPermissions() + " \n" + " \"startblock\" (numeric, optional) Block to apply permissions from (inclusive). Default - 0\n" + " \"endblock\" (numeric, optional) Block to apply permissions to (exclusive). Default - 4294967295\n" + " \"timestamp\" (numeric, optional) This helps resolve conflicts between permissions assigned by the same administrator. Default - current time\n" + " ,...\n" + " }\n" + " ,...\n" + " }\n" + " ,...\n" + " }\n" + "3. data (array, optional) Array of hexadecimal strings or data objects, see help appendrawdata for details.\n" + "4. action (string, optional, default \"\") Additional actions: \"lock\", \"sign\", \"lock,sign\", \"sign,lock\", \"send\". \n" + + + "\nResult:\n" + "\"transaction\" (string) hex string of the transaction (if action= \"\" or \"lock\")\n" + " or \n" + "{ (object) A json object (if action= \"sign\" or \"lock,sign\" or \"sign,lock\")\n" + " \"hex\": \"value\", (string) The raw transaction with signature(s) (hex-encoded string)\n" + " \"complete\": n (numeric) if transaction has a complete set of signature (0 if not)\n" + "}\n" + " or \n" + "\"hex\" (string) The transaction hash in hex (if action= \"send\")\n" + + "\nExamples\n" + + HelpExampleCli("createrawtransaction", "\"[{\\\"txid\\\":\\\"myid\\\",\\\"vout\\\":0}]\" \"{\\\"address\\\":0.01}\"") + + HelpExampleRpc("createrawtransaction", "\"[{\\\"txid\\\":\\\"myid\\\",\\\"vout\\\":0}]\", \"{\\\"address\\\":0.01}\"") + )); + + mapHelpStrings.insert(std::make_pair("decoderawtransaction", + "decoderawtransaction tx-hex\n" + "\nReturn a JSON object representing the serialized, hex-encoded transaction.\n" + + "\nArguments:\n" + "1. tx-hex (string, required) The transaction hex string\n" + + "\nResult:\n" + "{\n" + " \"txid\" : \"id\", (string) The transaction id\n" + " \"version\" : n, (numeric) The version\n" + " \"locktime\" : ttt, (numeric) The lock time\n" + " \"vin\" : [ (array of json objects)\n" + " {\n" + " \"txid\": \"id\", (string) The transaction id\n" + " \"vout\": n, (numeric) The output number\n" + " \"scriptSig\": { (json object) The script\n" + " \"asm\": \"asm\", (string) asm\n" + " \"hex\": \"hex\" (string) hex\n" + " },\n" + " \"sequence\": n (numeric) The script sequence number\n" + " }\n" + " ,...\n" + " ],\n" + " \"vout\" : [ (array of json objects)\n" + " {\n" + " \"value\" : x.xxx, (numeric) The value in btc\n" + " \"n\" : n, (numeric) index\n" + " \"scriptPubKey\" : { (json object)\n" + " \"asm\" : \"asm\", (string) the asm\n" + " \"hex\" : \"hex\", (string) the hex\n" + " \"reqSigs\" : n, (numeric) The required sigs\n" + " \"type\" : \"pubkeyhash\", (string) The type, eg 'pubkeyhash'\n" + " \"addresses\" : [ (json array of string)\n" + " \"12tvKAXCxZjSmdNbao16dKXC8tRWfcF5oc\" (string) address\n" + " ,...\n" + " ]\n" + " }\n" + " }\n" + " ,...\n" + " ],\n" + "}\n" + + "\nExamples:\n" + + HelpExampleCli("decoderawtransaction", "\"hexstring\"") + + HelpExampleRpc("decoderawtransaction", "\"hexstring\"") + )); + + mapHelpStrings.insert(std::make_pair("decodescript", + "decodescript script-hex\n" + "\nDecode a hex-encoded script.\n" + "\nArguments:\n" + "1. script-hex (string) the hex encoded script\n" + "\nResult:\n" + "{\n" + " \"asm\":\"asm\", (string) Script public key\n" + " \"hex\":\"hex\", (string) hex encoded public key\n" + " \"type\":\"type\", (string) The output type\n" + " \"reqSigs\": n, (numeric) The required signatures\n" + " \"addresses\": [ (json array of string)\n" + " \"address\" (string) address\n" + " ,...\n" + " ],\n" + " \"p2sh\",\"address\" (string) script address\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("decodescript", "\"hexstring\"") + + HelpExampleRpc("decodescript", "\"hexstring\"") + )); + + mapHelpStrings.insert(std::make_pair("getrawtransaction", + "getrawtransaction txid ( verbose )\n" + "\nNOTE: By default this function only works sometimes. This is when the tx is in the mempool\n" + "or there is an unspent output in the utxo for this transaction. To make it always work,\n" + "you need to maintain a transaction index, using the -txindex command line option.\n" + "\nReturn the raw transaction data.\n" + "\nIf verbose=0, returns a string that is serialized, hex-encoded data for 'txid'.\n" + "If verbose is non-zero, returns an Object with information about 'txid'.\n" + + "\nArguments:\n" + "1. txid (string, required) The transaction id\n" + "2. verbose (numeric, optional, default=0) If 0, return a string, other return a json object\n" + + "\nResult (if verbose is not set or set to 0):\n" + "\"data\" (string) The serialized, hex-encoded data for 'txid'\n" + + "\nResult (if verbose > 0):\n" + "{\n" + " \"hex\" : \"data\", (string) The serialized, hex-encoded data for 'txid'\n" + " \"txid\" : \"id\", (string) The transaction id (same as provided)\n" + " \"version\" : n, (numeric) The version\n" + " \"locktime\" : ttt, (numeric) The lock time\n" + " \"vin\" : [ (array of json objects)\n" + " {\n" + " \"txid\": \"id\", (string) The transaction id\n" + " \"vout\": n, (numeric) \n" + " \"scriptSig\": { (json object) The script\n" + " \"asm\": \"asm\", (string) asm\n" + " \"hex\": \"hex\" (string) hex\n" + " },\n" + " \"sequence\": n (numeric) The script sequence number\n" + " }\n" + " ,...\n" + " ],\n" + " \"vout\" : [ (array of json objects)\n" + " {\n" + " \"value\" : x.xxx, (numeric) The value in btc\n" + " \"n\" : n, (numeric) index\n" + " \"scriptPubKey\" : { (json object)\n" + " \"asm\" : \"asm\", (string) the asm\n" + " \"hex\" : \"hex\", (string) the hex\n" + " \"reqSigs\" : n, (numeric) The required sigs\n" + " \"type\" : \"pubkeyhash\", (string) The type, eg 'pubkeyhash'\n" + " \"addresses\" : [ (json array of string)\n" + " \"address\" (string) address\n" + " ,...\n" + " ]\n" + " }\n" + " }\n" + " ,...\n" + " ],\n" + " \"blockhash\" : \"hash\", (string) the block hash\n" + " \"confirmations\" : n, (numeric) The confirmations\n" + " \"time\" : ttt, (numeric) The transaction time in seconds since epoch (Jan 1 1970 GMT)\n" + " \"blocktime\" : ttt (numeric) The block time in seconds since epoch (Jan 1 1970 GMT)\n" + "}\n" + + "\nExamples:\n" + + HelpExampleCli("getrawtransaction", "\"mytxid\"") + + HelpExampleCli("getrawtransaction", "\"mytxid\" 1") + + HelpExampleRpc("getrawtransaction", "\"mytxid\", 1") + )); + + mapHelpStrings.insert(std::make_pair("sendrawtransaction", + "sendrawtransaction tx-hex ( allowhighfees )\n" + "\nSubmits raw transaction (serialized, hex-encoded) to local node and network.\n" + "\nAlso see createrawtransaction and signrawtransaction calls.\n" + "\nArguments:\n" + "1. tx-hex (string, required) The hex string of the raw transaction)\n" + "2. allowhighfees (boolean, optional, default=false) Allow high fees\n" + "\nResult:\n" + "\"hex\" (string) The transaction hash in hex\n" + "\nExamples:\n" + "\nCreate a transaction\n" + + HelpExampleCli("createrawtransaction", "\"[{\\\"txid\\\" : \\\"mytxid\\\",\\\"vout\\\":0}]\" \"{\\\"myaddress\\\":0.01}\"") + + "Sign the transaction, and get back the hex\n" + + HelpExampleCli("sendrawtransaction", "\"myhex\"") + + "\nSend the transaction (signed hex)\n" + + HelpExampleCli("sendrawtransaction", "\"signedhex\"") + + "\nAs a json rpc call\n" + + HelpExampleRpc("sendrawtransaction", "\"signedhex\"") + )); + + mapHelpStrings.insert(std::make_pair("signrawtransaction", + "signrawtransaction tx-hex ( [{\"txid\":\"id\",\"vout\":n,\"scriptPubKey\":\"hex\",\"redeemScript\":\"hex\"},...] [\"privatekey1\",...] sighashtype )\n" + "\nSign inputs for raw transaction (serialized, hex-encoded).\n" + "The second optional argument (may be null) is an array of previous transaction outputs that\n" + "this transaction depends on but may not yet be in the block chain.\n" + "The third optional argument (may be null) is an array of base58-encoded private\n" + "keys that, if given, will be the only keys used to sign the transaction.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. tx-hex (string, required) The transaction hex string\n" + "2. \"prevtxs\" (string, optional) An json array of previous dependent transaction outputs\n" + " [ (json array of json objects, or 'null' if none provided)\n" + " {\n" + " \"txid\":\"id\", (string, required) The transaction id\n" + " \"vout\":n, (numeric, required) The output number\n" + " \"scriptPubKey\": \"hex\", (string, required) script key\n" + " \"redeemScript\": \"hex\" (string, required for P2SH) redeem script\n" + " }\n" + " ,...\n" + " ]\n" + "3. \"privatekeys\" (string, optional) A json array of base58-encoded private keys for signing\n" + " [ (json array of strings, or 'null' if none provided)\n" + " \"privatekey\" (string) private key in base58-encoding\n" + " ,...\n" + " ]\n" + "4. \"sighashtype\" (string, optional, default=ALL) The signature hash type. Must be one of\n" + " \"ALL\"\n" + " \"NONE\"\n" + " \"SINGLE\"\n" + " \"ALL|ANYONECANPAY\"\n" + " \"NONE|ANYONECANPAY\"\n" + " \"SINGLE|ANYONECANPAY\"\n" + + "\nResult:\n" + "{\n" + " \"hex\": \"value\", (string) The raw transaction with signature(s) (hex-encoded string)\n" + " \"complete\": n (numeric) if transaction has a complete set of signature (0 if not)\n" + "}\n" + + "\nExamples:\n" + + HelpExampleCli("signrawtransaction", "\"myhex\"") + + HelpExampleRpc("signrawtransaction", "\"myhex\"") + )); + + mapHelpStrings.insert(std::make_pair("createkeypairs", + "createkeypairs ( count )\n" + "\nCreates public/private key pairs. These key pairs are not stored in the wallet.\n" + "\nArguments: \n" + "1. \"count\" (number, optional, default=1) Number of pairs to create.\n" + "\nResult:\n" + "[ (json array of )\n" + " {\n" + " \"address\" : \"address\", (string) Pay-to-pubkeyhash address\n" + " \"pubkey\" : \"pubkey\", (string) Public key (hexadecimal)\n" + " \"privkey\" : \"privatekey\", (string) Private key, base58-encoded as required for signrawtransaction\n" + " }\n" + "]\n" + "\nExamples:\n" + + HelpExampleCli("createkeypairs", "") + + HelpExampleRpc("createkeypairs", "") + )); + + mapHelpStrings.insert(std::make_pair("createmultisig", + "createmultisig nrequired [\"key\",...]\n" + "\nCreates a multi-signature address with n signature of m keys required.\n" + "It returns a json object with the address and redeemScript.\n" + + "\nArguments:\n" + "1. nrequired (numeric, required) The number of required signatures out of the n keys or addresses.\n" + "2. \"keys\" (string, required) A json array of keys which are addresses or hex-encoded public keys\n" + " [\n" + " \"key\" (string) address or hex-encoded public key\n" + " ,...\n" + " ]\n" + + "\nResult:\n" + "{\n" + " \"address\":\"multisigaddress\", (string) The value of the new multisig address.\n" + " \"redeemScript\":\"script\" (string) The string value of the hex-encoded redemption script.\n" + "}\n" + + "\nExamples:\n" + "\nCreate a multisig address from 2 addresses\n" + + HelpExampleCli("createmultisig", "2 \"[\\\"16sSauSf5pF2UkUwvKGq4qjNRzBZYqgEL5\\\",\\\"171sgjn4YtPu27adkKGrdDwzRTxnRkBfKV\\\"]\"") + + "\nAs a json rpc call\n" + + HelpExampleRpc("createmultisig", "2, \"[\\\"16sSauSf5pF2UkUwvKGq4qjNRzBZYqgEL5\\\",\\\"171sgjn4YtPu27adkKGrdDwzRTxnRkBfKV\\\"]\"") + )); + + mapHelpStrings.insert(std::make_pair("estimatefee", + "estimatefee nblocks\n" + "\nEstimates the approximate fee per kilobyte\n" + "needed for a transaction to begin confirmation\n" + "within nblocks blocks.\n" + "\nArguments:\n" + "1. nblocks (numeric)\n" + "\nResult:\n" + "n : (numeric) estimated fee-per-kilobyte\n" + "\n" + "-1.0 is returned if not enough transactions and\n" + "blocks have been observed to make an estimate.\n" + "\nExample:\n" + + HelpExampleCli("estimatefee", "6") + )); + +} + +void mc_InitRPCHelpMap06() +{ + mapHelpStrings.insert(std::make_pair("estimatepriority", + "estimatepriority nblocks\n" + "\nEstimates the approximate priority\n" + "a zero-fee transaction needs to begin confirmation\n" + "within nblocks blocks.\n" + "\nArguments:\n" + "1. nblocks (numeric)\n" + "\nResult:\n" + "n : (numeric) estimated priority\n" + "\n" + "-1.0 is returned if not enough transactions and\n" + "blocks have been observed to make an estimate.\n" + "\nExample:\n" + + HelpExampleCli("estimatepriority", "6") + )); + + mapHelpStrings.insert(std::make_pair("validateaddress", + "validateaddress address\n" + "\nReturn information about the given address.\n" + "\nArguments:\n" + "1. address (string, required) The address to validate\n" + "\nResult:\n" + "{\n" + " \"isvalid\" : true|false, (boolean) If the address is valid or not. If not, this is the only property returned.\n" + " \"address\" : \"address\", (string) The address validated\n" + " \"ismine\" : true|false, (boolean) If the address is yours or not\n" + " \"isscript\" : true|false, (boolean) If the key is a script\n" + " \"pubkey\" : \"publickeyhex\", (string) The hex value of the raw public key\n" + " \"iscompressed\" : true|false, (boolean) If the address is compressed\n" + " \"account\" : \"account\" (string) The account associated with the address, \"\" is the default account\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("validateaddress", "\"1PSSGeFHDnKNxiEyFrD1wcEaHr9hrQDDWc\"") + + HelpExampleRpc("validateaddress", "\"1PSSGeFHDnKNxiEyFrD1wcEaHr9hrQDDWc\"") + )); + + mapHelpStrings.insert(std::make_pair("verifymessage", + "verifymessage address signature \"message\"\n" + "\nVerify a signed message\n" + "\nArguments:\n" + "1. address (string, required) The address to use for the signature.\n" + "2. signature (string, required) The signature provided by the signer in base 64 encoding (see signmessage).\n" + "3. \"message\" (string, required) The message that was signed.\n" + "\nResult:\n" + "true|false (boolean) If the signature is verified or not.\n" + "\nExamples:\n" + "\nUnlock the wallet for 30 seconds\n" + + HelpExampleCli("walletpassphrase", "\"mypassphrase\" 30") + + "\nCreate the signature\n" + + HelpExampleCli("signmessage", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\" \"my message\"") + + "\nVerify the signature\n" + + HelpExampleCli("verifymessage", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\" \"signature\" \"my message\"") + + "\nAs json rpc\n" + + HelpExampleRpc("verifymessage", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\", \"signature\", \"my message\"") + )); + + mapHelpStrings.insert(std::make_pair("addmultisigaddress", + "addmultisigaddress nrequired [\"key\",...] ( \"account\" )\n" + "\nAdd a nrequired-to-sign multisignature address to the wallet.\n" + "Each key is a address or hex-encoded public key.\n" + "If 'account' is specified, assign address to that account.\n" + + "\nArguments:\n" + "1. nrequired (numeric, required) The number of required signatures out of the n keys or addresses.\n" + "2. \"keysobject\" (string, required) A json array of addresses or hex-encoded public keys\n" + " [\n" + " \"address\" (string) address or hex-encoded public key\n" + " ...,\n" + " ]\n" + "3. \"account\" (string, optional) An account to assign the addresses to.\n" + + "\nResult:\n" + "\"address\" (string) A address associated with the keys.\n" + + "\nExamples:\n" + "\nAdd a multisig address from 2 addresses\n" + + HelpExampleCli("addmultisigaddress", "2 \"[\\\"16sSauSf5pF2UkUwvKGq4qjNRzBZYqgEL5\\\",\\\"171sgjn4YtPu27adkKGrdDwzRTxnRkBfKV\\\"]\"") + + "\nAs json rpc call\n" + + HelpExampleRpc("addmultisigaddress", "2, \"[\\\"16sSauSf5pF2UkUwvKGq4qjNRzBZYqgEL5\\\",\\\"171sgjn4YtPu27adkKGrdDwzRTxnRkBfKV\\\"]\"") + )); + + mapHelpStrings.insert(std::make_pair("appendrawexchange", + "appendrawexchange hex txid vout ask-assets \n" + "\nAppends new offer/ask pair to existing exchange transaction adds fee if it completes exchange\n" + "Returns hex-encoded raw transaction.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. \"hex\" (string, required) The transaction hex string\n" + "2. \"txid\" (string, required) Transaction ID of the output prepared by preparelockunspent, if omitted last created output is used\n" + "3. \"vout\" (numeric, required) Output index\n" + "4. \"ask-assets\" (object, required) A json object of assets to ask\n" + " [\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " ]\n" + "\nResult:\n" + "{\n" + " \"hex\": \"value\", (string) The raw transaction with signature(s) (hex-encoded string)\n" + " \"complete\": true/false (boolean) if exchange is completed and can be sent \n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("appendrawexchange", "\"hexstring\" f4c3dd510dd55761015c9d96bff7793b0d501dd6f01a959fd7dd02478fb47dfb 1 \"{\\\"1234-5678-1234\\\":200}\"" ) + + HelpExampleRpc("appendrawexchange", "\"hexstring\",\"f4c3dd510dd55761015c9d96bff7793b0d501dd6f01a959fd7dd02478fb47dfb\",1,\"{\\\"1234-5678-1234\\\":200}\\\"") + )); + + mapHelpStrings.insert(std::make_pair("backupwallet", + "backupwallet \"destination\"\n" + "\nSafely copies wallet.dat to destination, which can be a directory or a path with filename.\n" + "\nArguments:\n" + "1. \"destination\" (string) The destination directory or file\n" + "\nExamples:\n" + + HelpExampleCli("backupwallet", "\"backup.dat\"") + + HelpExampleRpc("backupwallet", "\"backup.dat\"") + )); + + mapHelpStrings.insert(std::make_pair("combineunspent", + "combineunspent ( \"address(es)\" minconf maxcombines mininputs maxinputs maxtime )\n" + "\nOptimizes wallet performance by combining unspent txouts.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. \"address(es)\" (string, optional) Addresses to optimize (comma delimited). Default - \"*\", all.\n" + "2. minconf (numeric, optional) The minimum confirmations to filter. Default - 1\n" + "3. maxcombines (numeric, optional) Maximal number of transactions to send. Default - 1\n" + "4. mininputs (numeric, optional) Minimal number of txouts to combine in one transaction. Default - 10\n" + "5. maxinputs (numeric, optional) Maximal number of txouts to combine in one transaction. Default - 100\n" + "6. maxtime (numeric, optional) Maximal time for creating combining transactions, at least one transaction will be sent. Default - 15s\n" + "\nResult:\n" + "\"transactionids\" (Array) Array of transaction ids.\n" + "\nExamples:\n" + + HelpExampleCli("combineunspent", "\"*\" 1 100 5 20 120") + + HelpExampleCli("combineunspent", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" ") + + HelpExampleRpc("combineunspent", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\",1,100, 5, 20, 120" ) + )); + + mapHelpStrings.insert(std::make_pair("create", + "create stream \"stream-name\" open ( custom-fields )\n" + "\nCreates stream\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. entity-type (string, required) The only possible value: stream\n" + "2. \"stream-name\" (string, required) Stream name, if not \"\" should be unique.\n" + "3. open (boolean, required) Allow anyone to publish in this stream\n" + "4 custom-fields (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("create", "stream test false ") + + HelpExampleCli("create", "stream test false '{\"Description\":\"Test stream\"}'") + + HelpExampleRpc("create", "\"stream\", \"test\", false") + )); + + mapHelpStrings.insert(std::make_pair("createfrom", + "createfrom from-address stream \"stream-name\" open ( custom-fields )\n" + "\nCreates stream using specific address\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address used for creating.\n" + "2. entity-type (string, required) The only possible value: stream\n" + "3. \"stream-name\" (string, required) Stream name, if not \"\" should be unique.\n" + "4. open (boolean, required) Allow anyone to publish in this stream\n" + "5 custom-fields (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("createfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" stream test false ") + + HelpExampleCli("createfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" stream test false '{\"Description\":\"Test stream\"}'") + + HelpExampleRpc("createfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"stream\", \"test\", false") + )); + + mapHelpStrings.insert(std::make_pair("createrawexchange", + "createrawexchange txid vout ask-assets\n" + "\nCreates new exchange transaction\n" + "Returns hex-encoded raw transaction.\n" + "Note that the transaction should be completed by appendrawexchange\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. \"txid\" (string, required) Transaction ID of the output prepared by preparelockunspent, if omitted last created output is used\n" + "2. \"vout\" (numeric, required) Output index\n" + "3. \"ask-assets\" (object, required) A json object of assets to ask\n" + " [\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " ]\n" + "\nResult:\n" + "\"transaction\" (string) hex string of the transaction\n" + "\nExamples:\n" + + HelpExampleCli("createrawexchange", "f4c3dd510dd55761015c9d96bff7793b0d501dd6f01a959fd7dd02478fb47dfb 1 \"{\\\"1234-5678-1234\\\":200}\"") + + HelpExampleRpc("createrawexchange", "\"f4c3dd510dd55761015c9d96bff7793b0d501dd6f01a959fd7dd02478fb47dfb\",1,\"{\\\"1234-5678-1234\\\":200}\\\"") + )); + +} + +void mc_InitRPCHelpMap07() +{ + mapHelpStrings.insert(std::make_pair("createrawsendfrom", + "createrawsendfrom from-address {\"address\":amount,...} ( [data] action ) \n" + "\nCreate a transaction using the given sending address.\n" + + "\nArguments:\n" + "1. from-address (string, required) Address to send from.\n" + "2. \"addresses\" (string, required) a json object with addresses as keys and amounts as values\n" + " {\n" + " \"address\": \n" + " x.xxx (numeric, required) The key is the address, the value is the native currency amount\n" + " or \n" + " { (object) A json object of assets to send\n" + " \"asset-identifier\" : asset-quantity \n" + " ,...\n" + " }\n" + " or \n" + " { (object) A json object describing new asset issue\n" + " \"issue\" : \n" + " {\n" + " \"raw\" : n (numeric, required) The asset total amount in raw units \n" + " ,...\n" + " }\n" + " ,...\n" + " }\n" + " or \n" + " { (object) A json object describing follow-on asset issue\n" + " \"issuemore\" : \n" + " {\n" + " \"asset\" : \"asset-identifier\" (string, required) Asset identifier - one of the following: issue txid. asset reference, asset name.\n" + " \"raw\" : n (numeric, required) The asset total amount in raw units \n" + " ,...\n" + " }\n" + " ,...\n" + " }\n" + " or \n" + " { (object) A json object describing permission change\n" + " \"permissions\" : \n" + " {\n" + " \"type\" : \"permission(s)\" (string,required) Permission strings, comma delimited. Possible values: " + AllowedPermissions() + " \n" + " \"startblock\" (numeric, optional) Block to apply permissions from (inclusive). Default - 0\n" + " \"endblock\" (numeric, optional) Block to apply permissions to (exclusive). Default - 4294967295\n" + " \"timestamp\" (numeric, optional) This helps resolve conflicts between permissions assigned by the same administrator. Default - current time\n" + " ,...\n" + " }\n" + " ,...\n" + " }\n" + " ,...\n" + " }\n" + "3. data (array, optional) Array of hexadecimal strings or data objects, see help appendrawdata for details.\n" + "4. action (string, optional, default \"\") Additional actions: \"lock\", \"sign\", \"lock,sign\", \"sign,lock\", \"send\". \n" + + + "\nResult:\n" + "\"transaction\" (string) hex string of the transaction (if action= \"\" or \"lock\")\n" + " or \n" + "{ (object) A json object (if action= \"sign\" or \"lock,sign\" or \"sign,lock\")\n" + " \"hex\": \"value\", (string) The raw transaction with signature(s) (hex-encoded string)\n" + " \"complete\": n (numeric) if transaction has a complete set of signature (0 if not)\n" + "}\n" + " or \n" + "\"hex\" (string) The transaction hash in hex (if action= \"send\")\n" + + "\nExamples\n" + + HelpExampleCli("createrawsendfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"address\\\":0.01}\"") + + HelpExampleRpc("createrawsendfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"{\\\"address\\\":0.01}\"") + )); + + mapHelpStrings.insert(std::make_pair("decoderawexchange", + "decoderawexchange tx-hex ( verbose )\n" + "\nReturn a JSON object representing the serialized, hex-encoded exchange transaction.\n" + + "\nArguments:\n" + "1. tx-hex (string, required) The exchange transaction hex string\n" + "2. verbose (boolean, optional, default=false) If true, returns array of all exchanges created by createrawexchange or appendrawexchange\n" + + "\nResults is an object with exchange details\n" + "\nExamples:\n" + + HelpExampleCli("decoderawtransaction", "\"hexstring\"") + + HelpExampleRpc("decoderawtransaction", "\"hexstring\"") + )); + + mapHelpStrings.insert(std::make_pair("disablerawtransaction", + "disablerawtransaction tx-hex\n" + "\nDisable raw transaction by spending one of its inputs and sending it back to the wallet.\n" + + "\nArguments:\n" + "1. tx-hex (string, required) The transaction hex string\n" + + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("disablerawtransaction", "\"hexstring\"") + + HelpExampleRpc("disablerawtransaction", "\"hexstring\"") + )); + + mapHelpStrings.insert(std::make_pair("dumpprivkey", + "dumpprivkey address\n" + "\nReveals the private key corresponding to 'address'.\n" + "Then the importprivkey can be used with this output\n" + "\nArguments:\n" + "1. address (string, required) The MultiChain address for the private key\n" + "\nResult:\n" + "\"key\" (string) The private key\n" + "\nExamples:\n" + + HelpExampleCli("dumpprivkey", "\"myaddress\"") + + HelpExampleCli("importprivkey", "\"mykey\"") + + HelpExampleRpc("dumpprivkey", "\"myaddress\"") + )); + + mapHelpStrings.insert(std::make_pair("dumpwallet", + "dumpwallet \"filename\"\n" + "\nDumps all wallet keys in a human-readable format.\n" + "\nArguments:\n" + "1. \"filename\" (string, required) The filename\n" + "\nExamples:\n" + + HelpExampleCli("dumpwallet", "\"test\"") + + HelpExampleRpc("dumpwallet", "\"test\"") + )); + + mapHelpStrings.insert(std::make_pair("encryptwallet", + "encryptwallet \"passphrase\"\n" + "\nEncrypts the wallet with 'passphrase'. This is for first time encryption.\n" + "After this, any calls that interact with private keys such as sending or signing \n" + "will require the passphrase to be set prior the making these calls.\n" + "Use the walletpassphrase call for this, and then walletlock call.\n" + "If the wallet is already encrypted, use the walletpassphrasechange call.\n" + "Note that this will shutdown the server.\n" + "\nArguments:\n" + "1. \"passphrase\" (string) The pass phrase to encrypt the wallet with. It must be at least 1 character, but should be long.\n" + "\nExamples:\n" + "\nEncrypt you wallet\n" + + HelpExampleCli("encryptwallet", "\"my pass phrase\"") + + "\nNow set the passphrase to use the wallet, such as for signing or sending assets\n" + + HelpExampleCli("walletpassphrase", "\"my pass phrase\"") + + "\nNow we can so something like sign\n" + + HelpExampleCli("signmessage", "\"address\" \"test message\"") + + "\nNow lock the wallet again by removing the passphrase\n" + + HelpExampleCli("walletlock", "") + + "\nAs a json rpc call\n" + + HelpExampleRpc("encryptwallet", "\"my pass phrase\"") + )); + + mapHelpStrings.insert(std::make_pair("getaccount", + "getaccount address\n" + "\nReturns the account associated with the given address.\n" + "\nArguments:\n" + "1. address (string, required) The address for account lookup.\n" + "\nResult:\n" + "\"accountname\" (string) the account address\n" + "\nExamples:\n" + + HelpExampleCli("getaccount", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\"") + + HelpExampleRpc("getaccount", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\"") + )); + + mapHelpStrings.insert(std::make_pair("getaccountaddress", + "getaccountaddress \"account\"\n" + "\nReturns the current address for receiving payments to this account.\n" + "\nArguments:\n" + "1. \"account\" (string, required) The account name for the address. It can also be set to the empty string \"\" to represent the default account. The account does not need to exist, it will be created and a new address created if there is no account by the given name.\n" + "\nResult:\n" + "\"address\" (string) The account address\n" + "\nExamples:\n" + + HelpExampleCli("getaccountaddress", "") + + HelpExampleCli("getaccountaddress", "\"\"") + + HelpExampleCli("getaccountaddress", "\"myaccount\"") + + HelpExampleRpc("getaccountaddress", "\"myaccount\"") + )); + + mapHelpStrings.insert(std::make_pair("getaddressbalances", + "getaddressbalances address ( minconf includeLocked ) \n" + "\nReturns asset balances for specified address\n" + "\nArguments:\n" + "1. address (string, required) Address to return balance for.\n" + "2. minconf (numeric, optional, default=1) Only include transactions confirmed at least this many times.\n" + "3. includeLocked (bool, optional, default=false) Also take locked outputs into account\n" + "Results are an array of Objects with totals and details for each asset.\n" + "\nExamples:\n" + "\nThe total amount in the server across all accounts\n" + + HelpExampleCli("getaddressbalances", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\"") + + HelpExampleCli("getaddressbalances", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0 true") + + HelpExampleRpc("getaddressbalances", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\"") + )); + + mapHelpStrings.insert(std::make_pair("getaddresses", + "getaddresses ( verbose )\n" + "\nReturns the list of all addresses in the wallet.\n" + "\nArguments: \n" + "1. \"verbose\" (boolean, optional, default=false) The account name.\n" + "\nResult:\n" + "[ (json array of )\n" + " \"address\" (string) an address \n" + " or \n" + " \"address-datails\" (object) address details if verbose=true\n" + " ,...\n" + "]\n" + "\nExamples:\n" + + HelpExampleCli("getaddresses", "") + + HelpExampleRpc("getaddresses", "") + )); + +} + +void mc_InitRPCHelpMap08() +{ + mapHelpStrings.insert(std::make_pair("getaddressesbyaccount", + "getaddressesbyaccount \"account\"\n" + "\nReturns the list of addresses for the given account.\n" + "\nArguments:\n" + "1. \"account\" (string, required) The account name.\n" + "\nResult:\n" + "[ (json array of string)\n" + " \"address\" (string) an address associated with the given account\n" + " ,...\n" + "]\n" + "\nExamples:\n" + + HelpExampleCli("getaddressesbyaccount", "\"tabby\"") + + HelpExampleRpc("getaddressesbyaccount", "\"tabby\"") + )); + + mapHelpStrings.insert(std::make_pair("getaddresstransaction", + "getaddresstransaction address txid ( verbose )\n" + "\nGet detailed information about in-wallet transaction \n" + "\nArguments:\n" + "1. address (string, required) Address used for balance calculation.\n" + "2. txid (string, required) The transaction id\n" + "3. verbose (bool, optional, default=false) If true, returns detailed array of inputs and outputs and raw hex of transactions\n" + "\nResult:\n" + "\nResult:\n" + "[\n" + " {\n" + " \"balance\": {...}, (object) Changes in address balance. \n" + " {\n" + " \"amount\": x.xxx, (numeric) The amount in native currency. Negative value means amount was send by the wallet, positive - received\n" + " \"assets\": {...}, (object) Changes in asset amounts. \n" + " }\n" + " \"myaddresses\": [...], (array) Address passed as parameter\n" + " \"addresses\": [...], (array) Array of counterparty addresses involved in transaction \n" + " \"permissions\": [...], (array) Changes in permissions \n" + " \"issue\": {...}, (object) Issue details \n" + " \"data\" : \"metadata\", (array) Hexadecimal representation of metadata appended to the transaction\n" + " \"confirmations\": n, (numeric) The number of confirmations for the transaction. Available for 'send' and \n" + " 'receive' category of transactions.\n" + " \"blockhash\": \"hashvalue\",(string) The block hash containing the transaction. Available for 'send' and 'receive'\n" + " category of transactions.\n" + " \"blockindex\": n, (numeric) The block index containing the transaction. Available for 'send' and 'receive'\n" + " category of transactions.\n" + " \"txid\": \"transactionid\", (string) The transaction id. Available for 'send' and 'receive' category of transactions.\n" + " \"time\": xxx, (numeric) The transaction time in seconds since epoch (midnight Jan 1 1970 GMT).\n" + " \"timereceived\": xxx, (numeric) The time received in seconds since epoch (midnight Jan 1 1970 GMT). Available \n" + " for 'send' and 'receive' category of transactions.\n" + " \"comment\": \"...\", (string) If a comment is associated with the transaction.\n" + " \"vin\": [...], (array) If verbose=true. Array of input details\n" + " \"vout\": [...], (array) If verbose=true. Array of output details\n" + " \"hex\" : \"data\" (string) If verbose=true. Raw data for transaction\n" + " }\n" + "]\n" + + "\nExamples:\n" + + HelpExampleCli("getaddresstransaction", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1075db55d416d3ca199f55b6084e2115b9345e16c5cf302fc80e9d5fbf5d48d\"") + + HelpExampleCli("getaddresstransaction", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1075db55d416d3ca199f55b6084e2115b9345e16c5cf302fc80e9d5fbf5d48d\" true") + + HelpExampleRpc("getaddresstransaction", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1075db55d416d3ca199f55b6084e2115b9345e16c5cf302fc80e9d5fbf5d48d\"") + )); + + mapHelpStrings.insert(std::make_pair("getassetbalances", + "getassetbalances ( \"account\" minconf includeWatchonly includeLocked )\n" + "\nIf account is not specified, returns the server's total available asset balances.\n" + "If account is specified, returns the balances in the account.\n" + "Note that the account \"\" is not the same as leaving the parameter out.\n" + "The server total may be different to the balance in the default \"\" account.\n" + "\nArguments:\n" + "1. \"account\" (string, optional) The selected account, or \"*\" for entire wallet. It may be the default account using \"\".\n" + "2. minconf (numeric, optional, default=1) Only include transactions confirmed at least this many times.\n" + "3. includeWatchonly (bool, optional, default=false) Also include balance in watchonly addresses (see 'importaddress')\n" + "4. includeLocked (bool, optional, default=false) Also take locked outputs into account\n" + "Results are an array of Objects with totals and details for each asset.\n" + "\nExamples:\n" + "\nThe total amount in the server across all accounts\n" + + HelpExampleCli("getassetbalances", "") + + "\nThe total amount in the server across all accounts, with at least 5 confirmations\n" + + HelpExampleCli("getassetbalances", "\"*\" 6") + + "\nThe total amount in the default account with at least 1 confirmation\n" + + HelpExampleCli("getassetbalances", "\"\"") + + "\nThe total amount in the account named tabby with at least 6 confirmations\n" + + HelpExampleCli("getassetbalances", "\"tabby\" 6 true") + + "\nAs a json rpc call\n" + + HelpExampleRpc("getassetbalances", "\"tabby\", 6") + )); + + mapHelpStrings.insert(std::make_pair("getassettransaction", + "getassettransaction \"asset-identifier\" txid ( verbose )\n" + "\nReturns stream item.\n" + "\nArguments:\n" + "1. \"asset-identifier\"(string, required) Asset identifier - one of the following: asset txid, asset reference, asset name.\n" + "2. txid (string, required) The transaction id\n" + "3. verbose (boolean, optional, default=false) If true, returns information about item transaction \n" + "\nResult:\n" + "\"transaction\" (object) Information about an individual transaction from the perspective of a particular asset.\n" + "\nExamples:\n" + + HelpExampleCli("getassettransaction", "\"mytxid\"") + + HelpExampleCli("getassettransaction", "\"mytxid\" true") + + HelpExampleRpc("getassettransaction", "\"mytxid\", false") + )); + + mapHelpStrings.insert(std::make_pair("getbalance", + "getbalance ( \"account\" minconf includeWatchonly )\n" + "\nIf account is not specified, returns the server's total available balance.\n" + "If account is specified, returns the balance in the account.\n" + "Note that the account \"\" is not the same as leaving the parameter out.\n" + "The server total may be different to the balance in the default \"\" account.\n" + "\nArguments:\n" + "1. \"account\" (string, optional) The selected account, or \"*\" for entire wallet. It may be the default account using \"\".\n" + "2. minconf (numeric, optional, default=1) Only include transactions confirmed at least this many times.\n" + "3. includeWatchonly (bool, optional, default=false) Also include balance in watchonly addresses (see 'importaddress')\n" + "\nResult:\n" + "amount (numeric) The total amount in native currency received for this account.\n" + "\nExamples:\n" + "\nThe total amount in the server across all accounts\n" + + HelpExampleCli("getbalance", "") + + "\nThe total amount in the server across all accounts, with at least 5 confirmations\n" + + HelpExampleCli("getbalance", "\"*\" 6") + + "\nThe total amount in the default account with at least 1 confirmation\n" + + HelpExampleCli("getbalance", "\"\"") + + "\nThe total amount in the account named tabby with at least 6 confirmations\n" + + HelpExampleCli("getbalance", "\"tabby\" 6") + + "\nAs a json rpc call\n" + + HelpExampleRpc("getbalance", "\"tabby\", 6") + )); + + mapHelpStrings.insert(std::make_pair("getmultibalances", + "getmultibalances ( \"address(es)\" assets minconf includeLocked includeWatchonly ) \n" + "\nReturns asset balances for specified address\n" + "\nArguments:\n" + "1. \"address(es)\" (string, optional) Address(es) to return balance for, comma delimited. Default - all\n" + "or\n" + "1. \"address(es)\" (array, optional) A json array of addresses to return balance for\n" + "2. \"assets\" (array or string, optional) Single asset identifier or json array of asset identifiers to return balance for, default \"*\"\n" + "3. minconf (numeric, optional, default=1) Only include transactions confirmed at least this many times.\n" + "4. includeWatchonly (bool, optional, default=false) Include transactions to watchonly addresses (see 'importaddress')\n" + "5. includeLocked (bool, optional, default=false) Also take locked outputs into account\n" + "Results are an Object of balance arrays with totals and details for each asset.\n" + "\nExamples:\n" + + HelpExampleCli("getmultibalances", "") + + HelpExampleCli("getmultibalances", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\"") + + HelpExampleRpc("getmultibalances", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\"") + )); + + mapHelpStrings.insert(std::make_pair("getnewaddress", + "getnewaddress ( \"account\" )\n" + "\nReturns a new address for receiving payments.\n" + "If 'account' is specified (recommended), it is added to the address book \n" + "so payments received with the address will be credited to 'account'.\n" + "\nArguments:\n" + "1. \"account\" (string, optional) The account name for the address to be linked to. if not provided, the default account \"\" is used. It can also be set to the empty string \"\" to represent the default account. The account does not need to exist, it will be created if there is no account by the given name.\n" + "\nResult:\n" + "\"address\" (string) The new address\n" + "\nExamples:\n" + + HelpExampleCli("getnewaddress", "") + + HelpExampleCli("getnewaddress", "\"\"") + + HelpExampleCli("getnewaddress", "\"myaccount\"") + + HelpExampleRpc("getnewaddress", "\"myaccount\"") + )); + + mapHelpStrings.insert(std::make_pair("getrawchangeaddress", + "getrawchangeaddress\n" + "\nReturns a new address, for receiving change.\n" + "This is for use with raw transactions, NOT normal use.\n" + "\nResult:\n" + "\"address\" (string) The address\n" + "\nExamples:\n" + + HelpExampleCli("getrawchangeaddress", "") + + HelpExampleRpc("getrawchangeaddress", "") + )); + + mapHelpStrings.insert(std::make_pair("getreceivedbyaccount", + "getreceivedbyaccount \"account\" ( minconf )\n" + "\nReturns the total amount received by addresses with in transactions with at least [minconf] confirmations.\n" + "\nArguments:\n" + "1. \"account\" (string, required) The selected account, may be the default account using \"\".\n" + "2. minconf (numeric, optional, default=1) Only include transactions confirmed at least this many times.\n" + "\nResult:\n" + "amount (numeric) The total amount in native currency received for this account.\n" + "\nExamples:\n" + "\nAmount received by the default account with at least 1 confirmation\n" + + HelpExampleCli("getreceivedbyaccount", "\"\"") + + "\nAmount received at the tabby account including unconfirmed amounts with zero confirmations\n" + + HelpExampleCli("getreceivedbyaccount", "\"tabby\" 0") + + "\nThe amount with at least 6 confirmation, very safe\n" + + HelpExampleCli("getreceivedbyaccount", "\"tabby\" 6") + + "\nAs a json rpc call\n" + + HelpExampleRpc("getreceivedbyaccount", "\"tabby\", 6") + )); + + mapHelpStrings.insert(std::make_pair("getreceivedbyaddress", + "getreceivedbyaddress address ( minconf )\n" + "\nReturns the total amount received by the given address in transactions with at least minconf confirmations.\n" + "\nArguments:\n" + "1. address (string, required) The address for transactions.\n" + "2. minconf (numeric, optional, default=1) Only include transactions confirmed at least this many times.\n" + "\nResult:\n" + "amount (numeric) The total amount in native currency received at this address.\n" + "\nExamples:\n" + "\nThe amount from transactions with at least 1 confirmation\n" + + HelpExampleCli("getreceivedbyaddress", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\"") + + "\nThe amount including unconfirmed transactions, zero confirmations\n" + + HelpExampleCli("getreceivedbyaddress", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\" 0") + + "\nThe amount with at least 6 confirmation, very safe\n" + + HelpExampleCli("getreceivedbyaddress", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\" 6") + + "\nAs a json rpc call\n" + + HelpExampleRpc("getreceivedbyaddress", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\", 6") + )); + +} + +void mc_InitRPCHelpMap09() +{ + mapHelpStrings.insert(std::make_pair("getstreamitem", + "getstreamitem \"stream-identifier\" txid ( verbose )\n" + "\nReturns stream item.\n" + "\nArguments:\n" + "1. \"stream-identifier\"(string, required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + "2. txid (string, required) The transaction id\n" + "3. verbose (boolean, optional, default=false) If true, returns information about item transaction \n" + "\nResult:\n" + "\"stream-item\" (object) Stream item.\n" + "\nExamples:\n" + + HelpExampleCli("getstreamitem", "\"mytxid\"") + + HelpExampleCli("getstreamitem", "\"mytxid\" true") + + HelpExampleRpc("getstreamitem", "\"mytxid\", false") + )); + + mapHelpStrings.insert(std::make_pair("gettotalbalances", + "gettotalbalances ( minconf includeWatchonly includeLocked )\n" + "\nIf account is not specified, returns the server's total available asset balances.\n" + "If account is specified, returns the balances in the account.\n" + "Note that the account \"\" is not the same as leaving the parameter out.\n" + "The server total may be different to the balance in the default \"\" account.\n" + "\nArguments:\n" + "1. minconf (numeric, optional, default=1) Only include transactions confirmed at least this many times.\n" + "2. includeWatchonly (bool, optional, default=false) Also include balance in watchonly addresses (see 'importaddress')\n" + "3. includeLocked (bool, optional, default=false) Also take locked outputs into account\n" + "Results are an array of Objects with totals and details for each asset.\n" + "\nExamples:\n" + "\nThe total amount in the server across all accounts\n" + + HelpExampleCli("gettotalbalances", "") + + "\nThe total amount in the server across all accounts, with at least 5 confirmations\n" + + HelpExampleCli("gettotalbalances", "6") + + "\nThe total amount in the default account with at least 1 confirmation\n" + + HelpExampleCli("gettotalbalances", "") + + "\nThe total amount in the account named tabby with at least 6 confirmations\n" + + HelpExampleCli("gettotalbalances", "6 true") + + "\nAs a json rpc call\n" + + HelpExampleRpc("gettotalbalances", "\"tabby\", 6") + )); + + mapHelpStrings.insert(std::make_pair("gettransaction", + "gettransaction txid ( includeWatchonly )\n" + "\nGet detailed information about in-wallet transaction \n" + "\nArguments:\n" + "1. txid (string, required) The transaction id\n" + "2. \"includeWatchonly\" (bool, optional, default=false) Whether to include watchonly addresses in balance calculation and details[]\n" + "\nResult:\n" + "{\n" + " \"amount\" : x.xxx, (numeric) The transaction amount in native currency\n" + " \"confirmations\" : n, (numeric) The number of confirmations\n" + " \"blockhash\" : \"hash\", (string) The block hash\n" + " \"blockindex\" : xx, (numeric) The block index\n" + " \"blocktime\" : ttt, (numeric) The time in seconds since epoch (1 Jan 1970 GMT)\n" + " \"txid\" : \"transactionid\", (string) The transaction id.\n" + " \"time\" : ttt, (numeric) The transaction time in seconds since epoch (1 Jan 1970 GMT)\n" + " \"timereceived\" : ttt, (numeric) The time received in seconds since epoch (1 Jan 1970 GMT)\n" + " \"details\" : [\n" + " {\n" + " \"account\" : \"accountname\", (string) The account name involved in the transaction, can be \"\" for the default account.\n" + " \"address\" : \"address\", (string) The address involved in the transaction\n" + " \"category\" : \"send|receive\", (string) The category, either 'send' or 'receive'\n" + " \"amount\" : x.xxx (numeric) The amount in native currency\n" + " \"vout\" : n, (numeric) the vout value\n" + " }\n" + " ,...\n" + " ],\n" + " \"hex\" : \"data\" (string) Raw data for transaction\n" + "}\n" + + "\nExamples:\n" + + HelpExampleCli("gettransaction", "\"1075db55d416d3ca199f55b6084e2115b9345e16c5cf302fc80e9d5fbf5d48d\"") + + HelpExampleCli("gettransaction", "\"1075db55d416d3ca199f55b6084e2115b9345e16c5cf302fc80e9d5fbf5d48d\" true") + + HelpExampleRpc("gettransaction", "\"1075db55d416d3ca199f55b6084e2115b9345e16c5cf302fc80e9d5fbf5d48d\"") + )); + + mapHelpStrings.insert(std::make_pair("gettxoutdata", + "gettxoutdata txid n ( count-bytes start-byte )\n" + "\nReturns metadata of transaction output.\n" + "\nArguments:\n" + "1. txid (string, required) The transaction id\n" + "2. n (numeric, required) vout value\n" + "3. count-bytes (numeric, optional, default=INT_MAX) Number of bytes to return\n" + "4. start-byte (numeric, optional, default=0) start from specific byte \n" + "\nResult:\n" + "\"data-hex\" (string) transaction output metadata in hexadecimal form.\n" + "\nExamples:\n" + "\nView the data\n" + + HelpExampleCli("gettxoutdata", "\"txid\" 1") + + "\nAs a json rpc call\n" + + HelpExampleRpc("gettxoutdata", "\"txid\", 1") + )); + + mapHelpStrings.insert(std::make_pair("getunconfirmedbalance", + "getunconfirmedbalance\n" + "Returns the server's total unconfirmed balance\n" + )); + + mapHelpStrings.insert(std::make_pair("getwalletinfo", + "getwalletinfo\n" + "Returns an object containing various wallet state info.\n" + "\nResult:\n" + "{\n" + " \"walletversion\": xxxxx, (numeric) the wallet version\n" + " \"balance\": xxxxxxx, (numeric) the total native currency balance of the wallet\n" + " \"txcount\": xxxxxxx, (numeric) the total number of transactions in the wallet\n" + " \"walletdbversion\": xxxxx, (numeric) the wallet database version\n" + " \"keypoololdest\": xxxxxx, (numeric) the timestamp (seconds since GMT epoch) of the oldest pre-generated key in the key pool\n" + " \"keypoolsize\": xxxx, (numeric) how many new keys are pre-generated\n" + " \"unlocked_until\": ttt, (numeric) the timestamp in seconds since epoch (midnight Jan 1 1970 GMT) that the wallet is unlocked for transfers, or 0 if the wallet is locked\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("getwalletinfo", "") + + HelpExampleRpc("getwalletinfo", "") + )); + + mapHelpStrings.insert(std::make_pair("getwallettransaction", + "getwallettransaction txid ( includeWatchonly verbose )\n" + "\nGet detailed information about in-wallet transaction \n" + "\nArguments:\n" + "1. txid (string, required) The transaction id\n" + "2. \"includeWatchonly\" (bool, optional, default=false) Whether to include watchonly addresses in balance calculation and details[]\n" + "3. verbose (bool, optional, default=false) If true, returns detailed array of inputs and outputs and raw hex of transactions\n" + "\nResult:\n" + "\nResult:\n" + "[\n" + " {\n" + " \"balance\": {...}, (object) Changes in wallet balance. \n" + " {\n" + " \"amount\": x.xxx, (numeric) The amount in native currency. Negative value means amount was send by the wallet, positive - received\n" + " \"assets\": {...}, (object) Changes in asset amounts. \n" + " }\n" + " \"myaddresses\": [...], (array) Array of wallet addresses involved in transaction \n" + " \"addresses\": [...], (array) Array of counterparty addresses involved in transaction \n" + " \"permissions\": [...], (array) Changes in permissions \n" + " \"issue\": {...}, (object) Issue details \n" + " \"data\" : \"metadata\", (array) Hexadecimal representation of metadata appended to the transaction\n" + " \"confirmations\": n, (numeric) The number of confirmations for the transaction. Available for 'send' and \n" + " 'receive' category of transactions.\n" + " \"blockhash\": \"hashvalue\",(string) The block hash containing the transaction. Available for 'send' and 'receive'\n" + " category of transactions.\n" + " \"blockindex\": n, (numeric) The block index containing the transaction. Available for 'send' and 'receive'\n" + " category of transactions.\n" + " \"txid\": \"transactionid\", (string) The transaction id. Available for 'send' and 'receive' category of transactions.\n" + " \"time\": xxx, (numeric) The transaction time in seconds since epoch (midnight Jan 1 1970 GMT).\n" + " \"timereceived\": xxx, (numeric) The time received in seconds since epoch (midnight Jan 1 1970 GMT). Available \n" + " for 'send' and 'receive' category of transactions.\n" + " \"comment\": \"...\", (string) If a comment is associated with the transaction.\n" + " \"vin\": [...], (array) If verbose=true. Array of input details\n" + " \"vout\": [...], (array) If verbose=true. Array of output details\n" + " \"hex\" : \"data\" (string) If verbose=true. Raw data for transaction\n" + " }\n" + "]\n" + + "\nExamples:\n" + + HelpExampleCli("getwallettransaction", "\"1075db55d416d3ca199f55b6084e2115b9345e16c5cf302fc80e9d5fbf5d48d\"") + + HelpExampleCli("getwallettransaction", "\"1075db55d416d3ca199f55b6084e2115b9345e16c5cf302fc80e9d5fbf5d48d\" true") + + HelpExampleRpc("getwallettransaction", "\"1075db55d416d3ca199f55b6084e2115b9345e16c5cf302fc80e9d5fbf5d48d\"") + )); + + mapHelpStrings.insert(std::make_pair("grant", + "grant \"address(es)\" \"permission(s)\" ( native-amount \"comment\" \"comment-to\" startblock endblock )\n" + "\nGrant permission(s) to a given address. \n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. \"address(es)\" (string, required) The multichain addresses to send to (comma delimited)\n" + "2. \"permission(s)\" (string, required) Permission strings, comma delimited. Possible values: " + AllowedPermissions() + " \n" + "3. native-amount (numeric, optional) native currency amount to send. eg 0.1. Default - 0.0\n" + "4. startblock (numeric, optional) Block to apply permissions from (inclusive). Default - 0\n" + "5. endblock (numeric, optional) Block to apply permissions to (exclusive). Default - 4294967295\n" + " If -1 is specified default value is used.\n" + "6. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "7. \"comment-to\" (string, optional) A comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the \n" + " transaction, just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("grant", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 connect,send,receive") + + HelpExampleCli("grant", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 mystream.admin,write") + + HelpExampleCli("grant", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 mine \"permission to mine\" \"Miners Ltd.\"") + + HelpExampleRpc("grant", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, admin \"temporary admin\", \"Admins Ltd.\" 20000 30000") + )); + + mapHelpStrings.insert(std::make_pair("grantfrom", + "grantfrom from-address \"to-address(es)\" \"permission(s)\" ( native-amount \"comment\" \"comment-to\" startblock endblock )\n" + "\nGrant permission using specific address.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address used for grant.\n" + "2. \"to-address(es)\" (string, required) The multichain addresses to grant permissions to\n" + "3. \"permission(s)\" (string, required) Permission strings, comma delimited. Possible values: " + AllowedPermissions() + " \n" + "4. native-amount (numeric, optional) native currency amount to send. eg 0.1. Default - 0.0\n" + "5. startblock (numeric, optional) Block to apply permissions from (inclusive). Default - 0\n" + "6. endblock (numeric, optional) Block to apply permissions to (exclusive). Default - 4294967295\n" + " If -1 is specified default value is used.\n" + "7. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "8. \"comment-to\" (string, optional) A comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the \n" + " transaction, just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("grantfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 connect,send,receive") + + HelpExampleCli("grantfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 mine \"permission to mine\" \"Miners Ltd.\"") + + HelpExampleRpc("grantfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, admin \"temporary admin\", \"Admins Ltd.\" 20000 30000") + )); + + mapHelpStrings.insert(std::make_pair("grantwithdata", + "grantwithdata \"address(es)\" \"permission(s)\" data-hex|object ( native-amount startblock endblock )\n" + "\nGrant permission(s) with metadata to a given address. \n" + "\nArguments:\n" + "1. \"address(es)\" (string, required) The multichain addresses to send to (comma delimited)\n" + "2. \"permission(s)\" (string, required) Permission strings, comma delimited. Possible values: " + AllowedPermissions() + " \n" + "3. data-hex (string, required) Data hex string\n" + "or\n" + "3. publish-new-stream-item (object, required) A json object with stream item\n" + " {\n" + " \"for\" : stream-identifier (string,required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + " \"key\" : key (string,optional, default: \"\") Item key\n" + " \"data\" : data-hex (string,optional, default: \"\") Data hex string\n" + " }\n" + "4. native-amount (numeric, optional) native currency amount to send. eg 0.1. Default - 0.0\n" + "5. startblock (numeric, optional) Block to apply permissions from (inclusive). Default - 0\n" + "6. endblock (numeric, optional) Block to apply permissions to (exclusive). Default - 4294967295\n" + " If -1 is specified default value is used.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("grantwithdata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" connect,send,receive 48656C6C6F20576F726C64210A") + + HelpExampleCli("grantwithdata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" mine 48656C6C6F20576F726C64210A 0.1") + + HelpExampleRpc("grantwithdata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", admin, 48656C6C6F20576F726C64210A") + )); + +} + +void mc_InitRPCHelpMap10() +{ + mapHelpStrings.insert(std::make_pair("grantwithmetadata", + "grantwithdata \"address(es)\" \"permission(s)\" data-hex|object ( native-amount startblock endblock )\n" + "\nGrant permission(s) with metadata to a given address. \n" + "\nArguments:\n" + "1. \"address(es)\" (string, required) The multichain addresses to send to (comma delimited)\n" + "2. \"permission(s)\" (string, required) Permission strings, comma delimited. Possible values: " + AllowedPermissions() + " \n" + "3. data-hex (string, required) Data hex string\n" + "or\n" + "3. publish-new-stream-item (object, required) A json object with stream item\n" + " {\n" + " \"for\" : stream-identifier (string,required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + " \"key\" : key (string,optional, default: \"\") Item key\n" + " \"data\" : data-hex (string,optional, default: \"\") Data hex string\n" + " }\n" + "4. native-amount (numeric, optional) native currency amount to send. eg 0.1. Default - 0.0\n" + "5. startblock (numeric, optional) Block to apply permissions from (inclusive). Default - 0\n" + "6. endblock (numeric, optional) Block to apply permissions to (exclusive). Default - 4294967295\n" + " If -1 is specified default value is used.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("grantwithdata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" connect,send,receive 48656C6C6F20576F726C64210A") + + HelpExampleCli("grantwithdata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" mine 48656C6C6F20576F726C64210A 0.1") + + HelpExampleRpc("grantwithdata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", admin, 48656C6C6F20576F726C64210A") + )); + + mapHelpStrings.insert(std::make_pair("", + "grantwithdatafrom from-address \"to-address(es)\" \"permission(s)\" data-hex|object ( native-amount startblock endblock )\n" + "\nGrant permission with metadata using specific address.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address used for grant.\n" + "2. \"to-address(es)\" (string, required) The multichain addresses to grant permissions to\n" + "3. \"permission(s)\" (string, required) Permission strings, comma delimited. Possible values: " + AllowedPermissions() + " \n" + "4. data-hex (string, required) Data hex string\n" + "or\n" + "4. publish-new-stream-item (object, required) A json object with stream item\n" + " {\n" + " \"for\" : stream-identifier (string,required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + " \"key\" : key (string,optional, default: \"\") Item key\n" + " \"data\" : data-hex (string,optional, default: \"\") Data hex string\n" + " }\n" + "5. native-amount\" (numeric, optional) native currency amount to send. eg 0.1. Default - 0.0\n" + "6. startblock (numeric, optional) Block to apply permissions from (inclusive). Default - 0\n" + "7. endblock (numeric, optional) Block to apply permissions to (exclusive). Default - 4294967295\n" + " If -1 is specified default value is used.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("grantwithdatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" connect,send,receive 48656C6C6F20576F726C64210A") + + HelpExampleCli("grantwithdatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" mine 48656C6C6F20576F726C64210A 0.1 ") + + HelpExampleRpc("grantwithdatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", admin, 48656C6C6F20576F726C64210A") + )); + + mapHelpStrings.insert(std::make_pair("grantwithmetadatafrom", + "grantwithmetadatafrom from-address \"to-address(es)\" \"permission(s)\" data-hex|object ( native-amount startblock endblock )\n" + "\nGrant permission with metadata using specific address.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address used for grant.\n" + "2. \"to-address(es)\" (string, required) The multichain addresses to grant permissions to\n" + "3. \"permission(s)\" (string, required) Permission strings, comma delimited. Possible values: " + AllowedPermissions() + " \n" + "4. data-hex (string, required) Data hex string\n" + "or\n" + "4. publish-new-stream-item (object, required) A json object with stream item\n" + " {\n" + " \"for\" : stream-identifier (string,required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + " \"key\" : key (string,optional, default: \"\") Item key\n" + " \"data\" : data-hex (string,optional, default: \"\") Data hex string\n" + " }\n" + "5. native-amount\" (numeric, optional) native currency amount to send. eg 0.1. Default - 0.0\n" + "6. startblock (numeric, optional) Block to apply permissions from (inclusive). Default - 0\n" + "7. endblock (numeric, optional) Block to apply permissions to (exclusive). Default - 4294967295\n" + " If -1 is specified default value is used.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("grantwithmetadatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" connect,send,receive 48656C6C6F20576F726C64210A") + + HelpExampleCli("grantwithmetadatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" mine 48656C6C6F20576F726C64210A 0.1 ") + + HelpExampleRpc("grantwithmetadatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", admin, 48656C6C6F20576F726C64210A") + )); + + mapHelpStrings.insert(std::make_pair("importaddress", + "importaddress address(es) ( \"label\" rescan )\n" + "\nAdds an address or script (in hex) that can be watched as if it were in your wallet but cannot be used to spend.\n" + "\nArguments:\n" + "1. address(es) (string, required) The addresses, comma delimited\n" + "or\n" + "1. address(es) (array, optional) A json array of addresses \n" + "2. \"label\" (string, optional, default=\"\") An optional label\n" + "3. rescan (boolean, optional, default=true) Rescan the wallet for transactions\n" + "\nNote: This call can take minutes to complete if rescan is true.\n" + "\nExamples:\n" + "\nImport an address with rescan\n" + + HelpExampleCli("importaddress", "\"myaddress\"") + + "\nImport using a label without rescan\n" + + HelpExampleCli("importaddress", "\"myaddress\" \"testing\" false") + + "\nAs a JSON-RPC call\n" + + HelpExampleRpc("importaddress", "\"myaddress\", \"testing\", false") + )); + + mapHelpStrings.insert(std::make_pair("importprivkey", + "importprivkey privkey(s) ( \"label\" rescan )\n" + "\nAdds a private key (as returned by dumpprivkey) to your wallet.\n" + "\nArguments:\n" + "1. privkey(s) (string, required) The private key (see dumpprivkey), comma delimited\n" + "or\n" + "1. privkey(s) (array, optional) A json array of private keys \n" + "2. \"label\" (string, optional, default=\"\") An optional label\n" + "3. rescan (boolean, optional, default=true) Rescan the wallet for transactions\n" + "\nNote: This call can take minutes to complete if rescan is true.\n" + "\nExamples:\n" + "\nDump a private key\n" + + HelpExampleCli("dumpprivkey", "\"myaddress\"") + + "\nImport the private key with rescan\n" + + HelpExampleCli("importprivkey", "\"mykey\"") + + "\nImport using a label and without rescan\n" + + HelpExampleCli("importprivkey", "\"mykey\" \"testing\" false") + + "\nAs a JSON-RPC call\n" + + HelpExampleRpc("importprivkey", "\"mykey\", \"testing\", false") + )); + + mapHelpStrings.insert(std::make_pair("importwallet", + "importwallet \"filename\"\n" + "\nImports keys from a wallet dump file (see dumpwallet).\n" + "\nArguments:\n" + "1. \"filename\" (string, required) The wallet file\n" + "\nExamples:\n" + "\nDump the wallet\n" + + HelpExampleCli("dumpwallet", "\"test\"") + + "\nImport the wallet\n" + + HelpExampleCli("importwallet", "\"test\"") + + "\nImport using the json rpc call\n" + + HelpExampleRpc("importwallet", "\"test\"") + )); + + mapHelpStrings.insert(std::make_pair("issue", + "issue address \"asset-name\"|asset-params quantity ( smallest-unit native-amount custom-fields )\n" + "\nIssue new asset\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. \"address\" (string, required) The address to send newly created asset to.\n" + "2. \"asset-name\" (string, required) Asset name, if not \"\" should be unique.\n" + "or\n" + "2. asset-params (object, required) A json object of with asset params\n" + " {\n" + " \"name\" : \"asset-name\" (string, optional) Asset name\n" + " \"open\" : true|false (boolean, optional, default false) True if follow-on issues are allowed\n" + " ,...\n" + " }\n" + "3. quantity (numeric, required) The asset total amount in display units. eg. 1234.56\n" + "4. smallest-unit (numeric, optional, default=1) Number of raw units in one displayed unit, eg 0.01 for cents\n" + "5. native-amount (numeric, optional) native currency amount to send. eg 0.1, Default: minimum-per-output.\n" + "6 custom-fields (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("issue", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"Dollar\" 1000000") + + HelpExampleCli("issue", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" Dollar 1000000 0.01 0.01 ") + + HelpExampleRpc("issue", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"Dollar\", 1000000, 0.01, 0.01 \"description=1 Million dollars\"") + )); + + mapHelpStrings.insert(std::make_pair("issuefrom", + "issuefrom from-address to-address \"asset-name\"|asset-params quantity ( smallest-unit native-amount custom-fields )\n" + "\nIssue asset using specific address\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address used for issuing.\n" + "2. to-address (string, required) The address to send newly created asset to.\n" + "3. \"asset-name\" (string, required) Asset name, if not \"\" should be unique.\n" + "or\n" + "3. asset-params (object, required) A json object of with asset params\n" + " {\n" + " \"name\" : \"asset-name\" (string, optional) Asset name\n" + " \"open\" : true|false (boolean, optional, default false) True if follow-on issues are allowed\n" + " ,...\n" + " }\n" + "4. quantity (numeric, required) The asset total amount in display units. eg. 1234.56\n" + "5. smallest-unit (numeric, optional, default=1) Number of raw units in one displayed unit, eg 0.01 for cents\n" + "6. native-amount (numeric, optional) native currency amount to send. eg 0.1, Default: minimum-per-output.\n" + "7 custom-fields (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("issuefrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"Dollar\" 1000000") + + HelpExampleCli("issuefrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" Dollar 1000000 0.01 0.01 ") + + HelpExampleRpc("issuefrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"Dollar\", 1000000, 0.01, 0.01 \"description=1 Million dollars\"") + )); + + mapHelpStrings.insert(std::make_pair("issuemore", + "issuemore address \"asset-identifier\" quantity ( native-amount custom-fields )\n" + "\nCreate more units for asset\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. address (string, required) The address to send newly created asset to.\n" + "2. \"asset-identifier\" (string, required) Asset identifier - one of the following: issue txid, asset reference, asset name.\n" + "3. quantity (numeric, required) The asset total amount in display units. eg. 1234.56\n" + "4. native-amount (numeric, optional) native currency amount to send. eg 0.1, Default: minimum-per-output.\n" + "5 custom-fields (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("issuemore", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"Dollar\" 1000000") + + HelpExampleCli("issuemore", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" Dollar 1000000 0.01 ") + + HelpExampleRpc("issuemore", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"Dollar\", 1000000, 0.01 \"description=1 Million dollars\"") + )); + + mapHelpStrings.insert(std::make_pair("issuemorefrom", + "issuemorefrom from-address to-address \"asset-identifier\" quantity ( native-amount custom-fields )\n" + "\nCreate more units for asset from specific address\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address used for issuing.\n" + "2. to-address (string, required) The address to send newly created asset to.\n" + "3. \"asset-identifier\" (string, required) Asset identifier - one of the following: issue txid, asset reference, asset name.\n" + "4. quantity (numeric, required) The asset total amount in display units. eg. 1234.56\n" + "5. native-amount (numeric, optional) native currency amount to send. eg 0.1, Default: minimum-per-output.\n" + "6 custom-fields (object, optional) a json object with custom fields\n" + " {\n" + " \"param-name\": \"param-value\" (strings, required) The key is the parameter name, the value is parameter value\n" + " ,...\n" + " }\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("issuemorefrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"Dollar\" 1000000") + + HelpExampleCli("issuemorefrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" Dollar 1000000 0.01 ") + + HelpExampleRpc("issuemorefrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"Dollar\", 1000000, 0.01 \"description=1 Million dollars\"") + )); + +} + +void mc_InitRPCHelpMap11() +{ + mapHelpStrings.insert(std::make_pair("keypoolrefill", + "keypoolrefill ( newsize )\n" + "\nFills the keypool." + + HelpRequiringPassphraseWrapper() + "\n" + "\nArguments\n" + "1. newsize (numeric, optional, default=100) The new keypool size\n" + "\nExamples:\n" + + HelpExampleCli("keypoolrefill", "") + + HelpExampleRpc("keypoolrefill", "") + )); + + mapHelpStrings.insert(std::make_pair("listaccounts", + "listaccounts ( minconf includeWatchonly)\n" + "\nReturns Object that has account names as keys, account balances as values.\n" + "\nArguments:\n" + "1. minconf (numeric, optional, default=1) Only include transactions with at least this many confirmations\n" + "2. includeWatchonly (bool, optional, default=false) Include balances in watchonly addresses (see 'importaddress')\n" + "\nResult:\n" + "{ (json object where keys are account names, and values are numeric balances\n" + " \"account\": x.xxx, (numeric) The property name is the account name, and the value is the total balance for the account.\n" + " ...\n" + "}\n" + "\nExamples:\n" + "\nList account balances where there at least 1 confirmation\n" + + HelpExampleCli("listaccounts", "") + + "\nList account balances including zero confirmation transactions\n" + + HelpExampleCli("listaccounts", "0") + + "\nList account balances for 6 or more confirmations\n" + + HelpExampleCli("listaccounts", "6") + + "\nAs json rpc call\n" + + HelpExampleRpc("listaccounts", "6") + )); + + mapHelpStrings.insert(std::make_pair("listaddresses", + "listaddresses (address(es) verbose count start ) \n" + "\nReturns asset balances for specified address\n" + "\nArguments:\n" + "1. \"address(es)\" (string, optional, default *) Address(es) to return balance for, comma delimited. Default - all\n" + "or\n" + "1. \"address(es)\" (array, optional) A json array of addresses to return balance for\n" + "2. \"verbose\" (boolean, optional, default=false) If true return more information about address.\n" + "3. count (number, optional, default=INT_MAX - all) The number of addresses to display\n" + "4. start (number, optional, default=-count - last) Start from specific address, 0 based, if negative - from the end\n" + "Results are an Array of address Objects.\n" + "\nExamples:\n" + + HelpExampleCli("listaddresses", "") + + HelpExampleCli("listaddresses", "\"*\" true") + + HelpExampleCli("listaddresses", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" true") + + HelpExampleRpc("listaddresses", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\"") + )); + + mapHelpStrings.insert(std::make_pair("listaddressgroupings", + "listaddressgroupings\n" + "\nLists groups of addresses which have had their common ownership\n" + "made public by common use as inputs or as the resulting change\n" + "in past transactions\n" + "\nResult:\n" + "[\n" + " [\n" + " [\n" + " \"address\", (string) The address\n" + " amount, (numeric) The amount in native currency\n" + " \"account\" (string, optional) The account\n" + " ]\n" + " ,...\n" + " ]\n" + " ,...\n" + "]\n" + "\nExamples:\n" + + HelpExampleCli("listaddressgroupings", "") + + HelpExampleRpc("listaddressgroupings", "") + )); + + mapHelpStrings.insert(std::make_pair("listaddresstransactions", + "listaddresstransactions address ( count skip verbose )\n" + "\nReturns up to 'count' most recent transactions skipping the first 'from' transactions for account 'account'.\n" + "\nArguments:\n" + "1. address (string, required) Address to list transactions for.\n" + "2. count (numeric, optional, default=10) The number of transactions to return\n" + "3. skip (numeric, optional, default=0) The number of transactions to skip\n" + "4. verbose (boolean, optional, default=false) If true, returns detailed array of inputs and outputs and raw hex of transactions\n" + "\nResult:\n" + "[\n" + " {\n" + " \"balance\": {...}, (object) Changes in address balance. \n" + " {\n" + " \"amount\": x.xxx, (numeric) The amount in native currency. Negative value means amount was send by the wallet, positive - received\n" + " \"assets\": {...}, (object) Changes in asset amounts. \n" + " }\n" + " \"myaddresses\": [...], (array) Address passed as parameter. \n" + " \"addresses\": [...], (array) Array of counterparty addresses involved in transaction \n" + " \"permissions\": [...], (array) Changes in permissions \n" + " \"issue\": {...}, (object) Issue details \n" + " \"data\" : \"metadata\", (array) Hexadecimal representation of metadata appended to the transaction\n" + " \"confirmations\": n, (numeric) The number of confirmations for the transaction. Available for 'send' and \n" + " 'receive' category of transactions.\n" + " \"blockhash\": \"hashvalue\",(string) The block hash containing the transaction. Available for 'send' and 'receive'\n" + " category of transactions.\n" + " \"blockindex\": n, (numeric) The block index containing the transaction. Available for 'send' and 'receive'\n" + " category of transactions.\n" + " \"txid\": \"transactionid\", (string) The transaction id. Available for 'send' and 'receive' category of transactions.\n" + " \"time\": xxx, (numeric) The transaction time in seconds since epoch (midnight Jan 1 1970 GMT).\n" + " \"timereceived\": xxx, (numeric) The time received in seconds since epoch (midnight Jan 1 1970 GMT). Available \n" + " for 'send' and 'receive' category of transactions.\n" + " \"comment\": \"...\", (string) If a comment is associated with the transaction.\n" + " \"vin\": [...], (array) If verbose=true. Array of input details\n" + " \"vout\": [...], (array) If verbose=true. Array of output details\n" + " \"hex\" : \"data\" (string) If verbose=true. Raw data for transaction\n" + " }\n" + "]\n" + + "\nExamples:\n" + "\nList the most recent 10 transactions in the systems\n" + + HelpExampleCli("listaddresstransactions", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\"") + + "\nList transactions 100 to 120 \n" + + HelpExampleCli("listaddresstransactions", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 20 100") + + "\nAs a json rpc call\n" + + HelpExampleRpc("listaddresstransactions", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 20, 100") + )); + + mapHelpStrings.insert(std::make_pair("listassettransactions", + "listassettransactions \"asset-identifier\" ( verbose count start local-ordering )\n" + "\nReturns stream items.\n" + "\nArguments:\n" + "1. \"asset-identifier\"(string, required) Asset identifier - one of the following: asset txid, asset reference, asset name.\n" + "2. verbose (boolean, optional, default=false) If true, returns information about transaction \n" + "3. count (number, optional, default=10) The number of transactions to display\n" + "4. start (number, optional, default=-count - last) Start from specific transaction, 0 based, if negative - from the end\n" + "5. local-ordering (boolean, optional, default=false) If true, transactions appear in the order they were processed by the wallet, if false - in the order they appear in blockchain\n" + "\nResult:\n" + "\"stream-items\" (array) List of stream items.\n" + "\nExamples:\n" + + HelpExampleCli("listassettransactions", "\"test-asset\"") + + HelpExampleCli("listassettransactions", "\"test-asset\" true 10 100") + + HelpExampleRpc("listassettransactions", "\"test-asset\", false, 20") + )); + + mapHelpStrings.insert(std::make_pair("listlockunspent", + "listlockunspent\n" + "\nReturns list of temporarily unspendable outputs.\n" + "See the lockunspent call to lock and unlock transactions for spending.\n" + "\nResult:\n" + "[\n" + " {\n" + " \"txid\" : \"transactionid\", (string) The transaction id locked\n" + " \"vout\" : n (numeric) The vout value\n" + " }\n" + " ,...\n" + "]\n" + "\nExamples:\n" + "\nList the unspent transactions\n" + + HelpExampleCli("listunspent", "") + + "\nLock an unspent transaction\n" + + HelpExampleCli("lockunspent", "false \"[{\\\"txid\\\":\\\"a08e6907dbbd3d809776dbfc5d82e371b764ed838b5655e72f463568df1aadf0\\\",\\\"vout\\\":1}]\"") + + "\nList the locked transactions\n" + + HelpExampleCli("listlockunspent", "") + + "\nUnlock the transaction again\n" + + HelpExampleCli("lockunspent", "true \"[{\\\"txid\\\":\\\"a08e6907dbbd3d809776dbfc5d82e371b764ed838b5655e72f463568df1aadf0\\\",\\\"vout\\\":1}]\"") + + "\nAs a json rpc call\n" + + HelpExampleRpc("listlockunspent", "") + )); + + mapHelpStrings.insert(std::make_pair("listreceivedbyaccount", + "listreceivedbyaccount ( minconf includeempty includeWatchonly )\n" + "\nList balances by account.\n" + "\nArguments:\n" + "1. minconf (numeric, optional, default=1) The minimum number of confirmations before payments are included.\n" + "2. includeempty (boolean, optional, default=false) Whether to include accounts that haven't received any payments.\n" + "3. includeWatchonly (bool, optional, default=false) Whether to include watchonly addresses (see 'importaddress').\n" + + "\nResult:\n" + "[\n" + " {\n" + " \"involvesWatchonly\" : \"true\", (bool) Only returned if imported addresses were involved in transaction\n" + " \"account\" : \"accountname\", (string) The account name of the receiving account\n" + " \"amount\" : x.xxx, (numeric) The total amount received by addresses with this account\n" + " \"confirmations\" : n (numeric) The number of confirmations of the most recent transaction included\n" + " }\n" + " ,...\n" + "]\n" + + "\nExamples:\n" + + HelpExampleCli("listreceivedbyaccount", "") + + HelpExampleCli("listreceivedbyaccount", "6 true") + + HelpExampleRpc("listreceivedbyaccount", "6, true, true") + )); + + mapHelpStrings.insert(std::make_pair("listreceivedbyaddress", + "listreceivedbyaddress ( minconf includeempty includeWatchonly )\n" + "\nList balances by receiving address.\n" + "\nArguments:\n" + "1. minconf (numeric, optional, default=1) The minimum number of confirmations before payments are included.\n" + "2. includeempty (numeric, optional, default=false) Whether to include addresses that haven't received any payments.\n" + "3. includeWatchonly (bool, optional, default=false) Whether to include watchonly addresses (see 'importaddress').\n" + + "\nResult:\n" + "[\n" + " {\n" + " \"involvesWatchonly\" : \"true\", (bool) Only returned if imported addresses were involved in transaction\n" + " \"address\" : \"receivingaddress\", (string) The receiving address\n" + " \"account\" : \"accountname\", (string) The account of the receiving address. The default account is \"\".\n" + " \"amount\" : x.xxx, (numeric) The total amount in native currency received by the address\n" + " \"confirmations\" : n (numeric) The number of confirmations of the most recent transaction included\n" + " }\n" + " ,...\n" + "]\n" + + "\nExamples:\n" + + HelpExampleCli("listreceivedbyaddress", "") + + HelpExampleCli("listreceivedbyaddress", "6 true") + + HelpExampleRpc("listreceivedbyaddress", "6, true, true") + )); + + mapHelpStrings.insert(std::make_pair("listsinceblock", + "listsinceblock ( blockhash target-confirmations includeWatchonly )\n" + "\nGet all transactions in blocks since block [blockhash], or all transactions if omitted\n" + "\nArguments:\n" + "1. blockhash (string, optional) The block hash to list transactions since\n" + "2. target-confirmations: (numeric, optional) The confirmations required, must be 1 or more\n" + "3. includeWatchonly: (bool, optional, default=false) Include transactions to watchonly addresses (see 'importaddress')" + "\nResult:\n" + "{\n" + " \"transactions\": [\n" + " \"account\":\"accountname\", (string) The account name associated with the transaction. Will be \"\" for the default account.\n" + " \"address\":\"address\", (string) The address of the transaction. Not present for move transactions (category = move).\n" + " \"category\":\"send|receive\", (string) The transaction category. 'send' has negative amounts, 'receive' has positive amounts.\n" + " \"amount\": x.xxx, (numeric) The amount in btc. This is negative for the 'send' category, and for the 'move' category for moves \n" + " outbound. It is positive for the 'receive' category, and for the 'move' category for inbound funds.\n" + " \"vout\" : n, (numeric) the vout value\n" + " \"fee\": x.xxx, (numeric) The amount of the fee in btc. This is negative and only available for the 'send' category of transactions.\n" + " \"confirmations\": n, (numeric) The number of confirmations for the transaction. Available for 'send' and 'receive' category of transactions.\n" + " \"blockhash\": \"hashvalue\", (string) The block hash containing the transaction. Available for 'send' and 'receive' category of transactions.\n" + " \"blockindex\": n, (numeric) The block index containing the transaction. Available for 'send' and 'receive' category of transactions.\n" + " \"blocktime\": xxx, (numeric) The block time in seconds since epoch (1 Jan 1970 GMT).\n" + " \"txid\": \"transactionid\", (string) The transaction id. Available for 'send' and 'receive' category of transactions.\n" + " \"time\": xxx, (numeric) The transaction time in seconds since epoch (Jan 1 1970 GMT).\n" + " \"timereceived\": xxx, (numeric) The time received in seconds since epoch (Jan 1 1970 GMT). Available for 'send' and 'receive' category of transactions.\n" + " \"comment\": \"...\", (string) If a comment is associated with the transaction.\n" + " \"to\": \"...\", (string) If a comment to is associated with the transaction.\n" + " ],\n" + " \"lastblock\": \"lastblockhash\" (string) The hash of the last block\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("listsinceblock", "") + + HelpExampleCli("listsinceblock", "\"000000000000000bacf66f7497b7dc45ef753ee9a7d38571037cdb1a57f663ad\" 6") + + HelpExampleRpc("listsinceblock", "\"000000000000000bacf66f7497b7dc45ef753ee9a7d38571037cdb1a57f663ad\", 6") + )); + +} + +void mc_InitRPCHelpMap12() +{ + mapHelpStrings.insert(std::make_pair("liststreamitems", + "liststreamitems \"stream-identifier\" ( verbose count start local-ordering )\n" + "\nReturns stream items.\n" + "\nArguments:\n" + "1. \"stream-identifier\"(string, required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + "2. verbose (boolean, optional, default=false) If true, returns information about item transaction \n" + "3. count (number, optional, default=10) The number of items to display\n" + "4. start (number, optional, default=-count - last) Start from specific item, 0 based, if negative - from the end\n" + "5. local-ordering (boolean, optional, default=false) If true, items appear in the order they were processed by the wallet, if false - in the order they appear in blockchain\n" + "\nResult:\n" + "\"stream-items\" (array) List of stream items.\n" + "\nExamples:\n" + + HelpExampleCli("liststreamitems", "\"test-stream\"") + + HelpExampleCli("liststreamitems", "\"test-stream\" true 10 100") + + HelpExampleRpc("liststreamitems", "\"test-stream\", false, 20") + )); + + mapHelpStrings.insert(std::make_pair("liststreamkeyitems", + "liststreamkeyitems \"stream-identifier\" \"key\" ( verbose count start local-ordering )\n" + "\nReturns stream items for specific key.\n" + "\nArguments:\n" + "1. \"stream-identifier\"(string, required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + "2. \"key\" (string, required) Stream key\n" + "3. verbose (boolean, optional, default=false) If true, returns information about item transaction \n" + "4. count (number, optional, default=10) The number of items to display\n" + "5. start (number, optional, default=-count - last) Start from specific item, 0 based, if negative - from the end\n" + "6. local-ordering (boolean, optional, default=false) If true, items appear in the order they were processed by the wallet, if false - in the order they appear in blockchain\n" + "\nResult:\n" + "\"stream-items\" (array) List of stream items for specific key.\n" + "\nExamples:\n" + + HelpExampleCli("liststreamkeyitems", "\"test-stream\" \"key01\"") + + HelpExampleCli("liststreamkeyitems", "\"test-stream\" \"key01\" true 10 100") + + HelpExampleRpc("liststreamkeyitems", "\"test-stream\", \"key01\", false 20") + )); + + mapHelpStrings.insert(std::make_pair("liststreamkeys", + "liststreamkeys \"stream-identifier\" ( key(s) verbose count start local-ordering )\n" + "\nReturns stream keys.\n" + "\nArguments:\n" + "1. \"stream-identifier\"(string, required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + "2. \"key\" (string, optional, default=*) Stream key\n" + "or\n" + "2. key(s) (array, optional) A json array of stream keys \n" + "3. verbose (boolean, optional, default=false) If true, returns extended information about key \n" + "4. count (number, optional, default=INT_MAX - all) The number of items to display\n" + "5. start (number, optional, default=-count - last) Start from specific item, 0 based, if negative - from the end\n" + "6. local-ordering (boolean, optional, default=false) If true, items appear in the order they were processed by the wallet, if false - in the order they apppear in blockchain\n" + "\nResult:\n" + "\"stream-keys\" (array) List of stream keys.\n" + "\nExamples:\n" + + HelpExampleCli("liststreamkeys", "\"test-stream\" ") + + HelpExampleCli("liststreamkeys", "\"test-stream\" \"key01\"") + + HelpExampleCli("liststreamkeys", "\"test-stream\" \"*\" true 10 100") + + HelpExampleRpc("liststreamkeys", "\"test-stream\", \"key01\"") + )); + + mapHelpStrings.insert(std::make_pair("liststreampublisheritems", + "liststreampublisheritems \"stream-identifier\" address ( verbose count start local-ordering )\n" + "\nReturns stream items for specific publisher.\n" + "\nArguments:\n" + "1. \"stream-identifier\"(string, required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + "2. address (string, required) Publisher address\n" + "3. verbose (boolean, optional, default=false) If true, returns information about item transaction \n" + "4. count (number, optional, default=10) The number of items to display\n" + "5. start (number, optional, default=-count - last) Start from specific item, 0 based, if negative - from the end\n" + "6. local-ordering (boolean, optional, default=false) If true, items appear in the order they were processed by the wallet, if false - in the order they appear in blockchain\n" + "\nResult:\n" + "\"stream-items\" (array) List of stream items for specific publisher.\n" + "\nExamples:\n" + + HelpExampleCli("liststreampublisheritems", "\"test-stream\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\"") + + HelpExampleCli("liststreampublisheritems", "\"test-stream\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" true 10 100") + + HelpExampleRpc("liststreampublisheritems", "\"test-stream\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", false, 20") + )); + + mapHelpStrings.insert(std::make_pair("liststreampublishers", + "liststreampublishers \"stream-identifier\" ( address(es) verbose count start local-ordering )\n" + "\nReturns stream publishers.\n" + "\nArguments:\n" + "1. \"stream-identifier\"(string, required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + "2. address(es) (string, optional, default=*) Publisher addresses, comma delimited\n" + "or\n" + "2. address(es) (array, optional) A json array of publisher addresses \n" + "3. verbose (boolean, optional, default=false) If true, returns extended information about publisher \n" + "4. count (number, optional, default=INT_MAX - all) The number of items to display\n" + "5. start (number, optional, default=-count - last) Start from specific item, 0 based, if negative - from the end\n" + "6. local-ordering (boolean, optional, default=false) If true, items appear in the order they were processed by the wallet, if false - in the order they apppear in blockchain\n" + "\nResult:\n" + "\"stream-publishers\" (array) List of stream publishers.\n" + "\nExamples:\n" + + HelpExampleCli("liststreampublishers", "\"test-stream\" ") + + HelpExampleCli("liststreampublishers", "\"test-stream\" 1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd") + + HelpExampleCli("liststreampublishers", "\"test-stream\" \"*\" true 10 100") + + HelpExampleRpc("liststreampublishers", "\"test-stream\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\"") + )); + + mapHelpStrings.insert(std::make_pair("listtransactions", + "listtransactions ( \"account\" count from includeWatchonly )\n" + "\nReturns up to 'count' most recent transactions skipping the first 'from' transactions for account 'account'.\n" + "\nArguments:\n" + "1. \"account\" (string, optional) The account name. If not included, it will list all transactions for all accounts.\n" + " If \"\" is set, it will list transactions for the default account.\n" + "2. count (numeric, optional, default=10) The number of transactions to return\n" + "3. from (numeric, optional, default=0) The number of transactions to skip\n" + "4. includeWatchonly (bool, optional, default=false) Include transactions to watchonly addresses (see 'importaddress')\n" + "\nResult:\n" + "[\n" + " {\n" + " \"account\":\"accountname\", (string) The account name associated with the transaction. \n" + " It will be \"\" for the default account.\n" + " \"address\":\"address\", (string) The address of the transaction. Not present for \n" + " move transactions (category = move).\n" + " \"category\":\"send|receive|move\", (string) The transaction category. 'move' is a local (off blockchain)\n" + " transaction between accounts, and not associated with an address,\n" + " transaction id or block. 'send' and 'receive' transactions are \n" + " associated with an address, transaction id and block details\n" + " \"amount\": x.xxx, (numeric) The amount in btc. This is negative for the 'send' category, and for the\n" + " 'move' category for moves outbound. It is positive for the 'receive' category,\n" + " and for the 'move' category for inbound funds.\n" + " \"vout\" : n, (numeric) the vout value\n" + " \"fee\": x.xxx, (numeric) The amount of the fee in btc. This is negative and only available for the \n" + " 'send' category of transactions.\n" + " \"confirmations\": n, (numeric) The number of confirmations for the transaction. Available for 'send' and \n" + " 'receive' category of transactions.\n" + " \"blockhash\": \"hashvalue\", (string) The block hash containing the transaction. Available for 'send' and 'receive'\n" + " category of transactions.\n" + " \"blockindex\": n, (numeric) The block index containing the transaction. Available for 'send' and 'receive'\n" + " category of transactions.\n" + " \"txid\": \"transactionid\", (string) The transaction id. Available for 'send' and 'receive' category of transactions.\n" + " \"time\": xxx, (numeric) The transaction time in seconds since epoch (midnight Jan 1 1970 GMT).\n" + " \"timereceived\": xxx, (numeric) The time received in seconds since epoch (midnight Jan 1 1970 GMT). Available \n" + " for 'send' and 'receive' category of transactions.\n" + " \"comment\": \"...\", (string) If a comment is associated with the transaction.\n" + " \"otheraccount\": \"accountname\", (string) For the 'move' category of transactions, the account the funds came \n" + " from (for receiving funds, positive amounts), or went to (for sending funds,\n" + " negative amounts).\n" + " }\n" + "]\n" + + "\nExamples:\n" + "\nList the most recent 10 transactions in the systems\n" + + HelpExampleCli("listtransactions", "") + + "\nList the most recent 10 transactions for the tabby account\n" + + HelpExampleCli("listtransactions", "\"tabby\"") + + "\nList transactions 100 to 120 from the tabby account\n" + + HelpExampleCli("listtransactions", "\"tabby\" 20 100") + + "\nAs a json rpc call\n" + + HelpExampleRpc("listtransactions", "\"tabby\", 20, 100") + )); + + mapHelpStrings.insert(std::make_pair("listunspent", + "listunspent ( minconf maxconf [\"address\",...] )\n" + "\nReturns array of unspent transaction outputs\n" + "with between minconf and maxconf (inclusive) confirmations.\n" + "Optionally filter to only include txouts paid to specified addresses.\n" + "Results are an array of Objects, each of which has:\n" + "{txid, vout, scriptPubKey, amount, confirmations}\n" + "\nArguments:\n" + "1. minconf (numeric, optional, default=1) The minimum confirmations to filter\n" + "2. maxconf (numeric, optional, default=9999999) The maximum confirmations to filter\n" + "3. \"addresses\" (string) A json array of addresses to filter\n" + " [\n" + " \"address\" (string) address\n" + " ,...\n" + " ]\n" + "\nResult\n" + "[ (array of json object)\n" + " {\n" + " \"txid\" : \"txid\", (string) the transaction id \n" + " \"vout\" : n, (numeric) the vout value\n" + " \"address\" : \"address\", (string) the address\n" + " \"account\" : \"account\", (string) The associated account, or \"\" for the default account\n" + " \"scriptPubKey\" : \"key\", (string) the script key\n" + " \"amount\" : x.xxx, (numeric) the transaction amount in btc\n" + " \"confirmations\" : n (numeric) The number of confirmations\n" + " }\n" + " ,...\n" + "]\n" + + "\nExamples\n" + + HelpExampleCli("listunspent", "") + + HelpExampleCli("listunspent", "6 9999999 \"[\\\"1PGFqEzfmQch1gKD3ra4k18PNj3tTUUSqg\\\",\\\"1LtvqCaApEdUGFkpKMM4MstjcaL4dKg8SP\\\"]\"") + + HelpExampleRpc("listunspent", "6, 9999999 \"[\\\"1PGFqEzfmQch1gKD3ra4k18PNj3tTUUSqg\\\",\\\"1LtvqCaApEdUGFkpKMM4MstjcaL4dKg8SP\\\"]\"") + )); + + mapHelpStrings.insert(std::make_pair("listwallettransactions", + "listwallettransactions ( count skip includeWatchonly verbose )\n" + "\nReturns up to 'count' most recent transactions skipping the first 'from' transactions for account 'account'.\n" + "\nArguments:\n" + "1. count (numeric, optional, default=10) The number of transactions to return\n" + "2. skip (numeric, optional, default=0) The number of transactions to skip\n" + "3. includeWatchonly (bool, optional, default=false) Include transactions to watchonly addresses (see 'importaddress')\n" + "4. verbose (bool, optional, default=false) If true, returns detailed array of inputs and outputs and raw hex of transactions\n" + "\nResult:\n" + "[\n" + " {\n" + " \"balance\": {...}, (object) Changes in wallet balance. \n" + " {\n" + " \"amount\": x.xxx, (numeric) The amount in native currency. Negative value means amount was send by the wallet, positive - received\n" + " \"assets\": {...}, (object) Changes in asset amounts. \n" + " }\n" + " \"myaddresses\": [...], (array) Array of wallet addresses involved in transaction \n" + " \"addresses\": [...], (array) Array of counterparty addresses involved in transaction \n" + " \"permissions\": [...], (array) Changes in permissions \n" + " \"issue\": {...}, (object) Issue details \n" + " \"data\" : \"metadata\", (array) Hexadecimal representation of metadata appended to the transaction\n" + " \"confirmations\": n, (numeric) The number of confirmations for the transaction. Available for 'send' and \n" + " 'receive' category of transactions.\n" + " \"blockhash\": \"hashvalue\",(string) The block hash containing the transaction. Available for 'send' and 'receive'\n" + " category of transactions.\n" + " \"blockindex\": n, (numeric) The block index containing the transaction. Available for 'send' and 'receive'\n" + " category of transactions.\n" + " \"txid\": \"transactionid\", (string) The transaction id. Available for 'send' and 'receive' category of transactions.\n" + " \"time\": xxx, (numeric) The transaction time in seconds since epoch (midnight Jan 1 1970 GMT).\n" + " \"timereceived\": xxx, (numeric) The time received in seconds since epoch (midnight Jan 1 1970 GMT). Available \n" + " for 'send' and 'receive' category of transactions.\n" + " \"comment\": \"...\", (string) If a comment is associated with the transaction.\n" + " \"vin\": [...], (array) If verbose=true. Array of input details\n" + " \"vout\": [...], (array) If verbose=true. Array of output details\n" + " \"hex\" : \"data\" (string) If verbose=true. Raw data for transaction\n" + " }\n" + "]\n" + + "\nExamples:\n" + "\nList the most recent 10 transactions in the systems\n" + + HelpExampleCli("listwallettransactions", "") + + "\nList transactions 100 to 120 \n" + + HelpExampleCli("listwallettransactions", "20 100") + + "\nAs a json rpc call\n" + + HelpExampleRpc("listwallettransactions", "20, 100") + )); + + mapHelpStrings.insert(std::make_pair("lockunspent", + "lockunspent unlock [{\"txid\":\"txid\",\"vout\":n},...]\n" + "\nUpdates list of temporarily unspendable outputs.\n" + "Temporarily lock (unlock=false) or unlock (unlock=true) specified transaction outputs.\n" + "A locked transaction output will not be chosen by automatic coin selection, when spending assetss.\n" + "Locks are stored in memory only. Nodes start with zero locked outputs, and the locked output list\n" + "is always cleared (by virtue of process exit) when a node stops or fails.\n" + "Also see the listunspent call\n" + "\nArguments:\n" + "1. unlock (boolean, required) Whether to unlock (true) or lock (false) the specified transactions\n" + "2. \"transactions\" (string, required) A json array of objects. Each object the txid (string) vout (numeric)\n" + " [ (json array of json objects)\n" + " {\n" + " \"txid\":\"id\", (string) The transaction id\n" + " \"vout\": n (numeric) The output number\n" + " }\n" + " ,...\n" + " ]\n" + + "\nResult:\n" + "true|false (boolean) Whether the command was successful or not\n" + + "\nExamples:\n" + "\nList the unspent transactions\n" + + HelpExampleCli("listunspent", "") + + "\nLock an unspent transaction\n" + + HelpExampleCli("lockunspent", "false \"[{\\\"txid\\\":\\\"a08e6907dbbd3d809776dbfc5d82e371b764ed838b5655e72f463568df1aadf0\\\",\\\"vout\\\":1}]\"") + + "\nList the locked transactions\n" + + HelpExampleCli("listlockunspent", "") + + "\nUnlock the transaction again\n" + + HelpExampleCli("lockunspent", "true \"[{\\\"txid\\\":\\\"a08e6907dbbd3d809776dbfc5d82e371b764ed838b5655e72f463568df1aadf0\\\",\\\"vout\\\":1}]\"") + + "\nAs a json rpc call\n" + + HelpExampleRpc("lockunspent", "false, \"[{\\\"txid\\\":\\\"a08e6907dbbd3d809776dbfc5d82e371b764ed838b5655e72f463568df1aadf0\\\",\\\"vout\\\":1}]\"") + )); + + mapHelpStrings.insert(std::make_pair("move", + "move \"fromaccount\" \"toaccount\" amount ( minconf \"comment\" )\n" + "\nMove a specified amount from one account in your wallet to another.\n" + "\nArguments:\n" + "1. \"fromaccount\" (string, required) The name of the account to move funds from. May be the default account using \"\".\n" + "2. \"toaccount\" (string, required) The name of the account to move funds to. May be the default account using \"\".\n" + "3. minconf (numeric, optional, default=1) Only use funds with at least this many confirmations.\n" + "4. \"comment\" (string, optional) An optional comment, stored in the wallet only.\n" + "\nResult:\n" + "true|false (boolean) true if successfull.\n" + "\nExamples:\n" + "\nMove 0.01 btc from the default account to the account named tabby\n" + + HelpExampleCli("move", "\"\" \"tabby\" 0.01") + + "\nMove 0.01 btc timotei to akiko with a comment and funds have 6 confirmations\n" + + HelpExampleCli("move", "\"timotei\" \"akiko\" 0.01 6 \"happy birthday!\"") + + "\nAs a json rpc call\n" + + HelpExampleRpc("move", "\"timotei\", \"akiko\", 0.01, 6, \"happy birthday!\"") + )); + +} + +void mc_InitRPCHelpMap13() +{ + mapHelpStrings.insert(std::make_pair("preparelockunspent", + "preparelockunspent asset-quantities ( lock )\n" + "\nPrepares exchange transaction output for createrawexchange, appendrawexchange\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. asset-quantities (object, required) A json object of assets to send\n" + " [\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " ]\n" + "2. lock (boolean, optiona, default=true) Lock prepared unspent output\n" + "\nResult:\n" + "{\n" + " \"txid\": \"transactionid\", (string) Transaction ID of the output which can be spent in createrawexchange or createrawexchange\n" + " \"vout\": n (numeric) Output index\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("preparelockunspent", "\"{\\\"12345-6789-1234\\\":100}\"") + + HelpExampleCli("preparelockunspent", "\"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\"") + + HelpExampleRpc("preparelockunspent", "\"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\"") + )); + + mapHelpStrings.insert(std::make_pair("preparelockunspentfrom", + "preparelockunspentfrom from-address asset-quantities ( lock )\n" + "\nPrepares exchange transaction output for createrawexchange, appendrawexchange using specific address\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address to send from .\n" + "2. asset-quantities (object, required) A json object of assets to send\n" + " [\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " ]\n" + "3. lock (boolean, optiona, default=true) Lock prepared unspent output\n" + "\nResult:\n" + "{\n" + " \"txid\": \"transactionid\", (string) Transaction ID of the output which can be spent in createrawexchange or createrawexchange\n" + " \"vout\": n (numeric) Output index\n" + "}\n" + "\nExamples:\n" + + HelpExampleCli("preparelockunspentfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"12345-6789-1234\\\":100}\"") + + HelpExampleCli("preparelockunspentfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\"") + + HelpExampleRpc("preparelockunspentfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\"") + )); + + mapHelpStrings.insert(std::make_pair("publish", + "publish \"stream-identifier\" \"key\" data-hex\n" + "\nPublishes stream item\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. \"stream-identifier\" (string, required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + "2. \"key\" (string, required) Item key\n" + "3. data-hex (string, required) Item data hex string\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("publish", "test \"hello world\" 48656C6C6F20576F726C64210A") + + HelpExampleRpc("publish", "\"test\", \"hello world\", \"48656C6C6F20576F726C64210A\"") + )); + + mapHelpStrings.insert(std::make_pair("publishfrom", + "publishfrom from-address \"stream-identifier\" \"key\" data-hex\n" + "\nPublishes stream item\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address used for issuing.\n" + "2. \"stream-identifier\" (string, required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + "3. \"key\" (string, required) Item key\n" + "4. data-hex (string, required) Item data hex string\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("publishfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" test \"hello world\" 48656C6C6F20576F726C64210A") + + HelpExampleRpc("publishfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"test\", \"hello world\", \"48656C6C6F20576F726C64210A\"") + )); + + mapHelpStrings.insert(std::make_pair("resendwallettransactions", + "resendwallettransactions\n" + "\nStop Resends wallet transactions." + )); + + mapHelpStrings.insert(std::make_pair("revoke", + "revoke \"address(es)\" \"permission(s)\" ( native-amount \"comment\" \"comment-to\" )\n" + "\nRevoke permission from a given address. The amount is a real\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. \"address(es)\" (string, required) The addresses(es) to revoke permissions from\n" + "2. \"permission(s)\" (string, required) Permission strings, comma delimited. Possible values: connect,send,receive,issue,mine,admin \n" + "3. native-amount (numeric, optional) native currency amount to send. eg 0.1. Default - 0\n" + "4. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "5. \"comment-to\" (string, optional) A comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the \n" + " transaction, just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("revoke", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 connect,send,receive") + + HelpExampleCli("revoke", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 mine \"permission to mine\" \"Rogue Miner\"") + + HelpExampleRpc("revoke", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, admin \"disabling temporary admin\", \"Admins Ltd.\"") + )); + + mapHelpStrings.insert(std::make_pair("revokefrom", + "revokefrom from-address \"to-address(es)\" \"permission(s)\" ( native-amount \"comment\" \"comment-to\" )\n" + "\nRevoke permissions using specific address.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Addresses used for revoke.\n" + "2. \"to-address(es)\" (string, required) The addresses(es) to revoke permissions from. Comma delimited\n" + "3. \"permission(s)\" (string, required) Permission strings, comma delimited. Possible values: connect,send,receive,issue,mine,admin \n" + "4. native-amount (numeric, optional) native currency amount to send. eg 0.1. Default - 0\n" + "5. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "6. \"comment-to\" (string, optional) A comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the \n" + " transaction, just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("revokefrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 connect,send,receive") + + HelpExampleCli("revokefrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 mine \"permission to mine\" \"Rogue Miner\"") + + HelpExampleRpc("revokefrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, admin \"disabling temporary admin\", \"Admins Ltd.\"") + )); + + mapHelpStrings.insert(std::make_pair("send", + "send address amount|asset-quantities ( \"comment\" \"comment-to\" )\n" + "\nSend an amount (or several asset amounts) to a given address. The amount is a real and is rounded to the nearest 0.00000001\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. address (string, required) The address to send to.\n" + "2. amount (numeric, required) The amount in native currency to send. eg 0.1\n" + "or\n" + "2. \"asset-quantities\" (object, required) A json object of assets to send\n" + " {\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " }\n" + "3. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "4. \"comment-to\" (string, optional) A comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the \n" + " transaction, just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("send", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\"") + + HelpExampleCli("send", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 \"donation\" \"seans outpost\"") + + HelpExampleRpc("send", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, \"donation\", \"seans outpost\"") + )); + + mapHelpStrings.insert(std::make_pair("sendtoaddress", + "sendtoaddress address amount|asset-quantities ( \"comment\" \"comment-to\" )\n" + "\nSend an amount (or several asset amounts) to a given address. The amount is a real and is rounded to the nearest 0.00000001\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. address (string, required) The address to send to.\n" + "2. amount (numeric, required) The amount in native currency to send. eg 0.1\n" + "or\n" + "2. \"asset-quantities\" (object, required) A json object of assets to send\n" + " {\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " }\n" + "3. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "4. \"comment-to\" (string, optional) A comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the \n" + " transaction, just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("sendtoaddress", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\"") + + HelpExampleCli("sendtoaddress", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 \"donation\" \"seans outpost\"") + + HelpExampleRpc("sendtoaddress", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, \"donation\", \"seans outpost\"") + )); + + mapHelpStrings.insert(std::make_pair("sendasset", + "sendasset address \"asset-identifier\" asset-qty ( native-amount \"comment\" \"comment-to\" )\n" + "\nSend asset amount to a given address. The amounts are real.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. address (string, required) The address to send to.\n" + "2. \"asset-identifier\" (string, required) Asset identifier - one of the following: issue txid, asset reference, asset name.\n" + "3. asset-qty (numeric, required) Asset quantity to send. eg 0.1\n" + "4. native-amount (numeric, optional) native currency amount to send. eg 0.1, Default: minimum-per-output.\n" + "5. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "6. \"comment-to\" (string, optional) A comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the \n" + " transaction, just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("sendasset", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 12345-6789-1234 100") + + HelpExampleCli("sendasset", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 12345-6789-1234 100 0.1 \"donation\" \"seans outpost\"") + + HelpExampleRpc("sendasset", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 12345-6789-1234, 100, 0.1, \"donation\", \"seans outpost\"") + )); + +} + +void mc_InitRPCHelpMap14() +{ + mapHelpStrings.insert(std::make_pair("sendassettoaddress", + "sendassettoaddress address \"asset-identifier\" asset-qty ( native-amount \"comment\" \"comment-to\" )\n" + "\nSend asset amount to a given address. The amounts are real.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. address (string, required) The address to send to.\n" + "2. \"asset-identifier\" (string, required) Asset identifier - one of the following: issue txid, asset reference, asset name.\n" + "3. asset-qty (numeric, required) Asset quantity to send. eg 0.1\n" + "4. native-amount (numeric, optional) native currency amount to send. eg 0.1, Default: minimum-per-output.\n" + "5. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "6. \"comment-to\" (string, optional) A comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the \n" + " transaction, just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("sendassettoaddress", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 12345-6789-1234 100") + + HelpExampleCli("sendassettoaddress", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 12345-6789-1234 100 0.1 \"donation\" \"seans outpost\"") + + HelpExampleRpc("sendassettoaddress", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 12345-6789-1234, 100, 0.1, \"donation\", \"seans outpost\"") + )); + + mapHelpStrings.insert(std::make_pair("sendassetfrom", + "sendassetfrom from-address to-address \"asset-identifier\" asset-qty ( native-amount \"comment\" \"comment-to\" )\n" + "\nSend an asset amount using specific address. \n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address to send from. \n" + "2. to-address (string, required) The address to send to.\n" + "3. \"asset-identifier\" (string, required) Asset identifier - one of the following: issue txid, asset reference, asset name.\n" + "4. asset-qty (numeric, required) Asset quantity to send. eg 0.1\n" + "5. native-amount (numeric, optional) native currency amount to send. eg 0.1, Default: minimum-per-output.\n" + "6. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "7. \"comment-to\" (string, optional) A comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the \n" + " transaction, just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("sendassetfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 12345-6789-1234 100") + + HelpExampleCli("sendassetfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 12345-6789-1234 100 \"donation\" \"seans outpost\"") + + HelpExampleRpc("sendassetfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, 12345-6789-1234, 100, \"donation\", \"seans outpost\"") + )); + + mapHelpStrings.insert(std::make_pair("sendfrom", + "sendfrom from-address to-address amount|asset-quantities ( \"comment\" \"comment-to\" )\n" + "\nSend an amount (or several asset amounts) using specific address.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address to send from.\n" + "2. to-address (string, required) The address to send to.\n" + "3. amount (numeric, required) The amount in native currency to send. eg 0.1\n" + "or\n" + "3. asset-quantities (object, required) A json object of assets to send\n" + " {\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " }\n" + "4. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "5. \"comment-to\" (string, optional) A comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the \n" + " transaction, just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("sendfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\"") + + HelpExampleCli("sendfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 \"donation\" \"seans outpost\"") + + HelpExampleRpc("sendfrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, \"donation\", \"seans outpost\"") + )); + + mapHelpStrings.insert(std::make_pair("sendfromaddress", + "sendfrom from-address to-address amount|asset-quantities ( \"comment\" \"comment-to\" )\n" + "\nSend an amount (or several asset amounts) using specific address.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address to send from.\n" + "2. to-address (string, required) The address to send to.\n" + "3. amount (numeric, required) The amount in native currency to send. eg 0.1\n" + "or\n" + "3. asset-quantities (object, required) A json object of assets to send\n" + " {\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " }\n" + "4. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "5. \"comment-to\" (string, optional) A comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the \n" + " transaction, just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("sendfromaddress", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\"") + + HelpExampleCli("sendfromaddress", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 \"donation\" \"seans outpost\"") + + HelpExampleRpc("sendfromaddress", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, \"donation\", \"seans outpost\"") + )); + + mapHelpStrings.insert(std::make_pair("sendfromaccount", + "sendfromaccount \"fromaccount\" toaddress amount ( minconf \"comment\" \"comment-to\" )\n" + "\nSent an amount from an account to a address.\n" + "The amount is a real and is rounded to the nearest 0.00000001." + + HelpRequiringPassphraseWrapper() + "\n" + "\nArguments:\n" + "1. \"fromaccount\" (string, required) The name of the account to send funds from. May be the default account using \"\".\n" + "2. toaddress (string, required) The address to send funds to.\n" + "3. amount (numeric, required) The amount in native currency. (transaction fee is added on top).\n" + "4. minconf (numeric, optional, default=1) Only use funds with at least this many confirmations.\n" + "5. \"comment\" (string, optional) A comment used to store what the transaction is for. \n" + " This is not part of the transaction, just kept in your wallet.\n" + "6. \"comment-to\" (string, optional) An optional comment to store the name of the person or organization \n" + " to which you're sending the transaction. This is not part of the transaction, \n" + " it is just kept in your wallet.\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + "\nSend 0.01 btc from the default account to the address, must have at least 1 confirmation\n" + + HelpExampleCli("sendfromaccount", "\"\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.01") + + "\nSend 0.01 from the tabby account to the given address, funds must have at least 6 confirmations\n" + + HelpExampleCli("sendfromaccount", "\"tabby\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.01 6 \"donation\" \"seans outpost\"") + + "\nAs a json rpc call\n" + + HelpExampleRpc("sendfromaccount", "\"tabby\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.01, 6, \"donation\", \"seans outpost\"") + )); + + mapHelpStrings.insert(std::make_pair("sendmany", + "sendmany \"fromaccount\" {\"address\":amount,...} ( minconf \"comment\" )\n" + "\nSend multiple times. Amounts are double-precision floating point numbers." + + HelpRequiringPassphraseWrapper() + "\n" + "\nArguments:\n" + "1. \"fromaccount\" (string, required) The account to send the funds from, can be \"\" for the default account\n" + "2. \"amounts\" (string, required) A json object with addresses and amounts\n" + " {\n" + " \"address\":amount (numeric) The address is the key, the numeric amount in btc is the value\n" + " ,...\n" + " }\n" + "3. minconf (numeric, optional, default=1) Only use the balance confirmed at least this many times.\n" + "4. \"comment\" (string, optional) A comment\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id for the send. Only 1 transaction is created regardless of \n" + " the number of addresses.\n" + "\nExamples:\n" + "\nSend two amounts to two different addresses:\n" + + HelpExampleCli("sendmany", "\"tabby\" \"{\\\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\\\":0.01,\\\"1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz\\\":0.02}\"") + + "\nSend two amounts to two different addresses setting the confirmation and comment:\n" + + HelpExampleCli("sendmany", "\"tabby\" \"{\\\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\\\":0.01,\\\"1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz\\\":0.02}\" 6 \"testing\"") + + "\nAs a json rpc call\n" + + HelpExampleRpc("sendmany", "\"tabby\", \"{\\\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\\\":0.01,\\\"1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz\\\":0.02}\", 6, \"testing\"") + )); + + mapHelpStrings.insert(std::make_pair("sendwithdata", + "sendwithdata address amount|asset-quantities data-hex|object\n" + "\nSend an amount (or several asset amounts) to a given address with appended metadata. \n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. address (string, required) The address to send to.\n" + "2. amount (numeric, required) The amount in native currency to send. eg 0.1\n" + "or\n" + "2. asset-quantities (object, required) A json object of assets to send\n" + " [\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " ]\n" + "3. data-hex (string, required) Data hex string\n" + "or\n" + "3. publish-new-stream-item (object, required) A json object with stream item\n" + " {\n" + " \"for\" : stream-identifier (string,required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + " \"key\" : key (string,optional, default: \"\") Item key\n" + " \"data\" : data-hex (string,optional, default: \"\") Data hex string\n" + " }\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("sendwithdata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\" 48656C6C6F20576F726C64210A") + + HelpExampleCli("sendwithdata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 48656C6C6F20576F726C64210A") + + HelpExampleRpc("sendwithdata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, 48656C6C6F20576F726C64210A") + )); + + mapHelpStrings.insert(std::make_pair("sendwithmetadata", + "sendwithmetadata address amount|asset-quantities data-hex|object\n" + "\nSend an amount (or several asset amounts) to a given address with appended metadata. \n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. address (string, required) The address to send to.\n" + "2. amount (numeric, required) The amount in native currency to send. eg 0.1\n" + "or\n" + "2. asset-quantities (object, required) A json object of assets to send\n" + " [\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " ]\n" + "3. data-hex (string, required) Data hex string\n" + "or\n" + "3. publish-new-stream-item (object, required) A json object with stream item\n" + " {\n" + " \"for\" : stream-identifier (string,required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + " \"key\" : key (string,optional, default: \"\") Item key\n" + " \"data\" : data-hex (string,optional, default: \"\") Data hex string\n" + " }\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("sendwithmetadata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\" 48656C6C6F20576F726C64210A") + + HelpExampleCli("sendwithmetadata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 48656C6C6F20576F726C64210A") + + HelpExampleRpc("sendwithmetadata", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, 48656C6C6F20576F726C64210A") + )); + + mapHelpStrings.insert(std::make_pair("sendwithdatafrom", + "sendwithdatafrom from-address to-address amount|asset-quantities data-hex|object\n" + "\nSend an amount (or several asset amounts) using specific address.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address to send from.\n" + "2. to-address (string, required) The address to send to.\n" + "3. amount (numeric, required) The amount in native currency to send. eg 0.1\n" + "or\n" + "3. asset-quantities (object, required) A json object of assets to send\n" + " [\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " ]\n" + "4. data-hex (string, required) Data hex string\n" + "or\n" + "4. publish-new-stream-item (object, required) A json object with stream item\n" + " {\n" + " \"for\" : stream-identifier (string,required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + " \"key\" : key (string,optional, default: \"\") Item key\n" + " \"data\" : data-hex (string,optional, default: \"\") Data hex string\n" + " }\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("sendwithdatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\" 48656C6C6F20576F726C64210A") + + HelpExampleCli("sendwithdatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 48656C6C6F20576F726C64210A") + + HelpExampleRpc("sendwithdatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, 48656C6C6F20576F726C64210A") + )); + + mapHelpStrings.insert(std::make_pair("sendwithmetadatafrom", + "sendwithmetadatafrom from-address to-address amount|asset-quantities data-hex|object\n" + "\nSend an amount (or several asset amounts) using specific address.\n" + + HelpRequiringPassphraseWrapper() + + "\nArguments:\n" + "1. from-address (string, required) Address to send from.\n" + "2. to-address (string, required) The address to send to.\n" + "3. amount (numeric, required) The amount in native currency to send. eg 0.1\n" + "or\n" + "3. asset-quantities (object, required) A json object of assets to send\n" + " [\n" + " \"asset-identifier\" : asset-quantity\n" + " ,...\n" + " ]\n" + "4. data-hex (string, required) Data hex string\n" + "or\n" + "4. publish-new-stream-item (object, required) A json object with stream item\n" + " {\n" + " \"for\" : stream-identifier (string,required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + " \"key\" : key (string,optional, default: \"\") Item key\n" + " \"data\" : data-hex (string,optional, default: \"\") Data hex string\n" + " }\n" + "\nResult:\n" + "\"transactionid\" (string) The transaction id.\n" + "\nExamples:\n" + + HelpExampleCli("sendwithmetadatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"{\\\"12345-6789-1234\\\":100,\\\"1234-5678-1234\\\":200}\" 48656C6C6F20576F726C64210A") + + HelpExampleCli("sendwithmetadatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 0.1 48656C6C6F20576F726C64210A") + + HelpExampleRpc("sendwithmetadatafrom", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", \"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\", 0.1, 48656C6C6F20576F726C64210A") + )); + +} + +void mc_InitRPCHelpMap15() +{ + mapHelpStrings.insert(std::make_pair("setaccount", + "setaccount address \"account\"\n" + "\nSets the account associated with the given address.\n" + "\nArguments:\n" + "1. address (string, required) The address to be associated with an account.\n" + "2. \"account\" (string, required) The account to assign the address to.\n" + "\nExamples:\n" + + HelpExampleCli("setaccount", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\" \"tabby\"") + + HelpExampleRpc("setaccount", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\", \"tabby\"") + )); + + mapHelpStrings.insert(std::make_pair("settxfee", + "settxfee amount\n" + "\nSet the transaction fee per kB.\n" + "\nArguments:\n" + "1. amount (numeric, required) The transaction fee in /kB rounded to the nearest 0.00000001\n" + "\nResult\n" + "true|false (boolean) Returns true if successful\n" + "\nExamples:\n" + + HelpExampleCli("settxfee", "0.00001") + + HelpExampleRpc("settxfee", "0.00001") + )); + + mapHelpStrings.insert(std::make_pair("signmessage", + "signmessage address \"message\"\n" + "\nSign a message with the private key of an address" + + HelpRequiringPassphraseWrapper() + "\n" + "\nArguments:\n" + "1. address (string, required) The address to use for the private key.\n" + "2. \"message\" (string, required) The message to create a signature of.\n" + "\nResult:\n" + "\"signature\" (string) The signature of the message encoded in base 64\n" + "\nExamples:\n" + "\nUnlock the wallet for 30 seconds\n" + + HelpExampleCli("walletpassphrase", "\"mypassphrase\" 30") + + "\nCreate the signature\n" + + HelpExampleCli("signmessage", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\" \"my message\"") + + "\nVerify the signature\n" + + HelpExampleCli("verifymessage", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\" \"signature\" \"my message\"") + + "\nAs json rpc\n" + + HelpExampleRpc("signmessage", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\", \"my message\"") + )); + + mapHelpStrings.insert(std::make_pair("subscribe", + "subscribe entity-identifier(s) ( rescan )\n" + "\nSubscribes to the stream.\n" + "\nArguments:\n" + "1. \"stream-identifier\" (string, required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + "or\n" + "1. \"asset-identifier\" (string, required) Asset identifier - one of the following: asset txid, asset reference, asset name.\n" + "or\n" + "1. entity-identifier(s) (array, optional) A json array of stream or asset identifiers \n" + "2. rescan (boolean, optional, default=true) Rescan the wallet for transactions\n" + "\nNote: This call can take minutes to complete if rescan is true.\n" + "\nExamples:\n" + "\nSubscribe to the stream with rescan\n" + + HelpExampleCli("subscribe", "\"test-stream\"") + + "\nSubscribe to the stream without rescan\n" + + HelpExampleCli("subscribe", "\"test-stream\" false") + + "\nAs a JSON-RPC call\n" + + HelpExampleRpc("subscribe", "\"test-stream\", false") + )); + + mapHelpStrings.insert(std::make_pair("unsubscribe", + "unsubscribe entity-identifier(s)\n" + "\nUnsubscribes from the stream.\n" + "\nArguments:\n" + "1. \"stream-identifier\" (string, required) Stream identifier - one of the following: stream txid, stream reference, stream name.\n" + "or\n" + "1. \"asset-identifier\" (string, required) Asset identifier - one of the following: asset txid, asset reference, asset name.\n" + "or\n" + "1. entity-identifier(s) (array, optional) A json array of stream or asset identifiers \n" + "\nExamples:\n" + + HelpExampleCli("unsubscribe", "\"test-stream\"") + + HelpExampleRpc("unsubscribe", "\"test-stream\", false") + )); + + mapHelpStrings.insert(std::make_pair("invalidateblock", + "invalidateblock \"hash\"\n" + "\nPermanently marks a block as invalid, as if it violated a consensus rule.\n" + "\nArguments:\n" + "1. hash (string, required) the hash of the block to mark as invalid\n" + "\nResult:\n" + "\nExamples:\n" + + HelpExampleCli("invalidateblock", "\"blockhash\"") + + HelpExampleRpc("invalidateblock", "\"blockhash\"") + )); + + mapHelpStrings.insert(std::make_pair("reconsiderblock", + "reconsiderblock \"hash\"\n" + "\nRemoves invalidity status of a block and its descendants, reconsider them for activation.\n" + "This can be used to undo the effects of invalidateblock.\n" + "\nArguments:\n" + "1. hash (string, required) the hash of the block to reconsider\n" + "\nResult:\n" + "\nExamples:\n" + + HelpExampleCli("reconsiderblock", "\"blockhash\"") + + HelpExampleRpc("reconsiderblock", "\"blockhash\"") + )); + + mapHelpStrings.insert(std::make_pair("setmocktime", + "setmocktime timestamp\n" + "\nSet the local time to given timestamp (-regtest only)\n" + "\nArguments:\n" + "1. timestamp (integer, required) Unix seconds-since-epoch timestamp\n" + " Pass 0 to go back to using the system time." + )); + + mapHelpStrings.insert(std::make_pair("AAAAAAA", + "" + )); + + +} + + +void mc_InitRPCHelpMap() +{ + mc_InitRPCHelpMap01(); + mc_InitRPCHelpMap02(); + mc_InitRPCHelpMap03(); + mc_InitRPCHelpMap04(); + mc_InitRPCHelpMap05(); + mc_InitRPCHelpMap06(); + mc_InitRPCHelpMap07(); + mc_InitRPCHelpMap08(); + mc_InitRPCHelpMap09(); + mc_InitRPCHelpMap10(); + mc_InitRPCHelpMap11(); + mc_InitRPCHelpMap12(); + mc_InitRPCHelpMap13(); + mc_InitRPCHelpMap14(); + mc_InitRPCHelpMap15(); +} + diff --git a/src/rpc/rpcmining.cpp b/src/rpc/rpcmining.cpp new file mode 100644 index 00000000..4f729e99 --- /dev/null +++ b/src/rpc/rpcmining.cpp @@ -0,0 +1,549 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "structs/amount.h" +#include "chainparams/chainparams.h" +#include "utils/core_io.h" +#include "core/init.h" +#include "net/net.h" +#include "core/main.h" +#include "miner/miner.h" +#include "chain/pow.h" +#include "rpcserver.h" +#include "utils/util.h" +#ifdef ENABLE_WALLET +#include "wallet/db.h" +#include "wallet/wallet.h" +#endif + +#include + +#include + +#include "json/json_spirit_utils.h" +#include "json/json_spirit_value.h" + +using namespace json_spirit; +using namespace std; + +/** + * Return average network hashes per second based on the last 'lookup' blocks, + * or from the last difficulty change if 'lookup' is nonpositive. + * If 'height' is nonnegative, compute the estimate at the time when a given block was found. + */ +Value GetNetworkHashPS(int lookup, int height) { + CBlockIndex *pb = chainActive.Tip(); + + if (height >= 0 && height < chainActive.Height()) + pb = chainActive[height]; + + if (pb == NULL || !pb->nHeight) + return 0; + + // If lookup is -1, then use blocks since last difficulty change. + if (lookup <= 0) + lookup = pb->nHeight % 2016 + 1; + + // If lookup is larger than chain, then set it to chain length. + if (lookup > pb->nHeight) + lookup = pb->nHeight; + + CBlockIndex *pb0 = pb; + int64_t minTime = pb0->GetBlockTime(); + int64_t maxTime = minTime; + for (int i = 0; i < lookup; i++) { + pb0 = pb0->pprev; + int64_t time = pb0->GetBlockTime(); + minTime = std::min(time, minTime); + maxTime = std::max(time, maxTime); + } + + // In case there's a situation where minTime == maxTime, we don't want a divide by zero exception. + if (minTime == maxTime) + return 0; + + uint256 workDiff = pb->nChainWork - pb0->nChainWork; + int64_t timeDiff = maxTime - minTime; + + return (int64_t)(workDiff.getdouble() / timeDiff); +} + +Value getnetworkhashps(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 2) + throw runtime_error("Help message not found\n"); + + return GetNetworkHashPS(params.size() > 0 ? params[0].get_int() : 120, params.size() > 1 ? params[1].get_int() : -1); +} + +#ifdef ENABLE_WALLET +Value getgenerate(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + return GetBoolArg("-gen", false); +} + + +Value setgenerate(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) + throw runtime_error("Help message not found\n"); + + if (pwalletMain == NULL) + throw JSONRPCError(RPC_METHOD_NOT_FOUND, "Method not found (disabled)"); + + bool fGenerate = true; + if (params.size() > 0) + fGenerate = params[0].get_bool(); + + int nGenProcLimit = -1; + if (params.size() > 1) + { + nGenProcLimit = params[1].get_int(); + if (nGenProcLimit == 0) + fGenerate = false; + } + + // -regtest mode: don't return until nGenProcLimit blocks are generated + if (fGenerate && Params().MineBlocksOnDemand()) + { + int nHeightStart = 0; + int nHeightEnd = 0; + int nHeight = 0; + int nGenerate = (nGenProcLimit > 0 ? nGenProcLimit : 1); + CReserveKey reservekey(pwalletMain); + + { // Don't keep cs_main locked + LOCK(cs_main); + nHeightStart = chainActive.Height(); + nHeight = nHeightStart; + nHeightEnd = nHeightStart+nGenerate; + } + unsigned int nExtraNonce = 0; + Array blockHashes; + while (nHeight < nHeightEnd) + { + auto_ptr pblocktemplate(CreateNewBlockWithKey(reservekey)); + if (!pblocktemplate.get()) + throw JSONRPCError(RPC_INTERNAL_ERROR, "Wallet keypool empty"); + CBlock *pblock = &pblocktemplate->block; + { + LOCK(cs_main); +/* MCHN START */ + IncrementExtraNonce(pblock, chainActive.Tip(), nExtraNonce,pwalletMain); +// IncrementExtraNonce(pblock, chainActive.Tip(), nExtraNonce); +/* MCHN START */ + } + while (!CheckProofOfWork(pblock->GetHash(), pblock->nBits)) { + // Yes, there is a chance every nonce could fail to satisfy the -regtest + // target -- 1 in 2^(2^32). That ain't gonna happen. + ++pblock->nNonce; + } + CValidationState state; + if (!ProcessNewBlock(state, NULL, pblock)) + throw JSONRPCError(RPC_INTERNAL_ERROR, "ProcessNewBlock, block not accepted"); + ++nHeight; + blockHashes.push_back(pblock->GetHash().GetHex()); + } + return blockHashes; + } + else // Not -regtest: start generate thread, return immediately + { + mapArgs["-gen"] = (fGenerate ? "1" : "0"); + mapArgs ["-genproclimit"] = itostr(nGenProcLimit); + GenerateBitcoins(fGenerate, pwalletMain, nGenProcLimit); + } + + return Value::null; +} + +Value gethashespersec(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + if (GetTimeMillis() - nHPSTimerStart > 8000) + return (int64_t)0; + return (int64_t)dHashesPerSec; +} +#endif + + +Value getmininginfo(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + Object obj; + obj.push_back(Pair("blocks", (int)chainActive.Height())); + obj.push_back(Pair("currentblocksize", (uint64_t)nLastBlockSize)); + obj.push_back(Pair("currentblocktx", (uint64_t)nLastBlockTx)); + obj.push_back(Pair("difficulty", (double)GetDifficulty())); + obj.push_back(Pair("errors", GetWarnings("statusbar"))); + obj.push_back(Pair("genproclimit", (int)GetArg("-genproclimit", -1))); + obj.push_back(Pair("networkhashps", getnetworkhashps(params, false))); + obj.push_back(Pair("pooledtx", (uint64_t)mempool.size())); + obj.push_back(Pair("testnet", Params().TestnetToBeDeprecatedFieldRPC())); +/* MCHN START */ +// obj.push_back(Pair("chain", Params().NetworkIDString())); + obj.push_back(Pair("chain",Params().TestnetToBeDeprecatedFieldRPC() ? "test" : "main")); +/* MCHN END */ +#ifdef ENABLE_WALLET + obj.push_back(Pair("generate", getgenerate(params, false))); + obj.push_back(Pair("hashespersec", gethashespersec(params, false))); +#endif + return obj; +} + + +// NOTE: Unlike wallet RPC (which use BTC values), mining RPCs follow GBT (BIP 22) in using satoshi amounts +Value prioritisetransaction(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 3) + throw runtime_error("Help message not found\n"); + +/* MCHN START */ +// uint256 hash = ParseHashStr(params[0].get_str(), "txid"); + +// CAmount nAmount = params[2].get_int64(); + +// mempool.PrioritiseTransaction(hash, params[0].get_str(), params[1].get_real(), nAmount); +// return true; + return false; +/* MCHN END */ +} + + +// NOTE: Assumes a conclusive result; if result is inconclusive, it must be handled by caller +static Value BIP22ValidationResult(const CValidationState& state) +{ + if (state.IsValid()) + return Value::null; + + std::string strRejectReason = state.GetRejectReason(); + if (state.IsError()) + throw JSONRPCError(RPC_VERIFY_ERROR, strRejectReason); + if (state.IsInvalid()) + { + if (strRejectReason.empty()) + return "rejected"; + return strRejectReason; + } + // Should be impossible + return "valid?"; +} + +Value getblocktemplate(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 1) + throw runtime_error("Help message not found\n"); + + std::string strMode = "template"; + Value lpval = Value::null; + if (params.size() > 0) + { + const Object& oparam = params[0].get_obj(); + const Value& modeval = find_value(oparam, "mode"); + if (modeval.type() == str_type) + strMode = modeval.get_str(); + else if (modeval.type() == null_type) + { + /* Do nothing */ + } + else + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid mode"); + lpval = find_value(oparam, "longpollid"); + + if (strMode == "proposal") + { + const Value& dataval = find_value(oparam, "data"); + if (dataval.type() != str_type) + throw JSONRPCError(RPC_TYPE_ERROR, "Missing data String key for proposal"); + + CBlock block; + if (!DecodeHexBlk(block, dataval.get_str())) + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "Block decode failed"); + + uint256 hash = block.GetHash(); + BlockMap::iterator mi = mapBlockIndex.find(hash); + if (mi != mapBlockIndex.end()) { + CBlockIndex *pindex = mi->second; + if (pindex->IsValid(BLOCK_VALID_SCRIPTS)) + return "duplicate"; + if (pindex->nStatus & BLOCK_FAILED_MASK) + return "duplicate-invalid"; + return "duplicate-inconclusive"; + } + + CBlockIndex* const pindexPrev = chainActive.Tip(); + // TestBlockValidity only supports blocks built on the current Tip + if (block.hashPrevBlock != pindexPrev->GetBlockHash()) + return "inconclusive-not-best-prevblk"; + CValidationState state; + TestBlockValidity(state, block, pindexPrev, false, true); + return BIP22ValidationResult(state); + } + } + + if (strMode != "template") + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid mode"); + + if (vNodes.empty()) + throw JSONRPCError(RPC_CLIENT_NOT_CONNECTED, "MultiChain is not connected!"); + + if (IsInitialBlockDownload()) + throw JSONRPCError(RPC_CLIENT_IN_INITIAL_DOWNLOAD, "MultiChain is downloading blocks..."); + + static unsigned int nTransactionsUpdatedLast; + + if (lpval.type() != null_type) + { + // Wait to respond until either the best block changes, OR a minute has passed and there are more transactions + uint256 hashWatchedChain; + boost::system_time checktxtime; + unsigned int nTransactionsUpdatedLastLP; + + if (lpval.type() == str_type) + { + // Format: + std::string lpstr = lpval.get_str(); + + hashWatchedChain.SetHex(lpstr.substr(0, 64)); + nTransactionsUpdatedLastLP = atoi64(lpstr.substr(64)); + } + else + { + // NOTE: Spec does not specify behaviour for non-string longpollid, but this makes testing easier + hashWatchedChain = chainActive.Tip()->GetBlockHash(); + nTransactionsUpdatedLastLP = nTransactionsUpdatedLast; + } + + // Release the wallet and main lock while waiting +#ifdef ENABLE_WALLET + if(pwalletMain) + LEAVE_CRITICAL_SECTION(pwalletMain->cs_wallet); +#endif + LEAVE_CRITICAL_SECTION(cs_main); + { + checktxtime = boost::get_system_time() + boost::posix_time::minutes(1); + + boost::unique_lock lock(csBestBlock); + while (chainActive.Tip()->GetBlockHash() == hashWatchedChain && IsRPCRunning()) + { + if (!cvBlockChange.timed_wait(lock, checktxtime)) + { + // Timeout: Check transactions for update + if (mempool.GetTransactionsUpdated() != nTransactionsUpdatedLastLP) + break; + checktxtime += boost::posix_time::seconds(10); + } + } + } + ENTER_CRITICAL_SECTION(cs_main); +#ifdef ENABLE_WALLET + if(pwalletMain) + ENTER_CRITICAL_SECTION(pwalletMain->cs_wallet); +#endif + + if (!IsRPCRunning()) + throw JSONRPCError(RPC_CLIENT_NOT_CONNECTED, "Shutting down"); + // TODO: Maybe recheck connections/IBD and (if something wrong) send an expires-immediately template to stop miners? + } + + // Update block + static CBlockIndex* pindexPrev; + static int64_t nStart; + static CBlockTemplate* pblocktemplate; + if (pindexPrev != chainActive.Tip() || + (mempool.GetTransactionsUpdated() != nTransactionsUpdatedLast && GetTime() - nStart > 5)) + { + // Clear pindexPrev so future calls make a new block, despite any failures from here on + pindexPrev = NULL; + + // Store the pindexBest used before CreateNewBlock, to avoid races + nTransactionsUpdatedLast = mempool.GetTransactionsUpdated(); + CBlockIndex* pindexPrevNew = chainActive.Tip(); + nStart = GetTime(); + + // Create new block + if(pblocktemplate) + { + delete pblocktemplate; + pblocktemplate = NULL; + } + CScript scriptDummy = CScript() << OP_TRUE; + pblocktemplate = CreateNewBlock(scriptDummy); + if (!pblocktemplate) + throw JSONRPCError(RPC_OUT_OF_MEMORY, "Out of memory"); + + // Need to update only after we know CreateNewBlock succeeded + pindexPrev = pindexPrevNew; + } + CBlock* pblock = &pblocktemplate->block; // pointer for convenience + + // Update nTime + UpdateTime(pblock, pindexPrev); + pblock->nNonce = 0; + + static const Array aCaps = boost::assign::list_of("proposal"); + + Array transactions; + map setTxIndex; + int i = 0; + BOOST_FOREACH (CTransaction& tx, pblock->vtx) + { + uint256 txHash = tx.GetHash(); + setTxIndex[txHash] = i++; + + if (tx.IsCoinBase()) + continue; + + Object entry; + + entry.push_back(Pair("data", EncodeHexTx(tx))); + + entry.push_back(Pair("hash", txHash.GetHex())); + + Array deps; + BOOST_FOREACH (const CTxIn &in, tx.vin) + { + if (setTxIndex.count(in.prevout.hash)) + deps.push_back(setTxIndex[in.prevout.hash]); + } + entry.push_back(Pair("depends", deps)); + + int index_in_template = i - 1; + entry.push_back(Pair("fee", pblocktemplate->vTxFees[index_in_template])); + entry.push_back(Pair("sigops", pblocktemplate->vTxSigOps[index_in_template])); + + transactions.push_back(entry); + } + + Object aux; + aux.push_back(Pair("flags", HexStr(COINBASE_FLAGS.begin(), COINBASE_FLAGS.end()))); + + uint256 hashTarget = uint256().SetCompact(pblock->nBits); + + static Array aMutable; + if (aMutable.empty()) + { + aMutable.push_back("time"); + aMutable.push_back("transactions"); + aMutable.push_back("prevblock"); + } + + Object result; + result.push_back(Pair("capabilities", aCaps)); + result.push_back(Pair("version", pblock->nVersion)); + result.push_back(Pair("previousblockhash", pblock->hashPrevBlock.GetHex())); + result.push_back(Pair("transactions", transactions)); + result.push_back(Pair("coinbaseaux", aux)); + result.push_back(Pair("coinbasevalue", (int64_t)pblock->vtx[0].vout[0].nValue)); + result.push_back(Pair("longpollid", chainActive.Tip()->GetBlockHash().GetHex() + i64tostr(nTransactionsUpdatedLast))); + result.push_back(Pair("target", hashTarget.GetHex())); + result.push_back(Pair("mintime", (int64_t)pindexPrev->GetMedianTimePast()+1)); + result.push_back(Pair("mutable", aMutable)); + result.push_back(Pair("noncerange", "00000000ffffffff")); + result.push_back(Pair("sigoplimit", (int64_t)MAX_BLOCK_SIGOPS)); + result.push_back(Pair("sizelimit", (int64_t)MAX_BLOCK_SIZE)); + result.push_back(Pair("curtime", pblock->GetBlockTime())); + result.push_back(Pair("bits", strprintf("%08x", pblock->nBits))); + result.push_back(Pair("height", (int64_t)(pindexPrev->nHeight+1))); + + return result; +} + +class submitblock_StateCatcher : public CValidationInterface +{ +public: + uint256 hash; + bool found; + CValidationState state; + + submitblock_StateCatcher(const uint256 &hashIn) : hash(hashIn), found(false), state() {}; + +protected: + virtual void BlockChecked(const CBlock& block, const CValidationState& stateIn) { + if (block.GetHash() != hash) + return; + found = true; + state = stateIn; + }; +}; + +Value submitblock(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) + throw runtime_error("Help message not found\n"); + + CBlock block; + if (!DecodeHexBlk(block, params[0].get_str())) + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "Block decode failed"); + + uint256 hash = block.GetHash(); + BlockMap::iterator mi = mapBlockIndex.find(hash); + if (mi != mapBlockIndex.end()) { + CBlockIndex *pindex = mi->second; + if (pindex->IsValid(BLOCK_VALID_SCRIPTS)) + return "duplicate"; + if (pindex->nStatus & BLOCK_FAILED_MASK) + return "duplicate-invalid"; + // Otherwise, we might only have the header - process the block before returning + } + + CValidationState state; + submitblock_StateCatcher sc(block.GetHash()); + RegisterValidationInterface(&sc); + bool fAccepted = ProcessNewBlock(state, NULL, &block); + UnregisterValidationInterface(&sc); + if (mi != mapBlockIndex.end()) + { + if (fAccepted && !sc.found) + return "duplicate-inconclusive"; + return "duplicate"; + } + if (fAccepted) + { + if (!sc.found) + return "inconclusive"; + state = sc.state; + } + return BIP22ValidationResult(state); +} + +Value estimatefee(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + RPCTypeCheck(params, boost::assign::list_of(int_type)); + + int nBlocks = params[0].get_int(); + if (nBlocks < 1) + nBlocks = 1; + + CFeeRate feeRate = mempool.estimateFee(nBlocks); + if (feeRate == CFeeRate(0)) + return -1.0; + + return ValueFromAmount(feeRate.GetFeePerK()); +} + +Value estimatepriority(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + RPCTypeCheck(params, boost::assign::list_of(int_type)); + + int nBlocks = params[0].get_int(); + if (nBlocks < 1) + nBlocks = 1; + + return mempool.estimatePriority(nBlocks); +} diff --git a/src/rpc/rpcmisc.cpp b/src/rpc/rpcmisc.cpp new file mode 100644 index 00000000..02d3c82c --- /dev/null +++ b/src/rpc/rpcmisc.cpp @@ -0,0 +1,675 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "structs/base58.h" +#include "version/clientversion.h" +#include "core/init.h" +#include "core/main.h" +#include "net/net.h" +#include "net/netbase.h" +#include "rpc/rpcserver.h" +#include "utils/timedata.h" +#include "utils/util.h" +#ifdef ENABLE_WALLET +#include "wallet/wallet.h" +#include "wallet/walletdb.h" +#endif + +#include + +/* MCHN START */ +#include "multichain/multichain.h" +#include "wallet/wallettxs.h" +std::string BurnAddress(const std::vector& vchVersion); +/* MCHN END */ + +#include +#include "json/json_spirit_utils.h" +#include "json/json_spirit_value.h" + + +using namespace boost; +using namespace boost::assign; +using namespace json_spirit; +using namespace std; + +bool CBitcoinAddressFromTxEntity(CBitcoinAddress &address,mc_TxEntity *lpEntity); +Object AddressEntry(CBitcoinAddress& address,uint32_t verbose); + +/** + * @note Do not add or change anything in the information returned by this + * method. `getinfo` exists for backwards-compatibility only. It combines + * information from wildly different sources in the program, which is a mess, + * and is thus planned to be deprecated eventually. + * + * Based on the source of the information, new information should be added to: + * - `getblockchaininfo`, + * - `getnetworkinfo` or + * - `getwalletinfo` + * + * Or alternatively, create a specific query method for the information. + **/ +Value getinfo(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + proxyType proxy; + GetProxy(NET_IPV4, proxy); + + Object obj; +/* MCHN START */ +// obj.push_back(Pair("version", CLIENT_VERSION)); +// obj.push_back(Pair("protocolversion", PROTOCOL_VERSION)); + obj.push_back(Pair("version", mc_gState->GetVersion())); + obj.push_back(Pair("protocolversion", mc_gState->m_NetworkParams->GetInt64Param("protocolversion"))); + obj.push_back(Pair("chainname", string(mc_gState->m_NetworkParams->Name()))); + obj.push_back(Pair("description", string((char*)mc_gState->m_NetworkParams->GetParam("chaindescription",NULL)))); + obj.push_back(Pair("protocol", string((char*)mc_gState->m_NetworkParams->GetParam("chainprotocol",NULL)))); + obj.push_back(Pair("port", GetListenPort())); + obj.push_back(Pair("setupblocks", mc_gState->m_NetworkParams->GetInt64Param("setupfirstblocks"))); + + obj.push_back(Pair("nodeaddress", MultichainServerAddress() + strprintf(":%d",GetListenPort()))); + + obj.push_back(Pair("burnaddress", BurnAddress(Params().Base58Prefix(CChainParams::PUBKEY_ADDRESS)))); + obj.push_back(Pair("incomingpaused", (mc_gState->m_NodePausedState & MC_NPS_INCOMING) ? true : false)); + obj.push_back(Pair("miningpaused", (mc_gState->m_NodePausedState & MC_NPS_MINING) ? true : false)); + +/* MCHN END */ +#ifdef ENABLE_WALLET + if (pwalletMain) { + obj.push_back(Pair("walletversion", pwalletMain->GetVersion())); + obj.push_back(Pair("balance", ValueFromAmount(pwalletMain->GetBalance()))); + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + if(mc_gState->m_WalletMode & MC_WMD_MAP_TXS) + { + obj.push_back(Pair("walletdbversion", -1)); + } + else + { + obj.push_back(Pair("walletdbversion", 2)); + } + } + else + { + obj.push_back(Pair("walletdbversion", 1)); + } + } +#endif +/* MCHN START */ + obj.push_back(Pair("reindex", fReindex)); +/* MCHN END */ + obj.push_back(Pair("blocks", (int)chainActive.Height())); + obj.push_back(Pair("timeoffset", GetTimeOffset())); + obj.push_back(Pair("connections", (int)vNodes.size())); + obj.push_back(Pair("proxy", (proxy.IsValid() ? proxy.ToStringIPPort() : string()))); + obj.push_back(Pair("difficulty", (double)GetDifficulty())); + obj.push_back(Pair("testnet", Params().TestnetToBeDeprecatedFieldRPC())); +#ifdef ENABLE_WALLET + if (pwalletMain) { + obj.push_back(Pair("keypoololdest", pwalletMain->GetOldestKeyPoolTime())); + obj.push_back(Pair("keypoolsize", (int)pwalletMain->GetKeyPoolSize())); + } + if (pwalletMain && pwalletMain->IsCrypted()) + obj.push_back(Pair("unlocked_until", nWalletUnlockTime)); + obj.push_back(Pair("paytxfee", ValueFromAmount(payTxFee.GetFeePerK()))); +#endif + +/* MCHN START */ + ::minRelayTxFee = CFeeRate(MIN_RELAY_TX_FEE); +/* MCHN END */ + + obj.push_back(Pair("relayfee", ValueFromAmount(::minRelayTxFee.GetFeePerK()))); + obj.push_back(Pair("errors", GetWarnings("statusbar"))); + return obj; +} + +#ifdef ENABLE_WALLET +class DescribeAddressVisitor : public boost::static_visitor +{ +private: + isminetype mine; + +public: + DescribeAddressVisitor(isminetype mineIn) : mine(mineIn) {} + + Object operator()(const CNoDestination &dest) const { return Object(); } + + Object operator()(const CKeyID &keyID) const { + Object obj; + CPubKey vchPubKey; + obj.push_back(Pair("isscript", false)); + if (mine == ISMINE_SPENDABLE) { + pwalletMain->GetPubKey(keyID, vchPubKey); + obj.push_back(Pair("pubkey", HexStr(vchPubKey))); + obj.push_back(Pair("iscompressed", vchPubKey.IsCompressed())); + } + return obj; + } + + Object operator()(const CScriptID &scriptID) const { + Object obj; + obj.push_back(Pair("isscript", true)); + if (mine != ISMINE_NO) { + CScript subscript; + pwalletMain->GetCScript(scriptID, subscript); + std::vector addresses; + txnouttype whichType; + int nRequired; + ExtractDestinations(subscript, whichType, addresses, nRequired); + obj.push_back(Pair("script", GetTxnOutputType(whichType))); + obj.push_back(Pair("hex", HexStr(subscript.begin(), subscript.end()))); + Array a; + BOOST_FOREACH(const CTxDestination& addr, addresses) + a.push_back(CBitcoinAddress(addr).ToString()); + obj.push_back(Pair("addresses", a)); + if (whichType == TX_MULTISIG) + obj.push_back(Pair("sigsrequired", nRequired)); + } + return obj; + } +}; +#endif + +/* MCHN START */ +Value getblockchainparams(const json_spirit::Array& params, bool fHelp) +{ + if (fHelp || params.size() > 1) // MCHN + throw runtime_error("Help message not found\n"); + + + bool fDisplay = true; + if (params.size() > 0) + fDisplay = params[0].get_bool(); + + Object obj; + int protocol_version=(int)mc_gState->m_NetworkParams->GetInt64Param("protocolversion"); + for(int i=0;im_NetworkParams->m_Count;i++) + { + if((mc_gState->m_NetworkParams->m_lpParams+i)->IsRelevant(protocol_version)) + { + + string param_name; + Value param_value; + unsigned char* ptr; + int size; + string param_string="";; + + ptr=(unsigned char*)mc_gState->m_NetworkParams->GetParam((mc_gState->m_NetworkParams->m_lpParams+i)->m_Name,&size); + if(size == 0) + { + ptr=NULL; + } + else + { + if(((mc_gState->m_NetworkParams->m_lpParams+i)->m_Type & MC_PRM_DATA_TYPE_MASK) == MC_PRM_STRING) + { + if(size == 1) + { + ptr=NULL; + } + } + } + + if(fDisplay) + { + param_name=(mc_gState->m_NetworkParams->m_lpParams+i)->m_DisplayName; + } + else + { + param_name=(mc_gState->m_NetworkParams->m_lpParams+i)->m_Name; + } + + if(ptr) + { + switch((mc_gState->m_NetworkParams->m_lpParams+i)->m_Type & MC_PRM_DATA_TYPE_MASK) + { + case MC_PRM_BINARY: + for(int c=0;cm_NetworkParams->m_lpParams+i)->m_Type & MC_PRM_DECIMAL) + { + param_value=((double)mc_GetLE(ptr,4))/MC_PRM_DECIMAL_GRANULARITY; + } + else + { + param_value=(int)mc_GetLE(ptr,4); + } + break; + case MC_PRM_UINT32: + if((mc_gState->m_NetworkParams->m_lpParams+i)->m_Type & MC_PRM_DECIMAL) + { + param_value=((double)mc_GetLE(ptr,4))/MC_PRM_DECIMAL_GRANULARITY; + } + else + { + param_value=mc_GetLE(ptr,4); + } + break; + case MC_PRM_INT64: + param_value=mc_GetLE(ptr,8); + break; + case MC_PRM_DOUBLE: + param_value=*(double*)ptr; + break; + } + } + + obj.push_back(Pair(param_name,param_value)); + } + } + + return obj; +} + +void SetSynchronizedFlag(CTxDestination &dest,Object &ret) +{ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxEntityStat entStat; + CKeyID *lpKeyID=boost::get (&dest); + CScriptID *lpScriptID=boost::get (&dest); + entStat.Zero(); + if(lpKeyID) + { + memcpy(&entStat,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + entStat.m_Entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + } + if(lpScriptID) + { + memcpy(&entStat,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + entStat.m_Entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + } + if(pwalletTxsMain->FindEntity(&entStat)) + { + if(entStat.m_Flags & MC_EFL_NOT_IN_SYNC) + { + ret.push_back(Pair("synchronized",false)); + } + else + { + ret.push_back(Pair("synchronized",true)); + } + } + } +} + +Object AddressEntry(CBitcoinAddress& address,uint32_t verbose) +{ +// 0x01 add isvalid=true +// 0x02 verbose + + Object ret; + if(verbose & 0x01) + { + ret.push_back(Pair("isvalid", true)); + } + CTxDestination dest = address.Get(); + string currentAddress = address.ToString(); + ret.push_back(Pair("address", currentAddress)); +#ifdef ENABLE_WALLET + isminetype mine = pwalletMain ? IsMine(*pwalletMain, dest) : ISMINE_NO; + ret.push_back(Pair("ismine", (mine & ISMINE_SPENDABLE) ? true : false)); + if(verbose & 0x02) + { + if (mine != ISMINE_NO) { + ret.push_back(Pair("iswatchonly", (mine & ISMINE_WATCH_ONLY) ? true: false)); + Object detail = boost::apply_visitor(DescribeAddressVisitor(mine), dest); + ret.insert(ret.end(), detail.begin(), detail.end()); + } + if (pwalletMain && pwalletMain->mapAddressBook.count(dest)) + ret.push_back(Pair("account", pwalletMain->mapAddressBook[dest].name)); + #endif + SetSynchronizedFlag(dest,ret); + } + + return ret; +} + + +/* MCHN END */ + +Value validateaddress(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) // MCHN + throw runtime_error("Help message not found\n"); + + CBitcoinAddress address(params[0].get_str()); + bool isValid = address.IsValid(); + + Object ret; +/* MCHN START */ + if(!isValid) + { + ret.push_back(Pair("isvalid", false)); + } + else + { + return AddressEntry(address,0x03); + } +/* + ret.push_back(Pair("isvalid", isValid)); + if (isValid) + { + CTxDestination dest = address.Get(); + string currentAddress = address.ToString(); + ret.push_back(Pair("address", currentAddress)); +#ifdef ENABLE_WALLET + isminetype mine = pwalletMain ? IsMine(*pwalletMain, dest) : ISMINE_NO; + ret.push_back(Pair("ismine", (mine & ISMINE_SPENDABLE) ? true : false)); + if (mine != ISMINE_NO) { + ret.push_back(Pair("iswatchonly", (mine & ISMINE_WATCH_ONLY) ? true: false)); + Object detail = boost::apply_visitor(DescribeAddressVisitor(mine), dest); + ret.insert(ret.end(), detail.begin(), detail.end()); + } + if (pwalletMain && pwalletMain->mapAddressBook.count(dest)) + ret.push_back(Pair("account", pwalletMain->mapAddressBook[dest].name)); +#endif + SetSynchronizedFlag(dest,ret); + + } +*/ + return ret; +/* MCHN END */ +} + +/* MCHN START */ + +Value createkeypairs(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 1) // MCHN + throw runtime_error("Help message not found\n"); + + int count=1; + if (params.size() > 0) + { + if(params[0].type() == int_type) + { + count=params[0].get_int(); + if(count < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + + Array retArray; + bool fCompressed = true; + + for(int i=0;i 1) // MCHN + throw runtime_error("Help message not found\n"); + + int verbose=0x00; + + if (params.size() > 0) + { + if(params[0].type() == bool_type) + { + if(params[0].get_bool()) + { + verbose=0x02; + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid value for 'verbose' parameter, should be boolean"); + } + } + + Array ret; + if((mc_gState->m_WalletMode & MC_WMD_TXS)) + { + int entity_count; + mc_TxEntityStat *lpEntity; + + entity_count=pwalletTxsMain->GetEntityListCount(); + for(int i=0;iGetEntity(i); + CBitcoinAddress address; + if( (lpEntity->m_Entity.m_EntityType & MC_TET_ORDERMASK) == MC_TET_CHAINPOS) + { + if(CBitcoinAddressFromTxEntity(address,&(lpEntity->m_Entity))) + { + if(verbose) + { + ret.push_back(AddressEntry(address,verbose)); + } + else + { + ret.push_back(address.ToString()); + } + } + } + } + } + else + { + // Find all addresses that have the given account + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, pwalletMain->mapAddressBook) + { + const CBitcoinAddress& address = item.first; + + if(verbose) + { + Object addr; + CTxDestination dest = address.Get(); + string currentAddress = address.ToString(); + addr.push_back(Pair("address", currentAddress)); + isminetype mine = pwalletMain ? IsMine(*pwalletMain, dest) : ISMINE_NO; + addr.push_back(Pair("ismine", (mine & ISMINE_SPENDABLE) ? true : false)); + if (mine != ISMINE_NO) { + addr.push_back(Pair("iswatchonly", (mine & ISMINE_WATCH_ONLY) ? true: false)); + Object detail = boost::apply_visitor(DescribeAddressVisitor(mine), dest); + addr.insert(addr.end(), detail.begin(), detail.end()); + } + if (pwalletMain && pwalletMain->mapAddressBook.count(dest)) + addr.push_back(Pair("account", pwalletMain->mapAddressBook[dest].name)); + /* MCHN START */ + SetSynchronizedFlag(dest,addr); + /* MCHN END */ + ret.push_back(addr); + } + else + { + ret.push_back(address.ToString()); + } + + } + } + return ret; +} + +/* MCHN END */ +/** + * Used by addmultisigaddress / createmultisig: + */ +CScript _createmultisig_redeemScript(const Array& params) +{ + int nRequired = params[0].get_int(); + const Array& keys = params[1].get_array(); + + // Gather public keys + if (nRequired < 1) + throw runtime_error("a multisignature address must require at least one key to redeem"); + if ((int)keys.size() < nRequired) + throw runtime_error( + strprintf("not enough keys supplied " + "(got %u keys, but need at least %d to redeem)", keys.size(), nRequired)); + if (keys.size() > 16) + throw runtime_error("Number of addresses involved in the multisignature address creation > 16\nReduce the number"); + std::vector pubkeys; + pubkeys.resize(keys.size()); + for (unsigned int i = 0; i < keys.size(); i++) + { + const std::string& ks = keys[i].get_str(); +#ifdef ENABLE_WALLET + // Case 1: Bitcoin address and we have full public key: + CBitcoinAddress address(ks); + if (pwalletMain && address.IsValid()) + { + CKeyID keyID; + if (!address.GetKeyID(keyID)) + throw runtime_error( + strprintf("%s does not refer to a key",ks)); + CPubKey vchPubKey; + if (!pwalletMain->GetPubKey(keyID, vchPubKey)) + throw runtime_error( + strprintf("no full public key for address %s",ks)); + if (!vchPubKey.IsFullyValid()) + throw runtime_error(" Invalid public key: "+ks); + pubkeys[i] = vchPubKey; + } + + // Case 2: hex public key + else +#endif + if (IsHex(ks)) + { + CPubKey vchPubKey(ParseHex(ks)); + if (!vchPubKey.IsFullyValid()) + throw runtime_error(" Invalid public key: "+ks); + pubkeys[i] = vchPubKey; + } + else + { + throw runtime_error(" Invalid public key: "+ks); + } + } + CScript result = GetScriptForMultisig(nRequired, pubkeys); + + if(Params().RequireStandard()) + { + if (result.size() > MAX_SCRIPT_ELEMENT_SIZE) + throw runtime_error( + strprintf("redeemScript exceeds size limit: %d > %d", result.size(), MAX_SCRIPT_ELEMENT_SIZE)); + } + + return result; +} + +Value createmultisig(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 2) // MCHN + { + throw runtime_error("Help message not found\n"); + } + + +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(mc_gState->m_NetworkParams->GetInt64Param("allowp2shoutputs") == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "P2SH outputs are not allowed for this network"); + } + } +/* MCHN END */ + + + // Construct using pay-to-script-hash: + CScript inner = _createmultisig_redeemScript(params); + CScriptID innerID(inner); + CBitcoinAddress address(innerID); + + Object result; + result.push_back(Pair("address", address.ToString())); + result.push_back(Pair("redeemScript", HexStr(inner.begin(), inner.end()))); + + return result; +} + +Value verifymessage(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 3) // MCHN + throw runtime_error("Help message not found\n"); + + string strAddress = params[0].get_str(); + string strSign = params[1].get_str(); + string strMessage = params[2].get_str(); + + CBitcoinAddress addr(strAddress); + if (!addr.IsValid()) + throw JSONRPCError(RPC_TYPE_ERROR, "Invalid address"); + + CKeyID keyID; + if (!addr.GetKeyID(keyID)) + throw JSONRPCError(RPC_TYPE_ERROR, "Address does not refer to key"); + + bool fInvalid = false; + vector vchSig = DecodeBase64(strSign.c_str(), &fInvalid); + + if (fInvalid) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Malformed base64 encoding"); + + CHashWriter ss(SER_GETHASH, 0); + ss << strMessageMagic; + ss << strMessage; + + CPubKey pubkey; + if (!pubkey.RecoverCompact(ss.GetHash(), vchSig)) + return false; + + return (pubkey.GetID() == keyID); +} + +Value setmocktime(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + if (!Params().MineBlocksOnDemand()) + throw runtime_error("setmocktime for regression testing (-regtest mode) only"); + + RPCTypeCheck(params, boost::assign::list_of(int_type)); + SetMockTime(params[0].get_int64()); + + return Value::null; +} diff --git a/src/rpc/rpcnet.cpp b/src/rpc/rpcnet.cpp new file mode 100644 index 00000000..cee59ab3 --- /dev/null +++ b/src/rpc/rpcnet.cpp @@ -0,0 +1,342 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "rpc/rpcserver.h" + +#include "version/clientversion.h" +#include "core/main.h" +#include "net/net.h" +#include "net/netbase.h" +#include "protocol/netprotocol.h" +#include "utils/sync.h" +#include "utils/timedata.h" +#include "utils/util.h" +#include "version/bcversion.h" + +/* MCHN START */ +#include "structs/base58.h" +/* MCHN END */ + +#include + +#include "json/json_spirit_value.h" + +using namespace json_spirit; +using namespace std; + +Value getconnectioncount(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + LOCK(cs_vNodes); + return (int)vNodes.size(); +} + +Value ping(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + // Request that each node send a ping during next message processing pass + LOCK(cs_vNodes); + BOOST_FOREACH(CNode* pNode, vNodes) { + pNode->fPingQueued = true; + } + + return Value::null; +} + +static void CopyNodeStats(std::vector& vstats) +{ + vstats.clear(); + + LOCK(cs_vNodes); + vstats.reserve(vNodes.size()); + BOOST_FOREACH(CNode* pnode, vNodes) { + CNodeStats stats; + pnode->copyStats(stats); + vstats.push_back(stats); + } +} + +Value getpeerinfo(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + vector vstats; + CopyNodeStats(vstats); + + Array ret; + + BOOST_FOREACH(const CNodeStats& stats, vstats) { +/* MCHN START */ + if(stats.fSuccessfullyConnected) + { +/* MCHN END */ + Object obj; + CNodeStateStats statestats; + bool fStateStats = GetNodeStateStats(stats.nodeid, statestats); + obj.push_back(Pair("id", stats.nodeid)); + obj.push_back(Pair("addr", stats.addrName)); + if (!(stats.addrLocal.empty())) + obj.push_back(Pair("addrlocal", stats.addrLocal)); + obj.push_back(Pair("services", strprintf("%016x", stats.nServices))); + obj.push_back(Pair("lastsend", stats.nLastSend)); + obj.push_back(Pair("lastrecv", stats.nLastRecv)); + obj.push_back(Pair("bytessent", stats.nSendBytes)); + obj.push_back(Pair("bytesrecv", stats.nRecvBytes)); + obj.push_back(Pair("conntime", stats.nTimeConnected)); + obj.push_back(Pair("pingtime", stats.dPingTime)); + if (stats.dPingWait > 0.0) + obj.push_back(Pair("pingwait", stats.dPingWait)); + obj.push_back(Pair("version", stats.nVersion)); + // Use the sanitized form of subver here, to avoid tricksy remote peers from + // corrupting or modifiying the JSON output by putting special characters in + // their ver message. + obj.push_back(Pair("subver", stats.cleanSubVer)); +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecanconnect")) + { + Value null_value; + obj.push_back(Pair("handshakelocal", null_value)); + obj.push_back(Pair("handshake", null_value)); + } + else + { + obj.push_back(Pair("handshakelocal", CBitcoinAddress(stats.kAddrLocal).ToString())); + obj.push_back(Pair("handshake", CBitcoinAddress(stats.kAddrRemote).ToString())); + } + } +/* MCHN END */ + obj.push_back(Pair("inbound", stats.fInbound)); + obj.push_back(Pair("startingheight", stats.nStartingHeight)); + if (fStateStats) { + obj.push_back(Pair("banscore", statestats.nMisbehavior)); + obj.push_back(Pair("synced_headers", statestats.nSyncHeight)); + obj.push_back(Pair("synced_blocks", statestats.nCommonHeight)); + Array heights; + BOOST_FOREACH(int height, statestats.vHeightInFlight) { + heights.push_back(height); + } + obj.push_back(Pair("inflight", heights)); + } + obj.push_back(Pair("whitelisted", stats.fWhitelisted)); + + + + ret.push_back(obj); +/* MCHN START */ + } +/* MCHN END */ + } + + return ret; +} + +Value addnode(const Array& params, bool fHelp) +{ + string strCommand; + if (params.size() == 2) + strCommand = params[1].get_str(); + if (fHelp || params.size() != 2 || + (strCommand != "onetry" && strCommand != "add" && strCommand != "remove")) + throw runtime_error("Help message not found\n"); + + string strNode = params[0].get_str(); + + if (strCommand == "onetry") + { + CAddress addr; + OpenNetworkConnection(addr, NULL, strNode.c_str()); + return Value::null; + } + + LOCK(cs_vAddedNodes); + vector::iterator it = vAddedNodes.begin(); + for(; it != vAddedNodes.end(); it++) + if (strNode == *it) + break; + + if (strCommand == "add") + { + if (it != vAddedNodes.end()) + throw JSONRPCError(RPC_CLIENT_NODE_ALREADY_ADDED, "Error: Node already added"); + vAddedNodes.push_back(strNode); + } + else if(strCommand == "remove") + { + if (it == vAddedNodes.end()) + throw JSONRPCError(RPC_CLIENT_NODE_NOT_ADDED, "Error: Node has not been added."); + vAddedNodes.erase(it); + } + + return Value::null; +} + +Value getaddednodeinfo(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) + throw runtime_error("Help message not found\n"); + + bool fDns = params[0].get_bool(); + + list laddedNodes(0); + if (params.size() == 1) + { + LOCK(cs_vAddedNodes); + BOOST_FOREACH(string& strAddNode, vAddedNodes) + laddedNodes.push_back(strAddNode); + } + else + { + string strNode = params[1].get_str(); + LOCK(cs_vAddedNodes); + BOOST_FOREACH(string& strAddNode, vAddedNodes) + if (strAddNode == strNode) + { + laddedNodes.push_back(strAddNode); + break; + } + if (laddedNodes.size() == 0) + throw JSONRPCError(RPC_CLIENT_NODE_NOT_ADDED, "Error: Node has not been added."); + } + + Array ret; + if (!fDns) + { + BOOST_FOREACH(string& strAddNode, laddedNodes) + { + Object obj; + obj.push_back(Pair("addednode", strAddNode)); + ret.push_back(obj); + } + return ret; + } + + list > > laddedAddreses(0); + BOOST_FOREACH(string& strAddNode, laddedNodes) + { + vector vservNode(0); + if(Lookup(strAddNode.c_str(), vservNode, Params().GetDefaultPort(), fNameLookup, 0)) + laddedAddreses.push_back(make_pair(strAddNode, vservNode)); + else + { + Object obj; + obj.push_back(Pair("addednode", strAddNode)); + obj.push_back(Pair("connected", false)); + Array addresses; + obj.push_back(Pair("addresses", addresses)); + } + } + + LOCK(cs_vNodes); + for (list > >::iterator it = laddedAddreses.begin(); it != laddedAddreses.end(); it++) + { + Object obj; + obj.push_back(Pair("addednode", it->first)); + + Array addresses; + bool fConnected = false; + BOOST_FOREACH(CService& addrNode, it->second) + { + bool fFound = false; + Object node; + node.push_back(Pair("address", addrNode.ToString())); + BOOST_FOREACH(CNode* pnode, vNodes) + if (pnode->addr == addrNode) + { + fFound = true; + fConnected = true; + node.push_back(Pair("connected", pnode->fInbound ? "inbound" : "outbound")); + break; + } + if (!fFound) + node.push_back(Pair("connected", "false")); + addresses.push_back(node); + } + obj.push_back(Pair("connected", fConnected)); + obj.push_back(Pair("addresses", addresses)); + ret.push_back(obj); + } + + return ret; +} + +Value getnettotals(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 0) + throw runtime_error("Help message not found\n"); + + Object obj; + obj.push_back(Pair("totalbytesrecv", CNode::GetTotalBytesRecv())); + obj.push_back(Pair("totalbytessent", CNode::GetTotalBytesSent())); + obj.push_back(Pair("timemillis", GetTimeMillis())); + return obj; +} + +static Array GetNetworksInfo() +{ + Array networks; + for(int n=0; n(n); + if(network == NET_UNROUTABLE) + continue; + proxyType proxy; + Object obj; + GetProxy(network, proxy); + obj.push_back(Pair("name", GetNetworkName(network))); + obj.push_back(Pair("limited", IsLimited(network))); + obj.push_back(Pair("reachable", IsReachable(network))); + obj.push_back(Pair("proxy", proxy.IsValid() ? proxy.ToStringIPPort() : string())); + networks.push_back(obj); + } + return networks; +} + +Value getnetworkinfo(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) + throw runtime_error("Help message not found\n"); + + Object obj; + obj.push_back(Pair("version", CLIENT_VERSION)); +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + obj.push_back(Pair("subversion", + FormatSubVersion("MultiChain", mc_gState->GetProtocolVersion(), std::vector()))); + } + else + { + obj.push_back(Pair("subversion", + FormatSubVersion(CLIENT_NAME, CLIENT_VERSION, std::vector()))); + } +/* MCHN END */ + obj.push_back(Pair("protocolversion",PROTOCOL_VERSION)); + obj.push_back(Pair("localservices", strprintf("%016x", nLocalServices))); + obj.push_back(Pair("timeoffset", GetTimeOffset())); + obj.push_back(Pair("connections", (int)vNodes.size())); + obj.push_back(Pair("networks", GetNetworksInfo())); + obj.push_back(Pair("relayfee", ValueFromAmount(::minRelayTxFee.GetFeePerK()))); + Array localAddresses; + { + LOCK(cs_mapLocalHost); + BOOST_FOREACH(const PAIRTYPE(CNetAddr, LocalServiceInfo) &item, mapLocalHost) + { + Object rec; + rec.push_back(Pair("address", item.first.ToString())); + rec.push_back(Pair("port", item.second.nPort)); + rec.push_back(Pair("score", item.second.nScore)); + localAddresses.push_back(rec); + } + } + obj.push_back(Pair("localaddresses", localAddresses)); + return obj; +} diff --git a/src/rpc/rpcpermissions.cpp b/src/rpc/rpcpermissions.cpp new file mode 100644 index 00000000..d088a1ca --- /dev/null +++ b/src/rpc/rpcpermissions.cpp @@ -0,0 +1,696 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + + +#include "rpc/rpcwallet.h" + + +string AllowedPermissions() +{ + string ret="connect,send,receive,issue,mine,admin"; + if(mc_gState->m_Features->ActivatePermission()) + { + ret += ",activate"; + } + if(mc_gState->m_Features->Streams()) + { + ret += ",create"; + } + + return ret; +} + +Value grantoperation(const Array& params) +{ + vector addresses; + set setAddress; + + stringstream ss(params[1].get_str()); + string tok; + + while(getline(ss, tok, ',')) + { + CBitcoinAddress address(tok); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, string("Invalid address: ")+tok); + if (setAddress.count(address)) + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid parameter, duplicated address: ")+tok); + addresses.push_back(address.Get()); + setAddress.insert(address); + } + + // Amount + CAmount nAmount = 0; + if (params.size() > 4 && params[4].type() != null_type) + { + nAmount = AmountFromValue(params[4]); + } + + // Wallet comments + CWalletTx wtx; + if (params.size() > 7 && params[7].type() != null_type && !params[7].get_str().empty()) + wtx.mapValue["comment"] = params[7].get_str(); + if (params.size() > 8 && params[8].type() != null_type && !params[8].get_str().empty()) + wtx.mapValue["to"] = params[8].get_str(); + + mc_Script *lpScript; + lpScript=new mc_Script; + + uint32_t type,from,to,timestamp; + + type=0; + string entity_identifier, permission_type; + entity_identifier=""; + if (params.size() > 2 && params[2].type() != null_type && !params[2].get_str().empty()) + { + permission_type=params[2].get_str(); + int period_pos=permission_type.find_last_of("."); + + if(period_pos >= 0) + { + entity_identifier=permission_type.substr(0,period_pos); + permission_type=permission_type.substr(period_pos+1,permission_type.size()); + } + } + + + + from=0; + if (params.size() > 5 && params[5].type() != null_type) + { + if(params[5].get_int64()<0) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "start_block should be non-negative"); + } + from=(uint32_t)params[5].get_int64(); + } + + to=4294967295U; + if (params.size() > 6 && params[6].type() != null_type && (params[6].get_int64() != -1)) + { + if(params[6].get_int64()<0) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "end_block should be non-negative or -1"); + } + to=(uint32_t)params[6].get_int64(); + } + + if(((type & MC_PTP_RECEIVE) == 0) || (from >= to)) + { + if(nAmount > 0) + { + BOOST_FOREACH(CTxDestination& txdest, addresses) + { + if(!AddressCanReceive(txdest)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Destination address doesn't have receive permission"); + } + } + } + } + timestamp=mc_TimeNowAsUInt(); + + mc_EntityDetails entity; + entity.Zero(); + if (entity_identifier.size()) + { + ParseEntityIdentifier(entity_identifier,&entity, MC_ENT_TYPE_ANY); + LogPrintf("mchn: Granting %s permission(s) to address %s (%ld-%ld), Entity TxID: %s, Name: %s\n",permission_type,params[1].get_str(),from,to, + ((uint256*)entity.GetTxID())->ToString().c_str(),entity.GetName()); + + type=mc_gState->m_Permissions->GetPermissionType(permission_type.c_str(),entity.GetEntityType()); + + if(type == 0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid permission"); + + lpScript->SetEntity(entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET); + } + else + { + type=mc_gState->m_Permissions->GetPermissionType(permission_type.c_str(),MC_ENT_TYPE_NONE); + + if(type == 0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid permission"); + + LogPrintf("mchn: Granting %s permission(s) to address %s (%ld-%ld)\n",permission_type,params[1].get_str(),from,to); + } + + + + lpScript->SetPermission(type,from,to,timestamp); + + vector fromaddresses; + if(params[0].get_str() != "*") + { + fromaddresses=ParseAddresses(params[0].get_str(),false,false); + } + + if(fromaddresses.size() > 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + mc_EntityDetails found_entity; + CScript scriptOpReturn=CScript(); + if (params.size() > 3) + { + scriptOpReturn=ParseRawMetadata(params[3],0x0002,NULL,&found_entity); + } + + if(fromaddresses.size() == 1) + { + if( (IsMine(*pwalletMain, fromaddresses[0]) & ISMINE_SPENDABLE) != ISMINE_SPENDABLE ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key for from-address is not found in this wallet"); + } + + CKeyID *lpKeyID=boost::get (&fromaddresses[0]); + if(lpKeyID != NULL) + { + if(entity.GetEntityType()) + { + if(mc_gState->m_Permissions->IsActivateEnough(type)) + { + if(mc_gState->m_Permissions->CanActivate(entity.GetTxID(),(unsigned char*)(lpKeyID)) == 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "from-address doesn't have activate or admin permission for this entity"); + } + } + else + { + if(mc_gState->m_Permissions->CanAdmin(entity.GetTxID(),(unsigned char*)(lpKeyID)) == 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "from-address doesn't have admin permission for this entity"); + } + } + } + else + { + if(mc_gState->m_Permissions->IsActivateEnough(type)) + { + if(mc_gState->m_Permissions->CanActivate(NULL,(unsigned char*)(lpKeyID)) == 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "from-address doesn't have activate or admin permission"); + } + } + else + { + if(mc_gState->m_Permissions->CanAdmin(NULL,(unsigned char*)(lpKeyID)) == 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "from-address doesn't have admin permission"); + } + } + } + } + else + { + throw JSONRPCError(RPC_INTERNAL_ERROR, "Please use createrawtransaction to grant/revoke from P2PKH addresses"); + } + if(found_entity.GetEntityType() == MC_ENT_TYPE_STREAM) + { + FindAddressesWithPublishPermission(fromaddresses,&found_entity); + } + } + else + { + bool admin_found=false; + + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, pwalletMain->mapAddressBook) + { + const CBitcoinAddress& address = item.first; + CKeyID keyID; + + if(address.GetKeyID(keyID)) + { + bool valid_publisher=true; + if(found_entity.GetEntityType() == MC_ENT_TYPE_STREAM) + { + if((found_entity.AnyoneCanWrite() == 0) && (mc_gState->m_Permissions->CanWrite(found_entity.GetTxID(),(unsigned char*)(&keyID)) == 0)) + { + valid_publisher=false; + } + } + + if(valid_publisher) + { + if(entity.GetEntityType()) + { + if(mc_gState->m_Permissions->IsActivateEnough(type)) + { + if(mc_gState->m_Permissions->CanActivate(entity.GetTxID(),(unsigned char*)(&keyID))) + { + admin_found=true; + } + } + else + { + if(mc_gState->m_Permissions->CanAdmin(entity.GetTxID(),(unsigned char*)(&keyID))) + { + admin_found=true; + } + } + } + else + { + if(mc_gState->m_Permissions->IsActivateEnough(type)) + { + if(mc_gState->m_Permissions->CanActivate(NULL,(unsigned char*)(&keyID))) + { + admin_found=true; + } + } + else + { + if(mc_gState->m_Permissions->CanAdmin(NULL,(unsigned char*)(&keyID))) + { + admin_found=true; + } + } + } + } + } + } + + if(!admin_found) + { + string strErrorMessage="This wallet doesn't have addresses with "; + if(found_entity.GetEntityType() == MC_ENT_TYPE_STREAM) + { + strErrorMessage+="write permission for given stream and "; + } + if(mc_gState->m_Permissions->IsActivateEnough(type)) + { + strErrorMessage+="activate or admin permission"; + } + else + { + strErrorMessage+="admin permission"; + } + if(entity.GetEntityType()) + { + strErrorMessage+=" for this entity"; + } + throw JSONRPCError(RPC_INVALID_PARAMETER, strErrorMessage); + } + } + + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + SendMoneyToSeveralAddresses(addresses, nAmount, wtx, lpScript, scriptOpReturn, fromaddresses); + + delete lpScript; + return wtx.GetHash().GetHex(); + +} + +Value grantwithmetadatafrom(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 4 || params.size() > 7) + throw runtime_error("Help message not found\n"); + + return grantoperation(params); +} + +Value grantwithmetadata(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 3 || params.size() > 6) + throw runtime_error("Help message not found\n"); + + Array ext_params; + ext_params.push_back("*"); + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + } + return grantoperation(ext_params); +} + +Value grantfromcmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 3 || params.size() > 8) + throw runtime_error("Help message not found\n"); + + Array ext_params; + int param_count=0; + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + param_count++; + if(param_count==3) + { + ext_params.push_back(""); + } + } + return grantoperation(ext_params); +} + + +Value grantcmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 7) + throw runtime_error("Help message not found\n"); + + Array ext_params; + ext_params.push_back("*"); + int param_count=1; + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + param_count++; + if(param_count==3) + { + ext_params.push_back(""); + } + } + return grantoperation(ext_params); + +} + +Value revokefromcmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 3 || params.size() > 6) + throw runtime_error("Help message not found\n"); + + Array ext_params; + int param_count=0; + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + param_count++; + if(param_count==3) + { + ext_params.push_back(""); + } + if(param_count==4) + { + ext_params.push_back(0); + ext_params.push_back(0); + } + } + if(param_count < 4) + { + ext_params.push_back(0); + ext_params.push_back(0); + ext_params.push_back(0); + } + return grantoperation(ext_params); + +} + + + +Value revokecmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 5) + throw runtime_error("Help message not found\n"); + + Array ext_params; + ext_params.push_back("*"); + int param_count=1; + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + param_count++; + if(param_count==3) + { + ext_params.push_back(""); + } + if(param_count==4) + { + ext_params.push_back(0); + ext_params.push_back(0); + } + } + if(param_count < 4) + { + ext_params.push_back(0); + ext_params.push_back(0); + ext_params.push_back(0); + } + return grantoperation(ext_params); + +} + +Value listpermissions(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 3) + throw runtime_error("Help message not found\n"); + + mc_Buffer *permissions; + + Array results; + uint32_t type; + + type=0; + + string entity_identifier, permission_type; + entity_identifier=""; + permission_type="all"; + if (params.size() > 0 && params[0].type() != null_type)// && !params[0].get_str().empty()) + { + permission_type=params[0].get_str(); +// int period_pos=permission_type.find_last_of(".",permission_type.size()); + int period_pos=permission_type.find_last_of("."); + + if(period_pos >= 0) + { + entity_identifier=permission_type.substr(0,period_pos); + permission_type=permission_type.substr(period_pos+1,permission_type.size()); + } + } + + + mc_EntityDetails entity; + const unsigned char *lpEntity; + lpEntity=NULL; + entity.Zero(); + if (entity_identifier.size()) + { + ParseEntityIdentifier(entity_identifier,&entity, MC_ENT_TYPE_ANY); + lpEntity=entity.GetTxID(); + } + + type=mc_gState->m_Permissions->GetPermissionType(permission_type.c_str(),entity.GetEntityType()); + if(type == 0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid permission"); + + + int verbose=0; + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + verbose=params[2].get_int(); + } + if(params[2].type() == bool_type) + { + if(params[2].get_bool()) + { + verbose=1; + } + } + } + + permissions=NULL; + if (params.size() > 1 && params[1].type() != null_type && + ((params[1].type() != str_type) || ((params[1].get_str() !="*"))))// && (params[1].get_str() !="") )) ) + { + vector addresses; + vector inputStrings=ParseStringList(params[1]); + for(int is=0;is<(int)inputStrings.size();is++) + { + string tok=inputStrings[is]; + CBitcoinAddress address(tok); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, string("Invalid address: ")+tok); + addresses.push_back(address.Get()); + } + + if(addresses.size() == 0) + { + return results; + } + BOOST_FOREACH(CTxDestination& dest, addresses) + { + CKeyID *lpKeyID=boost::get (&dest); + CScriptID *lpScriptID=boost::get (&dest); + if(((lpKeyID == NULL) && (lpScriptID == NULL))) + { + mc_gState->m_Permissions->FreePermissionList(permissions); + LogPrintf("mchn: Invalid address"); + return false; + } + + { + LOCK(cs_main); + if(lpKeyID != NULL) + { + permissions=mc_gState->m_Permissions->GetPermissionList(lpEntity,(unsigned char*)(lpKeyID),type,permissions); + } + else + { + permissions=mc_gState->m_Permissions->GetPermissionList(lpEntity,(unsigned char*)(lpScriptID),type,permissions); + } + } + } + } + else + { + { + LOCK(cs_main); + permissions=mc_gState->m_Permissions->GetPermissionList(lpEntity,NULL,type,NULL); + } + } + + if(permissions == NULL) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Cannot open permission database"); + + for(int i=0;iGetCount();i++) + { + Object entry; + mc_PermissionDetails *plsRow; + mc_PermissionDetails *plsDet; + mc_PermissionDetails *plsPend; + bool take_it; + int flags,consensus,remaining; + plsRow=(mc_PermissionDetails *)(permissions->GetRow(i)); + + take_it=true; + flags=plsRow->m_Flags; + consensus=plsRow->m_RequiredAdmins; + if(flags & MC_PFL_IS_SCRIPTHASH) + { + uint160 addr; + memcpy(&addr,plsRow->m_Address,sizeof(uint160)); + CScriptID lpScriptID=CScriptID(addr); + entry.push_back(Pair("address", CBitcoinAddress(lpScriptID).ToString())); + entry.push_back(Pair("isp2shaddress", true)); + } + else + { + uint160 addr; + memcpy(&addr,plsRow->m_Address,sizeof(uint160)); + CKeyID lpKeyID=CKeyID(addr); + entry.push_back(Pair("address", CBitcoinAddress(lpKeyID).ToString())); + } + entry.push_back(Pair("for", PermissionForFieldEntry(&entity))); + switch(plsRow->m_Type) + { + case MC_PTP_CONNECT:entry.push_back(Pair("type", "connect"));break; + case MC_PTP_SEND :entry.push_back(Pair("type", "send"));break; + case MC_PTP_RECEIVE:entry.push_back(Pair("type", "receive"));break; + case MC_PTP_WRITE :entry.push_back(Pair("type", "write"));break; + case MC_PTP_CREATE :entry.push_back(Pair("type", "create"));break; + case MC_PTP_ISSUE :entry.push_back(Pair("type", "issue"));break; + case MC_PTP_MINE :entry.push_back(Pair("type", "mine"));break; + case MC_PTP_ADMIN :entry.push_back(Pair("type", "admin"));break; + case MC_PTP_ACTIVATE : + if(mc_gState->m_Features->ActivatePermission()) + { + entry.push_back(Pair("type", "activate")); + } + break; + default:take_it=false; + } + entry.push_back(Pair("startblock", (int64_t)plsRow->m_BlockFrom)); + entry.push_back(Pair("endblock", (int64_t)plsRow->m_BlockTo)); + if( (plsRow->m_BlockFrom >= plsRow->m_BlockTo) && + (((flags & MC_PFL_HAVE_PENDING) == 0) || !verbose) ) + { + take_it=false; + } + if(take_it) + { + Array admins; + Array pending; + mc_Buffer *details; + + if((flags & MC_PFL_HAVE_PENDING) || (consensus>1)) + { + details=mc_gState->m_Permissions->GetPermissionDetails(plsRow); + } + else + { + details=NULL; + } + + if(details) + { + for(int j=0;jGetCount();j++) + { + plsDet=(mc_PermissionDetails *)(details->GetRow(j)); + remaining=plsDet->m_RequiredAdmins; + if(remaining > 0) + { + uint160 addr; + memcpy(&addr,plsDet->m_LastAdmin,sizeof(uint160)); + CKeyID lpKeyID=CKeyID(addr); + admins.push_back(CBitcoinAddress(lpKeyID).ToString()); + } + } + for(int j=0;jGetCount();j++) + { + plsDet=(mc_PermissionDetails *)(details->GetRow(j)); + remaining=plsDet->m_RequiredAdmins; + if(remaining == 0) + { + Object pend_obj; + Array pend_admins; + uint32_t block_from=plsDet->m_BlockFrom; + uint32_t block_to=plsDet->m_BlockTo; + for(int k=j;kGetCount();k++) + { + plsPend=(mc_PermissionDetails *)(details->GetRow(k)); + remaining=plsPend->m_RequiredAdmins; + + if(remaining == 0) + { + if(block_from == plsPend->m_BlockFrom) + { + if(block_to == plsPend->m_BlockTo) + { + uint160 addr; + memcpy(&addr,plsPend->m_LastAdmin,sizeof(uint160)); + CKeyID lpKeyID=CKeyID(addr); +// CKeyID lpKeyID=CKeyID(*(uint160*)((void*)(plsPend->m_LastAdmin))); + pend_admins.push_back(CBitcoinAddress(lpKeyID).ToString()); + plsPend->m_RequiredAdmins=0x01010101; + } + } + } + } + pend_obj.push_back(Pair("startblock", (int64_t)block_from)); + pend_obj.push_back(Pair("endblock", (int64_t)block_to)); + pend_obj.push_back(Pair("admins", pend_admins)); + pend_obj.push_back(Pair("required", (int64_t)(consensus-pend_admins.size()))); + pending.push_back(pend_obj); + } + } + mc_gState->m_Permissions->FreePermissionList(details); + } + else + { + uint160 addr; + memcpy(&addr,plsRow->m_LastAdmin,sizeof(uint160)); + CKeyID lpKeyID=CKeyID(addr); + admins.push_back(CBitcoinAddress(lpKeyID).ToString()); + } + if(verbose) + { + entry.push_back(Pair("admins", admins)); + entry.push_back(Pair("pending", pending)); + } + results.push_back(entry); + } + } + + mc_gState->m_Permissions->FreePermissionList(permissions); + + return results; +} + diff --git a/src/rpc/rpcprotocol.cpp b/src/rpc/rpcprotocol.cpp new file mode 100644 index 00000000..59045275 --- /dev/null +++ b/src/rpc/rpcprotocol.cpp @@ -0,0 +1,298 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "rpc/rpcprotocol.h" + +#include "version/clientversion.h" +#include "utils/tinyformat.h" +#include "utils/util.h" +#include "utils/utilstrencodings.h" +#include "utils/utiltime.h" +#include "version/bcversion.h" + +#include + +#include "multichain/multichain.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "json/json_spirit_writer_template.h" + +using namespace std; +using namespace boost; +using namespace boost::asio; +using namespace json_spirit; + +//! Number of bytes to allocate and read at most at once in post data +const size_t POST_READ_SIZE = 256 * 1024; + +/** + * HTTP protocol + * + * This ain't Apache. We're just using HTTP header for the length field + * and to be compatible with other JSON-RPC implementations. + */ + +string HTTPPost(const string& strMsg, const map& mapRequestHeaders) +{ + ostringstream s; + s << "POST / HTTP/1.1\r\n" + << "User-Agent: bitcoin-json-rpc/" << FormatFullVersion() << "\r\n" + << "Host: 127.0.0.1\r\n" + << "Content-Type: application/json\r\n" + << "Content-Length: " << strMsg.size() << "\r\n" + << "Connection: close\r\n" + << "Accept: application/json\r\n"; + BOOST_FOREACH(const PAIRTYPE(string, string)& item, mapRequestHeaders) + s << item.first << ": " << item.second << "\r\n"; + s << "\r\n" << strMsg; + + return s.str(); +} + +static string rfc1123Time() +{ + return DateTimeStrFormat("%a, %d %b %Y %H:%M:%S +0000", GetTime()); +} + +static const char *httpStatusDescription(int nStatus) +{ + switch (nStatus) { + case HTTP_OK: return "OK"; + case HTTP_BAD_REQUEST: return "Bad Request"; + case HTTP_FORBIDDEN: return "Forbidden"; + case HTTP_NOT_FOUND: return "Not Found"; + case HTTP_INTERNAL_SERVER_ERROR: return "Internal Server Error"; + default: return ""; + } +} + +string HTTPError(int nStatus, bool keepalive, bool headersOnly) +{ + if (nStatus == HTTP_UNAUTHORIZED) + return strprintf("HTTP/1.0 401 Authorization Required\r\n" + "Date: %s\r\n" + "Server: bitcoin-json-rpc/%s\r\n" + "WWW-Authenticate: Basic realm=\"jsonrpc\"\r\n" + "Content-Type: text/html\r\n" + "Content-Length: 296\r\n" + "\r\n" + "\r\n" + "\r\n" + "\r\n" + "Error\r\n" + "\r\n" + "\r\n" + "

401 Unauthorized.

\r\n" + "\r\n", rfc1123Time(), FormatFullVersion()); + + return HTTPReply(nStatus, httpStatusDescription(nStatus), keepalive, + headersOnly, "text/plain"); +} + +string HTTPReplyHeader(int nStatus, bool keepalive, size_t contentLength, const char *contentType) +{ + return strprintf( + "HTTP/1.1 %d %s\r\n" + "Date: %s\r\n" + "Connection: %s\r\n" + "Content-Length: %u\r\n" + "Content-Type: %s\r\n" + "Server: bitcoin-json-rpc/%s\r\n" + "\r\n", + nStatus, + httpStatusDescription(nStatus), + rfc1123Time(), + keepalive ? "keep-alive" : "close", + contentLength, + contentType, + FormatFullVersion()); +} + +string HTTPReply(int nStatus, const string& strMsg, bool keepalive, + bool headersOnly, const char *contentType) +{ + if (headersOnly) + { + return HTTPReplyHeader(nStatus, keepalive, 0, contentType); + } else { + return HTTPReplyHeader(nStatus, keepalive, strMsg.size(), contentType) + strMsg; + } +} + +bool ReadHTTPRequestLine(std::basic_istream& stream, int &proto, + string& http_method, string& http_uri) +{ + string str; + getline(stream, str); + + // HTTP request line is space-delimited + vector vWords; + boost::split(vWords, str, boost::is_any_of(" ")); + if (vWords.size() < 2) + return false; + + // HTTP methods permitted: GET, POST + http_method = vWords[0]; + if (http_method != "GET" && http_method != "POST") + return false; + + // HTTP URI must be an absolute path, relative to current host + http_uri = vWords[1]; + if (http_uri.size() == 0 || http_uri[0] != '/') + return false; + + // parse proto, if present + string strProto = ""; + if (vWords.size() > 2) + strProto = vWords[2]; + + proto = 0; + const char *ver = strstr(strProto.c_str(), "HTTP/1."); + if (ver != NULL) + proto = atoi(ver+7); + + return true; +} + +int ReadHTTPStatus(std::basic_istream& stream, int &proto) +{ + string str; + getline(stream, str); + vector vWords; + boost::split(vWords, str, boost::is_any_of(" ")); + if (vWords.size() < 2) + return HTTP_INTERNAL_SERVER_ERROR; + proto = 0; + const char *ver = strstr(str.c_str(), "HTTP/1."); + if (ver != NULL) + proto = atoi(ver+7); + return atoi(vWords[1].c_str()); +} + +int ReadHTTPHeaders(std::basic_istream& stream, map& mapHeadersRet) +{ + int nLen = 0; + while (true) + { + string str; + std::getline(stream, str); + if (str.empty() || str == "\r") + break; + string::size_type nColon = str.find(":"); + if (nColon != string::npos) + { + string strHeader = str.substr(0, nColon); + boost::trim(strHeader); + boost::to_lower(strHeader); + string strValue = str.substr(nColon+1); + boost::trim(strValue); + mapHeadersRet[strHeader] = strValue; + if (strHeader == "content-length") + nLen = atoi(strValue.c_str()); + } + } + return nLen; +} + + +int ReadHTTPMessage(std::basic_istream& stream, map& mapHeadersRet, string& strMessageRet, + int nProto, size_t max_size) +{ + mapHeadersRet.clear(); + strMessageRet = ""; + + // Read header + int nLen = ReadHTTPHeaders(stream, mapHeadersRet); + if (nLen < 0 || (size_t)nLen > max_size) + return HTTP_INTERNAL_SERVER_ERROR; + + // Read message + if (nLen > 0) + { + vector vch; + size_t ptr = 0; + while (ptr < (size_t)nLen) + { + size_t bytes_to_read = std::min((size_t)nLen - ptr, POST_READ_SIZE); + vch.resize(ptr + bytes_to_read); + stream.read(&vch[ptr], bytes_to_read); + if (!stream) // Connection lost while reading + return HTTP_INTERNAL_SERVER_ERROR; + ptr += bytes_to_read; + } + strMessageRet = string(vch.begin(), vch.end()); + } + + string sConHdr = mapHeadersRet["connection"]; + + if ((sConHdr != "close") && (sConHdr != "keep-alive")) + { + if (nProto >= 1) + mapHeadersRet["connection"] = "keep-alive"; + else + mapHeadersRet["connection"] = "close"; + } + + return HTTP_OK; +} + +/** + * JSON-RPC protocol. Bitcoin speaks version 1.0 for maximum compatibility, + * but uses JSON-RPC 1.1/2.0 standards for parts of the 1.0 standard that were + * unspecified (HTTP errors and contents of 'error'). + * + * 1.0 spec: http://json-rpc.org/wiki/specification + * 1.2 spec: http://jsonrpc.org/historical/json-rpc-over-http.html + * http://www.codeproject.com/KB/recipes/JSON_Spirit.aspx + */ + +string JSONRPCRequest(const string& strMethod, const Array& params, const Value& id) +{ + Object request; + request.push_back(Pair("method", strMethod)); + request.push_back(Pair("params", params)); + request.push_back(Pair("id", id)); +/* MCHN START */ + request.push_back(Pair("chain_name", string(mc_gState->m_Params->NetworkName()))); +/* MCHN END */ + return write_string(Value(request), false) + "\n"; +} + +Object JSONRPCReplyObj(const Value& result, const Value& error, const Value& id) +{ + Object reply; + if (error.type() != null_type) + reply.push_back(Pair("result", Value::null)); + else + reply.push_back(Pair("result", result)); + reply.push_back(Pair("error", error)); + reply.push_back(Pair("id", id)); + return reply; +} + +string JSONRPCReply(const Value& result, const Value& error, const Value& id) +{ + Object reply = JSONRPCReplyObj(result, error, id); + return write_string(Value(reply), false) + "\n"; +} + +Object JSONRPCError(int code, const string& message) +{ + Object error; + error.push_back(Pair("code", code)); + error.push_back(Pair("message", message)); + return error; +} diff --git a/src/rpc/rpcprotocol.h b/src/rpc/rpcprotocol.h new file mode 100644 index 00000000..f51775f8 --- /dev/null +++ b/src/rpc/rpcprotocol.h @@ -0,0 +1,170 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_RPCPROTOCOL_H +#define BITCOIN_RPCPROTOCOL_H + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "json/json_spirit_reader_template.h" +#include "json/json_spirit_utils.h" +#include "json/json_spirit_writer_template.h" + +//! HTTP status codes +enum HTTPStatusCode +{ + HTTP_OK = 200, + HTTP_BAD_REQUEST = 400, + HTTP_UNAUTHORIZED = 401, + HTTP_FORBIDDEN = 403, + HTTP_NOT_FOUND = 404, + HTTP_INTERNAL_SERVER_ERROR = 500, + HTTP_SERVICE_UNAVAILABLE = 503, +}; + +//! Bitcoin RPC error codes +enum RPCErrorCode +{ + //! Standard JSON-RPC 2.0 errors + RPC_INVALID_REQUEST = -32600, + RPC_METHOD_NOT_FOUND = -32601, + RPC_INVALID_PARAMS = -32602, + RPC_INTERNAL_ERROR = -32603, + RPC_PARSE_ERROR = -32700, + + //! General application defined errors + RPC_MISC_ERROR = -1, //! std::exception thrown in command handling + RPC_FORBIDDEN_BY_SAFE_MODE = -2, //! Server is in safe mode, and command is not allowed in safe mode + RPC_TYPE_ERROR = -3, //! Unexpected type was passed as parameter + RPC_INVALID_ADDRESS_OR_KEY = -5, //! Invalid address or key + RPC_OUT_OF_MEMORY = -7, //! Ran out of memory during operation + RPC_INVALID_PARAMETER = -8, //! Invalid, missing or duplicate parameter + RPC_DATABASE_ERROR = -20, //! Database error + RPC_DESERIALIZATION_ERROR = -22, //! Error parsing or validating structure in raw format + RPC_VERIFY_ERROR = -25, //! General error during transaction or block submission + RPC_VERIFY_REJECTED = -26, //! Transaction or block was rejected by network rules + RPC_VERIFY_ALREADY_IN_CHAIN = -27, //! Transaction already in chain + RPC_IN_WARMUP = -28, //! Client still warming up + + //! Aliases for backward compatibility + RPC_TRANSACTION_ERROR = RPC_VERIFY_ERROR, + RPC_TRANSACTION_REJECTED = RPC_VERIFY_REJECTED, + RPC_TRANSACTION_ALREADY_IN_CHAIN= RPC_VERIFY_ALREADY_IN_CHAIN, + + //! P2P client errors + RPC_CLIENT_NOT_CONNECTED = -9, //! Bitcoin is not connected + RPC_CLIENT_IN_INITIAL_DOWNLOAD = -10, //! Still downloading initial blocks + RPC_CLIENT_NODE_ALREADY_ADDED = -23, //! Node is already added + RPC_CLIENT_NODE_NOT_ADDED = -24, //! Node has not been added before + + //! Wallet errors + RPC_WALLET_ERROR = -4, //! Unspecified problem with wallet (key not found etc.) + RPC_WALLET_INSUFFICIENT_FUNDS = -6, //! Not enough funds in wallet or account + RPC_WALLET_INVALID_ACCOUNT_NAME = -11, //! Invalid account name + RPC_WALLET_KEYPOOL_RAN_OUT = -12, //! Keypool ran out, call keypoolrefill first + RPC_WALLET_UNLOCK_NEEDED = -13, //! Enter the wallet passphrase with walletpassphrase first + RPC_WALLET_PASSPHRASE_INCORRECT = -14, //! The wallet passphrase entered was incorrect + RPC_WALLET_WRONG_ENC_STATE = -15, //! Command given in wrong wallet encryption state (encrypting an encrypted wallet etc.) + RPC_WALLET_ENCRYPTION_FAILED = -16, //! Failed to encrypt the wallet + RPC_WALLET_ALREADY_UNLOCKED = -17, //! Wallet is already unlocked +}; + +/** + * IOStream device that speaks SSL but can also speak non-SSL + */ +template +class SSLIOStreamDevice : public boost::iostreams::device { +public: + SSLIOStreamDevice(boost::asio::ssl::stream &streamIn, bool fUseSSLIn) : stream(streamIn) + { + fUseSSL = fUseSSLIn; + fNeedHandshake = fUseSSLIn; + } + + void handshake(boost::asio::ssl::stream_base::handshake_type role) + { + if (!fNeedHandshake) return; + fNeedHandshake = false; + stream.handshake(role); + } + std::streamsize read(char* s, std::streamsize n) + { + handshake(boost::asio::ssl::stream_base::server); // HTTPS servers read first + if (fUseSSL) return stream.read_some(boost::asio::buffer(s, n)); + return stream.next_layer().read_some(boost::asio::buffer(s, n)); + } + std::streamsize write(const char* s, std::streamsize n) + { + handshake(boost::asio::ssl::stream_base::client); // HTTPS clients write first + if (fUseSSL) return boost::asio::write(stream, boost::asio::buffer(s, n)); + return boost::asio::write(stream.next_layer(), boost::asio::buffer(s, n)); + } + bool connect(const std::string& server, const std::string& port) + { + using namespace boost::asio::ip; + tcp::resolver resolver(stream.get_io_service()); + tcp::resolver::iterator endpoint_iterator; +#if BOOST_VERSION >= 104300 + try { +#endif + // The default query (flags address_configured) tries IPv6 if + // non-localhost IPv6 configured, and IPv4 if non-localhost IPv4 + // configured. + tcp::resolver::query query(server.c_str(), port.c_str()); + endpoint_iterator = resolver.resolve(query); +#if BOOST_VERSION >= 104300 + } catch(boost::system::system_error &e) + { + // If we at first don't succeed, try blanket lookup (IPv4+IPv6 independent of configured interfaces) + tcp::resolver::query query(server.c_str(), port.c_str(), resolver_query_base::flags()); + endpoint_iterator = resolver.resolve(query); + } +#endif + boost::system::error_code error = boost::asio::error::host_not_found; + tcp::resolver::iterator end; + while (error && endpoint_iterator != end) + { + stream.lowest_layer().close(); + stream.lowest_layer().connect(*endpoint_iterator++, error); + } + if (error) + return false; + return true; + } + +private: + bool fNeedHandshake; + bool fUseSSL; + boost::asio::ssl::stream& stream; +}; + +std::string HTTPPost(const std::string& strMsg, const std::map& mapRequestHeaders); +std::string HTTPError(int nStatus, bool keepalive, + bool headerOnly = false); +std::string HTTPReplyHeader(int nStatus, bool keepalive, size_t contentLength, + const char *contentType = "application/json"); +std::string HTTPReply(int nStatus, const std::string& strMsg, bool keepalive, + bool headerOnly = false, + const char *contentType = "application/json"); +bool ReadHTTPRequestLine(std::basic_istream& stream, int &proto, + std::string& http_method, std::string& http_uri); +int ReadHTTPStatus(std::basic_istream& stream, int &proto); +int ReadHTTPHeaders(std::basic_istream& stream, std::map& mapHeadersRet); +int ReadHTTPMessage(std::basic_istream& stream, std::map& mapHeadersRet, + std::string& strMessageRet, int nProto, size_t max_size); +std::string JSONRPCRequest(const std::string& strMethod, const json_spirit::Array& params, const json_spirit::Value& id); +json_spirit::Object JSONRPCReplyObj(const json_spirit::Value& result, const json_spirit::Value& error, const json_spirit::Value& id); +std::string JSONRPCReply(const json_spirit::Value& result, const json_spirit::Value& error, const json_spirit::Value& id); +json_spirit::Object JSONRPCError(int code, const std::string& message); + +#endif // BITCOIN_RPCPROTOCOL_H diff --git a/src/rpc/rpcrawtransaction.cpp b/src/rpc/rpcrawtransaction.cpp new file mode 100644 index 00000000..89246d9d --- /dev/null +++ b/src/rpc/rpcrawtransaction.cpp @@ -0,0 +1,1738 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "structs/base58.h" +#include "primitives/transaction.h" +#include "utils/core_io.h" +#include "core/init.h" +#include "wallet/keystore.h" +#include "core/main.h" +#include "net/net.h" +#include "rpc/rpcserver.h" +#include "script/script.h" +#include "script/sign.h" +#include "script/standard.h" +#include "structs/uint256.h" +#ifdef ENABLE_WALLET +#include "wallet/wallet.h" +#endif + +#include + +#include +#include "json/json_spirit_utils.h" +#include "json/json_spirit_value.h" + +#include "multichain/multichain.h" +#include "rpc/rpcutils.h" + +using namespace boost; +using namespace boost::assign; +using namespace json_spirit; +using namespace std; + +/* MCHN START */ +#include "utils/util.h" +#include "utils/utilmoneystr.h" +#include "wallet/wallettxs.h" + +bool OutputCanSend(COutput out); + +/* MCHN END */ + +void ScriptPubKeyToJSON(const CScript& scriptPubKey, Object& out, bool fIncludeHex) +{ + txnouttype type; + vector addresses; + int nRequired; + +/* MCHN START */ + const CScript& scriptToShow=RemoveOpDropsIfNeeded(scriptPubKey); + +/* + out.push_back(Pair("asm", scriptPubKey.ToString())); + if (fIncludeHex) + out.push_back(Pair("hex", HexStr(scriptPubKey.begin(), scriptPubKey.end()))); + */ + out.push_back(Pair("asm", scriptToShow.ToString())); + if (fIncludeHex) + out.push_back(Pair("hex", HexStr(scriptToShow.begin(), scriptToShow.end()))); +/* MCHN END */ + + + if (!ExtractDestinations(scriptPubKey, type, addresses, nRequired)) { + out.push_back(Pair("type", GetTxnOutputType(type))); + return; + } + + out.push_back(Pair("reqSigs", nRequired)); + out.push_back(Pair("type", GetTxnOutputType(type))); + + Array a; + BOOST_FOREACH(const CTxDestination& addr, addresses) + a.push_back(CBitcoinAddress(addr).ToString()); + out.push_back(Pair("addresses", a)); +} + +void TxToJSON(const CTransaction& tx, const uint256 hashBlock, Object& entry) +{ + entry.push_back(Pair("txid", tx.GetHash().GetHex())); + entry.push_back(Pair("version", tx.nVersion)); + entry.push_back(Pair("locktime", (int64_t)tx.nLockTime)); + Array vin; + BOOST_FOREACH(const CTxIn& txin, tx.vin) { + Object in; + if (tx.IsCoinBase()) + in.push_back(Pair("coinbase", HexStr(txin.scriptSig.begin(), txin.scriptSig.end()))); + else { + in.push_back(Pair("txid", txin.prevout.hash.GetHex())); + in.push_back(Pair("vout", (int64_t)txin.prevout.n)); + Object o; + o.push_back(Pair("asm", txin.scriptSig.ToString())); + o.push_back(Pair("hex", HexStr(txin.scriptSig.begin(), txin.scriptSig.end()))); + in.push_back(Pair("scriptSig", o)); + } + in.push_back(Pair("sequence", (int64_t)txin.nSequence)); + vin.push_back(in); + } + entry.push_back(Pair("vin", vin)); + +/* MCHN START */ + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; +/* MCHN END */ + + Array vdata; + Array vInputCache; + bool is_issuefirst=false; + bool is_issuemore=false; + int64_t issuerawqty=0; + unsigned char details_script[MC_ENT_MAX_SCRIPT_SIZE]; + unsigned char short_txid[MC_AST_SHORT_TXID_SIZE]; + char asset_name[MC_ENT_MAX_NAME_SIZE+1]; + int multiple; + int details_script_size=0; + int err; + bool detals_script_found=false; + uint32_t new_entity_type; + new_entity_type=MC_ENT_TYPE_NONE; + set streams_already_seen; + + Array vout; + for (unsigned int i = 0; i < tx.vout.size(); i++) { + const CTxOut& txout = tx.vout[i]; + Object out; + out.push_back(Pair("value", ValueFromAmount(txout.nValue))); + out.push_back(Pair("n", (int64_t)i)); + Object o; + ScriptPubKeyToJSON(txout.scriptPubKey, o, true); + out.push_back(Pair("scriptPubKey", o)); + +/* MCHN START */ +// TODO too many duplicate code with ListWalletTransactions and may be AccepMultiChainTransaction + const CScript& script1 = tx.vout[i].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + lpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + Value op_return_data; + string full_script_hex=""; + int asset_update; + uint32_t value_offset; + size_t value_size; + vInputCache.clear(); + + if(lpScript->IsOpReturnScript()) + { +// int e=mc_gState->m_TmpScript->GetNumElements()-1; + int e=lpScript->GetNumElements()-1; + { + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + lpScript->SetElement(0); + err=lpScript->GetNewEntityType(&new_entity_type,&asset_update,details_script,&details_script_size); + if((err == 0) && (new_entity_type == MC_ENT_TYPE_ASSET)) + { + if(asset_update == 0) + { + detals_script_found=true; + is_issuefirst=true; + *asset_name=0x00; + multiple=1; + value_offset=mc_FindSpecialParamInDetailsScript(details_script,details_script_size,MC_ENT_SPRM_NAME,&value_size); + if(value_offset<(uint32_t)details_script_size) + { + memcpy(asset_name,details_script+value_offset,value_size); + asset_name[value_size]=0x00; + } + value_offset=mc_FindSpecialParamInDetailsScript(details_script,details_script_size,MC_ENT_SPRM_ASSET_MULTIPLE,&value_size); + if(value_offset<(uint32_t)details_script_size) + { + multiple=mc_GetLE(details_script+value_offset,value_size); + } + } + } + else + { + err=lpScript->GetEntity(short_txid); + if(err == 0) + { + lpScript->SetElement(1); + err=lpScript->GetNewEntityType(&new_entity_type,&asset_update,details_script,&details_script_size); + if((err == 0) && (new_entity_type == MC_ENT_TYPE_ASSET)) + { + if(asset_update) + { + detals_script_found=true; + is_issuemore=true; + } + } + } + } + } + else + { + lpScript->SetElement(e); + err=lpScript->GetAssetDetails(asset_name,&multiple,details_script,&details_script_size); + if(err == 0) + { + detals_script_found=true; + } + } + } + + size_t elem_size; + const unsigned char *elem; + int cs_err,cs_offset,cs_new_offset,cs_size,cs_vin; + unsigned char *cs_script; + + if(lpScript->GetNumElements()<=1) + { + if(lpScript->GetNumElements()==1) + { + elem = lpScript->GetData(lpScript->GetNumElements()-1,&elem_size); + vdata.push_back(OpReturnEntry(elem,elem_size,tx.GetHash(),i)); + } + } + else + { + elem = lpScript->GetData(lpScript->GetNumElements()-1,&elem_size); + if(elem_size) + { + vdata.push_back(OpReturnEntry(elem,elem_size,tx.GetHash(),i)); + } + + lpScript->SetElement(0); + if(lpScript->GetNewEntityType(&new_entity_type)) + { + cs_offset=0; + while( (cs_err=lpScript->GetCachedScript(cs_offset,&cs_new_offset,&cs_vin,&cs_script,&cs_size)) == MC_ERR_NOERROR ) + { + if(cs_offset) + { + Object cs_obj; + cs_obj.push_back(Pair("vin", cs_vin)); + CScript scriptInputCache(cs_script, cs_script + cs_size); + cs_obj.push_back(Pair("asm", scriptInputCache.ToString())); + cs_obj.push_back(Pair("hex", HexStr(scriptInputCache.begin(), scriptInputCache.end()))); + vInputCache.push_back(cs_obj); + } + cs_offset=cs_new_offset; + } + } + } + } + + if(vInputCache.size()) + { + out.push_back(Pair("inputcache", vInputCache)); + } + + int required=0; + bool is_genesis; + bool is_valid_asset; + + asset_amounts->Clear(); + if(CreateAssetBalanceList(txout,asset_amounts,lpScript,&required)) + { + Array assets; + unsigned char *ptr; + const unsigned char *txid; + + for(int a=0;aGetCount();a++) + { + Object asset_entry; + ptr=(unsigned char *)asset_amounts->GetRow(a); + + uint256 hash=tx.GetHash(); + txid=(unsigned char*)&hash; + is_genesis=true; + is_valid_asset=true; + if( (mc_GetABRefType(ptr) != MC_AST_ASSET_REF_TYPE_SPECIAL) && + (mc_GetABRefType(ptr) != MC_AST_ASSET_REF_TYPE_GENESIS) ) + { + mc_EntityDetails entity; + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ptr)) + { + is_genesis=false; + txid=entity.GetTxID(); + } + else + { + is_valid_asset=false; + } + } + + if(is_valid_asset) + { + asset_entry=AssetEntry(txid,mc_GetABQuantity(ptr),3); + if(is_genesis) + { + asset_entry.push_back(Pair("type", "issuefirst")); + issuerawqty+=mc_GetABQuantity(ptr); + is_issuefirst=true; + } + else + { + switch(mc_GetABScriptType(ptr)) + { + case MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER: + asset_entry.push_back(Pair("type", "transfer")); + break; + case MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON: + asset_entry.push_back(Pair("type", "issuemore")); + issuerawqty+=mc_GetABQuantity(ptr); + is_issuemore=true; + break; + case MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER | MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON: + asset_entry.push_back(Pair("type", "issuemore+transfer")); + issuerawqty+=mc_GetABQuantity(ptr); + is_issuemore=true; + break; + case 0: + asset_entry.push_back(Pair("type", "issuefirst")); + is_issuefirst=true; + break; + } + } + + assets.push_back(asset_entry); + } + } + + out.push_back(Pair("assets", assets)); + } + Array permissions=PermissionEntries(txout,lpScript,false); + out.push_back(Pair("permissions", permissions)); + + Array items; + Value data_item_entry=DataItemEntry(tx,i,streams_already_seen, 0x03); + if(!data_item_entry.is_null()) + { + items.push_back(data_item_entry); + } + out.push_back(Pair("items", items)); +/* MCHN END */ + vout.push_back(out); + } + entry.push_back(Pair("vout", vout)); + if(is_issuefirst || is_issuemore) + { + Object issue; + Object details; + uint32_t offset,next_offset,param_value_start; + size_t param_value_size; + bool is_open=false; + if(detals_script_found) + { + offset=0; + + while((int)offset 0) + { + if(details_script[offset]) + { + if(details_script[offset] != 0xff) + { + if(mc_gState->m_Features->SpecialParamsInDetailsScript() || + ((strcmp((char*)details_script+offset,"multiple") != 0) && (strcmp((char*)details_script+offset,"name") != 0) )) + { + string param_name((char*)details_script+offset); + string param_value((char*)details_script+param_value_start,(char*)details_script+param_value_start+param_value_size); + details.push_back(Pair(param_name, param_value)); + } + } + } + else + { + switch(details_script[offset+1]) + { + case MC_ENT_SPRM_FOLLOW_ONS: + if( (param_value_size > 0) && (param_value_size <= 4) ) + { + if(mc_GetLE(details_script+param_value_start,param_value_size)) + { + is_open=true; + } + } + break; + } + } + } + offset=next_offset; + } + if(is_issuefirst) + { + issue.push_back(Pair("type", "issuefirst")); + if(strlen(asset_name)) + { + issue.push_back(Pair("name", string(asset_name))); + } + issue.push_back(Pair("multiple", multiple)); + issue.push_back(Pair("open", is_open)); + } + if(is_issuemore) + { + issue.push_back(Pair("type", "issuemore")); + mc_EntityDetails genesis_entity; + unsigned char *ptr; + uint256 genesis_hash; + if(mc_gState->m_Assets->FindEntityByShortTxID(&genesis_entity,short_txid)) + { + ptr=(unsigned char *)genesis_entity.GetName(); + if(ptr && strlen((char*)ptr)) + { + issue.push_back(Pair("name", string((char*)ptr))); + } + genesis_hash=*(uint256*)genesis_entity.GetTxID(); + issue.push_back(Pair("issuetxid", genesis_hash.GetHex())); + ptr=(unsigned char *)genesis_entity.GetRef(); + string assetref=""; + if(genesis_entity.IsUnconfirmedGenesis()) + { + Value null_value; + issue.push_back(Pair("assetref",null_value)); + } + else + { + assetref += itostr((int)mc_GetLE(ptr,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(ptr+4,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(ptr+8,2)); + issue.push_back(Pair("assetref", assetref)); + } + } + } + issue.push_back(Pair("details", details)); + } + entry.push_back(Pair("issue", issue)); + } + if(new_entity_type == MC_ENT_TYPE_STREAM) + { + uint256 txid=tx.GetHash(); + entry.push_back(Pair("create", StreamEntry((unsigned char*)&txid,0x05))); + } + + entry.push_back(Pair("data", vdata)); + +/* MCHN START */ + + delete lpScript; + delete asset_amounts; + +/* MCHN END */ + + if (hashBlock != 0) { + entry.push_back(Pair("blockhash", hashBlock.GetHex())); + BlockMap::iterator mi = mapBlockIndex.find(hashBlock); + if (mi != mapBlockIndex.end() && (*mi).second) { + CBlockIndex* pindex = (*mi).second; + if (chainActive.Contains(pindex)) { + entry.push_back(Pair("confirmations", 1 + chainActive.Height() - pindex->nHeight)); + entry.push_back(Pair("time", pindex->GetBlockTime())); + entry.push_back(Pair("blocktime", pindex->GetBlockTime())); + } + else + entry.push_back(Pair("confirmations", 0)); + } + } +} + +Value getrawtransaction(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) // MCHN + throw runtime_error("Help message not found\n"); + + uint256 hash = ParseHashV(params[0], "parameter 1"); + + bool fVerbose = false; + if (params.size() > 1) + fVerbose = (params[1].get_int() != 0); + + CTransaction tx; + uint256 hashBlock = 0; + if (!GetTransaction(hash, tx, hashBlock, true)) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "No information available about transaction"); + +/* MCHN START */ + CMutableTransaction txToShow=tx; + + if (GetBoolArg("-hideknownopdrops", false)) + { + for (unsigned int i = 0; i < txToShow.vout.size(); i++) + { + txToShow.vout[i].scriptPubKey=RemoveOpDropsIfNeeded(txToShow.vout[i].scriptPubKey); + } + } + + string strHex = EncodeHexTx(txToShow); +/* MCHN END */ + + + if (!fVerbose) + return strHex; + + Object result; + result.push_back(Pair("hex", strHex)); + TxToJSON(tx, hashBlock, result); + return result; +} + +#ifdef ENABLE_WALLET +Value listunspent(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 3) // MCHN + throw runtime_error("Help message not found\n"); + + RPCTypeCheck(params, list_of(int_type)(int_type)(array_type)); + + int64_t nMinDepth = 1; + if (params.size() > 0) + nMinDepth = params[0].get_int64(); + + int64_t nMaxDepth = 9999999; + if (params.size() > 1) + nMaxDepth = params[1].get_int64(); + + set setAddress; + if (params.size() > 2) { + Array inputs = params[2].get_array(); + BOOST_FOREACH(Value& input, inputs) { + CBitcoinAddress address(input.get_str()); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, string("Invalid address: ")+input.get_str()); + if (setAddress.count(address)) + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid parameter, duplicated address: ")+input.get_str()); + setAddress.insert(address); + } + } +/* MCHN START */ + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; + + { + LOCK(pwalletMain->cs_wallet); + pwalletMain->nNextUnspentOptimization=mc_TimeNowAsUInt()+GetArg("-autocombinesuspend", 15); + } + +/* MCHN END */ + Array results; + vector vecOutputs; + assert(pwalletMain != NULL); +// pwalletMain->AvailableCoins(vecOutputs, false); + pwalletMain->AvailableCoins(vecOutputs, false, NULL, true, true); + BOOST_FOREACH(const COutput& out, vecOutputs) { + if (out.nDepth < nMinDepth || out.nDepth > nMaxDepth) + continue; + +/* MCHN START */ + CTxOut txout; + uint256 hash=out.GetHashAndTxOut(txout); +/* MCHN END */ + + if (setAddress.size()) { + CTxDestination address; + if (!ExtractDestination(txout.scriptPubKey, address)) + continue; + + if (!setAddress.count(address)) + continue; + } + + CAmount nValue = txout.nValue; +/* MCHN START */ +// const CScript& pk = out.tx->vout[out.i].scriptPubKey; + const CScript& pk = RemoveOpDropsIfNeeded(txout.scriptPubKey); +/* MCHN END */ + Object entry; + entry.push_back(Pair("txid", hash.GetHex())); + entry.push_back(Pair("vout", out.i)); + CTxDestination address; + if (ExtractDestination(txout.scriptPubKey, address)) { + entry.push_back(Pair("address", CBitcoinAddress(address).ToString())); + if (pwalletMain->mapAddressBook.count(address)) + entry.push_back(Pair("account", pwalletMain->mapAddressBook[address].name)); + } + entry.push_back(Pair("scriptPubKey", HexStr(pk.begin(), pk.end()))); + if (pk.IsPayToScriptHash()) { + CTxDestination address; + if (ExtractDestination(pk, address)) { +/* MCHN START */ + // Bitcoin issue #6056 with booust 1.58 fixed in https://github.com/bitcoin/bitcoin/commit/8b08d9530b93c7a98e7387167ecd2cd5b0563bfb +// const CScriptID& hash = boost::get(address); + const CScriptID& hash = boost::get(address); +/* MCHN END */ + CScript redeemScript; + if (pwalletMain->GetCScript(hash, redeemScript)) + entry.push_back(Pair("redeemScript", HexStr(redeemScript.begin(), redeemScript.end()))); + } + } + entry.push_back(Pair("amount",ValueFromAmount(nValue))); + entry.push_back(Pair("confirmations",out.nDepth)); + +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(OutputCanSend(out)) + { + entry.push_back(Pair("cansend", true)); + entry.push_back(Pair("spendable", out.fSpendable)); + } + else + { + entry.push_back(Pair("cansend", false)); + entry.push_back(Pair("spendable", false)); + } + } + else + { + entry.push_back(Pair("spendable", out.fSpendable)); + } + + asset_amounts->Clear(); + if(CreateAssetBalanceList(txout,asset_amounts,lpScript)) + { + Array assets; + unsigned char *ptr; + const unsigned char *txid; + + for(int a=0;aGetCount();a++) + { + Object asset_entry; + ptr=(unsigned char *)asset_amounts->GetRow(a); + + txid=(unsigned char*)&hash; + if( (mc_GetABRefType(ptr) != MC_AST_ASSET_REF_TYPE_SPECIAL) && + (mc_GetABRefType(ptr) != MC_AST_ASSET_REF_TYPE_GENESIS) ) + { + mc_EntityDetails entity; + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ptr)) + { + txid=entity.GetTxID(); + } + } + + asset_entry=AssetEntry(txid,mc_GetABQuantity(ptr),1); + assets.push_back(asset_entry); + } + + entry.push_back(Pair("assets", assets)); + } + Array permissions=PermissionEntries(txout,lpScript,false); + entry.push_back(Pair("permissions", permissions)); +// entry.push_back(Pair("spendable", out.fSpendable)); +/* MCHN END */ + results.push_back(entry); + } + +/* MCHN START */ + + delete lpScript; + delete asset_amounts; + +/* MCHN END */ + return results; +} +#endif + +/* MCHN START */ +Value appendrawchange(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 3) + throw runtime_error("Help message not found\n"); + + // parse hex string from parameter + CTransaction tx; + if (!DecodeHexTx(tx, params[0].get_str())) + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "TX decode failed"); + + CBitcoinAddress address(params[1].get_str()); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + + bool fFeeAmountIsSet=false; + CAmount nFeeAmount = 0; + if (params.size() > 2 && params[2].type() != null_type) + { + nFeeAmount = AmountFromValue(params[2]); + fFeeAmountIsSet=true; + } + + Object result; + string strError=""; + CAmount nAmount; + + vector input_txouts; + vector input_errors; + + if(!GetTxInputsAsTxOuts(tx, input_txouts, input_errors,strError)) + { + if(strError.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + BOOST_FOREACH(const string& err, input_errors) + { + if(err.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, err); + } + } + } + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + mc_Buffer *amounts; + amounts=new mc_Buffer; + mc_InitABufferMap(amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; + + int allowed=0; + int required=0; + + nAmount=0; + asset_amounts->Clear(); + BOOST_FOREACH(const CTxOut& txout, tx.vout) + { + if(ParseMultichainTxOutToBuffer(0,txout,asset_amounts,lpScript,&allowed,&required,strError)) + { + nAmount+=txout.nValue; + } + } + + nAmount=-nAmount; + for(int i=0;iGetCount();i++) + { + uint64_t quantity=mc_GetABQuantity(asset_amounts->GetRow(i)); + quantity=-quantity; + mc_SetABQuantity(asset_amounts->GetRow(i),quantity); + } + + allowed=0; + required=0; + int i=0; + BOOST_FOREACH(const CTxOut& txout, input_txouts) + { + if(ParseMultichainTxOutToBuffer(tx.vin[i].prevout.hash,txout,asset_amounts,lpScript,&allowed,&required,strError)) + { + nAmount+=txout.nValue; + } + i++; + } + + if(nAmount < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Insufficient funds, output native currency value is higher than input"); + } + + amounts->Clear(); + for(int i=0;iGetCount();i++) + { + if(mc_GetABRefType(asset_amounts->GetRow(i)) == MC_AST_ASSET_REF_TYPE_GENESIS) +// if((int)mc_GetLE(asset_amounts->GetRow(i)+4,4)<0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Unconfirmed issue transaction in input"); + } + if(mc_GetABRefType(asset_amounts->GetRow(i)) != MC_AST_ASSET_REF_TYPE_SPECIAL) +// if((int)mc_GetLE(asset_amounts->GetRow(i),4)>0) + { + int64_t quantity=mc_GetABQuantity(asset_amounts->GetRow(i)); + if(quantity < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Insufficient funds, output asset value is higher than input"); + } + if(quantity > 0) + { + amounts->Add(asset_amounts->GetRow(i)); + } + } + } + + if(mc_gState->m_Features->VerifySizeOfOpDropElements()) + { + int assets_per_opdrop=(mc_gState->m_NetworkParams->GetInt64Param("maxstdopdropsize")-4)/(mc_gState->m_NetworkParams->m_AssetRefSize+MC_AST_ASSET_QUANTITY_SIZE); + + if(mc_gState->m_Features->VerifySizeOfOpDropElements()) + { + assets_per_opdrop=(mc_gState->m_NetworkParams->GetInt64Param("maxstdelementsize")-4)/(mc_gState->m_NetworkParams->m_AssetRefSize+MC_AST_ASSET_QUANTITY_SIZE); + } + + if(amounts->GetCount() > assets_per_opdrop) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strprintf("Too many assets, maximal number for this chain - %d",assets_per_opdrop)); + } + } + + CScript scriptChange=GetScriptForDestination(address.Get()); + + size_t elem_size; + const unsigned char *elem; + lpScript->Clear(); + + if(amounts->GetCount()) + { + lpScript->SetAssetQuantities(amounts,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER); + for(int element=0;element < lpScript->GetNumElements();element++) + { + elem = lpScript->GetData(element,&elem_size); + if(elem) + { + scriptChange << vector(elem, elem + elem_size) << OP_DROP; + } + else + { + throw JSONRPCError(RPC_INTERNAL_ERROR, "Internal error: cannot create asset transfer script"); + } + } + } + + + CAmount change_amount; + CAmount min_output=-1; // Calculate minimal output for the change + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + min_output=mc_gState->m_NetworkParams->GetInt64Param("minimumperoutput"); + } + if(min_output<0) + { + min_output=3*(::minRelayTxFee.GetFee(GetSerializeSize(SER_DISK,0)+148u)); + } + + CMutableTransaction txNew; + + if(nAmount < nFeeAmount) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Insufficient funds, remaining native currency value is lower than required fee"); + } + + txNew.vin.clear(); + txNew.vout.clear(); + + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + txNew.vin.push_back(txin); + } + BOOST_FOREACH(const CTxOut& txout, tx.vout) + { + txNew.vout.push_back(txout); + } + + if( (nAmount >= nFeeAmount) || (amounts->GetCount() > 0) ) + { + change_amount=nAmount-nFeeAmount; + if(change_amountGetCount() > 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Insufficient funds, remaining native currency value is lower than required fee + change amount"); + } + } + else + { + CTxOut txout(change_amount, scriptChange); + txNew.vout.push_back(txout); + if(!fFeeAmountIsSet) + { + minRelayTxFee = CFeeRate(MIN_RELAY_TX_FEE); + unsigned int nBytes = ::GetSerializeSize(txNew, SER_NETWORK, PROTOCOL_VERSION); + nBytes+=txNew.vin.size()*108; + nBytes=((nBytes-1)/1000+1)*1000; + nFeeAmount=::GetMinRelayFee(tx,nBytes,false); + if(change_amount != (nAmount-nFeeAmount)) + { + change_amount=nAmount-nFeeAmount; + if(change_amountGetCount() > 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Insufficient funds, remaining native currency value is lower than required fee + change amount"); + } + else + { + txNew.vout.pop_back(); + } + } + else + { + txNew.vout.back().nValue=change_amount; + } + } + } + } + } + + delete lpScript; + delete asset_amounts; + delete amounts; + + + return EncodeHexTx(txNew); +} + + +void AddCacheInputScriptIfNeeded(CMutableTransaction& rawTx,Array inputs, bool fRequired) +{ + vector script_for_cache; + vector cache_array; + bool fMissingScript=false; + + BOOST_FOREACH(const Value& input, inputs) { + const Object& o = input.get_obj(); + +// uint256 txid = ParseHashO(o, "txid"); + + const Value& vout_v = find_value(o, "vout"); + if (vout_v.type() != int_type) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter, missing vout key"); + int nOutput = vout_v.get_int(); + if (nOutput < 0) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter, vout must be positive"); + + const Value& scriptPubKeyString = find_value(o, "scriptPubKey"); + + if(!scriptPubKeyString.is_null()) + { + bool fIsHex; + if( (scriptPubKeyString.type() != str_type) || (scriptPubKeyString.get_str().size() == 0) ) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter, scriptPubKey must be not empty string"); + } + + vector dataData(ParseHex(scriptPubKeyString.get_str().c_str(),fIsHex)); + if(!fIsHex) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter, scriptPubKey must be hexadecimal string"); + } + CScript scriptPubKey(dataData.begin(), dataData.end()); + + script_for_cache.push_back(scriptPubKey); + } + else + { + script_for_cache.push_back(CScript()); + } + + const Value& cache_this = find_value(o, "cache"); + int cache_value=-1; + if(!cache_this.is_null()) + { + cache_value=0; + if(cache_this.type() != bool_type) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter, cache must be boolean"); + } + if(mc_gState->m_Features->CachedInputScript()) + { + if(cache_this.get_bool()) + { + if(scriptPubKeyString.is_null()) + { + fMissingScript=true; + } + cache_value=1; + } + } + } + cache_array.push_back(cache_value); + } + + if(fRequired) + { + if(!fMissingScript) + { + for(int i=0;i<(int)script_for_cache.size();i++) + { + if(script_for_cache[i].size() == 0) + { + fMissingScript=true; + } + } + } + } + + if(fMissingScript) + { + CCoinsView viewDummy; + CCoinsViewCache view(&viewDummy); + { + LOCK(mempool.cs); + CCoinsViewCache &viewChain = *pcoinsTip; + CCoinsViewMemPool viewMempool(&viewChain, mempool); + view.SetBackend(viewMempool); // temporarily switch cache backend to db+mempool view + + BOOST_FOREACH(const CTxIn& txin, rawTx.vin) { + const uint256& prevHash = txin.prevout.hash; + CCoins coins; + view.AccessCoins(prevHash); // this is certainly allowed to fail + } + + view.SetBackend(viewDummy); // switch back to avoid locking mempool for too long + } + for(int i=0;i<(int)script_for_cache.size();i++) + { + if(script_for_cache[i].size() == 0) + { + const Object& o = inputs[i].get_obj(); + uint256 txid = ParseHashO(o, "txid"); + int vout_v = find_value(o, "vout").get_int(); + + CCoinsModifier coins = view.ModifyCoins(txid); + if (coins->IsAvailable(vout_v)) + { + script_for_cache[i]=coins->vout[vout_v].scriptPubKey; + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMS, "Previous output scriptPubKey not found"); + } + } + } + } + + bool fNewOutputs=false; + + for(int i=0;i<(int)script_for_cache.size();i++) + { + if( (cache_array[i] < 0) && fRequired) + { + CTxDestination addressRet; + if(ExtractDestinationScriptValid(script_for_cache[i], addressRet)) + { + const unsigned char *aptr; + aptr=GetAddressIDPtr(addressRet); + if(aptr) + { + if(mc_gState->m_Permissions->CanAdmin(NULL,aptr)) + { + cache_array[i]=1; + } + } + } + } + if(cache_array[i] == 1) + { + fNewOutputs=true; + } + } + + int cs_offset; + cs_offset=0; + cache_array.push_back(1); + + if(fNewOutputs) + { + mc_Script *lpDetails; + lpDetails=new mc_Script; + + lpDetails->Clear(); + for(int i=0;i<(int)cache_array.size();i++) + { + if(cache_array[i] == 1) + { + if( (cs_offset+9+script_for_cache[i].size() > MAX_SCRIPT_ELEMENT_SIZE) || (i == (int)cache_array.size() -1) ) + { + if(cs_offset>0) + { + size_t bytes; + const unsigned char *script; + CScript scriptOpReturn=CScript(); + + script=lpDetails->GetData(0,&bytes); + scriptOpReturn << vector(script, script + bytes) << OP_DROP << OP_RETURN; + CTxOut txout(0,scriptOpReturn); + rawTx.vout.push_back(txout); + lpDetails->Clear(); + } + } + + if(i < (int)cache_array.size() - 1) + { + if(cs_offset == 0) + { + lpDetails->SetCachedScript(cs_offset,&cs_offset,-1,NULL,-1); + } + const CScript& script3 = script_for_cache[i]; + CScript::const_iterator pc3 = script3.begin(); + + lpDetails->SetCachedScript(cs_offset,&cs_offset,i,(unsigned char*)&pc3[0],script3.size()); + } + } + } + } +} + + +/* MCHN END */ + +Value createrawtransaction(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 4) // MCHN + throw runtime_error("Help message not found\n"); + + RPCTypeCheck(params, list_of(array_type)(obj_type)); + + Array inputs = params[0].get_array(); + Object sendTo = params[1].get_obj(); + Array inputs_for_signature; + + CMutableTransaction rawTx; + + BOOST_FOREACH(const Value& input, inputs) { + const Object& o = input.get_obj(); + + uint256 txid = ParseHashO(o, "txid"); + + const Value& vout_v = find_value(o, "vout"); + if (vout_v.type() != int_type) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter, missing vout key"); + int nOutput = vout_v.get_int(); + if (nOutput < 0) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter, vout must be positive"); + + CTxIn in(COutPoint(txid, nOutput)); + rawTx.vin.push_back(in); + + const Value& scriptPubKey = find_value(o, "scriptPubKey"); + if(!scriptPubKey.is_null()) + { + inputs_for_signature.push_back(input); + } + } + + vector > vecSend; + int required=0; + bool fCachedInputScriptRequired=false; + vecSend=ParseRawOutputMultiObject(sendTo,&required); + BOOST_FOREACH (const PAIRTYPE(CScript, CAmount)& s, vecSend) + { + CTxOut out(s.second, s.first); + rawTx.vout.push_back(out); + } + + if( required & (MC_PTP_ADMIN | MC_PTP_MINE) ) + { + if(mc_gState->m_Features->CachedInputScript()) + { + if(mc_gState->m_NetworkParams->GetInt64Param("supportminerprecheck")) + { + fCachedInputScriptRequired=true; + } + } + } + + AddCacheInputScriptIfNeeded(rawTx,inputs,fCachedInputScriptRequired); + + mc_EntityDetails entity; + mc_gState->m_TmpAssetsOut->Clear(); + for(int i=0;i<(int)rawTx.vout.size();i++) + { + FindFollowOnsInScript(rawTx.vout[i].scriptPubKey,mc_gState->m_TmpAssetsOut,mc_gState->m_TmpScript); + } + entity.Zero(); + if(mc_gState->m_TmpAssetsOut->GetCount()) + { +/* + if(mc_gState->m_TmpAssetsOut->GetCount() > 1) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Follow-on script rejected - follow-on for several assets"); + } + */ + if(!mc_gState->m_Assets->FindEntityByFullRef(&entity,mc_gState->m_TmpAssetsOut->GetRow(0))) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Follow-on script rejected - asset not found"); + } + } + + + if (params.size() > 2 && params[2].type() != null_type) + { + BOOST_FOREACH(const Value& data, params[2].get_array()) + { + CScript scriptOpReturn=ParseRawMetadata(data,0xFFFF,&entity,NULL); + CTxOut out(0, scriptOpReturn); + rawTx.vout.push_back(out); + } + } + + string action=""; + string hex=EncodeHexTx(rawTx); + Value signedTx; + Value txid; + bool sign_it=false; + bool lock_it=false; + bool send_it=false; + if (params.size() > 3 && params[3].type() != null_type) + { + ParseRawAction(params[3].get_str(),lock_it,sign_it,send_it); + } + + if(sign_it) + { + Array signrawtransaction_params; + signrawtransaction_params.push_back(hex); + if(inputs_for_signature.size()) + { + signrawtransaction_params.push_back(inputs_for_signature); + } + signedTx=signrawtransaction(signrawtransaction_params,false); + } + if(lock_it) + { + BOOST_FOREACH(const CTxIn& txin, rawTx.vin) + { + COutPoint outpt(txin.prevout.hash, txin.prevout.n); + pwalletMain->LockCoin(outpt); + } + } + if(send_it) + { + Array sendrawtransaction_params; + BOOST_FOREACH(const Pair& s, signedTx.get_obj()) + { + if(s.name_=="complete") + { + if(!s.value_.get_bool()) + { + throw JSONRPCError(RPC_TRANSACTION_ERROR, "Transaction was not signed properly"); + } + } + if(s.name_=="hex") + { + sendrawtransaction_params.push_back(s.value_.get_str()); + } + } + txid=sendrawtransaction(sendrawtransaction_params,false); + } + + if(send_it) + { + return txid; + } + + if(sign_it) + { + return signedTx; + } + + return hex; +} + +/* MCHN START */ + +Value appendrawmetadata(const json_spirit::Array& params, bool fHelp) +{ + if (fHelp || params.size() != 2 ) + throw runtime_error("Help message not found\n"); + + CMutableTransaction tx; + + vector txData(ParseHex(params[0].get_str())); + + CDataStream ssData(txData, SER_NETWORK, PROTOCOL_VERSION); + ssData >> tx; + + mc_EntityDetails entity; + mc_gState->m_TmpAssetsOut->Clear(); + for(int i=0;i<(int)tx.vout.size();i++) + { + FindFollowOnsInScript(tx.vout[i].scriptPubKey,mc_gState->m_TmpAssetsOut,mc_gState->m_TmpScript); + } + entity.Zero(); + if(mc_gState->m_TmpAssetsOut->GetCount()) + { +/* + if(mc_gState->m_TmpAssetsOut->GetCount() > 1) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Follow-on script rejected - follow-on for several assets"); + } + */ + if(!mc_gState->m_Assets->FindEntityByFullRef(&entity,mc_gState->m_TmpAssetsOut->GetRow(0))) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Follow-on script rejected - asset not found"); + } + } + + CScript scriptOpReturn=ParseRawMetadata(params[1],0xFFFF,&entity,NULL); + + CTxOut txout(0, scriptOpReturn); + tx.vout.push_back(txout); + + return EncodeHexTx(tx); +} + +Value disablerawtransaction(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + RPCTypeCheck(params, list_of(str_type)); + + if( (mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) || + ( (pwalletMain->lpAssetGroups == NULL) || (pwalletMain->lpAssetGroups == 0) ) ) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "disablerawtransaction supported only for protocol=multichain"); + } + + CTransaction tx; + + if (!DecodeHexTx(tx, params[0].get_str())) + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "TX decode failed"); + + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + bool input_found=false; + bool permission_found=false; + string strLastError; + + for(int i=0;i<(int)tx.vin.size();i++) + { + string strError; + CTxOut preparedTxOut; + if(FindPreparedTxOut(preparedTxOut,tx.vin[i].prevout,strError)) + { + if( (pwalletMain->IsMine(preparedTxOut) & ISMINE_SPENDABLE) != ISMINE_NO ) + { + bool take_it=true; + CWalletTx wtx; + vector scriptPubKeys; + CReserveKey reservekey(pwalletMain); + CAmount nFeeRequired; + string strError; + set thisFromAddresses; + set *lpFromAddresses; + CScript scriptOpReturn; + vector vCoinsToUse; + COutPoint outpt=tx.vin[i].prevout; + + bool wasLocked=pwalletMain->IsLockedCoin(outpt.hash,outpt.n); + bool wasUnlocked=false; + + input_found=true; + vCoinsToUse.push_back(outpt); + scriptPubKeys.push_back(preparedTxOut.scriptPubKey); + + const CScript& script1 = preparedTxOut.scriptPubKey; + CTxDestination addressRet; + if(!ExtractDestination(script1, addressRet)) + { + take_it=false; + } + + CKeyID *lpKeyID=boost::get (&addressRet); + if(lpKeyID == NULL) + { + take_it=false; + } + + if(mc_gState->m_Permissions->CanSend(NULL,(unsigned char*)(lpKeyID)) == 0) + { + take_it=false; + } + + if(mc_gState->m_Permissions->CanReceive(NULL,(unsigned char*)(lpKeyID)) == 0) + { + take_it=false; + } + + + if(take_it) + { + permission_found=true; + thisFromAddresses.insert(addressRet); + + lpFromAddresses=NULL; + if(thisFromAddresses.size()) + { + lpFromAddresses=&thisFromAddresses; + } + + if(wasLocked) + { + pwalletMain->UnlockCoin(outpt); + wasUnlocked=true; + } + if (!pwalletMain->CreateTransaction(scriptPubKeys, preparedTxOut.nValue, scriptOpReturn, wtx, reservekey, nFeeRequired, strError, NULL, lpFromAddresses,1,-1,-1,&vCoinsToUse)) + { + take_it=false; + if (preparedTxOut.nValue + nFeeRequired > pwalletMain->GetBalance()) + { +// insufficient_fee=true; + strLastError=strprintf("Error: This transaction requires a transaction fee of at least %s because of its amount, complexity, or use of recently received funds!", FormatMoney(nFeeRequired));; + } + else + { + strLastError=strError; + } + } + } + + if(take_it) + { + string strRejectReason; + if (!pwalletMain->CommitTransaction(wtx, reservekey, strRejectReason)) + { + if(strLastError.size() == 0 ) + { + strLastError=strRejectReason; + } + take_it=false; + } + } + + if(take_it) + { + return wtx.GetHash().GetHex(); + } + else + { + if(wasUnlocked) + { + pwalletMain->LockCoin(outpt); + wasUnlocked=false; + } + } + } + } + } + + if(input_found) + { + if(!permission_found) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "At least one of the input addresses should have send and receive permission "); + } + else + { + throw JSONRPCError(RPC_INVALID_REQUEST, strLastError); + } + } + + throw JSONRPCError(RPC_INVALID_REQUEST, "Couldn't find unspent input in this tx belonging to this wallet."); +} + +/* MCHN END */ + + +Value decoderawtransaction(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + RPCTypeCheck(params, list_of(str_type)); + + CTransaction tx; + + if (!DecodeHexTx(tx, params[0].get_str())) + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "TX decode failed"); + + Object result; + TxToJSON(tx, 0, result); + + return result; +} + +Value decodescript(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) // MCHN + throw runtime_error("Help message not found\n"); + + RPCTypeCheck(params, list_of(str_type)); + + Object r; + CScript script; + if (params[0].get_str().size() > 0){ + vector scriptData(ParseHexV(params[0], "argument")); + script = CScript(scriptData.begin(), scriptData.end()); + } else { + // Empty scripts are valid + } + ScriptPubKeyToJSON(script, r, false); + + r.push_back(Pair("p2sh", CBitcoinAddress(CScriptID(script)).ToString())); + return r; +} + +Value signrawtransaction(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 4) + throw runtime_error("Help message not found\n"); + + RPCTypeCheck(params, list_of(str_type)(array_type)(array_type)(str_type), true); + + vector txData(ParseHexV(params[0], "argument 1")); + CDataStream ssData(txData, SER_NETWORK, PROTOCOL_VERSION); + vector txVariants; + while (!ssData.empty()) { + try { + CMutableTransaction tx; + ssData >> tx; + txVariants.push_back(tx); + } + catch (const std::exception &) { + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "TX decode failed"); + } + } + + if (txVariants.empty()) + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "Missing transaction"); + + // mergedTx will end up with all the signatures; it + // starts as a clone of the rawtx: + CMutableTransaction mergedTx(txVariants[0]); + bool fComplete = true; + + // Fetch previous transactions (inputs): + CCoinsView viewDummy; + CCoinsViewCache view(&viewDummy); + { + LOCK(mempool.cs); + CCoinsViewCache &viewChain = *pcoinsTip; + CCoinsViewMemPool viewMempool(&viewChain, mempool); + view.SetBackend(viewMempool); // temporarily switch cache backend to db+mempool view + + BOOST_FOREACH(const CTxIn& txin, mergedTx.vin) { + const uint256& prevHash = txin.prevout.hash; + CCoins coins; + view.AccessCoins(prevHash); // this is certainly allowed to fail + } + + view.SetBackend(viewDummy); // switch back to avoid locking mempool for too long + } + + bool fGivenKeys = false; + CBasicKeyStore tempKeystore; + if (params.size() > 2 && params[2].type() != null_type) { + fGivenKeys = true; + Array keys = params[2].get_array(); + BOOST_FOREACH(Value k, keys) { + CBitcoinSecret vchSecret; + bool fGood = vchSecret.SetString(k.get_str()); + if (!fGood) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid private key"); + CKey key = vchSecret.GetKey(); + if (!key.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key outside allowed range"); + tempKeystore.AddKey(key); + } + } +#ifdef ENABLE_WALLET + else + EnsureWalletIsUnlocked(); +#endif + + // Add previous txouts given in the RPC call: + if (params.size() > 1 && params[1].type() != null_type) { + Array prevTxs = params[1].get_array(); + BOOST_FOREACH(Value& p, prevTxs) { + if (p.type() != obj_type) + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "expected object with {\"txid'\",\"vout\",\"scriptPubKey\"}"); + + Object prevOut = p.get_obj(); + + RPCTypeCheck(prevOut, map_list_of("txid", str_type)("vout", int_type)("scriptPubKey", str_type)); + + uint256 txid = ParseHashO(prevOut, "txid"); + + int nOut = find_value(prevOut, "vout").get_int(); + if (nOut < 0) + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "vout must be positive"); + + vector pkData(ParseHexO(prevOut, "scriptPubKey")); + CScript scriptPubKey(pkData.begin(), pkData.end()); + + { + CCoinsModifier coins = view.ModifyCoins(txid); + if (coins->IsAvailable(nOut) && coins->vout[nOut].scriptPubKey != scriptPubKey) { + string err("Previous output scriptPubKey mismatch:\n"); + err = err + coins->vout[nOut].scriptPubKey.ToString() + "\nvs:\n"+ + scriptPubKey.ToString(); + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, err); + } + if ((unsigned int)nOut >= coins->vout.size()) + coins->vout.resize(nOut+1); + coins->vout[nOut].scriptPubKey = scriptPubKey; + coins->vout[nOut].nValue = 0; // we don't know the actual output value + } + + // if redeemScript given and not using the local wallet (private keys + // given), add redeemScript to the tempKeystore so it can be signed: + if (fGivenKeys && scriptPubKey.IsPayToScriptHash()) { + RPCTypeCheck(prevOut, map_list_of("txid", str_type)("vout", int_type)("scriptPubKey", str_type)("redeemScript",str_type)); + Value v = find_value(prevOut, "redeemScript"); + if (!(v == Value::null)) { + vector rsData(ParseHexV(v, "redeemScript")); + CScript redeemScript(rsData.begin(), rsData.end()); + tempKeystore.AddCScript(redeemScript); + } + } + } + } + +#ifdef ENABLE_WALLET + const CKeyStore& keystore = ((fGivenKeys || !pwalletMain) ? tempKeystore : *pwalletMain); +#else + const CKeyStore& keystore = tempKeystore; +#endif + + int nHashType = SIGHASH_ALL; + if (params.size() > 3 && params[3].type() != null_type) { + static map mapSigHashValues = + boost::assign::map_list_of + (string("ALL"), int(SIGHASH_ALL)) + (string("ALL|ANYONECANPAY"), int(SIGHASH_ALL|SIGHASH_ANYONECANPAY)) + (string("NONE"), int(SIGHASH_NONE)) + (string("NONE|ANYONECANPAY"), int(SIGHASH_NONE|SIGHASH_ANYONECANPAY)) + (string("SINGLE"), int(SIGHASH_SINGLE)) + (string("SINGLE|ANYONECANPAY"), int(SIGHASH_SINGLE|SIGHASH_ANYONECANPAY)) + ; + string strHashType = params[3].get_str(); + if (mapSigHashValues.count(strHashType)) + nHashType = mapSigHashValues[strHashType]; + else + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid sighash param"); + } + + bool fHashSingle = ((nHashType & ~SIGHASH_ANYONECANPAY) == SIGHASH_SINGLE); + + // Sign what we can: + for (unsigned int i = 0; i < mergedTx.vin.size(); i++) { + CTxIn& txin = mergedTx.vin[i]; + const CCoins* coins = view.AccessCoins(txin.prevout.hash); + if (coins == NULL || !coins->IsAvailable(txin.prevout.n)) { + fComplete = false; + continue; + } + const CScript& prevPubKey = coins->vout[txin.prevout.n].scriptPubKey; + + txin.scriptSig.clear(); + // Only sign SIGHASH_SINGLE if there's a corresponding output: + if (!fHashSingle || (i < mergedTx.vout.size())) + SignSignature(keystore, prevPubKey, mergedTx, i, nHashType); + + // ... and merge in other signatures: + BOOST_FOREACH(const CMutableTransaction& txv, txVariants) { + txin.scriptSig = CombineSignatures(prevPubKey, mergedTx, i, txin.scriptSig, txv.vin[i].scriptSig); + } + if (!VerifyScript(txin.scriptSig, prevPubKey, STANDARD_SCRIPT_VERIFY_FLAGS, MutableTransactionSignatureChecker(&mergedTx, i))) + fComplete = false; + } + + Object result; + result.push_back(Pair("hex", EncodeHexTx(mergedTx))); + result.push_back(Pair("complete", fComplete)); + + return result; +} + +Value sendrawtransaction(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) + throw runtime_error("Help message not found\n"); + + RPCTypeCheck(params, list_of(str_type)(bool_type)); + + // parse hex string from parameter + CTransaction tx; + if (!DecodeHexTx(tx, params[0].get_str())) + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "TX decode failed"); + uint256 hashTx = tx.GetHash(); + + bool fOverrideFees = false; + if (params.size() > 1) + fOverrideFees = params[1].get_bool(); + + CCoinsViewCache &view = *pcoinsTip; + const CCoins* existingCoins = view.AccessCoins(hashTx); + bool fHaveMempool = mempool.exists(hashTx); + bool fHaveChain = existingCoins && existingCoins->nHeight < 1000000000; + if (!fHaveMempool && !fHaveChain) { + // push to local node and sync with wallets + CValidationState state; +/* MCHN START */ + CPubKey pubkey; + uint32_t fCanMine=((pwalletMain != NULL) && pwalletMain->GetKeyFromAddressBook(pubkey,MC_PTP_MINE)) ? MC_PTP_MINE : 0; +/* MCHN END */ + if (!AcceptToMemoryPool(mempool, state, tx, false, NULL, !fOverrideFees)) { + if(state.IsInvalid()) + throw JSONRPCError(RPC_TRANSACTION_REJECTED, strprintf("%i: %s", state.GetRejectCode(), state.GetRejectReason())); + else +/* MCHN START */ + { + if(state.GetRejectReason().size()) + { + throw JSONRPCError(RPC_TRANSACTION_ERROR, state.GetRejectReason()); + } + else + { + if(!mempool.exists(hashTx)) + { + throw JSONRPCError(RPC_TRANSACTION_ALREADY_IN_CHAIN, "transaction already in block chain"); + } + } + } +/* MCHN START */ + } +/* MCHN START */ + if(pwalletMain) + { + if(fCanMine) + { + if(!pwalletMain->GetKeyFromAddressBook(pubkey,MC_PTP_MINE)) + { + LogPrint("mchn","mchn: Wallet lost mine permission on tx: %s (height %d) - sendrawtransaction, reactivating best chain\n", + tx.GetHash().ToString().c_str(), chainActive.Tip()->nHeight); + if (!ActivateBestChain(state, NULL)) + throw JSONRPCError(RPC_TRANSACTION_ERROR, state.GetRejectReason()); + } + } + } +/* MCHN END */ + + } else if (fHaveChain) { + throw JSONRPCError(RPC_TRANSACTION_ALREADY_IN_CHAIN, "transaction already in block chain"); + } + + RelayTransaction(tx); + + return hashTx.GetHex(); +} diff --git a/src/rpc/rpcserver.cpp b/src/rpc/rpcserver.cpp new file mode 100644 index 00000000..bca8ea91 --- /dev/null +++ b/src/rpc/rpcserver.cpp @@ -0,0 +1,1394 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "rpc/rpcserver.h" + +#include "structs/base58.h" +#include "core/init.h" +#include "core/main.h" +#include "ui/ui_interface.h" +#include "utils/util.h" +#ifdef ENABLE_WALLET +#include "wallet/wallet.h" +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "json/json_spirit_writer_template.h" + +using namespace boost; +using namespace boost::asio; +using namespace json_spirit; +using namespace std; + +static std::string strRPCUserColonPass; + +static bool fRPCRunning = false; +static bool fRPCInWarmup = true; +static std::string rpcWarmupStatus("RPC server started"); +static CCriticalSection cs_rpcWarmup; + +//! These are created by StartRPCThreads, destroyed in StopRPCThreads +static asio::io_service* rpc_io_service = NULL; +static map > deadlineTimers; +static ssl::context* rpc_ssl_context = NULL; +static boost::thread_group* rpc_worker_group = NULL; +static boost::asio::io_service::work *rpc_dummy_work = NULL; +static std::vector rpc_allow_subnets; //!< List of subnets to allow RPC connections from +static std::vector< boost::shared_ptr > rpc_acceptors; + +void RPCTypeCheck(const Array& params, + const list& typesExpected, + bool fAllowNull) +{ + unsigned int i = 0; + BOOST_FOREACH(Value_type t, typesExpected) + { + if (params.size() <= i) + break; + + const Value& v = params[i]; + if (!((v.type() == t) || (fAllowNull && (v.type() == null_type)))) + { + string err = strprintf("Expected type %s, got %s", + Value_type_name[t], Value_type_name[v.type()]); + throw JSONRPCError(RPC_TYPE_ERROR, err); + } + i++; + } +} + +void RPCTypeCheck(const Object& o, + const map& typesExpected, + bool fAllowNull) +{ + BOOST_FOREACH(const PAIRTYPE(string, Value_type)& t, typesExpected) + { + const Value& v = find_value(o, t.first); + if (!fAllowNull && v.type() == null_type) + throw JSONRPCError(RPC_TYPE_ERROR, strprintf("Missing %s", t.first)); + + if (!((v.type() == t.second) || (fAllowNull && (v.type() == null_type)))) + { + string err = strprintf("Expected type %s for %s, got %s", + Value_type_name[t.second], t.first, Value_type_name[v.type()]); + throw JSONRPCError(RPC_TYPE_ERROR, err); + } + } +} + +static inline int64_t roundint64(double d) +{ + return (int64_t)(d > 0 ? d + 0.5 : d - 0.5); +} + +CAmount AmountFromValue(const Value& value) +{ + double dAmount = value.get_real(); +/* MCHN START */ + if(COIN == 0) + { + if(dAmount != 0) + { + throw JSONRPCError(RPC_TYPE_ERROR, "Invalid amount"); + } + } + else + { + if (dAmount < 0.0 || dAmount > (double)MAX_MONEY/(double)COIN) // MCHN - was <= + throw JSONRPCError(RPC_TYPE_ERROR, "Invalid amount"); + } + +// if (dAmount < 0.0 || dAmount > 21000000.0) // MCHN - was <= +// throw JSONRPCError(RPC_TYPE_ERROR, "Invalid amount"); +/* MCHN END */ + CAmount nAmount = roundint64(dAmount * COIN); + if (!MoneyRange(nAmount)) + throw JSONRPCError(RPC_TYPE_ERROR, "Invalid amount"); + return nAmount; +} + +Value ValueFromAmount(const CAmount& amount) +{ +/* MCHN START */ + if(COIN == 0) + { + return (double)amount; + } +/* MCHN END */ + return (double)amount / (double)COIN; +} + +uint256 ParseHashV(const Value& v, string strName) +{ + string strHex; + if (v.type() == str_type) + strHex = v.get_str(); + if (!IsHex(strHex)) // Note: IsHex("") is false + throw JSONRPCError(RPC_INVALID_PARAMETER, strName+" must be hexadecimal string (not '"+strHex+"')"); + uint256 result; + result.SetHex(strHex); + return result; +} +uint256 ParseHashO(const Object& o, string strKey) +{ + return ParseHashV(find_value(o, strKey), strKey); +} +vector ParseHexV(const Value& v, string strName) +{ + string strHex; + if (v.type() == str_type) + strHex = v.get_str(); + if (!IsHex(strHex)) + throw JSONRPCError(RPC_INVALID_PARAMETER, strName+" must be hexadecimal string (not '"+strHex+"')"); + return ParseHex(strHex); +} +vector ParseHexO(const Object& o, string strKey) +{ + return ParseHexV(find_value(o, strKey), strKey); +} + + +/** + * Note: This interface may still be subject to change. + */ + +string CRPCTable::help(string strCommand) const +{ + string strRet; + string category; + set setDone; + vector > vCommands; + + for (map::const_iterator mi = mapCommands.begin(); mi != mapCommands.end(); ++mi) + vCommands.push_back(make_pair(mi->second->category + mi->first, mi->second)); + sort(vCommands.begin(), vCommands.end()); + + BOOST_FOREACH(const PAIRTYPE(string, const CRPCCommand*)& command, vCommands) + { + const CRPCCommand *pcmd = command.second; + string strMethod = pcmd->name; + // We already filter duplicates, but these deprecated screw up the sort order + if (strMethod.find("label") != string::npos) + continue; + if ((strCommand != "" || pcmd->category == "hidden") && strMethod != strCommand) + continue; +#ifdef ENABLE_WALLET + if (pcmd->reqWallet && !pwalletMain) + continue; +#endif + +/* MCHN START */ + string strHelp=""; + map::iterator it = mapHelpStrings.find(strMethod); + if (it == mapHelpStrings.end()) + { + try + { + Array params; + rpcfn_type pfn = pcmd->actor; + if (setDone.insert(pfn).second) + (*pfn)(params, true); + } + catch (std::exception& e) + { + strHelp = string(e.what()); + } + } + else + { + strHelp=it->second; + } + + if(strHelp != "") + { + if (strCommand == "") + { + if (strHelp.find('\n') != string::npos) + strHelp = strHelp.substr(0, strHelp.find('\n')); + + if (category != pcmd->category) + { + if (!category.empty()) + strRet += "\n"; + category = pcmd->category; + string firstLetter = category.substr(0,1); + boost::to_upper(firstLetter); + strRet += "== " + firstLetter + category.substr(1) + " ==\n"; + } + } + strRet += strHelp + "\n"; + } +/* + try + { + Array params; + rpcfn_type pfn = pcmd->actor; + if (setDone.insert(pfn).second) + (*pfn)(params, true); + } + catch (std::exception& e) + { + // Help text is returned in an exception + string strHelp = string(e.what()); + if (strCommand == "") + { + if (strHelp.find('\n') != string::npos) + strHelp = strHelp.substr(0, strHelp.find('\n')); + + if (category != pcmd->category) + { + if (!category.empty()) + strRet += "\n"; + category = pcmd->category; + string firstLetter = category.substr(0,1); + boost::to_upper(firstLetter); + strRet += "== " + firstLetter + category.substr(1) + " ==\n"; + } + } + strRet += strHelp + "\n"; + } + */ + } +/* MCHN END */ + if (strRet == "") + strRet = strprintf("help: unknown command: %s\n", strCommand); + strRet = strRet.substr(0,strRet.size()-1); + return strRet; +} + +Value help(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 1) + throw runtime_error("Help message not found\n"); + + string strCommand; + if (params.size() > 0) + strCommand = params[0].get_str(); + + return tableRPC.help(strCommand); +} + + +Value stop(const Array& params, bool fHelp) +{ + // Accept the deprecated and ignored 'detach' boolean argument + if (fHelp || params.size() > 1) + throw runtime_error("Help message not found\n"); + // Shutdown will take long enough that the response should get back + StartShutdown(); + return "MultiChain server stopping"; +} + +/* MCHN START */ + +string AllowedPausedServices() +{ + string ret="incoming,mining"; + + return ret; +} + +uint32_t GetPausedServices(const char *str) +{ + uint32_t result,type; + char* ptr; + char* start; + char* ptrEnd; + char c; + + ptr=(char*)str; + ptrEnd=ptr+strlen(ptr); + start=ptr; + + result=0; + + while(ptr<=ptrEnd) + { + c=*ptr; + if( (c == ',') || (c ==0x00)) + { + if(ptr > start) + { + type=0; + if(memcmp(start,"incoming", ptr-start) == 0)type = MC_NPS_INCOMING; + if(memcmp(start,"mining", ptr-start) == 0)type = MC_NPS_MINING; + + if(type == 0) + { + return 0; + } + result |= type; + start=ptr+1; + } + } + ptr++; + } + + return result; +} + + +Value pausecmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 1) + throw runtime_error("Help message not found\n"); + + uint32_t type=0; + if (params.size() > 0 && params[0].type() != null_type && !params[0].get_str().empty()) + { + type=GetPausedServices(params[0].get_str().c_str()); + } + + if(type == 0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid task"); + + LOCK(cs_main); + + mc_gState->m_NodePausedState |= type; + + LogPrintf("Node paused state is set to %08X\n",mc_gState->m_NodePausedState); + + return "Paused"; +} + +Value resumecmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 1) + throw runtime_error("Help message not found\n"); + + uint32_t type=0; + if (params.size() > 0 && params[0].type() != null_type && !params[0].get_str().empty()) + { + type=GetPausedServices(params[0].get_str().c_str()); + } + + if(type == 0) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid task"); + + LOCK(cs_main); + + mc_gState->m_NodePausedState &= (MC_NPS_ALL ^ type); + + LogPrintf("Node paused state is set to %08X\n",mc_gState->m_NodePausedState); + + return "Resumed"; +} + +/* END */ + +/* MCHN START */ +static const CRPCCommand vRPCWalletReadCommands[] = +{ // category name actor (function) okSafeMode threadSafe reqWallet + // --------------------- ------------------------ ----------------------- ---------- ---------- --------- + { "wallet", "getbalance", &getbalance, false, false, true }, + { "wallet", "getreceivedbyaccount", &getreceivedbyaccount, false, false, true }, + { "wallet", "getreceivedbyaddress", &getreceivedbyaddress, false, false, true }, + { "wallet", "gettransaction", &gettransaction, false, false, true }, + { "wallet", "getunconfirmedbalance", &getunconfirmedbalance, false, false, true }, + { "wallet", "getwalletinfo", &getwalletinfo, false, false, true }, + { "wallet", "listlockunspent", &listlockunspent, false, false, true }, + { "wallet", "listreceivedbyaccount", &listreceivedbyaccount, false, false, true }, + { "wallet", "listreceivedbyaddress", &listreceivedbyaddress, false, false, true }, + { "wallet", "listsinceblock", &listsinceblock, false, false, true }, + { "wallet", "listtransactions", &listtransactions, false, false, true }, + { "wallet", "listunspent", &listunspent, false, false, true }, + { "wallet", "getassetbalances", &getassetbalances, false, false, true }, + { "wallet", "gettotalbalances", &gettotalbalances, false, false, true }, + + { "wallet", "getmultibalances", &getmultibalances, false, false, true }, + { "wallet", "getaddressbalances", &getaddressbalances, false, false, true }, + + { "wallet", "listwallettransactions", &listwallettransactions, false, false, true }, + { "wallet", "listaddresstransactions",&listaddresstransactions,false, false, true }, + { "wallet", "getwallettransaction", &getwallettransaction, false, false, true }, + { "wallet", "getaddresstransaction", &getaddresstransaction, false, false, true }, +}; + +/* MCHN END */ + +/** + * Call Table + */ +static const CRPCCommand vRPCCommands[] = +{ // category name actor (function) okSafeMode threadSafe reqWallet + // --------------------- ------------------------ ----------------------- ---------- ---------- --------- + /* Overall control/query calls */ + { "control", "getinfo", &getinfo, true, false, false }, /* uses wallet if enabled */ + { "control", "help", &help, true, true, false }, + { "control", "stop", &stop, true, true, false }, +/* MCHN START */ + { "control", "pause", &pausecmd, true, false, false }, + { "control", "resume", &resumecmd, true, false, false }, + { "control", "clearmempool", &clearmempool, true, false, false }, + { "control", "setlastblock", &setlastblock, true, false, false }, + { "control", "getblockchainparams", &getblockchainparams, true, false, false }, +/* MCHN END */ + + /* P2P networking */ + { "network", "getnetworkinfo", &getnetworkinfo, true, false, false }, + { "network", "addnode", &addnode, true, true, false }, + { "network", "getaddednodeinfo", &getaddednodeinfo, true, true, false }, + { "network", "getconnectioncount", &getconnectioncount, true, false, false }, + { "network", "getnettotals", &getnettotals, true, true, false }, + { "network", "getpeerinfo", &getpeerinfo, true, false, false }, + { "network", "ping", &ping, true, false, false }, + + /* Block chain and UTXO */ + { "blockchain", "getblockchaininfo", &getblockchaininfo, true, false, false }, + { "blockchain", "getbestblockhash", &getbestblockhash, true, false, false }, + { "blockchain", "getblockcount", &getblockcount, true, false, false }, + { "blockchain", "getblock", &getblock, true, false, false }, + { "blockchain", "getblockhash", &getblockhash, true, false, false }, + { "blockchain", "getchaintips", &getchaintips, true, false, false }, + { "blockchain", "getdifficulty", &getdifficulty, true, false, false }, + { "blockchain", "getmempoolinfo", &getmempoolinfo, true, true, false }, + { "blockchain", "getrawmempool", &getrawmempool, true, false, false }, + { "blockchain", "gettxout", &gettxout, true, false, false }, + { "blockchain", "gettxoutsetinfo", &gettxoutsetinfo, true, false, false }, + { "blockchain", "verifychain", &verifychain, true, false, false }, + { "blockchain", "invalidateblock", &invalidateblock, true, true, false }, + { "blockchain", "reconsiderblock", &reconsiderblock, true, true, false }, + +/* MCHN START */ + { "blockchain", "listassets", &listassets, true, false, false }, + { "blockchain", "listpermissions", &listpermissions, true, false, false }, + { "blockchain", "liststreams", &liststreams, true, false, false }, +/* MCHN END */ + + /* Mining */ + { "mining", "getblocktemplate", &getblocktemplate, true, false, false }, + { "mining", "getmininginfo", &getmininginfo, true, false, false }, + { "mining", "getnetworkhashps", &getnetworkhashps, true, false, false }, + { "mining", "prioritisetransaction", &prioritisetransaction, true, false, false }, + { "mining", "submitblock", &submitblock, true, true, false }, + +#ifdef ENABLE_WALLET + /* Coin generation */ + { "generating", "getgenerate", &getgenerate, true, false, false }, + { "generating", "gethashespersec", &gethashespersec, true, false, false }, + { "generating", "setgenerate", &setgenerate, true, true, false }, +#endif + + /* Raw transactions */ + { "rawtransactions", "createrawtransaction", &createrawtransaction, true, false, false }, + { "rawtransactions", "decoderawtransaction", &decoderawtransaction, true, false, false }, + { "rawtransactions", "decodescript", &decodescript, true, false, false }, + { "rawtransactions", "getrawtransaction", &getrawtransaction, true, false, false }, + { "rawtransactions", "sendrawtransaction", &sendrawtransaction, false, false, false }, + { "rawtransactions", "signrawtransaction", &signrawtransaction, false, false, false }, /* uses wallet if enabled */ +/* MCHN START */ + { "rawtransactions", "appendrawchange", &appendrawchange, false, false, true }, + { "hidden", "appendrawmetadata", &appendrawmetadata, false, false, true }, + { "rawtransactions", "appendrawdata", &appendrawmetadata, false, false, true }, +/* MCHN END */ + + /* Utility functions */ + { "util", "createkeypairs", &createkeypairs, true, true , false }, + { "util", "createmultisig", &createmultisig, true, true , false }, + { "util", "validateaddress", &validateaddress, true, false, false }, /* uses wallet if enabled */ + { "util", "verifymessage", &verifymessage, true, false, false }, + { "util", "estimatefee", &estimatefee, true, true, false }, + { "util", "estimatepriority", &estimatepriority, true, true, false }, + + /* Not shown in help */ + { "hidden", "invalidateblock", &invalidateblock, true, true, false }, + { "hidden", "reconsiderblock", &reconsiderblock, true, true, false }, + { "hidden", "setmocktime", &setmocktime, true, false, false }, + +#ifdef ENABLE_WALLET + /* Wallet */ + { "wallet", "addmultisigaddress", &addmultisigaddress, true, false, true }, + { "wallet", "backupwallet", &backupwallet, true, false, true }, + { "wallet", "dumpprivkey", &dumpprivkey, true, false, true }, + { "wallet", "dumpwallet", &dumpwallet, true, false, true }, + { "wallet", "encryptwallet", &encryptwallet, true, false, true }, + { "wallet", "getaccountaddress", &getaccountaddress, true, false, true }, + { "wallet", "getaccount", &getaccount, true, false, true }, + { "wallet", "getaddressesbyaccount", &getaddressesbyaccount, true, false, true }, + { "wallet", "getbalance", &getbalance, false, false, true }, + { "wallet", "getnewaddress", &getnewaddress, true, false, true }, + { "wallet", "getrawchangeaddress", &getrawchangeaddress, true, false, true }, + { "wallet", "getreceivedbyaccount", &getreceivedbyaccount, false, false, true }, + { "wallet", "getreceivedbyaddress", &getreceivedbyaddress, false, false, true }, + { "wallet", "gettransaction", &gettransaction, false, false, true }, + { "wallet", "getunconfirmedbalance", &getunconfirmedbalance, false, false, true }, + { "wallet", "getwalletinfo", &getwalletinfo, false, false, true }, + { "wallet", "importprivkey", &importprivkey, true, false, true }, + { "wallet", "importwallet", &importwallet, true, false, true }, + { "wallet", "importaddress", &importaddress, true, false, true }, + { "wallet", "keypoolrefill", &keypoolrefill, true, false, true }, + { "wallet", "listaccounts", &listaccounts, false, false, true }, + { "wallet", "listaddressgroupings", &listaddressgroupings, false, false, true }, + { "wallet", "listlockunspent", &listlockunspent, false, false, true }, + { "wallet", "listreceivedbyaccount", &listreceivedbyaccount, false, false, true }, + { "wallet", "listreceivedbyaddress", &listreceivedbyaddress, false, false, true }, + { "wallet", "listsinceblock", &listsinceblock, false, false, true }, + { "wallet", "listtransactions", &listtransactions, false, false, true }, + { "wallet", "listunspent", &listunspent, false, false, true }, + { "wallet", "lockunspent", &lockunspent, true, false, true }, + { "wallet", "move", &movecmd, false, false, true }, +// { "wallet", "sendfrom", &sendfrom, false, false, true }, + { "wallet", "sendfromaccount", &sendfrom, false, false, true }, + { "wallet", "sendmany", &sendmany, false, false, true }, + { "hidden", "sendtoaddress", &sendtoaddress, false, false, true }, + { "wallet", "send", &sendtoaddress, false, false, true }, +/* MCHN START */ + { "wallet", "getaddresses", &getaddresses, true, false, true }, + { "wallet", "combineunspent", &combineunspent, false, false, true }, + { "wallet", "grant", &grantcmd, false, false, true }, + { "wallet", "revoke", &revokecmd, false, false, true }, + { "wallet", "issue", &issuecmd, false, false, true }, + { "wallet", "issuemore", &issuemorecmd, false, false, true }, + { "wallet", "getassetbalances", &getassetbalances, false, false, true }, + { "wallet", "gettotalbalances", &gettotalbalances, false, false, true }, + { "hidden", "sendassettoaddress", &sendassettoaddress, false, false, true }, + { "wallet", "sendasset", &sendassettoaddress, false, false, true }, + { "wallet", "preparelockunspent", &preparelockunspent, false, false, true }, + { "wallet", "createrawexchange", &createrawexchange, false, false, true }, + { "wallet", "appendrawexchange", &appendrawexchange, false, false, true }, + { "wallet", "completeexchange", &completeexchange, false, false, true }, + { "wallet", "decoderawexchange", &decoderawexchange, false, false, true }, + + { "wallet", "grantfrom", &grantfromcmd, false, false, true }, + { "wallet", "revokefrom", &revokefromcmd, false, false, true }, + { "wallet", "issuefrom", &issuefromcmd, false, false, true }, + { "wallet", "issuemorefrom", &issuemorefromcmd, false, false, true }, + { "wallet", "preparelockunspentfrom", &preparelockunspentfrom, false, false, true }, + { "wallet", "sendassetfrom", &sendassetfrom, false, false, true }, + { "hidden", "sendfromaddress", &sendfromaddress, false, false, true }, + { "wallet", "sendfrom", &sendfromaddress, false, false, true }, + { "wallet", "getmultibalances", &getmultibalances, false, false, true }, + { "wallet", "getaddressbalances", &getaddressbalances, false, false, true }, + { "wallet", "disablerawtransaction", &disablerawtransaction, false, false, true }, + { "hidden", "sendwithmetadata", &sendwithmetadata, false, false, true }, + { "wallet", "sendwithdata", &sendwithmetadata, false, false, true }, + { "hidden", "sendwithmetadatafrom", &sendwithmetadatafrom, false, false, true }, + { "wallet", "sendwithdatafrom", &sendwithmetadatafrom, false, false, true }, + { "hidden", "grantwithmetadata", &grantwithmetadata, false, false, true }, + { "wallet", "grantwithdata", &grantwithmetadata, false, false, true }, + { "hidden", "grantwithmetadatafrom", &grantwithmetadatafrom, false, false, true }, + { "wallet", "grantwithdatafrom", &grantwithmetadatafrom, false, false, true }, + { "wallet", "createrawsendfrom", &createrawsendfrom, false, false, true }, + + { "wallet", "listaddresses", &listaddresses, false, false, true }, + { "wallet", "listwallettransactions", &listwallettransactions, false, false, true }, + { "wallet", "listaddresstransactions",&listaddresstransactions,false, false, true }, + { "wallet", "getwallettransaction", &getwallettransaction, false, false, true }, + { "wallet", "getaddresstransaction", &getaddresstransaction, false, false, true }, + { "wallet", "resendwallettransactions",&resendwallettransactions, false, false, true }, + + { "wallet", "create", &createcmd, false, false, true }, + { "wallet", "createfrom", &createfromcmd, false, false, true }, + { "wallet", "publish", &publish, false, false, true }, + { "wallet", "publishfrom", &publishfrom, false, false, true }, + { "wallet", "subscribe", &subscribe, false, false, true }, + { "wallet", "unsubscribe", &unsubscribe, false, false, true }, + { "wallet", "listassettransactions", &listassettransactions, false, false, true }, + { "wallet", "getassettransaction", &getassettransaction, false, false, true }, + { "wallet", "getstreamitem", &getstreamitem, false, false, true }, + { "wallet", "liststreamitems", &liststreamitems, false, false, true }, + { "wallet", "liststreamkeyitems", &liststreamkeyitems, false, false, true }, + { "wallet", "liststreampublisheritems",&liststreampublisheritems,false, false, true }, + { "wallet", "liststreamkeys", &liststreamkeys, false, false, true }, + { "wallet", "liststreampublishers", &liststreampublishers, false, false, true }, + { "wallet", "gettxoutdata", &gettxoutdata, false, false, true }, + +/* MCHN END */ + { "wallet", "setaccount", &setaccount, true, false, true }, + { "wallet", "settxfee", &settxfee, true, false, true }, + { "wallet", "signmessage", &signmessage, true, false, true }, + { "wallet", "walletlock", &walletlock, true, false, true }, + { "wallet", "walletpassphrasechange", &walletpassphrasechange, true, false, true }, + { "wallet", "walletpassphrase", &walletpassphrase, true, false, true }, +#endif // ENABLE_WALLET +}; + +CRPCTable::CRPCTable() +{ + unsigned int vcidx; + for (vcidx = 0; vcidx < (sizeof(vRPCCommands) / sizeof(vRPCCommands[0])); vcidx++) + { + const CRPCCommand *pcmd; + + pcmd = &vRPCCommands[vcidx]; + mapCommands[pcmd->name] = pcmd; + } + for (vcidx = 0; vcidx < (sizeof(vRPCWalletReadCommands) / sizeof(vRPCWalletReadCommands[0])); vcidx++) + { + const CRPCCommand *pcmd; + + pcmd = &vRPCWalletReadCommands[vcidx]; + mapWalletReadCommands[pcmd->name] = pcmd; + } +} + +const CRPCCommand *CRPCTable::operator[](string name) const +{ + map::const_iterator it = mapCommands.find(name); + if (it == mapCommands.end()) + return NULL; + return (*it).second; +} + + +bool HTTPAuthorized(map& mapHeaders) +{ + string strAuth = mapHeaders["authorization"]; + if (strAuth.substr(0,6) != "Basic ") + return false; + string strUserPass64 = strAuth.substr(6); boost::trim(strUserPass64); + string strUserPass = DecodeBase64(strUserPass64); + return TimingResistantEqual(strUserPass, strRPCUserColonPass); +} + +void ErrorReply(std::ostream& stream, const Object& objError, const Value& id) +{ + // Send error reply from json-rpc error object + int nStatus = HTTP_INTERNAL_SERVER_ERROR; + int code = find_value(objError, "code").get_int(); + if (code == RPC_INVALID_REQUEST) nStatus = HTTP_BAD_REQUEST; + else if (code == RPC_METHOD_NOT_FOUND) nStatus = HTTP_NOT_FOUND; + string strReply = JSONRPCReply(Value::null, objError, id); + stream << HTTPReply(nStatus, strReply, false) << std::flush; +} + +CNetAddr BoostAsioToCNetAddr(boost::asio::ip::address address) +{ + CNetAddr netaddr; + // Make sure that IPv4-compatible and IPv4-mapped IPv6 addresses are treated as IPv4 addresses + if (address.is_v6() + && (address.to_v6().is_v4_compatible() + || address.to_v6().is_v4_mapped())) + address = address.to_v6().to_v4(); + + if(address.is_v4()) + { + boost::asio::ip::address_v4::bytes_type bytes = address.to_v4().to_bytes(); + netaddr.SetRaw(NET_IPV4, &bytes[0]); + } + else + { + boost::asio::ip::address_v6::bytes_type bytes = address.to_v6().to_bytes(); + netaddr.SetRaw(NET_IPV6, &bytes[0]); + } + return netaddr; +} + +bool ClientAllowed(const boost::asio::ip::address& address) +{ + CNetAddr netaddr = BoostAsioToCNetAddr(address); + BOOST_FOREACH(const CSubNet &subnet, rpc_allow_subnets) + if (subnet.Match(netaddr)) + return true; + return false; +} + +template +class AcceptedConnectionImpl : public AcceptedConnection +{ +public: + AcceptedConnectionImpl( + asio::io_service& io_service, + ssl::context &context, + bool fUseSSL) : + sslStream(io_service, context), + _d(sslStream, fUseSSL), + _stream(_d) + { + } + + virtual std::iostream& stream() + { + return _stream; + } + + virtual std::string peer_address_to_string() const + { + return peer.address().to_string(); + } + + virtual void close() + { + _stream.close(); + } + + typename Protocol::endpoint peer; + asio::ssl::stream sslStream; + +private: + SSLIOStreamDevice _d; + iostreams::stream< SSLIOStreamDevice > _stream; +}; + +void ServiceConnection(AcceptedConnection *conn); + +//! Forward declaration required for RPCListen +template +static void RPCAcceptHandler(boost::shared_ptr< basic_socket_acceptor > acceptor, + ssl::context& context, + bool fUseSSL, + boost::shared_ptr< AcceptedConnection > conn, + const boost::system::error_code& error); + +/** + * Sets up I/O resources to accept and handle a new connection. + */ +template +static void RPCListen(boost::shared_ptr< basic_socket_acceptor > acceptor, + ssl::context& context, + const bool fUseSSL) +{ + // Accept connection + boost::shared_ptr< AcceptedConnectionImpl > conn(new AcceptedConnectionImpl(acceptor->get_io_service(), context, fUseSSL)); + + acceptor->async_accept( + conn->sslStream.lowest_layer(), + conn->peer, + boost::bind(&RPCAcceptHandler, + acceptor, + boost::ref(context), + fUseSSL, + conn, + _1)); +} + + +/** + * Accept and handle incoming connection. + */ +template +static void RPCAcceptHandler(boost::shared_ptr< basic_socket_acceptor > acceptor, + ssl::context& context, + const bool fUseSSL, + boost::shared_ptr< AcceptedConnection > conn, + const boost::system::error_code& error) +{ + // Immediately start accepting new connections, except when we're cancelled or our socket is closed. + if (error != asio::error::operation_aborted && acceptor->is_open()) + RPCListen(acceptor, context, fUseSSL); + + AcceptedConnectionImpl* tcp_conn = dynamic_cast< AcceptedConnectionImpl* >(conn.get()); + + if (error) + { + // TODO: Actually handle errors + LogPrintf("%s: Error: %s\n", __func__, error.message()); + } + // Restrict callers by IP. It is important to + // do this before starting client thread, to filter out + // certain DoS and misbehaving clients. + else if (tcp_conn && !ClientAllowed(tcp_conn->peer.address())) + { + // Only send a 403 if we're not using SSL to prevent a DoS during the SSL handshake. + if (!fUseSSL) + conn->stream() << HTTPError(HTTP_FORBIDDEN, false) << std::flush; + conn->close(); + } + else { + ServiceConnection(conn.get()); + conn->close(); + } +} + +static ip::tcp::endpoint ParseEndpoint(const std::string &strEndpoint, int defaultPort) +{ + std::string addr; + int port = defaultPort; + SplitHostPort(strEndpoint, port, addr); + return ip::tcp::endpoint(asio::ip::address::from_string(addr), port); +} + +void StartRPCThreads() +{ + rpc_allow_subnets.clear(); + rpc_allow_subnets.push_back(CSubNet("127.0.0.0/8")); // always allow IPv4 local subnet + rpc_allow_subnets.push_back(CSubNet("::1")); // always allow IPv6 localhost + if (mapMultiArgs.count("-rpcallowip")) + { + const vector& vAllow = mapMultiArgs["-rpcallowip"]; + BOOST_FOREACH(string strAllow, vAllow) + { + CSubNet subnet(strAllow); + if(!subnet.IsValid()) + { + uiInterface.ThreadSafeMessageBox( + strprintf("Invalid -rpcallowip subnet specification: %s. Valid are a single IP (e.g. 1.2.3.4), a network/netmask (e.g. 1.2.3.4/255.255.255.0) or a network/CIDR (e.g. 1.2.3.4/24).", strAllow), + "", CClientUIInterface::MSG_ERROR); + StartShutdown(); + return; + } + rpc_allow_subnets.push_back(subnet); + } + } + std::string strAllowed; + BOOST_FOREACH(const CSubNet &subnet, rpc_allow_subnets) + strAllowed += subnet.ToString() + " "; + LogPrint("rpc", "Allowing RPC connections from: %s\n", strAllowed); + + strRPCUserColonPass = mapArgs["-rpcuser"] + ":" + mapArgs["-rpcpassword"]; + if (((mapArgs["-rpcpassword"] == "") || + (mapArgs["-rpcuser"] == mapArgs["-rpcpassword"])) && Params().RequireRPCPassword()) + { + unsigned char rand_pwd[32]; + GetRandBytes(rand_pwd, 32); + uiInterface.ThreadSafeMessageBox(strprintf( + _("To use multichaind, you must set an rpcpassword in the configuration file:\n" + "%s\n" + "It is recommended you use the following random password:\n" + "rpcuser=multichainrpc\n" + "rpcpassword=%s\n" + "(you do not need to remember this password)\n" + "The username and password MUST NOT be the same.\n" + "If the file does not exist, create it with owner-readable-only file permissions.\n" + "It is also recommended to set alertnotify so you are notified of problems;\n" + "for example: alertnotify=echo %%s | mail -s \"MultiChain Alert\" admin@foo.com\n"), + GetConfigFile().string(), + EncodeBase58(&rand_pwd[0],&rand_pwd[0]+32)), + "", CClientUIInterface::MSG_ERROR | CClientUIInterface::SECURE); + StartShutdown(); + return; + } + + assert(rpc_io_service == NULL); + rpc_io_service = new asio::io_service(); + rpc_ssl_context = new ssl::context(*rpc_io_service, ssl::context::sslv23); + + const bool fUseSSL = GetBoolArg("-rpcssl", false); + + if (fUseSSL) + { + rpc_ssl_context->set_options(ssl::context::no_sslv2 | ssl::context::no_sslv3); + + filesystem::path pathCertFile(GetArg("-rpcsslcertificatechainfile", "server.cert")); + if (!pathCertFile.is_complete()) pathCertFile = filesystem::path(GetDataDir()) / pathCertFile; + if (filesystem::exists(pathCertFile)) rpc_ssl_context->use_certificate_chain_file(pathCertFile.string()); + else LogPrintf("ThreadRPCServer ERROR: missing server certificate file %s\n", pathCertFile.string()); + + filesystem::path pathPKFile(GetArg("-rpcsslprivatekeyfile", "server.pem")); + if (!pathPKFile.is_complete()) pathPKFile = filesystem::path(GetDataDir()) / pathPKFile; + if (filesystem::exists(pathPKFile)) rpc_ssl_context->use_private_key_file(pathPKFile.string(), ssl::context::pem); + else LogPrintf("ThreadRPCServer ERROR: missing server private key file %s\n", pathPKFile.string()); + + string strCiphers = GetArg("-rpcsslciphers", "TLSv1.2+HIGH:TLSv1+HIGH:!SSLv2:!aNULL:!eNULL:!3DES:@STRENGTH"); + SSL_CTX_set_cipher_list(rpc_ssl_context->impl(), strCiphers.c_str()); + } + + std::vector vEndpoints; + bool bBindAny = false; + int defaultPort = GetArg("-rpcport", BaseParams().RPCPort()); + if (!mapArgs.count("-rpcallowip")) // Default to loopback if not allowing external IPs + { + vEndpoints.push_back(ip::tcp::endpoint(asio::ip::address_v6::loopback(), defaultPort)); + vEndpoints.push_back(ip::tcp::endpoint(asio::ip::address_v4::loopback(), defaultPort)); + if (mapArgs.count("-rpcbind")) + { + LogPrintf("WARNING: option -rpcbind was ignored because -rpcallowip was not specified, refusing to allow everyone to connect\n"); + } + } else if (mapArgs.count("-rpcbind")) // Specific bind address + { + BOOST_FOREACH(const std::string &addr, mapMultiArgs["-rpcbind"]) + { + try { + vEndpoints.push_back(ParseEndpoint(addr, defaultPort)); + } + catch(const boost::system::system_error &) + { + uiInterface.ThreadSafeMessageBox( + strprintf(_("Could not parse -rpcbind value %s as network address"), addr), + "", CClientUIInterface::MSG_ERROR); + StartShutdown(); + return; + } + } + } else { // No specific bind address specified, bind to any + vEndpoints.push_back(ip::tcp::endpoint(asio::ip::address_v6::any(), defaultPort)); + vEndpoints.push_back(ip::tcp::endpoint(asio::ip::address_v4::any(), defaultPort)); + // Prefer making the socket dual IPv6/IPv4 instead of binding + // to both addresses seperately. + bBindAny = true; + } + + bool fListening = false; + std::string strerr; + std::string straddress; + BOOST_FOREACH(const ip::tcp::endpoint &endpoint, vEndpoints) + { + try { + asio::ip::address bindAddress = endpoint.address(); + straddress = bindAddress.to_string(); + LogPrintf("Binding RPC on address %s port %i (IPv4+IPv6 bind any: %i)\n", straddress, endpoint.port(), bBindAny); + boost::system::error_code v6_only_error; + boost::shared_ptr acceptor(new ip::tcp::acceptor(*rpc_io_service)); + + acceptor->open(endpoint.protocol()); + acceptor->set_option(boost::asio::ip::tcp::acceptor::reuse_address(true)); + + // Try making the socket dual IPv6/IPv4 when listening on the IPv6 "any" address + acceptor->set_option(boost::asio::ip::v6_only( + !bBindAny || bindAddress != asio::ip::address_v6::any()), v6_only_error); + + acceptor->bind(endpoint); + acceptor->listen(socket_base::max_connections); + + RPCListen(acceptor, *rpc_ssl_context, fUseSSL); + + fListening = true; + rpc_acceptors.push_back(acceptor); + // If dual IPv6/IPv4 bind successful, skip binding to IPv4 separately + if(bBindAny && bindAddress == asio::ip::address_v6::any() && !v6_only_error) + break; + } + catch(boost::system::system_error &e) + { + LogPrintf("ERROR: Binding RPC on address %s port %i failed: %s\n", straddress, endpoint.port(), e.what()); + strerr = strprintf(_("An error occurred while setting up the RPC address %s port %u for listening: %s"), straddress, endpoint.port(), e.what()); + } + } + + if (!fListening) { + uiInterface.ThreadSafeMessageBox(strerr, "", CClientUIInterface::MSG_ERROR); + StartShutdown(); + return; + } + + rpc_worker_group = new boost::thread_group(); + for (int i = 0; i < GetArg("-rpcthreads", 4); i++) + rpc_worker_group->create_thread(boost::bind(&asio::io_service::run, rpc_io_service)); + + fRPCRunning = true; +} + +void StartDummyRPCThread() +{ + if(rpc_io_service == NULL) + { + rpc_io_service = new asio::io_service(); + /* Create dummy "work" to keep the thread from exiting when no timeouts active, + * see http://www.boost.org/doc/libs/1_51_0/doc/html/boost_asio/reference/io_service.html#boost_asio.reference.io_service.stopping_the_io_service_from_running_out_of_work */ + rpc_dummy_work = new asio::io_service::work(*rpc_io_service); + rpc_worker_group = new boost::thread_group(); + rpc_worker_group->create_thread(boost::bind(&asio::io_service::run, rpc_io_service)); + fRPCRunning = true; + } +} + +void StopRPCThreads() +{ + if (rpc_io_service == NULL) return; + // Set this to false first, so that longpolling loops will exit when woken up + fRPCRunning = false; + + // First, cancel all timers and acceptors + // This is not done automatically by ->stop(), and in some cases the destructor of + // asio::io_service can hang if this is skipped. + boost::system::error_code ec; + BOOST_FOREACH(const boost::shared_ptr &acceptor, rpc_acceptors) + { + acceptor->cancel(ec); + if (ec) + LogPrintf("%s: Warning: %s when cancelling acceptor", __func__, ec.message()); + } + rpc_acceptors.clear(); + BOOST_FOREACH(const PAIRTYPE(std::string, boost::shared_ptr) &timer, deadlineTimers) + { + timer.second->cancel(ec); + if (ec) + LogPrintf("%s: Warning: %s when cancelling timer", __func__, ec.message()); + } + deadlineTimers.clear(); + + rpc_io_service->stop(); + cvBlockChange.notify_all(); + if (rpc_worker_group != NULL) + rpc_worker_group->join_all(); + delete rpc_dummy_work; rpc_dummy_work = NULL; + delete rpc_worker_group; rpc_worker_group = NULL; + delete rpc_ssl_context; rpc_ssl_context = NULL; + delete rpc_io_service; rpc_io_service = NULL; +} + +bool IsRPCRunning() +{ + return fRPCRunning; +} + +void SetRPCWarmupStatus(const std::string& newStatus) +{ + LOCK(cs_rpcWarmup); + rpcWarmupStatus = newStatus; +} + +void SetRPCWarmupFinished() +{ + LOCK(cs_rpcWarmup); + assert(fRPCInWarmup); + fRPCInWarmup = false; +} + +bool RPCIsInWarmup(std::string *outStatus) +{ + LOCK(cs_rpcWarmup); + if (outStatus) + *outStatus = rpcWarmupStatus; + return fRPCInWarmup; +} + +void RPCRunHandler(const boost::system::error_code& err, boost::function func) +{ + if (!err) + func(); +} + +void RPCRunLater(const std::string& name, boost::function func, int64_t nSeconds) +{ + assert(rpc_io_service != NULL); + + if (deadlineTimers.count(name) == 0) + { + deadlineTimers.insert(make_pair(name, + boost::shared_ptr(new deadline_timer(*rpc_io_service)))); + } + deadlineTimers[name]->expires_from_now(posix_time::seconds(nSeconds)); + deadlineTimers[name]->async_wait(boost::bind(RPCRunHandler, _1, func)); +} + +class JSONRequest +{ +public: + Value id; + string strMethod; + Array params; + + JSONRequest() { id = Value::null; } + void parse(const Value& valRequest); +}; + +void JSONRequest::parse(const Value& valRequest) +{ + // Parse request + if (valRequest.type() != obj_type) + throw JSONRPCError(RPC_INVALID_REQUEST, "Invalid Request object"); + const Object& request = valRequest.get_obj(); + + // Parse id now so errors from here on will have the id + id = find_value(request, "id"); + + // Parse method + Value valMethod = find_value(request, "method"); + if (valMethod.type() == null_type) + throw JSONRPCError(RPC_INVALID_REQUEST, "Missing method"); + if (valMethod.type() != str_type) + throw JSONRPCError(RPC_INVALID_REQUEST, "Method must be a string"); + strMethod = valMethod.get_str(); + if (strMethod != "getblocktemplate") + LogPrint("rpc", "ThreadRPCServer method=%s\n", strMethod); + +/* MCHN START */ + Value valChainName = find_value(request, "chain_name"); + if (valChainName.type() != null_type) + { + if (valChainName.type() != str_type) + throw JSONRPCError(RPC_INVALID_REQUEST, "Chain name must be a string"); + if (strcmp(valChainName.get_str().c_str(),mc_gState->m_Params->NetworkName())) + throw JSONRPCError(RPC_INVALID_REQUEST, "Wrong chain name"); + } +/* MCHN END */ + // Parse params + Value valParams = find_value(request, "params"); + if (valParams.type() == array_type) + params = valParams.get_array(); + else if (valParams.type() == null_type) + params = Array(); + else + throw JSONRPCError(RPC_INVALID_REQUEST, "Params must be an array"); +} + + +static Object JSONRPCExecOne(const Value& req) +{ + Object rpc_result; + + JSONRequest jreq; +/* MCHN START */ + uint32_t wallet_mode=mc_gState->m_WalletMode; +/* MCHN END */ + try { + jreq.parse(req); + + Value result = tableRPC.execute(jreq.strMethod, jreq.params); + rpc_result = JSONRPCReplyObj(result, Value::null, jreq.id); + } + catch (Object& objError) + { +/* MCHN START */ + mc_gState->m_WalletMode=wallet_mode; + LogPrint("mcapi","mcapi: API request failure\n"); +/* MCHN END */ + rpc_result = JSONRPCReplyObj(Value::null, objError, jreq.id); + } + catch (std::exception& e) + { +/* MCHN START */ + mc_gState->m_WalletMode=wallet_mode; + LogPrint("mcapi","mcapi: API request failure\n"); +/* MCHN END */ + rpc_result = JSONRPCReplyObj(Value::null, + JSONRPCError(RPC_PARSE_ERROR, e.what()), jreq.id); + } + + return rpc_result; +} + +static string JSONRPCExecBatch(const Array& vReq) +{ + Array ret; + for (unsigned int reqIdx = 0; reqIdx < vReq.size(); reqIdx++) + ret.push_back(JSONRPCExecOne(vReq[reqIdx])); + + return write_string(Value(ret), false) + "\n"; +} + +static bool HTTPReq_JSONRPC(AcceptedConnection *conn, + string& strRequest, + map& mapHeaders, + bool fRun) +{ + // Check authorization + if (mapHeaders.count("authorization") == 0) + { + conn->stream() << HTTPError(HTTP_UNAUTHORIZED, false) << std::flush; + return false; + } + + if (!HTTPAuthorized(mapHeaders)) + { + LogPrintf("ThreadRPCServer incorrect password attempt from %s\n", conn->peer_address_to_string()); + /* Deter brute-forcing + If this results in a DoS the user really + shouldn't have their RPC port exposed. */ + MilliSleep(250); + + conn->stream() << HTTPError(HTTP_UNAUTHORIZED, false) << std::flush; + return false; + } + + JSONRequest jreq; +/* MCHN START */ + uint32_t wallet_mode=mc_gState->m_WalletMode; + LogPrint("mcapi","mcapi: API request from %s\n",conn->peer_address_to_string().c_str()); +/* MCHN END */ + try + { + // Parse request + Value valRequest; + if (!read_string(strRequest, valRequest)) + throw JSONRPCError(RPC_PARSE_ERROR, "Parse error"); + + // Return immediately if in warmup + { + LOCK(cs_rpcWarmup); + if (fRPCInWarmup) + throw JSONRPCError(RPC_IN_WARMUP, rpcWarmupStatus); + } + + string strReply; + + // singleton request + if (valRequest.type() == obj_type) { + jreq.parse(valRequest); + + Value result = tableRPC.execute(jreq.strMethod, jreq.params); + + // Send reply + strReply = JSONRPCReply(result, Value::null, jreq.id); + + // array of requests + } else if (valRequest.type() == array_type) + strReply = JSONRPCExecBatch(valRequest.get_array()); + else + throw JSONRPCError(RPC_PARSE_ERROR, "Top-level object parse error"); + + conn->stream() << HTTPReplyHeader(HTTP_OK, fRun, strReply.size()) << strReply << std::flush; + } + catch (Object& objError) + { +/* MCHN START */ + mc_gState->m_WalletMode=wallet_mode; + LogPrint("mcapi","mcapi: API request failure\n"); +/* MCHN END */ + ErrorReply(conn->stream(), objError, jreq.id); + return false; + } + catch (std::exception& e) + { +/* MCHN START */ + mc_gState->m_WalletMode=wallet_mode; + LogPrint("mcapi","mcapi: API request failure\n"); +/* MCHN END */ + ErrorReply(conn->stream(), JSONRPCError(RPC_PARSE_ERROR, e.what()), jreq.id); + return false; + } + return true; +} + +void ServiceConnection(AcceptedConnection *conn) +{ + bool fRun = true; + while (fRun && !ShutdownRequested()) + { + int nProto = 0; + map mapHeaders; + string strRequest, strMethod, strURI; + + // Read HTTP request line + if (!ReadHTTPRequestLine(conn->stream(), nProto, strMethod, strURI)) + break; + + // Read HTTP message headers and body + ReadHTTPMessage(conn->stream(), mapHeaders, strRequest, nProto, MAX_SIZE); + + // HTTP Keep-Alive is false; close connection immediately + if ((mapHeaders["connection"] == "close") || (!GetBoolArg("-rpckeepalive", true))) + fRun = false; + + // Process via JSON-RPC API + if (strURI == "/") { + if (!HTTPReq_JSONRPC(conn, strRequest, mapHeaders, fRun)) + break; + + // Process via HTTP REST API + } else if (strURI.substr(0, 6) == "/rest/" && GetBoolArg("-rest", false)) { + if (!HTTPReq_REST(conn, strURI, mapHeaders, fRun)) + break; + + } else { + conn->stream() << HTTPError(HTTP_NOT_FOUND, false) << std::flush; + break; + } + } +} + +json_spirit::Value CRPCTable::execute(const std::string &strMethod, const json_spirit::Array ¶ms) const +{ + // Find method + const CRPCCommand *pcmd = tableRPC[strMethod]; + if (!pcmd) + throw JSONRPCError(RPC_METHOD_NOT_FOUND, "Method not found"); +#ifdef ENABLE_WALLET + if (pcmd->reqWallet && !pwalletMain) + throw JSONRPCError(RPC_METHOD_NOT_FOUND, "Method not found (disabled)"); +#endif + + // Observe safe mode + string strWarning = GetWarnings("rpc"); + if (strWarning != "" && !GetBoolArg("-disablesafemode", false) && + !pcmd->okSafeMode) + throw JSONRPCError(RPC_FORBIDDEN_BY_SAFE_MODE, string("Safe mode: ") + strWarning); + + try + { + // Execute + string strRequest = JSONRPCRequest(strMethod, params, 1); + LogPrint("mcapi","mcapi: API request: %s\n",strRequest.c_str()); + + Value result; + { + if (pcmd->threadSafe) + result = pcmd->actor(params, false); +#ifdef ENABLE_WALLET + else if (!pwalletMain) { + LOCK(cs_main); + result = pcmd->actor(params, false); + } else { + LOCK2(cs_main, pwalletMain->cs_wallet); +/* MCHN START */ + uint32_t wallet_mode=mc_gState->m_WalletMode; + string strResultNone; + string strResult; + if(LogAcceptCategory("walletcompare")) + { + if(wallet_mode & MC_WMD_MAP_TXS) + { + if(mapWalletReadCommands.count(strMethod)) + { + mc_gState->m_WalletMode=MC_WMD_NONE; + result = pcmd->actor(params, false); + strResultNone=JSONRPCReply(result, Value::null, 1); + mc_gState->m_WalletMode=wallet_mode; + } + } + } +/* MCHN END */ + result = pcmd->actor(params, false); +/* MCHN START */ + if(LogAcceptCategory("walletcompare")) + { + if(wallet_mode & MC_WMD_MAP_TXS) + { + if(mapWalletReadCommands.count(strMethod)) + { + strResult=JSONRPCReply(result, Value::null, 1); + if(strcmp(strResultNone.c_str(),strResult.c_str())) + { + string strRequestBad = JSONRPCRequest(strMethod, params, 1); + LogPrint("walletcompare","walletcompare: ERROR: Result mismatch on API request: %s\n",strRequestBad.c_str()); + LogPrint("walletcompare","walletcompare: %s\n",strResultNone.c_str()); + LogPrint("walletcompare","walletcompare: %s\n",strResult.c_str()); + } + else + { + LogPrint("walletcompare","walletcompare: match: %s \n",strMethod.c_str()); + } + } + } + } +/* MCHN END */ + + } +#else // ENABLE_WALLET + else { + LOCK(cs_main); + result = pcmd->actor(params, false); + } +#endif // !ENABLE_WALLET + } +/* MCHN START */ + LogPrint("mcapi","mcapi: API request successful: %s\n",strMethod.c_str()); +/* MCHN END */ + return result; + } + catch (std::exception& e) + { + LogPrint("mcapi","mcapi: API request failure: %s\n",strMethod.c_str()); + if(strcmp(e.what(),"Help message not found\n") == 0) + { + throw JSONRPCError(RPC_MISC_ERROR, mc_RPCHelpString(strMethod).c_str()); + } + throw JSONRPCError(RPC_MISC_ERROR, e.what()); + } +} + +std::string HelpExampleCli(string methodname, string args){ + return "> multichain-cli " + std::string(mc_gState->m_NetworkParams->Name()) + " " + methodname + " " + args + "\n"; +} + +std::string HelpExampleRpc(string methodname, string args){ + return "> curl --user myusername --data-binary '{\"jsonrpc\": \"1.0\", \"id\":\"curltest\", " + "\"method\": \"" + std::string(mc_gState->m_NetworkParams->Name()) + " " + methodname + "\", \"params\": [" + args + "] }' -H 'content-type: text/plain;' http://127.0.0.1:"+ + strprintf("%d",(int)mc_gState->m_NetworkParams->GetInt64Param("defaultrpcport")) + "\n";// MCHN was hard-coded 8332 before +} + +const CRPCTable tableRPC; +std::map mapHelpStrings; \ No newline at end of file diff --git a/src/rpc/rpcserver.h b/src/rpc/rpcserver.h new file mode 100644 index 00000000..b1f2f8b6 --- /dev/null +++ b/src/rpc/rpcserver.h @@ -0,0 +1,305 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_RPCSERVER_H +#define BITCOIN_RPCSERVER_H + +#include "structs/amount.h" +#include "rpc/rpcprotocol.h" +#include "structs/uint256.h" + +#include +#include +#include +#include + +#include "json/json_spirit_reader_template.h" +#include "json/json_spirit_utils.h" +#include "json/json_spirit_writer_template.h" + +class CBlockIndex; +class CNetAddr; + +class AcceptedConnection +{ +public: + virtual ~AcceptedConnection() {} + + virtual std::iostream& stream() = 0; + virtual std::string peer_address_to_string() const = 0; + virtual void close() = 0; +}; + +/** Start RPC threads */ +void StartRPCThreads(); +/** + * Alternative to StartRPCThreads for the GUI, when no server is + * used. The RPC thread in this case is only used to handle timeouts. + * If real RPC threads have already been started this is a no-op. + */ +void StartDummyRPCThread(); +/** Stop RPC threads */ +void StopRPCThreads(); +/** Query whether RPC is running */ +bool IsRPCRunning(); + +/** + * Set the RPC warmup status. When this is done, all RPC calls will error out + * immediately with RPC_IN_WARMUP. + */ +void SetRPCWarmupStatus(const std::string& newStatus); +/* Mark warmup as done. RPC calls will be processed from now on. */ +void SetRPCWarmupFinished(); + +/* returns the current warmup state. */ +bool RPCIsInWarmup(std::string *statusOut); + +/** + * Type-check arguments; throws JSONRPCError if wrong type given. Does not check that + * the right number of arguments are passed, just that any passed are the correct type. + * Use like: RPCTypeCheck(params, boost::assign::list_of(str_type)(int_type)(obj_type)); + */ +void RPCTypeCheck(const json_spirit::Array& params, + const std::list& typesExpected, bool fAllowNull=false); +/** + * Check for expected keys/value types in an Object. + * Use like: RPCTypeCheck(object, boost::assign::map_list_of("name", str_type)("value", int_type)); + */ +void RPCTypeCheck(const json_spirit::Object& o, + const std::map& typesExpected, bool fAllowNull=false); + +/** + * Run func nSeconds from now. Uses boost deadline timers. + * Overrides previous timer (if any). + */ +void RPCRunLater(const std::string& name, boost::function func, int64_t nSeconds); + +//! Convert boost::asio address to CNetAddr +extern CNetAddr BoostAsioToCNetAddr(boost::asio::ip::address address); + +typedef json_spirit::Value(*rpcfn_type)(const json_spirit::Array& params, bool fHelp); + +class CRPCCommand +{ +public: + std::string category; + std::string name; + rpcfn_type actor; + bool okSafeMode; + bool threadSafe; + bool reqWallet; +}; + +/** + * Bitcoin RPC command dispatcher. + */ +class CRPCTable +{ +private: + std::map mapCommands; + std::map mapWalletReadCommands; +public: + CRPCTable(); + const CRPCCommand* operator[](std::string name) const; + std::string help(std::string name) const; + + /** + * Execute a method. + * @param method Method to execute + * @param params Array of arguments (JSON objects) + * @returns Result of the call. + * @throws an exception (json_spirit::Value) when an error happens. + */ + json_spirit::Value execute(const std::string &method, const json_spirit::Array ¶ms) const; +}; + +extern const CRPCTable tableRPC; + +/* MCHN START */ +extern std::map mapHelpStrings; +void mc_InitRPCHelpMap(); +std::string mc_RPCHelpString(std::string strMethod); +/* MCHN END */ + +/** + * Utilities: convert hex-encoded Values + * (throws error if not hex). + */ +extern uint256 ParseHashV(const json_spirit::Value& v, std::string strName); +extern uint256 ParseHashO(const json_spirit::Object& o, std::string strKey); +extern std::vector ParseHexV(const json_spirit::Value& v, std::string strName); +extern std::vector ParseHexO(const json_spirit::Object& o, std::string strKey); + +extern void InitRPCMining(); +extern void ShutdownRPCMining(); + +extern int64_t nWalletUnlockTime; +extern CAmount AmountFromValue(const json_spirit::Value& value); +extern json_spirit::Value ValueFromAmount(const CAmount& amount); +extern double GetDifficulty(const CBlockIndex* blockindex = NULL); +extern std::string HelpRequiringPassphrase(); +extern std::string HelpExampleCli(std::string methodname, std::string args); +extern std::string HelpExampleRpc(std::string methodname, std::string args); + +extern void EnsureWalletIsUnlocked(); + +extern json_spirit::Value getconnectioncount(const json_spirit::Array& params, bool fHelp); // in rpcnet.cpp +extern json_spirit::Value getpeerinfo(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value ping(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value addnode(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getaddednodeinfo(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getnettotals(const json_spirit::Array& params, bool fHelp); + +extern json_spirit::Value dumpprivkey(const json_spirit::Array& params, bool fHelp); // in rpcdump.cpp +extern json_spirit::Value importprivkey(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value importaddress(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value dumpwallet(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value importwallet(const json_spirit::Array& params, bool fHelp); + +extern json_spirit::Value getgenerate(const json_spirit::Array& params, bool fHelp); // in rpcmining.cpp +extern json_spirit::Value setgenerate(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getnetworkhashps(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value gethashespersec(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getmininginfo(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value prioritisetransaction(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getblocktemplate(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value submitblock(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value estimatefee(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value estimatepriority(const json_spirit::Array& params, bool fHelp); + +extern json_spirit::Value getnewaddress(const json_spirit::Array& params, bool fHelp); // in rpcwallet.cpp +extern json_spirit::Value getaccountaddress(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getrawchangeaddress(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value setaccount(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getaccount(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getaddressesbyaccount(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value sendtoaddress(const json_spirit::Array& params, bool fHelp); +/* MCHN START */ +extern json_spirit::Value createkeypairs(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getaddresses(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value combineunspent(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value grantcmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value revokecmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value issuecmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value issuemorecmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listassets(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listpermissions(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getassetbalances(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value gettotalbalances(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value sendassettoaddress(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getblockchainparams(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value preparelockunspent(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value createrawexchange(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value appendrawexchange(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value completeexchange(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value decoderawexchange(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value appendrawmetadata(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value grantfromcmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value revokefromcmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value issuefromcmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value issuemorefromcmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value sendassetfrom(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value preparelockunspentfrom(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value sendfromaddress(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value createrawsendfrom(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getmultibalances(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getaddressbalances(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value disablerawtransaction(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value sendwithmetadata(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value sendwithmetadatafrom(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value grantwithmetadata(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value grantwithmetadatafrom(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listwallettransactions(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listaddresstransactions(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getwallettransaction(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getaddresstransaction(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value appendrawchange(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value pausecmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value resumecmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value clearmempool(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value setlastblock(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value resendwallettransactions(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listaddresses(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value liststreams(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value createcmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value createfromcmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value publish(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value publishfrom(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value subscribe(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value unsubscribe(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listassettransactions(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getassettransaction(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getstreamitem(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value liststreamitems(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value liststreamkeyitems(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value liststreampublisheritems(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value liststreamkeys(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value liststreampublishers(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value gettxoutdata(const json_spirit::Array& params, bool fHelp); +/* MCHN END */ +extern json_spirit::Value signmessage(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value verifymessage(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getreceivedbyaddress(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getreceivedbyaccount(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getbalance(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getunconfirmedbalance(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value movecmd(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value sendfrom(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value sendmany(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value addmultisigaddress(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value createmultisig(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listreceivedbyaddress(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listreceivedbyaccount(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listtransactions(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listaddressgroupings(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listaccounts(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listsinceblock(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value gettransaction(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value backupwallet(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value keypoolrefill(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value walletpassphrase(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value walletpassphrasechange(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value walletlock(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value encryptwallet(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value validateaddress(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getinfo(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getwalletinfo(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getblockchaininfo(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getnetworkinfo(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value setmocktime(const json_spirit::Array& params, bool fHelp); + +extern json_spirit::Value getrawtransaction(const json_spirit::Array& params, bool fHelp); // in rcprawtransaction.cpp +extern json_spirit::Value listunspent(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value lockunspent(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value listlockunspent(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value createrawtransaction(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value decoderawtransaction(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value decodescript(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value signrawtransaction(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value sendrawtransaction(const json_spirit::Array& params, bool fHelp); + +extern json_spirit::Value getblockcount(const json_spirit::Array& params, bool fHelp); // in rpcblockchain.cpp +extern json_spirit::Value getbestblockhash(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getdifficulty(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value settxfee(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getmempoolinfo(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getrawmempool(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getblockhash(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getblock(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value gettxoutsetinfo(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value gettxout(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value verifychain(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value getchaintips(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value invalidateblock(const json_spirit::Array& params, bool fHelp); +extern json_spirit::Value reconsiderblock(const json_spirit::Array& params, bool fHelp); + +// in rest.cpp +extern bool HTTPReq_REST(AcceptedConnection *conn, + std::string& strURI, + std::map& mapHeaders, + bool fRun); + +#endif // BITCOIN_RPCSERVER_H diff --git a/src/rpc/rpcstreams.cpp b/src/rpc/rpcstreams.cpp new file mode 100644 index 00000000..57bc069d --- /dev/null +++ b/src/rpc/rpcstreams.cpp @@ -0,0 +1,1591 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + + +#include "rpc/rpcwallet.h" + +void parseStreamIdentifier(Value stream_identifier,mc_EntityDetails *entity) +{ + unsigned char buf[32]; + unsigned char buf_a[MC_AST_ASSET_REF_SIZE]; + unsigned char buf_n[MC_AST_ASSET_REF_SIZE]; + int ret; + + if (stream_identifier.type() != null_type && !stream_identifier.get_str().empty()) + { + string str=stream_identifier.get_str(); + + if(CoinSparkAssetRefDecode(buf_a,str.c_str(),str.size())) + { + memset(buf_n,0,MC_AST_ASSET_REF_SIZE); + if(memcmp(buf_a,buf_n,MC_AST_ASSET_REF_SIZE) == 0) + { + unsigned char *root_stream_name; + int root_stream_name_size; + root_stream_name=(unsigned char *)mc_gState->m_NetworkParams->GetParam("rootstreamname",&root_stream_name_size); + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + root_stream_name_size=0; + } + if(root_stream_name_size) + { + str=strprintf("%s",root_stream_name); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Stream with this stream reference not found: ")+str); + } + } + } + + ret=ParseAssetKey(str.c_str(),buf,NULL,NULL,NULL,NULL,MC_ENT_TYPE_STREAM); + switch(ret) + { + case -1: + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Stream with this txid not found: ")+str); + break; + case -2: + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Stream with this stream reference not found: ")+str); + break; + case -3: + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Stream with this name not found: ")+str); + break; + case -4: + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Could not parse stream key: ")+str); + break; +/* + case 1: + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Unconfirmed stream: ")+str); + break; + */ + } + } + else + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid stream identifier"); + } + + if(entity) + { + if(mc_gState->m_Assets->FindEntityByTxID(entity,buf)) + { + if(entity->GetEntityType() != MC_ENT_TYPE_STREAM) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid stream identifier, not stream"); + } + } + } +} + +Value liststreams(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 4) + throw runtime_error("Help message not found\n"); + + Array results; + mc_Buffer *streams; + unsigned char *txid; + uint32_t output_level; + + int count,start; + count=2147483647; + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + count=params[2].get_int(); + if(count < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + start=-count; + if (params.size() > 3) + { + if(params[3].type() == int_type) + { + start=params[3].get_int(); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid start")); + } + } + + streams=NULL; + + vector inputStrings; + if (params.size() > 0 && params[0].type() != null_type && ((params[0].type() != str_type) || (params[0].get_str() !="*" ) ) ) + { + if(params[0].type() == str_type) + { + inputStrings.push_back(params[0].get_str()); + if(params[0].get_str() == "") + { + return results; + } + } + else + { + inputStrings=ParseStringList(params[0]); + if(inputStrings.size() == 0) + { + return results; + } + } + } + if(inputStrings.size()) + { + { + LOCK(cs_main); + for(int is=0;is<(int)inputStrings.size();is++) + { + string param=inputStrings[is]; + + mc_EntityDetails stream_entity; + parseStreamIdentifier(param,&stream_entity); + + streams=mc_gState->m_Assets->GetEntityList(streams,stream_entity.GetTxID(),MC_ENT_TYPE_STREAM); + } + } + } + else + { + { + LOCK(cs_main); + streams=mc_gState->m_Assets->GetEntityList(streams,NULL,MC_ENT_TYPE_STREAM); + } + } + + if(streams == NULL) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Cannot open entity database"); + + output_level=0x1E; + + if (params.size() > 1) + { + if(params[1].type() == int_type) + { + if(params[1].get_int()) + { + output_level=0x3E; + } + } + if(params[1].type() == bool_type) + { + if(params[1].get_bool()) + { + output_level=0x3E; + } + } + } + + mc_AdjustStartAndCount(&count,&start,streams->GetCount()); + Array partial_results; + int unconfirmed_count=0; + if(count > 0) + { + for(int i=0;iGetCount();i++) + { + Object entry; + + txid=streams->GetRow(i); + entry=StreamEntry(txid,output_level); + if(entry.size()>0) + { + BOOST_FOREACH(const Pair& p, entry) + { + if(p.name_ == "streamref") + { + if(p.value_.type() == str_type) + { + results.push_back(entry); + } + else + { + unconfirmed_count++; + } + } + } + } + } + + sort(results.begin(), results.end(), AssetCompareByRef); + + for(int i=0;iGetCount();i++) + { + Object entry; + + txid=streams->GetRow(i); + + entry=StreamEntry(txid,output_level); + if(entry.size()>0) + { + BOOST_FOREACH(const Pair& p, entry) + { + if(p.name_ == "streamref") + { + if(p.value_.type() != str_type) + { + results.push_back(entry); + } + } + } + } + } + } + + bool return_partial=false; + if(count != streams->GetCount()) + { + return_partial=true; + } + mc_gState->m_Assets->FreeEntityList(streams); + if(return_partial) + { + for(int i=start;im_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + + Array ext_params; + ext_params.push_back("*"); + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + } + + return createfromcmd(ext_params,fHelp); +} + +Value createfromcmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 4) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + + if (strcmp(params[1].get_str().c_str(),"stream")) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid entity type, should be stream"); + + CWalletTx wtx; + + mc_Script *lpScript; + + mc_Script *lpDetailsScript; + lpDetailsScript=NULL; + + mc_Script *lpDetails; + + int ret,type; + string stream_name=""; + + if (params[2].type() != str_type) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid stream name, should be string"); + + if(!params[2].get_str().empty()) + { + stream_name=params[2].get_str(); + } + + if(params[3].type() != bool_type) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid open flag, should be boolean"); + + if(mc_gState->m_Features->Streams()) + { + if(stream_name == "*") + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid stream name"); + } + } + + unsigned char buf_a[MC_AST_ASSET_REF_SIZE]; + if(CoinSparkAssetRefDecode(buf_a,stream_name.c_str(),stream_name.size())) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid stream name"); + } + + + if(stream_name.size()) + { + ret=ParseAssetKey(stream_name.c_str(),NULL,NULL,NULL,NULL,&type,MC_ENT_TYPE_ANY); + if(ret != -3) + { + if(type == 3) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Stream or asset with this name already exists"); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid stream name"); + } + } + } + + lpScript=new mc_Script; + + lpDetails=new mc_Script; + lpDetails->AddElement(); + if(params[3].get_bool()) + { + unsigned char b=1; + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_ANYONE_CAN_WRITE,&b,1); + } + if(stream_name.size()) + { + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_NAME,(unsigned char*)(stream_name.c_str()),stream_name.size());//+1); + } + + + + if (params.size() > 4) + { + if(params[4].type() == obj_type) + { + Object objParams = params[4].get_obj(); + BOOST_FOREACH(const Pair& s, objParams) + { + lpDetails->SetParamValue(s.name_.c_str(),s.name_.size(),(unsigned char*)s.value_.get_str().c_str(),s.value_.get_str().size()); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid custom fields, expecting object"); + } + } + + int err; + size_t bytes; + const unsigned char *script; + script=lpDetails->GetData(0,&bytes); + + lpDetailsScript=new mc_Script; + + size_t elem_size; + const unsigned char *elem; + CScript scriptOpReturn=CScript(); + + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_STREAM,0,script,bytes); + + elem = lpDetailsScript->GetData(0,&elem_size); + scriptOpReturn << vector(elem, elem + elem_size) << OP_DROP << OP_RETURN; + } + else + { + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_STREAM); + + err=lpDetailsScript->SetGeneralDetails(script,bytes); + if(err) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid custom fields or stream name, too long"); + } + + for(int e=0;eGetNumElements();e++) + { + elem = lpDetailsScript->GetData(e,&elem_size); + if(e == (lpDetailsScript->GetNumElements() - 1) ) + { + if(elem_size > 0) + { + scriptOpReturn << OP_RETURN << vector(elem, elem + elem_size); + } + else + { + scriptOpReturn << OP_RETURN; + } + } + else + { + if(elem_size > 0) + { + scriptOpReturn << vector(elem, elem + elem_size) << OP_DROP; + } + } + } + } + + vector addresses; + + vector fromaddresses; + + if(params[0].get_str() != "*") + { + fromaddresses=ParseAddresses(params[0].get_str(),false,false); + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + if( (IsMine(*pwalletMain, fromaddresses[0]) & ISMINE_SPENDABLE) != ISMINE_SPENDABLE ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key for from-address is not found in this wallet"); + } + + set thisFromAddresses; + + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + thisFromAddresses.insert(fromaddress); + } + + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_CREATE,&thisFromAddresses)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "from-address doesn't have create permission"); + } + } + else + { + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_CREATE)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "This wallet doesn't have keys with create permission"); + } + } + + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + SendMoneyToSeveralAddresses(addresses, 0, wtx, lpScript, scriptOpReturn,fromaddresses); + + if(lpDetailsScript) + { + delete lpDetailsScript; + } + delete lpDetails; + delete lpScript; + + return wtx.GetHash().GetHex(); +} + +Value publish(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 3) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + + Array ext_params; + ext_params.push_back("*"); + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + } + + return publishfrom(ext_params,fHelp); +} + +Value publishfrom(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 4) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + + if(params[2].get_str() == "*") + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid item-key-string")); + } + + mc_Script *lpScript; + mc_EntityDetails stream_entity; + parseStreamIdentifier(params[1],&stream_entity); + + + // Wallet comments + CWalletTx wtx; + + vector addresses; + + vector fromaddresses; + + if(params[0].get_str() != "*") + { + fromaddresses=ParseAddresses(params[0].get_str(),false,false); + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + if( (IsMine(*pwalletMain, fromaddresses[0]) & ISMINE_SPENDABLE) != ISMINE_SPENDABLE ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key for from-address is not found in this wallet"); + } + } + + FindAddressesWithPublishPermission(fromaddresses,&stream_entity); + + if(params[2].get_str().size() > MC_ENT_MAX_ITEM_KEY_SIZE) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Item key is too long"); + } + + mc_Script *lpDetailsScript; + lpDetailsScript=NULL; + + + bool fIsHex; + vector dataData(ParseHex(params[3].get_str().c_str(),fIsHex)); + if(!fIsHex) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Item data should be hexadecimal string"); + } + + lpDetailsScript=new mc_Script; + lpDetailsScript->SetEntity(stream_entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET); + lpDetailsScript->SetItemKey((unsigned char*)params[2].get_str().c_str(),params[2].get_str().size()); + + lpDetailsScript->AddElement(); + + lpDetailsScript->SetData(&dataData[0],dataData.size()); + + size_t elem_size; + const unsigned char *elem; + CScript scriptOpReturn=CScript(); + + for(int e=0;eGetNumElements();e++) + { + elem = lpDetailsScript->GetData(e,&elem_size); + if(e == (lpDetailsScript->GetNumElements() - 1) ) + { + if(elem_size > 0) + { + scriptOpReturn << OP_RETURN << vector(elem, elem + elem_size); + } + else + { + scriptOpReturn << OP_RETURN; + } + } + else + { + if(elem_size > 0) + { + scriptOpReturn << vector(elem, elem + elem_size) << OP_DROP; + } + } + } + + + lpScript=new mc_Script; + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + SendMoneyToSeveralAddresses(addresses, 0, wtx, lpScript, scriptOpReturn,fromaddresses); + + delete lpDetailsScript; + delete lpScript; + + return wtx.GetHash().GetHex(); +} + +Value subscribe(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + if((mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "API is not supported with this wallet version. To get this functionality, run \"multichaind -walletdbversion=2 -rescan\" "); + } + + // Whether to perform rescan after import + bool fRescan = true; + if (params.size() > 1) + fRescan = params[1].get_bool(); + + vector inputEntities; + vector inputStrings; + if(params[0].type() == str_type) + { + inputStrings.push_back(params[0].get_str()); + } + else + { + inputStrings=ParseStringList(params[0]); + } + + for(int is=0;is<(int)inputStrings.size();is++) + { + mc_EntityDetails entity_to_subscribe; + Value param=inputStrings[is]; + ParseEntityIdentifier(param,&entity_to_subscribe, MC_ENT_TYPE_ANY); + inputEntities.push_back(entity_to_subscribe); + } + + bool fNewFound=false; + for(int is=0;is<(int)inputStrings.size();is++) + { + mc_EntityDetails* lpEntity; + lpEntity=&inputEntities[is]; + + mc_TxEntity entity; + if(lpEntity->GetEntityType() == MC_ENT_TYPE_STREAM) + { + entity.Zero(); + memcpy(entity.m_EntityID,lpEntity->GetTxID()+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + entity.m_EntityType=MC_TET_STREAM | MC_TET_CHAINPOS; + if(pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC) != MC_ERR_FOUND) + { + entity.m_EntityType=MC_TET_STREAM | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + fNewFound=true; + } + } + + if(lpEntity->GetEntityType() == MC_ENT_TYPE_ASSET) + { + entity.Zero(); + memcpy(entity.m_EntityID,lpEntity->GetShortRef(),mc_gState->m_NetworkParams->m_AssetRefSize); + entity.m_EntityType=MC_TET_ASSET | MC_TET_CHAINPOS; + if(pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC) != MC_ERR_FOUND) + { + entity.m_EntityType=MC_TET_ASSET | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + fNewFound=true; + } + } + } + + if (fRescan && fNewFound) + { + pwalletMain->ScanForWalletTransactions(chainActive.Genesis(), true, true); + } + + return Value::null; +} + + +Value unsubscribe(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + if((mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "API is not supported with this wallet version. To get this functionality, run \"multichaind -walletdbversion=2 -rescan\" "); + } + + + vector inputEntities; + vector inputStrings; + if(params[0].type() == str_type) + { + inputStrings.push_back(params[0].get_str()); + } + else + { + inputStrings=ParseStringList(params[0]); + } + + for(int is=0;is<(int)inputStrings.size();is++) + { + mc_EntityDetails entity_to_subscribe; + Value param=inputStrings[is]; + ParseEntityIdentifier(param,&entity_to_subscribe, MC_ENT_TYPE_ANY); + inputEntities.push_back(entity_to_subscribe); + } + + mc_Buffer *streams; + streams=new mc_Buffer; + streams->Initialize(sizeof(mc_TxEntity),sizeof(mc_TxEntity),MC_BUF_MODE_DEFAULT); + + + bool fNewFound=false; + for(int is=0;is<(int)inputStrings.size();is++) + { + mc_EntityDetails* lpEntity; + lpEntity=&inputEntities[is]; + + mc_TxEntity entity; + if(lpEntity->GetEntityType() == MC_ENT_TYPE_STREAM) + { + entity.Zero(); + memcpy(entity.m_EntityID,lpEntity->GetTxID()+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + entity.m_EntityType=MC_TET_STREAM | MC_TET_CHAINPOS; + streams->Add(&entity,NULL); + entity.m_EntityType=MC_TET_STREAM | MC_TET_TIMERECEIVED; + streams->Add(&entity,NULL); + entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_CHAINPOS; + streams->Add(&entity,NULL); + entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_TIMERECEIVED; + streams->Add(&entity,NULL); + entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_CHAINPOS; + streams->Add(&entity,NULL); + entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_TIMERECEIVED; + streams->Add(&entity,NULL); + fNewFound=true; + } + + if(lpEntity->GetEntityType() == MC_ENT_TYPE_ASSET) + { + entity.Zero(); + memcpy(entity.m_EntityID,lpEntity->GetShortRef(),mc_gState->m_NetworkParams->m_AssetRefSize); + entity.m_EntityType=MC_TET_ASSET | MC_TET_CHAINPOS; + streams->Add(&entity,NULL); + entity.m_EntityType=MC_TET_ASSET | MC_TET_TIMERECEIVED; + streams->Add(&entity,NULL); + fNewFound=true; + } + } + + if(fNewFound) + { + if(pwalletTxsMain->Unsubscribe(streams)) + { + delete streams; + throw JSONRPCError(RPC_INTERNAL_ERROR, string("Couldn't unsubscribe from stream")); + } + } + + delete streams; + return Value::null; +} + +Value getstreamitem(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 3) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + if((mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "API is not supported with this wallet version. For full streams functionality, run \"multichaind -walletdbversion=2 -rescan\" "); + } + + mc_EntityDetails stream_entity; + parseStreamIdentifier(params[0],&stream_entity); + + mc_TxEntityStat entStat; + entStat.Zero(); + memcpy(&entStat,stream_entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + entStat.m_Entity.m_EntityType=MC_TET_STREAM; + entStat.m_Entity.m_EntityType |= MC_TET_CHAINPOS; + + if(!pwalletTxsMain->FindEntity(&entStat)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Not subscribed to this stream"); + } + + + uint256 hash = ParseHashV(params[1], "parameter 2"); + + bool verbose=false; + + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + if(params[2].get_int()) + { + verbose=true; + } + } + if(params[2].type() == bool_type) + { + if(params[2].get_bool()) + { + verbose=true; + } + } + } + + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + + Object entry=StreamItemEntry(wtx,stream_entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,verbose); + + if(entry.size() == 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "This transaction does not contain items of this stream"); + } + + return entry; +} + +Value liststreamitems(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 5) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + if((mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "API is not supported with this wallet version. For full streams functionality, run \"multichaind -walletdbversion=2 -rescan\" "); + } + + mc_TxEntityStat entStat; + + mc_EntityDetails stream_entity; + parseStreamIdentifier(params[0],&stream_entity); + + int count,start; + bool verbose=false; + + if (params.size() > 1) + { + if(params[1].type() == int_type) + { + if(params[1].get_int()) + { + verbose=true; + } + } + if(params[1].type() == bool_type) + { + if(params[1].get_bool()) + { + verbose=true; + } + } + } + + count=10; + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + count=params[2].get_int(); + if(count < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + start=-count; + if (params.size() > 3) + { + if(params[3].type() == int_type) + { + start=params[3].get_int(); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid start")); + } + } + + bool fLocalOrdering = false; + if (params.size() > 4) + fLocalOrdering = params[4].get_bool(); + + entStat.Zero(); + memcpy(&entStat,stream_entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + entStat.m_Entity.m_EntityType=MC_TET_STREAM; + if(fLocalOrdering) + { + entStat.m_Entity.m_EntityType |= MC_TET_TIMERECEIVED; + } + else + { + entStat.m_Entity.m_EntityType |= MC_TET_CHAINPOS; + } + if(!pwalletTxsMain->FindEntity(&entStat)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Not subscribed to this stream"); + } + + mc_Buffer *entity_rows; + entity_rows=new mc_Buffer; + entity_rows->Initialize(MC_TDB_ENTITY_KEY_SIZE,MC_TDB_ROW_SIZE,MC_BUF_MODE_DEFAULT); + + mc_AdjustStartAndCount(&count,&start,entStat.m_LastPos); + + Array retArray; + pwalletTxsMain->GetList(&entStat.m_Entity,start+1,count,entity_rows); + + for(int i=0;iGetCount();i++) + { + mc_TxEntityRow *lpEntTx; + lpEntTx=(mc_TxEntityRow*)entity_rows->GetRow(i); + uint256 hash; + memcpy(&hash,lpEntTx->m_TxId,MC_TDB_TXID_SIZE); + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + Object entry=StreamItemEntry(wtx,stream_entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,verbose); + if(entry.size()) + { + retArray.push_back(entry); + } + } + + delete entity_rows; + + return retArray; +} + +void getSubKeyEntityFromKey(string str,mc_TxEntityStat entStat,mc_TxEntity *entity) +{ + if(str == "*") + { + return; + } + uint160 key_string_hash; + uint160 stream_subkey_hash; + key_string_hash=Hash160(str.begin(),str.end()); + mc_GetCompoundHash160(&stream_subkey_hash,entStat.m_Entity.m_EntityID,&key_string_hash); + memcpy(entity->m_EntityID,&stream_subkey_hash,MC_TDB_ENTITY_ID_SIZE); + entity->m_EntityType=entStat.m_Entity.m_EntityType | MC_TET_SUBKEY; +} + +void getSubKeyEntityFromPublisher(string str,mc_TxEntityStat entStat,mc_TxEntity *entity) +{ + if(str == "*") + { + return; + } + uint160 stream_subkey_hash; + CBitcoinAddress address(str); + if (!address.IsValid()) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + } + CTxDestination dest=address.Get(); + CKeyID *lpKeyID=boost::get (&dest); + CScriptID *lpScriptID=boost::get (&dest); + + + if ((lpKeyID == NULL) && (lpScriptID == NULL) ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + } + + if(lpKeyID) + { + mc_GetCompoundHash160(&stream_subkey_hash,entStat.m_Entity.m_EntityID,lpKeyID); + } + else + { + mc_GetCompoundHash160(&stream_subkey_hash,entStat.m_Entity.m_EntityID,lpScriptID); + } + + memcpy(entity->m_EntityID,&stream_subkey_hash,MC_TDB_ENTITY_ID_SIZE); + entity->m_EntityType=entStat.m_Entity.m_EntityType | MC_TET_SUBKEY; +} + +Value liststreamkeyitems(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 6) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + if((mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "API is not supported with this wallet version. For full streams functionality, run \"multichaind -walletdbversion=2 -rescan\" "); + } + + if(params[1].get_str() == "*") + { + int count=0; + Array ext_params; + BOOST_FOREACH(const Value& value, params) + { + if(count != 1) + { + ext_params.push_back(value); + } + count++; + } + + return liststreamitems(ext_params,fHelp); + } + + mc_TxEntityStat entStat; + mc_TxEntity entity; + + mc_EntityDetails stream_entity; + parseStreamIdentifier(params[0],&stream_entity); + + int count,start; + bool verbose=false; + + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + if(params[2].get_int()) + { + verbose=true; + } + } + if(params[2].type() == bool_type) + { + if(params[2].get_bool()) + { + verbose=true; + } + } + } + + count=10; + if (params.size() > 3) + { + if(params[3].type() == int_type) + { + count=params[3].get_int(); + if(count < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + start=-count; + if (params.size() > 4) + { + if(params[4].type() == int_type) + { + start=params[4].get_int(); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid start")); + } + } + + bool fLocalOrdering = false; + if (params.size() > 5) + fLocalOrdering = params[5].get_bool(); + + entStat.Zero(); + memcpy(&entStat,stream_entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + entStat.m_Entity.m_EntityType=MC_TET_STREAM_KEY; + if(fLocalOrdering) + { + entStat.m_Entity.m_EntityType |= MC_TET_TIMERECEIVED; + } + else + { + entStat.m_Entity.m_EntityType |= MC_TET_CHAINPOS; + } + if(!pwalletTxsMain->FindEntity(&entStat)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Not subscribed to this stream"); + } + + getSubKeyEntityFromKey(params[1].get_str(),entStat,&entity); + + + mc_Buffer *entity_rows; + entity_rows=new mc_Buffer; + entity_rows->Initialize(MC_TDB_ENTITY_KEY_SIZE,MC_TDB_ROW_SIZE,MC_BUF_MODE_DEFAULT); + + mc_AdjustStartAndCount(&count,&start,pwalletTxsMain->GetListSize(&entity,entStat.m_Generation,NULL)); + + Array retArray; + pwalletTxsMain->GetList(&entity,entStat.m_Generation,start+1,count,entity_rows); + + for(int i=0;iGetCount();i++) + { + mc_TxEntityRow *lpEntTx; + lpEntTx=(mc_TxEntityRow*)entity_rows->GetRow(i); + uint256 hash; + memcpy(&hash,lpEntTx->m_TxId,MC_TDB_TXID_SIZE); + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + Object entry=StreamItemEntry(wtx,stream_entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,verbose); + if(entry.size()) + { + retArray.push_back(entry); + } + } + + delete entity_rows; + + return retArray; +} + + +Value liststreampublisheritems(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 6) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + if((mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "API is not supported with this wallet version. For full streams functionality, run \"multichaind -walletdbversion=2 -rescan\" "); + } + + if(params[1].get_str() == "*") + { + int count=0; + Array ext_params; + BOOST_FOREACH(const Value& value, params) + { + if(count != 1) + { + ext_params.push_back(value); + } + count++; + } + + return liststreamitems(ext_params,fHelp); + } + + mc_TxEntityStat entStat; + mc_TxEntity entity; + uint160 stream_subkey_hash; + + mc_EntityDetails stream_entity; + parseStreamIdentifier(params[0],&stream_entity); + + int count,start; + bool verbose=false; + + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + if(params[2].get_int()) + { + verbose=true; + } + } + if(params[2].type() == bool_type) + { + if(params[2].get_bool()) + { + verbose=true; + } + } + } + + count=10; + if (params.size() > 3) + { + if(params[3].type() == int_type) + { + count=params[3].get_int(); + if(count < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + start=-count; + if (params.size() > 4) + { + if(params[4].type() == int_type) + { + start=params[4].get_int(); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid start")); + } + } + + bool fLocalOrdering = false; + if (params.size() > 5) + fLocalOrdering = params[5].get_bool(); + + entStat.Zero(); + memcpy(&entStat,stream_entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + entStat.m_Entity.m_EntityType=MC_TET_STREAM_PUBLISHER; + if(fLocalOrdering) + { + entStat.m_Entity.m_EntityType |= MC_TET_TIMERECEIVED; + } + else + { + entStat.m_Entity.m_EntityType |= MC_TET_CHAINPOS; + } + if(!pwalletTxsMain->FindEntity(&entStat)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Not subscribed to this stream"); + } + + getSubKeyEntityFromPublisher(params[1].get_str(),entStat,&entity); + + mc_Buffer *entity_rows; + entity_rows=new mc_Buffer; + entity_rows->Initialize(MC_TDB_ENTITY_KEY_SIZE,MC_TDB_ROW_SIZE,MC_BUF_MODE_DEFAULT); + + mc_AdjustStartAndCount(&count,&start,pwalletTxsMain->GetListSize(&entity,entStat.m_Generation,NULL)); + + Array retArray; + pwalletTxsMain->GetList(&entity,entStat.m_Generation,start+1,count,entity_rows); + + for(int i=0;iGetCount();i++) + { + mc_TxEntityRow *lpEntTx; + lpEntTx=(mc_TxEntityRow*)entity_rows->GetRow(i); + uint256 hash; + memcpy(&hash,lpEntTx->m_TxId,MC_TDB_TXID_SIZE); + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + Object entry=StreamItemEntry(wtx,stream_entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,verbose); + if(entry.size()) + { + retArray.push_back(entry); + } + } + + delete entity_rows; + + return retArray; +} + +bool IsAllowedMapMode(string mode) +{ + if(mode == "list") return true; + if(mode == "all") return true; + return false; +} + +//Value liststreammap_operation(mc_TxEntity *parent_entity,mc_TxEntity *subkey_entity,string subkey_string,int count, int start, string mode) +Value liststreammap_operation(mc_TxEntity *parent_entity,vector& inputEntities,vector& inputStrings,int count, int start, string mode) +{ + mc_TxEntity entity; + mc_TxEntityStat entStat; + Array retArray; + mc_Buffer *entity_rows; + mc_TxEntityRow erow; + uint160 stream_subkey_hash; + int row,enitity_count; + + entity_rows=NULL; + enitity_count=inputEntities.size(); + if(enitity_count == 0) + { + mc_AdjustStartAndCount(&count,&start,pwalletTxsMain->GetListSize(parent_entity,NULL)); + entity_rows=new mc_Buffer; + entity_rows->Initialize(MC_TDB_ENTITY_KEY_SIZE,MC_TDB_ROW_SIZE,MC_BUF_MODE_DEFAULT); + pwalletTxsMain->GetList(parent_entity,start+1,count,entity_rows); + enitity_count=entity_rows->GetCount(); + } + else + { + mc_AdjustStartAndCount(&count,&start,enitity_count); + enitity_count=count; + } + + entStat.Zero(); + if(enitity_count) + { + memcpy(&entStat,parent_entity,sizeof(mc_TxEntity)); + pwalletTxsMain->FindEntity(&entStat); + } + + for(int i=0;iGetRow(i); + key_string=pwalletTxsMain->GetSubKey(lpEntTx->m_TxId, NULL,NULL); + entity.Zero(); + mc_GetCompoundHash160(&stream_subkey_hash,parent_entity->m_EntityID,lpEntTx->m_TxId); + memcpy(entity.m_EntityID,&stream_subkey_hash,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=parent_entity->m_EntityType | MC_TET_SUBKEY; + } + else + { + memcpy(&entity,&(inputEntities[i+start]),sizeof(mc_TxEntity)); + key_string=inputStrings[i+start]; + } + + int total,confirmed; + total=pwalletTxsMain->GetListSize(&entity,entStat.m_Generation,&confirmed); + + Object all_entry; + int shift=total-1; + if(shift == 0) + { + shift=1; + } + if((parent_entity->m_EntityType & MC_TET_TYPE_MASK) == MC_TET_STREAM_PUBLISHER) + { + all_entry.push_back(Pair("publisher", key_string)); + } + else + { + all_entry.push_back(Pair("key", key_string)); + } + all_entry.push_back(Pair("items", total)); + all_entry.push_back(Pair("confirmed", confirmed)); + + if(mode == "all") + { + for(row=1;row<=total;row+=shift) + { + if( ( (row == 1) && (mode != "last") ) || ( (row == total) && (mode != "first") ) ) + { + erow.Zero(); + memcpy(&erow.m_Entity,&entity,sizeof(mc_TxEntity)); + erow.m_Generation=entStat.m_Generation; + erow.m_Pos=row; + + if(pwalletTxsMain->GetRow(&erow) == 0) + { + uint256 hash; + memcpy(&hash,erow.m_TxId,MC_TDB_TXID_SIZE); + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + + Value item_value; + + item_value=StreamItemEntry(wtx,parent_entity->m_EntityID,true); + if(row == 1) + { + all_entry.push_back(Pair("first", item_value)); + } + if(row == total) + { + all_entry.push_back(Pair("last", item_value)); + } + } + } + } + } + retArray.push_back(all_entry); + } + + if(entity_rows) + { + delete entity_rows; + } + + return retArray; +} + +Value liststreamkeys_or_publishers(const Array& params,bool is_publishers) +{ + mc_TxEntity entity; + mc_TxEntityStat entStat; + + mc_EntityDetails stream_entity; + + parseStreamIdentifier(params[0],&stream_entity); + + string mode="list"; + + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + if(params[2].get_int()) + { + mode="all"; + } + } + if(params[2].type() == bool_type) + { + if(params[2].get_bool()) + { + mode="all"; + } + } + } + + int count,start; + count=2147483647; + if (params.size() > 3) + { + if(params[3].type() == int_type) + { + count=params[3].get_int(); + if(count < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + start=-count; + if (params.size() > 4) + { + if(params[4].type() == int_type) + { + start=params[4].get_int(); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid start")); + } + } + + bool fLocalOrdering = false; + if (params.size() > 5) + fLocalOrdering = params[5].get_bool(); + + entStat.Zero(); + memcpy(&entStat,stream_entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + if(is_publishers) + { + entStat.m_Entity.m_EntityType=MC_TET_STREAM_PUBLISHER; + } + else + { + entStat.m_Entity.m_EntityType=MC_TET_STREAM_KEY; + } + if(fLocalOrdering) + { + entStat.m_Entity.m_EntityType |= MC_TET_TIMERECEIVED; + } + else + { + entStat.m_Entity.m_EntityType |= MC_TET_CHAINPOS; + } + + if(!pwalletTxsMain->FindEntity(&entStat)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Not subscribed to this stream"); + } + + vector inputStrings; + vector inputEntities; + + if(params.size() > 1) + { + if(!is_publishers && (params[1].type() == str_type) ) + { + inputStrings.push_back(params[1].get_str()); + } + else + { + inputStrings=ParseStringList(params[1]); + if(inputStrings.size() == 0) + { + Array retArray; + return retArray; + } + } + bool take_it=true; + if( (inputStrings.size() == 1) && (inputStrings[0] == "*") ) + { + take_it=false; + } + if(take_it) + { + for(int is=0;is<(int)inputStrings.size();is++) + { + string str=inputStrings[is]; + entity.Zero(); + + if(is_publishers) + { + getSubKeyEntityFromPublisher(str,entStat,&entity); + } + else + { + getSubKeyEntityFromKey(str,entStat,&entity); + } + inputEntities.push_back(entity); + } + } + } + + return liststreammap_operation(&(entStat.m_Entity),inputEntities,inputStrings,count,start,mode); +} + +Value liststreamkeys(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 6) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + if((mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "API is not supported with this wallet version. For full streams functionality, run \"multichaind -walletdbversion=2 -rescan\" "); + } + + return liststreamkeys_or_publishers(params,false); +} + +Value liststreampublishers(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 6) + throw runtime_error("Help message not found\n"); + if((mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "API is not supported with this wallet version. For full streams functionality, run \"multichaind -walletdbversion=2 -rescan\" "); + } + + if(mc_gState->m_Features->Streams() == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, string("API is not supported for this protocol version")); + } + + return liststreamkeys_or_publishers(params,true); +} + diff --git a/src/rpc/rpcutils.cpp b/src/rpc/rpcutils.cpp new file mode 100644 index 00000000..63791ee7 --- /dev/null +++ b/src/rpc/rpcutils.cpp @@ -0,0 +1,2642 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "rpc/rpcutils.h" +#ifdef ENABLE_WALLET +#include "wallet/wallet.h" +#endif + +#include "utils/util.h" +#include "utils/utilmoneystr.h" +#include "wallet/wallettxs.h" + +#include + +using namespace std; +using namespace json_spirit; + + +CScript RemoveOpDropsIfNeeded(const CScript& scriptInput) +{ + if (!GetBoolArg("-hideknownopdrops", false)) + { + return scriptInput; + } + + return scriptInput.RemoveOpDrops(); +} + +bool CoinSparkAssetRefDecode(unsigned char *bin, const char* string, const size_t stringLen) +{ + char buffer[1024]; + int txIDPrefixInteger; + long long blockNum, txOffset; + + if(stringLen <= 0) + return false; + + if (stringLen>=sizeof(buffer)) + return false; + + memcpy(buffer, string, stringLen); + buffer[stringLen]=0; // copy to our buffer and null terminate to allow scanf + + if (strchr(buffer, '+')) // special check for '+' character which would be accepted by sscanf() below + return false; + + if (sscanf(buffer, "%lld-%lld-%d", &blockNum, &txOffset, &txIDPrefixInteger)!=3) + return false; + + if ( (txIDPrefixInteger<0) || (txIDPrefixInteger>0xFFFF) ) + return false; + + mc_PutLE(bin+0,&blockNum,4); + mc_PutLE(bin+4,&txOffset,8); + bin[8]=(unsigned char)(txIDPrefixInteger%256); + bin[9]=(unsigned char)(txIDPrefixInteger/256); + + return true; +} + +int ParseAssetKey(const char* asset_key,unsigned char *txid,unsigned char *asset_ref,char *name,int *multiple,int *type,int entity_type) +{ + int ret=0; + int size=strlen(asset_key); + unsigned char buf[MC_AST_ASSET_REF_SIZE]; + mc_EntityDetails entity; + const unsigned char *ptr; + + if(size == 0) + { + return -5; + } + + uint256 hash=0; + if(size == 64) + { + if(type) + { + *type=1; + } + hash.SetHex(asset_key); + if(!mc_gState->m_Assets->FindEntityByTxID(&entity,(unsigned char*)&hash)) + { + ret=-1; + } + else + { + if( (entity_type != MC_ENT_TYPE_ANY) && ((int)entity.GetEntityType() != entity_type) ) + { + ret=-1; + } + if(entity.IsFollowOn()) + { + if(!mc_gState->m_Assets->FindEntityByFollowOn(&entity,(unsigned char*)&hash)) + { + ret=-1; + } + } + } + } + else + { + if(size<=MC_ENT_MAX_NAME_SIZE) + { + if(CoinSparkAssetRefDecode(buf,asset_key,size)) + { + if(type) + { + *type=2; + } + if(!mc_gState->m_Assets->FindEntityByRef(&entity,buf)) + { + ret=-2; + } + else + { + if( (entity_type != MC_ENT_TYPE_ANY) && ((int)entity.GetEntityType() != entity_type) ) + { + ret=-2; + } + } + } + else + { + if(type) + { + *type=3; + } + if(!mc_gState->m_Assets->FindEntityByName(&entity,(char*)asset_key)) + { + ret=-3; + } + else + { + if( (entity_type != MC_ENT_TYPE_ANY) && ((int)entity.GetEntityType() != entity_type) ) + { + ret=-3; + } + } + } + } + else + { + ret=-4; + } + } + + if(ret == 0) + { + if(txid) + { + memcpy(txid,entity.GetTxID(),32); + } + ptr=entity.GetRef(); +// if((int)mc_GetLE((unsigned char*)ptr+4,4)<=0) + if(entity.IsUnconfirmedGenesis()) + { + ret=1; + } + if(asset_ref) + { + memcpy(asset_ref,ptr,MC_AST_ASSET_REF_SIZE); + } + if(name) + { + strcpy(name,entity.GetName()); + } + if(multiple) + { + *multiple=entity.GetAssetMultiple(); + } + } + + + return ret; +} + +int ParseAssetKeyToFullAssetRef(const char* asset_key,unsigned char *full_asset_ref,int *multiple,int *type,int entity_type) +{ + int ret; + if(mc_gState->m_Features->ShortTxIDAsAssetRef()) + { + unsigned char txid[MC_ENT_KEY_SIZE]; + ret=ParseAssetKey(asset_key,txid,NULL,NULL,multiple,type,entity_type); + if(ret == 1) + { + ret=0; + } + memcpy(full_asset_ref+MC_AST_SHORT_TXID_OFFSET,txid+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + + mc_SetABRefType(full_asset_ref,MC_AST_ASSET_REF_TYPE_SHORT_TXID); + } + else + { + ret=ParseAssetKey(asset_key,NULL,full_asset_ref,NULL,multiple,type,entity_type); + } + return ret; +} + +Array AddressEntries(const CTxIn& txin,txnouttype& typeRet,mc_Script *lpScript) +{ + Array addrs; + const CScript& script2 = txin.scriptSig; + CScript::const_iterator pc2 = script2.begin(); + + lpScript->Clear(); + lpScript->SetScript((unsigned char*)(&pc2[0]),(size_t)(script2.end()-pc2),MC_SCR_TYPE_SCRIPTSIGRAW); + + if(lpScript->GetNumElements() < 2) + { + typeRet=TX_PUBKEY; + return addrs; + } + + size_t elem_size; + const unsigned char *elem; + + typeRet=TX_PUBKEYHASH; + elem = lpScript->GetData(0,&elem_size); + if(elem_size == 1) + { + elem = lpScript->GetData(lpScript->GetNumElements()-1,&elem_size); + if( (elem[0]<=0x50) || (elem[0]>0x60)) // it is not redeem script + { + typeRet=TX_MULTISIG; // whatever it is there is no address here + return addrs; + } + typeRet=TX_SCRIPTHASH; + } + else + { + elem = lpScript->GetData(lpScript->GetNumElements()-1,&elem_size); + } + + uint160 hash=Hash160(elem,elem+elem_size); + CBitcoinAddress address; + + if(typeRet == TX_PUBKEYHASH) + { + address=CBitcoinAddress(CKeyID(hash)); + } + if(typeRet == TX_SCRIPTHASH) + { + address=CBitcoinAddress(CScriptID(hash)); + } + + addrs.push_back(address.ToString()); + return addrs; +} + +Array AddressEntries(const CTxOut& txout,txnouttype& typeRet) +{ + Array addrs; + CBitcoinAddress address; + int nRequiredRet; + vector addressRets; + + if(ExtractDestinations(txout.scriptPubKey,typeRet,addressRets,nRequiredRet)) + { + for(int d=0;d<(int)addressRets.size();d++) + { + CKeyID *lpKeyID=boost::get (&addressRets[d]); + CScriptID *lpScriptID=boost::get (&addressRets[d]); + if( (lpKeyID != NULL) || (lpScriptID != NULL) ) + { + if(lpKeyID != NULL) + { + address=CBitcoinAddress(*lpKeyID); + } + if(lpScriptID != NULL) + { + address=CBitcoinAddress(*lpScriptID); + } + addrs.push_back(address.ToString()); + } + } + } + + return addrs; +} + +Value PermissionForFieldEntry(mc_EntityDetails *lpEntity) +{ + if(lpEntity->GetEntityType()) + { + Object entObject; + unsigned char *ptr; + + if(lpEntity->GetEntityType() == MC_ENT_TYPE_ASSET) + { + entObject.push_back(Pair("type", "asset")); + ptr=(unsigned char *)lpEntity->GetName(); + if(ptr && strlen((char*)ptr)) + { + entObject.push_back(Pair("name", string((char*)ptr))); + } + ptr=(unsigned char *)lpEntity->GetRef(); + string assetref=""; + if(lpEntity->IsUnconfirmedGenesis()) + { + Value null_value; + entObject.push_back(Pair("assetref",null_value)); + } + else + { + assetref += itostr((int)mc_GetLE(ptr,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(ptr+4,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(ptr+8,2)); + entObject.push_back(Pair("assetref", assetref)); + } + } + if(lpEntity->GetEntityType() == MC_ENT_TYPE_STREAM) + { + entObject.push_back(Pair("type", "stream")); + ptr=(unsigned char *)lpEntity->GetName(); + if(ptr && strlen((char*)ptr)) + { + entObject.push_back(Pair("name", string((char*)ptr))); + } + ptr=(unsigned char *)lpEntity->GetRef(); + string streamref=""; + if(lpEntity->IsUnconfirmedGenesis()) + { + Value null_value; + entObject.push_back(Pair("streamref",null_value)); + } + else + { + if((int)mc_GetLE(ptr,4)) + { + streamref += itostr((int)mc_GetLE(ptr,4)); + streamref += "-"; + streamref += itostr((int)mc_GetLE(ptr+4,4)); + streamref += "-"; + streamref += itostr((int)mc_GetLE(ptr+8,2)); + } + else + { + streamref="0-0-0"; + } + entObject.push_back(Pair("streamref", streamref)); + } + } + return entObject; + } + + return Value::null; +} + +Array PermissionEntries(const CTxOut& txout,mc_Script *lpScript,bool fLong) +{ + Array results; + + uint32_t type,from,to,timestamp,full_type; + unsigned char short_txid[MC_AST_SHORT_TXID_SIZE]; + mc_EntityDetails entity; + + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return results; + } + + const CScript& script1 = txout.scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + lpScript->Clear(); + lpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + entity.Zero(); // Permission processing + for (int e = 0; e < lpScript->GetNumElements(); e++) + { + lpScript->SetElement(e); + if(lpScript->GetEntity(short_txid) == 0) + { + if(entity.GetEntityType()) + { + goto exitlbl; // invalid transaction + } + if(mc_gState->m_Assets->FindEntityByShortTxID(&entity,short_txid) == 0) + { + goto exitlbl; // invalid transaction + } + } + else // Not entity element + { + if(lpScript->GetPermission(&type,&from,&to,×tamp) == 0) + { + Object entry; + entry.push_back(Pair("for", PermissionForFieldEntry(&entity))); + full_type=mc_gState->m_Permissions->GetPossiblePermissionTypes(entity.GetEntityType()); + if(full_type & MC_PTP_CONNECT)entry.push_back(Pair("connect", (type & MC_PTP_CONNECT) ? true : false)); + if(full_type & MC_PTP_SEND)entry.push_back(Pair("send", (type & MC_PTP_SEND) ? true : false)); + if(full_type & MC_PTP_RECEIVE)entry.push_back(Pair("receive", (type & MC_PTP_RECEIVE) ? true : false)); + if(mc_gState->m_Features->Streams()) + { + if(full_type & MC_PTP_WRITE)entry.push_back(Pair("write", (type & MC_PTP_WRITE) ? true : false)); + if(full_type & MC_PTP_CREATE)entry.push_back(Pair("create", (type & MC_PTP_CREATE) ? true : false)); + } + if(full_type & MC_PTP_ISSUE)entry.push_back(Pair("issue", (type & MC_PTP_ISSUE) ? true : false)); + if(full_type & MC_PTP_MINE)entry.push_back(Pair("mine", (type & MC_PTP_MINE) ? true : false)); + if(full_type & MC_PTP_ADMIN)entry.push_back(Pair("admin", (type & MC_PTP_ADMIN) ? true : false)); + if(mc_gState->m_Features->ActivatePermission()) + { + if(full_type & MC_PTP_ACTIVATE)entry.push_back(Pair("activate", (type & MC_PTP_ACTIVATE) ? true : false)); + } + entry.push_back(Pair("startblock",(int64_t)from)); + entry.push_back(Pair("endblock",(int64_t)to)); + entry.push_back(Pair("timestamp",(int64_t)timestamp)); + if(fLong) + { + txnouttype typeRet; + entry.push_back(Pair("addresses", AddressEntries(txout,typeRet))); + } + results.push_back(entry); + } + else + { + goto exitlbl; // invalid transaction + } + entity.Zero(); + } + } + +exitlbl: + + return results; +} + + + +Object StreamEntry(const unsigned char *txid,uint32_t output_level) +{ +// output_level constants +// 0x0001 type +// 0x0002 txid +// 0x0004 open/details +// 0x0008 subscribed/synchronized +// 0x0010 stats +// 0x0020 creators + + Object entry; + mc_EntityDetails entity; + mc_TxEntity tmp_entity; + mc_TxEntityStat entStat; + unsigned char *ptr; + + if(txid == NULL) + { + entry.push_back(Pair("streamref", "")); + return entry; + } + + uint256 hash=*(uint256*)txid; + if(mc_gState->m_Assets->FindEntityByTxID(&entity,txid)) + { + ptr=(unsigned char *)entity.GetName(); + + if(output_level & 0x001) + { + entry.push_back(Pair("type", "stream")); + } + + if(ptr && strlen((char*)ptr)) + { + entry.push_back(Pair("name", string((char*)ptr))); + } + if(output_level & 0x002) + { + entry.push_back(Pair("createtxid", hash.GetHex())); + } + ptr=(unsigned char *)entity.GetRef(); + string streamref=""; + if(entity.IsUnconfirmedGenesis()) + { + Value null_value; + entry.push_back(Pair("streamref",null_value)); + } + else + { + if((int)mc_GetLE(ptr,4)) + { + streamref += itostr((int)mc_GetLE(ptr,4)); + streamref += "-"; + streamref += itostr((int)mc_GetLE(ptr+4,4)); + streamref += "-"; + streamref += itostr((int)mc_GetLE(ptr+8,2)); + } + else + { + streamref="0-0-0"; + } + entry.push_back(Pair("streamref", streamref)); + } + + if(output_level & 0x0004) + { + if(entity.AnyoneCanWrite()) + { + entry.push_back(Pair("open",true)); + } + else + { + entry.push_back(Pair("open",false)); + } + } + + + size_t value_size; + int64_t offset,new_offset; + uint32_t value_offset; + const unsigned char *ptr; + + ptr=entity.GetScript(); + + Object fields; + Array openers; + if(output_level & 0x0004) + { + offset=0; + while(offset>=0) + { + new_offset=entity.NextParam(offset,&value_offset,&value_size); + if(value_offset > 0) + { + if(ptr[offset]) + { + string param_name((char*)ptr+offset); + string param_value((char*)ptr+value_offset,(char*)ptr+value_offset+value_size); + fields.push_back(Pair(param_name, param_value)); + } + else + { + if(ptr[offset+1] == MC_ENT_SPRM_ISSUER) + { + if(value_size == 24) + { + unsigned char tptr[4]; + memcpy(tptr,ptr+value_offset+sizeof(uint160),4); + if(mc_GetLE(tptr,4) & MC_PFL_IS_SCRIPTHASH) + { + openers.push_back(CBitcoinAddress(*(CScriptID*)(ptr+value_offset)).ToString()); + } + else + { + openers.push_back(CBitcoinAddress(*(CKeyID*)(ptr+value_offset)).ToString()); + } + } + } + } + } + offset=new_offset; + } + entry.push_back(Pair("details",fields)); + } + + if(output_level & 0x0020) + { + entry.push_back(Pair("creators",openers)); + } + if(output_level & 0x0018) + { + entStat.Zero(); + memcpy(&entStat,entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + entStat.m_Entity.m_EntityType=MC_TET_STREAM | MC_TET_CHAINPOS; + if(pwalletTxsMain->FindEntity(&entStat)) + { + if(output_level & 0x0008) + { + entry.push_back(Pair("subscribed",true)); + if(entStat.m_Flags & MC_EFL_NOT_IN_SYNC) + { + entry.push_back(Pair("synchronized",false)); + } + else + { + entry.push_back(Pair("synchronized",true)); + } + } + if(output_level & 0x0010) + { + entry.push_back(Pair("items",(int)entStat.m_LastPos)); + entry.push_back(Pair("confirmed",(int)entStat.m_LastClearedPos)); + tmp_entity.Zero(); + memcpy(tmp_entity.m_EntityID,entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + tmp_entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_CHAINPOS; + entry.push_back(Pair("keys",pwalletTxsMain->GetListSize(&tmp_entity,NULL))); + tmp_entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_CHAINPOS; + entry.push_back(Pair("publishers",pwalletTxsMain->GetListSize(&tmp_entity,NULL))); + } + } + else + { + entry.push_back(Pair("subscribed",false)); + } + } + } + else + { + Value null_value; + if(output_level & 0x001) + { + entry.push_back(Pair("type", "stream")); + } + entry.push_back(Pair("name",null_value)); + if(output_level & 0x002) + { + entry.push_back(Pair("createtxid",null_value)); + } + entry.push_back(Pair("streamref", null_value)); + } + + return entry; +} + +Value OpReturnEntry(const unsigned char *elem,size_t elem_size,uint256 txid, int vout) +{ + string metadata=""; + Object metadata_object; + if((int)elem_size <= GetArg("-maxshowndata",MAX_OP_RETURN_SHOWN)) + { + metadata=HexStr(elem,elem+elem_size); + return metadata; + } + metadata_object.push_back(Pair("txid", txid.ToString())); + metadata_object.push_back(Pair("vout", vout)); + metadata_object.push_back(Pair("size", (int)elem_size)); + return metadata_object; +} + + + +Value DataItemEntry(const CTransaction& tx,int n,set & already_seen,uint32_t stream_output_level) +{ + Object entry; + Array publishers; + set publishers_set; + Array items; + const unsigned char *ptr; + unsigned char item_key[MC_ENT_MAX_ITEM_KEY_SIZE+1]; + int item_key_size; + Value item_value; + mc_EntityDetails entity; + uint256 hash; + + const CScript& script1 = tx.vout[n].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + if(mc_gState->m_TmpScript->IsOpReturnScript() == 0) + { + return Value::null; + } + + if(mc_gState->m_TmpScript->GetNumElements() == 0) + { + return Value::null; + } + + unsigned char short_txid[MC_AST_SHORT_TXID_SIZE]; + mc_gState->m_TmpScript->SetElement(0); + + if(mc_gState->m_TmpScript->GetEntity(short_txid)) + { + return Value::null; + } + + if(mc_gState->m_Assets->FindEntityByShortTxID(&entity,short_txid) == 0) + { + return Value::null; + } + + hash=*(uint256*)entity.GetTxID(); + + if(already_seen.find(hash) != already_seen.end()) + { + return Value::null; + } + + mc_gState->m_TmpScript->SetElement(1); + // Should be spkk + if(mc_gState->m_TmpScript->GetItemKey(item_key,&item_key_size)) // Item key + { + return Value::null; + } + item_key[item_key_size]=0; + + size_t elem_size; + const unsigned char *elem; + + elem = mc_gState->m_TmpScript->GetData(2,&elem_size); + item_value=OpReturnEntry(elem,elem_size,tx.GetHash(),n); + + already_seen.insert(hash); + + publishers_set.clear(); + for (int i = 0; i < (int)tx.vin.size(); ++i) + { + int op_addr_offset,op_addr_size,is_redeem_script,sighash_type; + + const CScript& script2 = tx.vin[i].scriptSig; + CScript::const_iterator pc2 = script2.begin(); + + ptr=mc_ExtractAddressFromInputScript((unsigned char*)(&pc2[0]),(int)(script2.end()-pc2),&op_addr_offset,&op_addr_size,&is_redeem_script,&sighash_type,0); + if(ptr) + { + if( (sighash_type == SIGHASH_ALL) || ( (sighash_type == SIGHASH_SINGLE) && (i == n) ) ) + { + uint160 publisher_hash=Hash160(ptr+op_addr_offset,ptr+op_addr_offset+op_addr_size); + if(publishers_set.count(publisher_hash) == 0) + { + publishers_set.insert(publisher_hash); + if(is_redeem_script) + { + publishers.push_back(CBitcoinAddress((CScriptID)publisher_hash).ToString()); + } + else + { + publishers.push_back(CBitcoinAddress((CKeyID)publisher_hash).ToString()); + } + } + } + } + } + + entry=StreamEntry((unsigned char*)&hash,stream_output_level); + entry.push_back(Pair("publishers", publishers)); + entry.push_back(Pair("key", strprintf("%s",item_key))); + entry.push_back(Pair("data", item_value)); + + return entry; +} + + +// output level +// 1 - minimal - name, asset-ref, qty +// 3 - txout - name, asset-ref, geneis txid, qty,raw +// 9 - full - all fields + +Object AssetEntry(const unsigned char *txid,int64_t quantity,int output_level) +{ + Object entry; + mc_EntityDetails entity; + mc_EntityDetails genesis_entity; + mc_TxEntityStat entStat; + unsigned char *ptr; + + if(txid == NULL) + { + entry.push_back(Pair("assetref", "")); + entry.push_back(Pair("qty", ValueFromAmount(quantity))); + entry.push_back(Pair("raw", quantity)); + return entry; + } + + uint256 hash=*(uint256*)txid; + + if(mc_gState->m_Assets->FindEntityByTxID(&entity,txid)) + { + if(entity.IsFollowOn()) + { + mc_gState->m_Assets->FindEntityByFollowOn(&genesis_entity,txid); + } + else + { + memcpy(&genesis_entity,&entity,sizeof(mc_EntityDetails)); + } + + ptr=(unsigned char *)genesis_entity.GetName(); + if(ptr && strlen((char*)ptr)) + { + entry.push_back(Pair("name", string((char*)ptr))); + } + if((output_level >= 3) && (output_level != 7) ) + { + entry.push_back(Pair("issuetxid", hash.GetHex())); + } + ptr=(unsigned char *)genesis_entity.GetRef(); + string assetref=""; + if(genesis_entity.IsUnconfirmedGenesis()) + { + Value null_value; + entry.push_back(Pair("assetref",null_value)); + } + else + { + assetref += itostr((int)mc_GetLE(ptr,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(ptr+4,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(ptr+8,2)); + entry.push_back(Pair("assetref", assetref)); + } + + size_t value_size; + int64_t offset,new_offset; + uint32_t value_offset; + uint64_t multiple=1; + const unsigned char *ptr; + double units=1.; + + ptr=entity.GetScript(); + multiple=genesis_entity.GetAssetMultiple(); + units= 1./(double)multiple; + if(output_level >= 5) + { + if(!entity.IsFollowOn()) + { + entry.push_back(Pair("multiple", multiple)); + entry.push_back(Pair("units",units)); + if(mc_gState->m_Features->FollowOnIssues()) + { + if(entity.AllowedFollowOns()) + { + entry.push_back(Pair("open",true)); + } + else + { + entry.push_back(Pair("open",false)); + } + } + else + { + entry.push_back(Pair("open",false)); + } + } + } + + Object fields; + if(output_level >= 5) + { + offset=0; + while(offset>=0) + { + new_offset=entity.NextParam(offset,&value_offset,&value_size); + if(value_offset > 0) + { + if(ptr[offset]) + { + if(ptr[offset] != 0xff) + { + if(mc_gState->m_Features->SpecialParamsInDetailsScript() || + ((strcmp((char*)ptr+offset,"multiple") != 0) && (strcmp((char*)ptr+offset,"name") != 0) )) + { + string param_name((char*)ptr+offset); + string param_value((char*)ptr+value_offset,(char*)ptr+value_offset+value_size); + fields.push_back(Pair(param_name, param_value)); + } + } + } + } + offset=new_offset; + } + + } + if(output_level >= 5) + { + entry.push_back(Pair("details",fields)); + } + + Array issues; + int64_t total=0; + if((output_level == 9) || mc_gState->m_Assets->HasFollowOns(txid)) + { + int64_t qty; + mc_Buffer *followons; + followons=mc_gState->m_Assets->GetFollowOns(txid); + for(int i=followons->GetCount()-1;i>=0;i--) + { + Object issue; + mc_EntityDetails followon; + if(mc_gState->m_Assets->FindEntityByTxID(&followon,followons->GetRow(i))) + { + qty=followon.GetQuantity(); + total+=qty; + if(output_level == 9) + { + issue.push_back(Pair("txid", ((uint256*)(followon.GetTxID()))->ToString().c_str())); + issue.push_back(Pair("qty", (double)qty*units)); + issue.push_back(Pair("raw", qty)); + + Object followon_fields; + Array followon_issuers; + + ptr=followon.GetScript(); + offset=0; + while(offset>=0) + { + new_offset=followon.NextParam(offset,&value_offset,&value_size); + if(value_offset > 0) + { + if(ptr[offset]) + { + if(ptr[offset] != 0xff) + { + if(mc_gState->m_Features->SpecialParamsInDetailsScript() || + ((strcmp((char*)ptr+offset,"multiple") != 0) && (strcmp((char*)ptr+offset,"name") != 0) )) + { + string param_name((char*)ptr+offset); + string param_value((char*)ptr+value_offset,(char*)ptr+value_offset+value_size); + followon_fields.push_back(Pair(param_name, param_value)); + } + } + } + else + { + if(ptr[offset+1] == MC_ENT_SPRM_ISSUER) + { + if(value_size == 20) + { + followon_issuers.push_back(CBitcoinAddress(*(CKeyID*)(ptr+value_offset)).ToString()); + } + if(value_size == 24) + { + unsigned char tptr[4]; + memcpy(tptr,ptr+value_offset+sizeof(uint160),4); + if(mc_GetLE(tptr,4) & MC_PFL_IS_SCRIPTHASH) + { + followon_issuers.push_back(CBitcoinAddress(*(CScriptID*)(ptr+value_offset)).ToString()); + } + else + { + followon_issuers.push_back(CBitcoinAddress(*(CKeyID*)(ptr+value_offset)).ToString()); + } + } + } + } + } + offset=new_offset; + } + + issue.push_back(Pair("details",followon_fields)); + issue.push_back(Pair("issuers",followon_issuers)); + issues.push_back(issue); + } + } + } + mc_gState->m_Assets->FreeEntityList(followons); + } + else + { + total=entity.GetQuantity(); + } + + + if( ((quantity < 0) && (output_level != 2)) || (output_level == 7) ) + { +// quantity=entity.GetQuantity(); + if(output_level == 7) + { + entry.push_back(Pair("qty", (double)entity.GetQuantity()*units)); + entry.push_back(Pair("raw", entity.GetQuantity())); + } + else + { + entry.push_back(Pair("issueqty", (double)total*units)); + entry.push_back(Pair("issueraw", total)); + } + } + else + { + entry.push_back(Pair("qty", (double)quantity*units)); + if(output_level >= 3) + { + entry.push_back(Pair("raw", quantity)); + } + } + + if( (output_level >= 8) && ((mc_gState->m_WalletMode & MC_WMD_TXS) != 0) ) + { + entStat.Zero(); + memcpy(&entStat,genesis_entity.GetShortRef(),mc_gState->m_NetworkParams->m_AssetRefSize); + entStat.m_Entity.m_EntityType=MC_TET_ASSET | MC_TET_CHAINPOS; + if(pwalletTxsMain->FindEntity(&entStat)) + { + entry.push_back(Pair("subscribed",true)); + if(entStat.m_Flags & MC_EFL_NOT_IN_SYNC) + { + entry.push_back(Pair("synchronized",false)); + } + else + { + entry.push_back(Pair("synchronized",true)); + } + entry.push_back(Pair("transactions",(int)entStat.m_LastPos)); + entry.push_back(Pair("confirmed",(int)entStat.m_LastClearedPos)); + } + else + { + entry.push_back(Pair("subscribed",false)); + } + } + + if(mc_gState->m_Features->FollowOnIssues()) + { + if(output_level == 9) + { + entry.push_back(Pair("issues",issues)); + } + } + } + else + { + Value null_value; + entry.push_back(Pair("name",null_value)); + entry.push_back(Pair("issuetxid",null_value)); + entry.push_back(Pair("assetref", null_value)); + entry.push_back(Pair("qty", null_value)); + entry.push_back(Pair("raw", quantity)); + } + + return entry; +} + + +string ParseRawOutputObject(Value param,CAmount& nAmount,mc_Script *lpScript, int *required) +{ + string strError=""; + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + mc_Buffer *lpBuffer; + mc_Buffer *lpFollowonBuffer; + lpBuffer=new mc_Buffer; + lpFollowonBuffer=new mc_Buffer; + int assets_per_opdrop=(MAX_STANDARD_TX_SIZE)/(mc_gState->m_NetworkParams->m_AssetRefSize+MC_AST_ASSET_QUANTITY_SIZE); + int32_t verify_level=-1; + int asset_error=0; + int multiple; + int64_t max_block=0xffffffff; + string asset_name; + string type_string; + nAmount=0; + + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + + mc_InitABufferDefault(lpBuffer); + mc_InitABufferDefault(lpFollowonBuffer); + + if(mc_gState->m_Features->VerifySizeOfOpDropElements()) + { + if(mc_gState->m_Features->VerifySizeOfOpDropElements()) + { + assets_per_opdrop=(mc_gState->m_NetworkParams->GetInt64Param("maxstdelementsize")-4)/(mc_gState->m_NetworkParams->m_AssetRefSize+MC_AST_ASSET_QUANTITY_SIZE); + } + } + + BOOST_FOREACH(const Pair& a, param.get_obj()) + { + if(a.value_.type() == obj_type) + { + bool parsed=false; + + if(!parsed && (a.name_ == "hidden_verify_level")) + { + BOOST_FOREACH(const Pair& d, a.value_.get_obj()) + { + bool field_parsed=false; + if(!field_parsed && (d.name_ == "value")) + { + verify_level=d.value_.get_int(); + field_parsed=true; + } + if(!field_parsed) + { + strError=string("Invalid field for object ") + a.name_ + string(": ") + d.name_; + goto exitlbl; + } + } + parsed=true; + } + + if(!parsed && (a.name_ == "issue")) + { + int64_t quantity=-1; + BOOST_FOREACH(const Pair& d, a.value_.get_obj()) + { + bool field_parsed=false; + if(!field_parsed && (d.name_ == "raw")) + { + if (d.value_.type() != null_type) + { + quantity=d.value_.get_int64(); + if(quantity<0) + { + strError=string("Negative value for issue raw qty"); + goto exitlbl; + } + } + else + { + strError=string("Invalid value for issue raw qty"); + goto exitlbl; + + } + field_parsed=true; + } + if(!field_parsed) + { + strError=string("Invalid field for object ") + a.name_ + string(": ") + d.name_; + goto exitlbl; + } + } + if(quantity < 0) + { + strError=string("Issue raw qty not specified"); + goto exitlbl; + } + lpScript->SetAssetGenesis(quantity); + parsed=true; + } + + if(!parsed && (a.name_ == "issuemore")) + { + int64_t quantity=-1; + asset_name=""; + BOOST_FOREACH(const Pair& d, a.value_.get_obj()) + { + bool field_parsed=false; + if(!field_parsed && (d.name_ == "raw")) + { + if (d.value_.type() != null_type) + { + quantity=d.value_.get_int64(); + if(quantity<0) + { + strError=string("Negative value for issuemore raw qty"); + goto exitlbl; + } + } + else + { + strError=string("Invalid value for issuemore raw qty"); + goto exitlbl; + + } + field_parsed=true; + } + if(!field_parsed && (d.name_ == "asset")) + { + if(d.value_.type() != null_type && !d.value_.get_str().empty()) + { + asset_name=d.value_.get_str(); + } + if(asset_name.size()) + { +// asset_error=ParseAssetKey(asset_name.c_str(),NULL,buf,NULL,&multiple,NULL, MC_ENT_TYPE_ASSET); + asset_error=ParseAssetKeyToFullAssetRef(asset_name.c_str(),buf,&multiple,NULL, MC_ENT_TYPE_ASSET); + if(asset_error) + { + goto exitlbl; + } + field_parsed=true; + } + } + if(!field_parsed) + { + strError=string("Invalid field for object ") + a.name_ + string(": ") + d.name_; + goto exitlbl; + } + } + if(asset_name.size() == 0) + { + strError=string("Issuemore asset not specified"); + goto exitlbl; + } + if(quantity < 0) + { + strError=string("Issuemore raw qty not specified"); + goto exitlbl; + } + if(lpFollowonBuffer->GetCount()) + { + if(verify_level & 0x0008) + { + if(memcmp(buf,lpFollowonBuffer->GetRow(0),MC_AST_ASSET_QUANTITY_OFFSET)) + { + strError=string("Issuemore for different assets"); + goto exitlbl; + } + } + lpFollowonBuffer->Clear(); + } + mc_SetABQuantity(buf,quantity); + lpFollowonBuffer->Add(buf); + lpScript->SetAssetQuantities(lpFollowonBuffer,MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON); + parsed=true; + } + + if(!parsed && (a.name_ == "permissions")) + { + uint32_t type,from,to,timestamp; + int64_t v; + mc_EntityDetails entity; + entity.Zero(); + + type_string=""; + type=0; + from=0; + to=4294967295U; + timestamp=mc_TimeNowAsUInt(); + + BOOST_FOREACH(const Pair& d, a.value_.get_obj()) + { + bool field_parsed=false; + + if(!field_parsed && (d.name_ == "for")) + { + entity.Zero(); + if(d.value_.type() != null_type && !d.value_.get_str().empty()) + { + ParseEntityIdentifier(d.value_,&entity, MC_ENT_TYPE_ANY); + } + field_parsed=true; + } + if(!field_parsed && (d.name_ == "type")) + { + if(d.value_.type() == str_type && !d.value_.get_str().empty()) + { + type_string=d.value_.get_str(); + } + else + { + strError=string("Invalid value for permission type"); + goto exitlbl; + + } + field_parsed=true; + } + if(!field_parsed && (d.name_ == "startblock")) + { + if (d.value_.type() != null_type) + { + + v=d.value_.get_int64(); + if(v<0) + { + strError=string("Negative value for permissions startblock"); + goto exitlbl; + } + if(v>max_block) + { + strError=string("Invalid value for permissions endblock"); + goto exitlbl; + } + from=v; + } + else + { + strError=string("Invalid value for permissions startblock"); + goto exitlbl; + + } + field_parsed=true; + } + if(!field_parsed && (d.name_ == "endblock")) + { + if (d.value_.type() != null_type) + { + v=d.value_.get_int64(); + if(v<0) + { + strError=string("Negative value for permissions endblock"); + goto exitlbl; + } + if(v>max_block) + { + strError=string("Invalid value for permissions endblock"); + goto exitlbl; + } + to=v; + } + else + { + strError=string("Invalid value for permissions endblock"); + goto exitlbl; + + } + field_parsed=true; + } + if(!field_parsed && (d.name_ == "timestamp")) + { + if (d.value_.type() != null_type) + { + timestamp=(uint32_t)d.value_.get_uint64(); + } + else + { + strError=string("Invalid value for permissions timestamp"); + goto exitlbl; + + } + field_parsed=true; + } + if(!field_parsed) + { + strError=string("Invalid field for object ") + a.name_ + string(": ") + d.name_; + goto exitlbl; + } + } + + if(type_string.size()) + { + type=mc_gState->m_Permissions->GetPermissionType(type_string.c_str(),entity.GetEntityType()); + if(entity.GetEntityType() == MC_ENT_TYPE_NONE) + { + if(required) + { + *required |= type; + } + } + if(type == 0) + { + strError=string("Invalid value for permission type: ") + type_string; + goto exitlbl; + } + } + + if(type == 0) + { + strError=string("Permission type not specified"); + goto exitlbl; + } + + if(entity.GetEntityType()) + { + lpScript->SetEntity(entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET); + } + lpScript->SetPermission(type,from,to,timestamp); + parsed=true; + } + + if(!parsed) + { + strError=string("Invalid object: ") + a.name_; + goto exitlbl; + } + } + else + { + if(a.name_.size()) + { + asset_name=a.name_; +// asset_error=ParseAssetKey(asset_name.c_str(),NULL,buf,NULL,&multiple,NULL, MC_ENT_TYPE_ASSET); + asset_error=ParseAssetKeyToFullAssetRef(asset_name.c_str(),buf,&multiple,NULL, MC_ENT_TYPE_ASSET); + if(asset_error) + { + goto exitlbl; + } + int64_t quantity = (int64_t)(a.value_.get_real() * multiple + 0.499999); + if(verify_level & 0x0010) + { + if(quantity < 0) + { + strError=string("Negative asset quantity"); + goto exitlbl; + } + } + mc_SetABQuantity(buf,quantity); + lpBuffer->Add(buf); + if(verify_level & 0x0001) + { + if(lpBuffer->GetCount() >= assets_per_opdrop) + { + lpScript->SetAssetQuantities(lpBuffer,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER); + lpBuffer->Clear(); + } + } + } + else + { + nAmount += AmountFromValue(a.value_); + } + } + } + + if(lpBuffer->GetCount()) + { + if(Params().RequireStandard()) + { + if(verify_level & 0x0002) + { + if(lpBuffer->GetCount() > assets_per_opdrop) + { + strError=string("Too many assets in one group"); + goto exitlbl; + } + } + } + lpScript->SetAssetQuantities(lpBuffer,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER); + lpBuffer->Clear(); + } + + if(verify_level & 0x0004) + { + if(Params().RequireStandard()) + { + if(lpScript->GetNumElements() > mc_gState->m_NetworkParams->GetInt64Param("maxstdopdropscount")) + { + strError=string("Too many objects in output"); + goto exitlbl; + } + } + } + +exitlbl: + + switch(asset_error) + { + case -1: + strError=string("Issue transaction with this txid not found: ")+asset_name; + break; + case -2: + strError=string("Issue transaction with this asset reference not found: ")+asset_name; + break; + case -3: + strError=string("Issue transaction with this name not found: ")+asset_name; + break; + case -4: + strError=string("Could not parse asset key: ")+asset_name; + break; + case 1: + strError=string("Unconfirmed asset: ")+asset_name; + break; + } + + delete lpBuffer; + delete lpFollowonBuffer; + + return strError; + +} + +string ParseRawOutputObject(Value param,CAmount& nAmount,mc_Script *lpScript) +{ + return ParseRawOutputObject(param,nAmount,lpScript,NULL); +} + +bool FindPreparedTxOut(CTxOut& txout,COutPoint outpoint,string& reason) +{ + { + LOCK(mempool.cs); // protect pool.mapNextTx + if (mempool.mapNextTx.count(outpoint)) + { + reason="Conflicts with in-memory transactions"; + return false; + } + } + + { + CCoinsView dummy; + CCoinsViewCache view(&dummy); + + { + LOCK(mempool.cs); + CCoinsViewMemPool viewMemPool(pcoinsTip, mempool); + view.SetBackend(viewMemPool); + + if (!view.HaveCoins(outpoint.hash)) + { + reason="Missing inputs"; + return false; + } + else + { + const CCoins* coins = view.AccessCoins(outpoint.hash); + if (!coins || !coins->IsAvailable(outpoint.n)) + { + reason="Input already spent"; + return false; + } + else + { + txout=coins->vout[outpoint.n]; + } + } + + view.GetBestBlock(); + + // we have all inputs cached now, so switch back to dummy, so we don't need to keep lock on mempool + view.SetBackend(dummy); + } + } + + return true; +} + +bool GetTxInputsAsTxOuts(const CTransaction& tx, vector & inputs, vector & errors,string& reason) +{ + bool result=true; + + reason=""; + + inputs.clear(); + errors.clear(); + + inputs.resize(tx.vin.size()); + errors.resize(tx.vin.size()); + + // Check for conflicts with in-memory transactions + { + LOCK(mempool.cs); // protect pool.mapNextTx + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + COutPoint outpoint = tx.vin[i].prevout; + if (mempool.mapNextTx.count(outpoint)) + { + result=false; + errors[i]="Conflicts with in-memory transactions"; + } + } + } + + { + CCoinsView dummy; + CCoinsViewCache view(&dummy); + + { + LOCK(mempool.cs); + CCoinsViewMemPool viewMemPool(pcoinsTip, mempool); + view.SetBackend(viewMemPool); + + for (unsigned int i = 0; i < tx.vin.size(); i++) + { + COutPoint outpoint = tx.vin[i].prevout; + + if (!view.HaveCoins(outpoint.hash)) + { + result=false; + errors[i]="Missing inputs"; + } + else + { + const CCoins* coins = view.AccessCoins(outpoint.hash); + if (!coins || !coins->IsAvailable(outpoint.n)) + { + result=false; + errors[i]="Input already spent"; + } + else + { + if(errors[i].size() == 0) + { + inputs[i]=coins->vout[outpoint.n]; + } + } + } + } + + view.GetBestBlock(); + + // we have all inputs cached now, so switch back to dummy, so we don't need to keep lock on mempool + view.SetBackend(dummy); + } + } + + return result; +} + +CScript GetScriptForString(string source) +{ + vector destinations; + + string tok; + + stringstream ss(source); + while(getline(ss, tok, ',')) + { + destinations.push_back(tok); + } + + if(destinations.size() == 1) + { + CBitcoinAddress address(destinations[0]); + if (pwalletMain && address.IsValid()) + { + return GetScriptForDestination(address.Get()); + } + else + { + if (IsHex(destinations[0])) + { + CPubKey vchPubKey(ParseHex(destinations[0])); + if (!vchPubKey.IsFullyValid()) + throw runtime_error(" Invalid public key: "+destinations[0]); + return GetScriptForPubKey(vchPubKey); + } + else + { + throw runtime_error(" Invalid public key: "+destinations[0]); + } + } + } + + int required=atoi(destinations[0]); + if( (required <= 0) || (required > 16) ) + throw runtime_error(" Invalid required for bare multisig: "+destinations[0]); + + if(required > (int)destinations.size()-1) + throw runtime_error(" To few public keys"); + + vector vPubKeys; + for(int i=1;i<(int)destinations.size();i++) + { + CPubKey vchPubKey(ParseHex(destinations[i])); + if (!vchPubKey.IsFullyValid()) + throw runtime_error(" Invalid public key: "+destinations[i]); + vPubKeys.push_back(vchPubKey); + } + + return GetScriptForMultisig(required,vPubKeys); +} + + + +vector > ParseRawOutputMultiObject(Object sendTo,int *required) +{ + vector > vecSend; + + set setAddress; + BOOST_FOREACH(const Pair& s, sendTo) + { + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + CBitcoinAddress address(s.name_); + if (address.IsValid()) + { + if (setAddress.count(address)) + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid parameter, duplicated address: ")+s.name_); + setAddress.insert(address); + } + } + CScript scriptPubKey = GetScriptForString(s.name_); + + CAmount nAmount; + if (s.value_.type() != obj_type) + { + nAmount = AmountFromValue(s.value_); + } + else + { + mc_Script *lpScript; + lpScript=new mc_Script; +// uint256 offer_hash; + size_t elem_size; + const unsigned char *elem; + + nAmount=0; + + string strError=ParseRawOutputObject(s.value_,nAmount,lpScript, required); + if(strError.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + +/* + if(lpScript->GetNumElements() > mc_gState->m_NetworkParams->GetInt64Param("maxstdopdropscount") ) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid number of elements in script"); +*/ + + for(int element=0;element < lpScript->GetNumElements();element++) + { + elem = lpScript->GetData(element,&elem_size); + if(elem) + { + scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + } + else + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid script"); + } + delete lpScript; + } + + vecSend.push_back(make_pair(scriptPubKey, nAmount)); + + } + + return vecSend; +} + +CScript ParseRawMetadata(Value param,uint32_t allowed_objects,mc_EntityDetails *given_entity,mc_EntityDetails *found_entity) +{ +// codes for allowed_objects fields +// 0x0001 - create +// 0x0002 - publish +// 0x0004 - issue +// 0x0008 - follow-on +// 0x0010 - pure details +// 0x0100 - encode empty hex +// 0x0200 - cache input script + + CScript scriptOpReturn=CScript(); + if(found_entity) + { + found_entity->Zero(); + } + + if(param.type() == obj_type) + { + mc_Script *lpDetailsScript; + mc_Script *lpDetails; + lpDetailsScript=new mc_Script; + lpDetails=new mc_Script; + lpDetails->AddElement(); + int format=0; + string entity_name=""; + int new_type=0; + int multiple=1; + int is_open=0; + bool multiple_is_set=false; + string strError=""; + mc_EntityDetails entity; + vector vKey; + vector vValue; +// bool key_is_set=false; +// bool value_is_set=false; + + + format=0; + entity.Zero(); + BOOST_FOREACH(const Pair& d, param.get_obj()) + { + if(d.name_ == "name") + { + format |= 1; + } + if(d.name_ == "multiple") + { + format |= 2; + } + } + + BOOST_FOREACH(const Pair& d, param.get_obj()) + { + bool parsed=false; + + if(d.name_ == "inputcache") + { + new_type=-3; + if( ((allowed_objects & 0x0200) == 0) || (mc_gState->m_Features->CachedInputScript() == 0) ) + { + strError=string("Keyword not allowed in this API"); + } + else + { + if(d.value_.type() != array_type) + { + strError=string("Array should be specified for inputcache"); + } + else + { + int cs_offset,cs_vin,cs_size; + string cs_script=""; + Array csa=d.value_.get_array(); + lpDetails->Clear(); + lpDetails->SetCachedScript(0,&cs_offset,-1,NULL,-1); + for(int csi=0;csi<(int)csa.size();csi++) + { + if(strError.size() == 0) + { + if(csa[csi].type() != obj_type) + { + strError=string("Elements of inputcache should be objects"); + } + cs_vin=-1; + cs_size=-1; + BOOST_FOREACH(const Pair& csf, csa[csi].get_obj()) + { + bool cs_parsed=false; + if(csf.name_ == "vin") + { + cs_parsed=true; + if(csf.value_.type() != int_type) + { + strError=string("vin should be integer"); + } + else + { + cs_vin=csf.value_.get_int(); + } + } + if(csf.name_ == "scriptPubKey") + { + cs_parsed=true; + if(csf.value_.type() != str_type) + { + strError=string("scriptPubKey should be string"); + } + else + { + cs_script=csf.value_.get_str(); + cs_size=cs_script.size()/2; + } + } + if(!cs_parsed) + { + strError=string("Invalid field: ") + csf.name_; + } + } + if(strError.size() == 0) + { + if(cs_vin<0) + { + strError=string("Missing vin field"); + } + } + if(strError.size() == 0) + { + if(cs_size<0) + { + strError=string("Missing scriptPubKey field"); + } + } + if(strError.size() == 0) + { + bool fIsHex; + vector dataData(ParseHex(cs_script.c_str(),fIsHex)); + if(!fIsHex) + { + strError=string("scriptPubKey should be hexadecimal string"); + } + else + { + lpDetails->SetCachedScript(cs_offset,&cs_offset,cs_vin,&dataData[0],cs_size); + } + } + } + } + } + } + parsed=true; + } + + if(d.name_ == "create") + { + if(new_type != 0) + { + strError=string("Only one of the following keywords can appear in the object: create, update, for"); + } + if(d.value_.type() != null_type && !d.value_.get_str().empty()) + { + if(d.value_.get_str() == "stream") + { + if((allowed_objects & 0x0001) == 0) + { + strError=string("Keyword not allowed in this API"); + } + new_type=MC_ENT_TYPE_STREAM; + } + else + { + if(d.value_.get_str() == "asset") + { + if((allowed_objects & 0x0004) == 0) + { + strError=string("Keyword not allowed in this API"); + } + new_type=MC_ENT_TYPE_ASSET; + } + else + { + strError=string("Invalid new entity type"); + } + } + } + else + { + strError=string("Invalid new entity type"); + } + parsed=true; + } + + if(d.name_ == "for") + { + if(new_type != 0) + { + strError=string("Only one of the following keywords can appear in the object: create, update, for"); + } + if(d.value_.type() != null_type && !d.value_.get_str().empty()) + { + ParseEntityIdentifier(d.value_,&entity, MC_ENT_TYPE_STREAM); + if(found_entity) + { + memcpy(found_entity,&entity,sizeof(mc_EntityDetails)); + } + } + new_type=-1; + if((allowed_objects & 0x0002) == 0) + { + strError=string("Keyword not allowed in this API"); + } + else + { + if(entity.GetEntityType() != MC_ENT_TYPE_STREAM) + { + strError=string("Stream with this identifier not found"); + } + } + parsed=true; + } + + if(d.name_ == "update") + { + if(new_type != 0) + { + strError=string("Only one of the following keywords can appear in the object: create, update, for"); + } + if(d.value_.type() != null_type && !d.value_.get_str().empty()) + { + ParseEntityIdentifier(d.value_,&entity, MC_ENT_TYPE_ASSET); + if(found_entity) + { + memcpy(found_entity,&entity,sizeof(mc_EntityDetails)); + } + } + new_type=-2; + if((allowed_objects & 0x0010) == 0) + { + strError=string("Keyword not allowed in this API"); + } + else + { + if(entity.GetEntityType() != MC_ENT_TYPE_ASSET) + { + strError=string("Asset with this identifier not found"); + } + } + parsed=true; + } + if(d.name_ == "key") + { + if(d.value_.type() != null_type && (d.value_.type()==str_type)) + { + vKey=vector(d.value_.get_str().begin(), d.value_.get_str().end()); +// key_is_set=true; + } + else + { + strError=string("Invalid key"); + } + if((allowed_objects & 0x0002) == 0) + { + strError=string("Keyword not allowed in this API"); + } + parsed=true; + } + if(d.name_ == "key-hex") + { + if(d.value_.type() != null_type && (d.value_.type()==str_type)) + { + bool fIsHex; + vKey=ParseHex(d.value_.get_str().c_str(),fIsHex); + if(!fIsHex) + { + strError=string("key should be hexadecimal string"); + } +// key_is_set=true; + } + else + { + strError=string("Invalid key"); + } + if((allowed_objects & 0x0002) == 0) + { + strError=string("Keyword not allowed in this API"); + } + parsed=true; + } + if(d.name_ == "data") + { + if(d.value_.type() != null_type && (d.value_.type()==str_type)) + { + bool fIsHex; + vValue=ParseHex(d.value_.get_str().c_str(),fIsHex); + if(!fIsHex) + { + strError=string("value should be hexadecimal string"); + } +// value_is_set=true; + } + else + { + strError=string("Invalid value"); + } + if((allowed_objects & 0x0002) == 0) + { + strError=string("Keyword not allowed in this API"); + } + parsed=true; + } + + if(d.name_ == "name") + { + if(d.value_.type() != null_type && !d.value_.get_str().empty()) + { + entity_name=d.value_.get_str().c_str(); + } + else + { + strError=string("Invalid name"); + } + if((allowed_objects & 0x0005) == 0) + { + strError=string("Keyword not allowed in this API"); + } + parsed=true; + } + if(d.name_ == "multiple") + { + if(d.value_.type() == int_type) + { + multiple=d.value_.get_int(); + multiple_is_set=true; + } + else + { + strError=string("Invalid multiple"); + } + if((allowed_objects & 0x0004) == 0) + { + strError=string("Keyword not allowed in this API"); + } + parsed=true; + } + if(d.name_ == "open") + { + if(d.value_.get_bool()) + { + is_open=1; + } + if((allowed_objects & 0x0005) == 0) + { + strError=string("Keyword not allowed in this API"); + } + parsed=true; + } + if(d.name_ == "details") + { + if(new_type == -3) + { + strError=string("details and inputcache not allowed in the same object"); + } + if(d.value_.type() == obj_type) + { + BOOST_FOREACH(const Pair& p, d.value_.get_obj()) + { + lpDetails->SetParamValue(p.name_.c_str(),p.name_.size(),(unsigned char*)p.value_.get_str().c_str(),p.value_.get_str().size()); + } + } + if((allowed_objects & 0x000D) == 0) + { + strError=string("Keyword not allowed in this API"); + } + parsed=true; + } + if(!parsed) + { + strError=string("Invalid field: ") + d.name_; + } + } + + if(strError.size() == 0) + { + if(new_type == 0) + { +// strError=string("One of the following keywords can appear in the object: create, update, for"); + if(given_entity && given_entity->GetEntityType()) + { + memcpy(&entity,given_entity,sizeof(mc_EntityDetails)); + new_type=-2; + } + else + { + new_type=MC_ENT_TYPE_ASSET; + } + } + } + + if(strError.size() == 0) + { + if(new_type == -2) + { + if((allowed_objects & 0x0008) == 0) + { + strError=string("Follow-on issuance not allowed in this API"); + } + else + { + if(vKey.size()) + { + strError=string("Invalid field: key"); + } + if(vValue.size()) + { + strError=string("Invalid field: value"); + } + } + } + } + + if(strError.size() == 0) + { + if(new_type == MC_ENT_TYPE_ASSET) + { + if((allowed_objects & 0x0004) == 0) + { + strError=string("Issuing new assets not allowed in this API"); + } + else + { + if(is_open) + { + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_FOLLOW_ONS,(unsigned char*)&is_open,1); + } + if(multiple_is_set) + { + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_ASSET_MULTIPLE,(unsigned char*)&multiple,4); + } + } + if(entity_name.size()) + { + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_NAME,(const unsigned char*)(entity_name.c_str()),entity_name.size());//+1); + } + } + if(vKey.size()) + { + strError=string("Invalid field: key"); + } + if(vValue.size()) + { + strError=string("Invalid field: value"); + } + } + } + } + + if(strError.size() == 0) + { + if(new_type == MC_ENT_TYPE_STREAM) + { + if((allowed_objects & 0x0001) == 0) + { + strError=string("Follow-on issuance not allowed in this API"); + } + else + { + format=1; + if(entity_name.size()) + { + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_NAME,(const unsigned char*)(entity_name.c_str()),entity_name.size());//+1); + } + if(is_open) + { + lpDetails->SetSpecialParamValue(MC_ENT_SPRM_ANYONE_CAN_WRITE,(unsigned char*)&is_open,1); + } + if(multiple_is_set) + { + strError=string("Invalid field: multiple"); + } + if(vKey.size()) + { + strError=string("Invalid field: key"); + } + if(vValue.size()) + { + strError=string("Invalid field: value"); + } + } + } + if(new_type == -1) + { + format=-1; + } + } + + + if(strError.size() == 0) + { + size_t bytes; + const unsigned char *script; + + if(new_type == MC_ENT_TYPE_ASSET) + { + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + script=lpDetails->GetData(0,&bytes); + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_ASSET,0,script,bytes); + + script = lpDetailsScript->GetData(0,&bytes); + scriptOpReturn << vector(script, script + bytes) << OP_DROP << OP_RETURN; + } + else + { + script=lpDetails->GetData(0,&bytes); + lpDetailsScript->SetAssetDetails(entity_name.c_str(),multiple,script,bytes); + script = lpDetailsScript->GetData(0,&bytes); + if(bytes > 0) + { + scriptOpReturn << OP_RETURN << vector(script, script + bytes); + } + } + } + + if(new_type == MC_ENT_TYPE_STREAM) + { + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + script=lpDetails->GetData(0,&bytes); + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_STREAM,0,script,bytes); + + script = lpDetailsScript->GetData(0,&bytes); + scriptOpReturn << vector(script, script + bytes) << OP_DROP << OP_RETURN; + } + else + { + lpDetailsScript->Clear(); + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_STREAM); + script = lpDetailsScript->GetData(0,&bytes); + scriptOpReturn << vector(script, script + bytes) << OP_DROP; + + lpDetailsScript->Clear(); + script=lpDetails->GetData(0,&bytes); + lpDetailsScript->SetGeneralDetails(script,bytes); + script = lpDetailsScript->GetData(0,&bytes); + if(bytes > 0) + { + scriptOpReturn << OP_RETURN << vector(script, script + bytes); + } + } + } + + if(new_type == -2) + { + if(mc_gState->m_Features->OpDropDetailsScripts()) + { + lpDetailsScript->Clear(); + lpDetailsScript->SetEntity(entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET); + script = lpDetailsScript->GetData(0,&bytes); + scriptOpReturn << vector(script, script + bytes) << OP_DROP; + + lpDetailsScript->Clear(); + script=lpDetails->GetData(0,&bytes); + lpDetailsScript->SetNewEntityType(MC_ENT_TYPE_ASSET,1,script,bytes); + script = lpDetailsScript->GetData(0,&bytes); + scriptOpReturn << vector(script, script + bytes) << OP_DROP << OP_RETURN; + } + else + { + lpDetailsScript->Clear(); + script=lpDetails->GetData(0,&bytes); + lpDetailsScript->SetGeneralDetails(script,bytes); + script = lpDetailsScript->GetData(0,&bytes); + if(bytes > 0) + { + scriptOpReturn << OP_RETURN << vector(script, script + bytes); + } + } + } + + if(new_type == -1) + { + lpDetailsScript->Clear(); + lpDetailsScript->SetEntity(entity.GetTxID()+MC_AST_SHORT_TXID_OFFSET); + script = lpDetailsScript->GetData(0,&bytes); + scriptOpReturn << vector(script, script + bytes) << OP_DROP; + + lpDetailsScript->Clear(); + if(lpDetailsScript->SetItemKey(&vKey[0],vKey.size()) == MC_ERR_NOERROR) + { + script = lpDetailsScript->GetData(0,&bytes); + scriptOpReturn << vector(script, script + bytes) << OP_DROP; + } + + scriptOpReturn << OP_RETURN << vValue; + } + + if(new_type == -3) + { + script=lpDetails->GetData(0,&bytes); + scriptOpReturn << vector(script, script + bytes) << OP_DROP << OP_RETURN; + } + + } + + delete lpDetails; + delete lpDetailsScript; + if(strError.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + } + else + { + bool fIsHex; + if( ((allowed_objects & 0x0100) != 0) || (param.get_str().size() != 0) ) + { + vector dataData(ParseHex(param.get_str().c_str(),fIsHex)); + if(!fIsHex) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "data-hex should be hexadecimal string or recognized object format"); + } + scriptOpReturn << OP_RETURN << dataData; + } + } + + return scriptOpReturn; +} + +vector ParseStringList(Value param) +{ + vector vStrings; + set setStrings; + + if(param.type() == array_type) + { + BOOST_FOREACH(const Value& vtok, param.get_array()) + { + if(vtok.type() == str_type) + { + string tok=vtok.get_str(); + if (setStrings.count(tok)) + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid parameter, duplicate value: ")+tok); + setStrings.insert(tok); + vStrings.push_back(tok); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid value, expected array of strings")); + } + } + } + else + { + if(param.type() == str_type) + { + stringstream ss(param.get_str()); + string tok; + while(getline(ss, tok, ',')) + { + if (setStrings.count(tok)) + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid parameter, duplicate value: ")+tok); + setStrings.insert(tok); + vStrings.push_back(tok); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid value, expected string or array")); + } + } + + return vStrings; +} + +void ParseEntityIdentifier(Value entity_identifier,mc_EntityDetails *entity,uint32_t entity_type) +{ + unsigned char buf[32]; + unsigned char buf_a[MC_AST_ASSET_REF_SIZE]; + unsigned char buf_n[MC_AST_ASSET_REF_SIZE]; + int ret; + string entity_nameU; + string entity_nameL; + + switch(entity_type) + { + case MC_ENT_TYPE_STREAM: + entity_nameU="Stream"; + entity_nameL="stream"; + break; + case MC_ENT_TYPE_ASSET: + entity_nameU="Asset"; + entity_nameL="asset"; + break; + default: + entity_nameU="Entity"; + entity_nameL="entity"; + break; + } + + if (entity_identifier.type() != null_type && !entity_identifier.get_str().empty()) + { + string str=entity_identifier.get_str(); + + if(entity_type & MC_ENT_TYPE_STREAM) + { + if(CoinSparkAssetRefDecode(buf_a,str.c_str(),str.size())) + { + memset(buf_n,0,MC_AST_ASSET_REF_SIZE); + if(memcmp(buf_a,buf_n,MC_AST_ASSET_REF_SIZE) == 0) + { + unsigned char *root_stream_name; + int root_stream_name_size; + root_stream_name=(unsigned char *)mc_gState->m_NetworkParams->GetParam("rootstreamname",&root_stream_name_size); + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + root_stream_name_size=0; + } + if(root_stream_name_size) + { + str=strprintf("%s",root_stream_name); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Stream with this stream reference not found: ")+str); + } + } + } + } + + ret=ParseAssetKey(str.c_str(),buf,NULL,NULL,NULL,NULL,entity_type); + switch(ret) + { + case -1: + throw JSONRPCError(RPC_INVALID_PARAMETER, entity_nameU+string(" with this txid not found: ")+str); + break; + case -2: + throw JSONRPCError(RPC_INVALID_PARAMETER, entity_nameU+string(" with this reference not found: ")+str); + break; + case -3: + throw JSONRPCError(RPC_INVALID_PARAMETER, entity_nameU+string(" with this name not found: ")+str); + break; + case -4: + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Could not parse ")+entity_nameL+string(" key: ")+str); + break; +/* + case 1: + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Unconfirmed stream: ")+str); + break; + */ + } + } + else + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid "+entity_nameL+" identifier"); + } + + if(entity) + { + if(mc_gState->m_Assets->FindEntityByTxID(entity,buf)) + { + if((entity_type & entity->GetEntityType()) == 0) +// if(entity->GetEntityType() != MC_ENT_TYPE_STREAM) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid "+entity_nameL+" identifier, not "+entity_nameL); + } + } + } +} + +bool AssetCompareByRef(Value a,Value b) +{ + Value assetref_a; + Value assetref_b; + + BOOST_FOREACH(const Pair& p, a.get_obj()) + { + if(p.name_ == "assetref") + { + assetref_a=p.value_; + } + if(p.name_ == "streamref") + { + assetref_a=p.value_; + } + } + + BOOST_FOREACH(const Pair& p, b.get_obj()) + { + if(p.name_ == "assetref") + { + assetref_b=p.value_; + } + if(p.name_ == "streamref") + { + assetref_b=p.value_; + } + } + + if(assetref_b.type() != str_type) + { + return true; + } + else + { + if(assetref_a.type() != str_type) + { + return false; + } + } + + unsigned char buf_a[MC_AST_ASSET_REF_SIZE]; + unsigned char buf_b[MC_AST_ASSET_REF_SIZE]; + + CoinSparkAssetRefDecode(buf_a,assetref_a.get_str().c_str(),assetref_a.get_str().size()); + CoinSparkAssetRefDecode(buf_b,assetref_b.get_str().c_str(),assetref_b.get_str().size()); + + +// if(strcmp(assetref_a.get_str().c_str(),assetref_b.get_str().c_str()) < 0) + if(mc_GetLE(buf_a,4) < mc_GetLE(buf_b,4)) + { + return true; + } + else + { + if(mc_GetLE(buf_a,4) == mc_GetLE(buf_b,4)) + { + if(mc_GetLE(buf_a+4,4) < mc_GetLE(buf_b+4,4)) + { + return true; + } + } + } + + return false; +} + +Array AssetArrayFromAmounts(mc_Buffer *asset_amounts,int issue_asset_id,uint256 hash,int show_type) +{ + Array assets; + unsigned char *ptr; + const unsigned char *txid; + uint64_t quantity; + + for(int a=0;aGetCount();a++) + { + Object asset_entry; + ptr=(unsigned char *)asset_amounts->GetRow(a); + + if( (mc_GetABRefType(ptr) != MC_AST_ASSET_REF_TYPE_SPECIAL) && + (mc_GetABRefType(ptr) != MC_AST_ASSET_REF_TYPE_GENESIS) ) + { + mc_EntityDetails entity; + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ptr)) + { + quantity=mc_GetABQuantity(ptr); + if(quantity != 0) + { + txid=entity.GetTxID(); + if(a == issue_asset_id) + { + asset_entry=AssetEntry(txid,-quantity,8); + } + else + { + asset_entry=AssetEntry(txid,quantity,2); + } + if(show_type) + { + switch(mc_GetABScriptType(ptr)) + { + case MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER: + asset_entry.push_back(Pair("type", "transfer")); + break; + case MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON: + asset_entry.push_back(Pair("type", "issuemore")); + break; + case MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER | MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON: + asset_entry.push_back(Pair("type", "issuemore+transfer")); + break; + case 0: + asset_entry.push_back(Pair("type", "issuefirst")); + break; + } + } + assets.push_back(asset_entry); + } + } + } + else + { + if(mc_GetABRefType(ptr) == MC_AST_ASSET_REF_TYPE_GENESIS) + { + if(hash != 0) + { + mc_EntityDetails entity; + if(mc_gState->m_Assets->FindEntityByTxID(&entity,(unsigned char*)&hash)) + { + quantity=mc_GetABQuantity(ptr); + if(quantity != 0) + { + txid=(unsigned char*)&hash; + asset_entry=AssetEntry(txid,quantity,2); + if(show_type) + { + asset_entry.push_back(Pair("type", "issuefirst")); + } + assets.push_back(asset_entry); + } + } + } + } + } + } + + return assets; +} + +void ParseRawAction(string action,bool& lock_it, bool& sign_it,bool& send_it) +{ + bool valid_action=false; + + if(action == "") + { + valid_action=true; + } + if(action == "lock") + { + lock_it=true; + valid_action=true; + } + if(action == "sign") + { + sign_it=true; + valid_action=true; + } + if((action == "lock,sign") || (action == "sign,lock")) + { + sign_it=true; + lock_it=true; + valid_action=true; + } + if(action == "send") + { + sign_it=true; + send_it=true; + valid_action=true; + } + + if(!valid_action) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid action"); + } +} + diff --git a/src/rpc/rpcutils.h b/src/rpc/rpcutils.h new file mode 100644 index 00000000..e4924a84 --- /dev/null +++ b/src/rpc/rpcutils.h @@ -0,0 +1,61 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef RPCMULTICHAINUTILS_H +#define RPCMULTICHAINUTILS_H + +#include "primitives/transaction.h" +#include "core/init.h" +#include "core/main.h" +#include "wallet/keystore.h" +#include "rpc/rpcserver.h" +//#include "script/script.h" +//#include "script/standard.h" +#include "structs/uint256.h" + +#include "json/json_spirit_utils.h" +#include "json/json_spirit_value.h" +#include "multichain/multichain.h" +#include "utils/utilparse.h" + + +using namespace std; +using namespace json_spirit; +#include +#include + +//string HelpRequiringPassphrase(); +string AllowedPermissions(); +string AllowedPausedServices(); +uint32_t GetPausedServices(const char *str); + + + +CScript RemoveOpDropsIfNeeded(const CScript& scriptInput); +bool CoinSparkAssetRefDecode(unsigned char *bin, const char* string, const size_t stringLen); +int ParseAssetKey(const char* asset_key,unsigned char *txid,unsigned char *asset_ref,char *name,int *multiple,int *type,int entity_type); +int ParseAssetKeyToFullAssetRef(const char* asset_key,unsigned char *full_asset_ref,int *multiple,int *type,int entity_type); +Array AddressEntries(const CTxIn& txin,txnouttype& typeRet,mc_Script *lpScript); +Array AddressEntries(const CTxOut& txout,txnouttype& typeRet); +Value PermissionForFieldEntry(mc_EntityDetails *lpEntity); +Array PermissionEntries(const CTxOut& txout,mc_Script *lpScript,bool fLong); +Object StreamEntry(const unsigned char *txid,uint32_t output_level); +Value OpReturnEntry(const unsigned char *elem,size_t elem_size,uint256 txid, int vout); +Value DataItemEntry(const CTransaction& tx,int n,set & already_seen,uint32_t stream_output_level); +Object AssetEntry(const unsigned char *txid,int64_t quantity,int output_level); +string ParseRawOutputObject(Value param,CAmount& nAmount,mc_Script *lpScript); +bool FindPreparedTxOut(CTxOut& txout,COutPoint outpoint,string& reason); +bool GetTxInputsAsTxOuts(const CTransaction& tx, vector & inputs, vector & errors,string& reason); +CScript GetScriptForString(string source); +vector > ParseRawOutputMultiObject(Object sendTo,int *required); +CScript ParseRawMetadata(Value param,uint32_t allowed_objects,mc_EntityDetails *given_entity,mc_EntityDetails *found_entity); +vector ParseStringList(Value param); +void ParseEntityIdentifier(Value entity_identifier,mc_EntityDetails *entity,uint32_t entity_type); +bool AssetCompareByRef(Value a,Value b); +Array AssetArrayFromAmounts(mc_Buffer *asset_amounts,int issue_asset_id,uint256 hash,int show_type); +void ParseRawAction(string action,bool& lock_it, bool& sign_it,bool& send_it); + +#endif /* RPCMULTICHAINUTILS_H */ + diff --git a/src/rpc/rpcwallet.cpp b/src/rpc/rpcwallet.cpp new file mode 100644 index 00000000..7f704082 --- /dev/null +++ b/src/rpc/rpcwallet.cpp @@ -0,0 +1,2095 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "structs/amount.h" +#include "structs/base58.h" +#include "utils/core_io.h" +#include "rpc/rpcserver.h" +#include "core/init.h" +#include "net/net.h" +#include "net/netbase.h" +#include "utils/timedata.h" +#include "utils/util.h" +#include "utils/utilmoneystr.h" +#include "wallet/wallet.h" +#include "wallet/walletdb.h" +#include "rpc/rpcwallet.h" + +#include + +#include +#include "json/json_spirit_utils.h" +#include "json/json_spirit_value.h" + +using namespace std; +using namespace boost; +using namespace boost::assign; +using namespace json_spirit; + +/* MCHN START */ +#include "script/sign.h" +#include "multichain/multichain.h" +#include "wallet/wallettxs.h" + +#include "rpc/rpcutils.h" + +/* MCHN END */ + + +int64_t nWalletUnlockTime; +static CCriticalSection cs_nWalletUnlockTime; + + + +std::string HelpRequiringPassphrase() +{ + return pwalletMain && pwalletMain->IsCrypted() + ? "\nRequires wallet passphrase to be set with walletpassphrase call." + : ""; +} + +void EnsureWalletIsUnlocked() +{ + if (pwalletMain->IsLocked()) + throw JSONRPCError(RPC_WALLET_UNLOCK_NEEDED, "Error: Please enter the wallet passphrase with walletpassphrase first."); +} + +string AccountFromValue(const Value& value) +{ + string strAccount = value.get_str(); + if (strAccount == "*") + throw JSONRPCError(RPC_WALLET_INVALID_ACCOUNT_NAME, "Invalid account name"); + return strAccount; +} + +Value getnewaddress(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 1) // MCHN + throw runtime_error("Help message not found\n"); + + // Parse the account first so we don't generate a key if there's an error + string strAccount; + if (params.size() > 0) + { + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need accounts, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + strAccount = AccountFromValue(params[0]); + } + + if (!pwalletMain->IsLocked()) + pwalletMain->TopUpKeyPool(); + + // Generate a new key that is added to wallet + CPubKey newKey; + if (!pwalletMain->GetKeyFromPool(newKey)) + throw JSONRPCError(RPC_WALLET_KEYPOOL_RAN_OUT, "Error: Keypool ran out, please call keypoolrefill first"); + CKeyID keyID = newKey.GetID(); + + pwalletMain->SetAddressBook(keyID, strAccount, "receive"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxEntity entity; + entity.Zero(); + memcpy(entity.m_EntityID,&keyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,0); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,0); + } + + return CBitcoinAddress(keyID).ToString(); +} + + +CBitcoinAddress GetAccountAddress(string strAccount, bool bForceNew=false) +{ + CWalletDB walletdb(pwalletMain->strWalletFile); + + CAccount account; + walletdb.ReadAccount(strAccount, account); + + bool bKeyUsed = false; + + // Check if the current key has been used + if (account.vchPubKey.IsValid()) + { + CScript scriptPubKey = GetScriptForDestination(account.vchPubKey.GetID()); + for (map::iterator it = pwalletMain->mapWallet.begin(); + it != pwalletMain->mapWallet.end() && account.vchPubKey.IsValid(); + ++it) + { + const CWalletTx& wtx = (*it).second; + BOOST_FOREACH(const CTxOut& txout, wtx.vout) + if (txout.scriptPubKey == scriptPubKey) + bKeyUsed = true; + } + } + + // Generate a new key + if (!account.vchPubKey.IsValid() || bForceNew || bKeyUsed) + { + if (!pwalletMain->GetKeyFromPool(account.vchPubKey)) + throw JSONRPCError(RPC_WALLET_KEYPOOL_RAN_OUT, "Error: Keypool ran out, please call keypoolrefill first"); + + pwalletMain->SetAddressBook(account.vchPubKey.GetID(), strAccount, "receive"); + walletdb.WriteAccount(strAccount, account); + } + + return CBitcoinAddress(account.vchPubKey.GetID()); +} + +Value getaccountaddress(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) // MCHN + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need accounts, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + + // Parse the account first so we don't generate a key if there's an error + string strAccount = AccountFromValue(params[0]); + + Value ret; + + ret = GetAccountAddress(strAccount).ToString(); + + return ret; +} + + +Value getrawchangeaddress(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 1) // MCHN + throw runtime_error("Help message not found\n"); + + if (!pwalletMain->IsLocked()) + pwalletMain->TopUpKeyPool(); + + CReserveKey reservekey(pwalletMain); + CPubKey vchPubKey; + if (!reservekey.GetReservedKey(vchPubKey)) + throw JSONRPCError(RPC_WALLET_KEYPOOL_RAN_OUT, "Error: Keypool ran out, please call keypoolrefill first"); + + reservekey.KeepKey(); + + CKeyID keyID = vchPubKey.GetID(); + + return CBitcoinAddress(keyID).ToString(); +} + + +Value setaccount(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) // MCHN + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need accounts, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + + CBitcoinAddress address(params[0].get_str()); + if (!address.IsValid()) // MCHN + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + + + string strAccount; + if (params.size() > 1) + strAccount = AccountFromValue(params[1]); + + // Only add the account if the address is yours. + if (IsMine(*pwalletMain, address.Get())) + { + // Detect when changing the account of an address that is the 'unused current key' of another account: + if (pwalletMain->mapAddressBook.count(address.Get())) + { + string strOldAccount = pwalletMain->mapAddressBook[address.Get()].name; + if (address == GetAccountAddress(strOldAccount)) + GetAccountAddress(strOldAccount, true); + } + pwalletMain->SetAddressBook(address.Get(), strAccount, "receive"); + } + else + throw JSONRPCError(RPC_MISC_ERROR, "setaccount can only be used with own address"); + + return Value::null; +} + + +Value getaccount(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) // MCHN + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need accounts, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + + CBitcoinAddress address(params[0].get_str()); + if (!address.IsValid()) // MCHN + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + + string strAccount; + map::iterator mi = pwalletMain->mapAddressBook.find(address.Get()); + if (mi != pwalletMain->mapAddressBook.end() && !(*mi).second.name.empty()) + strAccount = (*mi).second.name; + return strAccount; +} + + +Value getaddressesbyaccount(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) // MCHN + throw runtime_error("Help message not found\n"); + + string strAccount = AccountFromValue(params[0]); + + if(strAccount != "") + { + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need accounts, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + } + + // Find all addresses that have the given account + Array ret; + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, pwalletMain->mapAddressBook) + { + const CBitcoinAddress& address = item.first; + const string& strName = item.second.name; + if (strName == strAccount) + ret.push_back(address.ToString()); + } + return ret; +} + + +/* MCHN START */ + +Value resendwallettransactions(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 0) + throw runtime_error("Help message not found\n"); + pwalletMain->ResendWalletTransactions(true); + return "Wallet transactions resent"; +} + + + +/* MCHN END */ + +void SendMoney(const CTxDestination &address, CAmount nValue, CWalletTx& wtxNew,mc_Script *dropscript,mc_Script *opreturnscript) // MCHN +{ + // Check amount +/* MCHN START */ +// if (nValue <= 0) + if (nValue < 0) +/* MCHN START */ + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid amount"); + + + if (nValue > pwalletMain->GetBalance()) + throw JSONRPCError(RPC_WALLET_INSUFFICIENT_FUNDS, "Insufficient funds"); + + string strError; + if (pwalletMain->IsLocked()) + { + strError = "Error: Wallet locked, unable to create transaction!"; + LogPrintf("SendMoney() : %s", strError); + throw JSONRPCError(RPC_WALLET_ERROR, strError); + } + + // Parse Bitcoin address + +/* MCHN START */ +// Pay-To_PubKey testing code + +/* + CKey key; + + CKeyID lpKeyID=boost::get (address); + if(!pwalletMain->GetKey(lpKeyID, key)) + { + strError = "Error: Cannot find private key!"; + LogPrintf("SendMoney() : %s", strError); + throw JSONRPCError(RPC_WALLET_ERROR, strError); + } + + CPubKey pkey=key.GetPubKey(); + + CScript scriptPubKey; + scriptPubKey << ToByteVector(pkey) << OP_CHECKSIG; + */ +/* MCHN END */ + + CScript scriptPubKey = GetScriptForDestination(address); + +/* MCHN START */ + size_t elem_size; + const unsigned char *elem; + + if(dropscript) + { + LogPrint("mchnminor","mchn: Sending script with %d OP_DROP element(s)",dropscript->GetNumElements()); + if(dropscript->GetNumElements() > mc_gState->m_NetworkParams->GetInt64Param("maxstdopdropscount") ) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid number of elements in script"); + + for(int element=0;element < dropscript->GetNumElements();element++) + { + elem = dropscript->GetData(element,&elem_size); + if(elem) + { + scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + } + else + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid script"); + } + } + + CScript scriptOpReturn=CScript(); + + if(opreturnscript) + { + elem = opreturnscript->GetData(0,&elem_size); + if(elem_size > 0) + { + scriptOpReturn << OP_RETURN << vector(elem, elem + elem_size); + } + } +/* MCHN END */ + + + // Create and send the transaction + CReserveKey reservekey(pwalletMain); + CAmount nFeeRequired; + if (!pwalletMain->CreateTransaction(scriptPubKey, nValue, scriptOpReturn, wtxNew, reservekey, nFeeRequired, strError)) + { + if (nValue + nFeeRequired > pwalletMain->GetBalance()) + strError = strprintf("Error: This transaction requires a transaction fee of at least %s because of its amount, complexity, or use of recently received funds!", FormatMoney(nFeeRequired)); + LogPrintf("SendMoney() : %s\n", strError); + throw JSONRPCError(RPC_WALLET_ERROR, strError); + } +/* MCHN START */ + string strRejectReason; + if (!pwalletMain->CommitTransaction(wtxNew, reservekey, strRejectReason)) + { + if(strRejectReason.size()) + { + throw JSONRPCError(RPC_WALLET_ERROR, "Error: The transaction was rejected: " + strRejectReason); + } + else + { + throw JSONRPCError(RPC_WALLET_ERROR, "Error: The transaction was rejected! This might happen if some of the coins in your wallet were already spent, such as if you used a copy of wallet.dat and coins were spent in the copy but not marked as spent here."); + } + } + +/* MCHN END */ +} + +/* MCHN START */ + + + + + +Value listaddresses(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 5) + throw runtime_error("Help message not found\n"); + + if((mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + throw JSONRPCError(RPC_INVALID_REQUEST, "API is not supported with this wallet version. To get this functionality, run \"multichaind -walletdbversion=2 -rescan\" "); + } + + Array result; + int entity_count,address_count; + mc_TxEntityStat *lpEntity; + + entity_count=pwalletTxsMain->GetEntityListCount(); + + set setAddresses; + set setAddressesWithBalances; + if(params.size() > 0) + { + if( (params[0].type() != str_type) || (params[0].get_str() != "*") ) + { + setAddresses=ParseAddresses(params[0],ISMINE_NO); + if(setAddresses.size() == 0) + { + return result; + } + } + } + + address_count=0; + for(int i=0;iGetEntity(i); + if((lpEntity->m_Entity.m_EntityType == (MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS)) || + (lpEntity->m_Entity.m_EntityType == (MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS))) + { + if(setAddresses.size()) + { + CBitcoinAddress address; + if(CBitcoinAddressFromTxEntity(address,&(lpEntity->m_Entity))) + { + if(setAddresses.count(address.ToString()) > 0) + { + address_count++; + } + } + } + else + { + address_count++; + } + } + } + + uint32_t verbose=0x00; + + if (params.size() > 1) + { + if(params[1].type() == bool_type) + { + if(params[1].get_bool()) + { + verbose=0x02; + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid value for 'verbose' parameter, should be boolean"); + } + } + + int count,start; + count=entity_count; + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + count=params[2].get_int(); + if(count < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + + start=-count; + if (params.size() > 3) + { + if(params[3].type() == int_type) + { + start=params[3].get_int(); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid start")); + } + } + + mc_AdjustStartAndCount(&count,&start,address_count); + + address_count=0; + for(int i=0;iGetEntity(i); + CBitcoinAddress address; + if( (lpEntity->m_Entity.m_EntityType & MC_TET_ORDERMASK) == MC_TET_CHAINPOS) + { + if(CBitcoinAddressFromTxEntity(address,&(lpEntity->m_Entity))) + { + if( (setAddresses.size() == 0) || (setAddresses.count(address.ToString()) > 0) ) + { + if((address_count >= start) && (address_count < start+count)) + { + result.push_back(AddressEntry(address,verbose)); + } + address_count++; + } + } + } + } + + return result; +} + + + + + + + + + +Value gettxoutdata(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 4) // MCHN + throw runtime_error("Help message not found\n"); + + uint256 hash(params[0].get_str()); + int n = params[1].get_int(); + + CTransaction tx; + bool found=false; + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + if(pwalletTxsMain->FindWalletTx(hash,NULL) == 0) + { + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + tx=CTransaction(wtx); + found=true; + } + } + else + { + if (pwalletMain->mapWallet.count(hash)) + { + const CWalletTx& wtx = pwalletMain->mapWallet[hash]; + tx=CTransaction(wtx); + found=true; + + } + } + + if(!found) + { + uint256 hashBlock = 0; + if (!GetTransaction(hash, tx, hashBlock, true)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "No information available about transaction"); + } + } + + if( (n<0) || (n >= (int)tx.vout.size()) ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid vout"); + } + + const CScript& script1 = tx.vout[n].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + string metadata=""; + + if(mc_gState->m_TmpScript->IsOpReturnScript() == 0) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Output without metadata"); + } + size_t elem_size; + const unsigned char *elem; + + elem = mc_gState->m_TmpScript->GetData(mc_gState->m_TmpScript->GetNumElements()-1,&elem_size); + + int count,start; + count=elem_size; + if (params.size() > 2) + { + if(params[2].type() == int_type) + { + count=params[2].get_int(); + if(count < 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid count")); + } + } + + start=0; + if (params.size() > 3) + { + if(params[3].type() == int_type) + { + start=params[3].get_int(); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid start")); + } + } + + if(start < 0) + { + start=elem_size+start; + if(start<0) + { + start=0; + } + } + + if(start > (int)elem_size) + { + start=elem_size; + } + if(start+count > (int)elem_size) + { + count=elem_size-start; + } + + return HexStr(elem+start,elem+start+count); +} + +/* MCHN END */ + +Value listaddressgroupings(const Array& params, bool fHelp) +{ + if (fHelp) // MCHN + throw runtime_error("Help message not found\n"); + + Array jsonGroupings; + map balances = pwalletMain->GetAddressBalances(); + BOOST_FOREACH(set grouping, pwalletMain->GetAddressGroupings()) + { + Array jsonGrouping; + BOOST_FOREACH(CTxDestination address, grouping) + { + Array addressInfo; + addressInfo.push_back(CBitcoinAddress(address).ToString()); + addressInfo.push_back(ValueFromAmount(balances[address])); + { + LOCK(pwalletMain->cs_wallet); + if (pwalletMain->mapAddressBook.find(CBitcoinAddress(address).Get()) != pwalletMain->mapAddressBook.end()) + addressInfo.push_back(pwalletMain->mapAddressBook.find(CBitcoinAddress(address).Get())->second.name); + } + jsonGrouping.push_back(addressInfo); + } + jsonGroupings.push_back(jsonGrouping); + } + return jsonGroupings; +} + +Value signmessage(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 2) // MCHN + throw runtime_error("Help message not found\n"); + + EnsureWalletIsUnlocked(); + + string strAddress = params[0].get_str(); + string strMessage = params[1].get_str(); + + CBitcoinAddress addr(strAddress); + if (!addr.IsValid()) + throw JSONRPCError(RPC_TYPE_ERROR, "Invalid address"); + + CKeyID keyID; + if (!addr.GetKeyID(keyID)) + throw JSONRPCError(RPC_TYPE_ERROR, "Address does not refer to key"); + + CKey key; + if (!pwalletMain->GetKey(keyID, key)) + throw JSONRPCError(RPC_WALLET_ERROR, "Private key not available"); + + CHashWriter ss(SER_GETHASH, 0); + ss << strMessageMagic; + ss << strMessage; + + vector vchSig; + if (!key.SignCompact(ss.GetHash(), vchSig)) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Sign failed"); + + return EncodeBase64(&vchSig[0], vchSig.size()); +} + +Value getreceivedbyaddress(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) // MCHN + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Not supported with scalable wallet - if you need accounts, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + + // Bitcoin address + CBitcoinAddress address = CBitcoinAddress(params[0].get_str()); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + CScript scriptPubKey = GetScriptForDestination(address.Get()); + if (!IsMine(*pwalletMain,scriptPubKey)) + return (double)0.0; + + // Minimum confirmations + int nMinDepth = 1; + if (params.size() > 1) + nMinDepth = params[1].get_int(); + + // Tally + CAmount nAmount = 0; + for (map::iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); ++it) + { + const CWalletTx& wtx = (*it).second; + if (wtx.IsCoinBase() || !IsFinalTx(wtx)) + continue; + + BOOST_FOREACH(const CTxOut& txout, wtx.vout) + if (txout.scriptPubKey == scriptPubKey) + if (wtx.GetDepthInMainChain() >= nMinDepth) + nAmount += txout.nValue; + } + + return ValueFromAmount(nAmount); +} + + +Value getreceivedbyaccount(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need accounts, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + + // Minimum confirmations + int nMinDepth = 1; + if (params.size() > 1) + nMinDepth = params[1].get_int(); + + // Get the set of pub keys assigned to account + string strAccount = AccountFromValue(params[0]); + set setAddress = pwalletMain->GetAccountAddresses(strAccount); + + // Tally + CAmount nAmount = 0; + for (map::iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); ++it) + { + const CWalletTx& wtx = (*it).second; + if (wtx.IsCoinBase() || !IsFinalTx(wtx)) + continue; + + BOOST_FOREACH(const CTxOut& txout, wtx.vout) + { + CTxDestination address; + if (ExtractDestination(txout.scriptPubKey, address) && IsMine(*pwalletMain, address) && setAddress.count(address)) + if (wtx.GetDepthInMainChain() >= nMinDepth) + nAmount += txout.nValue; + } + } + +/* MCHN START */ + if(COIN == 0) + { + return (double)nAmount; + } +/* MCHN END */ + return (double)nAmount / (double)COIN; +} + + +CAmount GetAccountBalance(CWalletDB& walletdb, const string& strAccount, int nMinDepth, const isminefilter& filter) +{ + CAmount nBalance = 0; + + // Tally wallet transactions + for (map::iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); ++it) + { + const CWalletTx& wtx = (*it).second; + if (!IsFinalTx(wtx) || wtx.GetBlocksToMaturity() > 0 || wtx.GetDepthInMainChain() < 0) + continue; + + CAmount nReceived, nSent, nFee; + wtx.GetAccountAmounts(strAccount, nReceived, nSent, nFee, filter); + + if (nReceived != 0 && wtx.GetDepthInMainChain() >= nMinDepth) + nBalance += nReceived; + nBalance -= nSent + nFee; + } + + // Tally internal accounting entries + nBalance += walletdb.GetAccountCreditDebit(strAccount); + + return nBalance; +} + +CAmount GetAccountBalance(const string& strAccount, int nMinDepth, const isminefilter& filter) +{ + CWalletDB walletdb(pwalletMain->strWalletFile); + return GetAccountBalance(walletdb, strAccount, nMinDepth, filter); +} + + +Value getbalance(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 3) // MCHN + throw runtime_error("Help message not found\n"); + + if (params.size() == 0) + return ValueFromAmount(pwalletMain->GetBalance()); + + int nMinDepth = 1; + if (params.size() > 1) + nMinDepth = params[1].get_int(); + isminefilter filter = ISMINE_SPENDABLE; + if(params.size() > 2) + if(params[2].get_bool()) + filter = filter | ISMINE_WATCH_ONLY; + + if (params[0].get_str() == "*") { + // Calculate total balance a different way from GetBalance() + // (GetBalance() sums up all unspent TxOuts) + // getbalance and getbalance '*' 0 should return the same number + CAmount nBalance = 0; + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + vector vecOutputs; + pwalletMain->AvailableCoins(vecOutputs, false, NULL, false,true); + BOOST_FOREACH(const COutput& out, vecOutputs) + { + CTxOut txout; + out.GetHashAndTxOut(txout); + if(out.IsTrusted() || (out.nDepth >= nMinDepth)) + { + nBalance+=txout.nValue; + } + } + } + else + { + for (map::iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); ++it) + { + const CWalletTx& wtx = (*it).second; + if (!wtx.IsTrusted() || wtx.GetBlocksToMaturity() > 0) + continue; + + CAmount allFee; + string strSentAccount; + list listReceived; + list listSent; + wtx.GetAmounts(listReceived, listSent, allFee, strSentAccount, filter); + if (wtx.GetDepthInMainChain() >= nMinDepth) + { + BOOST_FOREACH(const COutputEntry& r, listReceived) + nBalance += r.amount; + } + BOOST_FOREACH(const COutputEntry& s, listSent) + nBalance -= s.amount; + nBalance -= allFee; + } + } + return ValueFromAmount(nBalance); + } + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need getbalance, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + string strAccount = AccountFromValue(params[0]); + + CAmount nBalance = GetAccountBalance(strAccount, nMinDepth, filter); + + return ValueFromAmount(nBalance); +} + +Value getunconfirmedbalance(const Array ¶ms, bool fHelp) +{ + if (fHelp || params.size() > 0) + throw runtime_error("Help message not found\n"); + return ValueFromAmount(pwalletMain->GetUnconfirmedBalance()); +} + + +Value movecmd(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 3 || params.size() > 5) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need move, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + + string strFrom = AccountFromValue(params[0]); + string strTo = AccountFromValue(params[1]); + CAmount nAmount = AmountFromValue(params[2]); + if (params.size() > 3) + // unused parameter, used to be nMinDepth, keep type-checking it though + (void)params[3].get_int(); + string strComment; + if (params.size() > 4) + strComment = params[4].get_str(); + + CWalletDB walletdb(pwalletMain->strWalletFile); + if (!walletdb.TxnBegin()) + throw JSONRPCError(RPC_DATABASE_ERROR, "database error"); + + int64_t nNow = GetAdjustedTime(); + + // Debit + CAccountingEntry debit; + debit.nOrderPos = pwalletMain->IncOrderPosNext(&walletdb); + debit.strAccount = strFrom; + debit.nCreditDebit = -nAmount; + debit.nTime = nNow; + debit.strOtherAccount = strTo; + debit.strComment = strComment; + walletdb.WriteAccountingEntry(debit); + + // Credit + CAccountingEntry credit; + credit.nOrderPos = pwalletMain->IncOrderPosNext(&walletdb); + credit.strAccount = strTo; + credit.nCreditDebit = nAmount; + credit.nTime = nNow; + credit.strOtherAccount = strFrom; + credit.strComment = strComment; + walletdb.WriteAccountingEntry(credit); + + if (!walletdb.TxnCommit()) + throw JSONRPCError(RPC_DATABASE_ERROR, "database error"); + + return true; +} + + +Value sendfrom(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 3 || params.size() > 6) // MCHN + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need sendfrom, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + + string strAccount = AccountFromValue(params[0]); + CBitcoinAddress address(params[1].get_str()); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + CAmount nAmount = AmountFromValue(params[2]); + int nMinDepth = 1; + if (params.size() > 3) + nMinDepth = params[3].get_int(); + + CWalletTx wtx; + wtx.strFromAccount = strAccount; + if (params.size() > 4 && params[4].type() != null_type && !params[4].get_str().empty()) + wtx.mapValue["comment"] = params[4].get_str(); + if (params.size() > 5 && params[5].type() != null_type && !params[5].get_str().empty()) + wtx.mapValue["to"] = params[5].get_str(); + + EnsureWalletIsUnlocked(); + + // Check funds + CAmount nBalance = GetAccountBalance(strAccount, nMinDepth, ISMINE_SPENDABLE); + if (nAmount > nBalance) + throw JSONRPCError(RPC_WALLET_INSUFFICIENT_FUNDS, "Account has insufficient funds"); + + LOCK (pwalletMain->cs_wallet_send); + + SendMoney(address.Get(), nAmount, wtx, NULL, NULL); + + return wtx.GetHash().GetHex(); +} + + +Value sendmany(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 4) // MCHN + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need sendmany, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + + string strAccount = AccountFromValue(params[0]); + Object sendTo = params[1].get_obj(); + int nMinDepth = 1; + if (params.size() > 2) + nMinDepth = params[2].get_int(); + + CWalletTx wtx; + wtx.strFromAccount = strAccount; + if (params.size() > 3 && params[3].type() != null_type && !params[3].get_str().empty()) + wtx.mapValue["comment"] = params[3].get_str(); + + set setAddress; + vector > vecSend; + + CAmount totalAmount = 0; + BOOST_FOREACH(const Pair& s, sendTo) + { + CBitcoinAddress address(s.name_); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, string("Invalid address: ")+s.name_); + + if (setAddress.count(address)) + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid parameter, duplicated address: ")+s.name_); + setAddress.insert(address); + + CScript scriptPubKey = GetScriptForDestination(address.Get()); + CAmount nAmount = AmountFromValue(s.value_); + totalAmount += nAmount; + + vecSend.push_back(make_pair(scriptPubKey, nAmount)); + } + + EnsureWalletIsUnlocked(); + + // Check funds + CAmount nBalance = GetAccountBalance(strAccount, nMinDepth, ISMINE_SPENDABLE); + if (totalAmount > nBalance) + throw JSONRPCError(RPC_WALLET_INSUFFICIENT_FUNDS, "Account has insufficient funds"); + +/* MCHN START */ + LOCK(pwalletMain->cs_wallet_send); +/* MCHN END */ + + // Send + CReserveKey keyChange(pwalletMain); + CAmount nFeeRequired = 0; + string strFailReason; + bool fCreated = pwalletMain->CreateTransaction(vecSend, wtx, keyChange, nFeeRequired, strFailReason); + if (!fCreated) + throw JSONRPCError(RPC_WALLET_INSUFFICIENT_FUNDS, strFailReason); +/* MCHN START */ + string strRejectReason; + if (!pwalletMain->CommitTransaction(wtx, keyChange,strRejectReason)) + throw JSONRPCError(RPC_WALLET_ERROR, "Transaction commit failed: " + strRejectReason); +/* MCHN END */ + + return wtx.GetHash().GetHex(); +} + +// Defined in rpcmisc.cpp +extern CScript _createmultisig_redeemScript(const Array& params); + +Value addmultisigaddress(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 3) // MCHN + { + throw runtime_error("Help message not found\n"); + } + + string strAccount; + if (params.size() > 2) + { + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need accounts, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + strAccount = AccountFromValue(params[2]); + } + // Construct using pay-to-script-hash: + CScript inner = _createmultisig_redeemScript(params); + CScriptID innerID(inner); + pwalletMain->AddCScript(inner); + + pwalletMain->SetAddressBook(innerID, strAccount, "send"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxEntity entity; + entity.Zero(); + memcpy(entity.m_EntityID,&innerID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_TIMERECEIVED; + pwalletTxsMain->AddEntity(&entity,MC_EFL_NOT_IN_SYNC); + } + + + return CBitcoinAddress(innerID).ToString(); +} + + +struct tallyitem +{ + CAmount nAmount; + int nConf; + vector txids; + bool fIsWatchonly; + tallyitem() + { + nAmount = 0; + nConf = std::numeric_limits::max(); + fIsWatchonly = false; + } +}; + +Value ListReceived(const Array& params, bool fByAccounts) +{ + // Minimum confirmations + int nMinDepth = 1; + if (params.size() > 0) + nMinDepth = params[0].get_int(); + + // Whether to include empty accounts + bool fIncludeEmpty = false; + if (params.size() > 1) + fIncludeEmpty = params[1].get_bool(); + + isminefilter filter = ISMINE_SPENDABLE; + if(params.size() > 2) + if(params[2].get_bool()) + filter = filter | ISMINE_WATCH_ONLY; + + // Tally + map mapTally; + for (map::iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); ++it) + { + const CWalletTx& wtx = (*it).second; + + if (wtx.IsCoinBase() || !IsFinalTx(wtx)) + continue; + + int nDepth = wtx.GetDepthInMainChain(); + if (nDepth < nMinDepth) + continue; + + BOOST_FOREACH(const CTxOut& txout, wtx.vout) + { + CTxDestination address; + if (!ExtractDestination(txout.scriptPubKey, address)) + continue; + + isminefilter mine = IsMine(*pwalletMain, address); + if(!(mine & filter)) + continue; + + tallyitem& item = mapTally[address]; + item.nAmount += txout.nValue; + item.nConf = min(item.nConf, nDepth); + item.txids.push_back(wtx.GetHash()); + if (mine & ISMINE_WATCH_ONLY) + item.fIsWatchonly = true; + } + } + + // Reply + Array ret; + map mapAccountTally; + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, pwalletMain->mapAddressBook) + { + const CBitcoinAddress& address = item.first; + const string& strAccount = item.second.name; + map::iterator it = mapTally.find(address); + if (it == mapTally.end() && !fIncludeEmpty) + continue; + + CAmount nAmount = 0; + int nConf = std::numeric_limits::max(); + bool fIsWatchonly = false; + if (it != mapTally.end()) + { + nAmount = (*it).second.nAmount; + nConf = (*it).second.nConf; + fIsWatchonly = (*it).second.fIsWatchonly; + } + + if (fByAccounts) + { + tallyitem& item = mapAccountTally[strAccount]; + item.nAmount += nAmount; + item.nConf = min(item.nConf, nConf); + item.fIsWatchonly = fIsWatchonly; + } + else + { + Object obj; + if(fIsWatchonly) + obj.push_back(Pair("involvesWatchonly", true)); + obj.push_back(Pair("address", address.ToString())); + obj.push_back(Pair("account", strAccount)); + obj.push_back(Pair("amount", ValueFromAmount(nAmount))); + obj.push_back(Pair("confirmations", (nConf == std::numeric_limits::max() ? 0 : nConf))); + Array transactions; + if (it != mapTally.end()) + { + BOOST_FOREACH(const uint256& item, (*it).second.txids) + { + transactions.push_back(item.GetHex()); + } + } + obj.push_back(Pair("txids", transactions)); + ret.push_back(obj); + } + } + + if (fByAccounts) + { + for (map::iterator it = mapAccountTally.begin(); it != mapAccountTally.end(); ++it) + { + CAmount nAmount = (*it).second.nAmount; + int nConf = (*it).second.nConf; + Object obj; + if((*it).second.fIsWatchonly) + obj.push_back(Pair("involvesWatchonly", true)); + obj.push_back(Pair("account", (*it).first)); + obj.push_back(Pair("amount", ValueFromAmount(nAmount))); + obj.push_back(Pair("confirmations", (nConf == std::numeric_limits::max() ? 0 : nConf))); + ret.push_back(obj); + } + } + + return ret; +} + +Value listreceivedbyaddress(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 3) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Not supported with scalable wallet - if you need accounts, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + + return ListReceived(params, false); +} + +Value listreceivedbyaccount(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 3) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are ot supported with scalable wallet - if you need accounts, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + + return ListReceived(params, true); +} + +static void MaybePushAddress(Object & entry, const CTxDestination &dest) +{ + CBitcoinAddress addr; + if (addr.Set(dest)) + entry.push_back(Pair("address", addr.ToString())); +} + +/* MCHN START */ + + + + +/* MCHN END */ + +void ListTransactions(const CWalletTx& wtx, const string& strAccount, int nMinDepth, bool fLong, Array& ret, const isminefilter& filter, mc_TxDefRow *txdef) +{ + CAmount nFee; + string strSentAccount; + list listReceived; + list listSent; + + wtx.GetAmounts(listReceived, listSent, nFee, strSentAccount, filter); + + bool fAllAccounts = (strAccount == string("*")); + bool involvesWatchonly = wtx.IsFromMe(ISMINE_WATCH_ONLY); + + // Sent + if ((!listSent.empty() || nFee != 0) && (fAllAccounts || strAccount == strSentAccount)) + { + BOOST_FOREACH(const COutputEntry& s, listSent) + { + Object entry; + if(involvesWatchonly || (::IsMine(*pwalletMain, s.destination) & ISMINE_WATCH_ONLY)) + entry.push_back(Pair("involvesWatchonly", true)); + entry.push_back(Pair("account", strSentAccount)); + MaybePushAddress(entry, s.destination); + entry.push_back(Pair("category", "send")); + entry.push_back(Pair("amount", ValueFromAmount(-s.amount))); + entry.push_back(Pair("vout", s.vout)); + entry.push_back(Pair("fee", ValueFromAmount(-nFee))); + if (fLong) + WalletTxToJSON(wtx, entry); + ret.push_back(entry); + } + } + + int nDepth=0; + if(txdef) + { + if(txdef->m_Block >= 0) + { + nDepth= chainActive.Height()-txdef->m_Block+1; + if(nDepth<0) + { + nDepth=0; + } + } + } + else + { + nDepth=wtx.GetDepthInMainChain(); + } + + // Received + if (listReceived.size() > 0 && (nDepth >= nMinDepth) ) + { + BOOST_FOREACH(const COutputEntry& r, listReceived) + { + string account; + if (pwalletMain->mapAddressBook.count(r.destination)) + account = pwalletMain->mapAddressBook[r.destination].name; + if (fAllAccounts || (account == strAccount)) + { + Object entry; + if(involvesWatchonly || (::IsMine(*pwalletMain, r.destination) & ISMINE_WATCH_ONLY)) + entry.push_back(Pair("involvesWatchonly", true)); + entry.push_back(Pair("account", account)); + MaybePushAddress(entry, r.destination); + if (wtx.IsCoinBase()) + { + if (nDepth < 1) + entry.push_back(Pair("category", "orphan")); + else if (max(0, (COINBASE_MATURITY+1) - nDepth) > 0) + entry.push_back(Pair("category", "immature")); + else + entry.push_back(Pair("category", "generate")); + } + else + { + entry.push_back(Pair("category", "receive")); + } + entry.push_back(Pair("amount", ValueFromAmount(r.amount))); + entry.push_back(Pair("vout", r.vout)); + if (fLong) + WalletTxToJSON(wtx, entry); +/* MCHN START */ +/* + Object full_entry; + TxToJSON(wtx,wtx.hashBlock,full_entry); + entry.push_back(Pair("details",full_entry)); + */ +/* MCHN END */ + ret.push_back(entry); + } + } + } +} + +void ListTransactions(const CWalletTx& wtx, const string& strAccount, int nMinDepth, bool fLong, Array& ret, const isminefilter& filter) +{ + ListTransactions(wtx,strAccount,nMinDepth,fLong,ret,filter,NULL); +} + +void AcentryToJSON(const CAccountingEntry& acentry, const string& strAccount, Array& ret) +{ + bool fAllAccounts = (strAccount == string("*")); + + if (fAllAccounts || acentry.strAccount == strAccount) + { + Object entry; + entry.push_back(Pair("account", acentry.strAccount)); + entry.push_back(Pair("category", "move")); + entry.push_back(Pair("time", acentry.nTime)); + entry.push_back(Pair("amount", ValueFromAmount(acentry.nCreditDebit))); + entry.push_back(Pair("otheraccount", acentry.strOtherAccount)); + entry.push_back(Pair("comment", acentry.strComment)); + ret.push_back(entry); + } +} + +Value listtransactions(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 4) // MCHN + throw runtime_error("Help message not found\n"); + + string strAccount = "*"; + if (params.size() > 0) + strAccount = params[0].get_str(); + int nCount = 10; + if (params.size() > 1) + nCount = params[1].get_int(); + int nFrom = 0; + if (params.size() > 2) + nFrom = params[2].get_int(); + isminefilter filter = ISMINE_SPENDABLE; + if(params.size() > 3) + if(params[3].get_bool()) + filter = filter | ISMINE_WATCH_ONLY; + + if (nCount < 0) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Negative count"); + if (nFrom < 0) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Negative from"); + + Array ret; + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Not supported with scalable wallet - if you need listtransactions, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); +/* + mc_Buffer *entity_rows; + entity_rows=new mc_Buffer; + entity_rows->Initialize(MC_TDB_ENTITY_KEY_SIZE,MC_TDB_ROW_SIZE,MC_BUF_MODE_DEFAULT); + + mc_TxEntity wallet_by_time; + mc_TxEntityRow *lpEntTx; + wallet_by_time.Zero(); + wallet_by_time.m_EntityType=MC_TET_TIMERECEIVED; + if(filter & ISMINE_WATCH_ONLY) + { + wallet_by_time.m_EntityType |= MC_TET_WALLET_ALL; + } + else + { + wallet_by_time.m_EntityType |= MC_TET_WALLET_SPENDABLE; + } + pwalletTxsMain->GetList(&wallet_by_time,-nFrom,nCount,entity_rows); + for(int i=entity_rows->GetCount()-1;i>=0;i--) + { + lpEntTx=(mc_TxEntityRow*)entity_rows->GetRow(i); + uint256 hash; + mc_TxDefRow txdef; + memcpy(&hash,lpEntTx->m_TxId,MC_TDB_TXID_SIZE); + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,&txdef,NULL); + ListTransactions(wtx, strAccount, 0, true, ret, filter,&txdef); + } + delete entity_rows; + std::reverse(ret.begin(), ret.end()); // Return oldest to newest +*/ + } + else + { + std::list acentries; + CWallet::TxItems txOrdered = pwalletMain->OrderedTxItems(acentries, strAccount); + + // iterate backwards until we have nCount items to return: + for (CWallet::TxItems::reverse_iterator it = txOrdered.rbegin(); it != txOrdered.rend(); ++it) + { + CWalletTx *const pwtx = (*it).second.first; + if (pwtx != 0) + ListTransactions(*pwtx, strAccount, 0, true, ret, filter); + + CAccountingEntry *const pacentry = (*it).second.second; + if (pacentry != 0) + AcentryToJSON(*pacentry, strAccount, ret); + + if ((int)ret.size() >= (nCount+nFrom)) break; + } + // ret is newest to oldest + + if (nFrom > (int)ret.size()) + nFrom = ret.size(); + if ((nFrom + nCount) > (int)ret.size()) + nCount = ret.size() - nFrom; + Array::iterator first = ret.begin(); + std::advance(first, nFrom); + Array::iterator last = ret.begin(); + std::advance(last, nFrom+nCount); + + if (last != ret.end()) ret.erase(last, ret.end()); + if (first != ret.begin()) ret.erase(ret.begin(), first); + + std::reverse(ret.begin(), ret.end()); // Return oldest to newest + } + + + return ret; +} + +/* MCHN START */ + + +/* MCHN END */ + +Value listaccounts(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 2) + throw runtime_error("Help message not found\n"); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + throw JSONRPCError(RPC_TRANSACTION_REJECTED, "Accounts are not supported with scalable wallet - if you need listaccounts, run multichaind -walletdbversion=1 -rescan, but the wallet will perform worse"); + } + + int nMinDepth = 1; + if (params.size() > 0) + nMinDepth = params[0].get_int(); + isminefilter includeWatchonly = ISMINE_SPENDABLE; + if(params.size() > 1) + if(params[1].get_bool()) + includeWatchonly = includeWatchonly | ISMINE_WATCH_ONLY; + + map mapAccountBalances; + BOOST_FOREACH(const PAIRTYPE(CTxDestination, CAddressBookData)& entry, pwalletMain->mapAddressBook) { + if (IsMine(*pwalletMain, entry.first) & includeWatchonly) // This address belongs to me + mapAccountBalances[entry.second.name] = 0; + } + + for (map::iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); ++it) + { + const CWalletTx& wtx = (*it).second; + CAmount nFee; + string strSentAccount; + list listReceived; + list listSent; + int nDepth = wtx.GetDepthInMainChain(); + if (wtx.GetBlocksToMaturity() > 0 || nDepth < 0) + continue; + wtx.GetAmounts(listReceived, listSent, nFee, strSentAccount, includeWatchonly); + mapAccountBalances[strSentAccount] -= nFee; + BOOST_FOREACH(const COutputEntry& s, listSent) + mapAccountBalances[strSentAccount] -= s.amount; + if (nDepth >= nMinDepth) + { + BOOST_FOREACH(const COutputEntry& r, listReceived) + if (pwalletMain->mapAddressBook.count(r.destination)) + mapAccountBalances[pwalletMain->mapAddressBook[r.destination].name] += r.amount; + else + mapAccountBalances[""] += r.amount; + } + } + + list acentries; + CWalletDB(pwalletMain->strWalletFile).ListAccountCreditDebit("*", acentries); + BOOST_FOREACH(const CAccountingEntry& entry, acentries) + mapAccountBalances[entry.strAccount] += entry.nCreditDebit; + + Object ret; + BOOST_FOREACH(const PAIRTYPE(string, CAmount)& accountBalance, mapAccountBalances) { + ret.push_back(Pair(accountBalance.first, ValueFromAmount(accountBalance.second))); + } + return ret; +} + +Value listsinceblock(const Array& params, bool fHelp) +{ + if (fHelp) // MCHN + throw runtime_error("Help message not found\n"); + + CBlockIndex *pindex = NULL; + int target_confirms = 1; + isminefilter filter = ISMINE_SPENDABLE; + + if (params.size() > 0) + { + uint256 blockId = 0; + + blockId.SetHex(params[0].get_str()); + BlockMap::iterator it = mapBlockIndex.find(blockId); + if (it != mapBlockIndex.end()) + pindex = it->second; + } + + if (params.size() > 1) + { + target_confirms = params[1].get_int(); + + if (target_confirms < 1) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter"); + } + + if(params.size() > 2) + if(params[2].get_bool()) + filter = filter | ISMINE_WATCH_ONLY; + + int depth = pindex ? (1 + chainActive.Height() - pindex->nHeight) : -1; + + Array transactions; + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + int first_block=0; + int tx_count; + int this_tx, up_tx, down_tx,chunk_size; + int err; + mc_Buffer *lpEntRowBuffer; + mc_TxEntityRow *entrow; + mc_TxEntity wallet_entity; + set hashes; + if(pindex) + { + first_block=pindex->nHeight+1; + } + + wallet_entity.Zero(); + wallet_entity.m_EntityType=MC_TET_WALLET_SPENDABLE | MC_TET_CHAINPOS; + if(filter & ISMINE_WATCH_ONLY) + { + wallet_entity.m_EntityType=MC_TET_WALLET_ALL | MC_TET_CHAINPOS; + } + tx_count=pwalletTxsMain->GetListSize(&wallet_entity,NULL); + + if(tx_count) + { + lpEntRowBuffer=new mc_Buffer; + lpEntRowBuffer->Initialize(MC_TDB_ENTITY_KEY_SIZE,sizeof(mc_TxEntityRow),MC_BUF_MODE_DEFAULT); + + up_tx=tx_count; + down_tx=1; + this_tx=(up_tx+down_tx)/2; + chunk_size=10; + while(up_tx-down_tx>chunk_size) + { + err=pwalletTxsMain->GetList(&wallet_entity,this_tx,1,lpEntRowBuffer); + if( (err == MC_ERR_NOERROR) && (lpEntRowBuffer->GetCount() > 0) ) + { + entrow=(mc_TxEntityRow *)(lpEntRowBuffer->GetRow(0)); + if(entrow->m_Block >= first_block) + { + up_tx=this_tx; + } + else + { + down_tx=this_tx; + } + this_tx=(up_tx+down_tx)/2; + } + else // We are probably in reorg, start from what we have now + { + up_tx=down_tx; + } + } + + this_tx=down_tx; + while(this_tx<=tx_count) + { + err=pwalletTxsMain->GetList(&wallet_entity,this_tx,chunk_size,lpEntRowBuffer); + if( (err == MC_ERR_NOERROR) && (lpEntRowBuffer->GetCount() > 0) ) + { + for(int i=0;iGetCount();i++) + { + entrow=(mc_TxEntityRow*)lpEntRowBuffer->GetRow(i); + if( (entrow->m_Block < 0) || (entrow->m_Block >= first_block) ) + { + uint256 hash; + memcpy(&hash,entrow->m_TxId,MC_TDB_TXID_SIZE); + hashes.insert(hash); + } + } + } + this_tx+=chunk_size; + } + + BOOST_FOREACH(uint256 hash, hashes) + { + mc_TxDefRow txdef; + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,&txdef,NULL); + ListTransactions(wtx, "*", 0, true, transactions, filter,&txdef); + } + + delete lpEntRowBuffer; + } + } + else + { + for (map::iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); it++) + { + CWalletTx tx = (*it).second; + if (depth == -1 || tx.GetDepthInMainChain() < depth) + ListTransactions(tx, "*", 0, true, transactions, filter); + } + } + + CBlockIndex *pblockLast = chainActive[chainActive.Height() + 1 - target_confirms]; + uint256 lastblock = pblockLast ? pblockLast->GetBlockHash() : 0; + + Object ret; + ret.push_back(Pair("transactions", transactions)); + ret.push_back(Pair("lastblock", lastblock.GetHex())); + + return ret; +} + +Value gettransaction(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) // MCHN + throw runtime_error("Help message not found\n"); + + uint256 hash; + hash.SetHex(params[0].get_str()); + + isminefilter filter = ISMINE_SPENDABLE; + if(params.size() > 1) + if(params[1].get_bool()) + filter = filter | ISMINE_WATCH_ONLY; + + Object entry; + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + int err; + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,&err); + if(err) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid or non-wallet transaction id"); + } + CAmount nCredit = wtx.GetCredit(filter); + CAmount nDebit = wtx.GetDebit(filter); + CAmount nNet = nCredit - nDebit; + CAmount nFee = (wtx.IsFromMe(filter) ? wtx.GetValueOut() - nDebit : 0); + + entry.push_back(Pair("amount", ValueFromAmount(nNet - nFee))); + if (wtx.IsFromMe(filter)) + entry.push_back(Pair("fee", ValueFromAmount(nFee))); + + WalletTxToJSON(wtx, entry); + + Array details; + ListTransactions(wtx, "*", 0, false, details, filter); + entry.push_back(Pair("details", details)); + + string strHex = EncodeHexTx(static_cast(wtx)); + entry.push_back(Pair("hex", strHex)); + } + else + { + if (!pwalletMain->mapWallet.count(hash)) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid or non-wallet transaction id"); + const CWalletTx& wtx = pwalletMain->mapWallet[hash]; + + CAmount nCredit = wtx.GetCredit(filter); + CAmount nDebit = wtx.GetDebit(filter); + CAmount nNet = nCredit - nDebit; + CAmount nFee = (wtx.IsFromMe(filter) ? wtx.GetValueOut() - nDebit : 0); + + entry.push_back(Pair("amount", ValueFromAmount(nNet - nFee))); + if (wtx.IsFromMe(filter)) + entry.push_back(Pair("fee", ValueFromAmount(nFee))); + + WalletTxToJSON(wtx, entry); + + Array details; + ListTransactions(wtx, "*", 0, false, details, filter); + entry.push_back(Pair("details", details)); + + string strHex = EncodeHexTx(static_cast(wtx)); + entry.push_back(Pair("hex", strHex)); + } + + return entry; +} + + +Value backupwallet(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 1) + throw runtime_error("Help message not found\n"); + + string strDest = params[0].get_str(); + if (!BackupWallet(*pwalletMain, strDest)) + throw JSONRPCError(RPC_WALLET_ERROR, "Error: Wallet backup failed!"); + + return Value::null; +} + + +Value keypoolrefill(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 1) + throw runtime_error("Help message not found\n"); + + // 0 is interpreted by TopUpKeyPool() as the default keypool size given by -keypool + unsigned int kpSize = 0; + if (params.size() > 0) { + if (params[0].get_int() < 0) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter, expected valid size."); + kpSize = (unsigned int)params[0].get_int(); + } + + EnsureWalletIsUnlocked(); + pwalletMain->TopUpKeyPool(kpSize); + + if (pwalletMain->GetKeyPoolSize() < kpSize) + throw JSONRPCError(RPC_WALLET_ERROR, "Error refreshing keypool."); + + return Value::null; +} + + +static void LockWallet(CWallet* pWallet) +{ + LOCK(cs_nWalletUnlockTime); + nWalletUnlockTime = 0; + pWallet->Lock(); +} + +Value walletpassphrase(const Array& params, bool fHelp) +{ + if (pwalletMain->IsCrypted() && (fHelp || params.size() != 2)) // MCHN + throw runtime_error( + "walletpassphrase \"passphrase\" timeout\n" + "\nStores the wallet decryption key in memory for 'timeout' seconds.\n" + "This is needed prior to performing transactions related to private keys such as sending assets\n" + "\nArguments:\n" + "1. \"passphrase\" (string, required) The wallet passphrase\n" + "2. timeout (numeric, required) The time to keep the decryption key in seconds.\n" + "\nNote:\n" + "Issuing the walletpassphrase command while the wallet is already unlocked will set a new unlock\n" + "time that overrides the old one.\n" + "\nExamples:\n" + "\nunlock the wallet for 60 seconds\n" + + HelpExampleCli("walletpassphrase", "\"my pass phrase\" 60") + + "\nLock the wallet again (before 60 seconds)\n" + + HelpExampleCli("walletlock", "") + + "\nAs json rpc call\n" + + HelpExampleRpc("walletpassphrase", "\"my pass phrase\", 60") + ); + + if (fHelp) + return true; + if (!pwalletMain->IsCrypted()) + throw JSONRPCError(RPC_WALLET_WRONG_ENC_STATE, "Error: running with an unencrypted wallet, but walletpassphrase was called."); + + // Note that the walletpassphrase is stored in params[0] which is not mlock()ed + SecureString strWalletPass; + strWalletPass.reserve(100); + // TODO: get rid of this .c_str() by implementing SecureString::operator=(std::string) + // Alternately, find a way to make params[0] mlock()'d to begin with. + strWalletPass = params[0].get_str().c_str(); + + if (strWalletPass.length() > 0) + { + if (!pwalletMain->Unlock(strWalletPass)) + throw JSONRPCError(RPC_WALLET_PASSPHRASE_INCORRECT, "Error: The wallet passphrase entered was incorrect."); + } + else + throw runtime_error( + "walletpassphrase \n" + "Stores the wallet decryption key in memory for seconds."); + + pwalletMain->TopUpKeyPool(); + + int64_t nSleepTime = params[1].get_int64(); + LOCK(cs_nWalletUnlockTime); + nWalletUnlockTime = GetTime() + nSleepTime; + RPCRunLater("lockwallet", boost::bind(LockWallet, pwalletMain), nSleepTime); + + return Value::null; +} + + +Value walletpassphrasechange(const Array& params, bool fHelp) +{ + if (pwalletMain->IsCrypted() && (fHelp || params.size() != 2)) + throw runtime_error( + "walletpassphrasechange \"oldpassphrase\" \"newpassphrase\"\n" + "\nChanges the wallet passphrase from 'oldpassphrase' to 'newpassphrase'.\n" + "\nArguments:\n" + "1. \"oldpassphrase\" (string) The current passphrase\n" + "2. \"newpassphrase\" (string) The new passphrase\n" + "\nExamples:\n" + + HelpExampleCli("walletpassphrasechange", "\"old one\" \"new one\"") + + HelpExampleRpc("walletpassphrasechange", "\"old one\", \"new one\"") + ); + + if (fHelp) + return true; + if (!pwalletMain->IsCrypted()) + throw JSONRPCError(RPC_WALLET_WRONG_ENC_STATE, "Error: running with an unencrypted wallet, but walletpassphrasechange was called."); + + // TODO: get rid of these .c_str() calls by implementing SecureString::operator=(std::string) + // Alternately, find a way to make params[0] mlock()'d to begin with. + SecureString strOldWalletPass; + strOldWalletPass.reserve(100); + strOldWalletPass = params[0].get_str().c_str(); + + SecureString strNewWalletPass; + strNewWalletPass.reserve(100); + strNewWalletPass = params[1].get_str().c_str(); + + if (strOldWalletPass.length() < 1 || strNewWalletPass.length() < 1) + throw runtime_error( + "walletpassphrasechange \n" + "Changes the wallet passphrase from to ."); + + if (!pwalletMain->ChangeWalletPassphrase(strOldWalletPass, strNewWalletPass)) + throw JSONRPCError(RPC_WALLET_PASSPHRASE_INCORRECT, "Error: The wallet passphrase entered was incorrect."); + + return Value::null; +} + + +Value walletlock(const Array& params, bool fHelp) +{ + if (pwalletMain->IsCrypted() && (fHelp || params.size() != 0)) + throw runtime_error( + "walletlock\n" + "\nRemoves the wallet encryption key from memory, locking the wallet.\n" + "After calling this method, you will need to call walletpassphrase again\n" + "before being able to call any methods which require the wallet to be unlocked.\n" + "\nExamples:\n" + "\nSet the passphrase for 2 minutes to perform a transaction\n" + + HelpExampleCli("walletpassphrase", "\"my pass phrase\" 120") + + "\nPerform a send (requires passphrase set)\n" + + HelpExampleCli("sendtoaddress", "\"1M72Sfpbz1BPpXFHz9m3CdqATR44Jvaydd\" 1.0") + + "\nClear the passphrase since we are done before 2 minutes is up\n" + + HelpExampleCli("walletlock", "") + + "\nAs json rpc call\n" + + HelpExampleRpc("walletlock", "") + ); + + if (fHelp) + return true; + if (!pwalletMain->IsCrypted()) + throw JSONRPCError(RPC_WALLET_WRONG_ENC_STATE, "Error: running with an unencrypted wallet, but walletlock was called."); + + { + LOCK(cs_nWalletUnlockTime); + pwalletMain->Lock(); + nWalletUnlockTime = 0; + } + + return Value::null; +} + + +Value encryptwallet(const Array& params, bool fHelp) +{ + if (!pwalletMain->IsCrypted() && (fHelp || params.size() != 1)) // MCHN + throw runtime_error("Help message not found\n"); + + if (fHelp) + return true; + if (pwalletMain->IsCrypted()) + throw JSONRPCError(RPC_WALLET_WRONG_ENC_STATE, "Error: running with an encrypted wallet, but encryptwallet was called."); + + // TODO: get rid of this .c_str() by implementing SecureString::operator=(std::string) + // Alternately, find a way to make params[0] mlock()'d to begin with. + SecureString strWalletPass; + strWalletPass.reserve(100); + strWalletPass = params[0].get_str().c_str(); + + if (strWalletPass.length() < 1) + throw runtime_error( + "encryptwallet \n" + "Encrypts the wallet with ."); + + if (!pwalletMain->EncryptWallet(strWalletPass)) + throw JSONRPCError(RPC_WALLET_ENCRYPTION_FAILED, "Error: Failed to encrypt the wallet."); + + // BDB seems to have a bad habit of writing old data into + // slack space in .dat files; that is bad if the old data is + // unencrypted private keys. So: + StartShutdown(); // MCHN + return "wallet encrypted; server stopping, restart to run with encrypted wallet. The keypool has been flushed, you need to make a new backup."; +} + +Value lockunspent(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) // MCHN + throw runtime_error("Help message not found\n"); + + if (params.size() == 1) + RPCTypeCheck(params, list_of(bool_type)); + else + RPCTypeCheck(params, list_of(bool_type)(array_type)); + + bool fUnlock = params[0].get_bool(); + + if (params.size() == 1) { + if (fUnlock) + pwalletMain->UnlockAllCoins(); + return true; + } + + Array outputs = params[1].get_array(); + BOOST_FOREACH(Value& output, outputs) + { + if (output.type() != obj_type) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter, expected object"); + const Object& o = output.get_obj(); + + RPCTypeCheck(o, map_list_of("txid", str_type)("vout", int_type)); + + string txid = find_value(o, "txid").get_str(); + if (!IsHex(txid)) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter, expected hex txid"); + + int nOutput = find_value(o, "vout").get_int(); + if (nOutput < 0) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid parameter, vout must be positive"); + + COutPoint outpt(uint256(txid), nOutput); + + if (fUnlock) + pwalletMain->UnlockCoin(outpt); + else + pwalletMain->LockCoin(outpt); + } + + return true; +} + +Value listlockunspent(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 0) + throw runtime_error("Help message not found\n"); + + vector vOutpts; + pwalletMain->ListLockedCoins(vOutpts); + + Array ret; + + BOOST_FOREACH(COutPoint &outpt, vOutpts) { + Object o; + + o.push_back(Pair("txid", outpt.hash.GetHex())); + o.push_back(Pair("vout", (int)outpt.n)); + ret.push_back(o); + } + + return ret; +} + +Value settxfee(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 1) // MCHN + throw runtime_error("Help message not found\n"); + + // Amount + CAmount nAmount = 0; + if (params[0].get_real() != 0.0) + nAmount = AmountFromValue(params[0]); // rejects 0.0 amounts + + payTxFee = CFeeRate(nAmount, 1000); + return true; +} + +Value getwalletinfo(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 0) // MCHN + throw runtime_error("Help message not found\n"); + + Object obj; + obj.push_back(Pair("walletversion", pwalletMain->GetVersion())); + obj.push_back(Pair("balance", ValueFromAmount(pwalletMain->GetBalance()))); + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + obj.push_back(Pair("txcount", (int)pwalletTxsMain->m_Database->m_DBStat.m_Count)); + if(mc_gState->m_WalletMode & MC_WMD_MAP_TXS) + { + obj.push_back(Pair("walletdbversion", -1)); + } + else + { + obj.push_back(Pair("walletdbversion", 2)); + } + } + else + { + obj.push_back(Pair("txcount", (int)pwalletMain->mapWallet.size())); + obj.push_back(Pair("walletdbversion", 1)); + } + obj.push_back(Pair("keypoololdest", pwalletMain->GetOldestKeyPoolTime())); + obj.push_back(Pair("keypoolsize", (int)pwalletMain->GetKeyPoolSize())); + if (pwalletMain->IsCrypted()) + obj.push_back(Pair("unlocked_until", nWalletUnlockTime)); + return obj; +} diff --git a/src/rpc/rpcwallet.h b/src/rpc/rpcwallet.h new file mode 100644 index 00000000..cf8b817d --- /dev/null +++ b/src/rpc/rpcwallet.h @@ -0,0 +1,41 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef RPCWALLET_H +#define RPCWALLET_H + +#include "structs/base58.h" +#include "utils/core_io.h" +#include "rpc/rpcserver.h" +#include "core/init.h" +#include "utils/util.h" +#include "wallet/wallet.h" + +#include +#include "json/json_spirit_utils.h" +#include "json/json_spirit_value.h" + + +#include "multichain/multichain.h" +#include "wallet/wallettxs.h" +#include "rpc/rpcutils.h" + +void SendMoneyToSeveralAddresses(const std::vector addresses, CAmount nValue, CWalletTx& wtxNew,mc_Script *dropscript,CScript scriptOpReturn,const std::vector& fromaddresses); +vector ParseAddresses(string param, bool create_full_list, bool allow_scripthash); +void FindAddressesWithPublishPermission(std::vector& fromaddresses,mc_EntityDetails *stream_entity); +set ParseAddresses(Value param, isminefilter filter); +bool CBitcoinAddressFromTxEntity(CBitcoinAddress &address,mc_TxEntity *lpEntity); +Object StreamItemEntry(const CWalletTx& wtx,const unsigned char *stream_id, bool verbose); +Object TxOutEntry(const CTxOut& TxOutIn,int vout,const CTxIn& TxIn,uint256 hash,mc_Buffer *amounts,mc_Script *lpScript); +void WalletTxToJSON(const CWalletTx& wtx, Object& entry,bool skipWalletConflicts = false, int vout = -1); +void MinimalWalletTxToJSON(const CWalletTx& wtx, Object& entry); +Object AddressEntry(CBitcoinAddress& address,uint32_t verbose); +void SetSynchronizedFlag(CTxDestination &dest,Object &ret); + + + + +#endif /* RPCWALLET_H */ + diff --git a/src/rpc/rpcwalletsend.cpp b/src/rpc/rpcwalletsend.cpp new file mode 100644 index 00000000..3e3fb494 --- /dev/null +++ b/src/rpc/rpcwalletsend.cpp @@ -0,0 +1,1049 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + + +#include "rpc/rpcwallet.h" +bool CreateAssetGroupingTransaction(CWallet *lpWallet, const vector >& vecSend, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl* coinControl, + const set* addresses,int min_conf,int min_inputs,int max_inputs,const vector* lpCoinsToUse,uint32_t flags); + + + + +Value createrawsendfrom(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 4) // MCHN + throw runtime_error("Help message not found\n"); + + vector fromaddresses; + set thisFromAddresses; + + fromaddresses=ParseAddresses(params[0].get_str(),false,true); + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + thisFromAddresses.insert(fromaddress); + } + + Object sendTo = params[1].get_obj(); + + CWalletTx rawTx; + mc_EntityDetails found_entity; + + vector > vecSend; + vecSend=ParseRawOutputMultiObject(sendTo,NULL); + + mc_EntityDetails entity; + mc_gState->m_TmpAssetsOut->Clear(); + for(int i=0;i<(int)vecSend.size();i++) + { + FindFollowOnsInScript(vecSend[i].first,mc_gState->m_TmpAssetsOut,mc_gState->m_TmpScript); + } + entity.Zero(); + if(mc_gState->m_TmpAssetsOut->GetCount()) + { +/* + if(mc_gState->m_TmpAssetsOut->GetCount() > 1) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Follow-on script rejected - follow-on for several assets"); + } + */ + if(!mc_gState->m_Assets->FindEntityByFullRef(&entity,mc_gState->m_TmpAssetsOut->GetRow(0))) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Follow-on script rejected - asset not found"); + } + } + + + if (params.size() > 2 && params[2].type() != null_type) + { + BOOST_FOREACH(const Value& data, params[2].get_array()) + { + CScript scriptOpReturn=ParseRawMetadata(data,0x01FF,&entity,&found_entity); + if(found_entity.GetEntityType() == MC_ENT_TYPE_STREAM) + { + FindAddressesWithPublishPermission(fromaddresses,&found_entity); + } + vecSend.push_back(make_pair(scriptOpReturn, 0)); + } + } + + string action=""; + string hex; + Value signedTx; + Value txid; + bool sign_it=false; + bool lock_it=false; + bool send_it=false; + if (params.size() > 3 && params[3].type() != null_type) + { + ParseRawAction(params[3].get_str(),lock_it,sign_it,send_it); + } + + + CReserveKey reservekey(pwalletMain); + CAmount nFeeRequired; + string strError; + uint32_t flags=MC_CSF_ALLOW_NOT_SPENDABLE_P2SH | MC_CSF_ALLOW_SPENDABLE_P2SH; + + EnsureWalletIsUnlocked(); + { + LOCK (pwalletMain->cs_wallet_send); + + if(!CreateAssetGroupingTransaction(pwalletMain, vecSend, rawTx, reservekey, nFeeRequired, strError, NULL, &thisFromAddresses, 1, -1, -1, NULL, flags)) + { + LogPrintf("createrawsendfrom : %s\n", strError); + throw JSONRPCError(RPC_WALLET_ERROR, strError); + } + } + + hex=EncodeHexTx(rawTx); + + + if(sign_it) + { + Array signrawtransaction_params; + signrawtransaction_params.push_back(hex); + signedTx=signrawtransaction(signrawtransaction_params,false); + } + if(lock_it) + { + BOOST_FOREACH(const CTxIn& txin, rawTx.vin) + { + COutPoint outpt(txin.prevout.hash, txin.prevout.n); + pwalletMain->LockCoin(outpt); + } + } + if(send_it) + { + Array sendrawtransaction_params; + BOOST_FOREACH(const Pair& s, signedTx.get_obj()) + { + if(s.name_=="complete") + { + if(!s.value_.get_bool()) + { + throw JSONRPCError(RPC_TRANSACTION_ERROR, "Transaction was not signed properly"); + } + } + if(s.name_=="hex") + { + sendrawtransaction_params.push_back(s.value_.get_str()); + } + } + txid=sendrawtransaction(sendrawtransaction_params,false); + } + + if(send_it) + { + return txid; + } + + if(sign_it) + { + return signedTx; + } + + return hex; +} + + + +Value sendfromaddress(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 3 || params.size() > 5) // MCHN + throw runtime_error("Help message not found\n"); + + CBitcoinAddress address(params[1].get_str()); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + + // Wallet comments + CWalletTx wtx; + if (params.size() > 3 && params[3].type() != null_type && !params[3].get_str().empty()) + wtx.mapValue["comment"] = params[3].get_str(); + if (params.size() > 4 && params[4].type() != null_type && !params[4].get_str().empty()) + wtx.mapValue["to"] = params[4].get_str(); + + + if(!AddressCanReceive(address.Get())) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Destination address doesn't have receive permission"); + } + + + CAmount nAmount=0; + + vector addresses; + addresses.push_back(address.Get()); + + mc_Script *lpScript; + + lpScript=NULL; + + if (params[2].type() == obj_type) + { + lpScript=new mc_Script; + uint256 offer_hash; + + if (params[2].type() != obj_type) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid asset quantities object"); + } + else + { + string strError=ParseRawOutputObject(params[2],nAmount,lpScript); + if(strError.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + } + } + else + { + nAmount = AmountFromValue(params[2]); + } + + + vector fromaddresses; + + if(params[0].get_str() != "*") + { + fromaddresses=ParseAddresses(params[0].get_str(),false,false); + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + if( (IsMine(*pwalletMain, fromaddresses[0]) & ISMINE_SPENDABLE) != ISMINE_SPENDABLE ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key for from-address is not found in this wallet"); + } + + set thisFromAddresses; + + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + thisFromAddresses.insert(fromaddress); + } + + + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_SEND,&thisFromAddresses)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "from-address doesn't have send permission"); + } + } + else + { + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_SEND)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "This wallet doesn't have keys with send permission"); + } + } + + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + SendMoneyToSeveralAddresses(addresses, nAmount, wtx, lpScript, CScript(), fromaddresses); + + if(lpScript) + { + delete lpScript; + } + + return wtx.GetHash().GetHex(); +} + + +Value sendwithmetadatafrom(const Array& params, bool fHelp) +{ + if (fHelp || params.size() != 4) // MCHN + throw runtime_error("Help message not found\n"); + + CBitcoinAddress address(params[1].get_str()); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + + // Wallet comments + CWalletTx wtx; + + if(!AddressCanReceive(address.Get())) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Destination address doesn't have receive permission"); + } + + + CAmount nAmount=0; + + vector addresses; + addresses.push_back(address.Get()); + + mc_Script *lpScript; + + lpScript=NULL; + + if (params[2].type() == obj_type) + { + lpScript=new mc_Script; + uint256 offer_hash; + + if (params[2].type() != obj_type) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid asset quantities object"); + } + else + { + string strError=ParseRawOutputObject(params[2],nAmount,lpScript); + if(strError.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + } + } + else + { + nAmount = AmountFromValue(params[2]); + } + + mc_EntityDetails found_entity; + CScript scriptOpReturn=ParseRawMetadata(params[3],0x0002,NULL,&found_entity); + + vector fromaddresses; + set thisFromAddresses; + + if(params[0].get_str() != "*") + { + fromaddresses=ParseAddresses(params[0].get_str(),false,false); + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + if( (IsMine(*pwalletMain, fromaddresses[0]) & ISMINE_SPENDABLE) != ISMINE_SPENDABLE ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key for from-address is not found in this wallet"); + } + + + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + thisFromAddresses.insert(fromaddress); + } + + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_SEND,&thisFromAddresses)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "from-address doesn't have send permission"); + } + if(found_entity.GetEntityType() == MC_ENT_TYPE_STREAM) + { + FindAddressesWithPublishPermission(fromaddresses,&found_entity); + } + } + else + { + CPubKey pkey; + if(found_entity.GetEntityType() == MC_ENT_TYPE_STREAM) + { + FindAddressesWithPublishPermission(fromaddresses,&found_entity); + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + thisFromAddresses.insert(fromaddress); + } + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_SEND,&thisFromAddresses)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "This wallet doesn't have keys with write permission for given stream and/or global send permission"); + } + } + else + { + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_SEND)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "This wallet doesn't have keys with send permission"); + } + } + } + + + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + SendMoneyToSeveralAddresses(addresses, nAmount, wtx, lpScript, scriptOpReturn, fromaddresses); + + if(lpScript) + { + delete lpScript; + } + + return wtx.GetHash().GetHex(); +} + +Value sendwithmetadata(const Array& params, bool fHelp) +{ + if (fHelp || params.size() !=3 ) + throw runtime_error("Help message not found\n"); + + Array ext_params; + ext_params.push_back("*"); + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + } + + return sendwithmetadatafrom(ext_params,fHelp); +} + +Value sendtoaddress(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 4) // MCHN + throw runtime_error("Help message not found\n"); + + Array ext_params; + ext_params.push_back("*"); + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + } + + return sendfromaddress(ext_params,fHelp); +} + + + +Value combineunspent(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 6) + throw runtime_error("Help message not found\n"); + + int nMinConf = 1; + if (params.size() > 1) + nMinConf = params[1].get_int(); + + int nMinInputs = 10; + if (params.size() > 3) + nMinInputs = params[3].get_int(); + + if((nMinInputs < 2) || (nMinInputs > 1000)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid mininputs. Valid Range [2 - 1000]."); + } + + int nMaxInputs = 100; + if (params.size() > 4) + nMaxInputs = params[4].get_int(); + + if((nMaxInputs < 2) || (nMaxInputs > 1000)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid maxinputs. Valid Range [2 - 1000]."); + } + + if(nMaxInputs < nMinInputs) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "maxinputs below mininouts."); + } + + int nMaxTransactions = 1; + if (params.size() > 2) + nMaxTransactions = params[2].get_int(); + + if((nMaxTransactions < 1) || (nMaxTransactions > 20)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid maximum-transactions. Valid Range [1 - 20]."); + } + + int nMaxTime = 30; + if (params.size() > 5) + nMaxTime = params[5].get_int(); + + vector addresses; + set thisAddresses; + set setAddress; + set* lpAddresses; + + string tok; + + lpAddresses=NULL; + + if( (params.size() == 0) || (params[0].get_str() == "*") ) + { + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, pwalletMain->mapAddressBook) + { + const CBitcoinAddress& address = item.first; + CTxDestination dest=address.Get(); + CKeyID *lpKeyID=boost::get (&dest); + + if(lpKeyID) + { + addresses.push_back(address.Get()); + } + } + } + else + { + stringstream ss(params[0].get_str()); + while(getline(ss, tok, ',')) + { + CBitcoinAddress address(tok); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, string("Invalid address: ")+tok); + if (setAddress.count(address)) + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid parameter, duplicated address: ")+tok); + CTxDestination dest=address.Get(); + CKeyID *lpKeyID=boost::get (&dest); + if(lpKeyID) + { + addresses.push_back(address.Get()); + setAddress.insert(address); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid address (only pubkeyhash addresses are supported) : ")+tok); + } + } + } + + EnsureWalletIsUnlocked(); + + string strError; + if (pwalletMain->IsLocked()) + { + strError = "Error: Wallet locked, unable to create transaction!"; + LogPrintf("CombineUnspent() : %s", strError); + throw JSONRPCError(RPC_WALLET_ERROR, strError); + } + + + // Create and send the transaction + CReserveKey reservekey(pwalletMain); + + double start_time=mc_TimeNowAsDouble(); + int txcount=0; + int pos=0; + std::vector scriptPubKeys; + CScript scriptOpReturn=CScript(); + Array results; + + + while((pos<(int)addresses.size()) && (txcount < nMaxTransactions) && ((mc_TimeNowAsDouble()-start_time < (double)nMaxTime) || (txcount == 0))) + { + thisAddresses.clear(); + thisAddresses.insert(addresses[pos]); + lpAddresses=&thisAddresses; + + CKeyID *lpKeyID=boost::get (&addresses[pos]); + CBitcoinAddress bitcoin_address=CBitcoinAddress(*lpKeyID); + + + CWalletTx wtx; + if(!pwalletMain->CreateAndCommitOptimizeTransaction(wtx,strError,lpAddresses,nMinConf,nMinInputs,nMaxInputs)) + { + pos++; + } + else + { + txcount++; + results.push_back(wtx.GetHash().GetHex()); + } + + } + + if(results.size() == 0) + { + strError="Not enough inputs"; + LogPrintf("CombineUnspent() : %s\n", strError); + throw JSONRPCError(RPC_WALLET_ERROR, strError); + } + + return results; +} + + +Value preparelockunspentfrom(const json_spirit::Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 3) + throw runtime_error("Help message not found\n"); + + vector fromaddresses; + fromaddresses=ParseAddresses(params[0].get_str(),false,false); + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + if( (IsMine(*pwalletMain, fromaddresses[0]) & ISMINE_SPENDABLE) != ISMINE_SPENDABLE ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key for from-address is not found in this wallet"); + } + + set thisFromAddresses; + + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + thisFromAddresses.insert(fromaddress); + } + + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_SEND | MC_PTP_RECEIVE,&thisFromAddresses)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "from-address must have send and receive permission"); + } + vector addresses; + addresses.push_back(CTxDestination(pkey.GetID())); + + mc_Script *lpScript; + lpScript=new mc_Script; + CAmount nAmount=0; + uint256 offer_hash; + bool lock_it=true; + + + if (params[1].type() != obj_type) + { + nAmount = AmountFromValue(params[1]); + } + else + { + string strError=ParseRawOutputObject(params[1],nAmount,lpScript); + if(strError.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + } + + if ((params.size() > 2) && (params[2].type() == bool_type)) + { + if(!params[2].get_bool()) + { + lock_it=false; + } + } + + CWalletTx wtx; + + + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + SendMoneyToSeveralAddresses(addresses, nAmount, wtx, lpScript, CScript(), fromaddresses); + + int vout=-1; + + CScript scriptPubKey = GetScriptForDestination(addresses[0]); + + for(int element=0;element < lpScript->GetNumElements();element++) + { + size_t elem_size; + const unsigned char *elem; + elem = lpScript->GetData(element,&elem_size); + if(elem) + { + scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + } + else + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid script"); + } + + CScript::const_iterator pc0 = scriptPubKey.begin(); + + for (unsigned int j = 0; j < wtx.vout.size(); j++) + { + CTxOut txout=wtx.vout[j]; + + const CScript& script1 = txout.scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + int script_size=(int)(script1.end()-script1.begin()); + + if(nAmount == txout.nValue) + { + if(script_size == (int)(scriptPubKey.end()-scriptPubKey.begin())) + { + if(memcmp((&pc1[0]),(&pc0[0]),script_size) == 0) + { + LOCK(pwalletMain->cs_wallet); + LogPrint("mchn","mchn: New lockunspent (%s,%d), Offer hash: %s\n",wtx.GetHash().GetHex().c_str(),j,offer_hash.GetHex().c_str()); + vout=j; + } + } + } + } + + delete lpScript; + + + if(lock_it) + { + COutPoint outpt(wtx.GetHash(),vout); + + pwalletMain->LockCoin(outpt); + } + + Object result; + + result.push_back(Pair("txid", wtx.GetHash().GetHex())); + result.push_back(Pair("vout", vout)); + + return result; +} + + + +Value preparelockunspent(const json_spirit::Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 2) + throw runtime_error("Help message not found\n"); + + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_SEND | MC_PTP_RECEIVE)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "This wallet doesn't have keys with send and receive permission"); + } + + vector addresses; + addresses.push_back(CTxDestination(pkey.GetID())); + + mc_Script *lpScript; + lpScript=new mc_Script; + CAmount nAmount=0; + uint256 offer_hash; + bool lock_it=true; + + + if (params[0].type() != obj_type) + { + nAmount = AmountFromValue(params[0]); +// throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid asset quantities object"); + } + else + { + string strError=ParseRawOutputObject(params[0],nAmount,lpScript); + if(strError.size()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, strError); + } + } + + if ((params.size() > 1) && (params[1].type() == bool_type)) + { + if(!params[1].get_bool()) + { + lock_it=false; + } + } + + CWalletTx wtx; + + + vector fromaddresses; + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + SendMoneyToSeveralAddresses(addresses, nAmount, wtx, lpScript, CScript(), fromaddresses); + + int vout=-1; + + CScript scriptPubKey = GetScriptForDestination(addresses[0]); + + for(int element=0;element < lpScript->GetNumElements();element++) + { + size_t elem_size; + const unsigned char *elem; + elem = lpScript->GetData(element,&elem_size); + if(elem) + { + scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + } + else + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid script"); + } + + CScript::const_iterator pc0 = scriptPubKey.begin(); + + for (unsigned int j = 0; j < wtx.vout.size(); j++) + { + CTxOut txout=wtx.vout[j]; + + const CScript& script1 = txout.scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + int script_size=(int)(script1.end()-script1.begin()); + + if(nAmount == txout.nValue) + { + if(script_size == (int)(scriptPubKey.end()-scriptPubKey.begin())) + { + if(memcmp((&pc1[0]),(&pc0[0]),script_size) == 0) + { + LOCK(pwalletMain->cs_wallet); + LogPrint("mchn","mchn: New lockunspent (%s,%d), Offer hash: %s\n",wtx.GetHash().GetHex().c_str(),j,offer_hash.GetHex().c_str()); + vout=j; +// pwalletMain->mapExchanges.insert(make_pair(COutPoint(wtx.GetHash(),j),CExchangeStatus(offer_hash,0,mc_TimeNowAsUInt()))); + } + } + } + } + + delete lpScript; + + + if(lock_it) + { + COutPoint outpt(wtx.GetHash(),vout); + + pwalletMain->LockCoin(outpt); + } + + Object result; + + result.push_back(Pair("txid", wtx.GetHash().GetHex())); + result.push_back(Pair("vout", vout)); + + return result; +} + +Value sendassetfrom(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 4 || params.size() > 7) + throw runtime_error("Help message not found\n"); + + CBitcoinAddress address(params[1].get_str()); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + + // Amount + CAmount nAmount = mc_gState->m_NetworkParams->GetInt64Param("minimumperoutput"); + if (params.size() > 4 && params[4].type() != null_type) + { + nAmount = AmountFromValue(params[4]); + } + + // Wallet comments + CWalletTx wtx; + if (params.size() > 5 && params[5].type() != null_type && !params[5].get_str().empty()) + wtx.mapValue["comment"] = params[5].get_str(); + if (params.size() > 6 && params[6].type() != null_type && !params[6].get_str().empty()) + wtx.mapValue["to"] = params[6].get_str(); + + if(!AddressCanReceive(address.Get())) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Destination address doesn't have receive permission"); + } + + mc_Script *lpScript; + lpScript=new mc_Script; + + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + int multiple=1; + + if (params.size() > 2 && params[2].type() != null_type && !params[2].get_str().empty()) + { + mc_EntityDetails entity; + ParseEntityIdentifier(params[2],&entity, MC_ENT_TYPE_ASSET); + memcpy(buf,entity.GetFullRef(),MC_AST_ASSET_FULLREF_SIZE); + if(mc_gState->m_Features->ShortTxIDAsAssetRef() == 0) + { + if(entity.IsUnconfirmedGenesis()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Unconfirmed asset: ")+params[2].get_str()); + } + } + multiple=entity.GetAssetMultiple(); + } + else + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid asset reference"); + } + Value raw_qty=params[3]; + + int64_t quantity = (int64_t)(raw_qty.get_real() * multiple + 0.499999); + if(quantity<0) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid asset quantity"); + } + + + mc_SetABQuantity(buf,quantity); + + mc_Buffer *lpBuffer; + lpBuffer=new mc_Buffer; + + mc_InitABufferDefault(lpBuffer); + + lpBuffer->Add(buf); + + lpScript->SetAssetQuantities(lpBuffer,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER); + + delete lpBuffer; + + + vector addresses; + addresses.push_back(address.Get()); + + vector fromaddresses; + + + if(params[0].get_str() != "*") + { + fromaddresses=ParseAddresses(params[0].get_str(),false,false); + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + if( (IsMine(*pwalletMain, fromaddresses[0]) & ISMINE_SPENDABLE) != ISMINE_SPENDABLE ) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Private key for from-address is not found in this wallet"); + } + + set thisFromAddresses; + + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + thisFromAddresses.insert(fromaddress); + } + + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_SEND,&thisFromAddresses)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "from-address doesn't have send permission"); + } + } + else + { + CPubKey pkey; + if(!pwalletMain->GetKeyFromAddressBook(pkey,MC_PTP_SEND)) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "This wallet doesn't have keys with send permission"); + } + } + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + SendMoneyToSeveralAddresses(addresses, nAmount, wtx, lpScript, CScript(),fromaddresses); + + delete lpScript; + + return wtx.GetHash().GetHex(); +} + + +Value sendassettoaddress(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 3 || params.size() > 6) + throw runtime_error("Help message not found\n"); + + Array ext_params; + ext_params.push_back("*"); + BOOST_FOREACH(const Value& value, params) + { + ext_params.push_back(value); + } + + return sendassetfrom(ext_params,fHelp); + + CBitcoinAddress address(params[0].get_str()); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid address"); + + // Amount + CAmount nAmount = mc_gState->m_NetworkParams->GetInt64Param("minimumperoutput"); + if (params.size() > 3 && params[3].type() != null_type) + { + nAmount = AmountFromValue(params[3]); + } + + // Wallet comments + CWalletTx wtx; + if (params.size() > 4 && params[4].type() != null_type && !params[4].get_str().empty()) + wtx.mapValue["comment"] = params[4].get_str(); + if (params.size() > 5 && params[5].type() != null_type && !params[5].get_str().empty()) + wtx.mapValue["to"] = params[5].get_str(); + + if(!AddressCanReceive(address.Get())) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Destination address doesn't have receive permission"); + } + + mc_Script *lpScript; + lpScript=new mc_Script; + + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + int multiple=1; + + if (params.size() > 1 && params[1].type() != null_type && !params[1].get_str().empty()) + { + mc_EntityDetails entity; + ParseEntityIdentifier(params[1],&entity, MC_ENT_TYPE_ASSET); + memcpy(buf,entity.GetFullRef(),MC_AST_ASSET_FULLREF_SIZE); + if(mc_gState->m_Features->ShortTxIDAsAssetRef() == 0) + { + if(entity.IsUnconfirmedGenesis()) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Unconfirmed asset: ")+params[1].get_str()); + } + } + multiple=entity.GetAssetMultiple(); + } + else + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid asset reference"); + } + Value raw_qty=params[2]; + + int64_t quantity = (int64_t)(raw_qty.get_real() * multiple + 0.499999); + if(quantity<=0) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid asset quantity"); + } + + + mc_SetABQuantity(buf,quantity); + + mc_Buffer *lpBuffer; + lpBuffer=new mc_Buffer; + + mc_InitABufferDefault(lpBuffer); + + lpBuffer->Add(buf); + + lpScript->SetAssetQuantities(lpBuffer,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER); + + delete lpBuffer; + + + vector addresses; + addresses.push_back(address.Get()); + + vector fromaddresses; + + EnsureWalletIsUnlocked(); + LOCK (pwalletMain->cs_wallet_send); + + SendMoneyToSeveralAddresses(addresses, nAmount, wtx, lpScript, CScript(),fromaddresses); + + delete lpScript; + + return wtx.GetHash().GetHex(); +} diff --git a/src/rpc/rpcwallettxs.cpp b/src/rpc/rpcwallettxs.cpp new file mode 100644 index 00000000..4e265e55 --- /dev/null +++ b/src/rpc/rpcwallettxs.cpp @@ -0,0 +1,841 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + + +#include "rpc/rpcwallet.h" + +bool IsTxOutMineAndInList(CWallet *lpWallet,const CTxOut& txout,const set* addresses,const isminefilter& filter) +{ + isminetype fIsMine=lpWallet->IsMine(txout); + + if(fIsMine & filter) + { + if(addresses) + { + CTxDestination addressRet; + if(ExtractDestination(txout.scriptPubKey, addressRet)) + { + if(addresses->count(addressRet)) + { + return true; + } + } + } + else + { + return true; + } + } + + return false; +} + +int GetInputOffer(const CWalletTx& wtx, const set* addresses,const isminefilter& filter,CAmount& nAmount,mc_Buffer *amounts,mc_Script *lpScript,set* from_addresses,set* my_addresses) +{ + int nIsMineCount=0; + amounts->Clear(); + nAmount=0; + bool fIsMine; + + BOOST_FOREACH(const CTxIn& txin, wtx.vin) + { + CTxDestination addressFrom; + bool take_it=false; + fIsMine=false; + + CTxOut prevout; + bool prevout_found=false; + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + int err; + const CWalletTx& prev=pwalletTxsMain->GetWalletTx(txin.prevout.hash,NULL,&err); + if(err == MC_ERR_NOERROR) + { + if (txin.prevout.n < prev.vout.size()) + { + prevout=prev.vout[txin.prevout.n]; + prevout_found=true; + } + } + } + else + { + map::const_iterator mi = pwalletMain->mapWallet.find(txin.prevout.hash); + if (mi != pwalletMain->mapWallet.end()) + { + const CWalletTx& prev = (*mi).second; + if (txin.prevout.n < prev.vout.size()) + { + prevout=prev.vout[txin.prevout.n]; + prevout_found=true; + } + } + } + + if (prevout_found) + { + if(IsTxOutMineAndInList(pwalletMain,prevout,addresses,filter)) + { + string strFailReason; + if(ParseMultichainTxOutToBuffer(txin.prevout.hash,prevout,amounts,lpScript,NULL,NULL,strFailReason)) + { + nAmount+=prevout.nValue; + nIsMineCount++; + fIsMine=true; + if(ExtractDestination(prevout.scriptPubKey, addressFrom)) + { + take_it=true; + } + } + } + } + if( (from_addresses != NULL) && !wtx.IsCoinBase()) + { + const CScript& script2 = txin.scriptSig; + CScript::const_iterator pc2 = script2.begin(); + + lpScript->Clear(); + lpScript->SetScript((unsigned char*)(&pc2[0]),(size_t)(script2.end()-pc2),MC_SCR_TYPE_SCRIPTSIGRAW); + + size_t elem_size; + const unsigned char *elem; + if(lpScript->GetNumElements() >= 2) + { + elem = lpScript->GetData(0,&elem_size); + if(elem_size == 1) + { + elem = lpScript->GetData(lpScript->GetNumElements()-1,&elem_size); + if( (elem[0]>0x50) && (elem[0]<0x60)) + { + uint160 hash=Hash160(elem,elem+elem_size); + addressFrom=CScriptID(hash); + take_it=true; + } + } + else + { + elem = lpScript->GetData(lpScript->GetNumElements()-1,&elem_size); + uint160 hash=Hash160(elem,elem+elem_size); + addressFrom=CKeyID(hash); + take_it=true; + } + } + + if(take_it) + { + if( (addresses == NULL) || (addresses->find(addressFrom) == addresses->end()) ) + { + if(fIsMine) + { + if(my_addresses->find(addressFrom) == my_addresses->end()) + { + my_addresses->insert(addressFrom); + } + } + else + { + if(from_addresses->find(addressFrom) == from_addresses->end()) + { + from_addresses->insert(addressFrom); + } + } + } + else + { + if(my_addresses->find(addressFrom) == my_addresses->end()) + { + my_addresses->insert(addressFrom); + } + } + } + } + } + + return nIsMineCount; +} + +Object ListWalletTransactions(const CWalletTx& wtx, bool fLong, const isminefilter& filter,const set* addresses,mc_Buffer *amounts,mc_Script *lpScript) +{ + Object entry; + int nIsFromMeCount,nIsToMeCount; + uint32_t new_entity_type; + CAmount nAmount; + Array issue; + Array assets; + set my_addresses; + set to_addresses; + set from_addresses; + set streams_already_seen; + Array aToAddresses; + Array aFromAddresses; + Array aMyAddresses; + Array aPermissions; + Array aMetaData; + Array aItems; + + nIsFromMeCount=GetInputOffer(wtx,addresses,filter,nAmount,amounts,lpScript,&from_addresses,&my_addresses); + nIsToMeCount=0; + + nAmount=-nAmount; + for(int i=0;iGetCount();i++) + { + uint64_t quantity=mc_GetABQuantity(amounts->GetRow(i)); + quantity=-quantity; + mc_SetABQuantity(amounts->GetRow(i),quantity); + } + + new_entity_type=MC_ENT_TYPE_NONE; + nIsToMeCount=0; + for (int i = 0; i < (int)wtx.vout.size(); ++i) + { + const CTxOut& txout = wtx.vout[i]; + if(!txout.scriptPubKey.IsUnspendable()) + { + string strFailReason; + int required=0; + + txnouttype typeRet; + int nRequiredRet; + vector addressRets; + bool fIsMine=false; + + if(IsTxOutMineAndInList(pwalletMain,txout,addresses,filter)) + { + fIsMine=true; + nIsToMeCount++; + ParseMultichainTxOutToBuffer(wtx.GetHash(),txout,amounts,lpScript,NULL,&required,strFailReason); +// CreateAssetBalanceList(txout,amounts,lpScript,&required); + nAmount+=txout.nValue; + } + if(ExtractDestinations(txout.scriptPubKey,typeRet,addressRets,nRequiredRet)) + { + if(addressRets.size() == 1) // Details of multisig should not appear top-level myaddresses/addresses + { + for (int j = 0; j < (int)addressRets.size(); j++) + { + if( !fIsMine ) + { + if(to_addresses.find(addressRets[j]) == to_addresses.end()) + { + to_addresses.insert(addressRets[j]); + } + } + else + { + if(my_addresses.find(addressRets[j]) == my_addresses.end()) + { + my_addresses.insert(addressRets[j]); + } + } + } + } + } + } + else + { + const CScript& script2 = wtx.vout[i].scriptPubKey; + CScript::const_iterator pc2 = script2.begin(); + + lpScript->Clear(); + lpScript->SetScript((unsigned char*)(&pc2[0]),(size_t)(script2.end()-pc2),MC_SCR_TYPE_SCRIPTPUBKEY); + + size_t elem_size; + const unsigned char *elem; + + if(lpScript->GetNumElements()<=1) + { + if(lpScript->GetNumElements()==1) + { + elem = lpScript->GetData(lpScript->GetNumElements()-1,&elem_size); + aMetaData.push_back(OpReturnEntry(elem,elem_size,wtx.GetHash(),i)); + } + } + else + { + elem = lpScript->GetData(lpScript->GetNumElements()-1,&elem_size); + if(elem_size) + { + aMetaData.push_back(OpReturnEntry(elem,elem_size,wtx.GetHash(),i)); + } + + lpScript->SetElement(0); + if(lpScript->GetNewEntityType(&new_entity_type)) + { + if(lpScript->GetNumElements()==3) + { + unsigned char short_txid[MC_AST_SHORT_TXID_SIZE]; + if(lpScript->GetEntity(short_txid) == 0) + { + unsigned char details_script[MC_ENT_MAX_SCRIPT_SIZE]; + int details_script_size,entity_update; + + lpScript->SetElement(1);; + lpScript->GetNewEntityType(&new_entity_type,&entity_update,details_script,&details_script_size); + } + } + } + + } + } + } + + assets=AssetArrayFromAmounts(amounts,-1,wtx.GetHash(),0); + + uint64_t total=0; + Array aIssueAddresses; + set issue_addresses; + Object asset_entry; + + for (int i = 0; i < (int)wtx.vout.size(); ++i) + { + const CTxOut& txout = wtx.vout[i]; + if(!txout.scriptPubKey.IsUnspendable()) + { + string strFailReason; + int required=0; + amounts->Clear(); +// ParseMultichainTxOutToBuffer(wtx.GetHash(),txout,amounts,lpScript,NULL,&required,strFailReason); + CreateAssetBalanceList(txout,amounts,lpScript,&required); + if(required & (MC_PTP_ADMIN | MC_PTP_ACTIVATE)) + { + Array this_permissions=PermissionEntries(txout,lpScript,true); + for(int j=0;j<(int)this_permissions.size();j++) + { + aPermissions.push_back(this_permissions[j]); + } + } + if(required & MC_PTP_ISSUE) + { + total=1; + uint256 txid=wtx.GetHash(); + asset_entry=AssetEntry((unsigned char*)&txid,-total,7); + txnouttype typeRet; + int nRequiredRet; + vector addressRets; + + if(ExtractDestinations(txout.scriptPubKey,typeRet,addressRets,nRequiredRet)) + { + for (int j = 0; j < (int)addressRets.size(); j++) + { + if(issue_addresses.find(addressRets[j]) == issue_addresses.end()) + { + issue_addresses.insert(addressRets[j]); + } + } + } + + } + } + } + + if(total) + { + BOOST_FOREACH(const CTxDestination& addr, issue_addresses) + { + aIssueAddresses.push_back(CBitcoinAddress(addr).ToString()); + } + asset_entry.push_back(Pair("addresses", aIssueAddresses)); + } + + //aIssue.push_back(asset_entry); + + Array vin; + Array vout; + + if(fLong) + { + BOOST_FOREACH(const CTxIn& txin, wtx.vin) + { + CTxOut TxOutIn; + vin.push_back(TxOutEntry(TxOutIn,-1,txin,txin.prevout.hash,amounts,lpScript)); + } + } + for (int i = 0; i < (int)wtx.vout.size(); ++i) + { + CTxIn TxIn; + Value data_item_entry=DataItemEntry(wtx,i,streams_already_seen,0x01); + if(!data_item_entry.is_null()) + { + aItems.push_back(data_item_entry); + } + if(fLong) + { + Array aTxOutItems; + if(!data_item_entry.is_null()) + { + aTxOutItems.push_back(data_item_entry); + } + Object txout_entry=TxOutEntry(wtx.vout[i],i,TxIn,wtx.GetHash(),amounts,lpScript); + txout_entry.push_back(Pair("items", aTxOutItems)); + vout.push_back(txout_entry); + } + } + + if( (nIsFromMeCount+nIsToMeCount) == 0 ) + { + return entry; + } + + + + + BOOST_FOREACH(const CTxDestination& addr, my_addresses) + { + aMyAddresses.push_back(CBitcoinAddress(addr).ToString()); + } + + BOOST_FOREACH(const CTxDestination& addr, from_addresses) + { + aFromAddresses.push_back(CBitcoinAddress(addr).ToString()); + } + + BOOST_FOREACH(const CTxDestination& addr, to_addresses) + { +// aToAddresses.push_back(CBitcoinAddress(addr).ToString()); + if(from_addresses.find(addr) == from_addresses.end()) + { + aFromAddresses.push_back(CBitcoinAddress(addr).ToString()); + } + } + + Object oBalance; + oBalance.push_back(Pair("amount", ValueFromAmount(nAmount))); + oBalance.push_back(Pair("assets", assets)); + entry.push_back(Pair("balance", oBalance)); + entry.push_back(Pair("myaddresses", aMyAddresses)); + entry.push_back(Pair("addresses", aFromAddresses)); +// entry.push_back(Pair("fromaddresses", aFromAddresses)); +// entry.push_back(Pair("toaddresses", aToAddresses)); + entry.push_back(Pair("permissions", aPermissions)); + if(new_entity_type == MC_ENT_TYPE_STREAM) + { + uint256 txid=wtx.GetHash(); + entry.push_back(Pair("create", StreamEntry((unsigned char*)&txid,0x05))); + } + if(asset_entry.size()) + { + entry.push_back(Pair("issue", asset_entry)); + } + else + { + if(new_entity_type == MC_ENT_TYPE_ASSET) + { + uint256 txid=wtx.GetHash(); + entry.push_back(Pair("issue", AssetEntry((unsigned char*)&txid,-total,7))); + } + } + entry.push_back(Pair("items", aItems)); + entry.push_back(Pair("data", aMetaData)); + + WalletTxToJSON(wtx, entry, true); + + if(fLong) + { + entry.push_back(Pair("vin", vin)); + entry.push_back(Pair("vout", vout)); + string strHex = EncodeHexTx(static_cast(wtx)); + entry.push_back(Pair("hex", strHex)); + } +// ret.push_back(entry); + + return entry; +} + +Value listwallettransactions(const Array& params, bool fHelp) +{ + if (fHelp || params.size() > 4) // MCHN + throw runtime_error("Help message not found\n"); + + int nCount = 10; + if (params.size() > 0) + nCount = params[0].get_int(); + int nFrom = 0; + if (params.size() > 1) + nFrom = params[1].get_int(); + isminefilter filter = ISMINE_SPENDABLE; + if(params.size() > 2) + if(params[2].get_bool()) + filter = filter | ISMINE_WATCH_ONLY; + bool fLong=false; + if(params.size() > 3) + if(params[3].get_bool()) + fLong=true; + + if (nCount < 0) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Negative count"); + if (nFrom < 0) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Negative from"); + + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; + + mc_Buffer *entity_rows; + entity_rows=new mc_Buffer; + entity_rows->Initialize(MC_TDB_ENTITY_KEY_SIZE,MC_TDB_ROW_SIZE,MC_BUF_MODE_DEFAULT); + + Array ret; + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxEntity wallet_by_time; + mc_TxEntityRow *lpEntTx; + wallet_by_time.Zero(); + wallet_by_time.m_EntityType=MC_TET_TIMERECEIVED; + if(filter & ISMINE_WATCH_ONLY) + { + wallet_by_time.m_EntityType |= MC_TET_WALLET_ALL; + } + else + { + wallet_by_time.m_EntityType |= MC_TET_WALLET_SPENDABLE; + } + pwalletTxsMain->GetList(&wallet_by_time,-nFrom,nCount,entity_rows); + for(int i=0;iGetCount();i++) + { + lpEntTx=(mc_TxEntityRow*)entity_rows->GetRow(i); + uint256 hash; + memcpy(&hash,lpEntTx->m_TxId,MC_TDB_TXID_SIZE); + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + Object entry=ListWalletTransactions(wtx,fLong,filter,NULL,asset_amounts,lpScript); + if(entry.size()) + { + ret.push_back(entry); + } + } + } + else + { + std::list acentries; + CWallet::TxItems txOrdered = pwalletMain->OrderedTxItems(acentries, "*"); + // iterate backwards until we have nCount items to return: + for (CWallet::TxItems::reverse_iterator it = txOrdered.rbegin(); it != txOrdered.rend(); ++it) + { + CWalletTx *const pwtx = (*it).second.first; + if (pwtx != 0) + { + Object entry=ListWalletTransactions(*pwtx,fLong,filter,NULL,asset_amounts,lpScript); + if(entry.size()) + { + ret.push_back(entry); + } + } + + if ((int)ret.size() >= (nCount+nFrom)) break; + } + // ret is newest to oldest + + if (nFrom > (int)ret.size()) + nFrom = ret.size(); + if ((nFrom + nCount) > (int)ret.size()) + nCount = ret.size() - nFrom; + Array::iterator first = ret.begin(); + std::advance(first, nFrom); + Array::iterator last = ret.begin(); + std::advance(last, nFrom+nCount); + + if (last != ret.end()) ret.erase(last, ret.end()); + if (first != ret.begin()) ret.erase(ret.begin(), first); + + std::reverse(ret.begin(), ret.end()); // Return oldest to newest + } + + + delete lpScript; + delete asset_amounts; + delete entity_rows; + + + return ret; +} + +Value listaddresstransactions(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 5) + throw runtime_error("Help message not found\n"); + + int nCount = 10; + if (params.size() > 1) + nCount = params[1].get_int(); + int nFrom = 0; + if (params.size() > 2) + nFrom = params[2].get_int(); + isminefilter filter = ISMINE_ALL; + bool fLong=false; + if(params.size() > 3) + if(params[3].get_bool()) + fLong=true; + + if (nCount < 0) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Negative count"); + if (nFrom < 0) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Negative from"); + + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; + + vector fromaddresses; + + if (params.size() > 0) + { + fromaddresses=ParseAddresses(params[0].get_str(),false,true); + } + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + set thisFromAddresses; + set *lpFromAddresses; + + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + thisFromAddresses.insert(fromaddress); + } + + lpFromAddresses=NULL; + if(thisFromAddresses.size()) + { + lpFromAddresses=&thisFromAddresses; + } + + + mc_Buffer *entity_rows; + entity_rows=new mc_Buffer; + entity_rows->Initialize(MC_TDB_ENTITY_KEY_SIZE,MC_TDB_ROW_SIZE,MC_BUF_MODE_DEFAULT); + + Array ret; + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxEntity address_by_time; + mc_TxEntityRow *lpEntTx; + address_by_time.Zero(); + const CKeyID *lpKeyID=boost::get (&fromaddresses[0]); + const CScriptID *lpScriptID=boost::get (&fromaddresses[0]); + if(lpKeyID) + { + memcpy(address_by_time.m_EntityID,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + address_by_time.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_TIMERECEIVED; + } + if(lpScriptID) + { + memcpy(address_by_time.m_EntityID,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + address_by_time.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_TIMERECEIVED; + } + pwalletTxsMain->GetList(&address_by_time,-nFrom,nCount,entity_rows); + for(int i=0;iGetCount();i++) + { + lpEntTx=(mc_TxEntityRow*)entity_rows->GetRow(i); + uint256 hash; + memcpy(&hash,lpEntTx->m_TxId,MC_TDB_TXID_SIZE); + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + Object entry=ListWalletTransactions(wtx,fLong,filter,lpFromAddresses,asset_amounts,lpScript); + if(entry.size()) + { + ret.push_back(entry); + } + } + } + else + { + std::list acentries; + CWallet::TxItems txOrdered = pwalletMain->OrderedTxItems(acentries, "*"); + + // iterate backwards until we have nCount items to return: + for (CWallet::TxItems::reverse_iterator it = txOrdered.rbegin(); it != txOrdered.rend(); ++it) + { + CWalletTx *const pwtx = (*it).second.first; + if (pwtx != 0) + { + Object entry=ListWalletTransactions(*pwtx,fLong,filter,lpFromAddresses,asset_amounts,lpScript); + if(entry.size()) + { + ret.push_back(entry); + } + } + + if ((int)ret.size() >= (nCount+nFrom)) break; + } + // ret is newest to oldest + + if (nFrom > (int)ret.size()) + nFrom = ret.size(); + if ((nFrom + nCount) > (int)ret.size()) + nCount = ret.size() - nFrom; + Array::iterator first = ret.begin(); + std::advance(first, nFrom); + Array::iterator last = ret.begin(); + std::advance(last, nFrom+nCount); + + if (last != ret.end()) ret.erase(last, ret.end()); + if (first != ret.begin()) ret.erase(ret.begin(), first); + + std::reverse(ret.begin(), ret.end()); // Return oldest to newest + } + + delete lpScript; + delete asset_amounts; + delete entity_rows; + + + return ret; +} + +Value getwallettransaction(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 1 || params.size() > 3) // MCHN + throw runtime_error("Help message not found\n"); + + uint256 hash; + hash.SetHex(params[0].get_str()); + + isminefilter filter = ISMINE_SPENDABLE; + if(params.size() > 1) + if(params[1].get_bool()) + filter = filter | ISMINE_WATCH_ONLY; + + bool fLong=false; + if(params.size() > 2) + if(params[2].get_bool()) + fLong=true; + + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + if(pwalletTxsMain->FindWalletTx(hash,NULL)) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid or non-wallet transaction id"); + } + else + { + if (!pwalletMain->mapWallet.count(hash)) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid or non-wallet transaction id"); + } + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; + + Object entry; + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + entry=ListWalletTransactions(wtx,fLong,filter,NULL,asset_amounts,lpScript); + } + else + { + const CWalletTx& wtx = pwalletMain->mapWallet[hash]; + entry=ListWalletTransactions(wtx,fLong,filter,NULL,asset_amounts,lpScript); + } + + delete lpScript; + delete asset_amounts; + + if(entry.size() == 0) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Wallet addresses with specified criteria are not involved in transaction"); + } + + + return entry; +} + +Value getaddresstransaction(const Array& params, bool fHelp) +{ + if (fHelp || params.size() < 2 || params.size() > 3) // MCHN + throw runtime_error("Help message not found\n"); + + uint256 hash; + hash.SetHex(params[1].get_str()); + + isminefilter filter = ISMINE_ALL; + + bool fLong=false; + if(params.size() > 2) + if(params[2].get_bool()) + fLong=true; + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + if(pwalletTxsMain->FindWalletTx(hash,NULL)) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid or non-wallet transaction id"); + } + else + { + if (!pwalletMain->mapWallet.count(hash)) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid or non-wallet transaction id"); + } + + + mc_Buffer *asset_amounts; + asset_amounts=new mc_Buffer; + mc_InitABufferMap(asset_amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; + + vector fromaddresses; + + if (params.size() > 0) + { + fromaddresses=ParseAddresses(params[0].get_str(),false,true); + } + + if(fromaddresses.size() != 1) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Single from-address should be specified"); + } + + set thisFromAddresses; + set *lpFromAddresses; + + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + thisFromAddresses.insert(fromaddress); + } + + lpFromAddresses=NULL; + if(thisFromAddresses.size()) + { + lpFromAddresses=&thisFromAddresses; + } + + Object entry; + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + const CWalletTx& wtx=pwalletTxsMain->GetWalletTx(hash,NULL,NULL); + entry=ListWalletTransactions(wtx,fLong,filter,lpFromAddresses,asset_amounts,lpScript); + } + else + { + const CWalletTx& wtx = pwalletMain->mapWallet[hash]; + entry=ListWalletTransactions(wtx,fLong,filter,lpFromAddresses,asset_amounts,lpScript); + } + + if(entry.size() == 0) + { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Specified address is not involved in transaction"); + } + + delete lpScript; + delete asset_amounts; + + return entry; +} + diff --git a/src/rpc/rpcwalletutils.cpp b/src/rpc/rpcwalletutils.cpp new file mode 100644 index 00000000..9988b79a --- /dev/null +++ b/src/rpc/rpcwalletutils.cpp @@ -0,0 +1,644 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + + +#include "rpc/rpcwallet.h" + +void MinimalWalletTxToJSON(const CWalletTx& wtx, Object& entry) +{ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + uint256 blockHash; + int block=wtx.txDef.m_Block; + int confirms = wtx.GetDepthInMainChain(); + entry.push_back(Pair("confirmations", confirms)); + if (confirms > 0) + { + blockHash=chainActive[block]->GetBlockHash(); + entry.push_back(Pair("blocktime", mapBlockIndex[blockHash]->GetBlockTime())); + } + uint256 hash = wtx.GetHash(); + entry.push_back(Pair("txid", hash.GetHex())); + } + else + { + int confirms = wtx.GetDepthInMainChain(); + entry.push_back(Pair("confirmations", confirms)); + if (confirms > 0) + { + entry.push_back(Pair("blocktime", mapBlockIndex[wtx.hashBlock]->GetBlockTime())); + } + uint256 hash = wtx.GetHash(); + entry.push_back(Pair("txid", hash.GetHex())); + } +} + +void WalletTxToJSON(const CWalletTx& wtx, Object& entry,bool skipWalletConflicts, int vout) +{ + /* MCHN START */ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + int block=wtx.txDef.m_Block; + uint256 blockHash; + int confirms = wtx.GetDepthInMainChain(); + entry.push_back(Pair("confirmations", confirms)); + if (wtx.IsCoinBase()) + entry.push_back(Pair("generated", true)); + int64_t nTimeSmart=(int64_t)wtx.txDef.m_TimeReceived; + if (confirms > 0) + { + blockHash=chainActive[block]->GetBlockHash(); + entry.push_back(Pair("blockhash", chainActive[block]->GetBlockHash().GetHex())); + entry.push_back(Pair("blockindex", (int64_t)wtx.txDef.m_BlockIndex)); + entry.push_back(Pair("blocktime", mapBlockIndex[blockHash]->GetBlockTime())); + if(mapBlockIndex[blockHash]->GetBlockTime()GetBlockTime(); + } + } + uint256 hash = wtx.GetHash(); + entry.push_back(Pair("txid", hash.GetHex())); + if(vout >= 0) + { + entry.push_back(Pair("vout", vout)); + } + Array conflicts; + if(!skipWalletConflicts) + { + entry.push_back(Pair("walletconflicts", conflicts)); + } + if(wtx.txDef.m_Flags & MC_TFL_INVALID) + { + entry.push_back(Pair("valid", false)); + } + else + { + entry.push_back(Pair("valid", true)); + } + entry.push_back(Pair("time", nTimeSmart)); + entry.push_back(Pair("timereceived", (int64_t)wtx.txDef.m_TimeReceived)); + BOOST_FOREACH(const PAIRTYPE(string,string)& item, wtx.mapValue) + { + if((item.first.size() <13) || (memcmp(item.first.c_str(),"fullsize",8) != 0)) + { + entry.push_back(Pair(item.first, item.second)); + } + } + } + else + { + /* MCHN END */ + int confirms = wtx.GetDepthInMainChain(); + entry.push_back(Pair("confirmations", confirms)); + if (wtx.IsCoinBase()) + entry.push_back(Pair("generated", true)); + if (confirms > 0) + { + entry.push_back(Pair("blockhash", wtx.hashBlock.GetHex())); + entry.push_back(Pair("blockindex", wtx.nIndex)); + entry.push_back(Pair("blocktime", mapBlockIndex[wtx.hashBlock]->GetBlockTime())); + } + uint256 hash = wtx.GetHash(); + entry.push_back(Pair("txid", hash.GetHex())); + Array conflicts; + BOOST_FOREACH(const uint256& conflict, wtx.GetConflicts()) + conflicts.push_back(conflict.GetHex()); + /* MCHN START */ + if(!skipWalletConflicts) + { + /* MCHN END */ + entry.push_back(Pair("walletconflicts", conflicts)); + /* MCHN START */ + } + if(conflicts.size()) + { + if(confirms <= 0) + { + entry.push_back(Pair("valid", false)); + } + else + { + entry.push_back(Pair("valid", true)); + } + } + else + { + entry.push_back(Pair("valid", true)); + } + /* MCHN END */ + entry.push_back(Pair("time", wtx.GetTxTime())); + entry.push_back(Pair("timereceived", (int64_t)wtx.nTimeReceived)); + BOOST_FOREACH(const PAIRTYPE(string,string)& item, wtx.mapValue) + entry.push_back(Pair(item.first, item.second)); + } +} + + +//void SendMoneyToSeveralAddresses(const std::vector addresses, CAmount nValue, CWalletTx& wtxNew,mc_Script *dropscript,mc_Script *opreturnscript,const std::vector& fromaddresses) +void SendMoneyToSeveralAddresses(const std::vector addresses, CAmount nValue, CWalletTx& wtxNew,mc_Script *dropscript,CScript scriptOpReturn,const std::vector& fromaddresses) +{ + // Check amount + +/* + if (nValue < 0) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid amount"); +*/ + string strError; + if (pwalletMain->IsLocked()) + { + strError = "Error: Wallet locked, unable to create transaction!"; + LogPrintf("SendMoney() : %s", strError); + throw JSONRPCError(RPC_WALLET_ERROR, strError); + } + + set thisFromAddresses; + set *lpFromAddresses; + + BOOST_FOREACH(const CTxDestination& fromaddress, fromaddresses) + { + thisFromAddresses.insert(fromaddress); + } + + lpFromAddresses=NULL; + if(thisFromAddresses.size()) + { + lpFromAddresses=&thisFromAddresses; + } + + // Parse Bitcoin address + + + std::vector scriptPubKeys; + size_t elem_size; + const unsigned char *elem; + + BOOST_FOREACH (const CTxDestination& address, addresses) + { + + CScript scriptPubKey = GetScriptForDestination(address); + + + if(dropscript) + { + LogPrint("mchnminor","mchn: Sending script with %d OP_DROP element(s)",dropscript->GetNumElements()); + if(dropscript->GetNumElements() > mc_gState->m_NetworkParams->GetInt64Param("maxstdopdropscount") ) + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid number of elements in script"); + + for(int element=0;element < dropscript->GetNumElements();element++) + { + elem = dropscript->GetData(element,&elem_size); + if(elem) + { + scriptPubKey << vector(elem, elem + elem_size) << OP_DROP; + } + else + throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid script"); + } + } + + scriptPubKeys.push_back(scriptPubKey); + } + + // Create and send the transaction + + CReserveKey reservekey(pwalletMain); + CAmount nFeeRequired; + if (!pwalletMain->CreateTransaction(scriptPubKeys, nValue, scriptOpReturn, wtxNew, reservekey, nFeeRequired, strError, NULL, lpFromAddresses)) + { +/* + if (nValue + nFeeRequired > pwalletMain->GetBalance()) + strError = strprintf("Error: This transaction requires a transaction fee of at least %s because of its amount, complexity, or use of recently received funds!", FormatMoney(nFeeRequired)); + */ + LogPrintf("SendMoney() : %s\n", strError); + throw JSONRPCError(RPC_WALLET_ERROR, strError); + } + + string strRejectReason; + if (!pwalletMain->CommitTransaction(wtxNew, reservekey, strRejectReason)) + { + if(strRejectReason.size()) + { + throw JSONRPCError(RPC_WALLET_ERROR, "Error: The transaction was rejected: " + strRejectReason); + } + else + { + throw JSONRPCError(RPC_WALLET_ERROR, "Error: The transaction was rejected! This might happen if some of the coins in your wallet were already spent, such as if you used a copy of wallet.dat and coins were spent in the copy but not marked as spent here."); + } + } +} + +vector ParseAddresses(string param, bool create_full_list, bool allow_scripthash) +{ + vector addresses; + set thisAddresses; + set setAddress; + + string tok; + + CKeyID *lpKeyID; + if( param == "*" ) + { + if(create_full_list) + { + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, pwalletMain->mapAddressBook) + { + const CBitcoinAddress& address = item.first; + CTxDestination dest=address.Get(); + lpKeyID=boost::get (&dest); + + if(lpKeyID) + { + addresses.push_back(address.Get()); + } + } + } + } + else + { + stringstream ss(param); + while(getline(ss, tok, ',')) + { + CBitcoinAddress address(tok); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, string("Invalid address: ")+tok); + if (setAddress.count(address)) + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid parameter, duplicated address: ")+tok); + CTxDestination dest=address.Get(); + CKeyID *lpKeyID=boost::get (&dest); + CScriptID *lpScriptID=boost::get (&dest); + + if( (lpKeyID != NULL) || (( lpScriptID != NULL) && allow_scripthash )) + { + addresses.push_back(address.Get()); + setAddress.insert(address); + } + else + { + if(allow_scripthash) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid address (only pubkeyhash and scripthash addresses are supported) : ")+tok); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid address (only pubkeyhash addresses are supported) : ")+tok); + } + } + } + } + + return addresses; +} + +void FindAddressesWithPublishPermission(vector& fromaddresses,mc_EntityDetails *stream_entity) +{ + if(fromaddresses.size() == 1) + { + const unsigned char *aptr; + + aptr=GetAddressIDPtr(fromaddresses[0]); + if(aptr) + { + if((stream_entity->AnyoneCanWrite() == 0) && (mc_gState->m_Permissions->CanWrite(stream_entity->GetTxID(),aptr) == 0)) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Publishing in this stream is not allowed from this address"); + } + if(mc_gState->m_Permissions->CanSend(NULL,aptr) == 0) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "from-address doesn't have send permission"); + } + } + } + else + { + bool publisher_found=false; + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, pwalletMain->mapAddressBook) + { + const CBitcoinAddress& address = item.first; + CKeyID keyID; + + if(address.GetKeyID(keyID)) + { + if((stream_entity->AnyoneCanWrite() != 0) || (mc_gState->m_Permissions->CanWrite(stream_entity->GetTxID(),(unsigned char*)(&keyID)) != 0)) + { + if(mc_gState->m_Permissions->CanSend(NULL,(unsigned char*)(&keyID))) + { + publisher_found=true; + fromaddresses.push_back(address.Get()); + } + } + } + } + if(!publisher_found) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, "This wallet contains no addresses with permission to write to this stream and global send permission."); + } + } +} + +set ParseAddresses(Value param, isminefilter filter) +{ + set setAddresses; + + if(param.type() == array_type) + { + BOOST_FOREACH(const Value& vtok, param.get_array()) + { + if(vtok.type() == str_type) + { + string tok=vtok.get_str(); + CBitcoinAddress address(tok); + if (!address.IsValid()) + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, string("Invalid address: ")+tok); + if (setAddresses.count(address.ToString())) + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid parameter, duplicate address: ")+tok); + const CTxDestination dest=address.Get(); + const CKeyID *lpKeyID=boost::get (&dest); + const CScriptID *lpScriptID=boost::get (&dest); + + if( (lpKeyID != NULL) || ( lpScriptID != NULL)) + { + if(filter != ISMINE_NO) + { + isminetype fIsMine=IsMine(*pwalletMain,dest); + if (!(fIsMine & filter)) + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Non-wallet address : ")+tok); + } + } + + setAddresses.insert(address.ToString()); + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid address (only pubkeyhash and scripthash addresses are supported) : ")+tok); + } + } + else + { + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid address, expected string")); + } + } + } + else + { + vector fromaddresses; + fromaddresses=ParseAddresses(param.get_str(),false,true); + BOOST_FOREACH(const CTxDestination& dest, fromaddresses) + { + isminetype fIsMine=IsMine(*pwalletMain,dest); + if(filter != ISMINE_NO) + { + if (!(fIsMine & filter)) + { + CBitcoinAddress address(dest); + throw JSONRPCError(RPC_INVALID_PARAMETER, string("Non-wallet address : ")+address.ToString().c_str()); + } + } + setAddresses.insert(CBitcoinAddress(dest).ToString()); + } + } + + return setAddresses; +} + +bool CBitcoinAddressFromTxEntity(CBitcoinAddress &address,mc_TxEntity *lpEntity) +{ + bool is_address=false; + CTxDestination dest=CTxDestination(); + address=CBitcoinAddress(); + if((lpEntity->m_EntityType & MC_TET_TYPE_MASK) == MC_TET_PUBKEY_ADDRESS) + { + unsigned char *ptr=lpEntity->m_EntityID; + dest=CKeyID(*(uint160*)ptr); + is_address=true; + } + if((lpEntity->m_EntityType & MC_TET_TYPE_MASK) == MC_TET_SCRIPT_ADDRESS) + { + unsigned char *ptr=lpEntity->m_EntityID; + dest=CScriptID(*(uint160*)ptr); + is_address=true; + } + if(is_address) + { + address=CBitcoinAddress(dest); + return true; + } + return false; +} + +Object StreamItemEntry(const CWalletTx& wtx,const unsigned char *stream_id, bool verbose) +{ + Object entry; + Array publishers; + set publishers_set; + Array items; + int stream_output; + const unsigned char *ptr; + unsigned char item_key[MC_ENT_MAX_ITEM_KEY_SIZE+1]; + int item_key_size; + Value item_value; + + stream_output=-1; + for (int j = 0; j < (int)wtx.vout.size(); ++j) + { + if(stream_output < 0) + { + const CScript& script1 = wtx.vout[j].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + if(mc_gState->m_TmpScript->IsOpReturnScript()) + { + if(mc_gState->m_TmpScript->GetNumElements()) // 2 OP_DROPs + OP_RETURN - item key + { + unsigned char short_txid[MC_AST_SHORT_TXID_SIZE]; + mc_gState->m_TmpScript->SetElement(0); + + if(mc_gState->m_TmpScript->GetEntity(short_txid) == 0) + { + if(memcmp(short_txid,stream_id,MC_AST_SHORT_TXID_SIZE) == 0) + { + stream_output=j; + mc_gState->m_TmpScript->SetElement(1); + // Should be spkk + if(mc_gState->m_TmpScript->GetItemKey(item_key,&item_key_size)) // Item key + { + return entry; + } + item_key[item_key_size]=0; + + size_t elem_size; + const unsigned char *elem; + + elem = mc_gState->m_TmpScript->GetData(2,&elem_size); + item_value=OpReturnEntry(elem,elem_size,wtx.GetHash(),j); + } + } + } + } + } + } + + if(stream_output < 0) + { + return entry; + } + + publishers_set.clear(); + for (int i = 0; i < (int)wtx.vin.size(); ++i) + { + int op_addr_offset,op_addr_size,is_redeem_script,sighash_type; + + const CScript& script2 = wtx.vin[i].scriptSig; + CScript::const_iterator pc2 = script2.begin(); + + ptr=mc_ExtractAddressFromInputScript((unsigned char*)(&pc2[0]),(int)(script2.end()-pc2),&op_addr_offset,&op_addr_size,&is_redeem_script,&sighash_type,0); + if(ptr) + { + if( (sighash_type == SIGHASH_ALL) || ( (sighash_type == SIGHASH_SINGLE) && (i == stream_output) ) ) + { + uint160 publisher_hash=Hash160(ptr+op_addr_offset,ptr+op_addr_offset+op_addr_size); + if(publishers_set.count(publisher_hash) == 0) + { + publishers_set.insert(publisher_hash); + if(is_redeem_script) + { + publishers.push_back(CBitcoinAddress((CScriptID)publisher_hash).ToString()); + } + else + { + publishers.push_back(CBitcoinAddress((CKeyID)publisher_hash).ToString()); + } + } + } + } + } + + entry.push_back(Pair("publishers", publishers)); + entry.push_back(Pair("key", strprintf("%s",item_key))); + entry.push_back(Pair("data", item_value)); + + if(verbose) + { + WalletTxToJSON(wtx, entry, true, stream_output); + } + else + { + MinimalWalletTxToJSON(wtx, entry); + } + + return entry; +} + + +Object TxOutEntry(const CTxOut& TxOutIn,int vout,const CTxIn& TxIn,uint256 hash,mc_Buffer *amounts,mc_Script *lpScript) +{ + bool fIsInput=false; + bool fIsFound=false; + if(TxIn.prevout.hash != 0) + { + fIsInput=true; + } + + Object txout_entry; + Array permissions; + isminetype fIsMine=ISMINE_NO; + amounts->Clear(); +// int iad=-1; + CTxOut txout; + if(fIsInput) + { + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + int err; + const CWalletTx& prev=pwalletTxsMain->GetWalletTx(TxIn.prevout.hash,NULL,&err); + if(err == MC_ERR_NOERROR) + { + if (TxIn.prevout.n < prev.vout.size()) + { + txout=prev.vout[TxIn.prevout.n]; + fIsFound=true; + } + } + } + else + { + map::const_iterator mi = pwalletMain->mapWallet.find(TxIn.prevout.hash); + if (mi != pwalletMain->mapWallet.end()) + { + const CWalletTx& prev = (*mi).second; + if (TxIn.prevout.n < prev.vout.size()) + { + txout=prev.vout[TxIn.prevout.n]; + fIsFound=true; + } + } + } + } + else + { + txout=TxOutIn; + fIsFound=true; + } + + if(fIsInput) + { + txout_entry.push_back(Pair("txid", TxIn.prevout.hash.ToString())); + txout_entry.push_back(Pair("vout", (int)TxIn.prevout.n)); + } + else + { + txout_entry.push_back(Pair("n", vout)); + } + + if(fIsFound) + { + txnouttype typeRet; + txout_entry.push_back(Pair("addresses", AddressEntries(txout,typeRet))); + txout_entry.push_back(Pair("type", GetTxnOutputType(typeRet))); + + fIsMine=pwalletMain->IsMine(txout); + string strFailReason; + int required=0; + if(ParseMultichainTxOutToBuffer(hash,txout,amounts,lpScript,NULL,&required,strFailReason)) + { + if(required & (MC_PTP_ADMIN | MC_PTP_ACTIVATE) ) + { + if(!fIsInput) + { + permissions=PermissionEntries(txout,lpScript,false); + } + } + } + } + else + { + if(fIsInput) + { + txnouttype typeRet; + txout_entry.push_back(Pair("addresses", AddressEntries(TxIn,typeRet,lpScript))); + txout_entry.push_back(Pair("type", GetTxnOutputType(typeRet))); + } + } + + txout_entry.push_back(Pair("ismine", (fIsMine & ISMINE_SPENDABLE) ? true : false)); + txout_entry.push_back(Pair("iswatchonly", (fIsMine & ISMINE_WATCH_ONLY) ? true : false)); + + if(fIsFound) + { + txout_entry.push_back(Pair("amount", ValueFromAmount(txout.nValue))); + txout_entry.push_back(Pair("assets", AssetArrayFromAmounts(amounts,-1,hash,fIsInput ? 0 : 1))); + if(!fIsInput) + { + txout_entry.push_back(Pair("permissions", permissions)); + } + } + + return txout_entry; +} + diff --git a/src/script/bitcoinconsensus.cpp b/src/script/bitcoinconsensus.cpp new file mode 100644 index 00000000..05bb1734 --- /dev/null +++ b/src/script/bitcoinconsensus.cpp @@ -0,0 +1,99 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "bitcoinconsensus.h" + +#include "primitives/transaction.h" +#include "keys/pubkey.h" +#include "script/interpreter.h" +#include "version/bcversion.h" + +namespace { + +/** A class that deserializes a single CTransaction one time. */ +class TxInputStream +{ +public: + TxInputStream(int nTypeIn, int nVersionIn, const unsigned char *txTo, size_t txToLen) : + m_type(nTypeIn), + m_version(nVersionIn), + m_data(txTo), + m_remaining(txToLen) + {} + + TxInputStream& read(char* pch, size_t nSize) + { + if (nSize > m_remaining) + throw std::ios_base::failure(std::string(__func__) + ": end of data"); + + if (pch == NULL) + throw std::ios_base::failure(std::string(__func__) + ": bad destination buffer"); + + if (m_data == NULL) + throw std::ios_base::failure(std::string(__func__) + ": bad source buffer"); + + memcpy(pch, m_data, nSize); + m_remaining -= nSize; + m_data += nSize; + return *this; + } + + template + TxInputStream& operator>>(T& obj) + { + ::Unserialize(*this, obj, m_type, m_version); + return *this; + } + +private: + const int m_type; + const int m_version; + const unsigned char* m_data; + size_t m_remaining; +}; + +inline int set_error(bitcoinconsensus_error* ret, bitcoinconsensus_error serror) +{ + if (ret) + *ret = serror; + return 0; +} + +struct ECCryptoClosure +{ + ECCVerifyHandle handle; +}; + +ECCryptoClosure instance_of_eccryptoclosure; +} + +int bitcoinconsensus_verify_script(const unsigned char *scriptPubKey, unsigned int scriptPubKeyLen, + const unsigned char *txTo , unsigned int txToLen, + unsigned int nIn, unsigned int flags, bitcoinconsensus_error* err) +{ + try { + TxInputStream stream(SER_NETWORK, PROTOCOL_VERSION, txTo, txToLen); + CTransaction tx; + stream >> tx; + if (nIn >= tx.vin.size()) + return set_error(err, bitcoinconsensus_ERR_TX_INDEX); + if (tx.GetSerializeSize(SER_NETWORK, PROTOCOL_VERSION) != txToLen) + return set_error(err, bitcoinconsensus_ERR_TX_SIZE_MISMATCH); + + // Regardless of the verification result, the tx did not error. + set_error(err, bitcoinconsensus_ERR_OK); + + return VerifyScript(tx.vin[nIn].scriptSig, CScript(scriptPubKey, scriptPubKey + scriptPubKeyLen), flags, TransactionSignatureChecker(&tx, nIn), NULL); + } catch (const std::exception&) { + return set_error(err, bitcoinconsensus_ERR_TX_DESERIALIZE); // Error deserializing + } +} + +unsigned int bitcoinconsensus_version() +{ + // Just use the API version for now + return BITCOINCONSENSUS_API_VER; +} diff --git a/src/script/bitcoinconsensus.h b/src/script/bitcoinconsensus.h new file mode 100644 index 00000000..40a094a9 --- /dev/null +++ b/src/script/bitcoinconsensus.h @@ -0,0 +1,69 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_BITCOINCONSENSUS_H +#define BITCOIN_BITCOINCONSENSUS_H + +#if defined(BUILD_BITCOIN_INTERNAL) && defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" + #if defined(_WIN32) + #if defined(DLL_EXPORT) + #if defined(HAVE_FUNC_ATTRIBUTE_DLLEXPORT) + #define EXPORT_SYMBOL __declspec(dllexport) + #else + #define EXPORT_SYMBOL + #endif + #endif + #elif defined(HAVE_FUNC_ATTRIBUTE_VISIBILITY) + #define EXPORT_SYMBOL __attribute__ ((visibility ("default"))) + #endif +#elif defined(MSC_VER) && !defined(STATIC_LIBBITCOINCONSENSUS) + #define EXPORT_SYMBOL __declspec(dllimport) +#endif + +#ifndef EXPORT_SYMBOL + #define EXPORT_SYMBOL +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +#define BITCOINCONSENSUS_API_VER 0 + +typedef enum bitcoinconsensus_error_t +{ + bitcoinconsensus_ERR_OK = 0, + bitcoinconsensus_ERR_TX_INDEX, + bitcoinconsensus_ERR_TX_SIZE_MISMATCH, + bitcoinconsensus_ERR_TX_DESERIALIZE, +} bitcoinconsensus_error; + +/** Script verification flags */ +enum +{ + bitcoinconsensus_SCRIPT_FLAGS_VERIFY_NONE = 0, + bitcoinconsensus_SCRIPT_FLAGS_VERIFY_P2SH = (1U << 0), // evaluate P2SH (BIP16) subscripts + bitcoinconsensus_SCRIPT_FLAGS_VERIFY_DERSIG = (1U << 2), // enforce strict DER (BIP66) compliance +}; + +/// Returns 1 if the input nIn of the serialized transaction pointed to by +/// txTo correctly spends the scriptPubKey pointed to by scriptPubKey under +/// the additional constraints specified by flags. +/// If not NULL, err will contain an error/success code for the operation +EXPORT_SYMBOL int bitcoinconsensus_verify_script(const unsigned char *scriptPubKey, unsigned int scriptPubKeyLen, + const unsigned char *txTo , unsigned int txToLen, + unsigned int nIn, unsigned int flags, bitcoinconsensus_error* err); + +EXPORT_SYMBOL unsigned int bitcoinconsensus_version(); + +#ifdef __cplusplus +} // extern "C" +#endif + +#undef EXPORT_SYMBOL + +#endif // BITCOIN_BITCOINCONSENSUS_H diff --git a/src/script/interpreter.cpp b/src/script/interpreter.cpp new file mode 100644 index 00000000..73a5f926 --- /dev/null +++ b/src/script/interpreter.cpp @@ -0,0 +1,1252 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "interpreter.h" +/* MCHN START */ +#include "script/standard.h" +/* MCHN END */ + +#include "primitives/transaction.h" +#include "crypto/ripemd160.h" +#include "crypto/sha1.h" +#include "crypto/sha256.h" +#include "keys/pubkey.h" +#include "script/script.h" +#include "structs/uint256.h" + +using namespace std; + +typedef vector valtype; + +namespace { + +inline bool set_success(ScriptError* ret) +{ + if (ret) + *ret = SCRIPT_ERR_OK; + return true; +} + +inline bool set_error(ScriptError* ret, const ScriptError serror) +{ + if (ret) + *ret = serror; + return false; +} + +} // anon namespace + +bool CastToBool(const valtype& vch) +{ + for (unsigned int i = 0; i < vch.size(); i++) + { + if (vch[i] != 0) + { + // Can be negative zero + if (i == vch.size()-1 && vch[i] == 0x80) + return false; + return true; + } + } + return false; +} + +/** + * Script is a stack machine (like Forth) that evaluates a predicate + * returning a bool indicating valid or not. There are no loops. + */ +#define stacktop(i) (stack.at(stack.size()+(i))) +#define altstacktop(i) (altstack.at(altstack.size()+(i))) +static inline void popstack(vector& stack) +{ + if (stack.empty()) + throw runtime_error("popstack() : stack empty"); + stack.pop_back(); +} + +bool static IsCompressedOrUncompressedPubKey(const valtype &vchPubKey) { + if (vchPubKey.size() < 33) { + // Non-canonical public key: too short + return false; + } + if (vchPubKey[0] == 0x04) { + if (vchPubKey.size() != 65) { + // Non-canonical public key: invalid length for uncompressed key + return false; + } + } else if (vchPubKey[0] == 0x02 || vchPubKey[0] == 0x03) { + if (vchPubKey.size() != 33) { + // Non-canonical public key: invalid length for compressed key + return false; + } + } else { + // Non-canonical public key: neither compressed nor uncompressed + return false; + } + return true; +} + +/** + * A canonical signature exists of: <30> <02> <02> + * Where R and S are not negative (their first byte has its highest bit not set), and not + * excessively padded (do not start with a 0 byte, unless an otherwise negative number follows, + * in which case a single 0 byte is necessary and even required). + * + * See https://bitcointalk.org/index.php?topic=8392.msg127623#msg127623 + * + * This function is consensus-critical since BIP66. + */ +bool static IsValidSignatureEncoding(const std::vector &sig) { + // Format: 0x30 [total-length] 0x02 [R-length] [R] 0x02 [S-length] [S] [sighash] + // * total-length: 1-byte length descriptor of everything that follows, + // excluding the sighash byte. + // * R-length: 1-byte length descriptor of the R value that follows. + // * R: arbitrary-length big-endian encoded R value. It must use the shortest + // possible encoding for a positive integers (which means no null bytes at + // the start, except a single one when the next byte has its highest bit set). + // * S-length: 1-byte length descriptor of the S value that follows. + // * S: arbitrary-length big-endian encoded S value. The same rules apply. + // * sighash: 1-byte value indicating what data is hashed (not part of the DER + // signature) + + // Minimum and maximum size constraints. + if (sig.size() < 9) return false; + if (sig.size() > 73) return false; + + // A signature is of type 0x30 (compound). + if (sig[0] != 0x30) return false; + + // Make sure the length covers the entire signature. + if (sig[1] != sig.size() - 3) return false; + + // Extract the length of the R element. + unsigned int lenR = sig[3]; + + // Make sure the length of the S element is still inside the signature. + if (5 + lenR >= sig.size()) return false; + + // Extract the length of the S element. + unsigned int lenS = sig[5 + lenR]; + + // Verify that the length of the signature matches the sum of the length + // of the elements. + if ((size_t)(lenR + lenS + 7) != sig.size()) return false; + + // Check whether the R element is an integer. + if (sig[2] != 0x02) return false; + + // Zero-length integers are not allowed for R. + if (lenR == 0) return false; + + // Negative numbers are not allowed for R. + if (sig[4] & 0x80) return false; + + // Null bytes at the start of R are not allowed, unless R would + // otherwise be interpreted as a negative number. + if (lenR > 1 && (sig[4] == 0x00) && !(sig[5] & 0x80)) return false; + + // Check whether the S element is an integer. + if (sig[lenR + 4] != 0x02) return false; + + // Zero-length integers are not allowed for S. + if (lenS == 0) return false; + + // Negative numbers are not allowed for S. + if (sig[lenR + 6] & 0x80) return false; + + // Null bytes at the start of S are not allowed, unless S would otherwise be + // interpreted as a negative number. + if (lenS > 1 && (sig[lenR + 6] == 0x00) && !(sig[lenR + 7] & 0x80)) return false; + + return true; +} + +bool static IsLowDERSignature(const valtype &vchSig, ScriptError* serror) { + if (!IsValidSignatureEncoding(vchSig)) { + return set_error(serror, SCRIPT_ERR_SIG_DER); + } + std::vector vchSigCopy(vchSig.begin(), vchSig.begin() + vchSig.size() - 1); + return CPubKey::CheckLowS(vchSigCopy); + return true; +} + +bool static IsDefinedHashtypeSignature(const valtype &vchSig) { + if (vchSig.size() == 0) { + return false; + } + unsigned char nHashType = vchSig[vchSig.size() - 1] & (~(SIGHASH_ANYONECANPAY)); + if (nHashType < SIGHASH_ALL || nHashType > SIGHASH_SINGLE) + return false; + + return true; +} + +bool static CheckSignatureEncoding(const valtype &vchSig, unsigned int flags, ScriptError* serror) { + // Empty signature. Not strictly DER encoded, but allowed to provide a + // compact way to provide an invalid signature for use with CHECK(MULTI)SIG + if (vchSig.size() == 0) { + return true; + } + if ((flags & (SCRIPT_VERIFY_DERSIG | SCRIPT_VERIFY_LOW_S | SCRIPT_VERIFY_STRICTENC)) != 0 && !IsValidSignatureEncoding(vchSig)) { + return set_error(serror, SCRIPT_ERR_SIG_DER); + } else if ((flags & SCRIPT_VERIFY_LOW_S) != 0 && !IsLowDERSignature(vchSig, serror)) { + // serror is set + return false; + } else if ((flags & SCRIPT_VERIFY_STRICTENC) != 0 && !IsDefinedHashtypeSignature(vchSig)) { + return set_error(serror, SCRIPT_ERR_SIG_HASHTYPE); + } + return true; +} + +bool static CheckPubKeyEncoding(const valtype &vchSig, unsigned int flags, ScriptError* serror) { + if ((flags & SCRIPT_VERIFY_STRICTENC) != 0 && !IsCompressedOrUncompressedPubKey(vchSig)) { + return set_error(serror, SCRIPT_ERR_PUBKEYTYPE); + } + return true; +} + +bool static CheckMinimalPush(const valtype& data, opcodetype opcode) { + if (data.size() == 0) { + // Could have used OP_0. + return opcode == OP_0; + } else if (data.size() == 1 && data[0] >= 1 && data[0] <= 16) { + // Could have used OP_1 .. OP_16. + return opcode == OP_1 + (data[0] - 1); + } else if (data.size() == 1 && data[0] == 0x81) { + // Could have used OP_1NEGATE. + return opcode == OP_1NEGATE; + } else if (data.size() <= 75) { + // Could have used a direct push (opcode indicating number of bytes pushed + those bytes). +/* MCHN START */ +// return opcode == data.size(); + return (unsigned int)opcode == data.size(); +/* MCHN END */ + } else if (data.size() <= 255) { + // Could have used OP_PUSHDATA. + return opcode == OP_PUSHDATA1; + } else if (data.size() <= 65535) { + // Could have used OP_PUSHDATA2. + return opcode == OP_PUSHDATA2; + } + return true; +} + +bool EvalScript(vector >& stack, const CScript& script, unsigned int flags, const BaseSignatureChecker& checker, ScriptError* serror) +{ + static const CScriptNum bnZero(0); + static const CScriptNum bnOne(1); + static const CScriptNum bnFalse(0); + static const CScriptNum bnTrue(1); + static const valtype vchFalse(0); + static const valtype vchZero(0); + static const valtype vchTrue(1, 1); + + CScript::const_iterator pc = script.begin(); + CScript::const_iterator pend = script.end(); + CScript::const_iterator pbegincodehash = script.begin(); + opcodetype opcode; + valtype vchPushValue; + vector vfExec; + vector altstack; + set_error(serror, SCRIPT_ERR_UNKNOWN_ERROR); + if (script.size() > 10000) + return set_error(serror, SCRIPT_ERR_SCRIPT_SIZE); + int nOpCount = 0; + bool fRequireMinimal = (flags & SCRIPT_VERIFY_MINIMALDATA) != 0; + +/* MCHN START */ + bool fLongElement=false; +/* MCHN END */ + + try + { + while (pc < pend) + { + bool fExec = !count(vfExec.begin(), vfExec.end(), false); + + // + // Read instruction + // + if (!script.GetOp(pc, opcode, vchPushValue)) + return set_error(serror, SCRIPT_ERR_BAD_OPCODE); +/* MCHN START */ + if(fLongElement) + { + if(mc_gState->m_Features->FixedIn10007() == 0) + { + if(opcode != OP_DROP) + { + return set_error(serror, SCRIPT_ERR_PUSH_SIZE); + } + } + } + if(mc_gState->m_Features->Streams()) // It is now part of the IsStandard check + { + fLongElement=false; + } + else + { + if (vchPushValue.size() > MAX_SCRIPT_ELEMENT_SIZE) + { + if(mc_gState->m_Features->VerifySizeOfOpDropElements() == 0) + { + fLongElement=true; + } + else + { + return set_error(serror, SCRIPT_ERR_PUSH_SIZE); + } + } + else + { + fLongElement=false; + } + } +/* MCHN END */ + // Note how OP_RESERVED does not count towards the opcode limit. + if (opcode > OP_16 && ++nOpCount > 201) + return set_error(serror, SCRIPT_ERR_OP_COUNT); + + if (opcode == OP_CAT || + opcode == OP_SUBSTR || + opcode == OP_LEFT || + opcode == OP_RIGHT || + opcode == OP_INVERT || + opcode == OP_AND || + opcode == OP_OR || + opcode == OP_XOR || + opcode == OP_2MUL || + opcode == OP_2DIV || + opcode == OP_MUL || + opcode == OP_DIV || + opcode == OP_MOD || + opcode == OP_LSHIFT || + opcode == OP_RSHIFT) + return set_error(serror, SCRIPT_ERR_DISABLED_OPCODE); // Disabled opcodes. + + if (fExec && 0 <= opcode && opcode <= OP_PUSHDATA4) { + if (fRequireMinimal && !CheckMinimalPush(vchPushValue, opcode)) { + return set_error(serror, SCRIPT_ERR_MINIMALDATA); + } + stack.push_back(vchPushValue); + } else if (fExec || (OP_IF <= opcode && opcode <= OP_ENDIF)) + switch (opcode) + { + // + // Push value + // + case OP_1NEGATE: + case OP_1: + case OP_2: + case OP_3: + case OP_4: + case OP_5: + case OP_6: + case OP_7: + case OP_8: + case OP_9: + case OP_10: + case OP_11: + case OP_12: + case OP_13: + case OP_14: + case OP_15: + case OP_16: + { + // ( -- value) + CScriptNum bn((int)opcode - (int)(OP_1 - 1)); + stack.push_back(bn.getvch()); + // The result of these opcodes should always be the minimal way to push the data + // they push, so no need for a CheckMinimalPush here. + } + break; + + + // + // Control + // + case OP_NOP: + break; + + case OP_NOP1: case OP_NOP2: case OP_NOP3: case OP_NOP4: case OP_NOP5: + case OP_NOP6: case OP_NOP7: case OP_NOP8: case OP_NOP9: case OP_NOP10: + { + if (flags & SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_NOPS) + return set_error(serror, SCRIPT_ERR_DISCOURAGE_UPGRADABLE_NOPS); + } + break; + + case OP_IF: + case OP_NOTIF: + { + // if [statements] [else [statements]] endif + bool fValue = false; + if (fExec) + { + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_UNBALANCED_CONDITIONAL); + valtype& vch = stacktop(-1); + fValue = CastToBool(vch); + if (opcode == OP_NOTIF) + fValue = !fValue; + popstack(stack); + } + vfExec.push_back(fValue); + } + break; + + case OP_ELSE: + { + if (vfExec.empty()) + return set_error(serror, SCRIPT_ERR_UNBALANCED_CONDITIONAL); + vfExec.back() = !vfExec.back(); + } + break; + + case OP_ENDIF: + { + if (vfExec.empty()) + return set_error(serror, SCRIPT_ERR_UNBALANCED_CONDITIONAL); + vfExec.pop_back(); + } + break; + + case OP_VERIFY: + { + // (true -- ) or + // (false -- false) and return + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + bool fValue = CastToBool(stacktop(-1)); + if (fValue) + popstack(stack); + else + return set_error(serror, SCRIPT_ERR_VERIFY); + } + break; + + case OP_RETURN: + { + return set_error(serror, SCRIPT_ERR_OP_RETURN); + } + break; + + + // + // Stack ops + // + case OP_TOALTSTACK: + { + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + altstack.push_back(stacktop(-1)); + popstack(stack); + } + break; + + case OP_FROMALTSTACK: + { + if (altstack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_ALTSTACK_OPERATION); + stack.push_back(altstacktop(-1)); + popstack(altstack); + } + break; + + case OP_2DROP: + { + // (x1 x2 -- ) + if (stack.size() < 2) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + popstack(stack); + popstack(stack); + } + break; + + case OP_2DUP: + { + // (x1 x2 -- x1 x2 x1 x2) + if (stack.size() < 2) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + valtype vch1 = stacktop(-2); + valtype vch2 = stacktop(-1); + stack.push_back(vch1); + stack.push_back(vch2); + } + break; + + case OP_3DUP: + { + // (x1 x2 x3 -- x1 x2 x3 x1 x2 x3) + if (stack.size() < 3) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + valtype vch1 = stacktop(-3); + valtype vch2 = stacktop(-2); + valtype vch3 = stacktop(-1); + stack.push_back(vch1); + stack.push_back(vch2); + stack.push_back(vch3); + } + break; + + case OP_2OVER: + { + // (x1 x2 x3 x4 -- x1 x2 x3 x4 x1 x2) + if (stack.size() < 4) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + valtype vch1 = stacktop(-4); + valtype vch2 = stacktop(-3); + stack.push_back(vch1); + stack.push_back(vch2); + } + break; + + case OP_2ROT: + { + // (x1 x2 x3 x4 x5 x6 -- x3 x4 x5 x6 x1 x2) + if (stack.size() < 6) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + valtype vch1 = stacktop(-6); + valtype vch2 = stacktop(-5); + stack.erase(stack.end()-6, stack.end()-4); + stack.push_back(vch1); + stack.push_back(vch2); + } + break; + + case OP_2SWAP: + { + // (x1 x2 x3 x4 -- x3 x4 x1 x2) + if (stack.size() < 4) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + swap(stacktop(-4), stacktop(-2)); + swap(stacktop(-3), stacktop(-1)); + } + break; + + case OP_IFDUP: + { + // (x - 0 | x x) + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + valtype vch = stacktop(-1); + if (CastToBool(vch)) + stack.push_back(vch); + } + break; + + case OP_DEPTH: + { + // -- stacksize + CScriptNum bn(stack.size()); + stack.push_back(bn.getvch()); + } + break; + + case OP_DROP: + { + // (x -- ) + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + popstack(stack); + } + break; + + case OP_DUP: + { + // (x -- x x) + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + valtype vch = stacktop(-1); + stack.push_back(vch); + } + break; + + case OP_NIP: + { + // (x1 x2 -- x2) + if (stack.size() < 2) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + stack.erase(stack.end() - 2); + } + break; + + case OP_OVER: + { + // (x1 x2 -- x1 x2 x1) + if (stack.size() < 2) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + valtype vch = stacktop(-2); + stack.push_back(vch); + } + break; + + case OP_PICK: + case OP_ROLL: + { + // (xn ... x2 x1 x0 n - xn ... x2 x1 x0 xn) + // (xn ... x2 x1 x0 n - ... x2 x1 x0 xn) + if (stack.size() < 2) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + int n = CScriptNum(stacktop(-1), fRequireMinimal).getint(); + popstack(stack); + if (n < 0 || n >= (int)stack.size()) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + valtype vch = stacktop(-n-1); + if (opcode == OP_ROLL) + stack.erase(stack.end()-n-1); + stack.push_back(vch); + } + break; + + case OP_ROT: + { + // (x1 x2 x3 -- x2 x3 x1) + // x2 x1 x3 after first swap + // x2 x3 x1 after second swap + if (stack.size() < 3) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + swap(stacktop(-3), stacktop(-2)); + swap(stacktop(-2), stacktop(-1)); + } + break; + + case OP_SWAP: + { + // (x1 x2 -- x2 x1) + if (stack.size() < 2) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + swap(stacktop(-2), stacktop(-1)); + } + break; + + case OP_TUCK: + { + // (x1 x2 -- x2 x1 x2) + if (stack.size() < 2) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + valtype vch = stacktop(-1); + stack.insert(stack.end()-2, vch); + } + break; + + + case OP_SIZE: + { + // (in -- in size) + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + CScriptNum bn(stacktop(-1).size()); + stack.push_back(bn.getvch()); + } + break; + + + // + // Bitwise logic + // + case OP_EQUAL: + case OP_EQUALVERIFY: + //case OP_NOTEQUAL: // use OP_NUMNOTEQUAL + { + // (x1 x2 - bool) + if (stack.size() < 2) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + valtype& vch1 = stacktop(-2); + valtype& vch2 = stacktop(-1); + bool fEqual = (vch1 == vch2); + // OP_NOTEQUAL is disabled because it would be too easy to say + // something like n != 1 and have some wiseguy pass in 1 with extra + // zero bytes after it (numerically, 0x01 == 0x0001 == 0x000001) + //if (opcode == OP_NOTEQUAL) + // fEqual = !fEqual; + popstack(stack); + popstack(stack); + stack.push_back(fEqual ? vchTrue : vchFalse); + if (opcode == OP_EQUALVERIFY) + { + if (fEqual) + popstack(stack); + else + return set_error(serror, SCRIPT_ERR_EQUALVERIFY); + } + } + break; + + + // + // Numeric + // + case OP_1ADD: + case OP_1SUB: + case OP_NEGATE: + case OP_ABS: + case OP_NOT: + case OP_0NOTEQUAL: + { + // (in -- out) + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + CScriptNum bn(stacktop(-1), fRequireMinimal); + switch (opcode) + { + case OP_1ADD: bn += bnOne; break; + case OP_1SUB: bn -= bnOne; break; + case OP_NEGATE: bn = -bn; break; + case OP_ABS: if (bn < bnZero) bn = -bn; break; + case OP_NOT: bn = (bn == bnZero); break; + case OP_0NOTEQUAL: bn = (bn != bnZero); break; + default: assert(!"invalid opcode"); break; + } + popstack(stack); + stack.push_back(bn.getvch()); + } + break; + + case OP_ADD: + case OP_SUB: + case OP_BOOLAND: + case OP_BOOLOR: + case OP_NUMEQUAL: + case OP_NUMEQUALVERIFY: + case OP_NUMNOTEQUAL: + case OP_LESSTHAN: + case OP_GREATERTHAN: + case OP_LESSTHANOREQUAL: + case OP_GREATERTHANOREQUAL: + case OP_MIN: + case OP_MAX: + { + // (x1 x2 -- out) + if (stack.size() < 2) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + CScriptNum bn1(stacktop(-2), fRequireMinimal); + CScriptNum bn2(stacktop(-1), fRequireMinimal); + CScriptNum bn(0); + switch (opcode) + { + case OP_ADD: + bn = bn1 + bn2; + break; + + case OP_SUB: + bn = bn1 - bn2; + break; + + case OP_BOOLAND: bn = (bn1 != bnZero && bn2 != bnZero); break; + case OP_BOOLOR: bn = (bn1 != bnZero || bn2 != bnZero); break; + case OP_NUMEQUAL: bn = (bn1 == bn2); break; + case OP_NUMEQUALVERIFY: bn = (bn1 == bn2); break; + case OP_NUMNOTEQUAL: bn = (bn1 != bn2); break; + case OP_LESSTHAN: bn = (bn1 < bn2); break; + case OP_GREATERTHAN: bn = (bn1 > bn2); break; + case OP_LESSTHANOREQUAL: bn = (bn1 <= bn2); break; + case OP_GREATERTHANOREQUAL: bn = (bn1 >= bn2); break; + case OP_MIN: bn = (bn1 < bn2 ? bn1 : bn2); break; + case OP_MAX: bn = (bn1 > bn2 ? bn1 : bn2); break; + default: assert(!"invalid opcode"); break; + } + popstack(stack); + popstack(stack); + stack.push_back(bn.getvch()); + + if (opcode == OP_NUMEQUALVERIFY) + { + if (CastToBool(stacktop(-1))) + popstack(stack); + else + return set_error(serror, SCRIPT_ERR_NUMEQUALVERIFY); + } + } + break; + + case OP_WITHIN: + { + // (x min max -- out) + if (stack.size() < 3) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + CScriptNum bn1(stacktop(-3), fRequireMinimal); + CScriptNum bn2(stacktop(-2), fRequireMinimal); + CScriptNum bn3(stacktop(-1), fRequireMinimal); + bool fValue = (bn2 <= bn1 && bn1 < bn3); + popstack(stack); + popstack(stack); + popstack(stack); + stack.push_back(fValue ? vchTrue : vchFalse); + } + break; + + + // + // Crypto + // + case OP_RIPEMD160: + case OP_SHA1: + case OP_SHA256: + case OP_HASH160: + case OP_HASH256: + { + // (in -- hash) + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + valtype& vch = stacktop(-1); + valtype vchHash((opcode == OP_RIPEMD160 || opcode == OP_SHA1 || opcode == OP_HASH160) ? 20 : 32); + if (opcode == OP_RIPEMD160) + CRIPEMD160().Write(begin_ptr(vch), vch.size()).Finalize(begin_ptr(vchHash)); + else if (opcode == OP_SHA1) + CSHA1().Write(begin_ptr(vch), vch.size()).Finalize(begin_ptr(vchHash)); + else if (opcode == OP_SHA256) + CSHA256().Write(begin_ptr(vch), vch.size()).Finalize(begin_ptr(vchHash)); + else if (opcode == OP_HASH160) + CHash160().Write(begin_ptr(vch), vch.size()).Finalize(begin_ptr(vchHash)); + else if (opcode == OP_HASH256) + CHash256().Write(begin_ptr(vch), vch.size()).Finalize(begin_ptr(vchHash)); + popstack(stack); + stack.push_back(vchHash); + } + break; + + case OP_CODESEPARATOR: + { + // Hash starts after the code separator + pbegincodehash = pc; + } + break; + + case OP_CHECKSIG: + case OP_CHECKSIGVERIFY: + { + // (sig pubkey -- bool) + if (stack.size() < 2) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + + valtype& vchSig = stacktop(-2); + valtype& vchPubKey = stacktop(-1); + + // Subset of script starting at the most recent codeseparator + CScript scriptCode(pbegincodehash, pend); + + // Drop the signature, since there's no way for a signature to sign itself + scriptCode.FindAndDelete(CScript(vchSig)); + if (!CheckSignatureEncoding(vchSig, flags, serror) || !CheckPubKeyEncoding(vchPubKey, flags, serror)) { + //serror is set + return false; + } +/* MCHN START */ + bool cannot_send=true; +// bool fSuccess = checker.CheckSig(vchSig, vchPubKey, scriptCode); + bool fSuccess = checker.CheckSig(vchSig, vchPubKey, scriptCode, cannot_send); + fSuccess &= !cannot_send; +/* MCHN END */ + + popstack(stack); + popstack(stack); + stack.push_back(fSuccess ? vchTrue : vchFalse); + if (opcode == OP_CHECKSIGVERIFY) + { + if (fSuccess) + popstack(stack); + else + return set_error(serror, SCRIPT_ERR_CHECKSIGVERIFY); + } + } + break; + + case OP_CHECKMULTISIG: + case OP_CHECKMULTISIGVERIFY: + { + // ([sig ...] num_of_signatures [pubkey ...] num_of_pubkeys -- bool) + int i = 1; + if ((int)stack.size() < i) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + + int nKeysCount = CScriptNum(stacktop(-i), fRequireMinimal).getint(); + if (nKeysCount < 0 || nKeysCount > 20) + return set_error(serror, SCRIPT_ERR_PUBKEY_COUNT); + nOpCount += nKeysCount; + if (nOpCount > 201) + return set_error(serror, SCRIPT_ERR_OP_COUNT); + int ikey = ++i; + i += nKeysCount; + if ((int)stack.size() < i) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + + int nSigsCount = CScriptNum(stacktop(-i), fRequireMinimal).getint(); + if (nSigsCount < 0 || nSigsCount > nKeysCount) + return set_error(serror, SCRIPT_ERR_SIG_COUNT); + int isig = ++i; + i += nSigsCount; + if ((int)stack.size() < i) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + + // Subset of script starting at the most recent codeseparator + CScript scriptCode(pbegincodehash, pend); + + // Drop the signatures, since there's no way for a signature to sign itself + for (int k = 0; k < nSigsCount; k++) + { + valtype& vchSig = stacktop(-isig-k); + scriptCode.FindAndDelete(CScript(vchSig)); + } + +/* MCHN START */ + bool cannot_send=true; + if(flags & SCRIPT_VERIFY_SKIP_SEND_PERMISSION_CHECK) + { + cannot_send=false; + } +/* MCHN END */ + bool fSuccess = true; + while (fSuccess && nSigsCount > 0) + { + valtype& vchSig = stacktop(-isig); + valtype& vchPubKey = stacktop(-ikey); + + // Note how this makes the exact order of pubkey/signature evaluation + // distinguishable by CHECKMULTISIG NOT if the STRICTENC flag is set. + // See the script_(in)valid tests for details. + if (!CheckSignatureEncoding(vchSig, flags, serror) || !CheckPubKeyEncoding(vchPubKey, flags, serror)) { + // serror is set + return false; + } + + // Check signature +/* MCHN START */ +// bool fOk = checker.CheckSig(vchSig, vchPubKey, scriptCode); + bool fOk = checker.CheckSig(vchSig, vchPubKey, scriptCode, cannot_send); +/* MCHN END */ + + if (fOk) { + isig++; + nSigsCount--; + } + ikey++; + nKeysCount--; + + // If there are more signatures left than keys left, + // then too many signatures have failed. Exit early, + // without checking any further signatures. + if (nSigsCount > nKeysCount) + fSuccess = false; + } +/* MCHN START */ + fSuccess &= !cannot_send; +/* MCHN END */ + + // Clean up stack of actual arguments + while (i-- > 1) + popstack(stack); + + // A bug causes CHECKMULTISIG to consume one extra argument + // whose contents were not checked in any way. + // + // Unfortunately this is a potential source of mutability, + // so optionally verify it is exactly equal to zero prior + // to removing it from the stack. + if (stack.size() < 1) + return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION); + if ((flags & SCRIPT_VERIFY_NULLDUMMY) && stacktop(-1).size()) + return set_error(serror, SCRIPT_ERR_SIG_NULLDUMMY); + popstack(stack); + + stack.push_back(fSuccess ? vchTrue : vchFalse); + + if (opcode == OP_CHECKMULTISIGVERIFY) + { + if (fSuccess) + popstack(stack); + else + return set_error(serror, SCRIPT_ERR_CHECKMULTISIGVERIFY); + } + } + break; + + default: + return set_error(serror, SCRIPT_ERR_BAD_OPCODE); + } + + // Size limits + if (stack.size() + altstack.size() > 1000) + return set_error(serror, SCRIPT_ERR_STACK_SIZE); + } + } + catch (...) + { + return set_error(serror, SCRIPT_ERR_UNKNOWN_ERROR); + } + + if (!vfExec.empty()) + return set_error(serror, SCRIPT_ERR_UNBALANCED_CONDITIONAL); + + return set_success(serror); +} + +namespace { + +/** + * Wrapper that serializes like CTransaction, but with the modifications + * required for the signature hash done in-place + */ +class CTransactionSignatureSerializer { +private: + const CTransaction &txTo; //! reference to the spending transaction (the one being serialized) + const CScript &scriptCode; //! output script being consumed + const unsigned int nIn; //! input index of txTo being signed + const bool fAnyoneCanPay; //! whether the hashtype has the SIGHASH_ANYONECANPAY flag set + const bool fHashSingle; //! whether the hashtype is SIGHASH_SINGLE + const bool fHashNone; //! whether the hashtype is SIGHASH_NONE + +public: + CTransactionSignatureSerializer(const CTransaction &txToIn, const CScript &scriptCodeIn, unsigned int nInIn, int nHashTypeIn) : + txTo(txToIn), scriptCode(scriptCodeIn), nIn(nInIn), + fAnyoneCanPay(!!(nHashTypeIn & SIGHASH_ANYONECANPAY)), + fHashSingle((nHashTypeIn & 0x1f) == SIGHASH_SINGLE), + fHashNone((nHashTypeIn & 0x1f) == SIGHASH_NONE) {} + + /** Serialize the passed scriptCode, skipping OP_CODESEPARATORs */ + template + void SerializeScriptCode(S &s, int nType, int nVersion) const { + CScript::const_iterator it = scriptCode.begin(); + CScript::const_iterator itBegin = it; + opcodetype opcode; + unsigned int nCodeSeparators = 0; + while (scriptCode.GetOp(it, opcode)) { + if (opcode == OP_CODESEPARATOR) + nCodeSeparators++; + } + ::WriteCompactSize(s, scriptCode.size() - nCodeSeparators); + it = itBegin; + while (scriptCode.GetOp(it, opcode)) { + if (opcode == OP_CODESEPARATOR) { + s.write((char*)&itBegin[0], it-itBegin-1); + itBegin = it; + } + } + if (itBegin != scriptCode.end()) + s.write((char*)&itBegin[0], it-itBegin); + } + + /** Serialize an input of txTo */ + template + void SerializeInput(S &s, unsigned int nInput, int nType, int nVersion) const { + // In case of SIGHASH_ANYONECANPAY, only the input being signed is serialized + if (fAnyoneCanPay) + nInput = nIn; + // Serialize the prevout + ::Serialize(s, txTo.vin[nInput].prevout, nType, nVersion); + // Serialize the script + if (nInput != nIn) + // Blank out other inputs' signatures + ::Serialize(s, CScript(), nType, nVersion); + else + SerializeScriptCode(s, nType, nVersion); + // Serialize the nSequence + if (nInput != nIn && (fHashSingle || fHashNone)) + // let the others update at will + ::Serialize(s, (int)0, nType, nVersion); + else + ::Serialize(s, txTo.vin[nInput].nSequence, nType, nVersion); + } + + /** Serialize an output of txTo */ + template + void SerializeOutput(S &s, unsigned int nOutput, int nType, int nVersion) const { + if (fHashSingle && nOutput != nIn) + // Do not lock-in the txout payee at other indices as txin + ::Serialize(s, CTxOut(), nType, nVersion); + else + ::Serialize(s, txTo.vout[nOutput], nType, nVersion); + } + + /** Serialize txTo */ + template + void Serialize(S &s, int nType, int nVersion) const { + // Serialize nVersion + ::Serialize(s, txTo.nVersion, nType, nVersion); + // Serialize vin + unsigned int nInputs = fAnyoneCanPay ? 1 : txTo.vin.size(); + ::WriteCompactSize(s, nInputs); + for (unsigned int nInput = 0; nInput < nInputs; nInput++) + SerializeInput(s, nInput, nType, nVersion); + // Serialize vout + unsigned int nOutputs = fHashNone ? 0 : (fHashSingle ? nIn+1 : txTo.vout.size()); + ::WriteCompactSize(s, nOutputs); + for (unsigned int nOutput = 0; nOutput < nOutputs; nOutput++) + SerializeOutput(s, nOutput, nType, nVersion); + // Serialize nLockTime + ::Serialize(s, txTo.nLockTime, nType, nVersion); + } +}; + +} // anon namespace + +uint256 SignatureHash(const CScript& scriptCode, const CTransaction& txTo, unsigned int nIn, int nHashType) +{ + if (nIn >= txTo.vin.size()) { + // nIn out of range + return 1; + } + + // Check for invalid use of SIGHASH_SINGLE + if ((nHashType & 0x1f) == SIGHASH_SINGLE) { + if (nIn >= txTo.vout.size()) { + // nOut out of range + return 1; + } + } + + // Wrapper to serialize only the necessary parts of the transaction being signed + CTransactionSignatureSerializer txTmp(txTo, scriptCode, nIn, nHashType); + + // Serialize and hash + CHashWriter ss(SER_GETHASH, 0); + ss << txTmp << nHashType; + return ss.GetHash(); +} + +bool TransactionSignatureChecker::VerifySignature(const std::vector& vchSig, const CPubKey& pubkey, const uint256& sighash) const +{ + return pubkey.Verify(sighash, vchSig); +} + +/* MCHN START */ +//bool TransactionSignatureChecker::CheckSig(const vector& vchSigIn, const vector& vchPubKey, const CScript& scriptCode) const +bool TransactionSignatureChecker::CheckSig(const vector& vchSigIn, const vector& vchPubKey, const CScript& scriptCode, bool& CheckSendPermission) const +/* MCHN END */ + +{ + CPubKey pubkey(vchPubKey); + if (!pubkey.IsValid()) + return false; + // Hash type is one byte tacked on to the end of the signature + vector vchSig(vchSigIn); + if (vchSig.empty()) + return false; + int nHashType = vchSig.back(); + vchSig.pop_back(); + + uint256 sighash = SignatureHash(scriptCode, *txTo, nIn, nHashType); + + if (!VerifySignature(vchSig, pubkey, sighash)) + return false; + +/* MCHN START */ + if(CheckSendPermission) + { + + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + const unsigned char *pubkey_hash=(unsigned char *)Hash160(vchPubKey.begin(),vchPubKey.end()).begin(); + + if(mc_gState->m_Permissions->CanSend(NULL,pubkey_hash)) + { + CheckSendPermission=false; + } + } + else + { + CheckSendPermission=false; + } + } +/* MCHN END */ + return true; +} + +bool VerifyScript(const CScript& scriptSig, const CScript& scriptPubKey, unsigned int flags, const BaseSignatureChecker& checker, ScriptError* serror) +{ + set_error(serror, SCRIPT_ERR_UNKNOWN_ERROR); + if ((flags & SCRIPT_VERIFY_SIGPUSHONLY) != 0 && !scriptSig.IsPushOnly()) { + return set_error(serror, SCRIPT_ERR_SIG_PUSHONLY); + } + + vector > stack, stackCopy; + if (!EvalScript(stack, scriptSig, flags, checker, serror)) + // serror is set + return false; + if (flags & SCRIPT_VERIFY_P2SH) + stackCopy = stack; + if (!EvalScript(stack, scriptPubKey, flags, checker, serror)) + // serror is set + return false; + if (stack.empty()) + return set_error(serror, SCRIPT_ERR_EVAL_FALSE); + + if (CastToBool(stack.back()) == false) + return set_error(serror, SCRIPT_ERR_CHECKSIGVERIFY); // MCHN +// return set_error(serror, SCRIPT_ERR_EVAL_FALSE); +/* MCHN START */ + + if(mc_gState->m_Features->Streams()) + { + if(!scriptSig.HasSmallIntegerInTheBeginning()) + { + if(stack.size() != 1) + { + return set_error(serror, SCRIPT_ERR_EVAL_FALSE); + } + } + } +/* MCHN END */ + // Additional validation for spend-to-script-hash transactions: + if ((flags & SCRIPT_VERIFY_P2SH) && scriptPubKey.IsPayToScriptHash()) + { + // scriptSig must be literals-only or validation fails + if (!scriptSig.IsPushOnly()) + return set_error(serror, SCRIPT_ERR_SIG_PUSHONLY); + +/* MCHN START */ + unsigned int p2shflags=flags; + CTxDestination addressRet; + + if(mc_gState->m_Features->Streams()) + { + if(!scriptSig.HasSmallIntegerInTheBeginning()) + { + return set_error(serror, SCRIPT_ERR_EVAL_FALSE); + } + if(!ExtractDestination(scriptPubKey, addressRet)) + { + return set_error(serror, SCRIPT_ERR_VERIFY); + } + + CScriptID *lpScriptID=boost::get (&addressRet); + if(lpScriptID) + { + if(mc_gState->m_Permissions->CanSend(NULL,(unsigned char*)(lpScriptID))) + { + p2shflags |= SCRIPT_VERIFY_SKIP_SEND_PERMISSION_CHECK; + } + } + else + { + return set_error(serror, SCRIPT_ERR_VERIFY); + } + } + +/* MCHN END */ + + // stackCopy cannot be empty here, because if it was the + // P2SH HASH <> EQUAL scriptPubKey would be evaluated with + // an empty stack and the EvalScript above would return false. + assert(!stackCopy.empty()); + + const valtype& pubKeySerialized = stackCopy.back(); + CScript pubKey2(pubKeySerialized.begin(), pubKeySerialized.end()); + popstack(stackCopy); + + if (!EvalScript(stackCopy, pubKey2, p2shflags, checker, serror)) + // serror is set + return false; + if (stackCopy.empty()) + return set_error(serror, SCRIPT_ERR_EVAL_FALSE); + if (!CastToBool(stackCopy.back())) + return set_error(serror, SCRIPT_ERR_EVAL_FALSE); + else + return set_success(serror); + } + + return set_success(serror); +} diff --git a/src/script/interpreter.h b/src/script/interpreter.h new file mode 100644 index 00000000..0406c234 --- /dev/null +++ b/src/script/interpreter.h @@ -0,0 +1,124 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_SCRIPT_INTERPRETER_H +#define BITCOIN_SCRIPT_INTERPRETER_H + +#include "script_error.h" +#include "primitives/transaction.h" + +#include +#include +#include + +class CPubKey; +class CScript; +class CTransaction; +class uint256; + +/** Signature hash types/flags */ +enum +{ + SIGHASH_ALL = 1, + SIGHASH_NONE = 2, + SIGHASH_SINGLE = 3, + SIGHASH_ANYONECANPAY = 0x80, +}; + +/** Script verification flags */ +enum +{ + SCRIPT_VERIFY_NONE = 0, + + // Evaluate P2SH subscripts (softfork safe, BIP16). + SCRIPT_VERIFY_P2SH = (1U << 0), + + // Passing a non-strict-DER signature or one with undefined hashtype to a checksig operation causes script failure. + // Evaluating a pubkey that is not (0x04 + 64 bytes) or (0x02 or 0x03 + 32 bytes) by checksig causes script failure. + // (softfork safe, but not used or intended as a consensus rule). + SCRIPT_VERIFY_STRICTENC = (1U << 1), + + // Passing a non-strict-DER signature to a checksig operation causes script failure (softfork safe, BIP62 rule 1) + SCRIPT_VERIFY_DERSIG = (1U << 2), + + // Passing a non-strict-DER signature or one with S > order/2 to a checksig operation causes script failure + // (softfork safe, BIP62 rule 5). + SCRIPT_VERIFY_LOW_S = (1U << 3), + + // verify dummy stack item consumed by CHECKMULTISIG is of zero-length (softfork safe, BIP62 rule 7). + SCRIPT_VERIFY_NULLDUMMY = (1U << 4), + + // Using a non-push operator in the scriptSig causes script failure (softfork safe, BIP62 rule 2). + SCRIPT_VERIFY_SIGPUSHONLY = (1U << 5), + + // Require minimal encodings for all push operations (OP_0... OP_16, OP_1NEGATE where possible, direct + // pushes up to 75 bytes, OP_PUSHDATA up to 255 bytes, OP_PUSHDATA2 for anything larger). Evaluating + // any other push causes the script to fail (BIP62 rule 3). + // In addition, whenever a stack element is interpreted as a number, it must be of minimal length (BIP62 rule 4). + // (softfork safe) + SCRIPT_VERIFY_MINIMALDATA = (1U << 6), + + // Discourage use of NOPs reserved for upgrades (NOP1-10) + // + // Provided so that nodes can avoid accepting or mining transactions + // containing executed NOP's whose meaning may change after a soft-fork, + // thus rendering the script invalid; with this flag set executing + // discouraged NOPs fails the script. This verification flag will never be + // a mandatory flag applied to scripts in a block. NOPs that are not + // executed, e.g. within an unexecuted IF ENDIF block, are *not* rejected. + SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_NOPS = (1U << 7), +/* MCHN START */ + // Skips verifying of the send permission + SCRIPT_VERIFY_SKIP_SEND_PERMISSION_CHECK = (1U << 24) +/* MCHN END */ +}; + +uint256 SignatureHash(const CScript &scriptCode, const CTransaction& txTo, unsigned int nIn, int nHashType); + +class BaseSignatureChecker +{ +public: +/* MCHN START */ +// virtual bool CheckSig(const std::vector& scriptSig, const std::vector& vchPubKey, const CScript& scriptCode) const + virtual bool CheckSig(const std::vector& scriptSig, const std::vector& vchPubKey, const CScript& scriptCode, bool& CheckSendPermission) const +/* MCHN END */ + { + return false; + } + + virtual ~BaseSignatureChecker() {} +}; + +class TransactionSignatureChecker : public BaseSignatureChecker +{ +private: + const CTransaction* txTo; + unsigned int nIn; + +protected: + virtual bool VerifySignature(const std::vector& vchSig, const CPubKey& vchPubKey, const uint256& sighash) const; + +public: + TransactionSignatureChecker(const CTransaction* txToIn, unsigned int nInIn) : txTo(txToIn), nIn(nInIn) {} +/* MCHN START */ +// bool CheckSig(const std::vector& scriptSig, const std::vector& vchPubKey, const CScript& scriptCode) const; + bool CheckSig(const std::vector& scriptSig, const std::vector& vchPubKey, const CScript& scriptCode, bool& CheckSendPermission) const; +/* MCHN END */ +}; + +class MutableTransactionSignatureChecker : public TransactionSignatureChecker +{ +private: + const CTransaction txTo; + +public: + MutableTransactionSignatureChecker(const CMutableTransaction* txToIn, unsigned int nInIn) : TransactionSignatureChecker(&txTo, nInIn), txTo(*txToIn) {} +}; + +bool EvalScript(std::vector >& stack, const CScript& script, unsigned int flags, const BaseSignatureChecker& checker, ScriptError* error = NULL); +bool VerifyScript(const CScript& scriptSig, const CScript& scriptPubKey, unsigned int flags, const BaseSignatureChecker& checker, ScriptError* error = NULL); + +#endif // BITCOIN_SCRIPT_INTERPRETER_H diff --git a/src/script/script.cpp b/src/script/script.cpp new file mode 100644 index 00000000..863f1387 --- /dev/null +++ b/src/script/script.cpp @@ -0,0 +1,348 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "script.h" + +#include "utils/tinyformat.h" +#include "utils/utilstrencodings.h" + +namespace { +inline std::string ValueString(const std::vector& vch) +{ + if (vch.size() <= 4) + return strprintf("%d", CScriptNum(vch, false).getint()); + else + return HexStr(vch); +} +} // anon namespace + +using namespace std; + +const char* GetOpName(opcodetype opcode) +{ + switch (opcode) + { + // push value + case OP_0 : return "0"; + case OP_PUSHDATA1 : return "OP_PUSHDATA1"; + case OP_PUSHDATA2 : return "OP_PUSHDATA2"; + case OP_PUSHDATA4 : return "OP_PUSHDATA4"; + case OP_1NEGATE : return "-1"; + case OP_RESERVED : return "OP_RESERVED"; + case OP_1 : return "1"; + case OP_2 : return "2"; + case OP_3 : return "3"; + case OP_4 : return "4"; + case OP_5 : return "5"; + case OP_6 : return "6"; + case OP_7 : return "7"; + case OP_8 : return "8"; + case OP_9 : return "9"; + case OP_10 : return "10"; + case OP_11 : return "11"; + case OP_12 : return "12"; + case OP_13 : return "13"; + case OP_14 : return "14"; + case OP_15 : return "15"; + case OP_16 : return "16"; + + // control + case OP_NOP : return "OP_NOP"; + case OP_VER : return "OP_VER"; + case OP_IF : return "OP_IF"; + case OP_NOTIF : return "OP_NOTIF"; + case OP_VERIF : return "OP_VERIF"; + case OP_VERNOTIF : return "OP_VERNOTIF"; + case OP_ELSE : return "OP_ELSE"; + case OP_ENDIF : return "OP_ENDIF"; + case OP_VERIFY : return "OP_VERIFY"; + case OP_RETURN : return "OP_RETURN"; + + // stack ops + case OP_TOALTSTACK : return "OP_TOALTSTACK"; + case OP_FROMALTSTACK : return "OP_FROMALTSTACK"; + case OP_2DROP : return "OP_2DROP"; + case OP_2DUP : return "OP_2DUP"; + case OP_3DUP : return "OP_3DUP"; + case OP_2OVER : return "OP_2OVER"; + case OP_2ROT : return "OP_2ROT"; + case OP_2SWAP : return "OP_2SWAP"; + case OP_IFDUP : return "OP_IFDUP"; + case OP_DEPTH : return "OP_DEPTH"; + case OP_DROP : return "OP_DROP"; + case OP_DUP : return "OP_DUP"; + case OP_NIP : return "OP_NIP"; + case OP_OVER : return "OP_OVER"; + case OP_PICK : return "OP_PICK"; + case OP_ROLL : return "OP_ROLL"; + case OP_ROT : return "OP_ROT"; + case OP_SWAP : return "OP_SWAP"; + case OP_TUCK : return "OP_TUCK"; + + // splice ops + case OP_CAT : return "OP_CAT"; + case OP_SUBSTR : return "OP_SUBSTR"; + case OP_LEFT : return "OP_LEFT"; + case OP_RIGHT : return "OP_RIGHT"; + case OP_SIZE : return "OP_SIZE"; + + // bit logic + case OP_INVERT : return "OP_INVERT"; + case OP_AND : return "OP_AND"; + case OP_OR : return "OP_OR"; + case OP_XOR : return "OP_XOR"; + case OP_EQUAL : return "OP_EQUAL"; + case OP_EQUALVERIFY : return "OP_EQUALVERIFY"; + case OP_RESERVED1 : return "OP_RESERVED1"; + case OP_RESERVED2 : return "OP_RESERVED2"; + + // numeric + case OP_1ADD : return "OP_1ADD"; + case OP_1SUB : return "OP_1SUB"; + case OP_2MUL : return "OP_2MUL"; + case OP_2DIV : return "OP_2DIV"; + case OP_NEGATE : return "OP_NEGATE"; + case OP_ABS : return "OP_ABS"; + case OP_NOT : return "OP_NOT"; + case OP_0NOTEQUAL : return "OP_0NOTEQUAL"; + case OP_ADD : return "OP_ADD"; + case OP_SUB : return "OP_SUB"; + case OP_MUL : return "OP_MUL"; + case OP_DIV : return "OP_DIV"; + case OP_MOD : return "OP_MOD"; + case OP_LSHIFT : return "OP_LSHIFT"; + case OP_RSHIFT : return "OP_RSHIFT"; + case OP_BOOLAND : return "OP_BOOLAND"; + case OP_BOOLOR : return "OP_BOOLOR"; + case OP_NUMEQUAL : return "OP_NUMEQUAL"; + case OP_NUMEQUALVERIFY : return "OP_NUMEQUALVERIFY"; + case OP_NUMNOTEQUAL : return "OP_NUMNOTEQUAL"; + case OP_LESSTHAN : return "OP_LESSTHAN"; + case OP_GREATERTHAN : return "OP_GREATERTHAN"; + case OP_LESSTHANOREQUAL : return "OP_LESSTHANOREQUAL"; + case OP_GREATERTHANOREQUAL : return "OP_GREATERTHANOREQUAL"; + case OP_MIN : return "OP_MIN"; + case OP_MAX : return "OP_MAX"; + case OP_WITHIN : return "OP_WITHIN"; + + // crypto + case OP_RIPEMD160 : return "OP_RIPEMD160"; + case OP_SHA1 : return "OP_SHA1"; + case OP_SHA256 : return "OP_SHA256"; + case OP_HASH160 : return "OP_HASH160"; + case OP_HASH256 : return "OP_HASH256"; + case OP_CODESEPARATOR : return "OP_CODESEPARATOR"; + case OP_CHECKSIG : return "OP_CHECKSIG"; + case OP_CHECKSIGVERIFY : return "OP_CHECKSIGVERIFY"; + case OP_CHECKMULTISIG : return "OP_CHECKMULTISIG"; + case OP_CHECKMULTISIGVERIFY : return "OP_CHECKMULTISIGVERIFY"; + + // expanson + case OP_NOP1 : return "OP_NOP1"; + case OP_NOP2 : return "OP_NOP2"; + case OP_NOP3 : return "OP_NOP3"; + case OP_NOP4 : return "OP_NOP4"; + case OP_NOP5 : return "OP_NOP5"; + case OP_NOP6 : return "OP_NOP6"; + case OP_NOP7 : return "OP_NOP7"; + case OP_NOP8 : return "OP_NOP8"; + case OP_NOP9 : return "OP_NOP9"; + case OP_NOP10 : return "OP_NOP10"; + + case OP_INVALIDOPCODE : return "OP_INVALIDOPCODE"; + + // Note: + // The template matching params OP_SMALLDATA/etc are defined in opcodetype enum + // as kind of implementation hack, they are *NOT* real opcodes. If found in real + // Script, just let the default: case deal with them. + + default: + return "OP_UNKNOWN"; + } +} + +unsigned int CScript::GetSigOpCount(bool fAccurate) const +{ + unsigned int n = 0; + const_iterator pc = begin(); + opcodetype lastOpcode = OP_INVALIDOPCODE; + while (pc < end()) + { + opcodetype opcode; + if (!GetOp(pc, opcode)) + break; + if (opcode == OP_CHECKSIG || opcode == OP_CHECKSIGVERIFY) + n++; + else if (opcode == OP_CHECKMULTISIG || opcode == OP_CHECKMULTISIGVERIFY) + { + if (fAccurate && lastOpcode >= OP_1 && lastOpcode <= OP_16) + n += DecodeOP_N(lastOpcode); + else + n += 20; + } + lastOpcode = opcode; + } + return n; +} + +unsigned int CScript::GetSigOpCount(const CScript& scriptSig) const +{ + if (!IsPayToScriptHash()) + return GetSigOpCount(true); + + // This is a pay-to-script-hash scriptPubKey; + // get the last item that the scriptSig + // pushes onto the stack: + const_iterator pc = scriptSig.begin(); + vector data; + while (pc < scriptSig.end()) + { + opcodetype opcode; + if (!scriptSig.GetOp(pc, opcode, data)) + return 0; + if (opcode > OP_16) + return 0; + } + + /// ... and return its opcount: + CScript subscript(data.begin(), data.end()); + return subscript.GetSigOpCount(true); +} + +bool CScript::IsPayToScriptHash() const +{ + // Extra-fast test for pay-to-script-hash CScripts: +/* MCHN START*/ +// return (this->size() == 23 && + return (this->size() >= 23 && +/* MCHN END*/ + this->at(0) == OP_HASH160 && + this->at(1) == 0x14 && + this->at(22) == OP_EQUAL); +} + +bool CScript::IsPushOnly() const +{ + const_iterator pc = begin(); + while (pc < end()) + { + opcodetype opcode; + if (!GetOp(pc, opcode)) + return false; + // Note that IsPushOnly() *does* consider OP_RESERVED to be a + // push-type opcode, however execution of OP_RESERVED fails, so + // it's not relevant to P2SH/BIP62 as the scriptSig would fail prior to + // the P2SH special validation code being executed. + if (opcode > OP_16) + return false; + } + return true; +} + +bool CScript::HasSmallIntegerInTheBeginning() const +{ + const_iterator pc = begin(); + unsigned char opcode; + if(pc < end()) + { + opcode=*(unsigned char*)&begin()[0]; + if ( (opcode == OP_0) || (opcode >= OP_1 && opcode <= OP_16) ) + { + return true; + } + } + return false; +} + +std::string CScript::ToString() const +{ + std::string str; + opcodetype opcode; + std::vector vch; + const_iterator pc = begin(); + while (pc < end()) + { + if (!str.empty()) + str += " "; + if (!GetOp(pc, opcode, vch)) + { + str += "[error]"; + return str; + } + if (0 <= opcode && opcode <= OP_PUSHDATA4) + str += ValueString(vch); + else + str += GetOpName(opcode); + } + return str; +} + +/* MCHN START */ + +CScript CScript::RemoveOpDrops() const +{ + CScript scriptOutput; + + std::string str; + opcodetype opcode; + std::vector vch; + int opDropPos=-1; + int count=0; + + const_iterator pc = begin(); + while (pc < end()) + { + if (GetOp(pc, opcode, vch)) + { + if(opcode == OP_RETURN) + { + return *this; + } + } + } + + pc = begin(); + while ((pc < end()) && (opDropPos < 0)) + { + if (GetOp(pc, opcode, vch)) + { + if(opcode == OP_DROP) + { + opDropPos=count; + } + } + else + { + return *this; + } + count++; + } + + if(opDropPos < 0) + { + return *this; + } + + pc = begin(); + count=0; + while (count +#include +#include +#include +#include +#include +#include +#include + +/* MCHN START */ +#include "chainparams/state.h" +/* MCHN END */ + +extern unsigned int MAX_SCRIPT_ELEMENT_SIZE; // MCHN global +//static const unsigned int MAX_SCRIPT_ELEMENT_SIZE = 520; // bytes + +template +std::vector ToByteVector(const T& in) +{ + return std::vector(in.begin(), in.end()); +} + +/** Script opcodes */ +enum opcodetype +{ + // push value + OP_0 = 0x00, + OP_FALSE = OP_0, + OP_PUSHDATA1 = 0x4c, + OP_PUSHDATA2 = 0x4d, + OP_PUSHDATA4 = 0x4e, + OP_1NEGATE = 0x4f, + OP_RESERVED = 0x50, + OP_1 = 0x51, + OP_TRUE=OP_1, + OP_2 = 0x52, + OP_3 = 0x53, + OP_4 = 0x54, + OP_5 = 0x55, + OP_6 = 0x56, + OP_7 = 0x57, + OP_8 = 0x58, + OP_9 = 0x59, + OP_10 = 0x5a, + OP_11 = 0x5b, + OP_12 = 0x5c, + OP_13 = 0x5d, + OP_14 = 0x5e, + OP_15 = 0x5f, + OP_16 = 0x60, + + // control + OP_NOP = 0x61, + OP_VER = 0x62, + OP_IF = 0x63, + OP_NOTIF = 0x64, + OP_VERIF = 0x65, + OP_VERNOTIF = 0x66, + OP_ELSE = 0x67, + OP_ENDIF = 0x68, + OP_VERIFY = 0x69, + OP_RETURN = 0x6a, + + // stack ops + OP_TOALTSTACK = 0x6b, + OP_FROMALTSTACK = 0x6c, + OP_2DROP = 0x6d, + OP_2DUP = 0x6e, + OP_3DUP = 0x6f, + OP_2OVER = 0x70, + OP_2ROT = 0x71, + OP_2SWAP = 0x72, + OP_IFDUP = 0x73, + OP_DEPTH = 0x74, + OP_DROP = 0x75, + OP_DUP = 0x76, + OP_NIP = 0x77, + OP_OVER = 0x78, + OP_PICK = 0x79, + OP_ROLL = 0x7a, + OP_ROT = 0x7b, + OP_SWAP = 0x7c, + OP_TUCK = 0x7d, + + // splice ops + OP_CAT = 0x7e, + OP_SUBSTR = 0x7f, + OP_LEFT = 0x80, + OP_RIGHT = 0x81, + OP_SIZE = 0x82, + + // bit logic + OP_INVERT = 0x83, + OP_AND = 0x84, + OP_OR = 0x85, + OP_XOR = 0x86, + OP_EQUAL = 0x87, + OP_EQUALVERIFY = 0x88, + OP_RESERVED1 = 0x89, + OP_RESERVED2 = 0x8a, + + // numeric + OP_1ADD = 0x8b, + OP_1SUB = 0x8c, + OP_2MUL = 0x8d, + OP_2DIV = 0x8e, + OP_NEGATE = 0x8f, + OP_ABS = 0x90, + OP_NOT = 0x91, + OP_0NOTEQUAL = 0x92, + + OP_ADD = 0x93, + OP_SUB = 0x94, + OP_MUL = 0x95, + OP_DIV = 0x96, + OP_MOD = 0x97, + OP_LSHIFT = 0x98, + OP_RSHIFT = 0x99, + + OP_BOOLAND = 0x9a, + OP_BOOLOR = 0x9b, + OP_NUMEQUAL = 0x9c, + OP_NUMEQUALVERIFY = 0x9d, + OP_NUMNOTEQUAL = 0x9e, + OP_LESSTHAN = 0x9f, + OP_GREATERTHAN = 0xa0, + OP_LESSTHANOREQUAL = 0xa1, + OP_GREATERTHANOREQUAL = 0xa2, + OP_MIN = 0xa3, + OP_MAX = 0xa4, + + OP_WITHIN = 0xa5, + + // crypto + OP_RIPEMD160 = 0xa6, + OP_SHA1 = 0xa7, + OP_SHA256 = 0xa8, + OP_HASH160 = 0xa9, + OP_HASH256 = 0xaa, + OP_CODESEPARATOR = 0xab, + OP_CHECKSIG = 0xac, + OP_CHECKSIGVERIFY = 0xad, + OP_CHECKMULTISIG = 0xae, + OP_CHECKMULTISIGVERIFY = 0xaf, + + // expansion + OP_NOP1 = 0xb0, + OP_NOP2 = 0xb1, + OP_NOP3 = 0xb2, + OP_NOP4 = 0xb3, + OP_NOP5 = 0xb4, + OP_NOP6 = 0xb5, + OP_NOP7 = 0xb6, + OP_NOP8 = 0xb7, + OP_NOP9 = 0xb8, + OP_NOP10 = 0xb9, + + + // template matching params +/* MCHN START */ + OP_DROPDATA = 0xf8, +/* MCHN END */ + + OP_SMALLDATA = 0xf9, + OP_SMALLINTEGER = 0xfa, + OP_PUBKEYS = 0xfb, + OP_PUBKEYHASH = 0xfd, + OP_PUBKEY = 0xfe, + + OP_INVALIDOPCODE = 0xff, +}; + +const char* GetOpName(opcodetype opcode); + +class scriptnum_error : public std::runtime_error +{ +public: + explicit scriptnum_error(const std::string& str) : std::runtime_error(str) {} +}; + +class CScriptNum +{ +/** + * Numeric opcodes (OP_1ADD, etc) are restricted to operating on 4-byte integers. + * The semantics are subtle, though: operands must be in the range [-2^31 +1...2^31 -1], + * but results may overflow (and are valid as long as they are not used in a subsequent + * numeric operation). CScriptNum enforces those semantics by storing results as + * an int64 and allowing out-of-range values to be returned as a vector of bytes but + * throwing an exception if arithmetic is done or the result is interpreted as an integer. + */ +public: + + explicit CScriptNum(const int64_t& n) + { + m_value = n; + } + + explicit CScriptNum(const std::vector& vch, bool fRequireMinimal) + { + if (vch.size() > nMaxNumSize) { + throw scriptnum_error("script number overflow"); + } + if (fRequireMinimal && vch.size() > 0) { + // Check that the number is encoded with the minimum possible + // number of bytes. + // + // If the most-significant-byte - excluding the sign bit - is zero + // then we're not minimal. Note how this test also rejects the + // negative-zero encoding, 0x80. + if ((vch.back() & 0x7f) == 0) { + // One exception: if there's more than one byte and the most + // significant bit of the second-most-significant-byte is set + // it would conflict with the sign bit. An example of this case + // is +-255, which encode to 0xff00 and 0xff80 respectively. + // (big-endian). + if (vch.size() <= 1 || (vch[vch.size() - 2] & 0x80) == 0) { + throw scriptnum_error("non-minimally encoded script number"); + } + } + } + m_value = set_vch(vch); + } + + inline bool operator==(const int64_t& rhs) const { return m_value == rhs; } + inline bool operator!=(const int64_t& rhs) const { return m_value != rhs; } + inline bool operator<=(const int64_t& rhs) const { return m_value <= rhs; } + inline bool operator< (const int64_t& rhs) const { return m_value < rhs; } + inline bool operator>=(const int64_t& rhs) const { return m_value >= rhs; } + inline bool operator> (const int64_t& rhs) const { return m_value > rhs; } + + inline bool operator==(const CScriptNum& rhs) const { return operator==(rhs.m_value); } + inline bool operator!=(const CScriptNum& rhs) const { return operator!=(rhs.m_value); } + inline bool operator<=(const CScriptNum& rhs) const { return operator<=(rhs.m_value); } + inline bool operator< (const CScriptNum& rhs) const { return operator< (rhs.m_value); } + inline bool operator>=(const CScriptNum& rhs) const { return operator>=(rhs.m_value); } + inline bool operator> (const CScriptNum& rhs) const { return operator> (rhs.m_value); } + + inline CScriptNum operator+( const int64_t& rhs) const { return CScriptNum(m_value + rhs);} + inline CScriptNum operator-( const int64_t& rhs) const { return CScriptNum(m_value - rhs);} + inline CScriptNum operator+( const CScriptNum& rhs) const { return operator+(rhs.m_value); } + inline CScriptNum operator-( const CScriptNum& rhs) const { return operator-(rhs.m_value); } + + inline CScriptNum& operator+=( const CScriptNum& rhs) { return operator+=(rhs.m_value); } + inline CScriptNum& operator-=( const CScriptNum& rhs) { return operator-=(rhs.m_value); } + + inline CScriptNum operator-() const + { + assert(m_value != std::numeric_limits::min()); + return CScriptNum(-m_value); + } + + inline CScriptNum& operator=( const int64_t& rhs) + { + m_value = rhs; + return *this; + } + + inline CScriptNum& operator+=( const int64_t& rhs) + { + assert(rhs == 0 || (rhs > 0 && m_value <= std::numeric_limits::max() - rhs) || + (rhs < 0 && m_value >= std::numeric_limits::min() - rhs)); + m_value += rhs; + return *this; + } + + inline CScriptNum& operator-=( const int64_t& rhs) + { + assert(rhs == 0 || (rhs > 0 && m_value >= std::numeric_limits::min() + rhs) || + (rhs < 0 && m_value <= std::numeric_limits::max() + rhs)); + m_value -= rhs; + return *this; + } + + int getint() const + { + if (m_value > std::numeric_limits::max()) + return std::numeric_limits::max(); + else if (m_value < std::numeric_limits::min()) + return std::numeric_limits::min(); + return m_value; + } + + std::vector getvch() const + { + return serialize(m_value); + } + + static std::vector serialize(const int64_t& value) + { + if(value == 0) + return std::vector(); + + std::vector result; + const bool neg = value < 0; + uint64_t absvalue = neg ? -value : value; + + while(absvalue) + { + result.push_back(absvalue & 0xff); + absvalue >>= 8; + } + +// - If the most significant byte is >= 0x80 and the value is positive, push a +// new zero-byte to make the significant byte < 0x80 again. + +// - If the most significant byte is >= 0x80 and the value is negative, push a +// new 0x80 byte that will be popped off when converting to an integral. + +// - If the most significant byte is < 0x80 and the value is negative, add +// 0x80 to it, since it will be subtracted and interpreted as a negative when +// converting to an integral. + + if (result.back() & 0x80) + result.push_back(neg ? 0x80 : 0); + else if (neg) + result.back() |= 0x80; + + return result; + } + + static const size_t nMaxNumSize = 4; + +private: + static int64_t set_vch(const std::vector& vch) + { + if (vch.empty()) + return 0; + + int64_t result = 0; + for (size_t i = 0; i != vch.size(); ++i) + result |= static_cast(vch[i]) << 8*i; + + // If the input vector's most significant byte is 0x80, remove it from + // the result's msb and return a negative. + if (vch.back() & 0x80) + return -((int64_t)(result & ~(0x80ULL << (8 * (vch.size() - 1))))); + + return result; + } + + int64_t m_value; +}; + +/** Serialized script, used inside transaction inputs and outputs */ +class CScript : public std::vector +{ +protected: + CScript& push_int64(int64_t n) + { + if (n == -1 || (n >= 1 && n <= 16)) + { + push_back(n + (OP_1 - 1)); + } + else if (n == 0) + { + push_back(OP_0); + } + else + { + *this << CScriptNum::serialize(n); + } + return *this; + } +public: + CScript() { } + CScript(const CScript& b) : std::vector(b.begin(), b.end()) { } + CScript(const_iterator pbegin, const_iterator pend) : std::vector(pbegin, pend) { } + CScript(const unsigned char* pbegin, const unsigned char* pend) : std::vector(pbegin, pend) { } + + CScript& operator+=(const CScript& b) + { + insert(end(), b.begin(), b.end()); + return *this; + } + + friend CScript operator+(const CScript& a, const CScript& b) + { + CScript ret = a; + ret += b; + return ret; + } + + CScript(int64_t b) { operator<<(b); } + + explicit CScript(opcodetype b) { operator<<(b); } + explicit CScript(const CScriptNum& b) { operator<<(b); } + explicit CScript(const std::vector& b) { operator<<(b); } + + + CScript& operator<<(int64_t b) { return push_int64(b); } + + CScript& operator<<(opcodetype opcode) + { + if (opcode < 0 || opcode > 0xff) + throw std::runtime_error("CScript::operator<<() : invalid opcode"); + insert(end(), (unsigned char)opcode); + return *this; + } + + CScript& operator<<(const CScriptNum& b) + { + *this << b.getvch(); + return *this; + } + + CScript& operator<<(const std::vector& b) + { + if (b.size() < OP_PUSHDATA1) + { + insert(end(), (unsigned char)b.size()); + } + else if (b.size() <= 0xff) + { + insert(end(), OP_PUSHDATA1); + insert(end(), (unsigned char)b.size()); + } + else if (b.size() <= 0xffff) + { + insert(end(), OP_PUSHDATA2); + unsigned short nSize = b.size(); + insert(end(), (unsigned char*)&nSize, (unsigned char*)&nSize + sizeof(nSize)); + } + else + { + insert(end(), OP_PUSHDATA4); + unsigned int nSize = b.size(); + insert(end(), (unsigned char*)&nSize, (unsigned char*)&nSize + sizeof(nSize)); + } + insert(end(), b.begin(), b.end()); + return *this; + } + + CScript& operator<<(const CScript& b) + { + // I'm not sure if this should push the script or concatenate scripts. + // If there's ever a use for pushing a script onto a script, delete this member fn + assert(!"Warning: Pushing a CScript onto a CScript with << is probably not intended, use + to concatenate!"); + return *this; + } + + + bool GetOp(iterator& pc, opcodetype& opcodeRet, std::vector& vchRet) + { + // Wrapper so it can be called with either iterator or const_iterator + const_iterator pc2 = pc; + bool fRet = GetOp2(pc2, opcodeRet, &vchRet); + pc = begin() + (pc2 - begin()); + return fRet; + } + + bool GetOp(iterator& pc, opcodetype& opcodeRet) + { + const_iterator pc2 = pc; + bool fRet = GetOp2(pc2, opcodeRet, NULL); + pc = begin() + (pc2 - begin()); + return fRet; + } + + bool GetOp(const_iterator& pc, opcodetype& opcodeRet, std::vector& vchRet) const + { + return GetOp2(pc, opcodeRet, &vchRet); + } + + bool GetOp(const_iterator& pc, opcodetype& opcodeRet) const + { + return GetOp2(pc, opcodeRet, NULL); + } + + bool GetOp2(const_iterator& pc, opcodetype& opcodeRet, std::vector* pvchRet) const + { + opcodeRet = OP_INVALIDOPCODE; + if (pvchRet) + pvchRet->clear(); + if (pc >= end()) + return false; + + // Read instruction + if (end() - pc < 1) + return false; + unsigned int opcode = *pc++; + + // Immediate operand + if (opcode <= OP_PUSHDATA4) + { + unsigned int nSize = 0; + if (opcode < OP_PUSHDATA1) + { + nSize = opcode; + } + else if (opcode == OP_PUSHDATA1) + { + if (end() - pc < 1) + return false; + nSize = *pc++; + } + else if (opcode == OP_PUSHDATA2) + { + if (end() - pc < 2) + return false; + nSize = 0; + memcpy(&nSize, &pc[0], 2); + pc += 2; + } + else if (opcode == OP_PUSHDATA4) + { + if (end() - pc < 4) + return false; + memcpy(&nSize, &pc[0], 4); + pc += 4; + } + if (end() - pc < 0 || (unsigned int)(end() - pc) < nSize) + return false; + if (pvchRet) + pvchRet->assign(pc, pc + nSize); + pc += nSize; + } + + opcodeRet = (opcodetype)opcode; + return true; + } + + /** Encode/decode small integers: */ + static int DecodeOP_N(opcodetype opcode) + { + if (opcode == OP_0) + return 0; + assert(opcode >= OP_1 && opcode <= OP_16); + return (int)opcode - (int)(OP_1 - 1); + } + static opcodetype EncodeOP_N(int n) + { + assert(n >= 0 && n <= 16); + if (n == 0) + return OP_0; + return (opcodetype)(OP_1+n-1); + } + + int FindAndDelete(const CScript& b) + { + int nFound = 0; + if (b.empty()) + return nFound; + iterator pc = begin(); + opcodetype opcode; + do + { + while (end() - pc >= (long)b.size() && memcmp(&pc[0], &b[0], b.size()) == 0) + { + pc = erase(pc, pc + b.size()); + ++nFound; + } + } + while (GetOp(pc, opcode)); + return nFound; + } + int Find(opcodetype op) const + { + int nFound = 0; + opcodetype opcode; + for (const_iterator pc = begin(); pc != end() && GetOp(pc, opcode);) + if (opcode == op) + ++nFound; + return nFound; + } + + /** + * Pre-version-0.6, Bitcoin always counted CHECKMULTISIGs + * as 20 sigops. With pay-to-script-hash, that changed: + * CHECKMULTISIGs serialized in scriptSigs are + * counted more accurately, assuming they are of the form + * ... OP_N CHECKMULTISIG ... + */ + unsigned int GetSigOpCount(bool fAccurate) const; + + /** + * Accurately count sigOps, including sigOps in + * pay-to-script-hash transactions: + */ + unsigned int GetSigOpCount(const CScript& scriptSig) const; + + bool IsPayToScriptHash() const; + + /** Called by IsStandardTx and P2SH/BIP62 VerifyScript (which makes it consensus-critical). */ + bool IsPushOnly() const; + +/* MCHN START */ + bool HasSmallIntegerInTheBeginning() const; +/* MCHN END */ + + /** + * Returns whether the script is guaranteed to fail at execution, + * regardless of the initial stack. This allows outputs to be pruned + * instantly when entering the UTXO set. + */ + bool IsUnspendable() const + { +/* MCHN START */ + int op_drop_offset[2]; + int op_drop_size[2]; + int op_return_offset,op_return_size; + return (mc_ParseOpDropOpReturnScript((unsigned char*)&begin()[0],(int)size(),op_drop_offset,op_drop_size,2,&op_return_offset,&op_return_size)) != NULL; +// return (size() > 0 && *begin() == OP_RETURN); +/* MCHN END */ + } + + std::string ToString() const; + +/* MCHN START */ + CScript RemoveOpDrops() const; +/* MCHN END */ + + void clear() + { + // The default std::vector::clear() does not release memory. + std::vector().swap(*this); + } +}; + +#endif // BITCOIN_SCRIPT_SCRIPT_H diff --git a/src/script/script_error.cpp b/src/script/script_error.cpp new file mode 100644 index 00000000..cf39d40f --- /dev/null +++ b/src/script/script_error.cpp @@ -0,0 +1,72 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "script_error.h" + +const char* ScriptErrorString(const ScriptError serror) +{ + switch (serror) + { + case SCRIPT_ERR_OK: + return "No error"; + case SCRIPT_ERR_EVAL_FALSE: + return "Script evaluated without error but finished with a false/empty top stack element"; + case SCRIPT_ERR_VERIFY: + return "Script failed an OP_VERIFY operation"; + case SCRIPT_ERR_EQUALVERIFY: + return "Script failed an OP_EQUALVERIFY operation"; + case SCRIPT_ERR_CHECKMULTISIGVERIFY: + return "Script failed an OP_CHECKMULTISIGVERIFY operation"; + case SCRIPT_ERR_CHECKSIGVERIFY: + return "Script failed an OP_CHECKSIGVERIFY operation"; + case SCRIPT_ERR_NUMEQUALVERIFY: + return "Script failed an OP_NUMEQUALVERIFY operation"; + case SCRIPT_ERR_SCRIPT_SIZE: + return "Script is too big"; + case SCRIPT_ERR_PUSH_SIZE: + return "Push value size limit exceeded"; + case SCRIPT_ERR_OP_COUNT: + return "Operation limit exceeded"; + case SCRIPT_ERR_STACK_SIZE: + return "Stack size limit exceeded"; + case SCRIPT_ERR_SIG_COUNT: + return "Signature count negative or greater than pubkey count"; + case SCRIPT_ERR_PUBKEY_COUNT: + return "Pubkey count negative or limit exceeded"; + case SCRIPT_ERR_BAD_OPCODE: + return "Opcode missing or not understood"; + case SCRIPT_ERR_DISABLED_OPCODE: + return "Attempted to use a disabled opcode"; + case SCRIPT_ERR_INVALID_STACK_OPERATION: + return "Operation not valid with the current stack size"; + case SCRIPT_ERR_INVALID_ALTSTACK_OPERATION: + return "Operation not valid with the current altstack size"; + case SCRIPT_ERR_OP_RETURN: + return "OP_RETURN was encountered"; + case SCRIPT_ERR_UNBALANCED_CONDITIONAL: + return "Invalid OP_IF construction"; + case SCRIPT_ERR_SIG_HASHTYPE: + return "Signature hash type missing or not understood"; + case SCRIPT_ERR_SIG_DER: + return "Non-canonical DER signature"; + case SCRIPT_ERR_MINIMALDATA: + return "Data push larger than necessary"; + case SCRIPT_ERR_SIG_PUSHONLY: + return "Only non-push operators allowed in signatures"; + case SCRIPT_ERR_SIG_HIGH_S: + return "Non-canonical signature: S value is unnecessarily high"; + case SCRIPT_ERR_SIG_NULLDUMMY: + return "Dummy CHECKMULTISIG argument must be zero"; + case SCRIPT_ERR_DISCOURAGE_UPGRADABLE_NOPS: + return "NOPx reserved for soft-fork upgrades"; + case SCRIPT_ERR_PUBKEYTYPE: + return "Public key is neither compressed or uncompressed"; + case SCRIPT_ERR_UNKNOWN_ERROR: + case SCRIPT_ERR_ERROR_COUNT: + default: break; + } + return "unknown error"; +} diff --git a/src/script/script_error.h b/src/script/script_error.h new file mode 100644 index 00000000..4bdfccca --- /dev/null +++ b/src/script/script_error.h @@ -0,0 +1,58 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_SCRIPT_SCRIPT_ERROR_H +#define BITCOIN_SCRIPT_SCRIPT_ERROR_H + +typedef enum ScriptError_t +{ + SCRIPT_ERR_OK = 0, + SCRIPT_ERR_UNKNOWN_ERROR, + SCRIPT_ERR_EVAL_FALSE, + SCRIPT_ERR_OP_RETURN, + + /* Max sizes */ + SCRIPT_ERR_SCRIPT_SIZE, + SCRIPT_ERR_PUSH_SIZE, + SCRIPT_ERR_OP_COUNT, + SCRIPT_ERR_STACK_SIZE, + SCRIPT_ERR_SIG_COUNT, + SCRIPT_ERR_PUBKEY_COUNT, + + /* Failed verify operations */ + SCRIPT_ERR_VERIFY, + SCRIPT_ERR_EQUALVERIFY, + SCRIPT_ERR_CHECKMULTISIGVERIFY, + SCRIPT_ERR_CHECKSIGVERIFY, + SCRIPT_ERR_NUMEQUALVERIFY, + + /* Logical/Format/Canonical errors */ + SCRIPT_ERR_BAD_OPCODE, + SCRIPT_ERR_DISABLED_OPCODE, + SCRIPT_ERR_INVALID_STACK_OPERATION, + SCRIPT_ERR_INVALID_ALTSTACK_OPERATION, + SCRIPT_ERR_UNBALANCED_CONDITIONAL, + + /* BIP62 */ + SCRIPT_ERR_SIG_HASHTYPE, + SCRIPT_ERR_SIG_DER, + SCRIPT_ERR_MINIMALDATA, + SCRIPT_ERR_SIG_PUSHONLY, + SCRIPT_ERR_SIG_HIGH_S, + SCRIPT_ERR_SIG_NULLDUMMY, + SCRIPT_ERR_PUBKEYTYPE, + + /* softfork safeness */ + SCRIPT_ERR_DISCOURAGE_UPGRADABLE_NOPS, + + SCRIPT_ERR_ERROR_COUNT +} ScriptError; + +#define SCRIPT_ERR_LAST SCRIPT_ERR_ERROR_COUNT + +const char* ScriptErrorString(const ScriptError error); + +#endif // BITCOIN_SCRIPT_SCRIPT_ERROR_H diff --git a/src/script/sigcache.cpp b/src/script/sigcache.cpp new file mode 100644 index 00000000..f52a9b7a --- /dev/null +++ b/src/script/sigcache.cpp @@ -0,0 +1,91 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "sigcache.h" + +#include "keys/pubkey.h" +#include "utils/random.h" +#include "structs/uint256.h" +#include "utils/util.h" + +#include +#include + +namespace { + +/** + * Valid signature cache, to avoid doing expensive ECDSA signature checking + * twice for every transaction (once when accepted into memory pool, and + * again when accepted into the block chain) + */ +class CSignatureCache +{ +private: + //! sigdata_type is (signature hash, signature, public key): + typedef boost::tuple, CPubKey> sigdata_type; + std::set< sigdata_type> setValid; + boost::shared_mutex cs_sigcache; + +public: + bool + Get(const uint256 &hash, const std::vector& vchSig, const CPubKey& pubKey) + { + boost::shared_lock lock(cs_sigcache); + + sigdata_type k(hash, vchSig, pubKey); + std::set::iterator mi = setValid.find(k); + if (mi != setValid.end()) + return true; + return false; + } + + void Set(const uint256 &hash, const std::vector& vchSig, const CPubKey& pubKey) + { + // DoS prevention: limit cache size to less than 10MB + // (~200 bytes per cache entry times 50,000 entries) + // Since there are a maximum of 20,000 signature operations per block + // 50,000 is a reasonable default. + int64_t nMaxCacheSize = GetArg("-maxsigcachesize", 50000); + if (nMaxCacheSize <= 0) return; + + boost::unique_lock lock(cs_sigcache); + + while (static_cast(setValid.size()) > nMaxCacheSize) + { + // Evict a random entry. Random because that helps + // foil would-be DoS attackers who might try to pre-generate + // and re-use a set of valid signatures just-slightly-greater + // than our cache size. + uint256 randomHash = GetRandHash(); + std::vector unused; + std::set::iterator it = + setValid.lower_bound(sigdata_type(randomHash, unused, unused)); + if (it == setValid.end()) + it = setValid.begin(); + setValid.erase(*it); + } + + sigdata_type k(hash, vchSig, pubKey); + setValid.insert(k); + } +}; + +} + +bool CachingTransactionSignatureChecker::VerifySignature(const std::vector& vchSig, const CPubKey& pubkey, const uint256& sighash) const +{ + static CSignatureCache signatureCache; + + if (signatureCache.Get(sighash, vchSig, pubkey)) + return true; + + if (!TransactionSignatureChecker::VerifySignature(vchSig, pubkey, sighash)) + return false; + + if (store) + signatureCache.Set(sighash, vchSig, pubkey); + return true; +} diff --git a/src/script/sigcache.h b/src/script/sigcache.h new file mode 100644 index 00000000..dd53157c --- /dev/null +++ b/src/script/sigcache.h @@ -0,0 +1,27 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_SCRIPT_SIGCACHE_H +#define BITCOIN_SCRIPT_SIGCACHE_H + +#include "script/interpreter.h" + +#include + +class CPubKey; + +class CachingTransactionSignatureChecker : public TransactionSignatureChecker +{ +private: + bool store; + +public: + CachingTransactionSignatureChecker(const CTransaction* txToIn, unsigned int nInIn, bool storeIn=true) : TransactionSignatureChecker(txToIn, nInIn), store(storeIn) {} + + bool VerifySignature(const std::vector& vchSig, const CPubKey& vchPubKey, const uint256& sighash) const; +}; + +#endif // BITCOIN_SCRIPT_SIGCACHE_H diff --git a/src/script/sign.cpp b/src/script/sign.cpp new file mode 100644 index 00000000..140171ee --- /dev/null +++ b/src/script/sign.cpp @@ -0,0 +1,296 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "utils/util.h" +#include "script/sign.h" + +#include "primitives/transaction.h" +#include "keys/key.h" +#include "wallet/keystore.h" +#include "script/standard.h" +#include "structs/uint256.h" + +#include + +using namespace std; + +typedef vector valtype; + +bool Sign1(const CKeyID& address, const CKeyStore& keystore, uint256 hash, int nHashType, CScript& scriptSigRet) +{ +/* MCHN START */ +/* + if(mc_gState->m_Permissions->CanSend(NULL,(unsigned char*)(&address)) == 0) + { + return false; + } + */ +/* MCHN END */ + + CKey key; + if (!keystore.GetKey(address, key)) + return false; + + vector vchSig; + if (!key.Sign(hash, vchSig)) + return false; + vchSig.push_back((unsigned char)nHashType); + scriptSigRet << vchSig; + + return true; +} + +bool SignN(const vector& multisigdata, const CKeyStore& keystore, uint256 hash, int nHashType, CScript& scriptSigRet) +{ + int nSigned = 0; + int nRequired = multisigdata.front()[0]; + bool send_address_found=false; + for (unsigned int i = 1; i < multisigdata.size()-1 && ((nSigned < nRequired) || !send_address_found); i++) + { + const valtype& pubkey = multisigdata[i]; + CKeyID keyID = CPubKey(pubkey).GetID(); + +/* MCHN START */ + // We are trying to use at least one address with send permission + // otherwise we can sign completely with addresses without send permission + // and entire signature will fail + if(mc_gState->m_Permissions->CanSend(NULL,(unsigned char*)(&keyID))) + { + send_address_found=true; + } +/* MCHN END */ + + if (Sign1(keyID, keystore, hash, nHashType, scriptSigRet)) + ++nSigned; + } +/* MCHN START */ + // As a result of looking for send permission we may can sign more than required + return nSigned>=nRequired; +// return nSigned==nRequired; +/* MCHN END */ +} + +/** + * Sign scriptPubKey with private keys stored in keystore, given transaction hash and hash type. + * Signatures are returned in scriptSigRet (or returns false if scriptPubKey can't be signed), + * unless whichTypeRet is TX_SCRIPTHASH, in which case scriptSigRet is the redemption script. + * Returns false if scriptPubKey could not be completely satisfied. + */ +bool Solver(const CKeyStore& keystore, const CScript& scriptPubKey, uint256 hash, int nHashType, + CScript& scriptSigRet, txnouttype& whichTypeRet) +{ + scriptSigRet.clear(); + + vector vSolutions; + if (!Solver(scriptPubKey, whichTypeRet, vSolutions)) + return false; + + CKeyID keyID; + switch (whichTypeRet) + { + case TX_NONSTANDARD: + case TX_NULL_DATA: + return false; + case TX_PUBKEY: + keyID = CPubKey(vSolutions[0]).GetID(); + return Sign1(keyID, keystore, hash, nHashType, scriptSigRet); + case TX_PUBKEYHASH: + keyID = CKeyID(uint160(vSolutions[0])); + if (!Sign1(keyID, keystore, hash, nHashType, scriptSigRet)) + return false; + else + { + CPubKey vch; + keystore.GetPubKey(keyID, vch); + scriptSigRet << ToByteVector(vch); + } + return true; + case TX_SCRIPTHASH: + return keystore.GetCScript(uint160(vSolutions[0]), scriptSigRet); + + case TX_MULTISIG: + scriptSigRet << OP_0; // workaround CHECKMULTISIG bug + return (SignN(vSolutions, keystore, hash, nHashType, scriptSigRet)); + } + return false; +} + +bool SignSignature(const CKeyStore &keystore, const CScript& fromPubKey, CMutableTransaction& txTo, unsigned int nIn, int nHashType) +{ + assert(nIn < txTo.vin.size()); + CTxIn& txin = txTo.vin[nIn]; + + // Leave out the signature from the hash, since a signature can't sign itself. + // The checksig op will also drop the signatures from its hash. + uint256 hash = SignatureHash(fromPubKey, txTo, nIn, nHashType); + txnouttype whichType; + if (!Solver(keystore, fromPubKey, hash, nHashType, txin.scriptSig, whichType)) + return false; + + if (whichType == TX_SCRIPTHASH) + { + // Solver returns the subscript that need to be evaluated; + // the final scriptSig is the signatures from that + // and then the serialized subscript: + CScript subscript = txin.scriptSig; + + // Recompute txn hash using subscript in place of scriptPubKey: + uint256 hash2 = SignatureHash(subscript, txTo, nIn, nHashType); + + txnouttype subType; + bool fSolved = + Solver(keystore, subscript, hash2, nHashType, txin.scriptSig, subType) && subType != TX_SCRIPTHASH; + // Append serialized subscript whether or not it is completely signed: + txin.scriptSig << static_cast(subscript); + if (!fSolved) return false; + } + + // Test solution + if(GetBoolArg("-verifyjustsigned",true)) + { + return VerifyScript(txin.scriptSig, fromPubKey, STANDARD_SCRIPT_VERIFY_FLAGS, MutableTransactionSignatureChecker(&txTo, nIn)); + } + return true; +} + +bool SignSignature(const CKeyStore &keystore, const CTransaction& txFrom, CMutableTransaction& txTo, unsigned int nIn, int nHashType) +{ + assert(nIn < txTo.vin.size()); + CTxIn& txin = txTo.vin[nIn]; + assert(txin.prevout.n < txFrom.vout.size()); + const CTxOut& txout = txFrom.vout[txin.prevout.n]; + + return SignSignature(keystore, txout.scriptPubKey, txTo, nIn, nHashType); +} + +static CScript PushAll(const vector& values) +{ + CScript result; + BOOST_FOREACH(const valtype& v, values) + result << v; + return result; +} + +static CScript CombineMultisig(const CScript& scriptPubKey, const CTransaction& txTo, unsigned int nIn, + const vector& vSolutions, + const vector& sigs1, const vector& sigs2) +{ + // Combine all the signatures we've got: + set allsigs; + BOOST_FOREACH(const valtype& v, sigs1) + { + if (!v.empty()) + allsigs.insert(v); + } + BOOST_FOREACH(const valtype& v, sigs2) + { + if (!v.empty()) + allsigs.insert(v); + } + + // Build a map of pubkey -> signature by matching sigs to pubkeys: + assert(vSolutions.size() > 1); + unsigned int nSigsRequired = vSolutions.front()[0]; + unsigned int nPubKeys = vSolutions.size()-2; + map sigs; +/* MCHN START */ + bool cannot_send=true; +/* MCHN END */ + BOOST_FOREACH(const valtype& sig, allsigs) + { + for (unsigned int i = 0; i < nPubKeys; i++) + { + const valtype& pubkey = vSolutions[i+1]; + if (sigs.count(pubkey)) + continue; // Already got a sig for this pubkey + +/* MCHN START */ +// if (TransactionSignatureChecker(&txTo, nIn).CheckSig(sig, pubkey, scriptPubKey)) + if (TransactionSignatureChecker(&txTo, nIn).CheckSig(sig, pubkey, scriptPubKey, cannot_send)) + { +/* MCHN END */ + sigs[pubkey] = sig; + break; + } + } + } + // Now build a merged CScript: + unsigned int nSigsHave = 0; + CScript result; result << OP_0; // pop-one-too-many workaround + for (unsigned int i = 0; i < nPubKeys && nSigsHave < nSigsRequired; i++) + { + if (sigs.count(vSolutions[i+1])) + { + result << sigs[vSolutions[i+1]]; + ++nSigsHave; + } + } + // Fill any missing with OP_0: + for (unsigned int i = nSigsHave; i < nSigsRequired; i++) + result << OP_0; + + return result; +} + +static CScript CombineSignatures(const CScript& scriptPubKey, const CTransaction& txTo, unsigned int nIn, + const txnouttype txType, const vector& vSolutions, + vector& sigs1, vector& sigs2) +{ + switch (txType) + { + case TX_NONSTANDARD: + case TX_NULL_DATA: + // Don't know anything about this, assume bigger one is correct: + if (sigs1.size() >= sigs2.size()) + return PushAll(sigs1); + return PushAll(sigs2); + case TX_PUBKEY: + case TX_PUBKEYHASH: + // Signatures are bigger than placeholders or empty scripts: + if (sigs1.empty() || sigs1[0].empty()) + return PushAll(sigs2); + return PushAll(sigs1); + case TX_SCRIPTHASH: + if (sigs1.empty() || sigs1.back().empty()) + return PushAll(sigs2); + else if (sigs2.empty() || sigs2.back().empty()) + return PushAll(sigs1); + else + { + // Recur to combine: + valtype spk = sigs1.back(); + CScript pubKey2(spk.begin(), spk.end()); + + txnouttype txType2; + vector > vSolutions2; + Solver(pubKey2, txType2, vSolutions2); + sigs1.pop_back(); + sigs2.pop_back(); + CScript result = CombineSignatures(pubKey2, txTo, nIn, txType2, vSolutions2, sigs1, sigs2); + result << spk; + return result; + } + case TX_MULTISIG: + return CombineMultisig(scriptPubKey, txTo, nIn, vSolutions, sigs1, sigs2); + } + + return CScript(); +} + +CScript CombineSignatures(const CScript& scriptPubKey, const CTransaction& txTo, unsigned int nIn, + const CScript& scriptSig1, const CScript& scriptSig2) +{ + txnouttype txType; + vector > vSolutions; + Solver(scriptPubKey, txType, vSolutions); + + vector stack1; + EvalScript(stack1, scriptSig1, SCRIPT_VERIFY_STRICTENC, BaseSignatureChecker()); + vector stack2; + EvalScript(stack2, scriptSig2, SCRIPT_VERIFY_STRICTENC, BaseSignatureChecker()); + + return CombineSignatures(scriptPubKey, txTo, nIn, txType, vSolutions, stack1, stack2); +} diff --git a/src/script/sign.h b/src/script/sign.h new file mode 100644 index 00000000..81e71992 --- /dev/null +++ b/src/script/sign.h @@ -0,0 +1,27 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_SCRIPT_SIGN_H +#define BITCOIN_SCRIPT_SIGN_H + +#include "script/interpreter.h" + +class CKeyStore; +class CScript; +class CTransaction; + +struct CMutableTransaction; + +bool SignSignature(const CKeyStore& keystore, const CScript& fromPubKey, CMutableTransaction& txTo, unsigned int nIn, int nHashType=SIGHASH_ALL); +bool SignSignature(const CKeyStore& keystore, const CTransaction& txFrom, CMutableTransaction& txTo, unsigned int nIn, int nHashType=SIGHASH_ALL); + +/** + * Given two sets of signatures for scriptPubKey, possibly with OP_0 placeholders, + * combine them intelligently and return the result. + */ +CScript CombineSignatures(const CScript& scriptPubKey, const CTransaction& txTo, unsigned int nIn, const CScript& scriptSig1, const CScript& scriptSig2); + +#endif // BITCOIN_SCRIPT_SIGN_H diff --git a/src/script/standard.cpp b/src/script/standard.cpp new file mode 100644 index 00000000..d92acb04 --- /dev/null +++ b/src/script/standard.cpp @@ -0,0 +1,435 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "script/standard.h" + +#include "keys/pubkey.h" +#include "script/script.h" +#include "utils/util.h" +#include "utils/utilstrencodings.h" + +#include "multichain/multichain.h" + +#include + +using namespace std; + +typedef vector valtype; + +unsigned nMaxDatacarrierBytes = MAX_OP_RETURN_RELAY; + +CScriptID::CScriptID(const CScript& in) : uint160(Hash160(in.begin(), in.end())) {} + +const char* GetTxnOutputType(txnouttype t) +{ + switch (t) + { + case TX_NONSTANDARD: return "nonstandard"; + case TX_PUBKEY: return "pubkey"; + case TX_PUBKEYHASH: return "pubkeyhash"; + case TX_SCRIPTHASH: return "scripthash"; + case TX_MULTISIG: return "multisig"; + case TX_NULL_DATA: return "nulldata"; + } + return NULL; +} + +/** + * Return public keys or hashes from scriptPubKey, for 'standard' transaction types. + */ +bool Solver(const CScript& scriptPubKey, txnouttype& typeRet, vector >& vSolutionsRet) +{ + // Templates + static multimap mTemplates; + if (mTemplates.empty()) + { + // Standard tx, sender provides pubkey, receiver adds signature + mTemplates.insert(make_pair(TX_PUBKEY, CScript() << OP_PUBKEY << OP_CHECKSIG)); + + // Bitcoin address tx, sender provides hash of pubkey, receiver provides signature and pubkey + mTemplates.insert(make_pair(TX_PUBKEYHASH, CScript() << OP_DUP << OP_HASH160 << OP_PUBKEYHASH << OP_EQUALVERIFY << OP_CHECKSIG)); + + // Sender provides N pubkeys, receivers provides M signatures + mTemplates.insert(make_pair(TX_MULTISIG, CScript() << OP_SMALLINTEGER << OP_PUBKEYS << OP_SMALLINTEGER << OP_CHECKMULTISIG)); + + // Empty, provably prunable, data-carrying output + if (GetBoolArg("-datacarrier", true)) + mTemplates.insert(make_pair(TX_NULL_DATA, CScript() << OP_RETURN << OP_SMALLDATA)); + + mTemplates.insert(make_pair(TX_NULL_DATA, CScript() << OP_RETURN)); + +/* MCHN START*/ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(mc_gState->m_Features->Streams()) + { + mTemplates.insert(make_pair(TX_NULL_DATA, CScript() << OP_DROPDATA << OP_DROP << OP_RETURN)); + mTemplates.insert(make_pair(TX_NULL_DATA, CScript() << OP_DROPDATA << OP_DROP << OP_RETURN << OP_SMALLDATA)); + mTemplates.insert(make_pair(TX_NULL_DATA, CScript() << OP_DROPDATA << OP_DROP << OP_DROPDATA << OP_DROP << OP_RETURN)); + mTemplates.insert(make_pair(TX_NULL_DATA, CScript() << OP_DROPDATA << OP_DROP << OP_DROPDATA << OP_DROP << OP_RETURN << OP_SMALLDATA)); + } + + mTemplates.insert(make_pair(TX_SCRIPTHASH, CScript() << OP_HASH160 << OP_PUBKEYHASH << OP_EQUAL)); + } +/* MCHN END */ + } + + // Shortcut for pay-to-script-hash, which are more constrained than the other types: + // it is always OP_HASH160 20 [20 byte hash] OP_EQUAL +/* MCHN START - commented out*/ +/* + if (scriptPubKey.IsPayToScriptHash()) + { + typeRet = TX_SCRIPTHASH; + vector hashBytes(scriptPubKey.begin()+2, scriptPubKey.begin()+22); + vSolutionsRet.push_back(hashBytes); + return true; + } + */ +/* MCHN END */ + + // Scan templates + const CScript& script1 = scriptPubKey; + BOOST_FOREACH(const PAIRTYPE(txnouttype, CScript)& tplate, mTemplates) + { +/* MCHN START*/ +// const CScript& script2 = tplate.second; + CScript script2 = CScript(tplate.second); + + for(int d=0;d<=mc_gState->m_NetworkParams->GetInt64Param("maxstdopdropscount");d++) + { +/* MCHN END*/ + + vSolutionsRet.clear(); + + opcodetype opcode1, opcode2; + vector vch1, vch2; + + // Compare + CScript::const_iterator pc1 = script1.begin(); + CScript::const_iterator pc2 = script2.begin(); + while (true) + { + if (pc1 == script1.end() && pc2 == script2.end()) + { + // Found a match + typeRet = tplate.first; + if (typeRet == TX_MULTISIG) + { + // Additional checks for TX_MULTISIG: + unsigned char m = vSolutionsRet.front()[0]; + unsigned char n = vSolutionsRet.back()[0]; + if (m < 1 || n < 1 || m > n || vSolutionsRet.size()-2 != n) + return false; + } +/* MCHN START */ + if(d) + { +// LogPrint("mchnminor","mchn: %d OP_DROP element(s) found in script\n",d); + } +/* MCHN END */ + return true; + } + if (!script1.GetOp(pc1, opcode1, vch1)) + break; + if (!script2.GetOp(pc2, opcode2, vch2)) + break; + + // Template matching opcodes: + if (opcode2 == OP_PUBKEYS) + { + while (vch1.size() >= 33 && vch1.size() <= 65) + { + vSolutionsRet.push_back(vch1); + if (!script1.GetOp(pc1, opcode1, vch1)) + break; + } + if (!script2.GetOp(pc2, opcode2, vch2)) + break; + // Normal situation is to fall through + // to other if/else statements + } + + if (opcode2 == OP_PUBKEY) + { + if (vch1.size() < 33 || vch1.size() > 65) + break; + vSolutionsRet.push_back(vch1); + } + else if (opcode2 == OP_PUBKEYHASH) + { + if (vch1.size() != sizeof(uint160)) + break; + vSolutionsRet.push_back(vch1); + } + else if (opcode2 == OP_SMALLINTEGER) + { // Single-byte small integer pushed onto vSolutions + if (opcode1 == OP_0 || + (opcode1 >= OP_1 && opcode1 <= OP_16)) + { + char n = (char)CScript::DecodeOP_N(opcode1); + vSolutionsRet.push_back(valtype(1, n)); + } + else + break; + } + else if (opcode2 == OP_SMALLDATA) + { + // small pushdata, <= nMaxDatacarrierBytes +/* MCHN START */ + nMaxDatacarrierBytes=MAX_OP_RETURN_RELAY; +/* MCHN END */ + if (vch1.size() > nMaxDatacarrierBytes) + break; + } +/* MCHN START */ + else if (opcode2 == OP_DROPDATA) + { + // small pushdata, <= nMaxDatacarrierBytes + if(mc_gState->m_Features->VerifySizeOfOpDropElements()) + { + if (vch1.size() > MAX_SCRIPT_ELEMENT_SIZE) + break; + } + } +/* MCHN END */ + else if (opcode1 != opcode2 || vch1 != vch2) + { + // Others must match exactly + break; + } + } + +/* MCHN START */ + script2 << OP_DROPDATA << OP_DROP; + } +/* MCHN END */ + } + + vSolutionsRet.clear(); + typeRet = TX_NONSTANDARD; + return false; +} + +int ScriptSigArgsExpected(txnouttype t, const std::vector >& vSolutions) +{ + switch (t) + { + case TX_NONSTANDARD: + case TX_NULL_DATA: + return -1; + case TX_PUBKEY: + return 1; + case TX_PUBKEYHASH: + return 2; + case TX_MULTISIG: + if (vSolutions.size() < 1 || vSolutions[0].size() < 1) + return -1; + return vSolutions[0][0] + 1; + case TX_SCRIPTHASH: + return 1; // doesn't include args needed by the script + } + return -1; +} + +bool IsStandard(const CScript& scriptPubKey, txnouttype& whichType) +{ + vector vSolutions; + if (!Solver(scriptPubKey, whichType, vSolutions)) + return false; + + if (whichType == TX_MULTISIG) + { + unsigned char m = vSolutions.front()[0]; + unsigned char n = vSolutions.back()[0]; + // Support up to x-of-3 multisig txns as standard + if (n < 1 || n > 3) + return false; + if (m < 1 || m > n) + return false; + } + + return whichType != TX_NONSTANDARD; +} + +bool ExtractDestinationScriptValid(const CScript& scriptPubKey, CTxDestination& addressRet) +{ + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return ExtractDestination(scriptPubKey, addressRet); + } + CScript::const_iterator pc1 = scriptPubKey.begin(); + unsigned char *ptr; + int size; + + ptr=(unsigned char*)(&pc1[0]); + size=scriptPubKey.end()-pc1; + + if( (size >= 25) && (ptr[0] == OP_DUP) ) // pay-to-pubkeyhash + { + addressRet = CKeyID(*(uint160*)(ptr+3)); + return true; + } + + if( (size >= 23) && (ptr[0] == OP_HASH160) ) // pay-to-scripthash + { + addressRet = CScriptID(*(uint160*)(ptr+2)); + return true; + + } + + if( size >= 35 ) // pay-to-pubkey + { + if( (ptr[0] >= 33) && (ptr[0] <= 65) ) + { + if(size >= 2+ptr[0]) + { + CPubKey pubKey(ptr+1, ptr+1+ptr[0]); + if (!pubKey.IsValid()) + return false; + + addressRet = pubKey.GetID(); + return true; + } + } + } + + return false; +} + +bool ExtractDestination(const CScript& scriptPubKey, CTxDestination& addressRet) +{ + vector vSolutions; + txnouttype whichType; + if (!Solver(scriptPubKey, whichType, vSolutions)) + return false; + + if (whichType == TX_PUBKEY) + { + CPubKey pubKey(vSolutions[0]); + if (!pubKey.IsValid()) + return false; + + addressRet = pubKey.GetID(); + return true; + } + else if (whichType == TX_PUBKEYHASH) + { + addressRet = CKeyID(uint160(vSolutions[0])); + return true; + } + else if (whichType == TX_SCRIPTHASH) + { + addressRet = CScriptID(uint160(vSolutions[0])); + return true; + } + // Multisig txns have more than one address... + return false; +} + +bool ExtractDestinations(const CScript& scriptPubKey, txnouttype& typeRet, vector& addressRet, int& nRequiredRet) +{ + addressRet.clear(); + typeRet = TX_NONSTANDARD; + vector vSolutions; + +/* MCHN START */ + const CScript& scriptPubKeyDestinationOnly=scriptPubKey.RemoveOpDrops(); + if (!Solver(scriptPubKeyDestinationOnly, typeRet, vSolutions)) +// if (!Solver(scriptPubKey, typeRet, vSolutions)) + return false; +/* MCHN END */ + if (typeRet == TX_NULL_DATA){ + // This is data, not addresses + return false; + } + + if (typeRet == TX_MULTISIG) + { + nRequiredRet = vSolutions.front()[0]; + for (unsigned int i = 1; i < vSolutions.size()-1; i++) + { + CPubKey pubKey(vSolutions[i]); + if (!pubKey.IsValid()) + continue; + + CTxDestination address = pubKey.GetID(); + addressRet.push_back(address); + } + + if (addressRet.empty()) + return false; + } + else + { + nRequiredRet = 1; + CTxDestination address; +// if (!ExtractDestination(scriptPubKey, address)) + if (!ExtractDestinationScriptValid(scriptPubKeyDestinationOnly, address)) + return false; + addressRet.push_back(address); + } + + return true; +} + +namespace +{ +class CScriptVisitor : public boost::static_visitor +{ +private: + CScript *script; +public: + CScriptVisitor(CScript *scriptin) { script = scriptin; } + + bool operator()(const CNoDestination &dest) const { + script->clear(); + return false; + } + + bool operator()(const CKeyID &keyID) const { + script->clear(); + *script << OP_DUP << OP_HASH160 << ToByteVector(keyID) << OP_EQUALVERIFY << OP_CHECKSIG; + return true; + } + + bool operator()(const CScriptID &scriptID) const { + script->clear(); + *script << OP_HASH160 << ToByteVector(scriptID) << OP_EQUAL; + return true; + } +}; +} + +CScript GetScriptForDestination(const CTxDestination& dest) +{ + CScript script; + + boost::apply_visitor(CScriptVisitor(&script), dest); + return script; +} + +CScript GetScriptForMultisig(int nRequired, const std::vector& keys) +{ + CScript script; + + script << CScript::EncodeOP_N(nRequired); + BOOST_FOREACH(const CPubKey& key, keys) + script << ToByteVector(key); + script << CScript::EncodeOP_N(keys.size()) << OP_CHECKMULTISIG; + return script; +} + +/* MCHN START */ +CScript GetScriptForPubKey(const CPubKey& key) +{ + CScript script; + script << ToByteVector(key); + script << OP_CHECKSIG; + + return script; +} +/* MCHN END */ diff --git a/src/script/standard.h b/src/script/standard.h new file mode 100644 index 00000000..f42067ba --- /dev/null +++ b/src/script/standard.h @@ -0,0 +1,97 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_SCRIPT_STANDARD_H +#define BITCOIN_SCRIPT_STANDARD_H + +#include "script/interpreter.h" +#include "structs/uint256.h" + +#include + +#include + +class CKeyID; +class CScript; + +/** A reference to a CScript: the Hash160 of its serialization (see script.h) */ +class CScriptID : public uint160 +{ +public: + CScriptID() : uint160(0) {} + CScriptID(const CScript& in); + CScriptID(const uint160& in) : uint160(in) {} +}; + +extern unsigned int MAX_OP_RETURN_RELAY; // MCHN global +extern unsigned nMaxDatacarrierBytes; + +/** + * Mandatory script verification flags that all new blocks must comply with for + * them to be valid. (but old blocks may not comply with) Currently just P2SH, + * but in the future other flags may be added, such as a soft-fork to enforce + * strict DER encoding. + * + * Failing one of these tests may trigger a DoS ban - see CheckInputs() for + * details. + */ +static const unsigned int MANDATORY_SCRIPT_VERIFY_FLAGS = SCRIPT_VERIFY_P2SH; + +/** + * Standard script verification flags that standard transactions will comply + * with. However scripts violating these flags may still be present in valid + * blocks and we must accept those blocks. + */ +static const unsigned int STANDARD_SCRIPT_VERIFY_FLAGS = MANDATORY_SCRIPT_VERIFY_FLAGS | + SCRIPT_VERIFY_DERSIG | + SCRIPT_VERIFY_STRICTENC | + SCRIPT_VERIFY_MINIMALDATA | + SCRIPT_VERIFY_NULLDUMMY | + SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_NOPS; + +/** For convenience, standard but not mandatory verify flags. */ +static const unsigned int STANDARD_NOT_MANDATORY_VERIFY_FLAGS = STANDARD_SCRIPT_VERIFY_FLAGS & ~MANDATORY_SCRIPT_VERIFY_FLAGS; + +enum txnouttype +{ + TX_NONSTANDARD, + // 'standard' transaction types: + TX_PUBKEY, + TX_PUBKEYHASH, + TX_SCRIPTHASH, + TX_MULTISIG, + TX_NULL_DATA, +}; + +class CNoDestination { +public: + friend bool operator==(const CNoDestination &a, const CNoDestination &b) { return true; } + friend bool operator<(const CNoDestination &a, const CNoDestination &b) { return true; } +}; + +/** + * A txout script template with a specific destination. It is either: + * * CNoDestination: no destination set + * * CKeyID: TX_PUBKEYHASH destination + * * CScriptID: TX_SCRIPTHASH destination + * A CTxDestination is the internal data type encoded in a CBitcoinAddress + */ +typedef boost::variant CTxDestination; + +const char* GetTxnOutputType(txnouttype t); + +bool Solver(const CScript& scriptPubKey, txnouttype& typeRet, std::vector >& vSolutionsRet); +int ScriptSigArgsExpected(txnouttype t, const std::vector >& vSolutions); +bool IsStandard(const CScript& scriptPubKey, txnouttype& whichType); +bool ExtractDestination(const CScript& scriptPubKey, CTxDestination& addressRet); +bool ExtractDestinations(const CScript& scriptPubKey, txnouttype& typeRet, std::vector& addressRet, int& nRequiredRet); + +CScript GetScriptForDestination(const CTxDestination& dest); +CScript GetScriptForMultisig(int nRequired, const std::vector& keys); +/* MCHN START */ +CScript GetScriptForPubKey(const CPubKey& key); +/* MCHN END */ +#endif // BITCOIN_SCRIPT_STANDARD_H diff --git a/src/secp256k1/.gitignore b/src/secp256k1/.gitignore new file mode 100644 index 00000000..e0b7b7a4 --- /dev/null +++ b/src/secp256k1/.gitignore @@ -0,0 +1,41 @@ +bench_inv +bench_ecdh +bench_sign +bench_verify +bench_schnorr_verify +bench_recover +bench_internal +tests +gen_context +*.exe +*.so +*.a +!.gitignore + +Makefile +configure +.libs/ +Makefile.in +aclocal.m4 +autom4te.cache/ +config.log +config.status +*.tar.gz +*.la +libtool +.deps/ +.dirstamp +build-aux/ +*.lo +*.o +*~ +src/libsecp256k1-config.h +src/libsecp256k1-config.h.in +src/ecmult_static_context.h +m4/libtool.m4 +m4/ltoptions.m4 +m4/ltsugar.m4 +m4/ltversion.m4 +m4/lt~obsolete.m4 +src/stamp-h1 +libsecp256k1.pc diff --git a/src/secp256k1/.travis.yml b/src/secp256k1/.travis.yml new file mode 100644 index 00000000..4e1e73c3 --- /dev/null +++ b/src/secp256k1/.travis.yml @@ -0,0 +1,63 @@ +language: c +sudo: false +addons: + apt: + packages: libgmp-dev +compiler: + - clang + - gcc +env: + global: + - FIELD=auto BIGNUM=auto SCALAR=auto ENDOMORPHISM=no STATICPRECOMPUTATION=yes ASM=no BUILD=check EXTRAFLAGS= HOST= ECDH=no schnorr=no RECOVERY=no + matrix: + - SCALAR=32bit RECOVERY=yes + - SCALAR=32bit FIELD=32bit ECDH=yes + - SCALAR=64bit + - FIELD=64bit RECOVERY=yes + - FIELD=64bit ENDOMORPHISM=yes + - FIELD=64bit ENDOMORPHISM=yes ECDH=yes + - FIELD=64bit ASM=x86_64 + - FIELD=64bit ENDOMORPHISM=yes ASM=x86_64 + - FIELD=32bit SCHNORR=yes + - FIELD=32bit ENDOMORPHISM=yes + - BIGNUM=no + - BIGNUM=no ENDOMORPHISM=yes SCHNORR=yes RECOVERY=yes + - BIGNUM=no STATICPRECOMPUTATION=no + - BUILD=distcheck + - EXTRAFLAGS=CPPFLAGS=-DDETERMINISTIC + - EXTRAFLAGS=CFLAGS=-O0 +matrix: + fast_finish: true + include: + - compiler: clang + env: HOST=i686-linux-gnu ENDOMORPHISM=yes + addons: + apt: + packages: + - gcc-multilib + - libgmp-dev:i386 + - compiler: clang + env: HOST=i686-linux-gnu + addons: + apt: + packages: + - gcc-multilib + - compiler: gcc + env: HOST=i686-linux-gnu ENDOMORPHISM=yes + addons: + apt: + packages: + - gcc-multilib + - compiler: gcc + env: HOST=i686-linux-gnu + addons: + apt: + packages: + - gcc-multilib + - libgmp-dev:i386 +before_script: ./autogen.sh +script: + - if [ -n "$HOST" ]; then export USE_HOST="--host=$HOST"; fi + - if [ "x$HOST" = "xi686-linux-gnu" ]; then export CC="$CC -m32"; fi + - ./configure --enable-endomorphism=$ENDOMORPHISM --with-field=$FIELD --with-bignum=$BIGNUM --with-scalar=$SCALAR --enable-ecmult-static-precomputation=$STATICPRECOMPUTATION --enable-module-ecdh=$ECDH --enable-module-schnorr=$SCHNORR --enable-module-recovery=$RECOVERY $EXTRAFLAGS $USE_HOST && make -j2 $BUILD +os: linux diff --git a/src/secp256k1/COPYING b/src/secp256k1/COPYING new file mode 100644 index 00000000..4522a599 --- /dev/null +++ b/src/secp256k1/COPYING @@ -0,0 +1,19 @@ +Copyright (c) 2013 Pieter Wuille + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. diff --git a/src/secp256k1/Makefile.am b/src/secp256k1/Makefile.am new file mode 100644 index 00000000..93801cb4 --- /dev/null +++ b/src/secp256k1/Makefile.am @@ -0,0 +1,115 @@ +ACLOCAL_AMFLAGS = -I build-aux/m4 + +CC_FOR_BUILD = gcc + +lib_LTLIBRARIES = libsecp256k1.la +include_HEADERS = include/secp256k1.h +noinst_HEADERS = +noinst_HEADERS += src/scalar.h +noinst_HEADERS += src/scalar_4x64.h +noinst_HEADERS += src/scalar_8x32.h +noinst_HEADERS += src/scalar_impl.h +noinst_HEADERS += src/scalar_4x64_impl.h +noinst_HEADERS += src/scalar_8x32_impl.h +noinst_HEADERS += src/group.h +noinst_HEADERS += src/group_impl.h +noinst_HEADERS += src/num_gmp.h +noinst_HEADERS += src/num_gmp_impl.h +noinst_HEADERS += src/ecdsa.h +noinst_HEADERS += src/ecdsa_impl.h +noinst_HEADERS += src/eckey.h +noinst_HEADERS += src/eckey_impl.h +noinst_HEADERS += src/ecmult.h +noinst_HEADERS += src/ecmult_impl.h +noinst_HEADERS += src/ecmult_const.h +noinst_HEADERS += src/ecmult_const_impl.h +noinst_HEADERS += src/ecmult_gen.h +noinst_HEADERS += src/ecmult_gen_impl.h +noinst_HEADERS += src/num.h +noinst_HEADERS += src/num_impl.h +noinst_HEADERS += src/field_10x26.h +noinst_HEADERS += src/field_10x26_impl.h +noinst_HEADERS += src/field_5x52.h +noinst_HEADERS += src/field_5x52_impl.h +noinst_HEADERS += src/field_5x52_int128_impl.h +noinst_HEADERS += src/field_5x52_asm_impl.h +noinst_HEADERS += src/java/org_bitcoin_NativeSecp256k1.h +noinst_HEADERS += src/util.h +noinst_HEADERS += src/testrand.h +noinst_HEADERS += src/testrand_impl.h +noinst_HEADERS += src/hash.h +noinst_HEADERS += src/hash_impl.h +noinst_HEADERS += src/field.h +noinst_HEADERS += src/field_impl.h +noinst_HEADERS += src/bench.h +noinst_HEADERS += contrib/lax_der_parsing.h +noinst_HEADERS += contrib/lax_der_parsing.c +noinst_HEADERS += contrib/lax_der_privatekey_parsing.h +noinst_HEADERS += contrib/lax_der_privatekey_parsing.c + +pkgconfigdir = $(libdir)/pkgconfig +pkgconfig_DATA = libsecp256k1.pc + +libsecp256k1_la_SOURCES = src/secp256k1.c +libsecp256k1_la_CPPFLAGS = -I$(top_srcdir)/include -I$(top_srcdir)/src $(SECP_INCLUDES) +libsecp256k1_la_LIBADD = $(SECP_LIBS) + + +noinst_PROGRAMS = +if USE_BENCHMARK +noinst_PROGRAMS += bench_verify bench_sign bench_internal +bench_verify_SOURCES = src/bench_verify.c +bench_verify_LDADD = libsecp256k1.la $(SECP_LIBS) +bench_sign_SOURCES = src/bench_sign.c +bench_sign_LDADD = libsecp256k1.la $(SECP_LIBS) +bench_internal_SOURCES = src/bench_internal.c +bench_internal_LDADD = $(SECP_LIBS) +bench_internal_CPPFLAGS = $(SECP_INCLUDES) +endif + +if USE_TESTS +noinst_PROGRAMS += tests +tests_SOURCES = src/tests.c +tests_CPPFLAGS = -DVERIFY -I$(top_srcdir)/src -I$(top_srcdir)/include $(SECP_INCLUDES) $(SECP_TEST_INCLUDES) +tests_LDADD = $(SECP_LIBS) $(SECP_TEST_LIBS) +tests_LDFLAGS = -static +TESTS = tests +endif + +if USE_ECMULT_STATIC_PRECOMPUTATION +#CPPFLAGS_FOR_BUILD +=-I$(top_srcdir) +#CFLAGS_FOR_BUILD += -Wall -Wextra -Wno-unused-function +CPPFLAGS_FOR_BUILD =-I$(top_srcdir) +CFLAGS_FOR_BUILD = -O2 -Wall -Wextra -Wno-unused-function + +gen_context_OBJECTS = gen_context.o +gen_context_BIN = gen_context$(BUILD_EXEEXT) +gen_%.o: src/gen_%.c + $(CC_FOR_BUILD) $(CPPFLAGS_FOR_BUILD) $(CFLAGS_FOR_BUILD) -c $< -o $@ + +$(gen_context_BIN): $(gen_context_OBJECTS) + $(CC_FOR_BUILD) $^ -o $@ + +$(libsecp256k1_la_OBJECTS): src/ecmult_static_context.h +$(tests_OBJECTS): src/ecmult_static_context.h +$(bench_internal_OBJECTS): src/ecmult_static_context.h + +src/ecmult_static_context.h: $(gen_context_BIN) + ./$(gen_context_BIN) + +CLEANFILES = $(gen_context_BIN) src/ecmult_static_context.h +endif + +EXTRA_DIST = autogen.sh src/gen_context.c src/basic-config.h + +if ENABLE_MODULE_ECDH +include src/modules/ecdh/Makefile.am.include +endif + +if ENABLE_MODULE_SCHNORR +include src/modules/schnorr/Makefile.am.include +endif + +if ENABLE_MODULE_RECOVERY +include src/modules/recovery/Makefile.am.include +endif diff --git a/src/secp256k1/README.md b/src/secp256k1/README.md new file mode 100644 index 00000000..6095db42 --- /dev/null +++ b/src/secp256k1/README.md @@ -0,0 +1,61 @@ +libsecp256k1 +============ + +[![Build Status](https://travis-ci.org/bitcoin/secp256k1.svg?branch=master)](https://travis-ci.org/bitcoin/secp256k1) + +Optimized C library for EC operations on curve secp256k1. + +This library is a work in progress and is being used to research best practices. Use at your own risk. + +Features: +* secp256k1 ECDSA signing/verification and key generation. +* Adding/multiplying private/public keys. +* Serialization/parsing of private keys, public keys, signatures. +* Constant time, constant memory access signing and pubkey generation. +* Derandomized DSA (via RFC6979 or with a caller provided function.) +* Very efficient implementation. + +Implementation details +---------------------- + +* General + * No runtime heap allocation. + * Extensive testing infrastructure. + * Structured to facilitate review and analysis. + * Intended to be portable to any system with a C89 compiler and uint64_t support. + * Expose only higher level interfaces to minimize the API surface and improve application security. ("Be difficult to use insecurely.") +* Field operations + * Optimized implementation of arithmetic modulo the curve's field size (2^256 - 0x1000003D1). + * Using 5 52-bit limbs (including hand-optimized assembly for x86_64, by Diederik Huys). + * Using 10 26-bit limbs. + * Field inverses and square roots using a sliding window over blocks of 1s (by Peter Dettman). +* Scalar operations + * Optimized implementation without data-dependent branches of arithmetic modulo the curve's order. + * Using 4 64-bit limbs (relying on __int128 support in the compiler). + * Using 8 32-bit limbs. +* Group operations + * Point addition formula specifically simplified for the curve equation (y^2 = x^3 + 7). + * Use addition between points in Jacobian and affine coordinates where possible. + * Use a unified addition/doubling formula where necessary to avoid data-dependent branches. + * Point/x comparison without a field inversion by comparison in the Jacobian coordinate space. +* Point multiplication for verification (a*P + b*G). + * Use wNAF notation for point multiplicands. + * Use a much larger window for multiples of G, using precomputed multiples. + * Use Shamir's trick to do the multiplication with the public key and the generator simultaneously. + * Optionally (off by default) use secp256k1's efficiently-computable endomorphism to split the P multiplicand into 2 half-sized ones. +* Point multiplication for signing + * Use a precomputed table of multiples of powers of 16 multiplied with the generator, so general multiplication becomes a series of additions. + * Access the table with branch-free conditional moves so memory access is uniform. + * No data-dependent branches + * The precomputed tables add and eventually subtract points for which no known scalar (private key) is known, preventing even an attacker with control over the private key used to control the data internally. + +Build steps +----------- + +libsecp256k1 is built using autotools: + + $ ./autogen.sh + $ ./configure + $ make + $ ./tests + $ sudo make install # optional diff --git a/src/secp256k1/TODO b/src/secp256k1/TODO new file mode 100644 index 00000000..a300e1c5 --- /dev/null +++ b/src/secp256k1/TODO @@ -0,0 +1,3 @@ +* Unit tests for fieldelem/groupelem, including ones intended to + trigger fieldelem's boundary cases. +* Complete constant-time operations for signing/keygen diff --git a/src/secp256k1/autogen.sh b/src/secp256k1/autogen.sh new file mode 100755 index 00000000..65286b93 --- /dev/null +++ b/src/secp256k1/autogen.sh @@ -0,0 +1,3 @@ +#!/bin/sh +set -e +autoreconf -if --warnings=all diff --git a/src/secp256k1/configure.ac b/src/secp256k1/configure.ac new file mode 100644 index 00000000..786d8dcf --- /dev/null +++ b/src/secp256k1/configure.ac @@ -0,0 +1,376 @@ +AC_PREREQ([2.60]) +AC_INIT([libsecp256k1],[0.1]) +AC_CONFIG_AUX_DIR([build-aux]) +AC_CONFIG_MACRO_DIR([build-aux/m4]) +AC_CANONICAL_HOST +AH_TOP([#ifndef LIBSECP256K1_CONFIG_H]) +AH_TOP([#define LIBSECP256K1_CONFIG_H]) +AH_BOTTOM([#endif /*LIBSECP256K1_CONFIG_H*/]) +AM_INIT_AUTOMAKE([foreign subdir-objects]) +LT_INIT + +dnl make the compilation flags quiet unless V=1 is used +m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) + +PKG_PROG_PKG_CONFIG + +AC_PATH_TOOL(AR, ar) +AC_PATH_TOOL(RANLIB, ranlib) +AC_PATH_TOOL(STRIP, strip) +AX_PROG_CC_FOR_BUILD + +if test "x$CFLAGS" = "x"; then + CFLAGS="-O3 -g" +fi + +AM_PROG_CC_C_O + +AC_PROG_CC_C89 +if test x"$ac_cv_prog_cc_c89" = x"no"; then + AC_MSG_ERROR([c89 compiler support required]) +fi + +case $host_os in + *darwin*) + if test x$cross_compiling != xyes; then + AC_PATH_PROG([BREW],brew,) + if test x$BREW != x; then + dnl These Homebrew packages may be keg-only, meaning that they won't be found + dnl in expected paths because they may conflict with system files. Ask + dnl Homebrew where each one is located, then adjust paths accordingly. + + openssl_prefix=`$BREW --prefix openssl 2>/dev/null` + gmp_prefix=`$BREW --prefix gmp 2>/dev/null` + if test x$openssl_prefix != x; then + PKG_CONFIG_PATH="$openssl_prefix/lib/pkgconfig:$PKG_CONFIG_PATH" + export PKG_CONFIG_PATH + fi + if test x$gmp_prefix != x; then + GMP_CPPFLAGS="-I$gmp_prefix/include" + GMP_LIBS="-L$gmp_prefix/lib" + fi + else + AC_PATH_PROG([PORT],port,) + dnl if homebrew isn't installed and macports is, add the macports default paths + dnl as a last resort. + if test x$PORT != x; then + CPPFLAGS="$CPPFLAGS -isystem /opt/local/include" + LDFLAGS="$LDFLAGS -L/opt/local/lib" + fi + fi + fi + ;; +esac + +CFLAGS="$CFLAGS -W" + +warn_CFLAGS="-std=c89 -pedantic -Wall -Wextra -Wcast-align -Wnested-externs -Wshadow -Wstrict-prototypes -Wno-unused-function -Wno-long-long -Wno-overlength-strings" +saved_CFLAGS="$CFLAGS" +CFLAGS="$CFLAGS $warn_CFLAGS" +AC_MSG_CHECKING([if ${CC} supports ${warn_CFLAGS}]) +AC_COMPILE_IFELSE([AC_LANG_SOURCE([[char foo;]])], + [ AC_MSG_RESULT([yes]) ], + [ AC_MSG_RESULT([no]) + CFLAGS="$saved_CFLAGS" + ]) + +saved_CFLAGS="$CFLAGS" +CFLAGS="$CFLAGS -fvisibility=hidden" +AC_MSG_CHECKING([if ${CC} supports -fvisibility=hidden]) +AC_COMPILE_IFELSE([AC_LANG_SOURCE([[char foo;]])], + [ AC_MSG_RESULT([yes]) ], + [ AC_MSG_RESULT([no]) + CFLAGS="$saved_CFLAGS" + ]) + +AC_ARG_ENABLE(benchmark, + AS_HELP_STRING([--enable-benchmark],[compile benchmark (default is no)]), + [use_benchmark=$enableval], + [use_benchmark=no]) + +AC_ARG_ENABLE(tests, + AS_HELP_STRING([--enable-tests],[compile tests (default is yes)]), + [use_tests=$enableval], + [use_tests=yes]) + +AC_ARG_ENABLE(endomorphism, + AS_HELP_STRING([--enable-endomorphism],[enable endomorphism (default is no)]), + [use_endomorphism=$enableval], + [use_endomorphism=no]) + +AC_ARG_ENABLE(ecmult_static_precomputation, + AS_HELP_STRING([--enable-ecmult-static-precomputation],[enable precomputed ecmult table for signing (default is yes)]), + [use_ecmult_static_precomputation=$enableval], + [use_ecmult_static_precomputation=yes]) + +AC_ARG_ENABLE(module_ecdh, + AS_HELP_STRING([--enable-module-ecdh],[enable ECDH shared secret computation (default is no)]), + [enable_module_ecdh=$enableval], + [enable_module_ecdh=no]) + +AC_ARG_ENABLE(module_schnorr, + AS_HELP_STRING([--enable-module-schnorr],[enable Schnorr signature module (default is no)]), + [enable_module_schnorr=$enableval], + [enable_module_schnorr=no]) + +AC_ARG_ENABLE(module_recovery, + AS_HELP_STRING([--enable-module-recovery],[enable ECDSA pubkey recovery module (default is no)]), + [enable_module_recovery=$enableval], + [enable_module_recovery=no]) + +AC_ARG_WITH([field], [AS_HELP_STRING([--with-field=64bit|32bit|auto], +[Specify Field Implementation. Default is auto])],[req_field=$withval], [req_field=auto]) + +AC_ARG_WITH([bignum], [AS_HELP_STRING([--with-bignum=gmp|no|auto], +[Specify Bignum Implementation. Default is auto])],[req_bignum=$withval], [req_bignum=auto]) + +AC_ARG_WITH([scalar], [AS_HELP_STRING([--with-scalar=64bit|32bit|auto], +[Specify scalar implementation. Default is auto])],[req_scalar=$withval], [req_scalar=auto]) + +AC_ARG_WITH([asm], [AS_HELP_STRING([--with-asm=x86_64|no|auto] +[Specify assembly optimizations to use. Default is auto])],[req_asm=$withval], [req_asm=auto]) + +AC_CHECK_TYPES([__int128]) + +AC_MSG_CHECKING([for __builtin_expect]) +AC_COMPILE_IFELSE([AC_LANG_SOURCE([[void myfunc() {__builtin_expect(0,0);}]])], + [ AC_MSG_RESULT([yes]);AC_DEFINE(HAVE_BUILTIN_EXPECT,1,[Define this symbol if __builtin_expect is available]) ], + [ AC_MSG_RESULT([no]) + ]) + +if test x"$req_asm" = x"auto"; then + SECP_64BIT_ASM_CHECK + if test x"$has_64bit_asm" = x"yes"; then + set_asm=x86_64 + fi + if test x"$set_asm" = x; then + set_asm=no + fi +else + set_asm=$req_asm + case $set_asm in + x86_64) + SECP_64BIT_ASM_CHECK + if test x"$has_64bit_asm" != x"yes"; then + AC_MSG_ERROR([x86_64 assembly optimization requested but not available]) + fi + ;; + no) + ;; + *) + AC_MSG_ERROR([invalid assembly optimization selection]) + ;; + esac +fi + +if test x"$req_field" = x"auto"; then + if test x"set_asm" = x"x86_64"; then + set_field=64bit + fi + if test x"$set_field" = x; then + SECP_INT128_CHECK + if test x"$has_int128" = x"yes"; then + set_field=64bit + fi + fi + if test x"$set_field" = x; then + set_field=32bit + fi +else + set_field=$req_field + case $set_field in + 64bit) + if test x"$set_asm" != x"x86_64"; then + SECP_INT128_CHECK + if test x"$has_int128" != x"yes"; then + AC_MSG_ERROR([64bit field explicitly requested but neither __int128 support or x86_64 assembly available]) + fi + fi + ;; + 32bit) + ;; + *) + AC_MSG_ERROR([invalid field implementation selection]) + ;; + esac +fi + +if test x"$req_scalar" = x"auto"; then + SECP_INT128_CHECK + if test x"$has_int128" = x"yes"; then + set_scalar=64bit + fi + if test x"$set_scalar" = x; then + set_scalar=32bit + fi +else + set_scalar=$req_scalar + case $set_scalar in + 64bit) + SECP_INT128_CHECK + if test x"$has_int128" != x"yes"; then + AC_MSG_ERROR([64bit scalar explicitly requested but __int128 support not available]) + fi + ;; + 32bit) + ;; + *) + AC_MSG_ERROR([invalid scalar implementation selected]) + ;; + esac +fi + +if test x"$req_bignum" = x"auto"; then + SECP_GMP_CHECK + if test x"$has_gmp" = x"yes"; then + set_bignum=gmp + fi + + if test x"$set_bignum" = x; then + set_bignum=no + fi +else + set_bignum=$req_bignum + case $set_bignum in + gmp) + SECP_GMP_CHECK + if test x"$has_gmp" != x"yes"; then + AC_MSG_ERROR([gmp bignum explicitly requested but libgmp not available]) + fi + ;; + no) + ;; + *) + AC_MSG_ERROR([invalid bignum implementation selection]) + ;; + esac +fi + +# select assembly optimization +case $set_asm in +x86_64) + AC_DEFINE(USE_ASM_X86_64, 1, [Define this symbol to enable x86_64 assembly optimizations]) + ;; +no) + ;; +*) + AC_MSG_ERROR([invalid assembly optimizations]) + ;; +esac + +# select field implementation +case $set_field in +64bit) + AC_DEFINE(USE_FIELD_5X52, 1, [Define this symbol to use the FIELD_5X52 implementation]) + ;; +32bit) + AC_DEFINE(USE_FIELD_10X26, 1, [Define this symbol to use the FIELD_10X26 implementation]) + ;; +*) + AC_MSG_ERROR([invalid field implementation]) + ;; +esac + +# select bignum implementation +case $set_bignum in +gmp) + AC_DEFINE(HAVE_LIBGMP, 1, [Define this symbol if libgmp is installed]) + AC_DEFINE(USE_NUM_GMP, 1, [Define this symbol to use the gmp implementation for num]) + AC_DEFINE(USE_FIELD_INV_NUM, 1, [Define this symbol to use the num-based field inverse implementation]) + AC_DEFINE(USE_SCALAR_INV_NUM, 1, [Define this symbol to use the num-based scalar inverse implementation]) + ;; +no) + AC_DEFINE(USE_NUM_NONE, 1, [Define this symbol to use no num implementation]) + AC_DEFINE(USE_FIELD_INV_BUILTIN, 1, [Define this symbol to use the native field inverse implementation]) + AC_DEFINE(USE_SCALAR_INV_BUILTIN, 1, [Define this symbol to use the native scalar inverse implementation]) + ;; +*) + AC_MSG_ERROR([invalid bignum implementation]) + ;; +esac + +#select scalar implementation +case $set_scalar in +64bit) + AC_DEFINE(USE_SCALAR_4X64, 1, [Define this symbol to use the 4x64 scalar implementation]) + ;; +32bit) + AC_DEFINE(USE_SCALAR_8X32, 1, [Define this symbol to use the 8x32 scalar implementation]) + ;; +*) + AC_MSG_ERROR([invalid scalar implementation]) + ;; +esac + +if test x"$use_tests" = x"yes"; then + SECP_OPENSSL_CHECK + if test x"$has_openssl_ec" = x"yes"; then + AC_DEFINE(ENABLE_OPENSSL_TESTS, 1, [Define this symbol if OpenSSL EC functions are available]) + SECP_TEST_INCLUDES="$SSL_CFLAGS $CRYPTO_CFLAGS" + SECP_TEST_LIBS="$CRYPTO_LIBS" + + case $host in + *mingw*) + SECP_TEST_LIBS="$SECP_TEST_LIBS -lgdi32" + ;; + esac + + fi +fi + +if test x"$set_bignum" = x"gmp"; then + SECP_LIBS="$SECP_LIBS $GMP_LIBS" + SECP_INCLUDES="$SECP_INCLUDES $GMP_CPPFLAGS" +fi + +if test x"$use_endomorphism" = x"yes"; then + AC_DEFINE(USE_ENDOMORPHISM, 1, [Define this symbol to use endomorphism optimization]) +fi + +if test x"$use_ecmult_static_precomputation" = x"yes"; then + AC_DEFINE(USE_ECMULT_STATIC_PRECOMPUTATION, 1, [Define this symbol to use a statically generated ecmult table]) +fi + +if test x"$enable_module_ecdh" = x"yes"; then + AC_DEFINE(ENABLE_MODULE_ECDH, 1, [Define this symbol to enable the ECDH module]) +fi + +if test x"$enable_module_schnorr" = x"yes"; then + AC_DEFINE(ENABLE_MODULE_SCHNORR, 1, [Define this symbol to enable the Schnorr signature module]) +fi + +if test x"$enable_module_recovery" = x"yes"; then + AC_DEFINE(ENABLE_MODULE_RECOVERY, 1, [Define this symbol to enable the ECDSA pubkey recovery module]) +fi + +AC_C_BIGENDIAN() + +AC_MSG_NOTICE([Using assembly optimizations: $set_asm]) +AC_MSG_NOTICE([Using field implementation: $set_field]) +AC_MSG_NOTICE([Using bignum implementation: $set_bignum]) +AC_MSG_NOTICE([Using scalar implementation: $set_scalar]) +AC_MSG_NOTICE([Using endomorphism optimizations: $use_endomorphism]) +AC_MSG_NOTICE([Building ECDH module: $enable_module_ecdh]) + +AC_MSG_NOTICE([Building Schnorr signatures module: $enable_module_schnorr]) +AC_MSG_NOTICE([Building ECDSA pubkey recovery module: $enable_module_recovery]) + +AC_CONFIG_HEADERS([src/libsecp256k1-config.h]) +AC_CONFIG_FILES([Makefile libsecp256k1.pc]) +AC_SUBST(SECP_INCLUDES) +AC_SUBST(SECP_LIBS) +AC_SUBST(SECP_TEST_LIBS) +AC_SUBST(SECP_TEST_INCLUDES) +AM_CONDITIONAL([USE_TESTS], [test x"$use_tests" != x"no"]) +AM_CONDITIONAL([USE_BENCHMARK], [test x"$use_benchmark" = x"yes"]) +AM_CONDITIONAL([USE_ECMULT_STATIC_PRECOMPUTATION], [test x"$use_ecmult_static_precomputation" = x"yes"]) +AM_CONDITIONAL([ENABLE_MODULE_ECDH], [test x"$enable_module_ecdh" = x"yes"]) +AM_CONDITIONAL([ENABLE_MODULE_SCHNORR], [test x"$enable_module_schnorr" = x"yes"]) +AM_CONDITIONAL([ENABLE_MODULE_RECOVERY], [test x"$enable_module_recovery" = x"yes"]) + +dnl make sure nothing new is exported so that we don't break the cache +PKGCONFIG_PATH_TEMP="$PKG_CONFIG_PATH" +unset PKG_CONFIG_PATH +PKG_CONFIG_PATH="$PKGCONFIG_PATH_TEMP" + +AC_OUTPUT diff --git a/src/secp256k1/contrib/lax_der_parsing.c b/src/secp256k1/contrib/lax_der_parsing.c new file mode 100644 index 00000000..5b141a99 --- /dev/null +++ b/src/secp256k1/contrib/lax_der_parsing.c @@ -0,0 +1,150 @@ +/********************************************************************** + * Copyright (c) 2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#include +#include + +#include "lax_der_parsing.h" + +int ecdsa_signature_parse_der_lax(const secp256k1_context* ctx, secp256k1_ecdsa_signature* sig, const unsigned char *input, size_t inputlen) { + size_t rpos, rlen, spos, slen; + size_t pos = 0; + size_t lenbyte; + unsigned char tmpsig[64] = {0}; + int overflow = 0; + + /* Hack to initialize sig with a correctly-parsed but invalid signature. */ + secp256k1_ecdsa_signature_parse_compact(ctx, sig, tmpsig); + + /* Sequence tag byte */ + if (pos == inputlen || input[pos] != 0x30) { + return 0; + } + pos++; + + /* Sequence length bytes */ + if (pos == inputlen) { + return 0; + } + lenbyte = input[pos++]; + if (lenbyte & 0x80) { + lenbyte -= 0x80; + if (pos + lenbyte > inputlen) { + return 0; + } + pos += lenbyte; + } + + /* Integer tag byte for R */ + if (pos == inputlen || input[pos] != 0x02) { + return 0; + } + pos++; + + /* Integer length for R */ + if (pos == inputlen) { + return 0; + } + lenbyte = input[pos++]; + if (lenbyte & 0x80) { + lenbyte -= 0x80; + if (pos + lenbyte > inputlen) { + return 0; + } + while (lenbyte > 0 && input[pos] == 0) { + pos++; + lenbyte--; + } + if (lenbyte >= sizeof(size_t)) { + return 0; + } + rlen = 0; + while (lenbyte > 0) { + rlen = (rlen << 8) + input[pos]; + pos++; + lenbyte--; + } + } else { + rlen = lenbyte; + } + if (rlen > inputlen - pos) { + return 0; + } + rpos = pos; + pos += rlen; + + /* Integer tag byte for S */ + if (pos == inputlen || input[pos] != 0x02) { + return 0; + } + pos++; + + /* Integer length for S */ + if (pos == inputlen) { + return 0; + } + lenbyte = input[pos++]; + if (lenbyte & 0x80) { + lenbyte -= 0x80; + if (pos + lenbyte > inputlen) { + return 0; + } + while (lenbyte > 0 && input[pos] == 0) { + pos++; + lenbyte--; + } + if (lenbyte >= sizeof(size_t)) { + return 0; + } + slen = 0; + while (lenbyte > 0) { + slen = (slen << 8) + input[pos]; + pos++; + lenbyte--; + } + } else { + slen = lenbyte; + } + if (slen > inputlen - pos) { + return 0; + } + spos = pos; + pos += slen; + + /* Ignore leading zeroes in R */ + while (rlen > 0 && input[rpos] == 0) { + rlen--; + rpos++; + } + /* Copy R value */ + if (rlen > 32) { + overflow = 1; + } else { + memcpy(tmpsig + 32 - rlen, input + rpos, rlen); + } + + /* Ignore leading zeroes in S */ + while (slen > 0 && input[spos] == 0) { + slen--; + spos++; + } + /* Copy S value */ + if (slen > 32) { + overflow = 1; + } else { + memcpy(tmpsig + 64 - slen, input + spos, slen); + } + + if (!overflow) { + overflow = !secp256k1_ecdsa_signature_parse_compact(ctx, sig, tmpsig); + } + if (overflow) { + memset(tmpsig, 0, 64); + secp256k1_ecdsa_signature_parse_compact(ctx, sig, tmpsig); + } + return 1; +} + diff --git a/src/secp256k1/contrib/lax_der_parsing.h b/src/secp256k1/contrib/lax_der_parsing.h new file mode 100644 index 00000000..6d27871a --- /dev/null +++ b/src/secp256k1/contrib/lax_der_parsing.h @@ -0,0 +1,91 @@ +/********************************************************************** + * Copyright (c) 2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +/**** + * Please do not link this file directly. It is not part of the libsecp256k1 + * project and does not promise any stability in its API, functionality or + * presence. Projects which use this code should instead copy this header + * and its accompanying .c file directly into their codebase. + ****/ + +/* This file defines a function that parses DER with various errors and + * violations. This is not a part of the library itself, because the allowed + * violations are chosen arbitrarily and do not follow or establish any + * standard. + * + * In many places it matters that different implementations do not only accept + * the same set of valid signatures, but also reject the same set of signatures. + * The only means to accomplish that is by strictly obeying a standard, and not + * accepting anything else. + * + * Nonetheless, sometimes there is a need for compatibility with systems that + * use signatures which do not strictly obey DER. The snippet below shows how + * certain violations are easily supported. You may need to adapt it. + * + * Do not use this for new systems. Use well-defined DER or compact signatures + * instead if you have the choice (see secp256k1_ecdsa_signature_parse_der and + * secp256k1_ecdsa_signature_parse_compact). + * + * The supported violations are: + * - All numbers are parsed as nonnegative integers, even though X.609-0207 + * section 8.3.3 specifies that integers are always encoded as two's + * complement. + * - Integers can have length 0, even though section 8.3.1 says they can't. + * - Integers with overly long padding are accepted, violation section + * 8.3.2. + * - 127-byte long length descriptors are accepted, even though section + * 8.1.3.5.c says that they are not. + * - Trailing garbage data inside or after the signature is ignored. + * - The length descriptor of the sequence is ignored. + * + * Compared to for example OpenSSL, many violations are NOT supported: + * - Using overly long tag descriptors for the sequence or integers inside, + * violating section 8.1.2.2. + * - Encoding primitive integers as constructed values, violating section + * 8.3.1. + */ + +#ifndef _SECP256K1_CONTRIB_LAX_DER_PARSING_H_ +#define _SECP256K1_CONTRIB_LAX_DER_PARSING_H_ + +#include + +# ifdef __cplusplus +extern "C" { +# endif + +/** Parse a signature in "lax DER" format + * + * Returns: 1 when the signature could be parsed, 0 otherwise. + * Args: ctx: a secp256k1 context object + * Out: sig: a pointer to a signature object + * In: input: a pointer to the signature to be parsed + * inputlen: the length of the array pointed to be input + * + * This function will accept any valid DER encoded signature, even if the + * encoded numbers are out of range. In addition, it will accept signatures + * which violate the DER spec in various ways. Its purpose is to allow + * validation of the Bitcoin blockchain, which includes non-DER signatures + * from before the network rules were updated to enforce DER. Note that + * the set of supported violations is a strict subset of what OpenSSL will + * accept. + * + * After the call, sig will always be initialized. If parsing failed or the + * encoded numbers are out of range, signature validation with it is + * guaranteed to fail for every message and public key. + */ +int ecdsa_signature_parse_der_lax( + const secp256k1_context* ctx, + secp256k1_ecdsa_signature* sig, + const unsigned char *input, + size_t inputlen +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/secp256k1/contrib/lax_der_privatekey_parsing.c b/src/secp256k1/contrib/lax_der_privatekey_parsing.c new file mode 100644 index 00000000..c2e63b4b --- /dev/null +++ b/src/secp256k1/contrib/lax_der_privatekey_parsing.c @@ -0,0 +1,113 @@ +/********************************************************************** + * Copyright (c) 2014, 2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#include +#include + +#include "lax_der_privatekey_parsing.h" + +int ec_privkey_import_der(const secp256k1_context* ctx, unsigned char *out32, const unsigned char *privkey, size_t privkeylen) { + const unsigned char *end = privkey + privkeylen; + int lenb = 0; + int len = 0; + memset(out32, 0, 32); + /* sequence header */ + if (end < privkey+1 || *privkey != 0x30) { + return 0; + } + privkey++; + /* sequence length constructor */ + if (end < privkey+1 || !(*privkey & 0x80)) { + return 0; + } + lenb = *privkey & ~0x80; privkey++; + if (lenb < 1 || lenb > 2) { + return 0; + } + if (end < privkey+lenb) { + return 0; + } + /* sequence length */ + len = privkey[lenb-1] | (lenb > 1 ? privkey[lenb-2] << 8 : 0); + privkey += lenb; + if (end < privkey+len) { + return 0; + } + /* sequence element 0: version number (=1) */ + if (end < privkey+3 || privkey[0] != 0x02 || privkey[1] != 0x01 || privkey[2] != 0x01) { + return 0; + } + privkey += 3; + /* sequence element 1: octet string, up to 32 bytes */ + if (end < privkey+2 || privkey[0] != 0x04 || privkey[1] > 0x20 || end < privkey+2+privkey[1]) { + return 0; + } + memcpy(out32 + 32 - privkey[1], privkey + 2, privkey[1]); + if (!secp256k1_ec_seckey_verify(ctx, out32)) { + memset(out32, 0, 32); + return 0; + } + return 1; +} + +int ec_privkey_export_der(const secp256k1_context *ctx, unsigned char *privkey, size_t *privkeylen, const unsigned char *key32, int compressed) { + secp256k1_pubkey pubkey; + size_t pubkeylen = 0; + if (!secp256k1_ec_pubkey_create(ctx, &pubkey, key32)) { + *privkeylen = 0; + return 0; + } + if (compressed) { + static const unsigned char begin[] = { + 0x30,0x81,0xD3,0x02,0x01,0x01,0x04,0x20 + }; + static const unsigned char middle[] = { + 0xA0,0x81,0x85,0x30,0x81,0x82,0x02,0x01,0x01,0x30,0x2C,0x06,0x07,0x2A,0x86,0x48, + 0xCE,0x3D,0x01,0x01,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFE,0xFF,0xFF,0xFC,0x2F,0x30,0x06,0x04,0x01,0x00,0x04,0x01,0x07,0x04, + 0x21,0x02,0x79,0xBE,0x66,0x7E,0xF9,0xDC,0xBB,0xAC,0x55,0xA0,0x62,0x95,0xCE,0x87, + 0x0B,0x07,0x02,0x9B,0xFC,0xDB,0x2D,0xCE,0x28,0xD9,0x59,0xF2,0x81,0x5B,0x16,0xF8, + 0x17,0x98,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFE,0xBA,0xAE,0xDC,0xE6,0xAF,0x48,0xA0,0x3B,0xBF,0xD2,0x5E, + 0x8C,0xD0,0x36,0x41,0x41,0x02,0x01,0x01,0xA1,0x24,0x03,0x22,0x00 + }; + unsigned char *ptr = privkey; + memcpy(ptr, begin, sizeof(begin)); ptr += sizeof(begin); + memcpy(ptr, key32, 32); ptr += 32; + memcpy(ptr, middle, sizeof(middle)); ptr += sizeof(middle); + pubkeylen = 33; + secp256k1_ec_pubkey_serialize(ctx, ptr, &pubkeylen, &pubkey, SECP256K1_EC_COMPRESSED); + ptr += pubkeylen; + *privkeylen = ptr - privkey; + } else { + static const unsigned char begin[] = { + 0x30,0x82,0x01,0x13,0x02,0x01,0x01,0x04,0x20 + }; + static const unsigned char middle[] = { + 0xA0,0x81,0xA5,0x30,0x81,0xA2,0x02,0x01,0x01,0x30,0x2C,0x06,0x07,0x2A,0x86,0x48, + 0xCE,0x3D,0x01,0x01,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFE,0xFF,0xFF,0xFC,0x2F,0x30,0x06,0x04,0x01,0x00,0x04,0x01,0x07,0x04, + 0x41,0x04,0x79,0xBE,0x66,0x7E,0xF9,0xDC,0xBB,0xAC,0x55,0xA0,0x62,0x95,0xCE,0x87, + 0x0B,0x07,0x02,0x9B,0xFC,0xDB,0x2D,0xCE,0x28,0xD9,0x59,0xF2,0x81,0x5B,0x16,0xF8, + 0x17,0x98,0x48,0x3A,0xDA,0x77,0x26,0xA3,0xC4,0x65,0x5D,0xA4,0xFB,0xFC,0x0E,0x11, + 0x08,0xA8,0xFD,0x17,0xB4,0x48,0xA6,0x85,0x54,0x19,0x9C,0x47,0xD0,0x8F,0xFB,0x10, + 0xD4,0xB8,0x02,0x21,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFE,0xBA,0xAE,0xDC,0xE6,0xAF,0x48,0xA0,0x3B,0xBF,0xD2,0x5E, + 0x8C,0xD0,0x36,0x41,0x41,0x02,0x01,0x01,0xA1,0x44,0x03,0x42,0x00 + }; + unsigned char *ptr = privkey; + memcpy(ptr, begin, sizeof(begin)); ptr += sizeof(begin); + memcpy(ptr, key32, 32); ptr += 32; + memcpy(ptr, middle, sizeof(middle)); ptr += sizeof(middle); + pubkeylen = 65; + secp256k1_ec_pubkey_serialize(ctx, ptr, &pubkeylen, &pubkey, SECP256K1_EC_UNCOMPRESSED); + ptr += pubkeylen; + *privkeylen = ptr - privkey; + } + return 1; +} diff --git a/src/secp256k1/contrib/lax_der_privatekey_parsing.h b/src/secp256k1/contrib/lax_der_privatekey_parsing.h new file mode 100644 index 00000000..2fd088f8 --- /dev/null +++ b/src/secp256k1/contrib/lax_der_privatekey_parsing.h @@ -0,0 +1,90 @@ +/********************************************************************** + * Copyright (c) 2014, 2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +/**** + * Please do not link this file directly. It is not part of the libsecp256k1 + * project and does not promise any stability in its API, functionality or + * presence. Projects which use this code should instead copy this header + * and its accompanying .c file directly into their codebase. + ****/ + +/* This file contains code snippets that parse DER private keys with + * various errors and violations. This is not a part of the library + * itself, because the allowed violations are chosen arbitrarily and + * do not follow or establish any standard. + * + * It also contains code to serialize private keys in a compatible + * manner. + * + * These functions are meant for compatibility with applications + * that require BER encoded keys. When working with secp256k1-specific + * code, the simple 32-byte private keys normally used by the + * library are sufficient. + */ + +#ifndef _SECP256K1_CONTRIB_BER_PRIVATEKEY_H_ +#define _SECP256K1_CONTRIB_BER_PRIVATEKEY_H_ + +#include + +# ifdef __cplusplus +extern "C" { +# endif + +/** Export a private key in DER format. + * + * Returns: 1 if the private key was valid. + * Args: ctx: pointer to a context object, initialized for signing (cannot + * be NULL) + * Out: privkey: pointer to an array for storing the private key in BER. + * Should have space for 279 bytes, and cannot be NULL. + * privkeylen: Pointer to an int where the length of the private key in + * privkey will be stored. + * In: seckey: pointer to a 32-byte secret key to export. + * compressed: 1 if the key should be exported in + * compressed format, 0 otherwise + * + * This function is purely meant for compatibility with applications that + * require BER encoded keys. When working with secp256k1-specific code, the + * simple 32-byte private keys are sufficient. + * + * Note that this function does not guarantee correct DER output. It is + * guaranteed to be parsable by secp256k1_ec_privkey_import_der + */ +SECP256K1_WARN_UNUSED_RESULT int ec_privkey_export_der( + const secp256k1_context* ctx, + unsigned char *privkey, + size_t *privkeylen, + const unsigned char *seckey, + int compressed +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +/** Import a private key in DER format. + * Returns: 1 if a private key was extracted. + * Args: ctx: pointer to a context object (cannot be NULL). + * Out: seckey: pointer to a 32-byte array for storing the private key. + * (cannot be NULL). + * In: privkey: pointer to a private key in DER format (cannot be NULL). + * privkeylen: length of the DER private key pointed to be privkey. + * + * This function will accept more than just strict DER, and even allow some BER + * violations. The public key stored inside the DER-encoded private key is not + * verified for correctness, nor are the curve parameters. Use this function + * only if you know in advance it is supposed to contain a secp256k1 private + * key. + */ +SECP256K1_WARN_UNUSED_RESULT int ec_privkey_import_der( + const secp256k1_context* ctx, + unsigned char *seckey, + const unsigned char *privkey, + size_t privkeylen +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/secp256k1/include/secp256k1.h b/src/secp256k1/include/secp256k1.h new file mode 100644 index 00000000..7145dbcc --- /dev/null +++ b/src/secp256k1/include/secp256k1.h @@ -0,0 +1,583 @@ +#ifndef _SECP256K1_ +# define _SECP256K1_ + +# ifdef __cplusplus +extern "C" { +# endif + +#include + +/* These rules specify the order of arguments in API calls: + * + * 1. Context pointers go first, followed by output arguments, combined + * output/input arguments, and finally input-only arguments. + * 2. Array lengths always immediately the follow the argument whose length + * they describe, even if this violates rule 1. + * 3. Within the OUT/OUTIN/IN groups, pointers to data that is typically generated + * later go first. This means: signatures, public nonces, private nonces, + * messages, public keys, secret keys, tweaks. + * 4. Arguments that are not data pointers go last, from more complex to less + * complex: function pointers, algorithm names, messages, void pointers, + * counts, flags, booleans. + * 5. Opaque data pointers follow the function pointer they are to be passed to. + */ + +/** Opaque data structure that holds context information (precomputed tables etc.). + * + * The purpose of context structures is to cache large precomputed data tables + * that are expensive to construct, and also to maintain the randomization data + * for blinding. + * + * Do not create a new context object for each operation, as construction is + * far slower than all other API calls (~100 times slower than an ECDSA + * verification). + * + * A constructed context can safely be used from multiple threads + * simultaneously, but API call that take a non-const pointer to a context + * need exclusive access to it. In particular this is the case for + * secp256k1_context_destroy and secp256k1_context_randomize. + * + * Regarding randomization, either do it once at creation time (in which case + * you do not need any locking for the other calls), or use a read-write lock. + */ +typedef struct secp256k1_context_struct secp256k1_context; + +/** Opaque data structure that holds a parsed and valid public key. + * + * The exact representation of data inside is implementation defined and not + * guaranteed to be portable between different platforms or versions. It is + * however guaranteed to be 64 bytes in size, and can be safely copied/moved. + * If you need to convert to a format suitable for storage or transmission, use + * secp256k1_ec_pubkey_serialize and secp256k1_ec_pubkey_parse. + * + * Furthermore, it is guaranteed that identical public keys (ignoring + * compression) will have identical representation, so they can be memcmp'ed. + */ +typedef struct { + unsigned char data[64]; +} secp256k1_pubkey; + +/** Opaque data structured that holds a parsed ECDSA signature. + * + * The exact representation of data inside is implementation defined and not + * guaranteed to be portable between different platforms or versions. It is + * however guaranteed to be 64 bytes in size, and can be safely copied/moved. + * If you need to convert to a format suitable for storage or transmission, use + * the secp256k1_ecdsa_signature_serialize_* and + * secp256k1_ecdsa_signature_serialize_* functions. + * + * Furthermore, it is guaranteed to identical signatures will have identical + * representation, so they can be memcmp'ed. + */ +typedef struct { + unsigned char data[64]; +} secp256k1_ecdsa_signature; + +/** A pointer to a function to deterministically generate a nonce. + * + * Returns: 1 if a nonce was successfully generated. 0 will cause signing to fail. + * Out: nonce32: pointer to a 32-byte array to be filled by the function. + * In: msg32: the 32-byte message hash being verified (will not be NULL) + * key32: pointer to a 32-byte secret key (will not be NULL) + * algo16: pointer to a 16-byte array describing the signature + * algorithm (will be NULL for ECDSA for compatibility). + * data: Arbitrary data pointer that is passed through. + * attempt: how many iterations we have tried to find a nonce. + * This will almost always be 0, but different attempt values + * are required to result in a different nonce. + * + * Except for test cases, this function should compute some cryptographic hash of + * the message, the algorithm, the key and the attempt. + */ +typedef int (*secp256k1_nonce_function)( + unsigned char *nonce32, + const unsigned char *msg32, + const unsigned char *key32, + const unsigned char *algo16, + void *data, + unsigned int attempt +); + +# if !defined(SECP256K1_GNUC_PREREQ) +# if defined(__GNUC__)&&defined(__GNUC_MINOR__) +# define SECP256K1_GNUC_PREREQ(_maj,_min) \ + ((__GNUC__<<16)+__GNUC_MINOR__>=((_maj)<<16)+(_min)) +# else +# define SECP256K1_GNUC_PREREQ(_maj,_min) 0 +# endif +# endif + +# if (!defined(__STDC_VERSION__) || (__STDC_VERSION__ < 199901L) ) +# if SECP256K1_GNUC_PREREQ(2,7) +# define SECP256K1_INLINE __inline__ +# elif (defined(_MSC_VER)) +# define SECP256K1_INLINE __inline +# else +# define SECP256K1_INLINE +# endif +# else +# define SECP256K1_INLINE inline +# endif + +#ifndef SECP256K1_API +# if defined(_WIN32) +# ifdef SECP256K1_BUILD +# define SECP256K1_API __declspec(dllexport) +# else +# define SECP256K1_API +# endif +# elif defined(__GNUC__) && defined(SECP256K1_BUILD) +# define SECP256K1_API __attribute__ ((visibility ("default"))) +# else +# define SECP256K1_API +# endif +#endif + +/**Warning attributes + * NONNULL is not used if SECP256K1_BUILD is set to avoid the compiler optimizing out + * some paranoid null checks. */ +# if defined(__GNUC__) && SECP256K1_GNUC_PREREQ(3, 4) +# define SECP256K1_WARN_UNUSED_RESULT __attribute__ ((__warn_unused_result__)) +# else +# define SECP256K1_WARN_UNUSED_RESULT +# endif +# if !defined(SECP256K1_BUILD) && defined(__GNUC__) && SECP256K1_GNUC_PREREQ(3, 4) +# define SECP256K1_ARG_NONNULL(_x) __attribute__ ((__nonnull__(_x))) +# else +# define SECP256K1_ARG_NONNULL(_x) +# endif + +/** All flags' lower 8 bits indicate what they're for. Do not use directly. */ +#define SECP256K1_FLAGS_TYPE_MASK ((1 << 8) - 1) +#define SECP256K1_FLAGS_TYPE_CONTEXT (1 << 0) +#define SECP256K1_FLAGS_TYPE_COMPRESSION (1 << 1) +/** The higher bits contain the actual data. Do not use directly. */ +#define SECP256K1_FLAGS_BIT_CONTEXT_VERIFY (1 << 8) +#define SECP256K1_FLAGS_BIT_CONTEXT_SIGN (1 << 9) +#define SECP256K1_FLAGS_BIT_COMPRESSION (1 << 8) + +/** Flags to pass to secp256k1_context_create. */ +#define SECP256K1_CONTEXT_VERIFY (SECP256K1_FLAGS_TYPE_CONTEXT | SECP256K1_FLAGS_BIT_CONTEXT_VERIFY) +#define SECP256K1_CONTEXT_SIGN (SECP256K1_FLAGS_TYPE_CONTEXT | SECP256K1_FLAGS_BIT_CONTEXT_SIGN) +#define SECP256K1_CONTEXT_NONE (SECP256K1_FLAGS_TYPE_CONTEXT) + +/** Flag to pass to secp256k1_ec_pubkey_serialize and secp256k1_ec_privkey_export. */ +#define SECP256K1_EC_COMPRESSED (SECP256K1_FLAGS_TYPE_COMPRESSION | SECP256K1_FLAGS_BIT_COMPRESSION) +#define SECP256K1_EC_UNCOMPRESSED (SECP256K1_FLAGS_TYPE_COMPRESSION) + +/** Create a secp256k1 context object. + * + * Returns: a newly created context object. + * In: flags: which parts of the context to initialize. + */ +SECP256K1_API secp256k1_context* secp256k1_context_create( + unsigned int flags +) SECP256K1_WARN_UNUSED_RESULT; + +/** Copies a secp256k1 context object. + * + * Returns: a newly created context object. + * Args: ctx: an existing context to copy (cannot be NULL) + */ +SECP256K1_API secp256k1_context* secp256k1_context_clone( + const secp256k1_context* ctx +) SECP256K1_ARG_NONNULL(1) SECP256K1_WARN_UNUSED_RESULT; + +/** Destroy a secp256k1 context object. + * + * The context pointer may not be used afterwards. + * Args: ctx: an existing context to destroy (cannot be NULL) + */ +SECP256K1_API void secp256k1_context_destroy( + secp256k1_context* ctx +); + +/** Set a callback function to be called when an illegal argument is passed to + * an API call. It will only trigger for violations that are mentioned + * explicitly in the header. + * + * The philosophy is that these shouldn't be dealt with through a + * specific return value, as calling code should not have branches to deal with + * the case that this code itself is broken. + * + * On the other hand, during debug stage, one would want to be informed about + * such mistakes, and the default (crashing) may be inadvisable. + * When this callback is triggered, the API function called is guaranteed not + * to cause a crash, though its return value and output arguments are + * undefined. + * + * Args: ctx: an existing context object (cannot be NULL) + * In: fun: a pointer to a function to call when an illegal argument is + * passed to the API, taking a message and an opaque pointer + * (NULL restores a default handler that calls abort). + * data: the opaque pointer to pass to fun above. + */ +SECP256K1_API void secp256k1_context_set_illegal_callback( + secp256k1_context* ctx, + void (*fun)(const char* message, void* data), + const void* data +) SECP256K1_ARG_NONNULL(1); + +/** Set a callback function to be called when an internal consistency check + * fails. The default is crashing. + * + * This can only trigger in case of a hardware failure, miscompilation, + * memory corruption, serious bug in the library, or other error would can + * otherwise result in undefined behaviour. It will not trigger due to mere + * incorrect usage of the API (see secp256k1_context_set_illegal_callback + * for that). After this callback returns, anything may happen, including + * crashing. + * + * Args: ctx: an existing context object (cannot be NULL) + * In: fun: a pointer to a function to call when an internal error occurs, + * taking a message and an opaque pointer (NULL restores a default + * handler that calls abort). + * data: the opaque pointer to pass to fun above. + */ +SECP256K1_API void secp256k1_context_set_error_callback( + secp256k1_context* ctx, + void (*fun)(const char* message, void* data), + const void* data +) SECP256K1_ARG_NONNULL(1); + +/** Parse a variable-length public key into the pubkey object. + * + * Returns: 1 if the public key was fully valid. + * 0 if the public key could not be parsed or is invalid. + * Args: ctx: a secp256k1 context object. + * Out: pubkey: pointer to a pubkey object. If 1 is returned, it is set to a + * parsed version of input. If not, its value is undefined. + * In: input: pointer to a serialized public key + * inputlen: length of the array pointed to by input + * + * This function supports parsing compressed (33 bytes, header byte 0x02 or + * 0x03), uncompressed (65 bytes, header byte 0x04), or hybrid (65 bytes, header + * byte 0x06 or 0x07) format public keys. + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_pubkey_parse( + const secp256k1_context* ctx, + secp256k1_pubkey* pubkey, + const unsigned char *input, + size_t inputlen +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Serialize a pubkey object into a serialized byte sequence. + * + * Returns: 1 always. + * Args: ctx: a secp256k1 context object. + * Out: output: a pointer to a 65-byte (if compressed==0) or 33-byte (if + * compressed==1) byte array to place the serialized key + * in. + * In/Out: outputlen: a pointer to an integer which is initially set to the + * size of output, and is overwritten with the written + * size. + * In: pubkey: a pointer to a secp256k1_pubkey containing an + * initialized public key. + * flags: SECP256K1_EC_COMPRESSED if serialization should be in + * compressed format, otherwise SECP256K1_EC_UNCOMPRESSED. + */ +SECP256K1_API int secp256k1_ec_pubkey_serialize( + const secp256k1_context* ctx, + unsigned char *output, + size_t *outputlen, + const secp256k1_pubkey* pubkey, + unsigned int flags +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +/** Parse an ECDSA signature in compact (64 bytes) format. + * + * Returns: 1 when the signature could be parsed, 0 otherwise. + * Args: ctx: a secp256k1 context object + * Out: sig: a pointer to a signature object + * In: input64: a pointer to the 64-byte array to parse + * + * The signature must consist of a 32-byte big endian R value, followed by a + * 32-byte big endian S value. If R or S fall outside of [0..order-1], the + * encoding is invalid. R and S with value 0 are allowed in the encoding. + * + * After the call, sig will always be initialized. If parsing failed or R or + * S are zero, the resulting sig value is guaranteed to fail validation for any + * message and public key. + */ +SECP256K1_API int secp256k1_ecdsa_signature_parse_compact( + const secp256k1_context* ctx, + secp256k1_ecdsa_signature* sig, + const unsigned char *input64 +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Parse a DER ECDSA signature. + * + * Returns: 1 when the signature could be parsed, 0 otherwise. + * Args: ctx: a secp256k1 context object + * Out: sig: a pointer to a signature object + * In: input: a pointer to the signature to be parsed + * inputlen: the length of the array pointed to be input + * + * This function will accept any valid DER encoded signature, even if the + * encoded numbers are out of range. + * + * After the call, sig will always be initialized. If parsing failed or the + * encoded numbers are out of range, signature validation with it is + * guaranteed to fail for every message and public key. + */ +SECP256K1_API int secp256k1_ecdsa_signature_parse_der( + const secp256k1_context* ctx, + secp256k1_ecdsa_signature* sig, + const unsigned char *input, + size_t inputlen +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Serialize an ECDSA signature in DER format. + * + * Returns: 1 if enough space was available to serialize, 0 otherwise + * Args: ctx: a secp256k1 context object + * Out: output: a pointer to an array to store the DER serialization + * In/Out: outputlen: a pointer to a length integer. Initially, this integer + * should be set to the length of output. After the call + * it will be set to the length of the serialization (even + * if 0 was returned). + * In: sig: a pointer to an initialized signature object + */ +SECP256K1_API int secp256k1_ecdsa_signature_serialize_der( + const secp256k1_context* ctx, + unsigned char *output, + size_t *outputlen, + const secp256k1_ecdsa_signature* sig +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +/** Serialize an ECDSA signature in compact (64 byte) format. + * + * Returns: 1 + * Args: ctx: a secp256k1 context object + * Out: output64: a pointer to a 64-byte array to store the compact serialization + * In: sig: a pointer to an initialized signature object + * + * See secp256k1_ecdsa_signature_parse_compact for details about the encoding. + */ +SECP256K1_API int secp256k1_ecdsa_signature_serialize_compact( + const secp256k1_context* ctx, + unsigned char *output64, + const secp256k1_ecdsa_signature* sig +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Verify an ECDSA signature. + * + * Returns: 1: correct signature + * 0: incorrect or unparseable signature + * Args: ctx: a secp256k1 context object, initialized for verification. + * In: sig: the signature being verified (cannot be NULL) + * msg32: the 32-byte message hash being verified (cannot be NULL) + * pubkey: pointer to an initialized public key to verify with (cannot be NULL) + * + * To avoid accepting malleable signatures, only ECDSA signatures in lower-S + * form are accepted. + * + * If you need to accept ECDSA signatures from sources that do not obey this + * rule, apply secp256k1_ecdsa_signature_normalize to the signature prior to + * validation, but be aware that doing so results in malleable signatures. + * + * For details, see the comments for that function. + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ecdsa_verify( + const secp256k1_context* ctx, + const secp256k1_ecdsa_signature *sig, + const unsigned char *msg32, + const secp256k1_pubkey *pubkey +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +/** Convert a signature to a normalized lower-S form. + * + * Returns: 1 if sigin was not normalized, 0 if it already was. + * Args: ctx: a secp256k1 context object + * Out: sigout: a pointer to a signature to fill with the normalized form, + * or copy if the input was already normalized. (can be NULL if + * you're only interested in whether the input was already + * normalized). + * In: sigin: a pointer to a signature to check/normalize (cannot be NULL, + * can be identical to sigout) + * + * With ECDSA a third-party can forge a second distinct signature of the same + * message, given a single initial signature, but without knowing the key. This + * is done by negating the S value modulo the order of the curve, 'flipping' + * the sign of the random point R which is not included in the signature. + * + * Forgery of the same message isn't universally problematic, but in systems + * where message malleability or uniqueness of signatures is important this can + * cause issues. This forgery can be blocked by all verifiers forcing signers + * to use a normalized form. + * + * The lower-S form reduces the size of signatures slightly on average when + * variable length encodings (such as DER) are used and is cheap to verify, + * making it a good choice. Security of always using lower-S is assured because + * anyone can trivially modify a signature after the fact to enforce this + * property anyway. + * + * The lower S value is always between 0x1 and + * 0x7FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF5D576E7357A4501DDFE92F46681B20A0, + * inclusive. + * + * No other forms of ECDSA malleability are known and none seem likely, but + * there is no formal proof that ECDSA, even with this additional restriction, + * is free of other malleability. Commonly used serialization schemes will also + * accept various non-unique encodings, so care should be taken when this + * property is required for an application. + * + * The secp256k1_ecdsa_sign function will by default create signatures in the + * lower-S form, and secp256k1_ecdsa_verify will not accept others. In case + * signatures come from a system that cannot enforce this property, + * secp256k1_ecdsa_signature_normalize must be called before verification. + */ +SECP256K1_API int secp256k1_ecdsa_signature_normalize( + const secp256k1_context* ctx, + secp256k1_ecdsa_signature *sigout, + const secp256k1_ecdsa_signature *sigin +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(3); + +/** An implementation of RFC6979 (using HMAC-SHA256) as nonce generation function. + * If a data pointer is passed, it is assumed to be a pointer to 32 bytes of + * extra entropy. + */ +SECP256K1_API extern const secp256k1_nonce_function secp256k1_nonce_function_rfc6979; + +/** A default safe nonce generation function (currently equal to secp256k1_nonce_function_rfc6979). */ +SECP256K1_API extern const secp256k1_nonce_function secp256k1_nonce_function_default; + +/** Create an ECDSA signature. + * + * Returns: 1: signature created + * 0: the nonce generation function failed, or the private key was invalid. + * Args: ctx: pointer to a context object, initialized for signing (cannot be NULL) + * Out: sig: pointer to an array where the signature will be placed (cannot be NULL) + * In: msg32: the 32-byte message hash being signed (cannot be NULL) + * seckey: pointer to a 32-byte secret key (cannot be NULL) + * noncefp:pointer to a nonce generation function. If NULL, secp256k1_nonce_function_default is used + * ndata: pointer to arbitrary data used by the nonce generation function (can be NULL) + * + * The created signature is always in lower-S form. See + * secp256k1_ecdsa_signature_normalize for more details. + */ +SECP256K1_API int secp256k1_ecdsa_sign( + const secp256k1_context* ctx, + secp256k1_ecdsa_signature *sig, + const unsigned char *msg32, + const unsigned char *seckey, + secp256k1_nonce_function noncefp, + const void *ndata +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +/** Verify an ECDSA secret key. + * + * Returns: 1: secret key is valid + * 0: secret key is invalid + * Args: ctx: pointer to a context object (cannot be NULL) + * In: seckey: pointer to a 32-byte secret key (cannot be NULL) + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_seckey_verify( + const secp256k1_context* ctx, + const unsigned char *seckey +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2); + +/** Compute the public key for a secret key. + * + * Returns: 1: secret was valid, public key stores + * 0: secret was invalid, try again + * Args: ctx: pointer to a context object, initialized for signing (cannot be NULL) + * Out: pubkey: pointer to the created public key (cannot be NULL) + * In: seckey: pointer to a 32-byte private key (cannot be NULL) + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_pubkey_create( + const secp256k1_context* ctx, + secp256k1_pubkey *pubkey, + const unsigned char *seckey +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Tweak a private key by adding tweak to it. + * Returns: 0 if the tweak was out of range (chance of around 1 in 2^128 for + * uniformly random 32-byte arrays, or if the resulting private key + * would be invalid (only when the tweak is the complement of the + * private key). 1 otherwise. + * Args: ctx: pointer to a context object (cannot be NULL). + * In/Out: seckey: pointer to a 32-byte private key. + * In: tweak: pointer to a 32-byte tweak. + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_privkey_tweak_add( + const secp256k1_context* ctx, + unsigned char *seckey, + const unsigned char *tweak +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Tweak a public key by adding tweak times the generator to it. + * Returns: 0 if the tweak was out of range (chance of around 1 in 2^128 for + * uniformly random 32-byte arrays, or if the resulting public key + * would be invalid (only when the tweak is the complement of the + * corresponding private key). 1 otherwise. + * Args: ctx: pointer to a context object initialized for validation + * (cannot be NULL). + * In/Out: pubkey: pointer to a public key object. + * In: tweak: pointer to a 32-byte tweak. + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_pubkey_tweak_add( + const secp256k1_context* ctx, + secp256k1_pubkey *pubkey, + const unsigned char *tweak +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Tweak a private key by multiplying it by a tweak. + * Returns: 0 if the tweak was out of range (chance of around 1 in 2^128 for + * uniformly random 32-byte arrays, or equal to zero. 1 otherwise. + * Args: ctx: pointer to a context object (cannot be NULL). + * In/Out: seckey: pointer to a 32-byte private key. + * In: tweak: pointer to a 32-byte tweak. + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_privkey_tweak_mul( + const secp256k1_context* ctx, + unsigned char *seckey, + const unsigned char *tweak +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Tweak a public key by multiplying it by a tweak value. + * Returns: 0 if the tweak was out of range (chance of around 1 in 2^128 for + * uniformly random 32-byte arrays, or equal to zero. 1 otherwise. + * Args: ctx: pointer to a context object initialized for validation + * (cannot be NULL). + * In/Out: pubkey: pointer to a public key obkect. + * In: tweak: pointer to a 32-byte tweak. + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_pubkey_tweak_mul( + const secp256k1_context* ctx, + secp256k1_pubkey *pubkey, + const unsigned char *tweak +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Updates the context randomization. + * Returns: 1: randomization successfully updated + * 0: error + * Args: ctx: pointer to a context object (cannot be NULL) + * In: seed32: pointer to a 32-byte random seed (NULL resets to initial state) + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_context_randomize( + secp256k1_context* ctx, + const unsigned char *seed32 +) SECP256K1_ARG_NONNULL(1); + +/** Add a number of public keys together. + * Returns: 1: the sum of the public keys is valid. + * 0: the sum of the public keys is not valid. + * Args: ctx: pointer to a context object + * Out: out: pointer to a public key object for placing the resulting public key + * (cannot be NULL) + * In: ins: pointer to array of pointers to public keys (cannot be NULL) + * n: the number of public keys to add together (must be at least 1) + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ec_pubkey_combine( + const secp256k1_context* ctx, + secp256k1_pubkey *out, + const secp256k1_pubkey * const * ins, + size_t n +) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +# ifdef __cplusplus +} +# endif + +#endif diff --git a/src/secp256k1/include/secp256k1_ecdh.h b/src/secp256k1/include/secp256k1_ecdh.h new file mode 100644 index 00000000..4b84d7a9 --- /dev/null +++ b/src/secp256k1/include/secp256k1_ecdh.h @@ -0,0 +1,31 @@ +#ifndef _SECP256K1_ECDH_ +# define _SECP256K1_ECDH_ + +# include "secp256k1.h" + +# ifdef __cplusplus +extern "C" { +# endif + +/** Compute an EC Diffie-Hellman secret in constant time + * Returns: 1: exponentiation was successful + * 0: scalar was invalid (zero or overflow) + * Args: ctx: pointer to a context object (cannot be NULL) + * Out: result: a 32-byte array which will be populated by an ECDH + * secret computed from the point and scalar + * In: pubkey: a pointer to a secp256k1_pubkey containing an + * initialized public key + * privkey: a 32-byte scalar with which to multiply the point + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ecdh( + const secp256k1_context* ctx, + unsigned char *result, + const secp256k1_pubkey *pubkey, + const unsigned char *privkey +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +# ifdef __cplusplus +} +# endif + +#endif diff --git a/src/secp256k1/include/secp256k1_recovery.h b/src/secp256k1/include/secp256k1_recovery.h new file mode 100644 index 00000000..05537972 --- /dev/null +++ b/src/secp256k1/include/secp256k1_recovery.h @@ -0,0 +1,110 @@ +#ifndef _SECP256K1_RECOVERY_ +# define _SECP256K1_RECOVERY_ + +# include "secp256k1.h" + +# ifdef __cplusplus +extern "C" { +# endif + +/** Opaque data structured that holds a parsed ECDSA signature, + * supporting pubkey recovery. + * + * The exact representation of data inside is implementation defined and not + * guaranteed to be portable between different platforms or versions. It is + * however guaranteed to be 65 bytes in size, and can be safely copied/moved. + * If you need to convert to a format suitable for storage or transmission, use + * the secp256k1_ecdsa_signature_serialize_* and + * secp256k1_ecdsa_signature_parse_* functions. + * + * Furthermore, it is guaranteed that identical signatures (including their + * recoverability) will have identical representation, so they can be + * memcmp'ed. + */ +typedef struct { + unsigned char data[65]; +} secp256k1_ecdsa_recoverable_signature; + +/** Parse a compact ECDSA signature (64 bytes + recovery id). + * + * Returns: 1 when the signature could be parsed, 0 otherwise + * Args: ctx: a secp256k1 context object + * Out: sig: a pointer to a signature object + * In: input64: a pointer to a 64-byte compact signature + * recid: the recovery id (0, 1, 2 or 3) + */ +SECP256K1_API int secp256k1_ecdsa_recoverable_signature_parse_compact( + const secp256k1_context* ctx, + secp256k1_ecdsa_recoverable_signature* sig, + const unsigned char *input64, + int recid +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Convert a recoverable signature into a normal signature. + * + * Returns: 1 + * Out: sig: a pointer to a normal signature (cannot be NULL). + * In: sigin: a pointer to a recoverable signature (cannot be NULL). + */ +SECP256K1_API int secp256k1_ecdsa_recoverable_signature_convert( + const secp256k1_context* ctx, + secp256k1_ecdsa_signature* sig, + const secp256k1_ecdsa_recoverable_signature* sigin +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Serialize an ECDSA signature in compact format (64 bytes + recovery id). + * + * Returns: 1 + * Args: ctx: a secp256k1 context object + * Out: output64: a pointer to a 64-byte array of the compact signature (cannot be NULL) + * recid: a pointer to an integer to hold the recovery id (can be NULL). + * In: sig: a pointer to an initialized signature object (cannot be NULL) + */ +SECP256K1_API int secp256k1_ecdsa_recoverable_signature_serialize_compact( + const secp256k1_context* ctx, + unsigned char *output64, + int *recid, + const secp256k1_ecdsa_recoverable_signature* sig +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +/** Create a recoverable ECDSA signature. + * + * Returns: 1: signature created + * 0: the nonce generation function failed, or the private key was invalid. + * Args: ctx: pointer to a context object, initialized for signing (cannot be NULL) + * Out: sig: pointer to an array where the signature will be placed (cannot be NULL) + * In: msg32: the 32-byte message hash being signed (cannot be NULL) + * seckey: pointer to a 32-byte secret key (cannot be NULL) + * noncefp:pointer to a nonce generation function. If NULL, secp256k1_nonce_function_default is used + * ndata: pointer to arbitrary data used by the nonce generation function (can be NULL) + */ +SECP256K1_API int secp256k1_ecdsa_sign_recoverable( + const secp256k1_context* ctx, + secp256k1_ecdsa_recoverable_signature *sig, + const unsigned char *msg32, + const unsigned char *seckey, + secp256k1_nonce_function noncefp, + const void *ndata +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +/** Recover an ECDSA public key from a signature. + * + * Returns: 1: public key successfully recovered (which guarantees a correct signature). + * 0: otherwise. + * Args: ctx: pointer to a context object, initialized for verification (cannot be NULL) + * Out: pubkey: pointer to the recovered public key (cannot be NULL) + * In: sig: pointer to initialized signature that supports pubkey recovery (cannot be NULL) + * msg32: the 32-byte message hash assumed to be signed (cannot be NULL) + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_ecdsa_recover( + const secp256k1_context* ctx, + secp256k1_pubkey *pubkey, + const secp256k1_ecdsa_recoverable_signature *sig, + const unsigned char *msg32 +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +# ifdef __cplusplus +} +# endif + +#endif diff --git a/src/secp256k1/include/secp256k1_schnorr.h b/src/secp256k1/include/secp256k1_schnorr.h new file mode 100644 index 00000000..dc32fec1 --- /dev/null +++ b/src/secp256k1/include/secp256k1_schnorr.h @@ -0,0 +1,173 @@ +#ifndef _SECP256K1_SCHNORR_ +# define _SECP256K1_SCHNORR_ + +# include "secp256k1.h" + +# ifdef __cplusplus +extern "C" { +# endif + +/** Create a signature using a custom EC-Schnorr-SHA256 construction. It + * produces non-malleable 64-byte signatures which support public key recovery + * batch validation, and multiparty signing. + * Returns: 1: signature created + * 0: the nonce generation function failed, or the private key was + * invalid. + * Args: ctx: pointer to a context object, initialized for signing + * (cannot be NULL) + * Out: sig64: pointer to a 64-byte array where the signature will be + * placed (cannot be NULL) + * In: msg32: the 32-byte message hash being signed (cannot be NULL) + * seckey: pointer to a 32-byte secret key (cannot be NULL) + * noncefp:pointer to a nonce generation function. If NULL, + * secp256k1_nonce_function_default is used + * ndata: pointer to arbitrary data used by the nonce generation + * function (can be NULL) + */ +SECP256K1_API int secp256k1_schnorr_sign( + const secp256k1_context* ctx, + unsigned char *sig64, + const unsigned char *msg32, + const unsigned char *seckey, + secp256k1_nonce_function noncefp, + const void *ndata +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +/** Verify a signature created by secp256k1_schnorr_sign. + * Returns: 1: correct signature + * 0: incorrect signature + * Args: ctx: a secp256k1 context object, initialized for verification. + * In: sig64: the 64-byte signature being verified (cannot be NULL) + * msg32: the 32-byte message hash being verified (cannot be NULL) + * pubkey: the public key to verify with (cannot be NULL) + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_schnorr_verify( + const secp256k1_context* ctx, + const unsigned char *sig64, + const unsigned char *msg32, + const secp256k1_pubkey *pubkey +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +/** Recover an EC public key from a Schnorr signature created using + * secp256k1_schnorr_sign. + * Returns: 1: public key successfully recovered (which guarantees a correct + * signature). + * 0: otherwise. + * Args: ctx: pointer to a context object, initialized for + * verification (cannot be NULL) + * Out: pubkey: pointer to a pubkey to set to the recovered public key + * (cannot be NULL). + * In: sig64: signature as 64 byte array (cannot be NULL) + * msg32: the 32-byte message hash assumed to be signed (cannot + * be NULL) + */ +SECP256K1_API int secp256k1_schnorr_recover( + const secp256k1_context* ctx, + secp256k1_pubkey *pubkey, + const unsigned char *sig64, + const unsigned char *msg32 +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4); + +/** Generate a nonce pair deterministically for use with + * secp256k1_schnorr_partial_sign. + * Returns: 1: valid nonce pair was generated. + * 0: otherwise (nonce generation function failed) + * Args: ctx: pointer to a context object, initialized for signing + * (cannot be NULL) + * Out: pubnonce: public side of the nonce (cannot be NULL) + * privnonce32: private side of the nonce (32 byte) (cannot be NULL) + * In: msg32: the 32-byte message hash assumed to be signed (cannot + * be NULL) + * sec32: the 32-byte private key (cannot be NULL) + * noncefp: pointer to a nonce generation function. If NULL, + * secp256k1_nonce_function_default is used + * noncedata: pointer to arbitrary data used by the nonce generation + * function (can be NULL) + * + * Do not use the output as a private/public key pair for signing/validation. + */ +SECP256K1_API int secp256k1_schnorr_generate_nonce_pair( + const secp256k1_context* ctx, + secp256k1_pubkey *pubnonce, + unsigned char *privnonce32, + const unsigned char *msg32, + const unsigned char *sec32, + secp256k1_nonce_function noncefp, + const void* noncedata +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +/** Produce a partial Schnorr signature, which can be combined using + * secp256k1_schnorr_partial_combine, to end up with a full signature that is + * verifiable using secp256k1_schnorr_verify. + * Returns: 1: signature created successfully. + * 0: no valid signature exists with this combination of keys, nonces + * and message (chance around 1 in 2^128) + * -1: invalid private key, nonce, or public nonces. + * Args: ctx: pointer to context object, initialized for signing (cannot + * be NULL) + * Out: sig64: pointer to 64-byte array to put partial signature in + * In: msg32: pointer to 32-byte message to sign + * sec32: pointer to 32-byte private key + * pubnonce_others: pointer to pubkey containing the sum of the other's + * nonces (see secp256k1_ec_pubkey_combine) + * secnonce32: pointer to 32-byte array containing our nonce + * + * The intended procedure for creating a multiparty signature is: + * - Each signer S[i] with private key x[i] and public key Q[i] runs + * secp256k1_schnorr_generate_nonce_pair to produce a pair (k[i],R[i]) of + * private/public nonces. + * - All signers communicate their public nonces to each other (revealing your + * private nonce can lead to discovery of your private key, so it should be + * considered secret). + * - All signers combine all the public nonces they received (excluding their + * own) using secp256k1_ec_pubkey_combine to obtain an + * Rall[i] = sum(R[0..i-1,i+1..n]). + * - All signers produce a partial signature using + * secp256k1_schnorr_partial_sign, passing in their own private key x[i], + * their own private nonce k[i], and the sum of the others' public nonces + * Rall[i]. + * - All signers communicate their partial signatures to each other. + * - Someone combines all partial signatures using + * secp256k1_schnorr_partial_combine, to obtain a full signature. + * - The resulting signature is validatable using secp256k1_schnorr_verify, with + * public key equal to the result of secp256k1_ec_pubkey_combine of the + * signers' public keys (sum(Q[0..n])). + * + * Note that secp256k1_schnorr_partial_combine and secp256k1_ec_pubkey_combine + * function take their arguments in any order, and it is possible to + * pre-combine several inputs already with one call, and add more inputs later + * by calling the function again (they are commutative and associative). + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_schnorr_partial_sign( + const secp256k1_context* ctx, + unsigned char *sig64, + const unsigned char *msg32, + const unsigned char *sec32, + const secp256k1_pubkey *pubnonce_others, + const unsigned char *secnonce32 +) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3) SECP256K1_ARG_NONNULL(4) SECP256K1_ARG_NONNULL(5) SECP256K1_ARG_NONNULL(6); + +/** Combine multiple Schnorr partial signatures. + * Returns: 1: the passed signatures were successfully combined. + * 0: the resulting signature is not valid (chance of 1 in 2^256) + * -1: some inputs were invalid, or the signatures were not created + * using the same set of nonces + * Args: ctx: pointer to a context object + * Out: sig64: pointer to a 64-byte array to place the combined signature + * (cannot be NULL) + * In: sig64sin: pointer to an array of n pointers to 64-byte input + * signatures + * n: the number of signatures to combine (at least 1) + */ +SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_schnorr_partial_combine( + const secp256k1_context* ctx, + unsigned char *sig64, + const unsigned char * const * sig64sin, + size_t n +) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); + +# ifdef __cplusplus +} +# endif + +#endif diff --git a/src/secp256k1/libsecp256k1.pc.in b/src/secp256k1/libsecp256k1.pc.in new file mode 100644 index 00000000..1c72dd00 --- /dev/null +++ b/src/secp256k1/libsecp256k1.pc.in @@ -0,0 +1,13 @@ +prefix=@prefix@ +exec_prefix=@exec_prefix@ +libdir=@libdir@ +includedir=@includedir@ + +Name: libsecp256k1 +Description: Optimized C library for EC operations on curve secp256k1 +URL: https://github.com/bitcoin/secp256k1 +Version: @PACKAGE_VERSION@ +Cflags: -I${includedir} +Libs.private: @SECP_LIBS@ +Libs: -L${libdir} -lsecp256k1 + diff --git a/src/secp256k1/obj/.gitignore b/src/secp256k1/obj/.gitignore new file mode 100644 index 00000000..e69de29b diff --git a/src/secp256k1/src/basic-config.h b/src/secp256k1/src/basic-config.h new file mode 100644 index 00000000..c4c16eb7 --- /dev/null +++ b/src/secp256k1/src/basic-config.h @@ -0,0 +1,32 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_BASIC_CONFIG_ +#define _SECP256K1_BASIC_CONFIG_ + +#ifdef USE_BASIC_CONFIG + +#undef USE_ASM_X86_64 +#undef USE_ENDOMORPHISM +#undef USE_FIELD_10X26 +#undef USE_FIELD_5X52 +#undef USE_FIELD_INV_BUILTIN +#undef USE_FIELD_INV_NUM +#undef USE_NUM_GMP +#undef USE_NUM_NONE +#undef USE_SCALAR_4X64 +#undef USE_SCALAR_8X32 +#undef USE_SCALAR_INV_BUILTIN +#undef USE_SCALAR_INV_NUM + +#define USE_NUM_NONE 1 +#define USE_FIELD_INV_BUILTIN 1 +#define USE_SCALAR_INV_BUILTIN 1 +#define USE_FIELD_10X26 1 +#define USE_SCALAR_8X32 1 + +#endif // USE_BASIC_CONFIG +#endif // _SECP256K1_BASIC_CONFIG_ diff --git a/src/secp256k1/src/bench.h b/src/secp256k1/src/bench.h new file mode 100644 index 00000000..3a71b4aa --- /dev/null +++ b/src/secp256k1/src/bench.h @@ -0,0 +1,66 @@ +/********************************************************************** + * Copyright (c) 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_BENCH_H_ +#define _SECP256K1_BENCH_H_ + +#include +#include +#include "sys/time.h" + +static double gettimedouble(void) { + struct timeval tv; + gettimeofday(&tv, NULL); + return tv.tv_usec * 0.000001 + tv.tv_sec; +} + +void print_number(double x) { + double y = x; + int c = 0; + if (y < 0.0) { + y = -y; + } + while (y < 100.0) { + y *= 10.0; + c++; + } + printf("%.*f", c, x); +} + +void run_benchmark(char *name, void (*benchmark)(void*), void (*setup)(void*), void (*teardown)(void*), void* data, int count, int iter) { + int i; + double min = HUGE_VAL; + double sum = 0.0; + double max = 0.0; + for (i = 0; i < count; i++) { + double begin, total; + if (setup != NULL) { + setup(data); + } + begin = gettimedouble(); + benchmark(data); + total = gettimedouble() - begin; + if (teardown != NULL) { + teardown(data); + } + if (total < min) { + min = total; + } + if (total > max) { + max = total; + } + sum += total; + } + printf("%s: min ", name); + print_number(min * 1000000.0 / iter); + printf("us / avg "); + print_number((sum / count) * 1000000.0 / iter); + printf("us / max "); + print_number(max * 1000000.0 / iter); + printf("us\n"); +} + +#endif diff --git a/src/secp256k1/src/bench_ecdh.c b/src/secp256k1/src/bench_ecdh.c new file mode 100644 index 00000000..5a7c6376 --- /dev/null +++ b/src/secp256k1/src/bench_ecdh.c @@ -0,0 +1,53 @@ +/********************************************************************** + * Copyright (c) 2015 Pieter Wuille, Andrew Poelstra * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#include + +#include "include/secp256k1.h" +#include "include/secp256k1_ecdh.h" +#include "util.h" +#include "bench.h" + +typedef struct { + secp256k1_context *ctx; + secp256k1_pubkey point; + unsigned char scalar[32]; +} bench_ecdh_t; + +static void bench_ecdh_setup(void* arg) { + int i; + bench_ecdh_t *data = (bench_ecdh_t*)arg; + const unsigned char point[] = { + 0x03, + 0x54, 0x94, 0xc1, 0x5d, 0x32, 0x09, 0x97, 0x06, + 0xc2, 0x39, 0x5f, 0x94, 0x34, 0x87, 0x45, 0xfd, + 0x75, 0x7c, 0xe3, 0x0e, 0x4e, 0x8c, 0x90, 0xfb, + 0xa2, 0xba, 0xd1, 0x84, 0xf8, 0x83, 0xc6, 0x9f + }; + + data->ctx = secp256k1_context_create(0); + for (i = 0; i < 32; i++) { + data->scalar[i] = i + 1; + } + CHECK(secp256k1_ec_pubkey_parse(data->ctx, &data->point, point, sizeof(point)) == 1); +} + +static void bench_ecdh(void* arg) { + int i; + unsigned char res[32]; + bench_ecdh_t *data = (bench_ecdh_t*)arg; + + for (i = 0; i < 20000; i++) { + CHECK(secp256k1_ecdh(data->ctx, res, &data->point, data->scalar) == 1); + } +} + +int main(void) { + bench_ecdh_t data; + + run_benchmark("ecdh", bench_ecdh, bench_ecdh_setup, NULL, &data, 10, 20000); + return 0; +} diff --git a/src/secp256k1/src/bench_internal.c b/src/secp256k1/src/bench_internal.c new file mode 100644 index 00000000..7809f5f8 --- /dev/null +++ b/src/secp256k1/src/bench_internal.c @@ -0,0 +1,354 @@ +/********************************************************************** + * Copyright (c) 2014-2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ +#include + +#include "include/secp256k1.h" + +#include "util.h" +#include "hash_impl.h" +#include "num_impl.h" +#include "field_impl.h" +#include "group_impl.h" +#include "scalar_impl.h" +#include "ecmult_const_impl.h" +#include "ecmult_impl.h" +#include "bench.h" +#include "secp256k1.c" + +typedef struct { + secp256k1_scalar scalar_x, scalar_y; + secp256k1_fe fe_x, fe_y; + secp256k1_ge ge_x, ge_y; + secp256k1_gej gej_x, gej_y; + unsigned char data[64]; + int wnaf[256]; +} bench_inv_t; + +void bench_setup(void* arg) { + bench_inv_t *data = (bench_inv_t*)arg; + + static const unsigned char init_x[32] = { + 0x02, 0x03, 0x05, 0x07, 0x0b, 0x0d, 0x11, 0x13, + 0x17, 0x1d, 0x1f, 0x25, 0x29, 0x2b, 0x2f, 0x35, + 0x3b, 0x3d, 0x43, 0x47, 0x49, 0x4f, 0x53, 0x59, + 0x61, 0x65, 0x67, 0x6b, 0x6d, 0x71, 0x7f, 0x83 + }; + + static const unsigned char init_y[32] = { + 0x82, 0x83, 0x85, 0x87, 0x8b, 0x8d, 0x81, 0x83, + 0x97, 0xad, 0xaf, 0xb5, 0xb9, 0xbb, 0xbf, 0xc5, + 0xdb, 0xdd, 0xe3, 0xe7, 0xe9, 0xef, 0xf3, 0xf9, + 0x11, 0x15, 0x17, 0x1b, 0x1d, 0xb1, 0xbf, 0xd3 + }; + + secp256k1_scalar_set_b32(&data->scalar_x, init_x, NULL); + secp256k1_scalar_set_b32(&data->scalar_y, init_y, NULL); + secp256k1_fe_set_b32(&data->fe_x, init_x); + secp256k1_fe_set_b32(&data->fe_y, init_y); + CHECK(secp256k1_ge_set_xo_var(&data->ge_x, &data->fe_x, 0)); + CHECK(secp256k1_ge_set_xo_var(&data->ge_y, &data->fe_y, 1)); + secp256k1_gej_set_ge(&data->gej_x, &data->ge_x); + secp256k1_gej_set_ge(&data->gej_y, &data->ge_y); + memcpy(data->data, init_x, 32); + memcpy(data->data + 32, init_y, 32); +} + +void bench_scalar_add(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 2000000; i++) { + secp256k1_scalar_add(&data->scalar_x, &data->scalar_x, &data->scalar_y); + } +} + +void bench_scalar_negate(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 2000000; i++) { + secp256k1_scalar_negate(&data->scalar_x, &data->scalar_x); + } +} + +void bench_scalar_sqr(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 200000; i++) { + secp256k1_scalar_sqr(&data->scalar_x, &data->scalar_x); + } +} + +void bench_scalar_mul(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 200000; i++) { + secp256k1_scalar_mul(&data->scalar_x, &data->scalar_x, &data->scalar_y); + } +} + +#ifdef USE_ENDOMORPHISM +void bench_scalar_split(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 20000; i++) { + secp256k1_scalar l, r; + secp256k1_scalar_split_lambda(&l, &r, &data->scalar_x); + secp256k1_scalar_add(&data->scalar_x, &data->scalar_x, &data->scalar_y); + } +} +#endif + +void bench_scalar_inverse(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 2000; i++) { + secp256k1_scalar_inverse(&data->scalar_x, &data->scalar_x); + secp256k1_scalar_add(&data->scalar_x, &data->scalar_x, &data->scalar_y); + } +} + +void bench_scalar_inverse_var(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 2000; i++) { + secp256k1_scalar_inverse_var(&data->scalar_x, &data->scalar_x); + secp256k1_scalar_add(&data->scalar_x, &data->scalar_x, &data->scalar_y); + } +} + +void bench_field_normalize(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 2000000; i++) { + secp256k1_fe_normalize(&data->fe_x); + } +} + +void bench_field_normalize_weak(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 2000000; i++) { + secp256k1_fe_normalize_weak(&data->fe_x); + } +} + +void bench_field_mul(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 200000; i++) { + secp256k1_fe_mul(&data->fe_x, &data->fe_x, &data->fe_y); + } +} + +void bench_field_sqr(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 200000; i++) { + secp256k1_fe_sqr(&data->fe_x, &data->fe_x); + } +} + +void bench_field_inverse(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 20000; i++) { + secp256k1_fe_inv(&data->fe_x, &data->fe_x); + secp256k1_fe_add(&data->fe_x, &data->fe_y); + } +} + +void bench_field_inverse_var(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 20000; i++) { + secp256k1_fe_inv_var(&data->fe_x, &data->fe_x); + secp256k1_fe_add(&data->fe_x, &data->fe_y); + } +} + +void bench_field_sqrt_var(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 20000; i++) { + secp256k1_fe_sqrt_var(&data->fe_x, &data->fe_x); + secp256k1_fe_add(&data->fe_x, &data->fe_y); + } +} + +void bench_group_double_var(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 200000; i++) { + secp256k1_gej_double_var(&data->gej_x, &data->gej_x, NULL); + } +} + +void bench_group_add_var(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 200000; i++) { + secp256k1_gej_add_var(&data->gej_x, &data->gej_x, &data->gej_y, NULL); + } +} + +void bench_group_add_affine(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 200000; i++) { + secp256k1_gej_add_ge(&data->gej_x, &data->gej_x, &data->ge_y); + } +} + +void bench_group_add_affine_var(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 200000; i++) { + secp256k1_gej_add_ge_var(&data->gej_x, &data->gej_x, &data->ge_y, NULL); + } +} + +void bench_ecmult_wnaf(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 20000; i++) { + secp256k1_ecmult_wnaf(data->wnaf, 256, &data->scalar_x, WINDOW_A); + secp256k1_scalar_add(&data->scalar_x, &data->scalar_x, &data->scalar_y); + } +} + +void bench_wnaf_const(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + + for (i = 0; i < 20000; i++) { + secp256k1_wnaf_const(data->wnaf, data->scalar_x, WINDOW_A); + secp256k1_scalar_add(&data->scalar_x, &data->scalar_x, &data->scalar_y); + } +} + + +void bench_sha256(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + secp256k1_sha256_t sha; + + for (i = 0; i < 20000; i++) { + secp256k1_sha256_initialize(&sha); + secp256k1_sha256_write(&sha, data->data, 32); + secp256k1_sha256_finalize(&sha, data->data); + } +} + +void bench_hmac_sha256(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + secp256k1_hmac_sha256_t hmac; + + for (i = 0; i < 20000; i++) { + secp256k1_hmac_sha256_initialize(&hmac, data->data, 32); + secp256k1_hmac_sha256_write(&hmac, data->data, 32); + secp256k1_hmac_sha256_finalize(&hmac, data->data); + } +} + +void bench_rfc6979_hmac_sha256(void* arg) { + int i; + bench_inv_t *data = (bench_inv_t*)arg; + secp256k1_rfc6979_hmac_sha256_t rng; + + for (i = 0; i < 20000; i++) { + secp256k1_rfc6979_hmac_sha256_initialize(&rng, data->data, 64); + secp256k1_rfc6979_hmac_sha256_generate(&rng, data->data, 32); + } +} + +void bench_context_verify(void* arg) { + int i; + (void)arg; + for (i = 0; i < 20; i++) { + secp256k1_context_destroy(secp256k1_context_create(SECP256K1_CONTEXT_VERIFY)); + } +} + +void bench_context_sign(void* arg) { + int i; + (void)arg; + for (i = 0; i < 200; i++) { + secp256k1_context_destroy(secp256k1_context_create(SECP256K1_CONTEXT_SIGN)); + } +} + + +int have_flag(int argc, char** argv, char *flag) { + char** argm = argv + argc; + argv++; + if (argv == argm) { + return 1; + } + while (argv != NULL && argv != argm) { + if (strcmp(*argv, flag) == 0) { + return 1; + } + argv++; + } + return 0; +} + +int main(int argc, char **argv) { + bench_inv_t data; + if (have_flag(argc, argv, "scalar") || have_flag(argc, argv, "add")) run_benchmark("scalar_add", bench_scalar_add, bench_setup, NULL, &data, 10, 2000000); + if (have_flag(argc, argv, "scalar") || have_flag(argc, argv, "negate")) run_benchmark("scalar_negate", bench_scalar_negate, bench_setup, NULL, &data, 10, 2000000); + if (have_flag(argc, argv, "scalar") || have_flag(argc, argv, "sqr")) run_benchmark("scalar_sqr", bench_scalar_sqr, bench_setup, NULL, &data, 10, 200000); + if (have_flag(argc, argv, "scalar") || have_flag(argc, argv, "mul")) run_benchmark("scalar_mul", bench_scalar_mul, bench_setup, NULL, &data, 10, 200000); +#ifdef USE_ENDOMORPHISM + if (have_flag(argc, argv, "scalar") || have_flag(argc, argv, "split")) run_benchmark("scalar_split", bench_scalar_split, bench_setup, NULL, &data, 10, 20000); +#endif + if (have_flag(argc, argv, "scalar") || have_flag(argc, argv, "inverse")) run_benchmark("scalar_inverse", bench_scalar_inverse, bench_setup, NULL, &data, 10, 2000); + if (have_flag(argc, argv, "scalar") || have_flag(argc, argv, "inverse")) run_benchmark("scalar_inverse_var", bench_scalar_inverse_var, bench_setup, NULL, &data, 10, 2000); + + if (have_flag(argc, argv, "field") || have_flag(argc, argv, "normalize")) run_benchmark("field_normalize", bench_field_normalize, bench_setup, NULL, &data, 10, 2000000); + if (have_flag(argc, argv, "field") || have_flag(argc, argv, "normalize")) run_benchmark("field_normalize_weak", bench_field_normalize_weak, bench_setup, NULL, &data, 10, 2000000); + if (have_flag(argc, argv, "field") || have_flag(argc, argv, "sqr")) run_benchmark("field_sqr", bench_field_sqr, bench_setup, NULL, &data, 10, 200000); + if (have_flag(argc, argv, "field") || have_flag(argc, argv, "mul")) run_benchmark("field_mul", bench_field_mul, bench_setup, NULL, &data, 10, 200000); + if (have_flag(argc, argv, "field") || have_flag(argc, argv, "inverse")) run_benchmark("field_inverse", bench_field_inverse, bench_setup, NULL, &data, 10, 20000); + if (have_flag(argc, argv, "field") || have_flag(argc, argv, "inverse")) run_benchmark("field_inverse_var", bench_field_inverse_var, bench_setup, NULL, &data, 10, 20000); + if (have_flag(argc, argv, "field") || have_flag(argc, argv, "sqrt")) run_benchmark("field_sqrt_var", bench_field_sqrt_var, bench_setup, NULL, &data, 10, 20000); + + if (have_flag(argc, argv, "group") || have_flag(argc, argv, "double")) run_benchmark("group_double_var", bench_group_double_var, bench_setup, NULL, &data, 10, 200000); + if (have_flag(argc, argv, "group") || have_flag(argc, argv, "add")) run_benchmark("group_add_var", bench_group_add_var, bench_setup, NULL, &data, 10, 200000); + if (have_flag(argc, argv, "group") || have_flag(argc, argv, "add")) run_benchmark("group_add_affine", bench_group_add_affine, bench_setup, NULL, &data, 10, 200000); + if (have_flag(argc, argv, "group") || have_flag(argc, argv, "add")) run_benchmark("group_add_affine_var", bench_group_add_affine_var, bench_setup, NULL, &data, 10, 200000); + + if (have_flag(argc, argv, "ecmult") || have_flag(argc, argv, "wnaf")) run_benchmark("wnaf_const", bench_wnaf_const, bench_setup, NULL, &data, 10, 20000); + if (have_flag(argc, argv, "ecmult") || have_flag(argc, argv, "wnaf")) run_benchmark("ecmult_wnaf", bench_ecmult_wnaf, bench_setup, NULL, &data, 10, 20000); + + if (have_flag(argc, argv, "hash") || have_flag(argc, argv, "sha256")) run_benchmark("hash_sha256", bench_sha256, bench_setup, NULL, &data, 10, 20000); + if (have_flag(argc, argv, "hash") || have_flag(argc, argv, "hmac")) run_benchmark("hash_hmac_sha256", bench_hmac_sha256, bench_setup, NULL, &data, 10, 20000); + if (have_flag(argc, argv, "hash") || have_flag(argc, argv, "rng6979")) run_benchmark("hash_rfc6979_hmac_sha256", bench_rfc6979_hmac_sha256, bench_setup, NULL, &data, 10, 20000); + + if (have_flag(argc, argv, "context") || have_flag(argc, argv, "verify")) run_benchmark("context_verify", bench_context_verify, bench_setup, NULL, &data, 10, 20); + if (have_flag(argc, argv, "context") || have_flag(argc, argv, "sign")) run_benchmark("context_sign", bench_context_sign, bench_setup, NULL, &data, 10, 200); + + return 0; +} diff --git a/src/secp256k1/src/bench_recover.c b/src/secp256k1/src/bench_recover.c new file mode 100644 index 00000000..6489378c --- /dev/null +++ b/src/secp256k1/src/bench_recover.c @@ -0,0 +1,60 @@ +/********************************************************************** + * Copyright (c) 2014-2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#include "include/secp256k1.h" +#include "include/secp256k1_recovery.h" +#include "util.h" +#include "bench.h" + +typedef struct { + secp256k1_context *ctx; + unsigned char msg[32]; + unsigned char sig[64]; +} bench_recover_t; + +void bench_recover(void* arg) { + int i; + bench_recover_t *data = (bench_recover_t*)arg; + secp256k1_pubkey pubkey; + unsigned char pubkeyc[33]; + + for (i = 0; i < 20000; i++) { + int j; + size_t pubkeylen = 33; + secp256k1_ecdsa_recoverable_signature sig; + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(data->ctx, &sig, data->sig, i % 2)); + CHECK(secp256k1_ecdsa_recover(data->ctx, &pubkey, &sig, data->msg)); + CHECK(secp256k1_ec_pubkey_serialize(data->ctx, pubkeyc, &pubkeylen, &pubkey, SECP256K1_EC_COMPRESSED)); + for (j = 0; j < 32; j++) { + data->sig[j + 32] = data->msg[j]; /* Move former message to S. */ + data->msg[j] = data->sig[j]; /* Move former R to message. */ + data->sig[j] = pubkeyc[j + 1]; /* Move recovered pubkey X coordinate to R (which must be a valid X coordinate). */ + } + } +} + +void bench_recover_setup(void* arg) { + int i; + bench_recover_t *data = (bench_recover_t*)arg; + + for (i = 0; i < 32; i++) { + data->msg[i] = 1 + i; + } + for (i = 0; i < 64; i++) { + data->sig[i] = 65 + i; + } +} + +int main(void) { + bench_recover_t data; + + data.ctx = secp256k1_context_create(SECP256K1_CONTEXT_VERIFY); + + run_benchmark("ecdsa_recover", bench_recover, bench_recover_setup, NULL, &data, 10, 20000); + + secp256k1_context_destroy(data.ctx); + return 0; +} diff --git a/src/secp256k1/src/bench_schnorr_verify.c b/src/secp256k1/src/bench_schnorr_verify.c new file mode 100644 index 00000000..5f137dda --- /dev/null +++ b/src/secp256k1/src/bench_schnorr_verify.c @@ -0,0 +1,73 @@ +/********************************************************************** + * Copyright (c) 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#include +#include + +#include "include/secp256k1.h" +#include "include/secp256k1_schnorr.h" +#include "util.h" +#include "bench.h" + +typedef struct { + unsigned char key[32]; + unsigned char sig[64]; + unsigned char pubkey[33]; + size_t pubkeylen; +} benchmark_schnorr_sig_t; + +typedef struct { + secp256k1_context *ctx; + unsigned char msg[32]; + benchmark_schnorr_sig_t sigs[64]; + int numsigs; +} benchmark_schnorr_verify_t; + +static void benchmark_schnorr_init(void* arg) { + int i, k; + benchmark_schnorr_verify_t* data = (benchmark_schnorr_verify_t*)arg; + + for (i = 0; i < 32; i++) { + data->msg[i] = 1 + i; + } + for (k = 0; k < data->numsigs; k++) { + secp256k1_pubkey pubkey; + for (i = 0; i < 32; i++) { + data->sigs[k].key[i] = 33 + i + k; + } + secp256k1_schnorr_sign(data->ctx, data->sigs[k].sig, data->msg, data->sigs[k].key, NULL, NULL); + data->sigs[k].pubkeylen = 33; + CHECK(secp256k1_ec_pubkey_create(data->ctx, &pubkey, data->sigs[k].key)); + CHECK(secp256k1_ec_pubkey_serialize(data->ctx, data->sigs[k].pubkey, &data->sigs[k].pubkeylen, &pubkey, SECP256K1_EC_COMPRESSED)); + } +} + +static void benchmark_schnorr_verify(void* arg) { + int i; + benchmark_schnorr_verify_t* data = (benchmark_schnorr_verify_t*)arg; + + for (i = 0; i < 20000 / data->numsigs; i++) { + secp256k1_pubkey pubkey; + data->sigs[0].sig[(i >> 8) % 64] ^= (i & 0xFF); + CHECK(secp256k1_ec_pubkey_parse(data->ctx, &pubkey, data->sigs[0].pubkey, data->sigs[0].pubkeylen)); + CHECK(secp256k1_schnorr_verify(data->ctx, data->sigs[0].sig, data->msg, &pubkey) == ((i & 0xFF) == 0)); + data->sigs[0].sig[(i >> 8) % 64] ^= (i & 0xFF); + } +} + + + +int main(void) { + benchmark_schnorr_verify_t data; + + data.ctx = secp256k1_context_create(SECP256K1_CONTEXT_SIGN | SECP256K1_CONTEXT_VERIFY); + + data.numsigs = 1; + run_benchmark("schnorr_verify", benchmark_schnorr_verify, benchmark_schnorr_init, NULL, &data, 10, 20000); + + secp256k1_context_destroy(data.ctx); + return 0; +} diff --git a/src/secp256k1/src/bench_sign.c b/src/secp256k1/src/bench_sign.c new file mode 100644 index 00000000..ed7224d7 --- /dev/null +++ b/src/secp256k1/src/bench_sign.c @@ -0,0 +1,56 @@ +/********************************************************************** + * Copyright (c) 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#include "include/secp256k1.h" +#include "util.h" +#include "bench.h" + +typedef struct { + secp256k1_context* ctx; + unsigned char msg[32]; + unsigned char key[32]; +} bench_sign_t; + +static void bench_sign_setup(void* arg) { + int i; + bench_sign_t *data = (bench_sign_t*)arg; + + for (i = 0; i < 32; i++) { + data->msg[i] = i + 1; + } + for (i = 0; i < 32; i++) { + data->key[i] = i + 65; + } +} + +static void bench_sign(void* arg) { + int i; + bench_sign_t *data = (bench_sign_t*)arg; + + unsigned char sig[74]; + for (i = 0; i < 20000; i++) { + size_t siglen = 74; + int j; + secp256k1_ecdsa_signature signature; + CHECK(secp256k1_ecdsa_sign(data->ctx, &signature, data->msg, data->key, NULL, NULL)); + CHECK(secp256k1_ecdsa_signature_serialize_der(data->ctx, sig, &siglen, &signature)); + for (j = 0; j < 32; j++) { + data->msg[j] = sig[j]; + data->key[j] = sig[j + 32]; + } + } +} + +int main(void) { + bench_sign_t data; + + data.ctx = secp256k1_context_create(SECP256K1_CONTEXT_SIGN); + + run_benchmark("ecdsa_sign", bench_sign, bench_sign_setup, NULL, &data, 10, 20000); + + secp256k1_context_destroy(data.ctx); + return 0; +} diff --git a/src/secp256k1/src/bench_verify.c b/src/secp256k1/src/bench_verify.c new file mode 100644 index 00000000..5718320c --- /dev/null +++ b/src/secp256k1/src/bench_verify.c @@ -0,0 +1,68 @@ +/********************************************************************** + * Copyright (c) 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#include +#include + +#include "include/secp256k1.h" +#include "util.h" +#include "bench.h" + +typedef struct { + secp256k1_context *ctx; + unsigned char msg[32]; + unsigned char key[32]; + unsigned char sig[72]; + size_t siglen; + unsigned char pubkey[33]; + size_t pubkeylen; +} benchmark_verify_t; + +static void benchmark_verify(void* arg) { + int i; + benchmark_verify_t* data = (benchmark_verify_t*)arg; + + for (i = 0; i < 20000; i++) { + secp256k1_pubkey pubkey; + secp256k1_ecdsa_signature sig; + data->sig[data->siglen - 1] ^= (i & 0xFF); + data->sig[data->siglen - 2] ^= ((i >> 8) & 0xFF); + data->sig[data->siglen - 3] ^= ((i >> 16) & 0xFF); + CHECK(secp256k1_ec_pubkey_parse(data->ctx, &pubkey, data->pubkey, data->pubkeylen) == 1); + CHECK(secp256k1_ecdsa_signature_parse_der(data->ctx, &sig, data->sig, data->siglen) == 1); + CHECK(secp256k1_ecdsa_verify(data->ctx, &sig, data->msg, &pubkey) == (i == 0)); + data->sig[data->siglen - 1] ^= (i & 0xFF); + data->sig[data->siglen - 2] ^= ((i >> 8) & 0xFF); + data->sig[data->siglen - 3] ^= ((i >> 16) & 0xFF); + } +} + +int main(void) { + int i; + secp256k1_pubkey pubkey; + secp256k1_ecdsa_signature sig; + benchmark_verify_t data; + + data.ctx = secp256k1_context_create(SECP256K1_CONTEXT_SIGN | SECP256K1_CONTEXT_VERIFY); + + for (i = 0; i < 32; i++) { + data.msg[i] = 1 + i; + } + for (i = 0; i < 32; i++) { + data.key[i] = 33 + i; + } + data.siglen = 72; + CHECK(secp256k1_ecdsa_sign(data.ctx, &sig, data.msg, data.key, NULL, NULL)); + CHECK(secp256k1_ecdsa_signature_serialize_der(data.ctx, data.sig, &data.siglen, &sig)); + CHECK(secp256k1_ec_pubkey_create(data.ctx, &pubkey, data.key)); + data.pubkeylen = 33; + CHECK(secp256k1_ec_pubkey_serialize(data.ctx, data.pubkey, &data.pubkeylen, &pubkey, SECP256K1_EC_COMPRESSED) == 1); + + run_benchmark("ecdsa_verify", benchmark_verify, NULL, NULL, &data, 10, 20000); + + secp256k1_context_destroy(data.ctx); + return 0; +} diff --git a/src/secp256k1/src/ecdsa.h b/src/secp256k1/src/ecdsa.h new file mode 100644 index 00000000..54ae101b --- /dev/null +++ b/src/secp256k1/src/ecdsa.h @@ -0,0 +1,21 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_ECDSA_ +#define _SECP256K1_ECDSA_ + +#include + +#include "scalar.h" +#include "group.h" +#include "ecmult.h" + +static int secp256k1_ecdsa_sig_parse(secp256k1_scalar *r, secp256k1_scalar *s, const unsigned char *sig, size_t size); +static int secp256k1_ecdsa_sig_serialize(unsigned char *sig, size_t *size, const secp256k1_scalar *r, const secp256k1_scalar *s); +static int secp256k1_ecdsa_sig_verify(const secp256k1_ecmult_context *ctx, const secp256k1_scalar* r, const secp256k1_scalar* s, const secp256k1_ge *pubkey, const secp256k1_scalar *message); +static int secp256k1_ecdsa_sig_sign(const secp256k1_ecmult_gen_context *ctx, secp256k1_scalar* r, secp256k1_scalar* s, const secp256k1_scalar *seckey, const secp256k1_scalar *message, const secp256k1_scalar *nonce, int *recid); + +#endif diff --git a/src/secp256k1/src/ecdsa_impl.h b/src/secp256k1/src/ecdsa_impl.h new file mode 100644 index 00000000..d110b4bb --- /dev/null +++ b/src/secp256k1/src/ecdsa_impl.h @@ -0,0 +1,303 @@ +/********************************************************************** + * Copyright (c) 2013-2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + + +#ifndef _SECP256K1_ECDSA_IMPL_H_ +#define _SECP256K1_ECDSA_IMPL_H_ + +#include "scalar.h" +#include "field.h" +#include "group.h" +#include "ecmult.h" +#include "ecmult_gen.h" +#include "ecdsa.h" + +/** Group order for secp256k1 defined as 'n' in "Standards for Efficient Cryptography" (SEC2) 2.7.1 + * sage: for t in xrange(1023, -1, -1): + * .. p = 2**256 - 2**32 - t + * .. if p.is_prime(): + * .. print '%x'%p + * .. break + * 'fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f' + * sage: a = 0 + * sage: b = 7 + * sage: F = FiniteField (p) + * sage: '%x' % (EllipticCurve ([F (a), F (b)]).order()) + * 'fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141' + */ +static const secp256k1_fe secp256k1_ecdsa_const_order_as_fe = SECP256K1_FE_CONST( + 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFEUL, + 0xBAAEDCE6UL, 0xAF48A03BUL, 0xBFD25E8CUL, 0xD0364141UL +); + +/** Difference between field and order, values 'p' and 'n' values defined in + * "Standards for Efficient Cryptography" (SEC2) 2.7.1. + * sage: p = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F + * sage: a = 0 + * sage: b = 7 + * sage: F = FiniteField (p) + * sage: '%x' % (p - EllipticCurve ([F (a), F (b)]).order()) + * '14551231950b75fc4402da1722fc9baee' + */ +static const secp256k1_fe secp256k1_ecdsa_const_p_minus_order = SECP256K1_FE_CONST( + 0, 0, 0, 1, 0x45512319UL, 0x50B75FC4UL, 0x402DA172UL, 0x2FC9BAEEUL +); + +static int secp256k1_der_read_len(const unsigned char **sigp, const unsigned char *sigend) { + int lenleft, b1; + size_t ret = 0; + if (*sigp >= sigend) { + return -1; + } + b1 = *((*sigp)++); + if (b1 == 0xFF) { + /* X.690-0207 8.1.3.5.c the value 0xFF shall not be used. */ + return -1; + } + if ((b1 & 0x80) == 0) { + /* X.690-0207 8.1.3.4 short form length octets */ + return b1; + } + if (b1 == 0x80) { + /* Indefinite length is not allowed in DER. */ + return -1; + } + /* X.690-207 8.1.3.5 long form length octets */ + lenleft = b1 & 0x7F; + if (lenleft > sigend - *sigp) { + return -1; + } + if (**sigp == 0) { + /* Not the shortest possible length encoding. */ + return -1; + } + if ((size_t)lenleft > sizeof(size_t)) { + /* The resulting length would exceed the range of a size_t, so + * certainly longer than the passed array size. + */ + return -1; + } + while (lenleft > 0) { + if ((ret >> ((sizeof(size_t) - 1) * 8)) != 0) { + } + ret = (ret << 8) | **sigp; + if (ret + lenleft > (size_t)(sigend - *sigp)) { + /* Result exceeds the length of the passed array. */ + return -1; + } + (*sigp)++; + lenleft--; + } + if (ret < 128) { + /* Not the shortest possible length encoding. */ + return -1; + } + return ret; +} + +static int secp256k1_der_parse_integer(secp256k1_scalar *r, const unsigned char **sig, const unsigned char *sigend) { + int overflow = 0; + unsigned char ra[32] = {0}; + int rlen; + + if (*sig == sigend || **sig != 0x02) { + /* Not a primitive integer (X.690-0207 8.3.1). */ + return 0; + } + (*sig)++; + rlen = secp256k1_der_read_len(sig, sigend); + if (rlen <= 0 || (*sig) + rlen > sigend) { + /* Exceeds bounds or not at least length 1 (X.690-0207 8.3.1). */ + return 0; + } + if (**sig == 0x00 && rlen > 1 && (((*sig)[1]) & 0x80) == 0x00) { + /* Excessive 0x00 padding. */ + return 0; + } + if (**sig == 0xFF && rlen > 1 && (((*sig)[1]) & 0x80) == 0x80) { + /* Excessive 0xFF padding. */ + return 0; + } + if ((**sig & 0x80) == 0x80) { + /* Negative. */ + overflow = 1; + } + while (rlen > 0 && **sig == 0) { + /* Skip leading zero bytes */ + rlen--; + (*sig)++; + } + if (rlen > 32) { + overflow = 1; + } + if (!overflow) { + memcpy(ra + 32 - rlen, *sig, rlen); + secp256k1_scalar_set_b32(r, ra, &overflow); + } + if (overflow) { + secp256k1_scalar_set_int(r, 0); + } + (*sig) += rlen; + return 1; +} + +static int secp256k1_ecdsa_sig_parse(secp256k1_scalar *rr, secp256k1_scalar *rs, const unsigned char *sig, size_t size) { + const unsigned char *sigend = sig + size; + int rlen; + if (sig == sigend || *(sig++) != 0x30) { + /* The encoding doesn't start with a constructed sequence (X.690-0207 8.9.1). */ + return 0; + } + rlen = secp256k1_der_read_len(&sig, sigend); + if (rlen < 0 || sig + rlen > sigend) { + /* Tuple exceeds bounds */ + return 0; + } + if (sig + rlen != sigend) { + /* Garbage after tuple. */ + return 0; + } + + if (!secp256k1_der_parse_integer(rr, &sig, sigend)) { + return 0; + } + if (!secp256k1_der_parse_integer(rs, &sig, sigend)) { + return 0; + } + + if (sig != sigend) { + /* Trailing garbage inside tuple. */ + return 0; + } + + return 1; +} + +static int secp256k1_ecdsa_sig_serialize(unsigned char *sig, size_t *size, const secp256k1_scalar* ar, const secp256k1_scalar* as) { + unsigned char r[33] = {0}, s[33] = {0}; + unsigned char *rp = r, *sp = s; + size_t lenR = 33, lenS = 33; + secp256k1_scalar_get_b32(&r[1], ar); + secp256k1_scalar_get_b32(&s[1], as); + while (lenR > 1 && rp[0] == 0 && rp[1] < 0x80) { lenR--; rp++; } + while (lenS > 1 && sp[0] == 0 && sp[1] < 0x80) { lenS--; sp++; } + if (*size < 6+lenS+lenR) { + *size = 6 + lenS + lenR; + return 0; + } + *size = 6 + lenS + lenR; + sig[0] = 0x30; + sig[1] = 4 + lenS + lenR; + sig[2] = 0x02; + sig[3] = lenR; + memcpy(sig+4, rp, lenR); + sig[4+lenR] = 0x02; + sig[5+lenR] = lenS; + memcpy(sig+lenR+6, sp, lenS); + return 1; +} + +static int secp256k1_ecdsa_sig_verify(const secp256k1_ecmult_context *ctx, const secp256k1_scalar *sigr, const secp256k1_scalar *sigs, const secp256k1_ge *pubkey, const secp256k1_scalar *message) { + unsigned char c[32]; + secp256k1_scalar sn, u1, u2; + secp256k1_fe xr; + secp256k1_gej pubkeyj; + secp256k1_gej pr; + + if (secp256k1_scalar_is_zero(sigr) || secp256k1_scalar_is_zero(sigs)) { + return 0; + } + + secp256k1_scalar_inverse_var(&sn, sigs); + secp256k1_scalar_mul(&u1, &sn, message); + secp256k1_scalar_mul(&u2, &sn, sigr); + secp256k1_gej_set_ge(&pubkeyj, pubkey); + secp256k1_ecmult(ctx, &pr, &pubkeyj, &u2, &u1); + if (secp256k1_gej_is_infinity(&pr)) { + return 0; + } + secp256k1_scalar_get_b32(c, sigr); + secp256k1_fe_set_b32(&xr, c); + + /** We now have the recomputed R point in pr, and its claimed x coordinate (modulo n) + * in xr. Naively, we would extract the x coordinate from pr (requiring a inversion modulo p), + * compute the remainder modulo n, and compare it to xr. However: + * + * xr == X(pr) mod n + * <=> exists h. (xr + h * n < p && xr + h * n == X(pr)) + * [Since 2 * n > p, h can only be 0 or 1] + * <=> (xr == X(pr)) || (xr + n < p && xr + n == X(pr)) + * [In Jacobian coordinates, X(pr) is pr.x / pr.z^2 mod p] + * <=> (xr == pr.x / pr.z^2 mod p) || (xr + n < p && xr + n == pr.x / pr.z^2 mod p) + * [Multiplying both sides of the equations by pr.z^2 mod p] + * <=> (xr * pr.z^2 mod p == pr.x) || (xr + n < p && (xr + n) * pr.z^2 mod p == pr.x) + * + * Thus, we can avoid the inversion, but we have to check both cases separately. + * secp256k1_gej_eq_x implements the (xr * pr.z^2 mod p == pr.x) test. + */ + if (secp256k1_gej_eq_x_var(&xr, &pr)) { + /* xr * pr.z^2 mod p == pr.x, so the signature is valid. */ + return 1; + } + if (secp256k1_fe_cmp_var(&xr, &secp256k1_ecdsa_const_p_minus_order) >= 0) { + /* xr + n >= p, so we can skip testing the second case. */ + return 0; + } + secp256k1_fe_add(&xr, &secp256k1_ecdsa_const_order_as_fe); + if (secp256k1_gej_eq_x_var(&xr, &pr)) { + /* (xr + n) * pr.z^2 mod p == pr.x, so the signature is valid. */ + return 1; + } + return 0; +} + +static int secp256k1_ecdsa_sig_sign(const secp256k1_ecmult_gen_context *ctx, secp256k1_scalar *sigr, secp256k1_scalar *sigs, const secp256k1_scalar *seckey, const secp256k1_scalar *message, const secp256k1_scalar *nonce, int *recid) { + unsigned char b[32]; + secp256k1_gej rp; + secp256k1_ge r; + secp256k1_scalar n; + int overflow = 0; + + secp256k1_ecmult_gen(ctx, &rp, nonce); + secp256k1_ge_set_gej(&r, &rp); + secp256k1_fe_normalize(&r.x); + secp256k1_fe_normalize(&r.y); + secp256k1_fe_get_b32(b, &r.x); + secp256k1_scalar_set_b32(sigr, b, &overflow); + if (secp256k1_scalar_is_zero(sigr)) { + /* P.x = order is on the curve, so technically sig->r could end up zero, which would be an invalid signature. + * This branch is cryptographically unreachable as hitting it requires finding the discrete log of P.x = N. + */ + secp256k1_gej_clear(&rp); + secp256k1_ge_clear(&r); + return 0; + } + if (recid) { + /* The overflow condition is cryptographically unreachable as hitting it requires finding the discrete log + * of some P where P.x >= order, and only 1 in about 2^127 points meet this criteria. + */ + *recid = (overflow ? 2 : 0) | (secp256k1_fe_is_odd(&r.y) ? 1 : 0); + } + secp256k1_scalar_mul(&n, sigr, seckey); + secp256k1_scalar_add(&n, &n, message); + secp256k1_scalar_inverse(sigs, nonce); + secp256k1_scalar_mul(sigs, sigs, &n); + secp256k1_scalar_clear(&n); + secp256k1_gej_clear(&rp); + secp256k1_ge_clear(&r); + if (secp256k1_scalar_is_zero(sigs)) { + return 0; + } + if (secp256k1_scalar_is_high(sigs)) { + secp256k1_scalar_negate(sigs, sigs); + if (recid) { + *recid ^= 1; + } + } + return 1; +} + +#endif diff --git a/src/secp256k1/src/eckey.h b/src/secp256k1/src/eckey.h new file mode 100644 index 00000000..42739a3b --- /dev/null +++ b/src/secp256k1/src/eckey.h @@ -0,0 +1,25 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_ECKEY_ +#define _SECP256K1_ECKEY_ + +#include + +#include "group.h" +#include "scalar.h" +#include "ecmult.h" +#include "ecmult_gen.h" + +static int secp256k1_eckey_pubkey_parse(secp256k1_ge *elem, const unsigned char *pub, size_t size); +static int secp256k1_eckey_pubkey_serialize(secp256k1_ge *elem, unsigned char *pub, size_t *size, int compressed); + +static int secp256k1_eckey_privkey_tweak_add(secp256k1_scalar *key, const secp256k1_scalar *tweak); +static int secp256k1_eckey_pubkey_tweak_add(const secp256k1_ecmult_context *ctx, secp256k1_ge *key, const secp256k1_scalar *tweak); +static int secp256k1_eckey_privkey_tweak_mul(secp256k1_scalar *key, const secp256k1_scalar *tweak); +static int secp256k1_eckey_pubkey_tweak_mul(const secp256k1_ecmult_context *ctx, secp256k1_ge *key, const secp256k1_scalar *tweak); + +#endif diff --git a/src/secp256k1/src/eckey_impl.h b/src/secp256k1/src/eckey_impl.h new file mode 100644 index 00000000..ce38071a --- /dev/null +++ b/src/secp256k1/src/eckey_impl.h @@ -0,0 +1,99 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_ECKEY_IMPL_H_ +#define _SECP256K1_ECKEY_IMPL_H_ + +#include "eckey.h" + +#include "scalar.h" +#include "field.h" +#include "group.h" +#include "ecmult_gen.h" + +static int secp256k1_eckey_pubkey_parse(secp256k1_ge *elem, const unsigned char *pub, size_t size) { + if (size == 33 && (pub[0] == 0x02 || pub[0] == 0x03)) { + secp256k1_fe x; + return secp256k1_fe_set_b32(&x, pub+1) && secp256k1_ge_set_xo_var(elem, &x, pub[0] == 0x03); + } else if (size == 65 && (pub[0] == 0x04 || pub[0] == 0x06 || pub[0] == 0x07)) { + secp256k1_fe x, y; + if (!secp256k1_fe_set_b32(&x, pub+1) || !secp256k1_fe_set_b32(&y, pub+33)) { + return 0; + } + secp256k1_ge_set_xy(elem, &x, &y); + if ((pub[0] == 0x06 || pub[0] == 0x07) && secp256k1_fe_is_odd(&y) != (pub[0] == 0x07)) { + return 0; + } + return secp256k1_ge_is_valid_var(elem); + } else { + return 0; + } +} + +static int secp256k1_eckey_pubkey_serialize(secp256k1_ge *elem, unsigned char *pub, size_t *size, int compressed) { + if (secp256k1_ge_is_infinity(elem)) { + return 0; + } + secp256k1_fe_normalize_var(&elem->x); + secp256k1_fe_normalize_var(&elem->y); + secp256k1_fe_get_b32(&pub[1], &elem->x); + if (compressed) { + *size = 33; + pub[0] = 0x02 | (secp256k1_fe_is_odd(&elem->y) ? 0x01 : 0x00); + } else { + *size = 65; + pub[0] = 0x04; + secp256k1_fe_get_b32(&pub[33], &elem->y); + } + return 1; +} + +static int secp256k1_eckey_privkey_tweak_add(secp256k1_scalar *key, const secp256k1_scalar *tweak) { + secp256k1_scalar_add(key, key, tweak); + if (secp256k1_scalar_is_zero(key)) { + return 0; + } + return 1; +} + +static int secp256k1_eckey_pubkey_tweak_add(const secp256k1_ecmult_context *ctx, secp256k1_ge *key, const secp256k1_scalar *tweak) { + secp256k1_gej pt; + secp256k1_scalar one; + secp256k1_gej_set_ge(&pt, key); + secp256k1_scalar_set_int(&one, 1); + secp256k1_ecmult(ctx, &pt, &pt, &one, tweak); + + if (secp256k1_gej_is_infinity(&pt)) { + return 0; + } + secp256k1_ge_set_gej(key, &pt); + return 1; +} + +static int secp256k1_eckey_privkey_tweak_mul(secp256k1_scalar *key, const secp256k1_scalar *tweak) { + if (secp256k1_scalar_is_zero(tweak)) { + return 0; + } + + secp256k1_scalar_mul(key, key, tweak); + return 1; +} + +static int secp256k1_eckey_pubkey_tweak_mul(const secp256k1_ecmult_context *ctx, secp256k1_ge *key, const secp256k1_scalar *tweak) { + secp256k1_scalar zero; + secp256k1_gej pt; + if (secp256k1_scalar_is_zero(tweak)) { + return 0; + } + + secp256k1_scalar_set_int(&zero, 0); + secp256k1_gej_set_ge(&pt, key); + secp256k1_ecmult(ctx, &pt, &pt, tweak, &zero); + secp256k1_ge_set_gej(key, &pt); + return 1; +} + +#endif diff --git a/src/secp256k1/src/ecmult.h b/src/secp256k1/src/ecmult.h new file mode 100644 index 00000000..20484134 --- /dev/null +++ b/src/secp256k1/src/ecmult.h @@ -0,0 +1,31 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_ECMULT_ +#define _SECP256K1_ECMULT_ + +#include "num.h" +#include "group.h" + +typedef struct { + /* For accelerating the computation of a*P + b*G: */ + secp256k1_ge_storage (*pre_g)[]; /* odd multiples of the generator */ +#ifdef USE_ENDOMORPHISM + secp256k1_ge_storage (*pre_g_128)[]; /* odd multiples of 2^128*generator */ +#endif +} secp256k1_ecmult_context; + +static void secp256k1_ecmult_context_init(secp256k1_ecmult_context *ctx); +static void secp256k1_ecmult_context_build(secp256k1_ecmult_context *ctx, const secp256k1_callback *cb); +static void secp256k1_ecmult_context_clone(secp256k1_ecmult_context *dst, + const secp256k1_ecmult_context *src, const secp256k1_callback *cb); +static void secp256k1_ecmult_context_clear(secp256k1_ecmult_context *ctx); +static int secp256k1_ecmult_context_is_built(const secp256k1_ecmult_context *ctx); + +/** Double multiply: R = na*A + ng*G */ +static void secp256k1_ecmult(const secp256k1_ecmult_context *ctx, secp256k1_gej *r, const secp256k1_gej *a, const secp256k1_scalar *na, const secp256k1_scalar *ng); + +#endif diff --git a/src/secp256k1/src/ecmult_const.h b/src/secp256k1/src/ecmult_const.h new file mode 100644 index 00000000..2b009765 --- /dev/null +++ b/src/secp256k1/src/ecmult_const.h @@ -0,0 +1,15 @@ +/********************************************************************** + * Copyright (c) 2015 Andrew Poelstra * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_ECMULT_CONST_ +#define _SECP256K1_ECMULT_CONST_ + +#include "scalar.h" +#include "group.h" + +static void secp256k1_ecmult_const(secp256k1_gej *r, const secp256k1_ge *a, const secp256k1_scalar *q); + +#endif diff --git a/src/secp256k1/src/ecmult_const_impl.h b/src/secp256k1/src/ecmult_const_impl.h new file mode 100644 index 00000000..90ac9477 --- /dev/null +++ b/src/secp256k1/src/ecmult_const_impl.h @@ -0,0 +1,260 @@ +/********************************************************************** + * Copyright (c) 2015 Pieter Wuille, Andrew Poelstra * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_ECMULT_CONST_IMPL_ +#define _SECP256K1_ECMULT_CONST_IMPL_ + +#include "scalar.h" +#include "group.h" +#include "ecmult_const.h" +#include "ecmult_impl.h" + +#ifdef USE_ENDOMORPHISM + #define WNAF_BITS 128 +#else + #define WNAF_BITS 256 +#endif +#define WNAF_SIZE(w) ((WNAF_BITS + (w) - 1) / (w)) + +/* This is like `ECMULT_TABLE_GET_GE` but is constant time */ +#define ECMULT_CONST_TABLE_GET_GE(r,pre,n,w) do { \ + int m; \ + int abs_n = (n) * (((n) > 0) * 2 - 1); \ + int idx_n = abs_n / 2; \ + secp256k1_fe neg_y; \ + VERIFY_CHECK(((n) & 1) == 1); \ + VERIFY_CHECK((n) >= -((1 << ((w)-1)) - 1)); \ + VERIFY_CHECK((n) <= ((1 << ((w)-1)) - 1)); \ + VERIFY_SETUP(secp256k1_fe_clear(&(r)->x)); \ + VERIFY_SETUP(secp256k1_fe_clear(&(r)->y)); \ + for (m = 0; m < ECMULT_TABLE_SIZE(w); m++) { \ + /* This loop is used to avoid secret data in array indices. See + * the comment in ecmult_gen_impl.h for rationale. */ \ + secp256k1_fe_cmov(&(r)->x, &(pre)[m].x, m == idx_n); \ + secp256k1_fe_cmov(&(r)->y, &(pre)[m].y, m == idx_n); \ + } \ + (r)->infinity = 0; \ + secp256k1_fe_negate(&neg_y, &(r)->y, 1); \ + secp256k1_fe_cmov(&(r)->y, &neg_y, (n) != abs_n); \ +} while(0) + + +/** Convert a number to WNAF notation. The number becomes represented by sum(2^{wi} * wnaf[i], i=0..return_val) + * with the following guarantees: + * - each wnaf[i] an odd integer between -(1 << w) and (1 << w) + * - each wnaf[i] is nonzero + * - the number of words set is returned; this is always (WNAF_BITS + w - 1) / w + * + * Adapted from `The Width-w NAF Method Provides Small Memory and Fast Elliptic Scalar + * Multiplications Secure against Side Channel Attacks`, Okeya and Tagaki. M. Joye (Ed.) + * CT-RSA 2003, LNCS 2612, pp. 328-443, 2003. Springer-Verlagy Berlin Heidelberg 2003 + * + * Numbers reference steps of `Algorithm SPA-resistant Width-w NAF with Odd Scalar` on pp. 335 + */ +static int secp256k1_wnaf_const(int *wnaf, secp256k1_scalar s, int w) { + int global_sign; + int skew = 0; + int word = 0; + /* 1 2 3 */ + int u_last; + int u; + +#ifdef USE_ENDOMORPHISM + int flip; + int bit; + secp256k1_scalar neg_s; + int not_neg_one; + /* If we are using the endomorphism, we cannot handle even numbers by negating + * them, since we are working with 128-bit numbers whose negations would be 256 + * bits, eliminating the performance advantage. Instead we use a technique from + * Section 4.2 of the Okeya/Tagaki paper, which is to add either 1 (for even) + * or 2 (for odd) to the number we are encoding, then compensating after the + * multiplication. */ + /* Negative 128-bit numbers will be negated, since otherwise they are 256-bit */ + flip = secp256k1_scalar_is_high(&s); + /* We add 1 to even numbers, 2 to odd ones, noting that negation flips parity */ + bit = flip ^ (s.d[0] & 1); + /* We check for negative one, since adding 2 to it will cause an overflow */ + secp256k1_scalar_negate(&neg_s, &s); + not_neg_one = !secp256k1_scalar_is_one(&neg_s); + secp256k1_scalar_cadd_bit(&s, bit, not_neg_one); + /* If we had negative one, flip == 1, s.d[0] == 0, bit == 1, so caller expects + * that we added two to it and flipped it. In fact for -1 these operations are + * identical. We only flipped, but since skewing is required (in the sense that + * the skew must be 1 or 2, never zero) and flipping is not, we need to change + * our flags to claim that we only skewed. */ + global_sign = secp256k1_scalar_cond_negate(&s, flip); + global_sign *= not_neg_one * 2 - 1; + skew = 1 << bit; +#else + /* Otherwise, we just negate to force oddness */ + int is_even = secp256k1_scalar_is_even(&s); + global_sign = secp256k1_scalar_cond_negate(&s, is_even); +#endif + + /* 4 */ + u_last = secp256k1_scalar_shr_int(&s, w); + while (word * w < WNAF_BITS) { + int sign; + int even; + + /* 4.1 4.4 */ + u = secp256k1_scalar_shr_int(&s, w); + /* 4.2 */ + even = ((u & 1) == 0); + sign = 2 * (u_last > 0) - 1; + u += sign * even; + u_last -= sign * even * (1 << w); + + /* 4.3, adapted for global sign change */ + wnaf[word++] = u_last * global_sign; + + u_last = u; + } + wnaf[word] = u * global_sign; + + VERIFY_CHECK(secp256k1_scalar_is_zero(&s)); + VERIFY_CHECK(word == WNAF_SIZE(w)); + return skew; +} + + +static void secp256k1_ecmult_const(secp256k1_gej *r, const secp256k1_ge *a, const secp256k1_scalar *scalar) { + secp256k1_ge pre_a[ECMULT_TABLE_SIZE(WINDOW_A)]; + secp256k1_ge tmpa; + secp256k1_fe Z; + +#ifdef USE_ENDOMORPHISM + secp256k1_ge pre_a_lam[ECMULT_TABLE_SIZE(WINDOW_A)]; + int wnaf_1[1 + WNAF_SIZE(WINDOW_A - 1)]; + int wnaf_lam[1 + WNAF_SIZE(WINDOW_A - 1)]; + int skew_1; + int skew_lam; + secp256k1_scalar q_1, q_lam; +#else + int wnaf[1 + WNAF_SIZE(WINDOW_A - 1)]; +#endif + + int i; + secp256k1_scalar sc = *scalar; + + /* build wnaf representation for q. */ +#ifdef USE_ENDOMORPHISM + /* split q into q_1 and q_lam (where q = q_1 + q_lam*lambda, and q_1 and q_lam are ~128 bit) */ + secp256k1_scalar_split_lambda(&q_1, &q_lam, &sc); + /* no need for zero correction when using endomorphism since even + * numbers have one added to them anyway */ + skew_1 = secp256k1_wnaf_const(wnaf_1, q_1, WINDOW_A - 1); + skew_lam = secp256k1_wnaf_const(wnaf_lam, q_lam, WINDOW_A - 1); +#else + int is_zero = secp256k1_scalar_is_zero(scalar); + /* the wNAF ladder cannot handle zero, so bump this to one .. we will + * correct the result after the fact */ + sc.d[0] += is_zero; + VERIFY_CHECK(!secp256k1_scalar_is_zero(&sc)); + + secp256k1_wnaf_const(wnaf, sc, WINDOW_A - 1); +#endif + + /* Calculate odd multiples of a. + * All multiples are brought to the same Z 'denominator', which is stored + * in Z. Due to secp256k1' isomorphism we can do all operations pretending + * that the Z coordinate was 1, use affine addition formulae, and correct + * the Z coordinate of the result once at the end. + */ + secp256k1_gej_set_ge(r, a); + secp256k1_ecmult_odd_multiples_table_globalz_windowa(pre_a, &Z, r); + for (i = 0; i < ECMULT_TABLE_SIZE(WINDOW_A); i++) { + secp256k1_fe_normalize_weak(&pre_a[i].y); + } +#ifdef USE_ENDOMORPHISM + for (i = 0; i < ECMULT_TABLE_SIZE(WINDOW_A); i++) { + secp256k1_ge_mul_lambda(&pre_a_lam[i], &pre_a[i]); + } +#endif + + /* first loop iteration (separated out so we can directly set r, rather + * than having it start at infinity, get doubled several times, then have + * its new value added to it) */ +#ifdef USE_ENDOMORPHISM + i = wnaf_1[WNAF_SIZE(WINDOW_A - 1)]; + VERIFY_CHECK(i != 0); + ECMULT_CONST_TABLE_GET_GE(&tmpa, pre_a, i, WINDOW_A); + secp256k1_gej_set_ge(r, &tmpa); + + i = wnaf_lam[WNAF_SIZE(WINDOW_A - 1)]; + VERIFY_CHECK(i != 0); + ECMULT_CONST_TABLE_GET_GE(&tmpa, pre_a_lam, i, WINDOW_A); + secp256k1_gej_add_ge(r, r, &tmpa); +#else + i = wnaf[WNAF_SIZE(WINDOW_A - 1)]; + VERIFY_CHECK(i != 0); + ECMULT_CONST_TABLE_GET_GE(&tmpa, pre_a, i, WINDOW_A); + secp256k1_gej_set_ge(r, &tmpa); +#endif + /* remaining loop iterations */ + for (i = WNAF_SIZE(WINDOW_A - 1) - 1; i >= 0; i--) { + int n; + int j; + for (j = 0; j < WINDOW_A - 1; ++j) { + secp256k1_gej_double_nonzero(r, r, NULL); + } +#ifdef USE_ENDOMORPHISM + n = wnaf_1[i]; + ECMULT_CONST_TABLE_GET_GE(&tmpa, pre_a, n, WINDOW_A); + VERIFY_CHECK(n != 0); + secp256k1_gej_add_ge(r, r, &tmpa); + + n = wnaf_lam[i]; + ECMULT_CONST_TABLE_GET_GE(&tmpa, pre_a_lam, n, WINDOW_A); + VERIFY_CHECK(n != 0); + secp256k1_gej_add_ge(r, r, &tmpa); +#else + n = wnaf[i]; + VERIFY_CHECK(n != 0); + ECMULT_CONST_TABLE_GET_GE(&tmpa, pre_a, n, WINDOW_A); + secp256k1_gej_add_ge(r, r, &tmpa); +#endif + } + + secp256k1_fe_mul(&r->z, &r->z, &Z); + +#ifdef USE_ENDOMORPHISM + { + /* Correct for wNAF skew */ + secp256k1_ge correction = *a; + secp256k1_ge_storage correction_1_stor; + secp256k1_ge_storage correction_lam_stor; + secp256k1_ge_storage a2_stor; + secp256k1_gej tmpj; + secp256k1_gej_set_ge(&tmpj, &correction); + secp256k1_gej_double_var(&tmpj, &tmpj, NULL); + secp256k1_ge_set_gej(&correction, &tmpj); + secp256k1_ge_to_storage(&correction_1_stor, a); + secp256k1_ge_to_storage(&correction_lam_stor, a); + secp256k1_ge_to_storage(&a2_stor, &correction); + + /* For odd numbers this is 2a (so replace it), for even ones a (so no-op) */ + secp256k1_ge_storage_cmov(&correction_1_stor, &a2_stor, skew_1 == 2); + secp256k1_ge_storage_cmov(&correction_lam_stor, &a2_stor, skew_lam == 2); + + /* Apply the correction */ + secp256k1_ge_from_storage(&correction, &correction_1_stor); + secp256k1_ge_neg(&correction, &correction); + secp256k1_gej_add_ge(r, r, &correction); + + secp256k1_ge_from_storage(&correction, &correction_lam_stor); + secp256k1_ge_neg(&correction, &correction); + secp256k1_ge_mul_lambda(&correction, &correction); + secp256k1_gej_add_ge(r, r, &correction); + } +#else + /* correct for zero */ + r->infinity |= is_zero; +#endif +} + +#endif diff --git a/src/secp256k1/src/ecmult_gen.h b/src/secp256k1/src/ecmult_gen.h new file mode 100644 index 00000000..eb2cc9ea --- /dev/null +++ b/src/secp256k1/src/ecmult_gen.h @@ -0,0 +1,43 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_ECMULT_GEN_ +#define _SECP256K1_ECMULT_GEN_ + +#include "scalar.h" +#include "group.h" + +typedef struct { + /* For accelerating the computation of a*G: + * To harden against timing attacks, use the following mechanism: + * * Break up the multiplicand into groups of 4 bits, called n_0, n_1, n_2, ..., n_63. + * * Compute sum(n_i * 16^i * G + U_i, i=0..63), where: + * * U_i = U * 2^i (for i=0..62) + * * U_i = U * (1-2^63) (for i=63) + * where U is a point with no known corresponding scalar. Note that sum(U_i, i=0..63) = 0. + * For each i, and each of the 16 possible values of n_i, (n_i * 16^i * G + U_i) is + * precomputed (call it prec(i, n_i)). The formula now becomes sum(prec(i, n_i), i=0..63). + * None of the resulting prec group elements have a known scalar, and neither do any of + * the intermediate sums while computing a*G. + */ + secp256k1_ge_storage (*prec)[64][16]; /* prec[j][i] = 16^j * i * G + U_i */ + secp256k1_scalar blind; + secp256k1_gej initial; +} secp256k1_ecmult_gen_context; + +static void secp256k1_ecmult_gen_context_init(secp256k1_ecmult_gen_context* ctx); +static void secp256k1_ecmult_gen_context_build(secp256k1_ecmult_gen_context* ctx, const secp256k1_callback* cb); +static void secp256k1_ecmult_gen_context_clone(secp256k1_ecmult_gen_context *dst, + const secp256k1_ecmult_gen_context* src, const secp256k1_callback* cb); +static void secp256k1_ecmult_gen_context_clear(secp256k1_ecmult_gen_context* ctx); +static int secp256k1_ecmult_gen_context_is_built(const secp256k1_ecmult_gen_context* ctx); + +/** Multiply with the generator: R = a*G */ +static void secp256k1_ecmult_gen(const secp256k1_ecmult_gen_context* ctx, secp256k1_gej *r, const secp256k1_scalar *a); + +static void secp256k1_ecmult_gen_blind(secp256k1_ecmult_gen_context *ctx, const unsigned char *seed32); + +#endif diff --git a/src/secp256k1/src/ecmult_gen_impl.h b/src/secp256k1/src/ecmult_gen_impl.h new file mode 100644 index 00000000..b63c4d86 --- /dev/null +++ b/src/secp256k1/src/ecmult_gen_impl.h @@ -0,0 +1,210 @@ +/********************************************************************** + * Copyright (c) 2013, 2014, 2015 Pieter Wuille, Gregory Maxwell * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_ECMULT_GEN_IMPL_H_ +#define _SECP256K1_ECMULT_GEN_IMPL_H_ + +#include "scalar.h" +#include "group.h" +#include "ecmult_gen.h" +#include "hash_impl.h" +#ifdef USE_ECMULT_STATIC_PRECOMPUTATION +#include "ecmult_static_context.h" +#endif +static void secp256k1_ecmult_gen_context_init(secp256k1_ecmult_gen_context *ctx) { + ctx->prec = NULL; +} + +static void secp256k1_ecmult_gen_context_build(secp256k1_ecmult_gen_context *ctx, const secp256k1_callback* cb) { +#ifndef USE_ECMULT_STATIC_PRECOMPUTATION + secp256k1_ge prec[1024]; + secp256k1_gej gj; + secp256k1_gej nums_gej; + int i, j; +#endif + + if (ctx->prec != NULL) { + return; + } +#ifndef USE_ECMULT_STATIC_PRECOMPUTATION + ctx->prec = (secp256k1_ge_storage (*)[64][16])checked_malloc(cb, sizeof(*ctx->prec)); + + /* get the generator */ + secp256k1_gej_set_ge(&gj, &secp256k1_ge_const_g); + + /* Construct a group element with no known corresponding scalar (nothing up my sleeve). */ + { + static const unsigned char nums_b32[33] = "The scalar for this x is unknown"; + secp256k1_fe nums_x; + secp256k1_ge nums_ge; + int r; + r = secp256k1_fe_set_b32(&nums_x, nums_b32); + (void)r; + VERIFY_CHECK(r); + r = secp256k1_ge_set_xo_var(&nums_ge, &nums_x, 0); + (void)r; + VERIFY_CHECK(r); + secp256k1_gej_set_ge(&nums_gej, &nums_ge); + /* Add G to make the bits in x uniformly distributed. */ + secp256k1_gej_add_ge_var(&nums_gej, &nums_gej, &secp256k1_ge_const_g, NULL); + } + + /* compute prec. */ + { + secp256k1_gej precj[1024]; /* Jacobian versions of prec. */ + secp256k1_gej gbase; + secp256k1_gej numsbase; + gbase = gj; /* 16^j * G */ + numsbase = nums_gej; /* 2^j * nums. */ + for (j = 0; j < 64; j++) { + /* Set precj[j*16 .. j*16+15] to (numsbase, numsbase + gbase, ..., numsbase + 15*gbase). */ + precj[j*16] = numsbase; + for (i = 1; i < 16; i++) { + secp256k1_gej_add_var(&precj[j*16 + i], &precj[j*16 + i - 1], &gbase, NULL); + } + /* Multiply gbase by 16. */ + for (i = 0; i < 4; i++) { + secp256k1_gej_double_var(&gbase, &gbase, NULL); + } + /* Multiply numbase by 2. */ + secp256k1_gej_double_var(&numsbase, &numsbase, NULL); + if (j == 62) { + /* In the last iteration, numsbase is (1 - 2^j) * nums instead. */ + secp256k1_gej_neg(&numsbase, &numsbase); + secp256k1_gej_add_var(&numsbase, &numsbase, &nums_gej, NULL); + } + } + secp256k1_ge_set_all_gej_var(1024, prec, precj, cb); + } + for (j = 0; j < 64; j++) { + for (i = 0; i < 16; i++) { + secp256k1_ge_to_storage(&(*ctx->prec)[j][i], &prec[j*16 + i]); + } + } +#else + (void)cb; + ctx->prec = (secp256k1_ge_storage (*)[64][16])secp256k1_ecmult_static_context; +#endif + secp256k1_ecmult_gen_blind(ctx, NULL); +} + +static int secp256k1_ecmult_gen_context_is_built(const secp256k1_ecmult_gen_context* ctx) { + return ctx->prec != NULL; +} + +static void secp256k1_ecmult_gen_context_clone(secp256k1_ecmult_gen_context *dst, + const secp256k1_ecmult_gen_context *src, const secp256k1_callback* cb) { + if (src->prec == NULL) { + dst->prec = NULL; + } else { +#ifndef USE_ECMULT_STATIC_PRECOMPUTATION + dst->prec = (secp256k1_ge_storage (*)[64][16])checked_malloc(cb, sizeof(*dst->prec)); + memcpy(dst->prec, src->prec, sizeof(*dst->prec)); +#else + (void)cb; + dst->prec = src->prec; +#endif + dst->initial = src->initial; + dst->blind = src->blind; + } +} + +static void secp256k1_ecmult_gen_context_clear(secp256k1_ecmult_gen_context *ctx) { +#ifndef USE_ECMULT_STATIC_PRECOMPUTATION + free(ctx->prec); +#endif + secp256k1_scalar_clear(&ctx->blind); + secp256k1_gej_clear(&ctx->initial); + ctx->prec = NULL; +} + +static void secp256k1_ecmult_gen(const secp256k1_ecmult_gen_context *ctx, secp256k1_gej *r, const secp256k1_scalar *gn) { + secp256k1_ge add; + secp256k1_ge_storage adds; + secp256k1_scalar gnb; + int bits; + int i, j; + memset(&adds, 0, sizeof(adds)); + *r = ctx->initial; + /* Blind scalar/point multiplication by computing (n-b)G + bG instead of nG. */ + secp256k1_scalar_add(&gnb, gn, &ctx->blind); + add.infinity = 0; + for (j = 0; j < 64; j++) { + bits = secp256k1_scalar_get_bits(&gnb, j * 4, 4); + for (i = 0; i < 16; i++) { + /** This uses a conditional move to avoid any secret data in array indexes. + * _Any_ use of secret indexes has been demonstrated to result in timing + * sidechannels, even when the cache-line access patterns are uniform. + * See also: + * "A word of warning", CHES 2013 Rump Session, by Daniel J. Bernstein and Peter Schwabe + * (https://cryptojedi.org/peter/data/chesrump-20130822.pdf) and + * "Cache Attacks and Countermeasures: the Case of AES", RSA 2006, + * by Dag Arne Osvik, Adi Shamir, and Eran Tromer + * (http://www.tau.ac.il/~tromer/papers/cache.pdf) + */ + secp256k1_ge_storage_cmov(&adds, &(*ctx->prec)[j][i], i == bits); + } + secp256k1_ge_from_storage(&add, &adds); + secp256k1_gej_add_ge(r, r, &add); + } + bits = 0; + secp256k1_ge_clear(&add); + secp256k1_scalar_clear(&gnb); +} + +/* Setup blinding values for secp256k1_ecmult_gen. */ +static void secp256k1_ecmult_gen_blind(secp256k1_ecmult_gen_context *ctx, const unsigned char *seed32) { + secp256k1_scalar b; + secp256k1_gej gb; + secp256k1_fe s; + unsigned char nonce32[32]; + secp256k1_rfc6979_hmac_sha256_t rng; + int retry; + unsigned char keydata[64] = {0}; + if (seed32 == NULL) { + /* When seed is NULL, reset the initial point and blinding value. */ + secp256k1_gej_set_ge(&ctx->initial, &secp256k1_ge_const_g); + secp256k1_gej_neg(&ctx->initial, &ctx->initial); + secp256k1_scalar_set_int(&ctx->blind, 1); + } + /* The prior blinding value (if not reset) is chained forward by including it in the hash. */ + secp256k1_scalar_get_b32(nonce32, &ctx->blind); + /** Using a CSPRNG allows a failure free interface, avoids needing large amounts of random data, + * and guards against weak or adversarial seeds. This is a simpler and safer interface than + * asking the caller for blinding values directly and expecting them to retry on failure. + */ + memcpy(keydata, nonce32, 32); + if (seed32 != NULL) { + memcpy(keydata + 32, seed32, 32); + } + secp256k1_rfc6979_hmac_sha256_initialize(&rng, keydata, seed32 ? 64 : 32); + memset(keydata, 0, sizeof(keydata)); + /* Retry for out of range results to achieve uniformity. */ + do { + secp256k1_rfc6979_hmac_sha256_generate(&rng, nonce32, 32); + retry = !secp256k1_fe_set_b32(&s, nonce32); + retry |= secp256k1_fe_is_zero(&s); + } while (retry); /* This branch true is cryptographically unreachable. Requires sha256_hmac output > Fp. */ + /* Randomize the projection to defend against multiplier sidechannels. */ + secp256k1_gej_rescale(&ctx->initial, &s); + secp256k1_fe_clear(&s); + do { + secp256k1_rfc6979_hmac_sha256_generate(&rng, nonce32, 32); + secp256k1_scalar_set_b32(&b, nonce32, &retry); + /* A blinding value of 0 works, but would undermine the projection hardening. */ + retry |= secp256k1_scalar_is_zero(&b); + } while (retry); /* This branch true is cryptographically unreachable. Requires sha256_hmac output > order. */ + secp256k1_rfc6979_hmac_sha256_finalize(&rng); + memset(nonce32, 0, 32); + secp256k1_ecmult_gen(ctx, &gb, &b); + secp256k1_scalar_negate(&b, &b); + ctx->blind = b; + ctx->initial = gb; + secp256k1_scalar_clear(&b); + secp256k1_gej_clear(&gb); +} + +#endif diff --git a/src/secp256k1/src/ecmult_impl.h b/src/secp256k1/src/ecmult_impl.h new file mode 100644 index 00000000..e6e5f471 --- /dev/null +++ b/src/secp256k1/src/ecmult_impl.h @@ -0,0 +1,389 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_ECMULT_IMPL_H_ +#define _SECP256K1_ECMULT_IMPL_H_ + +#include "group.h" +#include "scalar.h" +#include "ecmult.h" + +/* optimal for 128-bit and 256-bit exponents. */ +#define WINDOW_A 5 + +/** larger numbers may result in slightly better performance, at the cost of + exponentially larger precomputed tables. */ +#ifdef USE_ENDOMORPHISM +/** Two tables for window size 15: 1.375 MiB. */ +#define WINDOW_G 15 +#else +/** One table for window size 16: 1.375 MiB. */ +#define WINDOW_G 16 +#endif + +/** The number of entries a table with precomputed multiples needs to have. */ +#define ECMULT_TABLE_SIZE(w) (1 << ((w)-2)) + +/** Fill a table 'prej' with precomputed odd multiples of a. Prej will contain + * the values [1*a,3*a,...,(2*n-1)*a], so it space for n values. zr[0] will + * contain prej[0].z / a.z. The other zr[i] values = prej[i].z / prej[i-1].z. + * Prej's Z values are undefined, except for the last value. + */ +static void secp256k1_ecmult_odd_multiples_table(int n, secp256k1_gej *prej, secp256k1_fe *zr, const secp256k1_gej *a) { + secp256k1_gej d; + secp256k1_ge a_ge, d_ge; + int i; + + VERIFY_CHECK(!a->infinity); + + secp256k1_gej_double_var(&d, a, NULL); + + /* + * Perform the additions on an isomorphism where 'd' is affine: drop the z coordinate + * of 'd', and scale the 1P starting value's x/y coordinates without changing its z. + */ + d_ge.x = d.x; + d_ge.y = d.y; + d_ge.infinity = 0; + + secp256k1_ge_set_gej_zinv(&a_ge, a, &d.z); + prej[0].x = a_ge.x; + prej[0].y = a_ge.y; + prej[0].z = a->z; + prej[0].infinity = 0; + + zr[0] = d.z; + for (i = 1; i < n; i++) { + secp256k1_gej_add_ge_var(&prej[i], &prej[i-1], &d_ge, &zr[i]); + } + + /* + * Each point in 'prej' has a z coordinate too small by a factor of 'd.z'. Only + * the final point's z coordinate is actually used though, so just update that. + */ + secp256k1_fe_mul(&prej[n-1].z, &prej[n-1].z, &d.z); +} + +/** Fill a table 'pre' with precomputed odd multiples of a. + * + * There are two versions of this function: + * - secp256k1_ecmult_odd_multiples_table_globalz_windowa which brings its + * resulting point set to a single constant Z denominator, stores the X and Y + * coordinates as ge_storage points in pre, and stores the global Z in rz. + * It only operates on tables sized for WINDOW_A wnaf multiples. + * - secp256k1_ecmult_odd_multiples_table_storage_var, which converts its + * resulting point set to actually affine points, and stores those in pre. + * It operates on tables of any size, but uses heap-allocated temporaries. + * + * To compute a*P + b*G, we compute a table for P using the first function, + * and for G using the second (which requires an inverse, but it only needs to + * happen once). + */ +static void secp256k1_ecmult_odd_multiples_table_globalz_windowa(secp256k1_ge *pre, secp256k1_fe *globalz, const secp256k1_gej *a) { + secp256k1_gej prej[ECMULT_TABLE_SIZE(WINDOW_A)]; + secp256k1_fe zr[ECMULT_TABLE_SIZE(WINDOW_A)]; + + /* Compute the odd multiples in Jacobian form. */ + secp256k1_ecmult_odd_multiples_table(ECMULT_TABLE_SIZE(WINDOW_A), prej, zr, a); + /* Bring them to the same Z denominator. */ + secp256k1_ge_globalz_set_table_gej(ECMULT_TABLE_SIZE(WINDOW_A), pre, globalz, prej, zr); +} + +static void secp256k1_ecmult_odd_multiples_table_storage_var(int n, secp256k1_ge_storage *pre, const secp256k1_gej *a, const secp256k1_callback *cb) { + secp256k1_gej *prej = (secp256k1_gej*)checked_malloc(cb, sizeof(secp256k1_gej) * n); + secp256k1_ge *prea = (secp256k1_ge*)checked_malloc(cb, sizeof(secp256k1_ge) * n); + secp256k1_fe *zr = (secp256k1_fe*)checked_malloc(cb, sizeof(secp256k1_fe) * n); + int i; + + /* Compute the odd multiples in Jacobian form. */ + secp256k1_ecmult_odd_multiples_table(n, prej, zr, a); + /* Convert them in batch to affine coordinates. */ + secp256k1_ge_set_table_gej_var(n, prea, prej, zr); + /* Convert them to compact storage form. */ + for (i = 0; i < n; i++) { + secp256k1_ge_to_storage(&pre[i], &prea[i]); + } + + free(prea); + free(prej); + free(zr); +} + +/** The following two macro retrieves a particular odd multiple from a table + * of precomputed multiples. */ +#define ECMULT_TABLE_GET_GE(r,pre,n,w) do { \ + VERIFY_CHECK(((n) & 1) == 1); \ + VERIFY_CHECK((n) >= -((1 << ((w)-1)) - 1)); \ + VERIFY_CHECK((n) <= ((1 << ((w)-1)) - 1)); \ + if ((n) > 0) { \ + *(r) = (pre)[((n)-1)/2]; \ + } else { \ + secp256k1_ge_neg((r), &(pre)[(-(n)-1)/2]); \ + } \ +} while(0) + +#define ECMULT_TABLE_GET_GE_STORAGE(r,pre,n,w) do { \ + VERIFY_CHECK(((n) & 1) == 1); \ + VERIFY_CHECK((n) >= -((1 << ((w)-1)) - 1)); \ + VERIFY_CHECK((n) <= ((1 << ((w)-1)) - 1)); \ + if ((n) > 0) { \ + secp256k1_ge_from_storage((r), &(pre)[((n)-1)/2]); \ + } else { \ + secp256k1_ge_from_storage((r), &(pre)[(-(n)-1)/2]); \ + secp256k1_ge_neg((r), (r)); \ + } \ +} while(0) + +static void secp256k1_ecmult_context_init(secp256k1_ecmult_context *ctx) { + ctx->pre_g = NULL; +#ifdef USE_ENDOMORPHISM + ctx->pre_g_128 = NULL; +#endif +} + +static void secp256k1_ecmult_context_build(secp256k1_ecmult_context *ctx, const secp256k1_callback *cb) { + secp256k1_gej gj; + + if (ctx->pre_g != NULL) { + return; + } + + /* get the generator */ + secp256k1_gej_set_ge(&gj, &secp256k1_ge_const_g); + + ctx->pre_g = (secp256k1_ge_storage (*)[])checked_malloc(cb, sizeof((*ctx->pre_g)[0]) * ECMULT_TABLE_SIZE(WINDOW_G)); + + /* precompute the tables with odd multiples */ + secp256k1_ecmult_odd_multiples_table_storage_var(ECMULT_TABLE_SIZE(WINDOW_G), *ctx->pre_g, &gj, cb); + +#ifdef USE_ENDOMORPHISM + { + secp256k1_gej g_128j; + int i; + + ctx->pre_g_128 = (secp256k1_ge_storage (*)[])checked_malloc(cb, sizeof((*ctx->pre_g_128)[0]) * ECMULT_TABLE_SIZE(WINDOW_G)); + + /* calculate 2^128*generator */ + g_128j = gj; + for (i = 0; i < 128; i++) { + secp256k1_gej_double_var(&g_128j, &g_128j, NULL); + } + secp256k1_ecmult_odd_multiples_table_storage_var(ECMULT_TABLE_SIZE(WINDOW_G), *ctx->pre_g_128, &g_128j, cb); + } +#endif +} + +static void secp256k1_ecmult_context_clone(secp256k1_ecmult_context *dst, + const secp256k1_ecmult_context *src, const secp256k1_callback *cb) { + if (src->pre_g == NULL) { + dst->pre_g = NULL; + } else { + size_t size = sizeof((*dst->pre_g)[0]) * ECMULT_TABLE_SIZE(WINDOW_G); + dst->pre_g = (secp256k1_ge_storage (*)[])checked_malloc(cb, size); + memcpy(dst->pre_g, src->pre_g, size); + } +#ifdef USE_ENDOMORPHISM + if (src->pre_g_128 == NULL) { + dst->pre_g_128 = NULL; + } else { + size_t size = sizeof((*dst->pre_g_128)[0]) * ECMULT_TABLE_SIZE(WINDOW_G); + dst->pre_g_128 = (secp256k1_ge_storage (*)[])checked_malloc(cb, size); + memcpy(dst->pre_g_128, src->pre_g_128, size); + } +#endif +} + +static int secp256k1_ecmult_context_is_built(const secp256k1_ecmult_context *ctx) { + return ctx->pre_g != NULL; +} + +static void secp256k1_ecmult_context_clear(secp256k1_ecmult_context *ctx) { + free(ctx->pre_g); +#ifdef USE_ENDOMORPHISM + free(ctx->pre_g_128); +#endif + secp256k1_ecmult_context_init(ctx); +} + +/** Convert a number to WNAF notation. The number becomes represented by sum(2^i * wnaf[i], i=0..bits), + * with the following guarantees: + * - each wnaf[i] is either 0, or an odd integer between -(1<<(w-1) - 1) and (1<<(w-1) - 1) + * - two non-zero entries in wnaf are separated by at least w-1 zeroes. + * - the number of set values in wnaf is returned. This number is at most 256, and at most one more + * than the number of bits in the (absolute value) of the input. + */ +static int secp256k1_ecmult_wnaf(int *wnaf, int len, const secp256k1_scalar *a, int w) { + secp256k1_scalar s = *a; + int last_set_bit = -1; + int bit = 0; + int sign = 1; + int carry = 0; + + VERIFY_CHECK(wnaf != NULL); + VERIFY_CHECK(0 <= len && len <= 256); + VERIFY_CHECK(a != NULL); + VERIFY_CHECK(2 <= w && w <= 31); + + memset(wnaf, 0, len * sizeof(wnaf[0])); + + if (secp256k1_scalar_get_bits(&s, 255, 1)) { + secp256k1_scalar_negate(&s, &s); + sign = -1; + } + + while (bit < len) { + int now; + int word; + if (secp256k1_scalar_get_bits(&s, bit, 1) == (unsigned int)carry) { + bit++; + continue; + } + + now = w; + if (now > len - bit) { + now = len - bit; + } + + word = secp256k1_scalar_get_bits_var(&s, bit, now) + carry; + + carry = (word >> (w-1)) & 1; + word -= carry << w; + + wnaf[bit] = sign * word; + last_set_bit = bit; + + bit += now; + } +#ifdef VERIFY + CHECK(carry == 0); + while (bit < 256) { + CHECK(secp256k1_scalar_get_bits(&s, bit++, 1) == 0); + } +#endif + return last_set_bit + 1; +} + +static void secp256k1_ecmult(const secp256k1_ecmult_context *ctx, secp256k1_gej *r, const secp256k1_gej *a, const secp256k1_scalar *na, const secp256k1_scalar *ng) { + secp256k1_ge pre_a[ECMULT_TABLE_SIZE(WINDOW_A)]; + secp256k1_ge tmpa; + secp256k1_fe Z; +#ifdef USE_ENDOMORPHISM + secp256k1_ge pre_a_lam[ECMULT_TABLE_SIZE(WINDOW_A)]; + secp256k1_scalar na_1, na_lam; + /* Splitted G factors. */ + secp256k1_scalar ng_1, ng_128; + int wnaf_na_1[130]; + int wnaf_na_lam[130]; + int bits_na_1; + int bits_na_lam; + int wnaf_ng_1[129]; + int bits_ng_1; + int wnaf_ng_128[129]; + int bits_ng_128; +#else + int wnaf_na[256]; + int bits_na; + int wnaf_ng[256]; + int bits_ng; +#endif + int i; + int bits; + +#ifdef USE_ENDOMORPHISM + /* split na into na_1 and na_lam (where na = na_1 + na_lam*lambda, and na_1 and na_lam are ~128 bit) */ + secp256k1_scalar_split_lambda(&na_1, &na_lam, na); + + /* build wnaf representation for na_1 and na_lam. */ + bits_na_1 = secp256k1_ecmult_wnaf(wnaf_na_1, 130, &na_1, WINDOW_A); + bits_na_lam = secp256k1_ecmult_wnaf(wnaf_na_lam, 130, &na_lam, WINDOW_A); + VERIFY_CHECK(bits_na_1 <= 130); + VERIFY_CHECK(bits_na_lam <= 130); + bits = bits_na_1; + if (bits_na_lam > bits) { + bits = bits_na_lam; + } +#else + /* build wnaf representation for na. */ + bits_na = secp256k1_ecmult_wnaf(wnaf_na, 256, na, WINDOW_A); + bits = bits_na; +#endif + + /* Calculate odd multiples of a. + * All multiples are brought to the same Z 'denominator', which is stored + * in Z. Due to secp256k1' isomorphism we can do all operations pretending + * that the Z coordinate was 1, use affine addition formulae, and correct + * the Z coordinate of the result once at the end. + * The exception is the precomputed G table points, which are actually + * affine. Compared to the base used for other points, they have a Z ratio + * of 1/Z, so we can use secp256k1_gej_add_zinv_var, which uses the same + * isomorphism to efficiently add with a known Z inverse. + */ + secp256k1_ecmult_odd_multiples_table_globalz_windowa(pre_a, &Z, a); + +#ifdef USE_ENDOMORPHISM + for (i = 0; i < ECMULT_TABLE_SIZE(WINDOW_A); i++) { + secp256k1_ge_mul_lambda(&pre_a_lam[i], &pre_a[i]); + } + + /* split ng into ng_1 and ng_128 (where gn = gn_1 + gn_128*2^128, and gn_1 and gn_128 are ~128 bit) */ + secp256k1_scalar_split_128(&ng_1, &ng_128, ng); + + /* Build wnaf representation for ng_1 and ng_128 */ + bits_ng_1 = secp256k1_ecmult_wnaf(wnaf_ng_1, 129, &ng_1, WINDOW_G); + bits_ng_128 = secp256k1_ecmult_wnaf(wnaf_ng_128, 129, &ng_128, WINDOW_G); + if (bits_ng_1 > bits) { + bits = bits_ng_1; + } + if (bits_ng_128 > bits) { + bits = bits_ng_128; + } +#else + bits_ng = secp256k1_ecmult_wnaf(wnaf_ng, 256, ng, WINDOW_G); + if (bits_ng > bits) { + bits = bits_ng; + } +#endif + + secp256k1_gej_set_infinity(r); + + for (i = bits - 1; i >= 0; i--) { + int n; + secp256k1_gej_double_var(r, r, NULL); +#ifdef USE_ENDOMORPHISM + if (i < bits_na_1 && (n = wnaf_na_1[i])) { + ECMULT_TABLE_GET_GE(&tmpa, pre_a, n, WINDOW_A); + secp256k1_gej_add_ge_var(r, r, &tmpa, NULL); + } + if (i < bits_na_lam && (n = wnaf_na_lam[i])) { + ECMULT_TABLE_GET_GE(&tmpa, pre_a_lam, n, WINDOW_A); + secp256k1_gej_add_ge_var(r, r, &tmpa, NULL); + } + if (i < bits_ng_1 && (n = wnaf_ng_1[i])) { + ECMULT_TABLE_GET_GE_STORAGE(&tmpa, *ctx->pre_g, n, WINDOW_G); + secp256k1_gej_add_zinv_var(r, r, &tmpa, &Z); + } + if (i < bits_ng_128 && (n = wnaf_ng_128[i])) { + ECMULT_TABLE_GET_GE_STORAGE(&tmpa, *ctx->pre_g_128, n, WINDOW_G); + secp256k1_gej_add_zinv_var(r, r, &tmpa, &Z); + } +#else + if (i < bits_na && (n = wnaf_na[i])) { + ECMULT_TABLE_GET_GE(&tmpa, pre_a, n, WINDOW_A); + secp256k1_gej_add_ge_var(r, r, &tmpa, NULL); + } + if (i < bits_ng && (n = wnaf_ng[i])) { + ECMULT_TABLE_GET_GE_STORAGE(&tmpa, *ctx->pre_g, n, WINDOW_G); + secp256k1_gej_add_zinv_var(r, r, &tmpa, &Z); + } +#endif + } + + if (!r->infinity) { + secp256k1_fe_mul(&r->z, &r->z, &Z); + } +} + +#endif diff --git a/src/secp256k1/src/field.h b/src/secp256k1/src/field.h new file mode 100644 index 00000000..2d52af5e --- /dev/null +++ b/src/secp256k1/src/field.h @@ -0,0 +1,121 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_FIELD_ +#define _SECP256K1_FIELD_ + +/** Field element module. + * + * Field elements can be represented in several ways, but code accessing + * it (and implementations) need to take certain properties into account: + * - Each field element can be normalized or not. + * - Each field element has a magnitude, which represents how far away + * its representation is away from normalization. Normalized elements + * always have a magnitude of 1, but a magnitude of 1 doesn't imply + * normality. + */ + +#if defined HAVE_CONFIG_H +#include "libsecp256k1-config.h" +#endif + +#if defined(USE_FIELD_10X26) +#include "field_10x26.h" +#elif defined(USE_FIELD_5X52) +#include "field_5x52.h" +#else +#error "Please select field implementation" +#endif + +/** Normalize a field element. */ +static void secp256k1_fe_normalize(secp256k1_fe *r); + +/** Weakly normalize a field element: reduce it magnitude to 1, but don't fully normalize. */ +static void secp256k1_fe_normalize_weak(secp256k1_fe *r); + +/** Normalize a field element, without constant-time guarantee. */ +static void secp256k1_fe_normalize_var(secp256k1_fe *r); + +/** Verify whether a field element represents zero i.e. would normalize to a zero value. The field + * implementation may optionally normalize the input, but this should not be relied upon. */ +static int secp256k1_fe_normalizes_to_zero(secp256k1_fe *r); + +/** Verify whether a field element represents zero i.e. would normalize to a zero value. The field + * implementation may optionally normalize the input, but this should not be relied upon. */ +static int secp256k1_fe_normalizes_to_zero_var(secp256k1_fe *r); + +/** Set a field element equal to a small integer. Resulting field element is normalized. */ +static void secp256k1_fe_set_int(secp256k1_fe *r, int a); + +/** Verify whether a field element is zero. Requires the input to be normalized. */ +static int secp256k1_fe_is_zero(const secp256k1_fe *a); + +/** Check the "oddness" of a field element. Requires the input to be normalized. */ +static int secp256k1_fe_is_odd(const secp256k1_fe *a); + +/** Compare two field elements. Requires magnitude-1 inputs. */ +static int secp256k1_fe_equal_var(const secp256k1_fe *a, const secp256k1_fe *b); + +/** Compare two field elements. Requires both inputs to be normalized */ +static int secp256k1_fe_cmp_var(const secp256k1_fe *a, const secp256k1_fe *b); + +/** Set a field element equal to 32-byte big endian value. If successful, the resulting field element is normalized. */ +static int secp256k1_fe_set_b32(secp256k1_fe *r, const unsigned char *a); + +/** Convert a field element to a 32-byte big endian value. Requires the input to be normalized */ +static void secp256k1_fe_get_b32(unsigned char *r, const secp256k1_fe *a); + +/** Set a field element equal to the additive inverse of another. Takes a maximum magnitude of the input + * as an argument. The magnitude of the output is one higher. */ +static void secp256k1_fe_negate(secp256k1_fe *r, const secp256k1_fe *a, int m); + +/** Multiplies the passed field element with a small integer constant. Multiplies the magnitude by that + * small integer. */ +static void secp256k1_fe_mul_int(secp256k1_fe *r, int a); + +/** Adds a field element to another. The result has the sum of the inputs' magnitudes as magnitude. */ +static void secp256k1_fe_add(secp256k1_fe *r, const secp256k1_fe *a); + +/** Sets a field element to be the product of two others. Requires the inputs' magnitudes to be at most 8. + * The output magnitude is 1 (but not guaranteed to be normalized). */ +static void secp256k1_fe_mul(secp256k1_fe *r, const secp256k1_fe *a, const secp256k1_fe * SECP256K1_RESTRICT b); + +/** Sets a field element to be the square of another. Requires the input's magnitude to be at most 8. + * The output magnitude is 1 (but not guaranteed to be normalized). */ +static void secp256k1_fe_sqr(secp256k1_fe *r, const secp256k1_fe *a); + +/** If a has a square root, it is computed in r and 1 is returned. If a does not + * have a square root, the root of its negation is computed and 0 is returned. + * The input's magnitude can be at most 8. The output magnitude is 1 (but not + * guaranteed to be normalized). The result in r will always be a square + * itself. */ +static int secp256k1_fe_sqrt_var(secp256k1_fe *r, const secp256k1_fe *a); + +/** Sets a field element to be the (modular) inverse of another. Requires the input's magnitude to be + * at most 8. The output magnitude is 1 (but not guaranteed to be normalized). */ +static void secp256k1_fe_inv(secp256k1_fe *r, const secp256k1_fe *a); + +/** Potentially faster version of secp256k1_fe_inv, without constant-time guarantee. */ +static void secp256k1_fe_inv_var(secp256k1_fe *r, const secp256k1_fe *a); + +/** Calculate the (modular) inverses of a batch of field elements. Requires the inputs' magnitudes to be + * at most 8. The output magnitudes are 1 (but not guaranteed to be normalized). The inputs and + * outputs must not overlap in memory. */ +static void secp256k1_fe_inv_all_var(size_t len, secp256k1_fe *r, const secp256k1_fe *a); + +/** Convert a field element to the storage type. */ +static void secp256k1_fe_to_storage(secp256k1_fe_storage *r, const secp256k1_fe *a); + +/** Convert a field element back from the storage type. */ +static void secp256k1_fe_from_storage(secp256k1_fe *r, const secp256k1_fe_storage *a); + +/** If flag is true, set *r equal to *a; otherwise leave it. Constant-time. */ +static void secp256k1_fe_storage_cmov(secp256k1_fe_storage *r, const secp256k1_fe_storage *a, int flag); + +/** If flag is true, set *r equal to *a; otherwise leave it. Constant-time. */ +static void secp256k1_fe_cmov(secp256k1_fe *r, const secp256k1_fe *a, int flag); + +#endif diff --git a/src/secp256k1/src/field_10x26.h b/src/secp256k1/src/field_10x26.h new file mode 100644 index 00000000..61ee1e09 --- /dev/null +++ b/src/secp256k1/src/field_10x26.h @@ -0,0 +1,47 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_FIELD_REPR_ +#define _SECP256K1_FIELD_REPR_ + +#include + +typedef struct { + /* X = sum(i=0..9, elem[i]*2^26) mod n */ + uint32_t n[10]; +#ifdef VERIFY + int magnitude; + int normalized; +#endif +} secp256k1_fe; + +/* Unpacks a constant into a overlapping multi-limbed FE element. */ +#define SECP256K1_FE_CONST_INNER(d7, d6, d5, d4, d3, d2, d1, d0) { \ + (d0) & 0x3FFFFFFUL, \ + (((uint32_t)d0) >> 26) | (((uint32_t)(d1) & 0xFFFFFUL) << 6), \ + (((uint32_t)d1) >> 20) | (((uint32_t)(d2) & 0x3FFFUL) << 12), \ + (((uint32_t)d2) >> 14) | (((uint32_t)(d3) & 0xFFUL) << 18), \ + (((uint32_t)d3) >> 8) | (((uint32_t)(d4) & 0x3UL) << 24), \ + (((uint32_t)d4) >> 2) & 0x3FFFFFFUL, \ + (((uint32_t)d4) >> 28) | (((uint32_t)(d5) & 0x3FFFFFUL) << 4), \ + (((uint32_t)d5) >> 22) | (((uint32_t)(d6) & 0xFFFFUL) << 10), \ + (((uint32_t)d6) >> 16) | (((uint32_t)(d7) & 0x3FFUL) << 16), \ + (((uint32_t)d7) >> 10) \ +} + +#ifdef VERIFY +#define SECP256K1_FE_CONST(d7, d6, d5, d4, d3, d2, d1, d0) {SECP256K1_FE_CONST_INNER((d7), (d6), (d5), (d4), (d3), (d2), (d1), (d0)), 1, 1} +#else +#define SECP256K1_FE_CONST(d7, d6, d5, d4, d3, d2, d1, d0) {SECP256K1_FE_CONST_INNER((d7), (d6), (d5), (d4), (d3), (d2), (d1), (d0))} +#endif + +typedef struct { + uint32_t n[8]; +} secp256k1_fe_storage; + +#define SECP256K1_FE_STORAGE_CONST(d7, d6, d5, d4, d3, d2, d1, d0) {{ (d0), (d1), (d2), (d3), (d4), (d5), (d6), (d7) }} +#define SECP256K1_FE_STORAGE_CONST_GET(d) d.n[7], d.n[6], d.n[5], d.n[4],d.n[3], d.n[2], d.n[1], d.n[0] +#endif diff --git a/src/secp256k1/src/field_10x26_impl.h b/src/secp256k1/src/field_10x26_impl.h new file mode 100644 index 00000000..212cc539 --- /dev/null +++ b/src/secp256k1/src/field_10x26_impl.h @@ -0,0 +1,1138 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_FIELD_REPR_IMPL_H_ +#define _SECP256K1_FIELD_REPR_IMPL_H_ + +#include +#include +#include "util.h" +#include "num.h" +#include "field.h" + +#ifdef VERIFY +static void secp256k1_fe_verify(const secp256k1_fe *a) { + const uint32_t *d = a->n; + int m = a->normalized ? 1 : 2 * a->magnitude, r = 1; + r &= (d[0] <= 0x3FFFFFFUL * m); + r &= (d[1] <= 0x3FFFFFFUL * m); + r &= (d[2] <= 0x3FFFFFFUL * m); + r &= (d[3] <= 0x3FFFFFFUL * m); + r &= (d[4] <= 0x3FFFFFFUL * m); + r &= (d[5] <= 0x3FFFFFFUL * m); + r &= (d[6] <= 0x3FFFFFFUL * m); + r &= (d[7] <= 0x3FFFFFFUL * m); + r &= (d[8] <= 0x3FFFFFFUL * m); + r &= (d[9] <= 0x03FFFFFUL * m); + r &= (a->magnitude >= 0); + r &= (a->magnitude <= 32); + if (a->normalized) { + r &= (a->magnitude <= 1); + if (r && (d[9] == 0x03FFFFFUL)) { + uint32_t mid = d[8] & d[7] & d[6] & d[5] & d[4] & d[3] & d[2]; + if (mid == 0x3FFFFFFUL) { + r &= ((d[1] + 0x40UL + ((d[0] + 0x3D1UL) >> 26)) <= 0x3FFFFFFUL); + } + } + } + VERIFY_CHECK(r == 1); +} +#else +static void secp256k1_fe_verify(const secp256k1_fe *a) { + (void)a; +} +#endif + +static void secp256k1_fe_normalize(secp256k1_fe *r) { + uint32_t t0 = r->n[0], t1 = r->n[1], t2 = r->n[2], t3 = r->n[3], t4 = r->n[4], + t5 = r->n[5], t6 = r->n[6], t7 = r->n[7], t8 = r->n[8], t9 = r->n[9]; + + /* Reduce t9 at the start so there will be at most a single carry from the first pass */ + uint32_t m; + uint32_t x = t9 >> 22; t9 &= 0x03FFFFFUL; + + /* The first pass ensures the magnitude is 1, ... */ + t0 += x * 0x3D1UL; t1 += (x << 6); + t1 += (t0 >> 26); t0 &= 0x3FFFFFFUL; + t2 += (t1 >> 26); t1 &= 0x3FFFFFFUL; + t3 += (t2 >> 26); t2 &= 0x3FFFFFFUL; m = t2; + t4 += (t3 >> 26); t3 &= 0x3FFFFFFUL; m &= t3; + t5 += (t4 >> 26); t4 &= 0x3FFFFFFUL; m &= t4; + t6 += (t5 >> 26); t5 &= 0x3FFFFFFUL; m &= t5; + t7 += (t6 >> 26); t6 &= 0x3FFFFFFUL; m &= t6; + t8 += (t7 >> 26); t7 &= 0x3FFFFFFUL; m &= t7; + t9 += (t8 >> 26); t8 &= 0x3FFFFFFUL; m &= t8; + + /* ... except for a possible carry at bit 22 of t9 (i.e. bit 256 of the field element) */ + VERIFY_CHECK(t9 >> 23 == 0); + + /* At most a single final reduction is needed; check if the value is >= the field characteristic */ + x = (t9 >> 22) | ((t9 == 0x03FFFFFUL) & (m == 0x3FFFFFFUL) + & ((t1 + 0x40UL + ((t0 + 0x3D1UL) >> 26)) > 0x3FFFFFFUL)); + + /* Apply the final reduction (for constant-time behaviour, we do it always) */ + t0 += x * 0x3D1UL; t1 += (x << 6); + t1 += (t0 >> 26); t0 &= 0x3FFFFFFUL; + t2 += (t1 >> 26); t1 &= 0x3FFFFFFUL; + t3 += (t2 >> 26); t2 &= 0x3FFFFFFUL; + t4 += (t3 >> 26); t3 &= 0x3FFFFFFUL; + t5 += (t4 >> 26); t4 &= 0x3FFFFFFUL; + t6 += (t5 >> 26); t5 &= 0x3FFFFFFUL; + t7 += (t6 >> 26); t6 &= 0x3FFFFFFUL; + t8 += (t7 >> 26); t7 &= 0x3FFFFFFUL; + t9 += (t8 >> 26); t8 &= 0x3FFFFFFUL; + + /* If t9 didn't carry to bit 22 already, then it should have after any final reduction */ + VERIFY_CHECK(t9 >> 22 == x); + + /* Mask off the possible multiple of 2^256 from the final reduction */ + t9 &= 0x03FFFFFUL; + + r->n[0] = t0; r->n[1] = t1; r->n[2] = t2; r->n[3] = t3; r->n[4] = t4; + r->n[5] = t5; r->n[6] = t6; r->n[7] = t7; r->n[8] = t8; r->n[9] = t9; + +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 1; + secp256k1_fe_verify(r); +#endif +} + +static void secp256k1_fe_normalize_weak(secp256k1_fe *r) { + uint32_t t0 = r->n[0], t1 = r->n[1], t2 = r->n[2], t3 = r->n[3], t4 = r->n[4], + t5 = r->n[5], t6 = r->n[6], t7 = r->n[7], t8 = r->n[8], t9 = r->n[9]; + + /* Reduce t9 at the start so there will be at most a single carry from the first pass */ + uint32_t x = t9 >> 22; t9 &= 0x03FFFFFUL; + + /* The first pass ensures the magnitude is 1, ... */ + t0 += x * 0x3D1UL; t1 += (x << 6); + t1 += (t0 >> 26); t0 &= 0x3FFFFFFUL; + t2 += (t1 >> 26); t1 &= 0x3FFFFFFUL; + t3 += (t2 >> 26); t2 &= 0x3FFFFFFUL; + t4 += (t3 >> 26); t3 &= 0x3FFFFFFUL; + t5 += (t4 >> 26); t4 &= 0x3FFFFFFUL; + t6 += (t5 >> 26); t5 &= 0x3FFFFFFUL; + t7 += (t6 >> 26); t6 &= 0x3FFFFFFUL; + t8 += (t7 >> 26); t7 &= 0x3FFFFFFUL; + t9 += (t8 >> 26); t8 &= 0x3FFFFFFUL; + + /* ... except for a possible carry at bit 22 of t9 (i.e. bit 256 of the field element) */ + VERIFY_CHECK(t9 >> 23 == 0); + + r->n[0] = t0; r->n[1] = t1; r->n[2] = t2; r->n[3] = t3; r->n[4] = t4; + r->n[5] = t5; r->n[6] = t6; r->n[7] = t7; r->n[8] = t8; r->n[9] = t9; + +#ifdef VERIFY + r->magnitude = 1; + secp256k1_fe_verify(r); +#endif +} + +static void secp256k1_fe_normalize_var(secp256k1_fe *r) { + uint32_t t0 = r->n[0], t1 = r->n[1], t2 = r->n[2], t3 = r->n[3], t4 = r->n[4], + t5 = r->n[5], t6 = r->n[6], t7 = r->n[7], t8 = r->n[8], t9 = r->n[9]; + + /* Reduce t9 at the start so there will be at most a single carry from the first pass */ + uint32_t m; + uint32_t x = t9 >> 22; t9 &= 0x03FFFFFUL; + + /* The first pass ensures the magnitude is 1, ... */ + t0 += x * 0x3D1UL; t1 += (x << 6); + t1 += (t0 >> 26); t0 &= 0x3FFFFFFUL; + t2 += (t1 >> 26); t1 &= 0x3FFFFFFUL; + t3 += (t2 >> 26); t2 &= 0x3FFFFFFUL; m = t2; + t4 += (t3 >> 26); t3 &= 0x3FFFFFFUL; m &= t3; + t5 += (t4 >> 26); t4 &= 0x3FFFFFFUL; m &= t4; + t6 += (t5 >> 26); t5 &= 0x3FFFFFFUL; m &= t5; + t7 += (t6 >> 26); t6 &= 0x3FFFFFFUL; m &= t6; + t8 += (t7 >> 26); t7 &= 0x3FFFFFFUL; m &= t7; + t9 += (t8 >> 26); t8 &= 0x3FFFFFFUL; m &= t8; + + /* ... except for a possible carry at bit 22 of t9 (i.e. bit 256 of the field element) */ + VERIFY_CHECK(t9 >> 23 == 0); + + /* At most a single final reduction is needed; check if the value is >= the field characteristic */ + x = (t9 >> 22) | ((t9 == 0x03FFFFFUL) & (m == 0x3FFFFFFUL) + & ((t1 + 0x40UL + ((t0 + 0x3D1UL) >> 26)) > 0x3FFFFFFUL)); + + if (x) { + t0 += 0x3D1UL; t1 += (x << 6); + t1 += (t0 >> 26); t0 &= 0x3FFFFFFUL; + t2 += (t1 >> 26); t1 &= 0x3FFFFFFUL; + t3 += (t2 >> 26); t2 &= 0x3FFFFFFUL; + t4 += (t3 >> 26); t3 &= 0x3FFFFFFUL; + t5 += (t4 >> 26); t4 &= 0x3FFFFFFUL; + t6 += (t5 >> 26); t5 &= 0x3FFFFFFUL; + t7 += (t6 >> 26); t6 &= 0x3FFFFFFUL; + t8 += (t7 >> 26); t7 &= 0x3FFFFFFUL; + t9 += (t8 >> 26); t8 &= 0x3FFFFFFUL; + + /* If t9 didn't carry to bit 22 already, then it should have after any final reduction */ + VERIFY_CHECK(t9 >> 22 == x); + + /* Mask off the possible multiple of 2^256 from the final reduction */ + t9 &= 0x03FFFFFUL; + } + + r->n[0] = t0; r->n[1] = t1; r->n[2] = t2; r->n[3] = t3; r->n[4] = t4; + r->n[5] = t5; r->n[6] = t6; r->n[7] = t7; r->n[8] = t8; r->n[9] = t9; + +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 1; + secp256k1_fe_verify(r); +#endif +} + +static int secp256k1_fe_normalizes_to_zero(secp256k1_fe *r) { + uint32_t t0 = r->n[0], t1 = r->n[1], t2 = r->n[2], t3 = r->n[3], t4 = r->n[4], + t5 = r->n[5], t6 = r->n[6], t7 = r->n[7], t8 = r->n[8], t9 = r->n[9]; + + /* z0 tracks a possible raw value of 0, z1 tracks a possible raw value of P */ + uint32_t z0, z1; + + /* Reduce t9 at the start so there will be at most a single carry from the first pass */ + uint32_t x = t9 >> 22; t9 &= 0x03FFFFFUL; + + /* The first pass ensures the magnitude is 1, ... */ + t0 += x * 0x3D1UL; t1 += (x << 6); + t1 += (t0 >> 26); t0 &= 0x3FFFFFFUL; z0 = t0; z1 = t0 ^ 0x3D0UL; + t2 += (t1 >> 26); t1 &= 0x3FFFFFFUL; z0 |= t1; z1 &= t1 ^ 0x40UL; + t3 += (t2 >> 26); t2 &= 0x3FFFFFFUL; z0 |= t2; z1 &= t2; + t4 += (t3 >> 26); t3 &= 0x3FFFFFFUL; z0 |= t3; z1 &= t3; + t5 += (t4 >> 26); t4 &= 0x3FFFFFFUL; z0 |= t4; z1 &= t4; + t6 += (t5 >> 26); t5 &= 0x3FFFFFFUL; z0 |= t5; z1 &= t5; + t7 += (t6 >> 26); t6 &= 0x3FFFFFFUL; z0 |= t6; z1 &= t6; + t8 += (t7 >> 26); t7 &= 0x3FFFFFFUL; z0 |= t7; z1 &= t7; + t9 += (t8 >> 26); t8 &= 0x3FFFFFFUL; z0 |= t8; z1 &= t8; + z0 |= t9; z1 &= t9 ^ 0x3C00000UL; + + /* ... except for a possible carry at bit 22 of t9 (i.e. bit 256 of the field element) */ + VERIFY_CHECK(t9 >> 23 == 0); + + return (z0 == 0) | (z1 == 0x3FFFFFFUL); +} + +static int secp256k1_fe_normalizes_to_zero_var(secp256k1_fe *r) { + uint32_t t0, t1, t2, t3, t4, t5, t6, t7, t8, t9; + uint32_t z0, z1; + uint32_t x; + + t0 = r->n[0]; + t9 = r->n[9]; + + /* Reduce t9 at the start so there will be at most a single carry from the first pass */ + x = t9 >> 22; + + /* The first pass ensures the magnitude is 1, ... */ + t0 += x * 0x3D1UL; + + /* z0 tracks a possible raw value of 0, z1 tracks a possible raw value of P */ + z0 = t0 & 0x3FFFFFFUL; + z1 = z0 ^ 0x3D0UL; + + /* Fast return path should catch the majority of cases */ + if ((z0 != 0UL) & (z1 != 0x3FFFFFFUL)) { + return 0; + } + + t1 = r->n[1]; + t2 = r->n[2]; + t3 = r->n[3]; + t4 = r->n[4]; + t5 = r->n[5]; + t6 = r->n[6]; + t7 = r->n[7]; + t8 = r->n[8]; + + t9 &= 0x03FFFFFUL; + t1 += (x << 6); + + t1 += (t0 >> 26); + t2 += (t1 >> 26); t1 &= 0x3FFFFFFUL; z0 |= t1; z1 &= t1 ^ 0x40UL; + t3 += (t2 >> 26); t2 &= 0x3FFFFFFUL; z0 |= t2; z1 &= t2; + t4 += (t3 >> 26); t3 &= 0x3FFFFFFUL; z0 |= t3; z1 &= t3; + t5 += (t4 >> 26); t4 &= 0x3FFFFFFUL; z0 |= t4; z1 &= t4; + t6 += (t5 >> 26); t5 &= 0x3FFFFFFUL; z0 |= t5; z1 &= t5; + t7 += (t6 >> 26); t6 &= 0x3FFFFFFUL; z0 |= t6; z1 &= t6; + t8 += (t7 >> 26); t7 &= 0x3FFFFFFUL; z0 |= t7; z1 &= t7; + t9 += (t8 >> 26); t8 &= 0x3FFFFFFUL; z0 |= t8; z1 &= t8; + z0 |= t9; z1 &= t9 ^ 0x3C00000UL; + + /* ... except for a possible carry at bit 22 of t9 (i.e. bit 256 of the field element) */ + VERIFY_CHECK(t9 >> 23 == 0); + + return (z0 == 0) | (z1 == 0x3FFFFFFUL); +} + +SECP256K1_INLINE static void secp256k1_fe_set_int(secp256k1_fe *r, int a) { + r->n[0] = a; + r->n[1] = r->n[2] = r->n[3] = r->n[4] = r->n[5] = r->n[6] = r->n[7] = r->n[8] = r->n[9] = 0; +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 1; + secp256k1_fe_verify(r); +#endif +} + +SECP256K1_INLINE static int secp256k1_fe_is_zero(const secp256k1_fe *a) { + const uint32_t *t = a->n; +#ifdef VERIFY + VERIFY_CHECK(a->normalized); + secp256k1_fe_verify(a); +#endif + return (t[0] | t[1] | t[2] | t[3] | t[4] | t[5] | t[6] | t[7] | t[8] | t[9]) == 0; +} + +SECP256K1_INLINE static int secp256k1_fe_is_odd(const secp256k1_fe *a) { +#ifdef VERIFY + VERIFY_CHECK(a->normalized); + secp256k1_fe_verify(a); +#endif + return a->n[0] & 1; +} + +SECP256K1_INLINE static void secp256k1_fe_clear(secp256k1_fe *a) { + int i; +#ifdef VERIFY + a->magnitude = 0; + a->normalized = 1; +#endif + for (i=0; i<10; i++) { + a->n[i] = 0; + } +} + +static int secp256k1_fe_cmp_var(const secp256k1_fe *a, const secp256k1_fe *b) { + int i; +#ifdef VERIFY + VERIFY_CHECK(a->normalized); + VERIFY_CHECK(b->normalized); + secp256k1_fe_verify(a); + secp256k1_fe_verify(b); +#endif + for (i = 9; i >= 0; i--) { + if (a->n[i] > b->n[i]) { + return 1; + } + if (a->n[i] < b->n[i]) { + return -1; + } + } + return 0; +} + +static int secp256k1_fe_set_b32(secp256k1_fe *r, const unsigned char *a) { + int i; + r->n[0] = r->n[1] = r->n[2] = r->n[3] = r->n[4] = 0; + r->n[5] = r->n[6] = r->n[7] = r->n[8] = r->n[9] = 0; + for (i=0; i<32; i++) { + int j; + for (j=0; j<4; j++) { + int limb = (8*i+2*j)/26; + int shift = (8*i+2*j)%26; + r->n[limb] |= (uint32_t)((a[31-i] >> (2*j)) & 0x3) << shift; + } + } + if (r->n[9] == 0x3FFFFFUL && (r->n[8] & r->n[7] & r->n[6] & r->n[5] & r->n[4] & r->n[3] & r->n[2]) == 0x3FFFFFFUL && (r->n[1] + 0x40UL + ((r->n[0] + 0x3D1UL) >> 26)) > 0x3FFFFFFUL) { + return 0; + } +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 1; + secp256k1_fe_verify(r); +#endif + return 1; +} + +/** Convert a field element to a 32-byte big endian value. Requires the input to be normalized */ +static void secp256k1_fe_get_b32(unsigned char *r, const secp256k1_fe *a) { + int i; +#ifdef VERIFY + VERIFY_CHECK(a->normalized); + secp256k1_fe_verify(a); +#endif + for (i=0; i<32; i++) { + int j; + int c = 0; + for (j=0; j<4; j++) { + int limb = (8*i+2*j)/26; + int shift = (8*i+2*j)%26; + c |= ((a->n[limb] >> shift) & 0x3) << (2 * j); + } + r[31-i] = c; + } +} + +SECP256K1_INLINE static void secp256k1_fe_negate(secp256k1_fe *r, const secp256k1_fe *a, int m) { +#ifdef VERIFY + VERIFY_CHECK(a->magnitude <= m); + secp256k1_fe_verify(a); +#endif + r->n[0] = 0x3FFFC2FUL * 2 * (m + 1) - a->n[0]; + r->n[1] = 0x3FFFFBFUL * 2 * (m + 1) - a->n[1]; + r->n[2] = 0x3FFFFFFUL * 2 * (m + 1) - a->n[2]; + r->n[3] = 0x3FFFFFFUL * 2 * (m + 1) - a->n[3]; + r->n[4] = 0x3FFFFFFUL * 2 * (m + 1) - a->n[4]; + r->n[5] = 0x3FFFFFFUL * 2 * (m + 1) - a->n[5]; + r->n[6] = 0x3FFFFFFUL * 2 * (m + 1) - a->n[6]; + r->n[7] = 0x3FFFFFFUL * 2 * (m + 1) - a->n[7]; + r->n[8] = 0x3FFFFFFUL * 2 * (m + 1) - a->n[8]; + r->n[9] = 0x03FFFFFUL * 2 * (m + 1) - a->n[9]; +#ifdef VERIFY + r->magnitude = m + 1; + r->normalized = 0; + secp256k1_fe_verify(r); +#endif +} + +SECP256K1_INLINE static void secp256k1_fe_mul_int(secp256k1_fe *r, int a) { + r->n[0] *= a; + r->n[1] *= a; + r->n[2] *= a; + r->n[3] *= a; + r->n[4] *= a; + r->n[5] *= a; + r->n[6] *= a; + r->n[7] *= a; + r->n[8] *= a; + r->n[9] *= a; +#ifdef VERIFY + r->magnitude *= a; + r->normalized = 0; + secp256k1_fe_verify(r); +#endif +} + +SECP256K1_INLINE static void secp256k1_fe_add(secp256k1_fe *r, const secp256k1_fe *a) { +#ifdef VERIFY + secp256k1_fe_verify(a); +#endif + r->n[0] += a->n[0]; + r->n[1] += a->n[1]; + r->n[2] += a->n[2]; + r->n[3] += a->n[3]; + r->n[4] += a->n[4]; + r->n[5] += a->n[5]; + r->n[6] += a->n[6]; + r->n[7] += a->n[7]; + r->n[8] += a->n[8]; + r->n[9] += a->n[9]; +#ifdef VERIFY + r->magnitude += a->magnitude; + r->normalized = 0; + secp256k1_fe_verify(r); +#endif +} + +#ifdef VERIFY +#define VERIFY_BITS(x, n) VERIFY_CHECK(((x) >> (n)) == 0) +#else +#define VERIFY_BITS(x, n) do { } while(0) +#endif + +SECP256K1_INLINE static void secp256k1_fe_mul_inner(uint32_t *r, const uint32_t *a, const uint32_t * SECP256K1_RESTRICT b) { + uint64_t c, d; + uint64_t u0, u1, u2, u3, u4, u5, u6, u7, u8; + uint32_t t9, t1, t0, t2, t3, t4, t5, t6, t7; + const uint32_t M = 0x3FFFFFFUL, R0 = 0x3D10UL, R1 = 0x400UL; + + VERIFY_BITS(a[0], 30); + VERIFY_BITS(a[1], 30); + VERIFY_BITS(a[2], 30); + VERIFY_BITS(a[3], 30); + VERIFY_BITS(a[4], 30); + VERIFY_BITS(a[5], 30); + VERIFY_BITS(a[6], 30); + VERIFY_BITS(a[7], 30); + VERIFY_BITS(a[8], 30); + VERIFY_BITS(a[9], 26); + VERIFY_BITS(b[0], 30); + VERIFY_BITS(b[1], 30); + VERIFY_BITS(b[2], 30); + VERIFY_BITS(b[3], 30); + VERIFY_BITS(b[4], 30); + VERIFY_BITS(b[5], 30); + VERIFY_BITS(b[6], 30); + VERIFY_BITS(b[7], 30); + VERIFY_BITS(b[8], 30); + VERIFY_BITS(b[9], 26); + + /** [... a b c] is a shorthand for ... + a<<52 + b<<26 + c<<0 mod n. + * px is a shorthand for sum(a[i]*b[x-i], i=0..x). + * Note that [x 0 0 0 0 0 0 0 0 0 0] = [x*R1 x*R0]. + */ + + d = (uint64_t)a[0] * b[9] + + (uint64_t)a[1] * b[8] + + (uint64_t)a[2] * b[7] + + (uint64_t)a[3] * b[6] + + (uint64_t)a[4] * b[5] + + (uint64_t)a[5] * b[4] + + (uint64_t)a[6] * b[3] + + (uint64_t)a[7] * b[2] + + (uint64_t)a[8] * b[1] + + (uint64_t)a[9] * b[0]; + /* VERIFY_BITS(d, 64); */ + /* [d 0 0 0 0 0 0 0 0 0] = [p9 0 0 0 0 0 0 0 0 0] */ + t9 = d & M; d >>= 26; + VERIFY_BITS(t9, 26); + VERIFY_BITS(d, 38); + /* [d t9 0 0 0 0 0 0 0 0 0] = [p9 0 0 0 0 0 0 0 0 0] */ + + c = (uint64_t)a[0] * b[0]; + VERIFY_BITS(c, 60); + /* [d t9 0 0 0 0 0 0 0 0 c] = [p9 0 0 0 0 0 0 0 0 p0] */ + d += (uint64_t)a[1] * b[9] + + (uint64_t)a[2] * b[8] + + (uint64_t)a[3] * b[7] + + (uint64_t)a[4] * b[6] + + (uint64_t)a[5] * b[5] + + (uint64_t)a[6] * b[4] + + (uint64_t)a[7] * b[3] + + (uint64_t)a[8] * b[2] + + (uint64_t)a[9] * b[1]; + VERIFY_BITS(d, 63); + /* [d t9 0 0 0 0 0 0 0 0 c] = [p10 p9 0 0 0 0 0 0 0 0 p0] */ + u0 = d & M; d >>= 26; c += u0 * R0; + VERIFY_BITS(u0, 26); + VERIFY_BITS(d, 37); + VERIFY_BITS(c, 61); + /* [d u0 t9 0 0 0 0 0 0 0 0 c-u0*R0] = [p10 p9 0 0 0 0 0 0 0 0 p0] */ + t0 = c & M; c >>= 26; c += u0 * R1; + VERIFY_BITS(t0, 26); + VERIFY_BITS(c, 37); + /* [d u0 t9 0 0 0 0 0 0 0 c-u0*R1 t0-u0*R0] = [p10 p9 0 0 0 0 0 0 0 0 p0] */ + /* [d 0 t9 0 0 0 0 0 0 0 c t0] = [p10 p9 0 0 0 0 0 0 0 0 p0] */ + + c += (uint64_t)a[0] * b[1] + + (uint64_t)a[1] * b[0]; + VERIFY_BITS(c, 62); + /* [d 0 t9 0 0 0 0 0 0 0 c t0] = [p10 p9 0 0 0 0 0 0 0 p1 p0] */ + d += (uint64_t)a[2] * b[9] + + (uint64_t)a[3] * b[8] + + (uint64_t)a[4] * b[7] + + (uint64_t)a[5] * b[6] + + (uint64_t)a[6] * b[5] + + (uint64_t)a[7] * b[4] + + (uint64_t)a[8] * b[3] + + (uint64_t)a[9] * b[2]; + VERIFY_BITS(d, 63); + /* [d 0 t9 0 0 0 0 0 0 0 c t0] = [p11 p10 p9 0 0 0 0 0 0 0 p1 p0] */ + u1 = d & M; d >>= 26; c += u1 * R0; + VERIFY_BITS(u1, 26); + VERIFY_BITS(d, 37); + VERIFY_BITS(c, 63); + /* [d u1 0 t9 0 0 0 0 0 0 0 c-u1*R0 t0] = [p11 p10 p9 0 0 0 0 0 0 0 p1 p0] */ + t1 = c & M; c >>= 26; c += u1 * R1; + VERIFY_BITS(t1, 26); + VERIFY_BITS(c, 38); + /* [d u1 0 t9 0 0 0 0 0 0 c-u1*R1 t1-u1*R0 t0] = [p11 p10 p9 0 0 0 0 0 0 0 p1 p0] */ + /* [d 0 0 t9 0 0 0 0 0 0 c t1 t0] = [p11 p10 p9 0 0 0 0 0 0 0 p1 p0] */ + + c += (uint64_t)a[0] * b[2] + + (uint64_t)a[1] * b[1] + + (uint64_t)a[2] * b[0]; + VERIFY_BITS(c, 62); + /* [d 0 0 t9 0 0 0 0 0 0 c t1 t0] = [p11 p10 p9 0 0 0 0 0 0 p2 p1 p0] */ + d += (uint64_t)a[3] * b[9] + + (uint64_t)a[4] * b[8] + + (uint64_t)a[5] * b[7] + + (uint64_t)a[6] * b[6] + + (uint64_t)a[7] * b[5] + + (uint64_t)a[8] * b[4] + + (uint64_t)a[9] * b[3]; + VERIFY_BITS(d, 63); + /* [d 0 0 t9 0 0 0 0 0 0 c t1 t0] = [p12 p11 p10 p9 0 0 0 0 0 0 p2 p1 p0] */ + u2 = d & M; d >>= 26; c += u2 * R0; + VERIFY_BITS(u2, 26); + VERIFY_BITS(d, 37); + VERIFY_BITS(c, 63); + /* [d u2 0 0 t9 0 0 0 0 0 0 c-u2*R0 t1 t0] = [p12 p11 p10 p9 0 0 0 0 0 0 p2 p1 p0] */ + t2 = c & M; c >>= 26; c += u2 * R1; + VERIFY_BITS(t2, 26); + VERIFY_BITS(c, 38); + /* [d u2 0 0 t9 0 0 0 0 0 c-u2*R1 t2-u2*R0 t1 t0] = [p12 p11 p10 p9 0 0 0 0 0 0 p2 p1 p0] */ + /* [d 0 0 0 t9 0 0 0 0 0 c t2 t1 t0] = [p12 p11 p10 p9 0 0 0 0 0 0 p2 p1 p0] */ + + c += (uint64_t)a[0] * b[3] + + (uint64_t)a[1] * b[2] + + (uint64_t)a[2] * b[1] + + (uint64_t)a[3] * b[0]; + VERIFY_BITS(c, 63); + /* [d 0 0 0 t9 0 0 0 0 0 c t2 t1 t0] = [p12 p11 p10 p9 0 0 0 0 0 p3 p2 p1 p0] */ + d += (uint64_t)a[4] * b[9] + + (uint64_t)a[5] * b[8] + + (uint64_t)a[6] * b[7] + + (uint64_t)a[7] * b[6] + + (uint64_t)a[8] * b[5] + + (uint64_t)a[9] * b[4]; + VERIFY_BITS(d, 63); + /* [d 0 0 0 t9 0 0 0 0 0 c t2 t1 t0] = [p13 p12 p11 p10 p9 0 0 0 0 0 p3 p2 p1 p0] */ + u3 = d & M; d >>= 26; c += u3 * R0; + VERIFY_BITS(u3, 26); + VERIFY_BITS(d, 37); + /* VERIFY_BITS(c, 64); */ + /* [d u3 0 0 0 t9 0 0 0 0 0 c-u3*R0 t2 t1 t0] = [p13 p12 p11 p10 p9 0 0 0 0 0 p3 p2 p1 p0] */ + t3 = c & M; c >>= 26; c += u3 * R1; + VERIFY_BITS(t3, 26); + VERIFY_BITS(c, 39); + /* [d u3 0 0 0 t9 0 0 0 0 c-u3*R1 t3-u3*R0 t2 t1 t0] = [p13 p12 p11 p10 p9 0 0 0 0 0 p3 p2 p1 p0] */ + /* [d 0 0 0 0 t9 0 0 0 0 c t3 t2 t1 t0] = [p13 p12 p11 p10 p9 0 0 0 0 0 p3 p2 p1 p0] */ + + c += (uint64_t)a[0] * b[4] + + (uint64_t)a[1] * b[3] + + (uint64_t)a[2] * b[2] + + (uint64_t)a[3] * b[1] + + (uint64_t)a[4] * b[0]; + VERIFY_BITS(c, 63); + /* [d 0 0 0 0 t9 0 0 0 0 c t3 t2 t1 t0] = [p13 p12 p11 p10 p9 0 0 0 0 p4 p3 p2 p1 p0] */ + d += (uint64_t)a[5] * b[9] + + (uint64_t)a[6] * b[8] + + (uint64_t)a[7] * b[7] + + (uint64_t)a[8] * b[6] + + (uint64_t)a[9] * b[5]; + VERIFY_BITS(d, 62); + /* [d 0 0 0 0 t9 0 0 0 0 c t3 t2 t1 t0] = [p14 p13 p12 p11 p10 p9 0 0 0 0 p4 p3 p2 p1 p0] */ + u4 = d & M; d >>= 26; c += u4 * R0; + VERIFY_BITS(u4, 26); + VERIFY_BITS(d, 36); + /* VERIFY_BITS(c, 64); */ + /* [d u4 0 0 0 0 t9 0 0 0 0 c-u4*R0 t3 t2 t1 t0] = [p14 p13 p12 p11 p10 p9 0 0 0 0 p4 p3 p2 p1 p0] */ + t4 = c & M; c >>= 26; c += u4 * R1; + VERIFY_BITS(t4, 26); + VERIFY_BITS(c, 39); + /* [d u4 0 0 0 0 t9 0 0 0 c-u4*R1 t4-u4*R0 t3 t2 t1 t0] = [p14 p13 p12 p11 p10 p9 0 0 0 0 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 t9 0 0 0 c t4 t3 t2 t1 t0] = [p14 p13 p12 p11 p10 p9 0 0 0 0 p4 p3 p2 p1 p0] */ + + c += (uint64_t)a[0] * b[5] + + (uint64_t)a[1] * b[4] + + (uint64_t)a[2] * b[3] + + (uint64_t)a[3] * b[2] + + (uint64_t)a[4] * b[1] + + (uint64_t)a[5] * b[0]; + VERIFY_BITS(c, 63); + /* [d 0 0 0 0 0 t9 0 0 0 c t4 t3 t2 t1 t0] = [p14 p13 p12 p11 p10 p9 0 0 0 p5 p4 p3 p2 p1 p0] */ + d += (uint64_t)a[6] * b[9] + + (uint64_t)a[7] * b[8] + + (uint64_t)a[8] * b[7] + + (uint64_t)a[9] * b[6]; + VERIFY_BITS(d, 62); + /* [d 0 0 0 0 0 t9 0 0 0 c t4 t3 t2 t1 t0] = [p15 p14 p13 p12 p11 p10 p9 0 0 0 p5 p4 p3 p2 p1 p0] */ + u5 = d & M; d >>= 26; c += u5 * R0; + VERIFY_BITS(u5, 26); + VERIFY_BITS(d, 36); + /* VERIFY_BITS(c, 64); */ + /* [d u5 0 0 0 0 0 t9 0 0 0 c-u5*R0 t4 t3 t2 t1 t0] = [p15 p14 p13 p12 p11 p10 p9 0 0 0 p5 p4 p3 p2 p1 p0] */ + t5 = c & M; c >>= 26; c += u5 * R1; + VERIFY_BITS(t5, 26); + VERIFY_BITS(c, 39); + /* [d u5 0 0 0 0 0 t9 0 0 c-u5*R1 t5-u5*R0 t4 t3 t2 t1 t0] = [p15 p14 p13 p12 p11 p10 p9 0 0 0 p5 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 0 t9 0 0 c t5 t4 t3 t2 t1 t0] = [p15 p14 p13 p12 p11 p10 p9 0 0 0 p5 p4 p3 p2 p1 p0] */ + + c += (uint64_t)a[0] * b[6] + + (uint64_t)a[1] * b[5] + + (uint64_t)a[2] * b[4] + + (uint64_t)a[3] * b[3] + + (uint64_t)a[4] * b[2] + + (uint64_t)a[5] * b[1] + + (uint64_t)a[6] * b[0]; + VERIFY_BITS(c, 63); + /* [d 0 0 0 0 0 0 t9 0 0 c t5 t4 t3 t2 t1 t0] = [p15 p14 p13 p12 p11 p10 p9 0 0 p6 p5 p4 p3 p2 p1 p0] */ + d += (uint64_t)a[7] * b[9] + + (uint64_t)a[8] * b[8] + + (uint64_t)a[9] * b[7]; + VERIFY_BITS(d, 61); + /* [d 0 0 0 0 0 0 t9 0 0 c t5 t4 t3 t2 t1 t0] = [p16 p15 p14 p13 p12 p11 p10 p9 0 0 p6 p5 p4 p3 p2 p1 p0] */ + u6 = d & M; d >>= 26; c += u6 * R0; + VERIFY_BITS(u6, 26); + VERIFY_BITS(d, 35); + /* VERIFY_BITS(c, 64); */ + /* [d u6 0 0 0 0 0 0 t9 0 0 c-u6*R0 t5 t4 t3 t2 t1 t0] = [p16 p15 p14 p13 p12 p11 p10 p9 0 0 p6 p5 p4 p3 p2 p1 p0] */ + t6 = c & M; c >>= 26; c += u6 * R1; + VERIFY_BITS(t6, 26); + VERIFY_BITS(c, 39); + /* [d u6 0 0 0 0 0 0 t9 0 c-u6*R1 t6-u6*R0 t5 t4 t3 t2 t1 t0] = [p16 p15 p14 p13 p12 p11 p10 p9 0 0 p6 p5 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 0 0 t9 0 c t6 t5 t4 t3 t2 t1 t0] = [p16 p15 p14 p13 p12 p11 p10 p9 0 0 p6 p5 p4 p3 p2 p1 p0] */ + + c += (uint64_t)a[0] * b[7] + + (uint64_t)a[1] * b[6] + + (uint64_t)a[2] * b[5] + + (uint64_t)a[3] * b[4] + + (uint64_t)a[4] * b[3] + + (uint64_t)a[5] * b[2] + + (uint64_t)a[6] * b[1] + + (uint64_t)a[7] * b[0]; + /* VERIFY_BITS(c, 64); */ + VERIFY_CHECK(c <= 0x8000007C00000007ULL); + /* [d 0 0 0 0 0 0 0 t9 0 c t6 t5 t4 t3 t2 t1 t0] = [p16 p15 p14 p13 p12 p11 p10 p9 0 p7 p6 p5 p4 p3 p2 p1 p0] */ + d += (uint64_t)a[8] * b[9] + + (uint64_t)a[9] * b[8]; + VERIFY_BITS(d, 58); + /* [d 0 0 0 0 0 0 0 t9 0 c t6 t5 t4 t3 t2 t1 t0] = [p17 p16 p15 p14 p13 p12 p11 p10 p9 0 p7 p6 p5 p4 p3 p2 p1 p0] */ + u7 = d & M; d >>= 26; c += u7 * R0; + VERIFY_BITS(u7, 26); + VERIFY_BITS(d, 32); + /* VERIFY_BITS(c, 64); */ + VERIFY_CHECK(c <= 0x800001703FFFC2F7ULL); + /* [d u7 0 0 0 0 0 0 0 t9 0 c-u7*R0 t6 t5 t4 t3 t2 t1 t0] = [p17 p16 p15 p14 p13 p12 p11 p10 p9 0 p7 p6 p5 p4 p3 p2 p1 p0] */ + t7 = c & M; c >>= 26; c += u7 * R1; + VERIFY_BITS(t7, 26); + VERIFY_BITS(c, 38); + /* [d u7 0 0 0 0 0 0 0 t9 c-u7*R1 t7-u7*R0 t6 t5 t4 t3 t2 t1 t0] = [p17 p16 p15 p14 p13 p12 p11 p10 p9 0 p7 p6 p5 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 0 0 0 t9 c t7 t6 t5 t4 t3 t2 t1 t0] = [p17 p16 p15 p14 p13 p12 p11 p10 p9 0 p7 p6 p5 p4 p3 p2 p1 p0] */ + + c += (uint64_t)a[0] * b[8] + + (uint64_t)a[1] * b[7] + + (uint64_t)a[2] * b[6] + + (uint64_t)a[3] * b[5] + + (uint64_t)a[4] * b[4] + + (uint64_t)a[5] * b[3] + + (uint64_t)a[6] * b[2] + + (uint64_t)a[7] * b[1] + + (uint64_t)a[8] * b[0]; + /* VERIFY_BITS(c, 64); */ + VERIFY_CHECK(c <= 0x9000007B80000008ULL); + /* [d 0 0 0 0 0 0 0 0 t9 c t7 t6 t5 t4 t3 t2 t1 t0] = [p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + d += (uint64_t)a[9] * b[9]; + VERIFY_BITS(d, 57); + /* [d 0 0 0 0 0 0 0 0 t9 c t7 t6 t5 t4 t3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + u8 = d & M; d >>= 26; c += u8 * R0; + VERIFY_BITS(u8, 26); + VERIFY_BITS(d, 31); + /* VERIFY_BITS(c, 64); */ + VERIFY_CHECK(c <= 0x9000016FBFFFC2F8ULL); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 t7 t6 t5 t4 t3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + + r[3] = t3; + VERIFY_BITS(r[3], 26); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 t7 t6 t5 t4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[4] = t4; + VERIFY_BITS(r[4], 26); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 t7 t6 t5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[5] = t5; + VERIFY_BITS(r[5], 26); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 t7 t6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[6] = t6; + VERIFY_BITS(r[6], 26); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 t7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[7] = t7; + VERIFY_BITS(r[7], 26); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + + r[8] = c & M; c >>= 26; c += u8 * R1; + VERIFY_BITS(r[8], 26); + VERIFY_BITS(c, 39); + /* [d u8 0 0 0 0 0 0 0 0 t9+c-u8*R1 r8-u8*R0 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 0 0 0 0 t9+c r8 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + c += d * R0 + t9; + VERIFY_BITS(c, 45); + /* [d 0 0 0 0 0 0 0 0 0 c-d*R0 r8 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[9] = c & (M >> 4); c >>= 22; c += d * (R1 << 4); + VERIFY_BITS(r[9], 22); + VERIFY_BITS(c, 46); + /* [d 0 0 0 0 0 0 0 0 r9+((c-d*R1<<4)<<22)-d*R0 r8 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 0 0 -d*R1 r9+(c<<22)-d*R0 r8 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + /* [r9+(c<<22) r8 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + + d = c * (R0 >> 4) + t0; + VERIFY_BITS(d, 56); + /* [r9+(c<<22) r8 r7 r6 r5 r4 r3 t2 t1 d-c*R0>>4] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[0] = d & M; d >>= 26; + VERIFY_BITS(r[0], 26); + VERIFY_BITS(d, 30); + /* [r9+(c<<22) r8 r7 r6 r5 r4 r3 t2 t1+d r0-c*R0>>4] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + d += c * (R1 >> 4) + t1; + VERIFY_BITS(d, 53); + VERIFY_CHECK(d <= 0x10000003FFFFBFULL); + /* [r9+(c<<22) r8 r7 r6 r5 r4 r3 t2 d-c*R1>>4 r0-c*R0>>4] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + /* [r9 r8 r7 r6 r5 r4 r3 t2 d r0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[1] = d & M; d >>= 26; + VERIFY_BITS(r[1], 26); + VERIFY_BITS(d, 27); + VERIFY_CHECK(d <= 0x4000000ULL); + /* [r9 r8 r7 r6 r5 r4 r3 t2+d r1 r0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + d += t2; + VERIFY_BITS(d, 27); + /* [r9 r8 r7 r6 r5 r4 r3 d r1 r0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[2] = d; + VERIFY_BITS(r[2], 27); + /* [r9 r8 r7 r6 r5 r4 r3 r2 r1 r0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ +} + +SECP256K1_INLINE static void secp256k1_fe_sqr_inner(uint32_t *r, const uint32_t *a) { + uint64_t c, d; + uint64_t u0, u1, u2, u3, u4, u5, u6, u7, u8; + uint32_t t9, t0, t1, t2, t3, t4, t5, t6, t7; + const uint32_t M = 0x3FFFFFFUL, R0 = 0x3D10UL, R1 = 0x400UL; + + VERIFY_BITS(a[0], 30); + VERIFY_BITS(a[1], 30); + VERIFY_BITS(a[2], 30); + VERIFY_BITS(a[3], 30); + VERIFY_BITS(a[4], 30); + VERIFY_BITS(a[5], 30); + VERIFY_BITS(a[6], 30); + VERIFY_BITS(a[7], 30); + VERIFY_BITS(a[8], 30); + VERIFY_BITS(a[9], 26); + + /** [... a b c] is a shorthand for ... + a<<52 + b<<26 + c<<0 mod n. + * px is a shorthand for sum(a[i]*a[x-i], i=0..x). + * Note that [x 0 0 0 0 0 0 0 0 0 0] = [x*R1 x*R0]. + */ + + d = (uint64_t)(a[0]*2) * a[9] + + (uint64_t)(a[1]*2) * a[8] + + (uint64_t)(a[2]*2) * a[7] + + (uint64_t)(a[3]*2) * a[6] + + (uint64_t)(a[4]*2) * a[5]; + /* VERIFY_BITS(d, 64); */ + /* [d 0 0 0 0 0 0 0 0 0] = [p9 0 0 0 0 0 0 0 0 0] */ + t9 = d & M; d >>= 26; + VERIFY_BITS(t9, 26); + VERIFY_BITS(d, 38); + /* [d t9 0 0 0 0 0 0 0 0 0] = [p9 0 0 0 0 0 0 0 0 0] */ + + c = (uint64_t)a[0] * a[0]; + VERIFY_BITS(c, 60); + /* [d t9 0 0 0 0 0 0 0 0 c] = [p9 0 0 0 0 0 0 0 0 p0] */ + d += (uint64_t)(a[1]*2) * a[9] + + (uint64_t)(a[2]*2) * a[8] + + (uint64_t)(a[3]*2) * a[7] + + (uint64_t)(a[4]*2) * a[6] + + (uint64_t)a[5] * a[5]; + VERIFY_BITS(d, 63); + /* [d t9 0 0 0 0 0 0 0 0 c] = [p10 p9 0 0 0 0 0 0 0 0 p0] */ + u0 = d & M; d >>= 26; c += u0 * R0; + VERIFY_BITS(u0, 26); + VERIFY_BITS(d, 37); + VERIFY_BITS(c, 61); + /* [d u0 t9 0 0 0 0 0 0 0 0 c-u0*R0] = [p10 p9 0 0 0 0 0 0 0 0 p0] */ + t0 = c & M; c >>= 26; c += u0 * R1; + VERIFY_BITS(t0, 26); + VERIFY_BITS(c, 37); + /* [d u0 t9 0 0 0 0 0 0 0 c-u0*R1 t0-u0*R0] = [p10 p9 0 0 0 0 0 0 0 0 p0] */ + /* [d 0 t9 0 0 0 0 0 0 0 c t0] = [p10 p9 0 0 0 0 0 0 0 0 p0] */ + + c += (uint64_t)(a[0]*2) * a[1]; + VERIFY_BITS(c, 62); + /* [d 0 t9 0 0 0 0 0 0 0 c t0] = [p10 p9 0 0 0 0 0 0 0 p1 p0] */ + d += (uint64_t)(a[2]*2) * a[9] + + (uint64_t)(a[3]*2) * a[8] + + (uint64_t)(a[4]*2) * a[7] + + (uint64_t)(a[5]*2) * a[6]; + VERIFY_BITS(d, 63); + /* [d 0 t9 0 0 0 0 0 0 0 c t0] = [p11 p10 p9 0 0 0 0 0 0 0 p1 p0] */ + u1 = d & M; d >>= 26; c += u1 * R0; + VERIFY_BITS(u1, 26); + VERIFY_BITS(d, 37); + VERIFY_BITS(c, 63); + /* [d u1 0 t9 0 0 0 0 0 0 0 c-u1*R0 t0] = [p11 p10 p9 0 0 0 0 0 0 0 p1 p0] */ + t1 = c & M; c >>= 26; c += u1 * R1; + VERIFY_BITS(t1, 26); + VERIFY_BITS(c, 38); + /* [d u1 0 t9 0 0 0 0 0 0 c-u1*R1 t1-u1*R0 t0] = [p11 p10 p9 0 0 0 0 0 0 0 p1 p0] */ + /* [d 0 0 t9 0 0 0 0 0 0 c t1 t0] = [p11 p10 p9 0 0 0 0 0 0 0 p1 p0] */ + + c += (uint64_t)(a[0]*2) * a[2] + + (uint64_t)a[1] * a[1]; + VERIFY_BITS(c, 62); + /* [d 0 0 t9 0 0 0 0 0 0 c t1 t0] = [p11 p10 p9 0 0 0 0 0 0 p2 p1 p0] */ + d += (uint64_t)(a[3]*2) * a[9] + + (uint64_t)(a[4]*2) * a[8] + + (uint64_t)(a[5]*2) * a[7] + + (uint64_t)a[6] * a[6]; + VERIFY_BITS(d, 63); + /* [d 0 0 t9 0 0 0 0 0 0 c t1 t0] = [p12 p11 p10 p9 0 0 0 0 0 0 p2 p1 p0] */ + u2 = d & M; d >>= 26; c += u2 * R0; + VERIFY_BITS(u2, 26); + VERIFY_BITS(d, 37); + VERIFY_BITS(c, 63); + /* [d u2 0 0 t9 0 0 0 0 0 0 c-u2*R0 t1 t0] = [p12 p11 p10 p9 0 0 0 0 0 0 p2 p1 p0] */ + t2 = c & M; c >>= 26; c += u2 * R1; + VERIFY_BITS(t2, 26); + VERIFY_BITS(c, 38); + /* [d u2 0 0 t9 0 0 0 0 0 c-u2*R1 t2-u2*R0 t1 t0] = [p12 p11 p10 p9 0 0 0 0 0 0 p2 p1 p0] */ + /* [d 0 0 0 t9 0 0 0 0 0 c t2 t1 t0] = [p12 p11 p10 p9 0 0 0 0 0 0 p2 p1 p0] */ + + c += (uint64_t)(a[0]*2) * a[3] + + (uint64_t)(a[1]*2) * a[2]; + VERIFY_BITS(c, 63); + /* [d 0 0 0 t9 0 0 0 0 0 c t2 t1 t0] = [p12 p11 p10 p9 0 0 0 0 0 p3 p2 p1 p0] */ + d += (uint64_t)(a[4]*2) * a[9] + + (uint64_t)(a[5]*2) * a[8] + + (uint64_t)(a[6]*2) * a[7]; + VERIFY_BITS(d, 63); + /* [d 0 0 0 t9 0 0 0 0 0 c t2 t1 t0] = [p13 p12 p11 p10 p9 0 0 0 0 0 p3 p2 p1 p0] */ + u3 = d & M; d >>= 26; c += u3 * R0; + VERIFY_BITS(u3, 26); + VERIFY_BITS(d, 37); + /* VERIFY_BITS(c, 64); */ + /* [d u3 0 0 0 t9 0 0 0 0 0 c-u3*R0 t2 t1 t0] = [p13 p12 p11 p10 p9 0 0 0 0 0 p3 p2 p1 p0] */ + t3 = c & M; c >>= 26; c += u3 * R1; + VERIFY_BITS(t3, 26); + VERIFY_BITS(c, 39); + /* [d u3 0 0 0 t9 0 0 0 0 c-u3*R1 t3-u3*R0 t2 t1 t0] = [p13 p12 p11 p10 p9 0 0 0 0 0 p3 p2 p1 p0] */ + /* [d 0 0 0 0 t9 0 0 0 0 c t3 t2 t1 t0] = [p13 p12 p11 p10 p9 0 0 0 0 0 p3 p2 p1 p0] */ + + c += (uint64_t)(a[0]*2) * a[4] + + (uint64_t)(a[1]*2) * a[3] + + (uint64_t)a[2] * a[2]; + VERIFY_BITS(c, 63); + /* [d 0 0 0 0 t9 0 0 0 0 c t3 t2 t1 t0] = [p13 p12 p11 p10 p9 0 0 0 0 p4 p3 p2 p1 p0] */ + d += (uint64_t)(a[5]*2) * a[9] + + (uint64_t)(a[6]*2) * a[8] + + (uint64_t)a[7] * a[7]; + VERIFY_BITS(d, 62); + /* [d 0 0 0 0 t9 0 0 0 0 c t3 t2 t1 t0] = [p14 p13 p12 p11 p10 p9 0 0 0 0 p4 p3 p2 p1 p0] */ + u4 = d & M; d >>= 26; c += u4 * R0; + VERIFY_BITS(u4, 26); + VERIFY_BITS(d, 36); + /* VERIFY_BITS(c, 64); */ + /* [d u4 0 0 0 0 t9 0 0 0 0 c-u4*R0 t3 t2 t1 t0] = [p14 p13 p12 p11 p10 p9 0 0 0 0 p4 p3 p2 p1 p0] */ + t4 = c & M; c >>= 26; c += u4 * R1; + VERIFY_BITS(t4, 26); + VERIFY_BITS(c, 39); + /* [d u4 0 0 0 0 t9 0 0 0 c-u4*R1 t4-u4*R0 t3 t2 t1 t0] = [p14 p13 p12 p11 p10 p9 0 0 0 0 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 t9 0 0 0 c t4 t3 t2 t1 t0] = [p14 p13 p12 p11 p10 p9 0 0 0 0 p4 p3 p2 p1 p0] */ + + c += (uint64_t)(a[0]*2) * a[5] + + (uint64_t)(a[1]*2) * a[4] + + (uint64_t)(a[2]*2) * a[3]; + VERIFY_BITS(c, 63); + /* [d 0 0 0 0 0 t9 0 0 0 c t4 t3 t2 t1 t0] = [p14 p13 p12 p11 p10 p9 0 0 0 p5 p4 p3 p2 p1 p0] */ + d += (uint64_t)(a[6]*2) * a[9] + + (uint64_t)(a[7]*2) * a[8]; + VERIFY_BITS(d, 62); + /* [d 0 0 0 0 0 t9 0 0 0 c t4 t3 t2 t1 t0] = [p15 p14 p13 p12 p11 p10 p9 0 0 0 p5 p4 p3 p2 p1 p0] */ + u5 = d & M; d >>= 26; c += u5 * R0; + VERIFY_BITS(u5, 26); + VERIFY_BITS(d, 36); + /* VERIFY_BITS(c, 64); */ + /* [d u5 0 0 0 0 0 t9 0 0 0 c-u5*R0 t4 t3 t2 t1 t0] = [p15 p14 p13 p12 p11 p10 p9 0 0 0 p5 p4 p3 p2 p1 p0] */ + t5 = c & M; c >>= 26; c += u5 * R1; + VERIFY_BITS(t5, 26); + VERIFY_BITS(c, 39); + /* [d u5 0 0 0 0 0 t9 0 0 c-u5*R1 t5-u5*R0 t4 t3 t2 t1 t0] = [p15 p14 p13 p12 p11 p10 p9 0 0 0 p5 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 0 t9 0 0 c t5 t4 t3 t2 t1 t0] = [p15 p14 p13 p12 p11 p10 p9 0 0 0 p5 p4 p3 p2 p1 p0] */ + + c += (uint64_t)(a[0]*2) * a[6] + + (uint64_t)(a[1]*2) * a[5] + + (uint64_t)(a[2]*2) * a[4] + + (uint64_t)a[3] * a[3]; + VERIFY_BITS(c, 63); + /* [d 0 0 0 0 0 0 t9 0 0 c t5 t4 t3 t2 t1 t0] = [p15 p14 p13 p12 p11 p10 p9 0 0 p6 p5 p4 p3 p2 p1 p0] */ + d += (uint64_t)(a[7]*2) * a[9] + + (uint64_t)a[8] * a[8]; + VERIFY_BITS(d, 61); + /* [d 0 0 0 0 0 0 t9 0 0 c t5 t4 t3 t2 t1 t0] = [p16 p15 p14 p13 p12 p11 p10 p9 0 0 p6 p5 p4 p3 p2 p1 p0] */ + u6 = d & M; d >>= 26; c += u6 * R0; + VERIFY_BITS(u6, 26); + VERIFY_BITS(d, 35); + /* VERIFY_BITS(c, 64); */ + /* [d u6 0 0 0 0 0 0 t9 0 0 c-u6*R0 t5 t4 t3 t2 t1 t0] = [p16 p15 p14 p13 p12 p11 p10 p9 0 0 p6 p5 p4 p3 p2 p1 p0] */ + t6 = c & M; c >>= 26; c += u6 * R1; + VERIFY_BITS(t6, 26); + VERIFY_BITS(c, 39); + /* [d u6 0 0 0 0 0 0 t9 0 c-u6*R1 t6-u6*R0 t5 t4 t3 t2 t1 t0] = [p16 p15 p14 p13 p12 p11 p10 p9 0 0 p6 p5 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 0 0 t9 0 c t6 t5 t4 t3 t2 t1 t0] = [p16 p15 p14 p13 p12 p11 p10 p9 0 0 p6 p5 p4 p3 p2 p1 p0] */ + + c += (uint64_t)(a[0]*2) * a[7] + + (uint64_t)(a[1]*2) * a[6] + + (uint64_t)(a[2]*2) * a[5] + + (uint64_t)(a[3]*2) * a[4]; + /* VERIFY_BITS(c, 64); */ + VERIFY_CHECK(c <= 0x8000007C00000007ULL); + /* [d 0 0 0 0 0 0 0 t9 0 c t6 t5 t4 t3 t2 t1 t0] = [p16 p15 p14 p13 p12 p11 p10 p9 0 p7 p6 p5 p4 p3 p2 p1 p0] */ + d += (uint64_t)(a[8]*2) * a[9]; + VERIFY_BITS(d, 58); + /* [d 0 0 0 0 0 0 0 t9 0 c t6 t5 t4 t3 t2 t1 t0] = [p17 p16 p15 p14 p13 p12 p11 p10 p9 0 p7 p6 p5 p4 p3 p2 p1 p0] */ + u7 = d & M; d >>= 26; c += u7 * R0; + VERIFY_BITS(u7, 26); + VERIFY_BITS(d, 32); + /* VERIFY_BITS(c, 64); */ + VERIFY_CHECK(c <= 0x800001703FFFC2F7ULL); + /* [d u7 0 0 0 0 0 0 0 t9 0 c-u7*R0 t6 t5 t4 t3 t2 t1 t0] = [p17 p16 p15 p14 p13 p12 p11 p10 p9 0 p7 p6 p5 p4 p3 p2 p1 p0] */ + t7 = c & M; c >>= 26; c += u7 * R1; + VERIFY_BITS(t7, 26); + VERIFY_BITS(c, 38); + /* [d u7 0 0 0 0 0 0 0 t9 c-u7*R1 t7-u7*R0 t6 t5 t4 t3 t2 t1 t0] = [p17 p16 p15 p14 p13 p12 p11 p10 p9 0 p7 p6 p5 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 0 0 0 t9 c t7 t6 t5 t4 t3 t2 t1 t0] = [p17 p16 p15 p14 p13 p12 p11 p10 p9 0 p7 p6 p5 p4 p3 p2 p1 p0] */ + + c += (uint64_t)(a[0]*2) * a[8] + + (uint64_t)(a[1]*2) * a[7] + + (uint64_t)(a[2]*2) * a[6] + + (uint64_t)(a[3]*2) * a[5] + + (uint64_t)a[4] * a[4]; + /* VERIFY_BITS(c, 64); */ + VERIFY_CHECK(c <= 0x9000007B80000008ULL); + /* [d 0 0 0 0 0 0 0 0 t9 c t7 t6 t5 t4 t3 t2 t1 t0] = [p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + d += (uint64_t)a[9] * a[9]; + VERIFY_BITS(d, 57); + /* [d 0 0 0 0 0 0 0 0 t9 c t7 t6 t5 t4 t3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + u8 = d & M; d >>= 26; c += u8 * R0; + VERIFY_BITS(u8, 26); + VERIFY_BITS(d, 31); + /* VERIFY_BITS(c, 64); */ + VERIFY_CHECK(c <= 0x9000016FBFFFC2F8ULL); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 t7 t6 t5 t4 t3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + + r[3] = t3; + VERIFY_BITS(r[3], 26); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 t7 t6 t5 t4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[4] = t4; + VERIFY_BITS(r[4], 26); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 t7 t6 t5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[5] = t5; + VERIFY_BITS(r[5], 26); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 t7 t6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[6] = t6; + VERIFY_BITS(r[6], 26); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 t7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[7] = t7; + VERIFY_BITS(r[7], 26); + /* [d u8 0 0 0 0 0 0 0 0 t9 c-u8*R0 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + + r[8] = c & M; c >>= 26; c += u8 * R1; + VERIFY_BITS(r[8], 26); + VERIFY_BITS(c, 39); + /* [d u8 0 0 0 0 0 0 0 0 t9+c-u8*R1 r8-u8*R0 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 0 0 0 0 t9+c r8 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + c += d * R0 + t9; + VERIFY_BITS(c, 45); + /* [d 0 0 0 0 0 0 0 0 0 c-d*R0 r8 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[9] = c & (M >> 4); c >>= 22; c += d * (R1 << 4); + VERIFY_BITS(r[9], 22); + VERIFY_BITS(c, 46); + /* [d 0 0 0 0 0 0 0 0 r9+((c-d*R1<<4)<<22)-d*R0 r8 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + /* [d 0 0 0 0 0 0 0 -d*R1 r9+(c<<22)-d*R0 r8 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + /* [r9+(c<<22) r8 r7 r6 r5 r4 r3 t2 t1 t0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + + d = c * (R0 >> 4) + t0; + VERIFY_BITS(d, 56); + /* [r9+(c<<22) r8 r7 r6 r5 r4 r3 t2 t1 d-c*R0>>4] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[0] = d & M; d >>= 26; + VERIFY_BITS(r[0], 26); + VERIFY_BITS(d, 30); + /* [r9+(c<<22) r8 r7 r6 r5 r4 r3 t2 t1+d r0-c*R0>>4] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + d += c * (R1 >> 4) + t1; + VERIFY_BITS(d, 53); + VERIFY_CHECK(d <= 0x10000003FFFFBFULL); + /* [r9+(c<<22) r8 r7 r6 r5 r4 r3 t2 d-c*R1>>4 r0-c*R0>>4] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + /* [r9 r8 r7 r6 r5 r4 r3 t2 d r0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[1] = d & M; d >>= 26; + VERIFY_BITS(r[1], 26); + VERIFY_BITS(d, 27); + VERIFY_CHECK(d <= 0x4000000ULL); + /* [r9 r8 r7 r6 r5 r4 r3 t2+d r1 r0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + d += t2; + VERIFY_BITS(d, 27); + /* [r9 r8 r7 r6 r5 r4 r3 d r1 r0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[2] = d; + VERIFY_BITS(r[2], 27); + /* [r9 r8 r7 r6 r5 r4 r3 r2 r1 r0] = [p18 p17 p16 p15 p14 p13 p12 p11 p10 p9 p8 p7 p6 p5 p4 p3 p2 p1 p0] */ +} + + +static void secp256k1_fe_mul(secp256k1_fe *r, const secp256k1_fe *a, const secp256k1_fe * SECP256K1_RESTRICT b) { +#ifdef VERIFY + VERIFY_CHECK(a->magnitude <= 8); + VERIFY_CHECK(b->magnitude <= 8); + secp256k1_fe_verify(a); + secp256k1_fe_verify(b); + VERIFY_CHECK(r != b); +#endif + secp256k1_fe_mul_inner(r->n, a->n, b->n); +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 0; + secp256k1_fe_verify(r); +#endif +} + +static void secp256k1_fe_sqr(secp256k1_fe *r, const secp256k1_fe *a) { +#ifdef VERIFY + VERIFY_CHECK(a->magnitude <= 8); + secp256k1_fe_verify(a); +#endif + secp256k1_fe_sqr_inner(r->n, a->n); +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 0; + secp256k1_fe_verify(r); +#endif +} + +static SECP256K1_INLINE void secp256k1_fe_cmov(secp256k1_fe *r, const secp256k1_fe *a, int flag) { + uint32_t mask0, mask1; + mask0 = flag + ~((uint32_t)0); + mask1 = ~mask0; + r->n[0] = (r->n[0] & mask0) | (a->n[0] & mask1); + r->n[1] = (r->n[1] & mask0) | (a->n[1] & mask1); + r->n[2] = (r->n[2] & mask0) | (a->n[2] & mask1); + r->n[3] = (r->n[3] & mask0) | (a->n[3] & mask1); + r->n[4] = (r->n[4] & mask0) | (a->n[4] & mask1); + r->n[5] = (r->n[5] & mask0) | (a->n[5] & mask1); + r->n[6] = (r->n[6] & mask0) | (a->n[6] & mask1); + r->n[7] = (r->n[7] & mask0) | (a->n[7] & mask1); + r->n[8] = (r->n[8] & mask0) | (a->n[8] & mask1); + r->n[9] = (r->n[9] & mask0) | (a->n[9] & mask1); +#ifdef VERIFY + if (a->magnitude > r->magnitude) { + r->magnitude = a->magnitude; + } + r->normalized &= a->normalized; +#endif +} + +static SECP256K1_INLINE void secp256k1_fe_storage_cmov(secp256k1_fe_storage *r, const secp256k1_fe_storage *a, int flag) { + uint32_t mask0, mask1; + mask0 = flag + ~((uint32_t)0); + mask1 = ~mask0; + r->n[0] = (r->n[0] & mask0) | (a->n[0] & mask1); + r->n[1] = (r->n[1] & mask0) | (a->n[1] & mask1); + r->n[2] = (r->n[2] & mask0) | (a->n[2] & mask1); + r->n[3] = (r->n[3] & mask0) | (a->n[3] & mask1); + r->n[4] = (r->n[4] & mask0) | (a->n[4] & mask1); + r->n[5] = (r->n[5] & mask0) | (a->n[5] & mask1); + r->n[6] = (r->n[6] & mask0) | (a->n[6] & mask1); + r->n[7] = (r->n[7] & mask0) | (a->n[7] & mask1); +} + +static void secp256k1_fe_to_storage(secp256k1_fe_storage *r, const secp256k1_fe *a) { +#ifdef VERIFY + VERIFY_CHECK(a->normalized); +#endif + r->n[0] = a->n[0] | a->n[1] << 26; + r->n[1] = a->n[1] >> 6 | a->n[2] << 20; + r->n[2] = a->n[2] >> 12 | a->n[3] << 14; + r->n[3] = a->n[3] >> 18 | a->n[4] << 8; + r->n[4] = a->n[4] >> 24 | a->n[5] << 2 | a->n[6] << 28; + r->n[5] = a->n[6] >> 4 | a->n[7] << 22; + r->n[6] = a->n[7] >> 10 | a->n[8] << 16; + r->n[7] = a->n[8] >> 16 | a->n[9] << 10; +} + +static SECP256K1_INLINE void secp256k1_fe_from_storage(secp256k1_fe *r, const secp256k1_fe_storage *a) { + r->n[0] = a->n[0] & 0x3FFFFFFUL; + r->n[1] = a->n[0] >> 26 | ((a->n[1] << 6) & 0x3FFFFFFUL); + r->n[2] = a->n[1] >> 20 | ((a->n[2] << 12) & 0x3FFFFFFUL); + r->n[3] = a->n[2] >> 14 | ((a->n[3] << 18) & 0x3FFFFFFUL); + r->n[4] = a->n[3] >> 8 | ((a->n[4] << 24) & 0x3FFFFFFUL); + r->n[5] = (a->n[4] >> 2) & 0x3FFFFFFUL; + r->n[6] = a->n[4] >> 28 | ((a->n[5] << 4) & 0x3FFFFFFUL); + r->n[7] = a->n[5] >> 22 | ((a->n[6] << 10) & 0x3FFFFFFUL); + r->n[8] = a->n[6] >> 16 | ((a->n[7] << 16) & 0x3FFFFFFUL); + r->n[9] = a->n[7] >> 10; +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 1; +#endif +} + +#endif diff --git a/src/secp256k1/src/field_5x52.h b/src/secp256k1/src/field_5x52.h new file mode 100644 index 00000000..8e69a560 --- /dev/null +++ b/src/secp256k1/src/field_5x52.h @@ -0,0 +1,47 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_FIELD_REPR_ +#define _SECP256K1_FIELD_REPR_ + +#include + +typedef struct { + /* X = sum(i=0..4, elem[i]*2^52) mod n */ + uint64_t n[5]; +#ifdef VERIFY + int magnitude; + int normalized; +#endif +} secp256k1_fe; + +/* Unpacks a constant into a overlapping multi-limbed FE element. */ +#define SECP256K1_FE_CONST_INNER(d7, d6, d5, d4, d3, d2, d1, d0) { \ + (d0) | (((uint64_t)(d1) & 0xFFFFFUL) << 32), \ + ((uint64_t)(d1) >> 20) | (((uint64_t)(d2)) << 12) | (((uint64_t)(d3) & 0xFFUL) << 44), \ + ((uint64_t)(d3) >> 8) | (((uint64_t)(d4) & 0xFFFFFFFUL) << 24), \ + ((uint64_t)(d4) >> 28) | (((uint64_t)(d5)) << 4) | (((uint64_t)(d6) & 0xFFFFUL) << 36), \ + ((uint64_t)(d6) >> 16) | (((uint64_t)(d7)) << 16) \ +} + +#ifdef VERIFY +#define SECP256K1_FE_CONST(d7, d6, d5, d4, d3, d2, d1, d0) {SECP256K1_FE_CONST_INNER((d7), (d6), (d5), (d4), (d3), (d2), (d1), (d0)), 1, 1} +#else +#define SECP256K1_FE_CONST(d7, d6, d5, d4, d3, d2, d1, d0) {SECP256K1_FE_CONST_INNER((d7), (d6), (d5), (d4), (d3), (d2), (d1), (d0))} +#endif + +typedef struct { + uint64_t n[4]; +} secp256k1_fe_storage; + +#define SECP256K1_FE_STORAGE_CONST(d7, d6, d5, d4, d3, d2, d1, d0) {{ \ + (d0) | (((uint64_t)(d1)) << 32), \ + (d2) | (((uint64_t)(d3)) << 32), \ + (d4) | (((uint64_t)(d5)) << 32), \ + (d6) | (((uint64_t)(d7)) << 32) \ +}} + +#endif diff --git a/src/secp256k1/src/field_5x52_asm_impl.h b/src/secp256k1/src/field_5x52_asm_impl.h new file mode 100644 index 00000000..98cc004b --- /dev/null +++ b/src/secp256k1/src/field_5x52_asm_impl.h @@ -0,0 +1,502 @@ +/********************************************************************** + * Copyright (c) 2013-2014 Diederik Huys, Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +/** + * Changelog: + * - March 2013, Diederik Huys: original version + * - November 2014, Pieter Wuille: updated to use Peter Dettman's parallel multiplication algorithm + * - December 2014, Pieter Wuille: converted from YASM to GCC inline assembly + */ + +#ifndef _SECP256K1_FIELD_INNER5X52_IMPL_H_ +#define _SECP256K1_FIELD_INNER5X52_IMPL_H_ + +SECP256K1_INLINE static void secp256k1_fe_mul_inner(uint64_t *r, const uint64_t *a, const uint64_t * SECP256K1_RESTRICT b) { +/** + * Registers: rdx:rax = multiplication accumulator + * r9:r8 = c + * r15:rcx = d + * r10-r14 = a0-a4 + * rbx = b + * rdi = r + * rsi = a / t? + */ + uint64_t tmp1, tmp2, tmp3; +__asm__ __volatile__( + "movq 0(%%rsi),%%r10\n" + "movq 8(%%rsi),%%r11\n" + "movq 16(%%rsi),%%r12\n" + "movq 24(%%rsi),%%r13\n" + "movq 32(%%rsi),%%r14\n" + + /* d += a3 * b0 */ + "movq 0(%%rbx),%%rax\n" + "mulq %%r13\n" + "movq %%rax,%%rcx\n" + "movq %%rdx,%%r15\n" + /* d += a2 * b1 */ + "movq 8(%%rbx),%%rax\n" + "mulq %%r12\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += a1 * b2 */ + "movq 16(%%rbx),%%rax\n" + "mulq %%r11\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d = a0 * b3 */ + "movq 24(%%rbx),%%rax\n" + "mulq %%r10\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* c = a4 * b4 */ + "movq 32(%%rbx),%%rax\n" + "mulq %%r14\n" + "movq %%rax,%%r8\n" + "movq %%rdx,%%r9\n" + /* d += (c & M) * R */ + "movq $0xfffffffffffff,%%rdx\n" + "andq %%rdx,%%rax\n" + "movq $0x1000003d10,%%rdx\n" + "mulq %%rdx\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* c >>= 52 (%%r8 only) */ + "shrdq $52,%%r9,%%r8\n" + /* t3 (tmp1) = d & M */ + "movq %%rcx,%%rsi\n" + "movq $0xfffffffffffff,%%rdx\n" + "andq %%rdx,%%rsi\n" + "movq %%rsi,%q1\n" + /* d >>= 52 */ + "shrdq $52,%%r15,%%rcx\n" + "xorq %%r15,%%r15\n" + /* d += a4 * b0 */ + "movq 0(%%rbx),%%rax\n" + "mulq %%r14\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += a3 * b1 */ + "movq 8(%%rbx),%%rax\n" + "mulq %%r13\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += a2 * b2 */ + "movq 16(%%rbx),%%rax\n" + "mulq %%r12\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += a1 * b3 */ + "movq 24(%%rbx),%%rax\n" + "mulq %%r11\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += a0 * b4 */ + "movq 32(%%rbx),%%rax\n" + "mulq %%r10\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += c * R */ + "movq %%r8,%%rax\n" + "movq $0x1000003d10,%%rdx\n" + "mulq %%rdx\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* t4 = d & M (%%rsi) */ + "movq %%rcx,%%rsi\n" + "movq $0xfffffffffffff,%%rdx\n" + "andq %%rdx,%%rsi\n" + /* d >>= 52 */ + "shrdq $52,%%r15,%%rcx\n" + "xorq %%r15,%%r15\n" + /* tx = t4 >> 48 (tmp3) */ + "movq %%rsi,%%rax\n" + "shrq $48,%%rax\n" + "movq %%rax,%q3\n" + /* t4 &= (M >> 4) (tmp2) */ + "movq $0xffffffffffff,%%rax\n" + "andq %%rax,%%rsi\n" + "movq %%rsi,%q2\n" + /* c = a0 * b0 */ + "movq 0(%%rbx),%%rax\n" + "mulq %%r10\n" + "movq %%rax,%%r8\n" + "movq %%rdx,%%r9\n" + /* d += a4 * b1 */ + "movq 8(%%rbx),%%rax\n" + "mulq %%r14\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += a3 * b2 */ + "movq 16(%%rbx),%%rax\n" + "mulq %%r13\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += a2 * b3 */ + "movq 24(%%rbx),%%rax\n" + "mulq %%r12\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += a1 * b4 */ + "movq 32(%%rbx),%%rax\n" + "mulq %%r11\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* u0 = d & M (%%rsi) */ + "movq %%rcx,%%rsi\n" + "movq $0xfffffffffffff,%%rdx\n" + "andq %%rdx,%%rsi\n" + /* d >>= 52 */ + "shrdq $52,%%r15,%%rcx\n" + "xorq %%r15,%%r15\n" + /* u0 = (u0 << 4) | tx (%%rsi) */ + "shlq $4,%%rsi\n" + "movq %q3,%%rax\n" + "orq %%rax,%%rsi\n" + /* c += u0 * (R >> 4) */ + "movq $0x1000003d1,%%rax\n" + "mulq %%rsi\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* r[0] = c & M */ + "movq %%r8,%%rax\n" + "movq $0xfffffffffffff,%%rdx\n" + "andq %%rdx,%%rax\n" + "movq %%rax,0(%%rdi)\n" + /* c >>= 52 */ + "shrdq $52,%%r9,%%r8\n" + "xorq %%r9,%%r9\n" + /* c += a1 * b0 */ + "movq 0(%%rbx),%%rax\n" + "mulq %%r11\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* c += a0 * b1 */ + "movq 8(%%rbx),%%rax\n" + "mulq %%r10\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* d += a4 * b2 */ + "movq 16(%%rbx),%%rax\n" + "mulq %%r14\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += a3 * b3 */ + "movq 24(%%rbx),%%rax\n" + "mulq %%r13\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += a2 * b4 */ + "movq 32(%%rbx),%%rax\n" + "mulq %%r12\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* c += (d & M) * R */ + "movq %%rcx,%%rax\n" + "movq $0xfffffffffffff,%%rdx\n" + "andq %%rdx,%%rax\n" + "movq $0x1000003d10,%%rdx\n" + "mulq %%rdx\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* d >>= 52 */ + "shrdq $52,%%r15,%%rcx\n" + "xorq %%r15,%%r15\n" + /* r[1] = c & M */ + "movq %%r8,%%rax\n" + "movq $0xfffffffffffff,%%rdx\n" + "andq %%rdx,%%rax\n" + "movq %%rax,8(%%rdi)\n" + /* c >>= 52 */ + "shrdq $52,%%r9,%%r8\n" + "xorq %%r9,%%r9\n" + /* c += a2 * b0 */ + "movq 0(%%rbx),%%rax\n" + "mulq %%r12\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* c += a1 * b1 */ + "movq 8(%%rbx),%%rax\n" + "mulq %%r11\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* c += a0 * b2 (last use of %%r10 = a0) */ + "movq 16(%%rbx),%%rax\n" + "mulq %%r10\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* fetch t3 (%%r10, overwrites a0), t4 (%%rsi) */ + "movq %q2,%%rsi\n" + "movq %q1,%%r10\n" + /* d += a4 * b3 */ + "movq 24(%%rbx),%%rax\n" + "mulq %%r14\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* d += a3 * b4 */ + "movq 32(%%rbx),%%rax\n" + "mulq %%r13\n" + "addq %%rax,%%rcx\n" + "adcq %%rdx,%%r15\n" + /* c += (d & M) * R */ + "movq %%rcx,%%rax\n" + "movq $0xfffffffffffff,%%rdx\n" + "andq %%rdx,%%rax\n" + "movq $0x1000003d10,%%rdx\n" + "mulq %%rdx\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* d >>= 52 (%%rcx only) */ + "shrdq $52,%%r15,%%rcx\n" + /* r[2] = c & M */ + "movq %%r8,%%rax\n" + "movq $0xfffffffffffff,%%rdx\n" + "andq %%rdx,%%rax\n" + "movq %%rax,16(%%rdi)\n" + /* c >>= 52 */ + "shrdq $52,%%r9,%%r8\n" + "xorq %%r9,%%r9\n" + /* c += t3 */ + "addq %%r10,%%r8\n" + /* c += d * R */ + "movq %%rcx,%%rax\n" + "movq $0x1000003d10,%%rdx\n" + "mulq %%rdx\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* r[3] = c & M */ + "movq %%r8,%%rax\n" + "movq $0xfffffffffffff,%%rdx\n" + "andq %%rdx,%%rax\n" + "movq %%rax,24(%%rdi)\n" + /* c >>= 52 (%%r8 only) */ + "shrdq $52,%%r9,%%r8\n" + /* c += t4 (%%r8 only) */ + "addq %%rsi,%%r8\n" + /* r[4] = c */ + "movq %%r8,32(%%rdi)\n" +: "+S"(a), "=m"(tmp1), "=m"(tmp2), "=m"(tmp3) +: "b"(b), "D"(r) +: "%rax", "%rcx", "%rdx", "%r8", "%r9", "%r10", "%r11", "%r12", "%r13", "%r14", "%r15", "cc", "memory" +); +} + +SECP256K1_INLINE static void secp256k1_fe_sqr_inner(uint64_t *r, const uint64_t *a) { +/** + * Registers: rdx:rax = multiplication accumulator + * r9:r8 = c + * rcx:rbx = d + * r10-r14 = a0-a4 + * r15 = M (0xfffffffffffff) + * rdi = r + * rsi = a / t? + */ + uint64_t tmp1, tmp2, tmp3; +__asm__ __volatile__( + "movq 0(%%rsi),%%r10\n" + "movq 8(%%rsi),%%r11\n" + "movq 16(%%rsi),%%r12\n" + "movq 24(%%rsi),%%r13\n" + "movq 32(%%rsi),%%r14\n" + "movq $0xfffffffffffff,%%r15\n" + + /* d = (a0*2) * a3 */ + "leaq (%%r10,%%r10,1),%%rax\n" + "mulq %%r13\n" + "movq %%rax,%%rbx\n" + "movq %%rdx,%%rcx\n" + /* d += (a1*2) * a2 */ + "leaq (%%r11,%%r11,1),%%rax\n" + "mulq %%r12\n" + "addq %%rax,%%rbx\n" + "adcq %%rdx,%%rcx\n" + /* c = a4 * a4 */ + "movq %%r14,%%rax\n" + "mulq %%r14\n" + "movq %%rax,%%r8\n" + "movq %%rdx,%%r9\n" + /* d += (c & M) * R */ + "andq %%r15,%%rax\n" + "movq $0x1000003d10,%%rdx\n" + "mulq %%rdx\n" + "addq %%rax,%%rbx\n" + "adcq %%rdx,%%rcx\n" + /* c >>= 52 (%%r8 only) */ + "shrdq $52,%%r9,%%r8\n" + /* t3 (tmp1) = d & M */ + "movq %%rbx,%%rsi\n" + "andq %%r15,%%rsi\n" + "movq %%rsi,%q1\n" + /* d >>= 52 */ + "shrdq $52,%%rcx,%%rbx\n" + "xorq %%rcx,%%rcx\n" + /* a4 *= 2 */ + "addq %%r14,%%r14\n" + /* d += a0 * a4 */ + "movq %%r10,%%rax\n" + "mulq %%r14\n" + "addq %%rax,%%rbx\n" + "adcq %%rdx,%%rcx\n" + /* d+= (a1*2) * a3 */ + "leaq (%%r11,%%r11,1),%%rax\n" + "mulq %%r13\n" + "addq %%rax,%%rbx\n" + "adcq %%rdx,%%rcx\n" + /* d += a2 * a2 */ + "movq %%r12,%%rax\n" + "mulq %%r12\n" + "addq %%rax,%%rbx\n" + "adcq %%rdx,%%rcx\n" + /* d += c * R */ + "movq %%r8,%%rax\n" + "movq $0x1000003d10,%%rdx\n" + "mulq %%rdx\n" + "addq %%rax,%%rbx\n" + "adcq %%rdx,%%rcx\n" + /* t4 = d & M (%%rsi) */ + "movq %%rbx,%%rsi\n" + "andq %%r15,%%rsi\n" + /* d >>= 52 */ + "shrdq $52,%%rcx,%%rbx\n" + "xorq %%rcx,%%rcx\n" + /* tx = t4 >> 48 (tmp3) */ + "movq %%rsi,%%rax\n" + "shrq $48,%%rax\n" + "movq %%rax,%q3\n" + /* t4 &= (M >> 4) (tmp2) */ + "movq $0xffffffffffff,%%rax\n" + "andq %%rax,%%rsi\n" + "movq %%rsi,%q2\n" + /* c = a0 * a0 */ + "movq %%r10,%%rax\n" + "mulq %%r10\n" + "movq %%rax,%%r8\n" + "movq %%rdx,%%r9\n" + /* d += a1 * a4 */ + "movq %%r11,%%rax\n" + "mulq %%r14\n" + "addq %%rax,%%rbx\n" + "adcq %%rdx,%%rcx\n" + /* d += (a2*2) * a3 */ + "leaq (%%r12,%%r12,1),%%rax\n" + "mulq %%r13\n" + "addq %%rax,%%rbx\n" + "adcq %%rdx,%%rcx\n" + /* u0 = d & M (%%rsi) */ + "movq %%rbx,%%rsi\n" + "andq %%r15,%%rsi\n" + /* d >>= 52 */ + "shrdq $52,%%rcx,%%rbx\n" + "xorq %%rcx,%%rcx\n" + /* u0 = (u0 << 4) | tx (%%rsi) */ + "shlq $4,%%rsi\n" + "movq %q3,%%rax\n" + "orq %%rax,%%rsi\n" + /* c += u0 * (R >> 4) */ + "movq $0x1000003d1,%%rax\n" + "mulq %%rsi\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* r[0] = c & M */ + "movq %%r8,%%rax\n" + "andq %%r15,%%rax\n" + "movq %%rax,0(%%rdi)\n" + /* c >>= 52 */ + "shrdq $52,%%r9,%%r8\n" + "xorq %%r9,%%r9\n" + /* a0 *= 2 */ + "addq %%r10,%%r10\n" + /* c += a0 * a1 */ + "movq %%r10,%%rax\n" + "mulq %%r11\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* d += a2 * a4 */ + "movq %%r12,%%rax\n" + "mulq %%r14\n" + "addq %%rax,%%rbx\n" + "adcq %%rdx,%%rcx\n" + /* d += a3 * a3 */ + "movq %%r13,%%rax\n" + "mulq %%r13\n" + "addq %%rax,%%rbx\n" + "adcq %%rdx,%%rcx\n" + /* c += (d & M) * R */ + "movq %%rbx,%%rax\n" + "andq %%r15,%%rax\n" + "movq $0x1000003d10,%%rdx\n" + "mulq %%rdx\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* d >>= 52 */ + "shrdq $52,%%rcx,%%rbx\n" + "xorq %%rcx,%%rcx\n" + /* r[1] = c & M */ + "movq %%r8,%%rax\n" + "andq %%r15,%%rax\n" + "movq %%rax,8(%%rdi)\n" + /* c >>= 52 */ + "shrdq $52,%%r9,%%r8\n" + "xorq %%r9,%%r9\n" + /* c += a0 * a2 (last use of %%r10) */ + "movq %%r10,%%rax\n" + "mulq %%r12\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* fetch t3 (%%r10, overwrites a0),t4 (%%rsi) */ + "movq %q2,%%rsi\n" + "movq %q1,%%r10\n" + /* c += a1 * a1 */ + "movq %%r11,%%rax\n" + "mulq %%r11\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* d += a3 * a4 */ + "movq %%r13,%%rax\n" + "mulq %%r14\n" + "addq %%rax,%%rbx\n" + "adcq %%rdx,%%rcx\n" + /* c += (d & M) * R */ + "movq %%rbx,%%rax\n" + "andq %%r15,%%rax\n" + "movq $0x1000003d10,%%rdx\n" + "mulq %%rdx\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* d >>= 52 (%%rbx only) */ + "shrdq $52,%%rcx,%%rbx\n" + /* r[2] = c & M */ + "movq %%r8,%%rax\n" + "andq %%r15,%%rax\n" + "movq %%rax,16(%%rdi)\n" + /* c >>= 52 */ + "shrdq $52,%%r9,%%r8\n" + "xorq %%r9,%%r9\n" + /* c += t3 */ + "addq %%r10,%%r8\n" + /* c += d * R */ + "movq %%rbx,%%rax\n" + "movq $0x1000003d10,%%rdx\n" + "mulq %%rdx\n" + "addq %%rax,%%r8\n" + "adcq %%rdx,%%r9\n" + /* r[3] = c & M */ + "movq %%r8,%%rax\n" + "andq %%r15,%%rax\n" + "movq %%rax,24(%%rdi)\n" + /* c >>= 52 (%%r8 only) */ + "shrdq $52,%%r9,%%r8\n" + /* c += t4 (%%r8 only) */ + "addq %%rsi,%%r8\n" + /* r[4] = c */ + "movq %%r8,32(%%rdi)\n" +: "+S"(a), "=m"(tmp1), "=m"(tmp2), "=m"(tmp3) +: "D"(r) +: "%rax", "%rbx", "%rcx", "%rdx", "%r8", "%r9", "%r10", "%r11", "%r12", "%r13", "%r14", "%r15", "cc", "memory" +); +} + +#endif diff --git a/src/secp256k1/src/field_5x52_impl.h b/src/secp256k1/src/field_5x52_impl.h new file mode 100644 index 00000000..b31e24ab --- /dev/null +++ b/src/secp256k1/src/field_5x52_impl.h @@ -0,0 +1,456 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_FIELD_REPR_IMPL_H_ +#define _SECP256K1_FIELD_REPR_IMPL_H_ + +#if defined HAVE_CONFIG_H +#include "libsecp256k1-config.h" +#endif + +#include +#include "util.h" +#include "num.h" +#include "field.h" + +#if defined(USE_ASM_X86_64) +#include "field_5x52_asm_impl.h" +#else +#include "field_5x52_int128_impl.h" +#endif + +/** Implements arithmetic modulo FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFE FFFFFC2F, + * represented as 5 uint64_t's in base 2^52. The values are allowed to contain >52 each. In particular, + * each FieldElem has a 'magnitude' associated with it. Internally, a magnitude M means each element + * is at most M*(2^53-1), except the most significant one, which is limited to M*(2^49-1). All operations + * accept any input with magnitude at most M, and have different rules for propagating magnitude to their + * output. + */ + +#ifdef VERIFY +static void secp256k1_fe_verify(const secp256k1_fe *a) { + const uint64_t *d = a->n; + int m = a->normalized ? 1 : 2 * a->magnitude, r = 1; + /* secp256k1 'p' value defined in "Standards for Efficient Cryptography" (SEC2) 2.7.1. */ + r &= (d[0] <= 0xFFFFFFFFFFFFFULL * m); + r &= (d[1] <= 0xFFFFFFFFFFFFFULL * m); + r &= (d[2] <= 0xFFFFFFFFFFFFFULL * m); + r &= (d[3] <= 0xFFFFFFFFFFFFFULL * m); + r &= (d[4] <= 0x0FFFFFFFFFFFFULL * m); + r &= (a->magnitude >= 0); + r &= (a->magnitude <= 2048); + if (a->normalized) { + r &= (a->magnitude <= 1); + if (r && (d[4] == 0x0FFFFFFFFFFFFULL) && ((d[3] & d[2] & d[1]) == 0xFFFFFFFFFFFFFULL)) { + r &= (d[0] < 0xFFFFEFFFFFC2FULL); + } + } + VERIFY_CHECK(r == 1); +} +#else +static void secp256k1_fe_verify(const secp256k1_fe *a) { + (void)a; +} +#endif + +static void secp256k1_fe_normalize(secp256k1_fe *r) { + uint64_t t0 = r->n[0], t1 = r->n[1], t2 = r->n[2], t3 = r->n[3], t4 = r->n[4]; + + /* Reduce t4 at the start so there will be at most a single carry from the first pass */ + uint64_t m; + uint64_t x = t4 >> 48; t4 &= 0x0FFFFFFFFFFFFULL; + + /* The first pass ensures the magnitude is 1, ... */ + t0 += x * 0x1000003D1ULL; + t1 += (t0 >> 52); t0 &= 0xFFFFFFFFFFFFFULL; + t2 += (t1 >> 52); t1 &= 0xFFFFFFFFFFFFFULL; m = t1; + t3 += (t2 >> 52); t2 &= 0xFFFFFFFFFFFFFULL; m &= t2; + t4 += (t3 >> 52); t3 &= 0xFFFFFFFFFFFFFULL; m &= t3; + + /* ... except for a possible carry at bit 48 of t4 (i.e. bit 256 of the field element) */ + VERIFY_CHECK(t4 >> 49 == 0); + + /* At most a single final reduction is needed; check if the value is >= the field characteristic */ + x = (t4 >> 48) | ((t4 == 0x0FFFFFFFFFFFFULL) & (m == 0xFFFFFFFFFFFFFULL) + & (t0 >= 0xFFFFEFFFFFC2FULL)); + + /* Apply the final reduction (for constant-time behaviour, we do it always) */ + t0 += x * 0x1000003D1ULL; + t1 += (t0 >> 52); t0 &= 0xFFFFFFFFFFFFFULL; + t2 += (t1 >> 52); t1 &= 0xFFFFFFFFFFFFFULL; + t3 += (t2 >> 52); t2 &= 0xFFFFFFFFFFFFFULL; + t4 += (t3 >> 52); t3 &= 0xFFFFFFFFFFFFFULL; + + /* If t4 didn't carry to bit 48 already, then it should have after any final reduction */ + VERIFY_CHECK(t4 >> 48 == x); + + /* Mask off the possible multiple of 2^256 from the final reduction */ + t4 &= 0x0FFFFFFFFFFFFULL; + + r->n[0] = t0; r->n[1] = t1; r->n[2] = t2; r->n[3] = t3; r->n[4] = t4; + +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 1; + secp256k1_fe_verify(r); +#endif +} + +static void secp256k1_fe_normalize_weak(secp256k1_fe *r) { + uint64_t t0 = r->n[0], t1 = r->n[1], t2 = r->n[2], t3 = r->n[3], t4 = r->n[4]; + + /* Reduce t4 at the start so there will be at most a single carry from the first pass */ + uint64_t x = t4 >> 48; t4 &= 0x0FFFFFFFFFFFFULL; + + /* The first pass ensures the magnitude is 1, ... */ + t0 += x * 0x1000003D1ULL; + t1 += (t0 >> 52); t0 &= 0xFFFFFFFFFFFFFULL; + t2 += (t1 >> 52); t1 &= 0xFFFFFFFFFFFFFULL; + t3 += (t2 >> 52); t2 &= 0xFFFFFFFFFFFFFULL; + t4 += (t3 >> 52); t3 &= 0xFFFFFFFFFFFFFULL; + + /* ... except for a possible carry at bit 48 of t4 (i.e. bit 256 of the field element) */ + VERIFY_CHECK(t4 >> 49 == 0); + + r->n[0] = t0; r->n[1] = t1; r->n[2] = t2; r->n[3] = t3; r->n[4] = t4; + +#ifdef VERIFY + r->magnitude = 1; + secp256k1_fe_verify(r); +#endif +} + +static void secp256k1_fe_normalize_var(secp256k1_fe *r) { + uint64_t t0 = r->n[0], t1 = r->n[1], t2 = r->n[2], t3 = r->n[3], t4 = r->n[4]; + + /* Reduce t4 at the start so there will be at most a single carry from the first pass */ + uint64_t m; + uint64_t x = t4 >> 48; t4 &= 0x0FFFFFFFFFFFFULL; + + /* The first pass ensures the magnitude is 1, ... */ + t0 += x * 0x1000003D1ULL; + t1 += (t0 >> 52); t0 &= 0xFFFFFFFFFFFFFULL; + t2 += (t1 >> 52); t1 &= 0xFFFFFFFFFFFFFULL; m = t1; + t3 += (t2 >> 52); t2 &= 0xFFFFFFFFFFFFFULL; m &= t2; + t4 += (t3 >> 52); t3 &= 0xFFFFFFFFFFFFFULL; m &= t3; + + /* ... except for a possible carry at bit 48 of t4 (i.e. bit 256 of the field element) */ + VERIFY_CHECK(t4 >> 49 == 0); + + /* At most a single final reduction is needed; check if the value is >= the field characteristic */ + x = (t4 >> 48) | ((t4 == 0x0FFFFFFFFFFFFULL) & (m == 0xFFFFFFFFFFFFFULL) + & (t0 >= 0xFFFFEFFFFFC2FULL)); + + if (x) { + t0 += 0x1000003D1ULL; + t1 += (t0 >> 52); t0 &= 0xFFFFFFFFFFFFFULL; + t2 += (t1 >> 52); t1 &= 0xFFFFFFFFFFFFFULL; + t3 += (t2 >> 52); t2 &= 0xFFFFFFFFFFFFFULL; + t4 += (t3 >> 52); t3 &= 0xFFFFFFFFFFFFFULL; + + /* If t4 didn't carry to bit 48 already, then it should have after any final reduction */ + VERIFY_CHECK(t4 >> 48 == x); + + /* Mask off the possible multiple of 2^256 from the final reduction */ + t4 &= 0x0FFFFFFFFFFFFULL; + } + + r->n[0] = t0; r->n[1] = t1; r->n[2] = t2; r->n[3] = t3; r->n[4] = t4; + +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 1; + secp256k1_fe_verify(r); +#endif +} + +static int secp256k1_fe_normalizes_to_zero(secp256k1_fe *r) { + uint64_t t0 = r->n[0], t1 = r->n[1], t2 = r->n[2], t3 = r->n[3], t4 = r->n[4]; + + /* z0 tracks a possible raw value of 0, z1 tracks a possible raw value of P */ + uint64_t z0, z1; + + /* Reduce t4 at the start so there will be at most a single carry from the first pass */ + uint64_t x = t4 >> 48; t4 &= 0x0FFFFFFFFFFFFULL; + + /* The first pass ensures the magnitude is 1, ... */ + t0 += x * 0x1000003D1ULL; + t1 += (t0 >> 52); t0 &= 0xFFFFFFFFFFFFFULL; z0 = t0; z1 = t0 ^ 0x1000003D0ULL; + t2 += (t1 >> 52); t1 &= 0xFFFFFFFFFFFFFULL; z0 |= t1; z1 &= t1; + t3 += (t2 >> 52); t2 &= 0xFFFFFFFFFFFFFULL; z0 |= t2; z1 &= t2; + t4 += (t3 >> 52); t3 &= 0xFFFFFFFFFFFFFULL; z0 |= t3; z1 &= t3; + z0 |= t4; z1 &= t4 ^ 0xF000000000000ULL; + + /* ... except for a possible carry at bit 48 of t4 (i.e. bit 256 of the field element) */ + VERIFY_CHECK(t4 >> 49 == 0); + + return (z0 == 0) | (z1 == 0xFFFFFFFFFFFFFULL); +} + +static int secp256k1_fe_normalizes_to_zero_var(secp256k1_fe *r) { + uint64_t t0, t1, t2, t3, t4; + uint64_t z0, z1; + uint64_t x; + + t0 = r->n[0]; + t4 = r->n[4]; + + /* Reduce t4 at the start so there will be at most a single carry from the first pass */ + x = t4 >> 48; + + /* The first pass ensures the magnitude is 1, ... */ + t0 += x * 0x1000003D1ULL; + + /* z0 tracks a possible raw value of 0, z1 tracks a possible raw value of P */ + z0 = t0 & 0xFFFFFFFFFFFFFULL; + z1 = z0 ^ 0x1000003D0ULL; + + /* Fast return path should catch the majority of cases */ + if ((z0 != 0ULL) & (z1 != 0xFFFFFFFFFFFFFULL)) { + return 0; + } + + t1 = r->n[1]; + t2 = r->n[2]; + t3 = r->n[3]; + + t4 &= 0x0FFFFFFFFFFFFULL; + + t1 += (t0 >> 52); + t2 += (t1 >> 52); t1 &= 0xFFFFFFFFFFFFFULL; z0 |= t1; z1 &= t1; + t3 += (t2 >> 52); t2 &= 0xFFFFFFFFFFFFFULL; z0 |= t2; z1 &= t2; + t4 += (t3 >> 52); t3 &= 0xFFFFFFFFFFFFFULL; z0 |= t3; z1 &= t3; + z0 |= t4; z1 &= t4 ^ 0xF000000000000ULL; + + /* ... except for a possible carry at bit 48 of t4 (i.e. bit 256 of the field element) */ + VERIFY_CHECK(t4 >> 49 == 0); + + return (z0 == 0) | (z1 == 0xFFFFFFFFFFFFFULL); +} + +SECP256K1_INLINE static void secp256k1_fe_set_int(secp256k1_fe *r, int a) { + r->n[0] = a; + r->n[1] = r->n[2] = r->n[3] = r->n[4] = 0; +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 1; + secp256k1_fe_verify(r); +#endif +} + +SECP256K1_INLINE static int secp256k1_fe_is_zero(const secp256k1_fe *a) { + const uint64_t *t = a->n; +#ifdef VERIFY + VERIFY_CHECK(a->normalized); + secp256k1_fe_verify(a); +#endif + return (t[0] | t[1] | t[2] | t[3] | t[4]) == 0; +} + +SECP256K1_INLINE static int secp256k1_fe_is_odd(const secp256k1_fe *a) { +#ifdef VERIFY + VERIFY_CHECK(a->normalized); + secp256k1_fe_verify(a); +#endif + return a->n[0] & 1; +} + +SECP256K1_INLINE static void secp256k1_fe_clear(secp256k1_fe *a) { + int i; +#ifdef VERIFY + a->magnitude = 0; + a->normalized = 1; +#endif + for (i=0; i<5; i++) { + a->n[i] = 0; + } +} + +static int secp256k1_fe_cmp_var(const secp256k1_fe *a, const secp256k1_fe *b) { + int i; +#ifdef VERIFY + VERIFY_CHECK(a->normalized); + VERIFY_CHECK(b->normalized); + secp256k1_fe_verify(a); + secp256k1_fe_verify(b); +#endif + for (i = 4; i >= 0; i--) { + if (a->n[i] > b->n[i]) { + return 1; + } + if (a->n[i] < b->n[i]) { + return -1; + } + } + return 0; +} + +static int secp256k1_fe_set_b32(secp256k1_fe *r, const unsigned char *a) { + int i; + r->n[0] = r->n[1] = r->n[2] = r->n[3] = r->n[4] = 0; + for (i=0; i<32; i++) { + int j; + for (j=0; j<2; j++) { + int limb = (8*i+4*j)/52; + int shift = (8*i+4*j)%52; + r->n[limb] |= (uint64_t)((a[31-i] >> (4*j)) & 0xF) << shift; + } + } + if (r->n[4] == 0x0FFFFFFFFFFFFULL && (r->n[3] & r->n[2] & r->n[1]) == 0xFFFFFFFFFFFFFULL && r->n[0] >= 0xFFFFEFFFFFC2FULL) { + return 0; + } +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 1; + secp256k1_fe_verify(r); +#endif + return 1; +} + +/** Convert a field element to a 32-byte big endian value. Requires the input to be normalized */ +static void secp256k1_fe_get_b32(unsigned char *r, const secp256k1_fe *a) { + int i; +#ifdef VERIFY + VERIFY_CHECK(a->normalized); + secp256k1_fe_verify(a); +#endif + for (i=0; i<32; i++) { + int j; + int c = 0; + for (j=0; j<2; j++) { + int limb = (8*i+4*j)/52; + int shift = (8*i+4*j)%52; + c |= ((a->n[limb] >> shift) & 0xF) << (4 * j); + } + r[31-i] = c; + } +} + +SECP256K1_INLINE static void secp256k1_fe_negate(secp256k1_fe *r, const secp256k1_fe *a, int m) { +#ifdef VERIFY + VERIFY_CHECK(a->magnitude <= m); + secp256k1_fe_verify(a); +#endif + r->n[0] = 0xFFFFEFFFFFC2FULL * 2 * (m + 1) - a->n[0]; + r->n[1] = 0xFFFFFFFFFFFFFULL * 2 * (m + 1) - a->n[1]; + r->n[2] = 0xFFFFFFFFFFFFFULL * 2 * (m + 1) - a->n[2]; + r->n[3] = 0xFFFFFFFFFFFFFULL * 2 * (m + 1) - a->n[3]; + r->n[4] = 0x0FFFFFFFFFFFFULL * 2 * (m + 1) - a->n[4]; +#ifdef VERIFY + r->magnitude = m + 1; + r->normalized = 0; + secp256k1_fe_verify(r); +#endif +} + +SECP256K1_INLINE static void secp256k1_fe_mul_int(secp256k1_fe *r, int a) { + r->n[0] *= a; + r->n[1] *= a; + r->n[2] *= a; + r->n[3] *= a; + r->n[4] *= a; +#ifdef VERIFY + r->magnitude *= a; + r->normalized = 0; + secp256k1_fe_verify(r); +#endif +} + +SECP256K1_INLINE static void secp256k1_fe_add(secp256k1_fe *r, const secp256k1_fe *a) { +#ifdef VERIFY + secp256k1_fe_verify(a); +#endif + r->n[0] += a->n[0]; + r->n[1] += a->n[1]; + r->n[2] += a->n[2]; + r->n[3] += a->n[3]; + r->n[4] += a->n[4]; +#ifdef VERIFY + r->magnitude += a->magnitude; + r->normalized = 0; + secp256k1_fe_verify(r); +#endif +} + +static void secp256k1_fe_mul(secp256k1_fe *r, const secp256k1_fe *a, const secp256k1_fe * SECP256K1_RESTRICT b) { +#ifdef VERIFY + VERIFY_CHECK(a->magnitude <= 8); + VERIFY_CHECK(b->magnitude <= 8); + secp256k1_fe_verify(a); + secp256k1_fe_verify(b); + VERIFY_CHECK(r != b); +#endif + secp256k1_fe_mul_inner(r->n, a->n, b->n); +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 0; + secp256k1_fe_verify(r); +#endif +} + +static void secp256k1_fe_sqr(secp256k1_fe *r, const secp256k1_fe *a) { +#ifdef VERIFY + VERIFY_CHECK(a->magnitude <= 8); + secp256k1_fe_verify(a); +#endif + secp256k1_fe_sqr_inner(r->n, a->n); +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 0; + secp256k1_fe_verify(r); +#endif +} + +static SECP256K1_INLINE void secp256k1_fe_cmov(secp256k1_fe *r, const secp256k1_fe *a, int flag) { + uint64_t mask0, mask1; + mask0 = flag + ~((uint64_t)0); + mask1 = ~mask0; + r->n[0] = (r->n[0] & mask0) | (a->n[0] & mask1); + r->n[1] = (r->n[1] & mask0) | (a->n[1] & mask1); + r->n[2] = (r->n[2] & mask0) | (a->n[2] & mask1); + r->n[3] = (r->n[3] & mask0) | (a->n[3] & mask1); + r->n[4] = (r->n[4] & mask0) | (a->n[4] & mask1); +#ifdef VERIFY + if (a->magnitude > r->magnitude) { + r->magnitude = a->magnitude; + } + r->normalized &= a->normalized; +#endif +} + +static SECP256K1_INLINE void secp256k1_fe_storage_cmov(secp256k1_fe_storage *r, const secp256k1_fe_storage *a, int flag) { + uint64_t mask0, mask1; + mask0 = flag + ~((uint64_t)0); + mask1 = ~mask0; + r->n[0] = (r->n[0] & mask0) | (a->n[0] & mask1); + r->n[1] = (r->n[1] & mask0) | (a->n[1] & mask1); + r->n[2] = (r->n[2] & mask0) | (a->n[2] & mask1); + r->n[3] = (r->n[3] & mask0) | (a->n[3] & mask1); +} + +static void secp256k1_fe_to_storage(secp256k1_fe_storage *r, const secp256k1_fe *a) { +#ifdef VERIFY + VERIFY_CHECK(a->normalized); +#endif + r->n[0] = a->n[0] | a->n[1] << 52; + r->n[1] = a->n[1] >> 12 | a->n[2] << 40; + r->n[2] = a->n[2] >> 24 | a->n[3] << 28; + r->n[3] = a->n[3] >> 36 | a->n[4] << 16; +} + +static SECP256K1_INLINE void secp256k1_fe_from_storage(secp256k1_fe *r, const secp256k1_fe_storage *a) { + r->n[0] = a->n[0] & 0xFFFFFFFFFFFFFULL; + r->n[1] = a->n[0] >> 52 | ((a->n[1] << 12) & 0xFFFFFFFFFFFFFULL); + r->n[2] = a->n[1] >> 40 | ((a->n[2] << 24) & 0xFFFFFFFFFFFFFULL); + r->n[3] = a->n[2] >> 28 | ((a->n[3] << 36) & 0xFFFFFFFFFFFFFULL); + r->n[4] = a->n[3] >> 16; +#ifdef VERIFY + r->magnitude = 1; + r->normalized = 1; +#endif +} + +#endif diff --git a/src/secp256k1/src/field_5x52_int128_impl.h b/src/secp256k1/src/field_5x52_int128_impl.h new file mode 100644 index 00000000..9280bb5e --- /dev/null +++ b/src/secp256k1/src/field_5x52_int128_impl.h @@ -0,0 +1,277 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_FIELD_INNER5X52_IMPL_H_ +#define _SECP256K1_FIELD_INNER5X52_IMPL_H_ + +#include + +#ifdef VERIFY +#define VERIFY_BITS(x, n) VERIFY_CHECK(((x) >> (n)) == 0) +#else +#define VERIFY_BITS(x, n) do { } while(0) +#endif + +SECP256K1_INLINE static void secp256k1_fe_mul_inner(uint64_t *r, const uint64_t *a, const uint64_t * SECP256K1_RESTRICT b) { + uint128_t c, d; + uint64_t t3, t4, tx, u0; + uint64_t a0 = a[0], a1 = a[1], a2 = a[2], a3 = a[3], a4 = a[4]; + const uint64_t M = 0xFFFFFFFFFFFFFULL, R = 0x1000003D10ULL; + + VERIFY_BITS(a[0], 56); + VERIFY_BITS(a[1], 56); + VERIFY_BITS(a[2], 56); + VERIFY_BITS(a[3], 56); + VERIFY_BITS(a[4], 52); + VERIFY_BITS(b[0], 56); + VERIFY_BITS(b[1], 56); + VERIFY_BITS(b[2], 56); + VERIFY_BITS(b[3], 56); + VERIFY_BITS(b[4], 52); + VERIFY_CHECK(r != b); + + /* [... a b c] is a shorthand for ... + a<<104 + b<<52 + c<<0 mod n. + * px is a shorthand for sum(a[i]*b[x-i], i=0..x). + * Note that [x 0 0 0 0 0] = [x*R]. + */ + + d = (uint128_t)a0 * b[3] + + (uint128_t)a1 * b[2] + + (uint128_t)a2 * b[1] + + (uint128_t)a3 * b[0]; + VERIFY_BITS(d, 114); + /* [d 0 0 0] = [p3 0 0 0] */ + c = (uint128_t)a4 * b[4]; + VERIFY_BITS(c, 112); + /* [c 0 0 0 0 d 0 0 0] = [p8 0 0 0 0 p3 0 0 0] */ + d += (c & M) * R; c >>= 52; + VERIFY_BITS(d, 115); + VERIFY_BITS(c, 60); + /* [c 0 0 0 0 0 d 0 0 0] = [p8 0 0 0 0 p3 0 0 0] */ + t3 = d & M; d >>= 52; + VERIFY_BITS(t3, 52); + VERIFY_BITS(d, 63); + /* [c 0 0 0 0 d t3 0 0 0] = [p8 0 0 0 0 p3 0 0 0] */ + + d += (uint128_t)a0 * b[4] + + (uint128_t)a1 * b[3] + + (uint128_t)a2 * b[2] + + (uint128_t)a3 * b[1] + + (uint128_t)a4 * b[0]; + VERIFY_BITS(d, 115); + /* [c 0 0 0 0 d t3 0 0 0] = [p8 0 0 0 p4 p3 0 0 0] */ + d += c * R; + VERIFY_BITS(d, 116); + /* [d t3 0 0 0] = [p8 0 0 0 p4 p3 0 0 0] */ + t4 = d & M; d >>= 52; + VERIFY_BITS(t4, 52); + VERIFY_BITS(d, 64); + /* [d t4 t3 0 0 0] = [p8 0 0 0 p4 p3 0 0 0] */ + tx = (t4 >> 48); t4 &= (M >> 4); + VERIFY_BITS(tx, 4); + VERIFY_BITS(t4, 48); + /* [d t4+(tx<<48) t3 0 0 0] = [p8 0 0 0 p4 p3 0 0 0] */ + + c = (uint128_t)a0 * b[0]; + VERIFY_BITS(c, 112); + /* [d t4+(tx<<48) t3 0 0 c] = [p8 0 0 0 p4 p3 0 0 p0] */ + d += (uint128_t)a1 * b[4] + + (uint128_t)a2 * b[3] + + (uint128_t)a3 * b[2] + + (uint128_t)a4 * b[1]; + VERIFY_BITS(d, 115); + /* [d t4+(tx<<48) t3 0 0 c] = [p8 0 0 p5 p4 p3 0 0 p0] */ + u0 = d & M; d >>= 52; + VERIFY_BITS(u0, 52); + VERIFY_BITS(d, 63); + /* [d u0 t4+(tx<<48) t3 0 0 c] = [p8 0 0 p5 p4 p3 0 0 p0] */ + /* [d 0 t4+(tx<<48)+(u0<<52) t3 0 0 c] = [p8 0 0 p5 p4 p3 0 0 p0] */ + u0 = (u0 << 4) | tx; + VERIFY_BITS(u0, 56); + /* [d 0 t4+(u0<<48) t3 0 0 c] = [p8 0 0 p5 p4 p3 0 0 p0] */ + c += (uint128_t)u0 * (R >> 4); + VERIFY_BITS(c, 115); + /* [d 0 t4 t3 0 0 c] = [p8 0 0 p5 p4 p3 0 0 p0] */ + r[0] = c & M; c >>= 52; + VERIFY_BITS(r[0], 52); + VERIFY_BITS(c, 61); + /* [d 0 t4 t3 0 c r0] = [p8 0 0 p5 p4 p3 0 0 p0] */ + + c += (uint128_t)a0 * b[1] + + (uint128_t)a1 * b[0]; + VERIFY_BITS(c, 114); + /* [d 0 t4 t3 0 c r0] = [p8 0 0 p5 p4 p3 0 p1 p0] */ + d += (uint128_t)a2 * b[4] + + (uint128_t)a3 * b[3] + + (uint128_t)a4 * b[2]; + VERIFY_BITS(d, 114); + /* [d 0 t4 t3 0 c r0] = [p8 0 p6 p5 p4 p3 0 p1 p0] */ + c += (d & M) * R; d >>= 52; + VERIFY_BITS(c, 115); + VERIFY_BITS(d, 62); + /* [d 0 0 t4 t3 0 c r0] = [p8 0 p6 p5 p4 p3 0 p1 p0] */ + r[1] = c & M; c >>= 52; + VERIFY_BITS(r[1], 52); + VERIFY_BITS(c, 63); + /* [d 0 0 t4 t3 c r1 r0] = [p8 0 p6 p5 p4 p3 0 p1 p0] */ + + c += (uint128_t)a0 * b[2] + + (uint128_t)a1 * b[1] + + (uint128_t)a2 * b[0]; + VERIFY_BITS(c, 114); + /* [d 0 0 t4 t3 c r1 r0] = [p8 0 p6 p5 p4 p3 p2 p1 p0] */ + d += (uint128_t)a3 * b[4] + + (uint128_t)a4 * b[3]; + VERIFY_BITS(d, 114); + /* [d 0 0 t4 t3 c t1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + c += (d & M) * R; d >>= 52; + VERIFY_BITS(c, 115); + VERIFY_BITS(d, 62); + /* [d 0 0 0 t4 t3 c r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + + /* [d 0 0 0 t4 t3 c r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[2] = c & M; c >>= 52; + VERIFY_BITS(r[2], 52); + VERIFY_BITS(c, 63); + /* [d 0 0 0 t4 t3+c r2 r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + c += d * R + t3;; + VERIFY_BITS(c, 100); + /* [t4 c r2 r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[3] = c & M; c >>= 52; + VERIFY_BITS(r[3], 52); + VERIFY_BITS(c, 48); + /* [t4+c r3 r2 r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + c += t4; + VERIFY_BITS(c, 49); + /* [c r3 r2 r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[4] = c; + VERIFY_BITS(r[4], 49); + /* [r4 r3 r2 r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ +} + +SECP256K1_INLINE static void secp256k1_fe_sqr_inner(uint64_t *r, const uint64_t *a) { + uint128_t c, d; + uint64_t a0 = a[0], a1 = a[1], a2 = a[2], a3 = a[3], a4 = a[4]; + int64_t t3, t4, tx, u0; + const uint64_t M = 0xFFFFFFFFFFFFFULL, R = 0x1000003D10ULL; + + VERIFY_BITS(a[0], 56); + VERIFY_BITS(a[1], 56); + VERIFY_BITS(a[2], 56); + VERIFY_BITS(a[3], 56); + VERIFY_BITS(a[4], 52); + + /** [... a b c] is a shorthand for ... + a<<104 + b<<52 + c<<0 mod n. + * px is a shorthand for sum(a[i]*a[x-i], i=0..x). + * Note that [x 0 0 0 0 0] = [x*R]. + */ + + d = (uint128_t)(a0*2) * a3 + + (uint128_t)(a1*2) * a2; + VERIFY_BITS(d, 114); + /* [d 0 0 0] = [p3 0 0 0] */ + c = (uint128_t)a4 * a4; + VERIFY_BITS(c, 112); + /* [c 0 0 0 0 d 0 0 0] = [p8 0 0 0 0 p3 0 0 0] */ + d += (c & M) * R; c >>= 52; + VERIFY_BITS(d, 115); + VERIFY_BITS(c, 60); + /* [c 0 0 0 0 0 d 0 0 0] = [p8 0 0 0 0 p3 0 0 0] */ + t3 = d & M; d >>= 52; + VERIFY_BITS(t3, 52); + VERIFY_BITS(d, 63); + /* [c 0 0 0 0 d t3 0 0 0] = [p8 0 0 0 0 p3 0 0 0] */ + + a4 *= 2; + d += (uint128_t)a0 * a4 + + (uint128_t)(a1*2) * a3 + + (uint128_t)a2 * a2; + VERIFY_BITS(d, 115); + /* [c 0 0 0 0 d t3 0 0 0] = [p8 0 0 0 p4 p3 0 0 0] */ + d += c * R; + VERIFY_BITS(d, 116); + /* [d t3 0 0 0] = [p8 0 0 0 p4 p3 0 0 0] */ + t4 = d & M; d >>= 52; + VERIFY_BITS(t4, 52); + VERIFY_BITS(d, 64); + /* [d t4 t3 0 0 0] = [p8 0 0 0 p4 p3 0 0 0] */ + tx = (t4 >> 48); t4 &= (M >> 4); + VERIFY_BITS(tx, 4); + VERIFY_BITS(t4, 48); + /* [d t4+(tx<<48) t3 0 0 0] = [p8 0 0 0 p4 p3 0 0 0] */ + + c = (uint128_t)a0 * a0; + VERIFY_BITS(c, 112); + /* [d t4+(tx<<48) t3 0 0 c] = [p8 0 0 0 p4 p3 0 0 p0] */ + d += (uint128_t)a1 * a4 + + (uint128_t)(a2*2) * a3; + VERIFY_BITS(d, 114); + /* [d t4+(tx<<48) t3 0 0 c] = [p8 0 0 p5 p4 p3 0 0 p0] */ + u0 = d & M; d >>= 52; + VERIFY_BITS(u0, 52); + VERIFY_BITS(d, 62); + /* [d u0 t4+(tx<<48) t3 0 0 c] = [p8 0 0 p5 p4 p3 0 0 p0] */ + /* [d 0 t4+(tx<<48)+(u0<<52) t3 0 0 c] = [p8 0 0 p5 p4 p3 0 0 p0] */ + u0 = (u0 << 4) | tx; + VERIFY_BITS(u0, 56); + /* [d 0 t4+(u0<<48) t3 0 0 c] = [p8 0 0 p5 p4 p3 0 0 p0] */ + c += (uint128_t)u0 * (R >> 4); + VERIFY_BITS(c, 113); + /* [d 0 t4 t3 0 0 c] = [p8 0 0 p5 p4 p3 0 0 p0] */ + r[0] = c & M; c >>= 52; + VERIFY_BITS(r[0], 52); + VERIFY_BITS(c, 61); + /* [d 0 t4 t3 0 c r0] = [p8 0 0 p5 p4 p3 0 0 p0] */ + + a0 *= 2; + c += (uint128_t)a0 * a1; + VERIFY_BITS(c, 114); + /* [d 0 t4 t3 0 c r0] = [p8 0 0 p5 p4 p3 0 p1 p0] */ + d += (uint128_t)a2 * a4 + + (uint128_t)a3 * a3; + VERIFY_BITS(d, 114); + /* [d 0 t4 t3 0 c r0] = [p8 0 p6 p5 p4 p3 0 p1 p0] */ + c += (d & M) * R; d >>= 52; + VERIFY_BITS(c, 115); + VERIFY_BITS(d, 62); + /* [d 0 0 t4 t3 0 c r0] = [p8 0 p6 p5 p4 p3 0 p1 p0] */ + r[1] = c & M; c >>= 52; + VERIFY_BITS(r[1], 52); + VERIFY_BITS(c, 63); + /* [d 0 0 t4 t3 c r1 r0] = [p8 0 p6 p5 p4 p3 0 p1 p0] */ + + c += (uint128_t)a0 * a2 + + (uint128_t)a1 * a1; + VERIFY_BITS(c, 114); + /* [d 0 0 t4 t3 c r1 r0] = [p8 0 p6 p5 p4 p3 p2 p1 p0] */ + d += (uint128_t)a3 * a4; + VERIFY_BITS(d, 114); + /* [d 0 0 t4 t3 c r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + c += (d & M) * R; d >>= 52; + VERIFY_BITS(c, 115); + VERIFY_BITS(d, 62); + /* [d 0 0 0 t4 t3 c r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[2] = c & M; c >>= 52; + VERIFY_BITS(r[2], 52); + VERIFY_BITS(c, 63); + /* [d 0 0 0 t4 t3+c r2 r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + + c += d * R + t3;; + VERIFY_BITS(c, 100); + /* [t4 c r2 r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[3] = c & M; c >>= 52; + VERIFY_BITS(r[3], 52); + VERIFY_BITS(c, 48); + /* [t4+c r3 r2 r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + c += t4; + VERIFY_BITS(c, 49); + /* [c r3 r2 r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ + r[4] = c; + VERIFY_BITS(r[4], 49); + /* [r4 r3 r2 r1 r0] = [p8 p7 p6 p5 p4 p3 p2 p1 p0] */ +} + +#endif diff --git a/src/secp256k1/src/field_impl.h b/src/secp256k1/src/field_impl.h new file mode 100644 index 00000000..77f4aae2 --- /dev/null +++ b/src/secp256k1/src/field_impl.h @@ -0,0 +1,283 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_FIELD_IMPL_H_ +#define _SECP256K1_FIELD_IMPL_H_ + +#if defined HAVE_CONFIG_H +#include "libsecp256k1-config.h" +#endif + +#include "util.h" + +#if defined(USE_FIELD_10X26) +#include "field_10x26_impl.h" +#elif defined(USE_FIELD_5X52) +#include "field_5x52_impl.h" +#else +#error "Please select field implementation" +#endif + +SECP256K1_INLINE static int secp256k1_fe_equal_var(const secp256k1_fe *a, const secp256k1_fe *b) { + secp256k1_fe na; + secp256k1_fe_negate(&na, a, 1); + secp256k1_fe_add(&na, b); + return secp256k1_fe_normalizes_to_zero_var(&na); +} + +static int secp256k1_fe_sqrt_var(secp256k1_fe *r, const secp256k1_fe *a) { + /** Given that p is congruent to 3 mod 4, we can compute the square root of + * a mod p as the (p+1)/4'th power of a. + * + * As (p+1)/4 is an even number, it will have the same result for a and for + * (-a). Only one of these two numbers actually has a square root however, + * so we test at the end by squaring and comparing to the input. + * Also because (p+1)/4 is an even number, the computed square root is + * itself always a square (a ** ((p+1)/4) is the square of a ** ((p+1)/8)). + */ + secp256k1_fe x2, x3, x6, x9, x11, x22, x44, x88, x176, x220, x223, t1; + int j; + + /** The binary representation of (p + 1)/4 has 3 blocks of 1s, with lengths in + * { 2, 22, 223 }. Use an addition chain to calculate 2^n - 1 for each block: + * 1, [2], 3, 6, 9, 11, [22], 44, 88, 176, 220, [223] + */ + + secp256k1_fe_sqr(&x2, a); + secp256k1_fe_mul(&x2, &x2, a); + + secp256k1_fe_sqr(&x3, &x2); + secp256k1_fe_mul(&x3, &x3, a); + + x6 = x3; + for (j=0; j<3; j++) { + secp256k1_fe_sqr(&x6, &x6); + } + secp256k1_fe_mul(&x6, &x6, &x3); + + x9 = x6; + for (j=0; j<3; j++) { + secp256k1_fe_sqr(&x9, &x9); + } + secp256k1_fe_mul(&x9, &x9, &x3); + + x11 = x9; + for (j=0; j<2; j++) { + secp256k1_fe_sqr(&x11, &x11); + } + secp256k1_fe_mul(&x11, &x11, &x2); + + x22 = x11; + for (j=0; j<11; j++) { + secp256k1_fe_sqr(&x22, &x22); + } + secp256k1_fe_mul(&x22, &x22, &x11); + + x44 = x22; + for (j=0; j<22; j++) { + secp256k1_fe_sqr(&x44, &x44); + } + secp256k1_fe_mul(&x44, &x44, &x22); + + x88 = x44; + for (j=0; j<44; j++) { + secp256k1_fe_sqr(&x88, &x88); + } + secp256k1_fe_mul(&x88, &x88, &x44); + + x176 = x88; + for (j=0; j<88; j++) { + secp256k1_fe_sqr(&x176, &x176); + } + secp256k1_fe_mul(&x176, &x176, &x88); + + x220 = x176; + for (j=0; j<44; j++) { + secp256k1_fe_sqr(&x220, &x220); + } + secp256k1_fe_mul(&x220, &x220, &x44); + + x223 = x220; + for (j=0; j<3; j++) { + secp256k1_fe_sqr(&x223, &x223); + } + secp256k1_fe_mul(&x223, &x223, &x3); + + /* The final result is then assembled using a sliding window over the blocks. */ + + t1 = x223; + for (j=0; j<23; j++) { + secp256k1_fe_sqr(&t1, &t1); + } + secp256k1_fe_mul(&t1, &t1, &x22); + for (j=0; j<6; j++) { + secp256k1_fe_sqr(&t1, &t1); + } + secp256k1_fe_mul(&t1, &t1, &x2); + secp256k1_fe_sqr(&t1, &t1); + secp256k1_fe_sqr(r, &t1); + + /* Check that a square root was actually calculated */ + + secp256k1_fe_sqr(&t1, r); + return secp256k1_fe_equal_var(&t1, a); +} + +static void secp256k1_fe_inv(secp256k1_fe *r, const secp256k1_fe *a) { + secp256k1_fe x2, x3, x6, x9, x11, x22, x44, x88, x176, x220, x223, t1; + int j; + + /** The binary representation of (p - 2) has 5 blocks of 1s, with lengths in + * { 1, 2, 22, 223 }. Use an addition chain to calculate 2^n - 1 for each block: + * [1], [2], 3, 6, 9, 11, [22], 44, 88, 176, 220, [223] + */ + + secp256k1_fe_sqr(&x2, a); + secp256k1_fe_mul(&x2, &x2, a); + + secp256k1_fe_sqr(&x3, &x2); + secp256k1_fe_mul(&x3, &x3, a); + + x6 = x3; + for (j=0; j<3; j++) { + secp256k1_fe_sqr(&x6, &x6); + } + secp256k1_fe_mul(&x6, &x6, &x3); + + x9 = x6; + for (j=0; j<3; j++) { + secp256k1_fe_sqr(&x9, &x9); + } + secp256k1_fe_mul(&x9, &x9, &x3); + + x11 = x9; + for (j=0; j<2; j++) { + secp256k1_fe_sqr(&x11, &x11); + } + secp256k1_fe_mul(&x11, &x11, &x2); + + x22 = x11; + for (j=0; j<11; j++) { + secp256k1_fe_sqr(&x22, &x22); + } + secp256k1_fe_mul(&x22, &x22, &x11); + + x44 = x22; + for (j=0; j<22; j++) { + secp256k1_fe_sqr(&x44, &x44); + } + secp256k1_fe_mul(&x44, &x44, &x22); + + x88 = x44; + for (j=0; j<44; j++) { + secp256k1_fe_sqr(&x88, &x88); + } + secp256k1_fe_mul(&x88, &x88, &x44); + + x176 = x88; + for (j=0; j<88; j++) { + secp256k1_fe_sqr(&x176, &x176); + } + secp256k1_fe_mul(&x176, &x176, &x88); + + x220 = x176; + for (j=0; j<44; j++) { + secp256k1_fe_sqr(&x220, &x220); + } + secp256k1_fe_mul(&x220, &x220, &x44); + + x223 = x220; + for (j=0; j<3; j++) { + secp256k1_fe_sqr(&x223, &x223); + } + secp256k1_fe_mul(&x223, &x223, &x3); + + /* The final result is then assembled using a sliding window over the blocks. */ + + t1 = x223; + for (j=0; j<23; j++) { + secp256k1_fe_sqr(&t1, &t1); + } + secp256k1_fe_mul(&t1, &t1, &x22); + for (j=0; j<5; j++) { + secp256k1_fe_sqr(&t1, &t1); + } + secp256k1_fe_mul(&t1, &t1, a); + for (j=0; j<3; j++) { + secp256k1_fe_sqr(&t1, &t1); + } + secp256k1_fe_mul(&t1, &t1, &x2); + for (j=0; j<2; j++) { + secp256k1_fe_sqr(&t1, &t1); + } + secp256k1_fe_mul(r, a, &t1); +} + +static void secp256k1_fe_inv_var(secp256k1_fe *r, const secp256k1_fe *a) { +#if defined(USE_FIELD_INV_BUILTIN) + secp256k1_fe_inv(r, a); +#elif defined(USE_FIELD_INV_NUM) + secp256k1_num n, m; + static const secp256k1_fe negone = SECP256K1_FE_CONST( + 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFFUL, + 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFEUL, 0xFFFFFC2EUL + ); + /* secp256k1 field prime, value p defined in "Standards for Efficient Cryptography" (SEC2) 2.7.1. */ + static const unsigned char prime[32] = { + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFE,0xFF,0xFF,0xFC,0x2F + }; + unsigned char b[32]; + int res; + secp256k1_fe c = *a; + secp256k1_fe_normalize_var(&c); + secp256k1_fe_get_b32(b, &c); + secp256k1_num_set_bin(&n, b, 32); + secp256k1_num_set_bin(&m, prime, 32); + secp256k1_num_mod_inverse(&n, &n, &m); + secp256k1_num_get_bin(b, 32, &n); + res = secp256k1_fe_set_b32(r, b); + (void)res; + VERIFY_CHECK(res); + /* Verify the result is the (unique) valid inverse using non-GMP code. */ + secp256k1_fe_mul(&c, &c, r); + secp256k1_fe_add(&c, &negone); + CHECK(secp256k1_fe_normalizes_to_zero_var(&c)); +#else +#error "Please select field inverse implementation" +#endif +} + +static void secp256k1_fe_inv_all_var(size_t len, secp256k1_fe *r, const secp256k1_fe *a) { + secp256k1_fe u; + size_t i; + if (len < 1) { + return; + } + + VERIFY_CHECK((r + len <= a) || (a + len <= r)); + + r[0] = a[0]; + + i = 0; + while (++i < len) { + secp256k1_fe_mul(&r[i], &r[i - 1], &a[i]); + } + + secp256k1_fe_inv_var(&u, &r[--i]); + + while (i > 0) { + size_t j = i--; + secp256k1_fe_mul(&r[j], &r[i], &u); + secp256k1_fe_mul(&u, &u, &a[j]); + } + + r[0] = u; +} + +#endif diff --git a/src/secp256k1/src/gen_context.c b/src/secp256k1/src/gen_context.c new file mode 100644 index 00000000..1835fd49 --- /dev/null +++ b/src/secp256k1/src/gen_context.c @@ -0,0 +1,74 @@ +/********************************************************************** + * Copyright (c) 2013, 2014, 2015 Thomas Daede, Cory Fields * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#define USE_BASIC_CONFIG 1 + +#include "basic-config.h" +#include "include/secp256k1.h" +#include "field_impl.h" +#include "scalar_impl.h" +#include "group_impl.h" +#include "ecmult_gen_impl.h" + +static void default_error_callback_fn(const char* str, void* data) { + (void)data; + fprintf(stderr, "[libsecp256k1] internal consistency check failed: %s\n", str); + abort(); +} + +static const secp256k1_callback default_error_callback = { + default_error_callback_fn, + NULL +}; + +int main(int argc, char **argv) { + secp256k1_ecmult_gen_context ctx; + int inner; + int outer; + FILE* fp; + + (void)argc; + (void)argv; + + fp = fopen("src/ecmult_static_context.h","w"); + if (fp == NULL) { + fprintf(stderr, "Could not open src/ecmult_static_context.h for writing!\n"); + return -1; + } + + fprintf(fp, "#ifndef _SECP256K1_ECMULT_STATIC_CONTEXT_\n"); + fprintf(fp, "#define _SECP256K1_ECMULT_STATIC_CONTEXT_\n"); + fprintf(fp, "#include \"group.h\"\n"); + fprintf(fp, "#define SC SECP256K1_GE_STORAGE_CONST\n"); + fprintf(fp, "static const secp256k1_ge_storage secp256k1_ecmult_static_context[64][16] = {\n"); + + secp256k1_ecmult_gen_context_init(&ctx); + secp256k1_ecmult_gen_context_build(&ctx, &default_error_callback); + for(outer = 0; outer != 64; outer++) { + fprintf(fp,"{\n"); + for(inner = 0; inner != 16; inner++) { + fprintf(fp," SC(%uu, %uu, %uu, %uu, %uu, %uu, %uu, %uu, %uu, %uu, %uu, %uu, %uu, %uu, %uu, %uu)", SECP256K1_GE_STORAGE_CONST_GET((*ctx.prec)[outer][inner])); + if (inner != 15) { + fprintf(fp,",\n"); + } else { + fprintf(fp,"\n"); + } + } + if (outer != 63) { + fprintf(fp,"},\n"); + } else { + fprintf(fp,"}\n"); + } + } + fprintf(fp,"};\n"); + secp256k1_ecmult_gen_context_clear(&ctx); + + fprintf(fp, "#undef SC\n"); + fprintf(fp, "#endif\n"); + fclose(fp); + + return 0; +} diff --git a/src/secp256k1/src/group.h b/src/secp256k1/src/group.h new file mode 100644 index 00000000..ebfe1ca7 --- /dev/null +++ b/src/secp256k1/src/group.h @@ -0,0 +1,141 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_GROUP_ +#define _SECP256K1_GROUP_ + +#include "num.h" +#include "field.h" + +/** A group element of the secp256k1 curve, in affine coordinates. */ +typedef struct { + secp256k1_fe x; + secp256k1_fe y; + int infinity; /* whether this represents the point at infinity */ +} secp256k1_ge; + +#define SECP256K1_GE_CONST(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p) {SECP256K1_FE_CONST((a),(b),(c),(d),(e),(f),(g),(h)), SECP256K1_FE_CONST((i),(j),(k),(l),(m),(n),(o),(p)), 0} +#define SECP256K1_GE_CONST_INFINITY {SECP256K1_FE_CONST(0, 0, 0, 0, 0, 0, 0, 0), SECP256K1_FE_CONST(0, 0, 0, 0, 0, 0, 0, 0), 1} + +/** A group element of the secp256k1 curve, in jacobian coordinates. */ +typedef struct { + secp256k1_fe x; /* actual X: x/z^2 */ + secp256k1_fe y; /* actual Y: y/z^3 */ + secp256k1_fe z; + int infinity; /* whether this represents the point at infinity */ +} secp256k1_gej; + +#define SECP256K1_GEJ_CONST(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p) {SECP256K1_FE_CONST((a),(b),(c),(d),(e),(f),(g),(h)), SECP256K1_FE_CONST((i),(j),(k),(l),(m),(n),(o),(p)), SECP256K1_FE_CONST(0, 0, 0, 0, 0, 0, 0, 1), 0} +#define SECP256K1_GEJ_CONST_INFINITY {SECP256K1_FE_CONST(0, 0, 0, 0, 0, 0, 0, 0), SECP256K1_FE_CONST(0, 0, 0, 0, 0, 0, 0, 0), SECP256K1_FE_CONST(0, 0, 0, 0, 0, 0, 0, 0), 1} + +typedef struct { + secp256k1_fe_storage x; + secp256k1_fe_storage y; +} secp256k1_ge_storage; + +#define SECP256K1_GE_STORAGE_CONST(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p) {SECP256K1_FE_STORAGE_CONST((a),(b),(c),(d),(e),(f),(g),(h)), SECP256K1_FE_STORAGE_CONST((i),(j),(k),(l),(m),(n),(o),(p))} + +#define SECP256K1_GE_STORAGE_CONST_GET(t) SECP256K1_FE_STORAGE_CONST_GET(t.x), SECP256K1_FE_STORAGE_CONST_GET(t.y) + +/** Set a group element equal to the point with given X and Y coordinates */ +static void secp256k1_ge_set_xy(secp256k1_ge *r, const secp256k1_fe *x, const secp256k1_fe *y); + +/** Set a group element (affine) equal to the point with the given X coordinate + * and a Y coordinate that is a quadratic residue modulo p. The return value + * is true iff a coordinate with the given X coordinate exists. + */ +static int secp256k1_ge_set_xquad_var(secp256k1_ge *r, const secp256k1_fe *x); + +/** Set a group element (affine) equal to the point with the given X coordinate, and given oddness + * for Y. Return value indicates whether the result is valid. */ +static int secp256k1_ge_set_xo_var(secp256k1_ge *r, const secp256k1_fe *x, int odd); + +/** Check whether a group element is the point at infinity. */ +static int secp256k1_ge_is_infinity(const secp256k1_ge *a); + +/** Check whether a group element is valid (i.e., on the curve). */ +static int secp256k1_ge_is_valid_var(const secp256k1_ge *a); + +static void secp256k1_ge_neg(secp256k1_ge *r, const secp256k1_ge *a); + +/** Set a group element equal to another which is given in jacobian coordinates */ +static void secp256k1_ge_set_gej(secp256k1_ge *r, secp256k1_gej *a); + +/** Set a batch of group elements equal to the inputs given in jacobian coordinates */ +static void secp256k1_ge_set_all_gej_var(size_t len, secp256k1_ge *r, const secp256k1_gej *a, const secp256k1_callback *cb); + +/** Set a batch of group elements equal to the inputs given in jacobian + * coordinates (with known z-ratios). zr must contain the known z-ratios such + * that mul(a[i].z, zr[i+1]) == a[i+1].z. zr[0] is ignored. */ +static void secp256k1_ge_set_table_gej_var(size_t len, secp256k1_ge *r, const secp256k1_gej *a, const secp256k1_fe *zr); + +/** Bring a batch inputs given in jacobian coordinates (with known z-ratios) to + * the same global z "denominator". zr must contain the known z-ratios such + * that mul(a[i].z, zr[i+1]) == a[i+1].z. zr[0] is ignored. The x and y + * coordinates of the result are stored in r, the common z coordinate is + * stored in globalz. */ +static void secp256k1_ge_globalz_set_table_gej(size_t len, secp256k1_ge *r, secp256k1_fe *globalz, const secp256k1_gej *a, const secp256k1_fe *zr); + +/** Set a group element (jacobian) equal to the point at infinity. */ +static void secp256k1_gej_set_infinity(secp256k1_gej *r); + +/** Set a group element (jacobian) equal to another which is given in affine coordinates. */ +static void secp256k1_gej_set_ge(secp256k1_gej *r, const secp256k1_ge *a); + +/** Compare the X coordinate of a group element (jacobian). */ +static int secp256k1_gej_eq_x_var(const secp256k1_fe *x, const secp256k1_gej *a); + +/** Set r equal to the inverse of a (i.e., mirrored around the X axis) */ +static void secp256k1_gej_neg(secp256k1_gej *r, const secp256k1_gej *a); + +/** Check whether a group element is the point at infinity. */ +static int secp256k1_gej_is_infinity(const secp256k1_gej *a); + +/** Set r equal to the double of a. If rzr is not-NULL, r->z = a->z * *rzr (where infinity means an implicit z = 0). + * a may not be zero. Constant time. */ +static void secp256k1_gej_double_nonzero(secp256k1_gej *r, const secp256k1_gej *a, secp256k1_fe *rzr); + +/** Set r equal to the double of a. If rzr is not-NULL, r->z = a->z * *rzr (where infinity means an implicit z = 0). */ +static void secp256k1_gej_double_var(secp256k1_gej *r, const secp256k1_gej *a, secp256k1_fe *rzr); + +/** Set r equal to the sum of a and b. If rzr is non-NULL, r->z = a->z * *rzr (a cannot be infinity in that case). */ +static void secp256k1_gej_add_var(secp256k1_gej *r, const secp256k1_gej *a, const secp256k1_gej *b, secp256k1_fe *rzr); + +/** Set r equal to the sum of a and b (with b given in affine coordinates, and not infinity). */ +static void secp256k1_gej_add_ge(secp256k1_gej *r, const secp256k1_gej *a, const secp256k1_ge *b); + +/** Set r equal to the sum of a and b (with b given in affine coordinates). This is more efficient + than secp256k1_gej_add_var. It is identical to secp256k1_gej_add_ge but without constant-time + guarantee, and b is allowed to be infinity. If rzr is non-NULL, r->z = a->z * *rzr (a cannot be infinity in that case). */ +static void secp256k1_gej_add_ge_var(secp256k1_gej *r, const secp256k1_gej *a, const secp256k1_ge *b, secp256k1_fe *rzr); + +/** Set r equal to the sum of a and b (with the inverse of b's Z coordinate passed as bzinv). */ +static void secp256k1_gej_add_zinv_var(secp256k1_gej *r, const secp256k1_gej *a, const secp256k1_ge *b, const secp256k1_fe *bzinv); + +#ifdef USE_ENDOMORPHISM +/** Set r to be equal to lambda times a, where lambda is chosen in a way such that this is very fast. */ +static void secp256k1_ge_mul_lambda(secp256k1_ge *r, const secp256k1_ge *a); +#endif + +/** Clear a secp256k1_gej to prevent leaking sensitive information. */ +static void secp256k1_gej_clear(secp256k1_gej *r); + +/** Clear a secp256k1_ge to prevent leaking sensitive information. */ +static void secp256k1_ge_clear(secp256k1_ge *r); + +/** Convert a group element to the storage type. */ +static void secp256k1_ge_to_storage(secp256k1_ge_storage *r, const secp256k1_ge *a); + +/** Convert a group element back from the storage type. */ +static void secp256k1_ge_from_storage(secp256k1_ge *r, const secp256k1_ge_storage *a); + +/** If flag is true, set *r equal to *a; otherwise leave it. Constant-time. */ +static void secp256k1_ge_storage_cmov(secp256k1_ge_storage *r, const secp256k1_ge_storage *a, int flag); + +/** Rescale a jacobian point by b which must be non-zero. Constant-time. */ +static void secp256k1_gej_rescale(secp256k1_gej *r, const secp256k1_fe *b); + +#endif diff --git a/src/secp256k1/src/group_impl.h b/src/secp256k1/src/group_impl.h new file mode 100644 index 00000000..42e2f6e6 --- /dev/null +++ b/src/secp256k1/src/group_impl.h @@ -0,0 +1,626 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_GROUP_IMPL_H_ +#define _SECP256K1_GROUP_IMPL_H_ + +#include + +#include "num.h" +#include "field.h" +#include "group.h" + +/** Generator for secp256k1, value 'g' defined in + * "Standards for Efficient Cryptography" (SEC2) 2.7.1. + */ +static const secp256k1_ge secp256k1_ge_const_g = SECP256K1_GE_CONST( + 0x79BE667EUL, 0xF9DCBBACUL, 0x55A06295UL, 0xCE870B07UL, + 0x029BFCDBUL, 0x2DCE28D9UL, 0x59F2815BUL, 0x16F81798UL, + 0x483ADA77UL, 0x26A3C465UL, 0x5DA4FBFCUL, 0x0E1108A8UL, + 0xFD17B448UL, 0xA6855419UL, 0x9C47D08FUL, 0xFB10D4B8UL +); + +static void secp256k1_ge_set_gej_zinv(secp256k1_ge *r, const secp256k1_gej *a, const secp256k1_fe *zi) { + secp256k1_fe zi2; + secp256k1_fe zi3; + secp256k1_fe_sqr(&zi2, zi); + secp256k1_fe_mul(&zi3, &zi2, zi); + secp256k1_fe_mul(&r->x, &a->x, &zi2); + secp256k1_fe_mul(&r->y, &a->y, &zi3); + r->infinity = a->infinity; +} + +static void secp256k1_ge_set_xy(secp256k1_ge *r, const secp256k1_fe *x, const secp256k1_fe *y) { + r->infinity = 0; + r->x = *x; + r->y = *y; +} + +static int secp256k1_ge_is_infinity(const secp256k1_ge *a) { + return a->infinity; +} + +static void secp256k1_ge_neg(secp256k1_ge *r, const secp256k1_ge *a) { + *r = *a; + secp256k1_fe_normalize_weak(&r->y); + secp256k1_fe_negate(&r->y, &r->y, 1); +} + +static void secp256k1_ge_set_gej(secp256k1_ge *r, secp256k1_gej *a) { + secp256k1_fe z2, z3; + r->infinity = a->infinity; + secp256k1_fe_inv(&a->z, &a->z); + secp256k1_fe_sqr(&z2, &a->z); + secp256k1_fe_mul(&z3, &a->z, &z2); + secp256k1_fe_mul(&a->x, &a->x, &z2); + secp256k1_fe_mul(&a->y, &a->y, &z3); + secp256k1_fe_set_int(&a->z, 1); + r->x = a->x; + r->y = a->y; +} + +static void secp256k1_ge_set_gej_var(secp256k1_ge *r, secp256k1_gej *a) { + secp256k1_fe z2, z3; + r->infinity = a->infinity; + if (a->infinity) { + return; + } + secp256k1_fe_inv_var(&a->z, &a->z); + secp256k1_fe_sqr(&z2, &a->z); + secp256k1_fe_mul(&z3, &a->z, &z2); + secp256k1_fe_mul(&a->x, &a->x, &z2); + secp256k1_fe_mul(&a->y, &a->y, &z3); + secp256k1_fe_set_int(&a->z, 1); + r->x = a->x; + r->y = a->y; +} + +static void secp256k1_ge_set_all_gej_var(size_t len, secp256k1_ge *r, const secp256k1_gej *a, const secp256k1_callback *cb) { + secp256k1_fe *az; + secp256k1_fe *azi; + size_t i; + size_t count = 0; + az = (secp256k1_fe *)checked_malloc(cb, sizeof(secp256k1_fe) * len); + for (i = 0; i < len; i++) { + if (!a[i].infinity) { + az[count++] = a[i].z; + } + } + + azi = (secp256k1_fe *)checked_malloc(cb, sizeof(secp256k1_fe) * count); + secp256k1_fe_inv_all_var(count, azi, az); + free(az); + + count = 0; + for (i = 0; i < len; i++) { + r[i].infinity = a[i].infinity; + if (!a[i].infinity) { + secp256k1_ge_set_gej_zinv(&r[i], &a[i], &azi[count++]); + } + } + free(azi); +} + +static void secp256k1_ge_set_table_gej_var(size_t len, secp256k1_ge *r, const secp256k1_gej *a, const secp256k1_fe *zr) { + size_t i = len - 1; + secp256k1_fe zi; + + if (len > 0) { + /* Compute the inverse of the last z coordinate, and use it to compute the last affine output. */ + secp256k1_fe_inv(&zi, &a[i].z); + secp256k1_ge_set_gej_zinv(&r[i], &a[i], &zi); + + /* Work out way backwards, using the z-ratios to scale the x/y values. */ + while (i > 0) { + secp256k1_fe_mul(&zi, &zi, &zr[i]); + i--; + secp256k1_ge_set_gej_zinv(&r[i], &a[i], &zi); + } + } +} + +static void secp256k1_ge_globalz_set_table_gej(size_t len, secp256k1_ge *r, secp256k1_fe *globalz, const secp256k1_gej *a, const secp256k1_fe *zr) { + size_t i = len - 1; + secp256k1_fe zs; + + if (len > 0) { + /* The z of the final point gives us the "global Z" for the table. */ + r[i].x = a[i].x; + r[i].y = a[i].y; + *globalz = a[i].z; + r[i].infinity = 0; + zs = zr[i]; + + /* Work our way backwards, using the z-ratios to scale the x/y values. */ + while (i > 0) { + if (i != len - 1) { + secp256k1_fe_mul(&zs, &zs, &zr[i]); + } + i--; + secp256k1_ge_set_gej_zinv(&r[i], &a[i], &zs); + } + } +} + +static void secp256k1_gej_set_infinity(secp256k1_gej *r) { + r->infinity = 1; + secp256k1_fe_set_int(&r->x, 0); + secp256k1_fe_set_int(&r->y, 0); + secp256k1_fe_set_int(&r->z, 0); +} + +static void secp256k1_gej_clear(secp256k1_gej *r) { + r->infinity = 0; + secp256k1_fe_clear(&r->x); + secp256k1_fe_clear(&r->y); + secp256k1_fe_clear(&r->z); +} + +static void secp256k1_ge_clear(secp256k1_ge *r) { + r->infinity = 0; + secp256k1_fe_clear(&r->x); + secp256k1_fe_clear(&r->y); +} + +static int secp256k1_ge_set_xquad_var(secp256k1_ge *r, const secp256k1_fe *x) { + secp256k1_fe x2, x3, c; + r->x = *x; + secp256k1_fe_sqr(&x2, x); + secp256k1_fe_mul(&x3, x, &x2); + r->infinity = 0; + secp256k1_fe_set_int(&c, 7); + secp256k1_fe_add(&c, &x3); + return secp256k1_fe_sqrt_var(&r->y, &c); +} + +static int secp256k1_ge_set_xo_var(secp256k1_ge *r, const secp256k1_fe *x, int odd) { + if (!secp256k1_ge_set_xquad_var(r, x)) { + return 0; + } + secp256k1_fe_normalize_var(&r->y); + if (secp256k1_fe_is_odd(&r->y) != odd) { + secp256k1_fe_negate(&r->y, &r->y, 1); + } + return 1; + +} + +static void secp256k1_gej_set_ge(secp256k1_gej *r, const secp256k1_ge *a) { + r->infinity = a->infinity; + r->x = a->x; + r->y = a->y; + secp256k1_fe_set_int(&r->z, 1); +} + +static int secp256k1_gej_eq_x_var(const secp256k1_fe *x, const secp256k1_gej *a) { + secp256k1_fe r, r2; + VERIFY_CHECK(!a->infinity); + secp256k1_fe_sqr(&r, &a->z); secp256k1_fe_mul(&r, &r, x); + r2 = a->x; secp256k1_fe_normalize_weak(&r2); + return secp256k1_fe_equal_var(&r, &r2); +} + +static void secp256k1_gej_neg(secp256k1_gej *r, const secp256k1_gej *a) { + r->infinity = a->infinity; + r->x = a->x; + r->y = a->y; + r->z = a->z; + secp256k1_fe_normalize_weak(&r->y); + secp256k1_fe_negate(&r->y, &r->y, 1); +} + +static int secp256k1_gej_is_infinity(const secp256k1_gej *a) { + return a->infinity; +} + +static int secp256k1_gej_is_valid_var(const secp256k1_gej *a) { + secp256k1_fe y2, x3, z2, z6; + if (a->infinity) { + return 0; + } + /** y^2 = x^3 + 7 + * (Y/Z^3)^2 = (X/Z^2)^3 + 7 + * Y^2 / Z^6 = X^3 / Z^6 + 7 + * Y^2 = X^3 + 7*Z^6 + */ + secp256k1_fe_sqr(&y2, &a->y); + secp256k1_fe_sqr(&x3, &a->x); secp256k1_fe_mul(&x3, &x3, &a->x); + secp256k1_fe_sqr(&z2, &a->z); + secp256k1_fe_sqr(&z6, &z2); secp256k1_fe_mul(&z6, &z6, &z2); + secp256k1_fe_mul_int(&z6, 7); + secp256k1_fe_add(&x3, &z6); + secp256k1_fe_normalize_weak(&x3); + return secp256k1_fe_equal_var(&y2, &x3); +} + +static int secp256k1_ge_is_valid_var(const secp256k1_ge *a) { + secp256k1_fe y2, x3, c; + if (a->infinity) { + return 0; + } + /* y^2 = x^3 + 7 */ + secp256k1_fe_sqr(&y2, &a->y); + secp256k1_fe_sqr(&x3, &a->x); secp256k1_fe_mul(&x3, &x3, &a->x); + secp256k1_fe_set_int(&c, 7); + secp256k1_fe_add(&x3, &c); + secp256k1_fe_normalize_weak(&x3); + return secp256k1_fe_equal_var(&y2, &x3); +} + +static void secp256k1_gej_double_var(secp256k1_gej *r, const secp256k1_gej *a, secp256k1_fe *rzr) { + /* Operations: 3 mul, 4 sqr, 0 normalize, 12 mul_int/add/negate */ + secp256k1_fe t1,t2,t3,t4; + /** For secp256k1, 2Q is infinity if and only if Q is infinity. This is because if 2Q = infinity, + * Q must equal -Q, or that Q.y == -(Q.y), or Q.y is 0. For a point on y^2 = x^3 + 7 to have + * y=0, x^3 must be -7 mod p. However, -7 has no cube root mod p. + */ + r->infinity = a->infinity; + if (r->infinity) { + if (rzr != NULL) { + secp256k1_fe_set_int(rzr, 1); + } + return; + } + + if (rzr != NULL) { + *rzr = a->y; + secp256k1_fe_normalize_weak(rzr); + secp256k1_fe_mul_int(rzr, 2); + } + + secp256k1_fe_mul(&r->z, &a->z, &a->y); + secp256k1_fe_mul_int(&r->z, 2); /* Z' = 2*Y*Z (2) */ + secp256k1_fe_sqr(&t1, &a->x); + secp256k1_fe_mul_int(&t1, 3); /* T1 = 3*X^2 (3) */ + secp256k1_fe_sqr(&t2, &t1); /* T2 = 9*X^4 (1) */ + secp256k1_fe_sqr(&t3, &a->y); + secp256k1_fe_mul_int(&t3, 2); /* T3 = 2*Y^2 (2) */ + secp256k1_fe_sqr(&t4, &t3); + secp256k1_fe_mul_int(&t4, 2); /* T4 = 8*Y^4 (2) */ + secp256k1_fe_mul(&t3, &t3, &a->x); /* T3 = 2*X*Y^2 (1) */ + r->x = t3; + secp256k1_fe_mul_int(&r->x, 4); /* X' = 8*X*Y^2 (4) */ + secp256k1_fe_negate(&r->x, &r->x, 4); /* X' = -8*X*Y^2 (5) */ + secp256k1_fe_add(&r->x, &t2); /* X' = 9*X^4 - 8*X*Y^2 (6) */ + secp256k1_fe_negate(&t2, &t2, 1); /* T2 = -9*X^4 (2) */ + secp256k1_fe_mul_int(&t3, 6); /* T3 = 12*X*Y^2 (6) */ + secp256k1_fe_add(&t3, &t2); /* T3 = 12*X*Y^2 - 9*X^4 (8) */ + secp256k1_fe_mul(&r->y, &t1, &t3); /* Y' = 36*X^3*Y^2 - 27*X^6 (1) */ + secp256k1_fe_negate(&t2, &t4, 2); /* T2 = -8*Y^4 (3) */ + secp256k1_fe_add(&r->y, &t2); /* Y' = 36*X^3*Y^2 - 27*X^6 - 8*Y^4 (4) */ +} + +static SECP256K1_INLINE void secp256k1_gej_double_nonzero(secp256k1_gej *r, const secp256k1_gej *a, secp256k1_fe *rzr) { + VERIFY_CHECK(!secp256k1_gej_is_infinity(a)); + secp256k1_gej_double_var(r, a, rzr); +} + +static void secp256k1_gej_add_var(secp256k1_gej *r, const secp256k1_gej *a, const secp256k1_gej *b, secp256k1_fe *rzr) { + /* Operations: 12 mul, 4 sqr, 2 normalize, 12 mul_int/add/negate */ + secp256k1_fe z22, z12, u1, u2, s1, s2, h, i, i2, h2, h3, t; + + if (a->infinity) { + VERIFY_CHECK(rzr == NULL); + *r = *b; + return; + } + + if (b->infinity) { + if (rzr != NULL) { + secp256k1_fe_set_int(rzr, 1); + } + *r = *a; + return; + } + + r->infinity = 0; + secp256k1_fe_sqr(&z22, &b->z); + secp256k1_fe_sqr(&z12, &a->z); + secp256k1_fe_mul(&u1, &a->x, &z22); + secp256k1_fe_mul(&u2, &b->x, &z12); + secp256k1_fe_mul(&s1, &a->y, &z22); secp256k1_fe_mul(&s1, &s1, &b->z); + secp256k1_fe_mul(&s2, &b->y, &z12); secp256k1_fe_mul(&s2, &s2, &a->z); + secp256k1_fe_negate(&h, &u1, 1); secp256k1_fe_add(&h, &u2); + secp256k1_fe_negate(&i, &s1, 1); secp256k1_fe_add(&i, &s2); + if (secp256k1_fe_normalizes_to_zero_var(&h)) { + if (secp256k1_fe_normalizes_to_zero_var(&i)) { + secp256k1_gej_double_var(r, a, rzr); + } else { + if (rzr != NULL) { + secp256k1_fe_set_int(rzr, 0); + } + r->infinity = 1; + } + return; + } + secp256k1_fe_sqr(&i2, &i); + secp256k1_fe_sqr(&h2, &h); + secp256k1_fe_mul(&h3, &h, &h2); + secp256k1_fe_mul(&h, &h, &b->z); + if (rzr != NULL) { + *rzr = h; + } + secp256k1_fe_mul(&r->z, &a->z, &h); + secp256k1_fe_mul(&t, &u1, &h2); + r->x = t; secp256k1_fe_mul_int(&r->x, 2); secp256k1_fe_add(&r->x, &h3); secp256k1_fe_negate(&r->x, &r->x, 3); secp256k1_fe_add(&r->x, &i2); + secp256k1_fe_negate(&r->y, &r->x, 5); secp256k1_fe_add(&r->y, &t); secp256k1_fe_mul(&r->y, &r->y, &i); + secp256k1_fe_mul(&h3, &h3, &s1); secp256k1_fe_negate(&h3, &h3, 1); + secp256k1_fe_add(&r->y, &h3); +} + +static void secp256k1_gej_add_ge_var(secp256k1_gej *r, const secp256k1_gej *a, const secp256k1_ge *b, secp256k1_fe *rzr) { + /* 8 mul, 3 sqr, 4 normalize, 12 mul_int/add/negate */ + secp256k1_fe z12, u1, u2, s1, s2, h, i, i2, h2, h3, t; + if (a->infinity) { + VERIFY_CHECK(rzr == NULL); + secp256k1_gej_set_ge(r, b); + return; + } + if (b->infinity) { + if (rzr != NULL) { + secp256k1_fe_set_int(rzr, 1); + } + *r = *a; + return; + } + r->infinity = 0; + + secp256k1_fe_sqr(&z12, &a->z); + u1 = a->x; secp256k1_fe_normalize_weak(&u1); + secp256k1_fe_mul(&u2, &b->x, &z12); + s1 = a->y; secp256k1_fe_normalize_weak(&s1); + secp256k1_fe_mul(&s2, &b->y, &z12); secp256k1_fe_mul(&s2, &s2, &a->z); + secp256k1_fe_negate(&h, &u1, 1); secp256k1_fe_add(&h, &u2); + secp256k1_fe_negate(&i, &s1, 1); secp256k1_fe_add(&i, &s2); + if (secp256k1_fe_normalizes_to_zero_var(&h)) { + if (secp256k1_fe_normalizes_to_zero_var(&i)) { + secp256k1_gej_double_var(r, a, rzr); + } else { + if (rzr != NULL) { + secp256k1_fe_set_int(rzr, 0); + } + r->infinity = 1; + } + return; + } + secp256k1_fe_sqr(&i2, &i); + secp256k1_fe_sqr(&h2, &h); + secp256k1_fe_mul(&h3, &h, &h2); + if (rzr != NULL) { + *rzr = h; + } + secp256k1_fe_mul(&r->z, &a->z, &h); + secp256k1_fe_mul(&t, &u1, &h2); + r->x = t; secp256k1_fe_mul_int(&r->x, 2); secp256k1_fe_add(&r->x, &h3); secp256k1_fe_negate(&r->x, &r->x, 3); secp256k1_fe_add(&r->x, &i2); + secp256k1_fe_negate(&r->y, &r->x, 5); secp256k1_fe_add(&r->y, &t); secp256k1_fe_mul(&r->y, &r->y, &i); + secp256k1_fe_mul(&h3, &h3, &s1); secp256k1_fe_negate(&h3, &h3, 1); + secp256k1_fe_add(&r->y, &h3); +} + +static void secp256k1_gej_add_zinv_var(secp256k1_gej *r, const secp256k1_gej *a, const secp256k1_ge *b, const secp256k1_fe *bzinv) { + /* 9 mul, 3 sqr, 4 normalize, 12 mul_int/add/negate */ + secp256k1_fe az, z12, u1, u2, s1, s2, h, i, i2, h2, h3, t; + + if (b->infinity) { + *r = *a; + return; + } + if (a->infinity) { + secp256k1_fe bzinv2, bzinv3; + r->infinity = b->infinity; + secp256k1_fe_sqr(&bzinv2, bzinv); + secp256k1_fe_mul(&bzinv3, &bzinv2, bzinv); + secp256k1_fe_mul(&r->x, &b->x, &bzinv2); + secp256k1_fe_mul(&r->y, &b->y, &bzinv3); + secp256k1_fe_set_int(&r->z, 1); + return; + } + r->infinity = 0; + + /** We need to calculate (rx,ry,rz) = (ax,ay,az) + (bx,by,1/bzinv). Due to + * secp256k1's isomorphism we can multiply the Z coordinates on both sides + * by bzinv, and get: (rx,ry,rz*bzinv) = (ax,ay,az*bzinv) + (bx,by,1). + * This means that (rx,ry,rz) can be calculated as + * (ax,ay,az*bzinv) + (bx,by,1), when not applying the bzinv factor to rz. + * The variable az below holds the modified Z coordinate for a, which is used + * for the computation of rx and ry, but not for rz. + */ + secp256k1_fe_mul(&az, &a->z, bzinv); + + secp256k1_fe_sqr(&z12, &az); + u1 = a->x; secp256k1_fe_normalize_weak(&u1); + secp256k1_fe_mul(&u2, &b->x, &z12); + s1 = a->y; secp256k1_fe_normalize_weak(&s1); + secp256k1_fe_mul(&s2, &b->y, &z12); secp256k1_fe_mul(&s2, &s2, &az); + secp256k1_fe_negate(&h, &u1, 1); secp256k1_fe_add(&h, &u2); + secp256k1_fe_negate(&i, &s1, 1); secp256k1_fe_add(&i, &s2); + if (secp256k1_fe_normalizes_to_zero_var(&h)) { + if (secp256k1_fe_normalizes_to_zero_var(&i)) { + secp256k1_gej_double_var(r, a, NULL); + } else { + r->infinity = 1; + } + return; + } + secp256k1_fe_sqr(&i2, &i); + secp256k1_fe_sqr(&h2, &h); + secp256k1_fe_mul(&h3, &h, &h2); + r->z = a->z; secp256k1_fe_mul(&r->z, &r->z, &h); + secp256k1_fe_mul(&t, &u1, &h2); + r->x = t; secp256k1_fe_mul_int(&r->x, 2); secp256k1_fe_add(&r->x, &h3); secp256k1_fe_negate(&r->x, &r->x, 3); secp256k1_fe_add(&r->x, &i2); + secp256k1_fe_negate(&r->y, &r->x, 5); secp256k1_fe_add(&r->y, &t); secp256k1_fe_mul(&r->y, &r->y, &i); + secp256k1_fe_mul(&h3, &h3, &s1); secp256k1_fe_negate(&h3, &h3, 1); + secp256k1_fe_add(&r->y, &h3); +} + + +static void secp256k1_gej_add_ge(secp256k1_gej *r, const secp256k1_gej *a, const secp256k1_ge *b) { + /* Operations: 7 mul, 5 sqr, 4 normalize, 21 mul_int/add/negate/cmov */ + static const secp256k1_fe fe_1 = SECP256K1_FE_CONST(0, 0, 0, 0, 0, 0, 0, 1); + secp256k1_fe zz, u1, u2, s1, s2, t, tt, m, n, q, rr; + secp256k1_fe m_alt, rr_alt; + int infinity, degenerate; + VERIFY_CHECK(!b->infinity); + VERIFY_CHECK(a->infinity == 0 || a->infinity == 1); + + /** In: + * Eric Brier and Marc Joye, Weierstrass Elliptic Curves and Side-Channel Attacks. + * In D. Naccache and P. Paillier, Eds., Public Key Cryptography, vol. 2274 of Lecture Notes in Computer Science, pages 335-345. Springer-Verlag, 2002. + * we find as solution for a unified addition/doubling formula: + * lambda = ((x1 + x2)^2 - x1 * x2 + a) / (y1 + y2), with a = 0 for secp256k1's curve equation. + * x3 = lambda^2 - (x1 + x2) + * 2*y3 = lambda * (x1 + x2 - 2 * x3) - (y1 + y2). + * + * Substituting x_i = Xi / Zi^2 and yi = Yi / Zi^3, for i=1,2,3, gives: + * U1 = X1*Z2^2, U2 = X2*Z1^2 + * S1 = Y1*Z2^3, S2 = Y2*Z1^3 + * Z = Z1*Z2 + * T = U1+U2 + * M = S1+S2 + * Q = T*M^2 + * R = T^2-U1*U2 + * X3 = 4*(R^2-Q) + * Y3 = 4*(R*(3*Q-2*R^2)-M^4) + * Z3 = 2*M*Z + * (Note that the paper uses xi = Xi / Zi and yi = Yi / Zi instead.) + * + * This formula has the benefit of being the same for both addition + * of distinct points and doubling. However, it breaks down in the + * case that either point is infinity, or that y1 = -y2. We handle + * these cases in the following ways: + * + * - If b is infinity we simply bail by means of a VERIFY_CHECK. + * + * - If a is infinity, we detect this, and at the end of the + * computation replace the result (which will be meaningless, + * but we compute to be constant-time) with b.x : b.y : 1. + * + * - If a = -b, we have y1 = -y2, which is a degenerate case. + * But here the answer is infinity, so we simply set the + * infinity flag of the result, overriding the computed values + * without even needing to cmov. + * + * - If y1 = -y2 but x1 != x2, which does occur thanks to certain + * properties of our curve (specifically, 1 has nontrivial cube + * roots in our field, and the curve equation has no x coefficient) + * then the answer is not infinity but also not given by the above + * equation. In this case, we cmov in place an alternate expression + * for lambda. Specifically (y1 - y2)/(x1 - x2). Where both these + * expressions for lambda are defined, they are equal, and can be + * obtained from each other by multiplication by (y1 + y2)/(y1 + y2) + * then substitution of x^3 + 7 for y^2 (using the curve equation). + * For all pairs of nonzero points (a, b) at least one is defined, + * so this covers everything. + */ + + secp256k1_fe_sqr(&zz, &a->z); /* z = Z1^2 */ + u1 = a->x; secp256k1_fe_normalize_weak(&u1); /* u1 = U1 = X1*Z2^2 (1) */ + secp256k1_fe_mul(&u2, &b->x, &zz); /* u2 = U2 = X2*Z1^2 (1) */ + s1 = a->y; secp256k1_fe_normalize_weak(&s1); /* s1 = S1 = Y1*Z2^3 (1) */ + secp256k1_fe_mul(&s2, &b->y, &zz); /* s2 = Y2*Z1^2 (1) */ + secp256k1_fe_mul(&s2, &s2, &a->z); /* s2 = S2 = Y2*Z1^3 (1) */ + t = u1; secp256k1_fe_add(&t, &u2); /* t = T = U1+U2 (2) */ + m = s1; secp256k1_fe_add(&m, &s2); /* m = M = S1+S2 (2) */ + secp256k1_fe_sqr(&rr, &t); /* rr = T^2 (1) */ + secp256k1_fe_negate(&m_alt, &u2, 1); /* Malt = -X2*Z1^2 */ + secp256k1_fe_mul(&tt, &u1, &m_alt); /* tt = -U1*U2 (2) */ + secp256k1_fe_add(&rr, &tt); /* rr = R = T^2-U1*U2 (3) */ + /** If lambda = R/M = 0/0 we have a problem (except in the "trivial" + * case that Z = z1z2 = 0, and this is special-cased later on). */ + degenerate = secp256k1_fe_normalizes_to_zero(&m) & + secp256k1_fe_normalizes_to_zero(&rr); + /* This only occurs when y1 == -y2 and x1^3 == x2^3, but x1 != x2. + * This means either x1 == beta*x2 or beta*x1 == x2, where beta is + * a nontrivial cube root of one. In either case, an alternate + * non-indeterminate expression for lambda is (y1 - y2)/(x1 - x2), + * so we set R/M equal to this. */ + rr_alt = s1; + secp256k1_fe_mul_int(&rr_alt, 2); /* rr = Y1*Z2^3 - Y2*Z1^3 (2) */ + secp256k1_fe_add(&m_alt, &u1); /* Malt = X1*Z2^2 - X2*Z1^2 */ + + secp256k1_fe_cmov(&rr_alt, &rr, !degenerate); + secp256k1_fe_cmov(&m_alt, &m, !degenerate); + /* Now Ralt / Malt = lambda and is guaranteed not to be 0/0. + * From here on out Ralt and Malt represent the numerator + * and denominator of lambda; R and M represent the explicit + * expressions x1^2 + x2^2 + x1x2 and y1 + y2. */ + secp256k1_fe_sqr(&n, &m_alt); /* n = Malt^2 (1) */ + secp256k1_fe_mul(&q, &n, &t); /* q = Q = T*Malt^2 (1) */ + /* These two lines use the observation that either M == Malt or M == 0, + * so M^3 * Malt is either Malt^4 (which is computed by squaring), or + * zero (which is "computed" by cmov). So the cost is one squaring + * versus two multiplications. */ + secp256k1_fe_sqr(&n, &n); + secp256k1_fe_cmov(&n, &m, degenerate); /* n = M^3 * Malt (2) */ + secp256k1_fe_sqr(&t, &rr_alt); /* t = Ralt^2 (1) */ + secp256k1_fe_mul(&r->z, &a->z, &m_alt); /* r->z = Malt*Z (1) */ + infinity = secp256k1_fe_normalizes_to_zero(&r->z) * (1 - a->infinity); + secp256k1_fe_mul_int(&r->z, 2); /* r->z = Z3 = 2*Malt*Z (2) */ + secp256k1_fe_negate(&q, &q, 1); /* q = -Q (2) */ + secp256k1_fe_add(&t, &q); /* t = Ralt^2-Q (3) */ + secp256k1_fe_normalize_weak(&t); + r->x = t; /* r->x = Ralt^2-Q (1) */ + secp256k1_fe_mul_int(&t, 2); /* t = 2*x3 (2) */ + secp256k1_fe_add(&t, &q); /* t = 2*x3 - Q: (4) */ + secp256k1_fe_mul(&t, &t, &rr_alt); /* t = Ralt*(2*x3 - Q) (1) */ + secp256k1_fe_add(&t, &n); /* t = Ralt*(2*x3 - Q) + M^3*Malt (3) */ + secp256k1_fe_negate(&r->y, &t, 3); /* r->y = Ralt*(Q - 2x3) - M^3*Malt (4) */ + secp256k1_fe_normalize_weak(&r->y); + secp256k1_fe_mul_int(&r->x, 4); /* r->x = X3 = 4*(Ralt^2-Q) */ + secp256k1_fe_mul_int(&r->y, 4); /* r->y = Y3 = 4*Ralt*(Q - 2x3) - 4*M^3*Malt (4) */ + + /** In case a->infinity == 1, replace r with (b->x, b->y, 1). */ + secp256k1_fe_cmov(&r->x, &b->x, a->infinity); + secp256k1_fe_cmov(&r->y, &b->y, a->infinity); + secp256k1_fe_cmov(&r->z, &fe_1, a->infinity); + r->infinity = infinity; +} + +static void secp256k1_gej_rescale(secp256k1_gej *r, const secp256k1_fe *s) { + /* Operations: 4 mul, 1 sqr */ + secp256k1_fe zz; + VERIFY_CHECK(!secp256k1_fe_is_zero(s)); + secp256k1_fe_sqr(&zz, s); + secp256k1_fe_mul(&r->x, &r->x, &zz); /* r->x *= s^2 */ + secp256k1_fe_mul(&r->y, &r->y, &zz); + secp256k1_fe_mul(&r->y, &r->y, s); /* r->y *= s^3 */ + secp256k1_fe_mul(&r->z, &r->z, s); /* r->z *= s */ +} + +static void secp256k1_ge_to_storage(secp256k1_ge_storage *r, const secp256k1_ge *a) { + secp256k1_fe x, y; + VERIFY_CHECK(!a->infinity); + x = a->x; + secp256k1_fe_normalize(&x); + y = a->y; + secp256k1_fe_normalize(&y); + secp256k1_fe_to_storage(&r->x, &x); + secp256k1_fe_to_storage(&r->y, &y); +} + +static void secp256k1_ge_from_storage(secp256k1_ge *r, const secp256k1_ge_storage *a) { + secp256k1_fe_from_storage(&r->x, &a->x); + secp256k1_fe_from_storage(&r->y, &a->y); + r->infinity = 0; +} + +static SECP256K1_INLINE void secp256k1_ge_storage_cmov(secp256k1_ge_storage *r, const secp256k1_ge_storage *a, int flag) { + secp256k1_fe_storage_cmov(&r->x, &a->x, flag); + secp256k1_fe_storage_cmov(&r->y, &a->y, flag); +} + +#ifdef USE_ENDOMORPHISM +static void secp256k1_ge_mul_lambda(secp256k1_ge *r, const secp256k1_ge *a) { + static const secp256k1_fe beta = SECP256K1_FE_CONST( + 0x7ae96a2bul, 0x657c0710ul, 0x6e64479eul, 0xac3434e9ul, + 0x9cf04975ul, 0x12f58995ul, 0xc1396c28ul, 0x719501eeul + ); + *r = *a; + secp256k1_fe_mul(&r->x, &r->x, &beta); +} +#endif + +#endif diff --git a/src/secp256k1/src/hash.h b/src/secp256k1/src/hash.h new file mode 100644 index 00000000..0ff01e63 --- /dev/null +++ b/src/secp256k1/src/hash.h @@ -0,0 +1,41 @@ +/********************************************************************** + * Copyright (c) 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_HASH_ +#define _SECP256K1_HASH_ + +#include +#include + +typedef struct { + uint32_t s[32]; + uint32_t buf[16]; /* In big endian */ + size_t bytes; +} secp256k1_sha256_t; + +static void secp256k1_sha256_initialize(secp256k1_sha256_t *hash); +static void secp256k1_sha256_write(secp256k1_sha256_t *hash, const unsigned char *data, size_t size); +static void secp256k1_sha256_finalize(secp256k1_sha256_t *hash, unsigned char *out32); + +typedef struct { + secp256k1_sha256_t inner, outer; +} secp256k1_hmac_sha256_t; + +static void secp256k1_hmac_sha256_initialize(secp256k1_hmac_sha256_t *hash, const unsigned char *key, size_t size); +static void secp256k1_hmac_sha256_write(secp256k1_hmac_sha256_t *hash, const unsigned char *data, size_t size); +static void secp256k1_hmac_sha256_finalize(secp256k1_hmac_sha256_t *hash, unsigned char *out32); + +typedef struct { + unsigned char v[32]; + unsigned char k[32]; + int retry; +} secp256k1_rfc6979_hmac_sha256_t; + +static void secp256k1_rfc6979_hmac_sha256_initialize(secp256k1_rfc6979_hmac_sha256_t *rng, const unsigned char *key, size_t keylen); +static void secp256k1_rfc6979_hmac_sha256_generate(secp256k1_rfc6979_hmac_sha256_t *rng, unsigned char *out, size_t outlen); +static void secp256k1_rfc6979_hmac_sha256_finalize(secp256k1_rfc6979_hmac_sha256_t *rng); + +#endif diff --git a/src/secp256k1/src/hash_impl.h b/src/secp256k1/src/hash_impl.h new file mode 100644 index 00000000..ae55df6d --- /dev/null +++ b/src/secp256k1/src/hash_impl.h @@ -0,0 +1,283 @@ +/********************************************************************** + * Copyright (c) 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_HASH_IMPL_H_ +#define _SECP256K1_HASH_IMPL_H_ + +#include "hash.h" + +#include +#include +#include + +#define Ch(x,y,z) ((z) ^ ((x) & ((y) ^ (z)))) +#define Maj(x,y,z) (((x) & (y)) | ((z) & ((x) | (y)))) +#define Sigma0(x) (((x) >> 2 | (x) << 30) ^ ((x) >> 13 | (x) << 19) ^ ((x) >> 22 | (x) << 10)) +#define Sigma1(x) (((x) >> 6 | (x) << 26) ^ ((x) >> 11 | (x) << 21) ^ ((x) >> 25 | (x) << 7)) +#define sigma0(x) (((x) >> 7 | (x) << 25) ^ ((x) >> 18 | (x) << 14) ^ ((x) >> 3)) +#define sigma1(x) (((x) >> 17 | (x) << 15) ^ ((x) >> 19 | (x) << 13) ^ ((x) >> 10)) + +#define Round(a,b,c,d,e,f,g,h,k,w) do { \ + uint32_t t1 = (h) + Sigma1(e) + Ch((e), (f), (g)) + (k) + (w); \ + uint32_t t2 = Sigma0(a) + Maj((a), (b), (c)); \ + (d) += t1; \ + (h) = t1 + t2; \ +} while(0) + +#ifdef WORDS_BIGENDIAN +#define BE32(x) (x) +#else +#define BE32(p) ((((p) & 0xFF) << 24) | (((p) & 0xFF00) << 8) | (((p) & 0xFF0000) >> 8) | (((p) & 0xFF000000) >> 24)) +#endif + +static void secp256k1_sha256_initialize(secp256k1_sha256_t *hash) { + hash->s[0] = 0x6a09e667ul; + hash->s[1] = 0xbb67ae85ul; + hash->s[2] = 0x3c6ef372ul; + hash->s[3] = 0xa54ff53aul; + hash->s[4] = 0x510e527ful; + hash->s[5] = 0x9b05688cul; + hash->s[6] = 0x1f83d9abul; + hash->s[7] = 0x5be0cd19ul; + hash->bytes = 0; +} + +/** Perform one SHA-256 transformation, processing 16 big endian 32-bit words. */ +static void secp256k1_sha256_transform(uint32_t* s, const uint32_t* chunk) { + uint32_t a = s[0], b = s[1], c = s[2], d = s[3], e = s[4], f = s[5], g = s[6], h = s[7]; + uint32_t w0, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, w11, w12, w13, w14, w15; + + Round(a, b, c, d, e, f, g, h, 0x428a2f98, w0 = BE32(chunk[0])); + Round(h, a, b, c, d, e, f, g, 0x71374491, w1 = BE32(chunk[1])); + Round(g, h, a, b, c, d, e, f, 0xb5c0fbcf, w2 = BE32(chunk[2])); + Round(f, g, h, a, b, c, d, e, 0xe9b5dba5, w3 = BE32(chunk[3])); + Round(e, f, g, h, a, b, c, d, 0x3956c25b, w4 = BE32(chunk[4])); + Round(d, e, f, g, h, a, b, c, 0x59f111f1, w5 = BE32(chunk[5])); + Round(c, d, e, f, g, h, a, b, 0x923f82a4, w6 = BE32(chunk[6])); + Round(b, c, d, e, f, g, h, a, 0xab1c5ed5, w7 = BE32(chunk[7])); + Round(a, b, c, d, e, f, g, h, 0xd807aa98, w8 = BE32(chunk[8])); + Round(h, a, b, c, d, e, f, g, 0x12835b01, w9 = BE32(chunk[9])); + Round(g, h, a, b, c, d, e, f, 0x243185be, w10 = BE32(chunk[10])); + Round(f, g, h, a, b, c, d, e, 0x550c7dc3, w11 = BE32(chunk[11])); + Round(e, f, g, h, a, b, c, d, 0x72be5d74, w12 = BE32(chunk[12])); + Round(d, e, f, g, h, a, b, c, 0x80deb1fe, w13 = BE32(chunk[13])); + Round(c, d, e, f, g, h, a, b, 0x9bdc06a7, w14 = BE32(chunk[14])); + Round(b, c, d, e, f, g, h, a, 0xc19bf174, w15 = BE32(chunk[15])); + + Round(a, b, c, d, e, f, g, h, 0xe49b69c1, w0 += sigma1(w14) + w9 + sigma0(w1)); + Round(h, a, b, c, d, e, f, g, 0xefbe4786, w1 += sigma1(w15) + w10 + sigma0(w2)); + Round(g, h, a, b, c, d, e, f, 0x0fc19dc6, w2 += sigma1(w0) + w11 + sigma0(w3)); + Round(f, g, h, a, b, c, d, e, 0x240ca1cc, w3 += sigma1(w1) + w12 + sigma0(w4)); + Round(e, f, g, h, a, b, c, d, 0x2de92c6f, w4 += sigma1(w2) + w13 + sigma0(w5)); + Round(d, e, f, g, h, a, b, c, 0x4a7484aa, w5 += sigma1(w3) + w14 + sigma0(w6)); + Round(c, d, e, f, g, h, a, b, 0x5cb0a9dc, w6 += sigma1(w4) + w15 + sigma0(w7)); + Round(b, c, d, e, f, g, h, a, 0x76f988da, w7 += sigma1(w5) + w0 + sigma0(w8)); + Round(a, b, c, d, e, f, g, h, 0x983e5152, w8 += sigma1(w6) + w1 + sigma0(w9)); + Round(h, a, b, c, d, e, f, g, 0xa831c66d, w9 += sigma1(w7) + w2 + sigma0(w10)); + Round(g, h, a, b, c, d, e, f, 0xb00327c8, w10 += sigma1(w8) + w3 + sigma0(w11)); + Round(f, g, h, a, b, c, d, e, 0xbf597fc7, w11 += sigma1(w9) + w4 + sigma0(w12)); + Round(e, f, g, h, a, b, c, d, 0xc6e00bf3, w12 += sigma1(w10) + w5 + sigma0(w13)); + Round(d, e, f, g, h, a, b, c, 0xd5a79147, w13 += sigma1(w11) + w6 + sigma0(w14)); + Round(c, d, e, f, g, h, a, b, 0x06ca6351, w14 += sigma1(w12) + w7 + sigma0(w15)); + Round(b, c, d, e, f, g, h, a, 0x14292967, w15 += sigma1(w13) + w8 + sigma0(w0)); + + Round(a, b, c, d, e, f, g, h, 0x27b70a85, w0 += sigma1(w14) + w9 + sigma0(w1)); + Round(h, a, b, c, d, e, f, g, 0x2e1b2138, w1 += sigma1(w15) + w10 + sigma0(w2)); + Round(g, h, a, b, c, d, e, f, 0x4d2c6dfc, w2 += sigma1(w0) + w11 + sigma0(w3)); + Round(f, g, h, a, b, c, d, e, 0x53380d13, w3 += sigma1(w1) + w12 + sigma0(w4)); + Round(e, f, g, h, a, b, c, d, 0x650a7354, w4 += sigma1(w2) + w13 + sigma0(w5)); + Round(d, e, f, g, h, a, b, c, 0x766a0abb, w5 += sigma1(w3) + w14 + sigma0(w6)); + Round(c, d, e, f, g, h, a, b, 0x81c2c92e, w6 += sigma1(w4) + w15 + sigma0(w7)); + Round(b, c, d, e, f, g, h, a, 0x92722c85, w7 += sigma1(w5) + w0 + sigma0(w8)); + Round(a, b, c, d, e, f, g, h, 0xa2bfe8a1, w8 += sigma1(w6) + w1 + sigma0(w9)); + Round(h, a, b, c, d, e, f, g, 0xa81a664b, w9 += sigma1(w7) + w2 + sigma0(w10)); + Round(g, h, a, b, c, d, e, f, 0xc24b8b70, w10 += sigma1(w8) + w3 + sigma0(w11)); + Round(f, g, h, a, b, c, d, e, 0xc76c51a3, w11 += sigma1(w9) + w4 + sigma0(w12)); + Round(e, f, g, h, a, b, c, d, 0xd192e819, w12 += sigma1(w10) + w5 + sigma0(w13)); + Round(d, e, f, g, h, a, b, c, 0xd6990624, w13 += sigma1(w11) + w6 + sigma0(w14)); + Round(c, d, e, f, g, h, a, b, 0xf40e3585, w14 += sigma1(w12) + w7 + sigma0(w15)); + Round(b, c, d, e, f, g, h, a, 0x106aa070, w15 += sigma1(w13) + w8 + sigma0(w0)); + + Round(a, b, c, d, e, f, g, h, 0x19a4c116, w0 += sigma1(w14) + w9 + sigma0(w1)); + Round(h, a, b, c, d, e, f, g, 0x1e376c08, w1 += sigma1(w15) + w10 + sigma0(w2)); + Round(g, h, a, b, c, d, e, f, 0x2748774c, w2 += sigma1(w0) + w11 + sigma0(w3)); + Round(f, g, h, a, b, c, d, e, 0x34b0bcb5, w3 += sigma1(w1) + w12 + sigma0(w4)); + Round(e, f, g, h, a, b, c, d, 0x391c0cb3, w4 += sigma1(w2) + w13 + sigma0(w5)); + Round(d, e, f, g, h, a, b, c, 0x4ed8aa4a, w5 += sigma1(w3) + w14 + sigma0(w6)); + Round(c, d, e, f, g, h, a, b, 0x5b9cca4f, w6 += sigma1(w4) + w15 + sigma0(w7)); + Round(b, c, d, e, f, g, h, a, 0x682e6ff3, w7 += sigma1(w5) + w0 + sigma0(w8)); + Round(a, b, c, d, e, f, g, h, 0x748f82ee, w8 += sigma1(w6) + w1 + sigma0(w9)); + Round(h, a, b, c, d, e, f, g, 0x78a5636f, w9 += sigma1(w7) + w2 + sigma0(w10)); + Round(g, h, a, b, c, d, e, f, 0x84c87814, w10 += sigma1(w8) + w3 + sigma0(w11)); + Round(f, g, h, a, b, c, d, e, 0x8cc70208, w11 += sigma1(w9) + w4 + sigma0(w12)); + Round(e, f, g, h, a, b, c, d, 0x90befffa, w12 += sigma1(w10) + w5 + sigma0(w13)); + Round(d, e, f, g, h, a, b, c, 0xa4506ceb, w13 += sigma1(w11) + w6 + sigma0(w14)); + Round(c, d, e, f, g, h, a, b, 0xbef9a3f7, w14 + sigma1(w12) + w7 + sigma0(w15)); + Round(b, c, d, e, f, g, h, a, 0xc67178f2, w15 + sigma1(w13) + w8 + sigma0(w0)); + + s[0] += a; + s[1] += b; + s[2] += c; + s[3] += d; + s[4] += e; + s[5] += f; + s[6] += g; + s[7] += h; +} + +static void secp256k1_sha256_write(secp256k1_sha256_t *hash, const unsigned char *data, size_t len) { + size_t bufsize = hash->bytes & 0x3F; + hash->bytes += len; + while (bufsize + len >= 64) { + /* Fill the buffer, and process it. */ + memcpy(((unsigned char*)hash->buf) + bufsize, data, 64 - bufsize); + data += 64 - bufsize; + len -= 64 - bufsize; + secp256k1_sha256_transform(hash->s, hash->buf); + bufsize = 0; + } + if (len) { + /* Fill the buffer with what remains. */ + memcpy(((unsigned char*)hash->buf) + bufsize, data, len); + } +} + +static void secp256k1_sha256_finalize(secp256k1_sha256_t *hash, unsigned char *out32) { + static const unsigned char pad[64] = {0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}; + uint32_t sizedesc[2]; + uint32_t out[8]; + int i = 0; + sizedesc[0] = BE32(hash->bytes >> 29); + sizedesc[1] = BE32(hash->bytes << 3); + secp256k1_sha256_write(hash, pad, 1 + ((119 - (hash->bytes % 64)) % 64)); + secp256k1_sha256_write(hash, (const unsigned char*)sizedesc, 8); + for (i = 0; i < 8; i++) { + out[i] = BE32(hash->s[i]); + hash->s[i] = 0; + } + memcpy(out32, (const unsigned char*)out, 32); +} + +static void secp256k1_hmac_sha256_initialize(secp256k1_hmac_sha256_t *hash, const unsigned char *key, size_t keylen) { + int n; + unsigned char rkey[64]; + if (keylen <= 64) { + memcpy(rkey, key, keylen); + memset(rkey + keylen, 0, 64 - keylen); + } else { + secp256k1_sha256_t sha256; + secp256k1_sha256_initialize(&sha256); + secp256k1_sha256_write(&sha256, key, keylen); + secp256k1_sha256_finalize(&sha256, rkey); + memset(rkey + 32, 0, 32); + } + + secp256k1_sha256_initialize(&hash->outer); + for (n = 0; n < 64; n++) { + rkey[n] ^= 0x5c; + } + secp256k1_sha256_write(&hash->outer, rkey, 64); + + secp256k1_sha256_initialize(&hash->inner); + for (n = 0; n < 64; n++) { + rkey[n] ^= 0x5c ^ 0x36; + } + secp256k1_sha256_write(&hash->inner, rkey, 64); + memset(rkey, 0, 64); +} + +static void secp256k1_hmac_sha256_write(secp256k1_hmac_sha256_t *hash, const unsigned char *data, size_t size) { + secp256k1_sha256_write(&hash->inner, data, size); +} + +static void secp256k1_hmac_sha256_finalize(secp256k1_hmac_sha256_t *hash, unsigned char *out32) { + unsigned char temp[32]; + secp256k1_sha256_finalize(&hash->inner, temp); + secp256k1_sha256_write(&hash->outer, temp, 32); + memset(temp, 0, 32); + secp256k1_sha256_finalize(&hash->outer, out32); +} + + +static void secp256k1_rfc6979_hmac_sha256_initialize(secp256k1_rfc6979_hmac_sha256_t *rng, const unsigned char *key, size_t keylen) { + secp256k1_hmac_sha256_t hmac; + static const unsigned char zero[1] = {0x00}; + static const unsigned char one[1] = {0x01}; + + memset(rng->v, 0x01, 32); /* RFC6979 3.2.b. */ + memset(rng->k, 0x00, 32); /* RFC6979 3.2.c. */ + + /* RFC6979 3.2.d. */ + secp256k1_hmac_sha256_initialize(&hmac, rng->k, 32); + secp256k1_hmac_sha256_write(&hmac, rng->v, 32); + secp256k1_hmac_sha256_write(&hmac, zero, 1); + secp256k1_hmac_sha256_write(&hmac, key, keylen); + secp256k1_hmac_sha256_finalize(&hmac, rng->k); + secp256k1_hmac_sha256_initialize(&hmac, rng->k, 32); + secp256k1_hmac_sha256_write(&hmac, rng->v, 32); + secp256k1_hmac_sha256_finalize(&hmac, rng->v); + + /* RFC6979 3.2.f. */ + secp256k1_hmac_sha256_initialize(&hmac, rng->k, 32); + secp256k1_hmac_sha256_write(&hmac, rng->v, 32); + secp256k1_hmac_sha256_write(&hmac, one, 1); + secp256k1_hmac_sha256_write(&hmac, key, keylen); + secp256k1_hmac_sha256_finalize(&hmac, rng->k); + secp256k1_hmac_sha256_initialize(&hmac, rng->k, 32); + secp256k1_hmac_sha256_write(&hmac, rng->v, 32); + secp256k1_hmac_sha256_finalize(&hmac, rng->v); + rng->retry = 0; +} + +static void secp256k1_rfc6979_hmac_sha256_generate(secp256k1_rfc6979_hmac_sha256_t *rng, unsigned char *out, size_t outlen) { + /* RFC6979 3.2.h. */ + static const unsigned char zero[1] = {0x00}; + if (rng->retry) { + secp256k1_hmac_sha256_t hmac; + secp256k1_hmac_sha256_initialize(&hmac, rng->k, 32); + secp256k1_hmac_sha256_write(&hmac, rng->v, 32); + secp256k1_hmac_sha256_write(&hmac, zero, 1); + secp256k1_hmac_sha256_finalize(&hmac, rng->k); + secp256k1_hmac_sha256_initialize(&hmac, rng->k, 32); + secp256k1_hmac_sha256_write(&hmac, rng->v, 32); + secp256k1_hmac_sha256_finalize(&hmac, rng->v); + } + + while (outlen > 0) { + secp256k1_hmac_sha256_t hmac; + int now = outlen; + secp256k1_hmac_sha256_initialize(&hmac, rng->k, 32); + secp256k1_hmac_sha256_write(&hmac, rng->v, 32); + secp256k1_hmac_sha256_finalize(&hmac, rng->v); + if (now > 32) { + now = 32; + } + memcpy(out, rng->v, now); + out += now; + outlen -= now; + } + + rng->retry = 1; +} + +static void secp256k1_rfc6979_hmac_sha256_finalize(secp256k1_rfc6979_hmac_sha256_t *rng) { + memset(rng->k, 0, 32); + memset(rng->v, 0, 32); + rng->retry = 0; +} + + +#undef Round +#undef sigma0 +#undef sigma1 +#undef Sigma0 +#undef Sigma1 +#undef Ch +#undef Maj +#undef ReadBE32 +#undef WriteBE32 + +#endif diff --git a/src/secp256k1/src/java/org/bitcoin/NativeSecp256k1.java b/src/secp256k1/src/java/org/bitcoin/NativeSecp256k1.java new file mode 100644 index 00000000..90a498ea --- /dev/null +++ b/src/secp256k1/src/java/org/bitcoin/NativeSecp256k1.java @@ -0,0 +1,60 @@ +package org.bitcoin; + +import java.nio.ByteBuffer; +import java.nio.ByteOrder; + +import com.google.common.base.Preconditions; + + +/** + * This class holds native methods to handle ECDSA verification. + * You can find an example library that can be used for this at + * https://github.com/sipa/secp256k1 + */ +public class NativeSecp256k1 { + public static final boolean enabled; + static { + boolean isEnabled = true; + try { + System.loadLibrary("javasecp256k1"); + } catch (UnsatisfiedLinkError e) { + isEnabled = false; + } + enabled = isEnabled; + } + + private static ThreadLocal nativeECDSABuffer = new ThreadLocal(); + /** + * Verifies the given secp256k1 signature in native code. + * Calling when enabled == false is undefined (probably library not loaded) + * + * @param data The data which was signed, must be exactly 32 bytes + * @param signature The signature + * @param pub The public key which did the signing + */ + public static boolean verify(byte[] data, byte[] signature, byte[] pub) { + Preconditions.checkArgument(data.length == 32 && signature.length <= 520 && pub.length <= 520); + + ByteBuffer byteBuff = nativeECDSABuffer.get(); + if (byteBuff == null) { + byteBuff = ByteBuffer.allocateDirect(32 + 8 + 520 + 520); + byteBuff.order(ByteOrder.nativeOrder()); + nativeECDSABuffer.set(byteBuff); + } + byteBuff.rewind(); + byteBuff.put(data); + byteBuff.putInt(signature.length); + byteBuff.putInt(pub.length); + byteBuff.put(signature); + byteBuff.put(pub); + return secp256k1_ecdsa_verify(byteBuff) == 1; + } + + /** + * @param byteBuff signature format is byte[32] data, + * native-endian int signatureLength, native-endian int pubkeyLength, + * byte[signatureLength] signature, byte[pubkeyLength] pub + * @returns 1 for valid signature, anything else for invalid + */ + private static native int secp256k1_ecdsa_verify(ByteBuffer byteBuff); +} diff --git a/src/secp256k1/src/java/org_bitcoin_NativeSecp256k1.c b/src/secp256k1/src/java/org_bitcoin_NativeSecp256k1.c new file mode 100644 index 00000000..bb4cd707 --- /dev/null +++ b/src/secp256k1/src/java/org_bitcoin_NativeSecp256k1.c @@ -0,0 +1,23 @@ +#include "org_bitcoin_NativeSecp256k1.h" +#include "include/secp256k1.h" + +JNIEXPORT jint JNICALL Java_org_bitcoin_NativeSecp256k1_secp256k1_1ecdsa_1verify + (JNIEnv* env, jclass classObject, jobject byteBufferObject) +{ + unsigned char* data = (unsigned char*) (*env)->GetDirectBufferAddress(env, byteBufferObject); + int sigLen = *((int*)(data + 32)); + int pubLen = *((int*)(data + 32 + 4)); + + return secp256k1_ecdsa_verify(data, 32, data+32+8, sigLen, data+32+8+sigLen, pubLen); +} + +static void __javasecp256k1_attach(void) __attribute__((constructor)); +static void __javasecp256k1_detach(void) __attribute__((destructor)); + +static void __javasecp256k1_attach(void) { + secp256k1_start(SECP256K1_START_VERIFY); +} + +static void __javasecp256k1_detach(void) { + secp256k1_stop(); +} diff --git a/src/secp256k1/src/java/org_bitcoin_NativeSecp256k1.h b/src/secp256k1/src/java/org_bitcoin_NativeSecp256k1.h new file mode 100644 index 00000000..d7fb004f --- /dev/null +++ b/src/secp256k1/src/java/org_bitcoin_NativeSecp256k1.h @@ -0,0 +1,21 @@ +/* DO NOT EDIT THIS FILE - it is machine generated */ +#include +/* Header for class org_bitcoin_NativeSecp256k1 */ + +#ifndef _Included_org_bitcoin_NativeSecp256k1 +#define _Included_org_bitcoin_NativeSecp256k1 +#ifdef __cplusplus +extern "C" { +#endif +/* + * Class: org_bitcoin_NativeSecp256k1 + * Method: secp256k1_ecdsa_verify + * Signature: (Ljava/nio/ByteBuffer;)I + */ +JNIEXPORT jint JNICALL Java_org_bitcoin_NativeSecp256k1_secp256k1_1ecdsa_1verify + (JNIEnv *, jclass, jobject); + +#ifdef __cplusplus +} +#endif +#endif diff --git a/src/secp256k1/src/modules/ecdh/Makefile.am.include b/src/secp256k1/src/modules/ecdh/Makefile.am.include new file mode 100644 index 00000000..670b9c11 --- /dev/null +++ b/src/secp256k1/src/modules/ecdh/Makefile.am.include @@ -0,0 +1,8 @@ +include_HEADERS += include/secp256k1_ecdh.h +noinst_HEADERS += src/modules/ecdh/main_impl.h +noinst_HEADERS += src/modules/ecdh/tests_impl.h +if USE_BENCHMARK +noinst_PROGRAMS += bench_ecdh +bench_ecdh_SOURCES = src/bench_ecdh.c +bench_ecdh_LDADD = libsecp256k1.la $(SECP_LIBS) +endif diff --git a/src/secp256k1/src/modules/ecdh/main_impl.h b/src/secp256k1/src/modules/ecdh/main_impl.h new file mode 100644 index 00000000..c23e4f82 --- /dev/null +++ b/src/secp256k1/src/modules/ecdh/main_impl.h @@ -0,0 +1,54 @@ +/********************************************************************** + * Copyright (c) 2015 Andrew Poelstra * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_MODULE_ECDH_MAIN_ +#define _SECP256K1_MODULE_ECDH_MAIN_ + +#include "include/secp256k1_ecdh.h" +#include "ecmult_const_impl.h" + +int secp256k1_ecdh(const secp256k1_context* ctx, unsigned char *result, const secp256k1_pubkey *point, const unsigned char *scalar) { + int ret = 0; + int overflow = 0; + secp256k1_gej res; + secp256k1_ge pt; + secp256k1_scalar s; + ARG_CHECK(result != NULL); + ARG_CHECK(point != NULL); + ARG_CHECK(scalar != NULL); + (void)ctx; + + secp256k1_pubkey_load(ctx, &pt, point); + secp256k1_scalar_set_b32(&s, scalar, &overflow); + if (overflow || secp256k1_scalar_is_zero(&s)) { + ret = 0; + } else { + unsigned char x[32]; + unsigned char y[1]; + secp256k1_sha256_t sha; + + secp256k1_ecmult_const(&res, &pt, &s); + secp256k1_ge_set_gej(&pt, &res); + /* Compute a hash of the point in compressed form + * Note we cannot use secp256k1_eckey_pubkey_serialize here since it does not + * expect its output to be secret and has a timing sidechannel. */ + secp256k1_fe_normalize(&pt.x); + secp256k1_fe_normalize(&pt.y); + secp256k1_fe_get_b32(x, &pt.x); + y[0] = 0x02 | secp256k1_fe_is_odd(&pt.y); + + secp256k1_sha256_initialize(&sha); + secp256k1_sha256_write(&sha, y, sizeof(y)); + secp256k1_sha256_write(&sha, x, sizeof(x)); + secp256k1_sha256_finalize(&sha, result); + ret = 1; + } + + secp256k1_scalar_clear(&s); + return ret; +} + +#endif diff --git a/src/secp256k1/src/modules/ecdh/tests_impl.h b/src/secp256k1/src/modules/ecdh/tests_impl.h new file mode 100644 index 00000000..7badc903 --- /dev/null +++ b/src/secp256k1/src/modules/ecdh/tests_impl.h @@ -0,0 +1,75 @@ +/********************************************************************** + * Copyright (c) 2015 Andrew Poelstra * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_MODULE_ECDH_TESTS_ +#define _SECP256K1_MODULE_ECDH_TESTS_ + +void test_ecdh_generator_basepoint(void) { + unsigned char s_one[32] = { 0 }; + secp256k1_pubkey point[2]; + int i; + + s_one[31] = 1; + /* Check against pubkey creation when the basepoint is the generator */ + for (i = 0; i < 100; ++i) { + secp256k1_sha256_t sha; + unsigned char s_b32[32]; + unsigned char output_ecdh[32]; + unsigned char output_ser[32]; + unsigned char point_ser[33]; + size_t point_ser_len = sizeof(point_ser); + secp256k1_scalar s; + + random_scalar_order(&s); + secp256k1_scalar_get_b32(s_b32, &s); + + /* compute using ECDH function */ + CHECK(secp256k1_ec_pubkey_create(ctx, &point[0], s_one) == 1); + CHECK(secp256k1_ecdh(ctx, output_ecdh, &point[0], s_b32) == 1); + /* compute "explicitly" */ + CHECK(secp256k1_ec_pubkey_create(ctx, &point[1], s_b32) == 1); + CHECK(secp256k1_ec_pubkey_serialize(ctx, point_ser, &point_ser_len, &point[1], SECP256K1_EC_COMPRESSED) == 1); + CHECK(point_ser_len == sizeof(point_ser)); + secp256k1_sha256_initialize(&sha); + secp256k1_sha256_write(&sha, point_ser, point_ser_len); + secp256k1_sha256_finalize(&sha, output_ser); + /* compare */ + CHECK(memcmp(output_ecdh, output_ser, sizeof(output_ser)) == 0); + } +} + +void test_bad_scalar(void) { + unsigned char s_zero[32] = { 0 }; + unsigned char s_overflow[32] = { + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, + 0xba, 0xae, 0xdc, 0xe6, 0xaf, 0x48, 0xa0, 0x3b, + 0xbf, 0xd2, 0x5e, 0x8c, 0xd0, 0x36, 0x41, 0x41 + }; + unsigned char s_rand[32] = { 0 }; + unsigned char output[32]; + secp256k1_scalar rand; + secp256k1_pubkey point; + + /* Create random point */ + random_scalar_order(&rand); + secp256k1_scalar_get_b32(s_rand, &rand); + CHECK(secp256k1_ec_pubkey_create(ctx, &point, s_rand) == 1); + + /* Try to multiply it by bad values */ + CHECK(secp256k1_ecdh(ctx, output, &point, s_zero) == 0); + CHECK(secp256k1_ecdh(ctx, output, &point, s_overflow) == 0); + /* ...and a good one */ + s_overflow[31] -= 1; + CHECK(secp256k1_ecdh(ctx, output, &point, s_overflow) == 1); +} + +void run_ecdh_tests(void) { + test_ecdh_generator_basepoint(); + test_bad_scalar(); +} + +#endif diff --git a/src/secp256k1/src/modules/recovery/Makefile.am.include b/src/secp256k1/src/modules/recovery/Makefile.am.include new file mode 100644 index 00000000..5de3ea33 --- /dev/null +++ b/src/secp256k1/src/modules/recovery/Makefile.am.include @@ -0,0 +1,8 @@ +include_HEADERS += include/secp256k1_recovery.h +noinst_HEADERS += src/modules/recovery/main_impl.h +noinst_HEADERS += src/modules/recovery/tests_impl.h +if USE_BENCHMARK +noinst_PROGRAMS += bench_recover +bench_recover_SOURCES = src/bench_recover.c +bench_recover_LDADD = libsecp256k1.la $(SECP_LIBS) +endif diff --git a/src/secp256k1/src/modules/recovery/main_impl.h b/src/secp256k1/src/modules/recovery/main_impl.h new file mode 100644 index 00000000..ec42f4bb --- /dev/null +++ b/src/secp256k1/src/modules/recovery/main_impl.h @@ -0,0 +1,193 @@ +/********************************************************************** + * Copyright (c) 2013-2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_MODULE_RECOVERY_MAIN_ +#define _SECP256K1_MODULE_RECOVERY_MAIN_ + +#include "include/secp256k1_recovery.h" + +static void secp256k1_ecdsa_recoverable_signature_load(const secp256k1_context* ctx, secp256k1_scalar* r, secp256k1_scalar* s, int* recid, const secp256k1_ecdsa_recoverable_signature* sig) { + (void)ctx; + if (sizeof(secp256k1_scalar) == 32) { + /* When the secp256k1_scalar type is exactly 32 byte, use its + * representation inside secp256k1_ecdsa_signature, as conversion is very fast. + * Note that secp256k1_ecdsa_signature_save must use the same representation. */ + memcpy(r, &sig->data[0], 32); + memcpy(s, &sig->data[32], 32); + } else { + secp256k1_scalar_set_b32(r, &sig->data[0], NULL); + secp256k1_scalar_set_b32(s, &sig->data[32], NULL); + } + *recid = sig->data[64]; +} + +static void secp256k1_ecdsa_recoverable_signature_save(secp256k1_ecdsa_recoverable_signature* sig, const secp256k1_scalar* r, const secp256k1_scalar* s, int recid) { + if (sizeof(secp256k1_scalar) == 32) { + memcpy(&sig->data[0], r, 32); + memcpy(&sig->data[32], s, 32); + } else { + secp256k1_scalar_get_b32(&sig->data[0], r); + secp256k1_scalar_get_b32(&sig->data[32], s); + } + sig->data[64] = recid; +} + +int secp256k1_ecdsa_recoverable_signature_parse_compact(const secp256k1_context* ctx, secp256k1_ecdsa_recoverable_signature* sig, const unsigned char *input64, int recid) { + secp256k1_scalar r, s; + int ret = 1; + int overflow = 0; + + (void)ctx; + ARG_CHECK(sig != NULL); + ARG_CHECK(input64 != NULL); + ARG_CHECK(recid >= 0 && recid <= 3); + + secp256k1_scalar_set_b32(&r, &input64[0], &overflow); + ret &= !overflow; + secp256k1_scalar_set_b32(&s, &input64[32], &overflow); + ret &= !overflow; + if (ret) { + secp256k1_ecdsa_recoverable_signature_save(sig, &r, &s, recid); + } else { + memset(sig, 0, sizeof(*sig)); + } + return ret; +} + +int secp256k1_ecdsa_recoverable_signature_serialize_compact(const secp256k1_context* ctx, unsigned char *output64, int *recid, const secp256k1_ecdsa_recoverable_signature* sig) { + secp256k1_scalar r, s; + + (void)ctx; + ARG_CHECK(output64 != NULL); + ARG_CHECK(sig != NULL); + ARG_CHECK(recid != NULL); + + secp256k1_ecdsa_recoverable_signature_load(ctx, &r, &s, recid, sig); + secp256k1_scalar_get_b32(&output64[0], &r); + secp256k1_scalar_get_b32(&output64[32], &s); + return 1; +} + +int secp256k1_ecdsa_recoverable_signature_convert(const secp256k1_context* ctx, secp256k1_ecdsa_signature* sig, const secp256k1_ecdsa_recoverable_signature* sigin) { + secp256k1_scalar r, s; + int recid; + + (void)ctx; + ARG_CHECK(sig != NULL); + ARG_CHECK(sigin != NULL); + + secp256k1_ecdsa_recoverable_signature_load(ctx, &r, &s, &recid, sigin); + secp256k1_ecdsa_signature_save(sig, &r, &s); + return 1; +} + +static int secp256k1_ecdsa_sig_recover(const secp256k1_ecmult_context *ctx, const secp256k1_scalar *sigr, const secp256k1_scalar* sigs, secp256k1_ge *pubkey, const secp256k1_scalar *message, int recid) { + unsigned char brx[32]; + secp256k1_fe fx; + secp256k1_ge x; + secp256k1_gej xj; + secp256k1_scalar rn, u1, u2; + secp256k1_gej qj; + int r; + + if (secp256k1_scalar_is_zero(sigr) || secp256k1_scalar_is_zero(sigs)) { + return 0; + } + + secp256k1_scalar_get_b32(brx, sigr); + r = secp256k1_fe_set_b32(&fx, brx); + (void)r; + VERIFY_CHECK(r); /* brx comes from a scalar, so is less than the order; certainly less than p */ + if (recid & 2) { + if (secp256k1_fe_cmp_var(&fx, &secp256k1_ecdsa_const_p_minus_order) >= 0) { + return 0; + } + secp256k1_fe_add(&fx, &secp256k1_ecdsa_const_order_as_fe); + } + if (!secp256k1_ge_set_xo_var(&x, &fx, recid & 1)) { + return 0; + } + secp256k1_gej_set_ge(&xj, &x); + secp256k1_scalar_inverse_var(&rn, sigr); + secp256k1_scalar_mul(&u1, &rn, message); + secp256k1_scalar_negate(&u1, &u1); + secp256k1_scalar_mul(&u2, &rn, sigs); + secp256k1_ecmult(ctx, &qj, &xj, &u2, &u1); + secp256k1_ge_set_gej_var(pubkey, &qj); + return !secp256k1_gej_is_infinity(&qj); +} + +int secp256k1_ecdsa_sign_recoverable(const secp256k1_context* ctx, secp256k1_ecdsa_recoverable_signature *signature, const unsigned char *msg32, const unsigned char *seckey, secp256k1_nonce_function noncefp, const void* noncedata) { + secp256k1_scalar r, s; + secp256k1_scalar sec, non, msg; + int recid; + int ret = 0; + int overflow = 0; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_gen_context_is_built(&ctx->ecmult_gen_ctx)); + ARG_CHECK(msg32 != NULL); + ARG_CHECK(signature != NULL); + ARG_CHECK(seckey != NULL); + if (noncefp == NULL) { + noncefp = secp256k1_nonce_function_default; + } + + secp256k1_scalar_set_b32(&sec, seckey, &overflow); + /* Fail if the secret key is invalid. */ + if (!overflow && !secp256k1_scalar_is_zero(&sec)) { + unsigned int count = 0; + secp256k1_scalar_set_b32(&msg, msg32, NULL); + while (1) { + unsigned char nonce32[32]; + ret = noncefp(nonce32, msg32, seckey, NULL, (void*)noncedata, count); + if (!ret) { + break; + } + secp256k1_scalar_set_b32(&non, nonce32, &overflow); + memset(nonce32, 0, 32); + if (!secp256k1_scalar_is_zero(&non) && !overflow) { + if (secp256k1_ecdsa_sig_sign(&ctx->ecmult_gen_ctx, &r, &s, &sec, &msg, &non, &recid)) { + break; + } + } + count++; + } + secp256k1_scalar_clear(&msg); + secp256k1_scalar_clear(&non); + secp256k1_scalar_clear(&sec); + } + if (ret) { + secp256k1_ecdsa_recoverable_signature_save(signature, &r, &s, recid); + } else { + memset(signature, 0, sizeof(*signature)); + } + return ret; +} + +int secp256k1_ecdsa_recover(const secp256k1_context* ctx, secp256k1_pubkey *pubkey, const secp256k1_ecdsa_recoverable_signature *signature, const unsigned char *msg32) { + secp256k1_ge q; + secp256k1_scalar r, s; + secp256k1_scalar m; + int recid; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_context_is_built(&ctx->ecmult_ctx)); + ARG_CHECK(msg32 != NULL); + ARG_CHECK(signature != NULL); + ARG_CHECK(pubkey != NULL); + + secp256k1_ecdsa_recoverable_signature_load(ctx, &r, &s, &recid, signature); + ARG_CHECK(recid >= 0 && recid < 4); + secp256k1_scalar_set_b32(&m, msg32, NULL); + if (secp256k1_ecdsa_sig_recover(&ctx->ecmult_ctx, &r, &s, &q, &m, recid)) { + secp256k1_pubkey_save(pubkey, &q); + return 1; + } else { + memset(pubkey, 0, sizeof(*pubkey)); + return 0; + } +} + +#endif diff --git a/src/secp256k1/src/modules/recovery/tests_impl.h b/src/secp256k1/src/modules/recovery/tests_impl.h new file mode 100644 index 00000000..8932d5f0 --- /dev/null +++ b/src/secp256k1/src/modules/recovery/tests_impl.h @@ -0,0 +1,250 @@ +/********************************************************************** + * Copyright (c) 2013-2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_MODULE_RECOVERY_TESTS_ +#define _SECP256K1_MODULE_RECOVERY_TESTS_ + +void test_ecdsa_recovery_end_to_end(void) { + unsigned char extra[32] = {0x00}; + unsigned char privkey[32]; + unsigned char message[32]; + secp256k1_ecdsa_signature signature[5]; + secp256k1_ecdsa_recoverable_signature rsignature[5]; + unsigned char sig[74]; + secp256k1_pubkey pubkey; + secp256k1_pubkey recpubkey; + int recid = 0; + + /* Generate a random key and message. */ + { + secp256k1_scalar msg, key; + random_scalar_order_test(&msg); + random_scalar_order_test(&key); + secp256k1_scalar_get_b32(privkey, &key); + secp256k1_scalar_get_b32(message, &msg); + } + + /* Construct and verify corresponding public key. */ + CHECK(secp256k1_ec_seckey_verify(ctx, privkey) == 1); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, privkey) == 1); + + /* Serialize/parse compact and verify/recover. */ + extra[0] = 0; + CHECK(secp256k1_ecdsa_sign_recoverable(ctx, &rsignature[0], message, privkey, NULL, NULL) == 1); + CHECK(secp256k1_ecdsa_sign(ctx, &signature[0], message, privkey, NULL, NULL) == 1); + CHECK(secp256k1_ecdsa_sign_recoverable(ctx, &rsignature[4], message, privkey, NULL, NULL) == 1); + CHECK(secp256k1_ecdsa_sign_recoverable(ctx, &rsignature[1], message, privkey, NULL, extra) == 1); + extra[31] = 1; + CHECK(secp256k1_ecdsa_sign_recoverable(ctx, &rsignature[2], message, privkey, NULL, extra) == 1); + extra[31] = 0; + extra[0] = 1; + CHECK(secp256k1_ecdsa_sign_recoverable(ctx, &rsignature[3], message, privkey, NULL, extra) == 1); + CHECK(secp256k1_ecdsa_recoverable_signature_serialize_compact(ctx, sig, &recid, &rsignature[4]) == 1); + CHECK(secp256k1_ecdsa_recoverable_signature_convert(ctx, &signature[4], &rsignature[4]) == 1); + CHECK(memcmp(&signature[4], &signature[0], 64) == 0); + CHECK(secp256k1_ecdsa_verify(ctx, &signature[4], message, &pubkey) == 1); + memset(&rsignature[4], 0, sizeof(rsignature[4])); + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsignature[4], sig, recid) == 1); + CHECK(secp256k1_ecdsa_recoverable_signature_convert(ctx, &signature[4], &rsignature[4]) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &signature[4], message, &pubkey) == 1); + /* Parse compact (with recovery id) and recover. */ + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsignature[4], sig, recid) == 1); + CHECK(secp256k1_ecdsa_recover(ctx, &recpubkey, &rsignature[4], message) == 1); + CHECK(memcmp(&pubkey, &recpubkey, sizeof(pubkey)) == 0); + /* Serialize/destroy/parse signature and verify again. */ + CHECK(secp256k1_ecdsa_recoverable_signature_serialize_compact(ctx, sig, &recid, &rsignature[4]) == 1); + sig[secp256k1_rand_bits(6)] += 1 + secp256k1_rand_int(255); + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsignature[4], sig, recid) == 1); + CHECK(secp256k1_ecdsa_recoverable_signature_convert(ctx, &signature[4], &rsignature[4]) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &signature[4], message, &pubkey) == 0); + /* Recover again */ + CHECK(secp256k1_ecdsa_recover(ctx, &recpubkey, &rsignature[4], message) == 0 || + memcmp(&pubkey, &recpubkey, sizeof(pubkey)) != 0); +} + +/* Tests several edge cases. */ +void test_ecdsa_recovery_edge_cases(void) { + const unsigned char msg32[32] = { + 'T', 'h', 'i', 's', ' ', 'i', 's', ' ', + 'a', ' ', 'v', 'e', 'r', 'y', ' ', 's', + 'e', 'c', 'r', 'e', 't', ' ', 'm', 'e', + 's', 's', 'a', 'g', 'e', '.', '.', '.' + }; + const unsigned char sig64[64] = { + /* Generated by signing the above message with nonce 'This is the nonce we will use...' + * and secret key 0 (which is not valid), resulting in recid 0. */ + 0x67, 0xCB, 0x28, 0x5F, 0x9C, 0xD1, 0x94, 0xE8, + 0x40, 0xD6, 0x29, 0x39, 0x7A, 0xF5, 0x56, 0x96, + 0x62, 0xFD, 0xE4, 0x46, 0x49, 0x99, 0x59, 0x63, + 0x17, 0x9A, 0x7D, 0xD1, 0x7B, 0xD2, 0x35, 0x32, + 0x4B, 0x1B, 0x7D, 0xF3, 0x4C, 0xE1, 0xF6, 0x8E, + 0x69, 0x4F, 0xF6, 0xF1, 0x1A, 0xC7, 0x51, 0xDD, + 0x7D, 0xD7, 0x3E, 0x38, 0x7E, 0xE4, 0xFC, 0x86, + 0x6E, 0x1B, 0xE8, 0xEC, 0xC7, 0xDD, 0x95, 0x57 + }; + secp256k1_pubkey pubkey; + /* signature (r,s) = (4,4), which can be recovered with all 4 recids. */ + const unsigned char sigb64[64] = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, + }; + secp256k1_pubkey pubkeyb; + secp256k1_ecdsa_recoverable_signature rsig; + secp256k1_ecdsa_signature sig; + int recid; + + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsig, sig64, 0)); + CHECK(!secp256k1_ecdsa_recover(ctx, &pubkey, &rsig, msg32)); + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsig, sig64, 1)); + CHECK(secp256k1_ecdsa_recover(ctx, &pubkey, &rsig, msg32)); + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsig, sig64, 2)); + CHECK(!secp256k1_ecdsa_recover(ctx, &pubkey, &rsig, msg32)); + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsig, sig64, 3)); + CHECK(!secp256k1_ecdsa_recover(ctx, &pubkey, &rsig, msg32)); + + for (recid = 0; recid < 4; recid++) { + int i; + int recid2; + /* (4,4) encoded in DER. */ + unsigned char sigbder[8] = {0x30, 0x06, 0x02, 0x01, 0x04, 0x02, 0x01, 0x04}; + unsigned char sigcder_zr[7] = {0x30, 0x05, 0x02, 0x00, 0x02, 0x01, 0x01}; + unsigned char sigcder_zs[7] = {0x30, 0x05, 0x02, 0x01, 0x01, 0x02, 0x00}; + unsigned char sigbderalt1[39] = { + 0x30, 0x25, 0x02, 0x20, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x04, 0x02, 0x01, 0x04, + }; + unsigned char sigbderalt2[39] = { + 0x30, 0x25, 0x02, 0x01, 0x04, 0x02, 0x20, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, + }; + unsigned char sigbderalt3[40] = { + 0x30, 0x26, 0x02, 0x21, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x04, 0x02, 0x01, 0x04, + }; + unsigned char sigbderalt4[40] = { + 0x30, 0x26, 0x02, 0x01, 0x04, 0x02, 0x21, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, + }; + /* (order + r,4) encoded in DER. */ + unsigned char sigbderlong[40] = { + 0x30, 0x26, 0x02, 0x21, 0x00, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xFE, 0xBA, 0xAE, 0xDC, + 0xE6, 0xAF, 0x48, 0xA0, 0x3B, 0xBF, 0xD2, 0x5E, + 0x8C, 0xD0, 0x36, 0x41, 0x45, 0x02, 0x01, 0x04 + }; + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsig, sigb64, recid) == 1); + CHECK(secp256k1_ecdsa_recover(ctx, &pubkeyb, &rsig, msg32) == 1); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbder, sizeof(sigbder)) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &sig, msg32, &pubkeyb) == 1); + for (recid2 = 0; recid2 < 4; recid2++) { + secp256k1_pubkey pubkey2b; + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsig, sigb64, recid2) == 1); + CHECK(secp256k1_ecdsa_recover(ctx, &pubkey2b, &rsig, msg32) == 1); + /* Verifying with (order + r,4) should always fail. */ + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbderlong, sizeof(sigbderlong)) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &sig, msg32, &pubkeyb) == 0); + } + /* DER parsing tests. */ + /* Zero length r/s. */ + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigcder_zr, sizeof(sigcder_zr)) == 0); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigcder_zs, sizeof(sigcder_zs)) == 0); + /* Leading zeros. */ + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbderalt1, sizeof(sigbderalt1)) == 0); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbderalt2, sizeof(sigbderalt2)) == 0); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbderalt3, sizeof(sigbderalt3)) == 0); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbderalt4, sizeof(sigbderalt4)) == 0); + sigbderalt3[4] = 1; + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbderalt3, sizeof(sigbderalt3)) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &sig, msg32, &pubkeyb) == 0); + sigbderalt4[7] = 1; + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbderalt4, sizeof(sigbderalt4)) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &sig, msg32, &pubkeyb) == 0); + /* Damage signature. */ + sigbder[7]++; + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbder, sizeof(sigbder)) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &sig, msg32, &pubkeyb) == 0); + sigbder[7]--; + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbder, 6) == 0); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbder, sizeof(sigbder) - 1) == 0); + for(i = 0; i < 8; i++) { + int c; + unsigned char orig = sigbder[i]; + /*Try every single-byte change.*/ + for (c = 0; c < 256; c++) { + if (c == orig ) { + continue; + } + sigbder[i] = c; + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigbder, sizeof(sigbder)) == 0 || secp256k1_ecdsa_verify(ctx, &sig, msg32, &pubkeyb) == 0); + } + sigbder[i] = orig; + } + } + + /* Test r/s equal to zero */ + { + /* (1,1) encoded in DER. */ + unsigned char sigcder[8] = {0x30, 0x06, 0x02, 0x01, 0x01, 0x02, 0x01, 0x01}; + unsigned char sigc64[64] = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + }; + secp256k1_pubkey pubkeyc; + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsig, sigc64, 0) == 1); + CHECK(secp256k1_ecdsa_recover(ctx, &pubkeyc, &rsig, msg32) == 1); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigcder, sizeof(sigcder)) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &sig, msg32, &pubkeyc) == 1); + sigcder[4] = 0; + sigc64[31] = 0; + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsig, sigc64, 0) == 1); + CHECK(secp256k1_ecdsa_recover(ctx, &pubkeyb, &rsig, msg32) == 0); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigcder, sizeof(sigcder)) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &sig, msg32, &pubkeyc) == 0); + sigcder[4] = 1; + sigcder[7] = 0; + sigc64[31] = 1; + sigc64[63] = 0; + CHECK(secp256k1_ecdsa_recoverable_signature_parse_compact(ctx, &rsig, sigc64, 0) == 1); + CHECK(secp256k1_ecdsa_recover(ctx, &pubkeyb, &rsig, msg32) == 0); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, sigcder, sizeof(sigcder)) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &sig, msg32, &pubkeyc) == 0); + } +} + +void run_recovery_tests(void) { + int i; + for (i = 0; i < 64*count; i++) { + test_ecdsa_recovery_end_to_end(); + } + test_ecdsa_recovery_edge_cases(); +} + +#endif diff --git a/src/secp256k1/src/modules/schnorr/Makefile.am.include b/src/secp256k1/src/modules/schnorr/Makefile.am.include new file mode 100644 index 00000000..b3bfa7d5 --- /dev/null +++ b/src/secp256k1/src/modules/schnorr/Makefile.am.include @@ -0,0 +1,10 @@ +include_HEADERS += include/secp256k1_schnorr.h +noinst_HEADERS += src/modules/schnorr/main_impl.h +noinst_HEADERS += src/modules/schnorr/schnorr.h +noinst_HEADERS += src/modules/schnorr/schnorr_impl.h +noinst_HEADERS += src/modules/schnorr/tests_impl.h +if USE_BENCHMARK +noinst_PROGRAMS += bench_schnorr_verify +bench_schnorr_verify_SOURCES = src/bench_schnorr_verify.c +bench_schnorr_verify_LDADD = libsecp256k1.la $(SECP_LIBS) +endif diff --git a/src/secp256k1/src/modules/schnorr/main_impl.h b/src/secp256k1/src/modules/schnorr/main_impl.h new file mode 100644 index 00000000..fa176a17 --- /dev/null +++ b/src/secp256k1/src/modules/schnorr/main_impl.h @@ -0,0 +1,164 @@ +/********************************************************************** + * Copyright (c) 2014-2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef SECP256K1_MODULE_SCHNORR_MAIN +#define SECP256K1_MODULE_SCHNORR_MAIN + +#include "include/secp256k1_schnorr.h" +#include "modules/schnorr/schnorr_impl.h" + +static void secp256k1_schnorr_msghash_sha256(unsigned char *h32, const unsigned char *r32, const unsigned char *msg32) { + secp256k1_sha256_t sha; + secp256k1_sha256_initialize(&sha); + secp256k1_sha256_write(&sha, r32, 32); + secp256k1_sha256_write(&sha, msg32, 32); + secp256k1_sha256_finalize(&sha, h32); +} + +static const unsigned char secp256k1_schnorr_algo16[17] = "Schnorr+SHA256 "; + +int secp256k1_schnorr_sign(const secp256k1_context* ctx, unsigned char *sig64, const unsigned char *msg32, const unsigned char *seckey, secp256k1_nonce_function noncefp, const void* noncedata) { + secp256k1_scalar sec, non; + int ret = 0; + int overflow = 0; + unsigned int count = 0; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_gen_context_is_built(&ctx->ecmult_gen_ctx)); + ARG_CHECK(msg32 != NULL); + ARG_CHECK(sig64 != NULL); + ARG_CHECK(seckey != NULL); + if (noncefp == NULL) { + noncefp = secp256k1_nonce_function_default; + } + + secp256k1_scalar_set_b32(&sec, seckey, NULL); + while (1) { + unsigned char nonce32[32]; + ret = noncefp(nonce32, msg32, seckey, secp256k1_schnorr_algo16, (void*)noncedata, count); + if (!ret) { + break; + } + secp256k1_scalar_set_b32(&non, nonce32, &overflow); + memset(nonce32, 0, 32); + if (!secp256k1_scalar_is_zero(&non) && !overflow) { + if (secp256k1_schnorr_sig_sign(&ctx->ecmult_gen_ctx, sig64, &sec, &non, NULL, secp256k1_schnorr_msghash_sha256, msg32)) { + break; + } + } + count++; + } + if (!ret) { + memset(sig64, 0, 64); + } + secp256k1_scalar_clear(&non); + secp256k1_scalar_clear(&sec); + return ret; +} + +int secp256k1_schnorr_verify(const secp256k1_context* ctx, const unsigned char *sig64, const unsigned char *msg32, const secp256k1_pubkey *pubkey) { + secp256k1_ge q; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_context_is_built(&ctx->ecmult_ctx)); + ARG_CHECK(msg32 != NULL); + ARG_CHECK(sig64 != NULL); + ARG_CHECK(pubkey != NULL); + + secp256k1_pubkey_load(ctx, &q, pubkey); + return secp256k1_schnorr_sig_verify(&ctx->ecmult_ctx, sig64, &q, secp256k1_schnorr_msghash_sha256, msg32); +} + +int secp256k1_schnorr_recover(const secp256k1_context* ctx, secp256k1_pubkey *pubkey, const unsigned char *sig64, const unsigned char *msg32) { + secp256k1_ge q; + + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_context_is_built(&ctx->ecmult_ctx)); + ARG_CHECK(msg32 != NULL); + ARG_CHECK(sig64 != NULL); + ARG_CHECK(pubkey != NULL); + + if (secp256k1_schnorr_sig_recover(&ctx->ecmult_ctx, sig64, &q, secp256k1_schnorr_msghash_sha256, msg32)) { + secp256k1_pubkey_save(pubkey, &q); + return 1; + } else { + memset(pubkey, 0, sizeof(*pubkey)); + return 0; + } +} + +int secp256k1_schnorr_generate_nonce_pair(const secp256k1_context* ctx, secp256k1_pubkey *pubnonce, unsigned char *privnonce32, const unsigned char *sec32, const unsigned char *msg32, secp256k1_nonce_function noncefp, const void* noncedata) { + int count = 0; + int ret = 1; + secp256k1_gej Qj; + secp256k1_ge Q; + secp256k1_scalar sec; + + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_gen_context_is_built(&ctx->ecmult_gen_ctx)); + ARG_CHECK(msg32 != NULL); + ARG_CHECK(sec32 != NULL); + ARG_CHECK(pubnonce != NULL); + ARG_CHECK(privnonce32 != NULL); + + if (noncefp == NULL) { + noncefp = secp256k1_nonce_function_default; + } + + do { + int overflow; + ret = noncefp(privnonce32, sec32, msg32, secp256k1_schnorr_algo16, (void*)noncedata, count++); + if (!ret) { + break; + } + secp256k1_scalar_set_b32(&sec, privnonce32, &overflow); + if (overflow || secp256k1_scalar_is_zero(&sec)) { + continue; + } + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &Qj, &sec); + secp256k1_ge_set_gej(&Q, &Qj); + + secp256k1_pubkey_save(pubnonce, &Q); + break; + } while(1); + + secp256k1_scalar_clear(&sec); + if (!ret) { + memset(pubnonce, 0, sizeof(*pubnonce)); + } + return ret; +} + +int secp256k1_schnorr_partial_sign(const secp256k1_context* ctx, unsigned char *sig64, const unsigned char *msg32, const unsigned char *sec32, const secp256k1_pubkey *pubnonce_others, const unsigned char *secnonce32) { + int overflow = 0; + secp256k1_scalar sec, non; + secp256k1_ge pubnon; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_gen_context_is_built(&ctx->ecmult_gen_ctx)); + ARG_CHECK(msg32 != NULL); + ARG_CHECK(sig64 != NULL); + ARG_CHECK(sec32 != NULL); + ARG_CHECK(secnonce32 != NULL); + ARG_CHECK(pubnonce_others != NULL); + + secp256k1_scalar_set_b32(&sec, sec32, &overflow); + if (overflow || secp256k1_scalar_is_zero(&sec)) { + return -1; + } + secp256k1_scalar_set_b32(&non, secnonce32, &overflow); + if (overflow || secp256k1_scalar_is_zero(&non)) { + return -1; + } + secp256k1_pubkey_load(ctx, &pubnon, pubnonce_others); + return secp256k1_schnorr_sig_sign(&ctx->ecmult_gen_ctx, sig64, &sec, &non, &pubnon, secp256k1_schnorr_msghash_sha256, msg32); +} + +int secp256k1_schnorr_partial_combine(const secp256k1_context* ctx, unsigned char *sig64, const unsigned char * const *sig64sin, size_t n) { + ARG_CHECK(sig64 != NULL); + ARG_CHECK(n >= 1); + ARG_CHECK(sig64sin != NULL); + return secp256k1_schnorr_sig_combine(sig64, n, sig64sin); +} + +#endif diff --git a/src/secp256k1/src/modules/schnorr/schnorr.h b/src/secp256k1/src/modules/schnorr/schnorr.h new file mode 100644 index 00000000..de18147b --- /dev/null +++ b/src/secp256k1/src/modules/schnorr/schnorr.h @@ -0,0 +1,20 @@ +/*********************************************************************** + * Copyright (c) 2014-2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php. * + ***********************************************************************/ + +#ifndef _SECP256K1_MODULE_SCHNORR_H_ +#define _SECP256K1_MODULE_SCHNORR_H_ + +#include "scalar.h" +#include "group.h" + +typedef void (*secp256k1_schnorr_msghash)(unsigned char *h32, const unsigned char *r32, const unsigned char *msg32); + +static int secp256k1_schnorr_sig_sign(const secp256k1_ecmult_gen_context* ctx, unsigned char *sig64, const secp256k1_scalar *key, const secp256k1_scalar *nonce, const secp256k1_ge *pubnonce, secp256k1_schnorr_msghash hash, const unsigned char *msg32); +static int secp256k1_schnorr_sig_verify(const secp256k1_ecmult_context* ctx, const unsigned char *sig64, const secp256k1_ge *pubkey, secp256k1_schnorr_msghash hash, const unsigned char *msg32); +static int secp256k1_schnorr_sig_recover(const secp256k1_ecmult_context* ctx, const unsigned char *sig64, secp256k1_ge *pubkey, secp256k1_schnorr_msghash hash, const unsigned char *msg32); +static int secp256k1_schnorr_sig_combine(unsigned char *sig64, size_t n, const unsigned char * const *sig64ins); + +#endif diff --git a/src/secp256k1/src/modules/schnorr/schnorr_impl.h b/src/secp256k1/src/modules/schnorr/schnorr_impl.h new file mode 100644 index 00000000..e13ab6db --- /dev/null +++ b/src/secp256k1/src/modules/schnorr/schnorr_impl.h @@ -0,0 +1,207 @@ +/*********************************************************************** + * Copyright (c) 2014-2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php. * + ***********************************************************************/ + +#ifndef _SECP256K1_SCHNORR_IMPL_H_ +#define _SECP256K1_SCHNORR_IMPL_H_ + +#include + +#include "schnorr.h" +#include "num.h" +#include "field.h" +#include "group.h" +#include "ecmult.h" +#include "ecmult_gen.h" + +/** + * Custom Schnorr-based signature scheme. They support multiparty signing, public key + * recovery and batch validation. + * + * Rationale for verifying R's y coordinate: + * In order to support batch validation and public key recovery, the full R point must + * be known to verifiers, rather than just its x coordinate. In order to not risk + * being more strict in batch validation than normal validation, validators must be + * required to reject signatures with incorrect y coordinate. This is only possible + * by including a (relatively slow) field inverse, or a field square root. However, + * batch validation offers potentially much higher benefits than this cost. + * + * Rationale for having an implicit y coordinate oddness: + * If we commit to having the full R point known to verifiers, there are two mechanism. + * Either include its oddness in the signature, or give it an implicit fixed value. + * As the R y coordinate can be flipped by a simple negation of the nonce, we choose the + * latter, as it comes with nearly zero impact on signing or validation performance, and + * saves a byte in the signature. + * + * Signing: + * Inputs: 32-byte message m, 32-byte scalar key x (!=0), 32-byte scalar nonce k (!=0) + * + * Compute point R = k * G. Reject nonce if R's y coordinate is odd (or negate nonce). + * Compute 32-byte r, the serialization of R's x coordinate. + * Compute scalar h = Hash(r || m). Reject nonce if h == 0 or h >= order. + * Compute scalar s = k - h * x. + * The signature is (r, s). + * + * + * Verification: + * Inputs: 32-byte message m, public key point Q, signature: (32-byte r, scalar s) + * + * Signature is invalid if s >= order. + * Signature is invalid if r >= p. + * Compute scalar h = Hash(r || m). Signature is invalid if h == 0 or h >= order. + * Option 1 (faster for single verification): + * Compute point R = h * Q + s * G. Signature is invalid if R is infinity or R's y coordinate is odd. + * Signature is valid if the serialization of R's x coordinate equals r. + * Option 2 (allows batch validation and pubkey recovery): + * Decompress x coordinate r into point R, with odd y coordinate. Fail if R is not on the curve. + * Signature is valid if R + h * Q + s * G == 0. + */ + +static int secp256k1_schnorr_sig_sign(const secp256k1_ecmult_gen_context* ctx, unsigned char *sig64, const secp256k1_scalar *key, const secp256k1_scalar *nonce, const secp256k1_ge *pubnonce, secp256k1_schnorr_msghash hash, const unsigned char *msg32) { + secp256k1_gej Rj; + secp256k1_ge Ra; + unsigned char h32[32]; + secp256k1_scalar h, s; + int overflow; + secp256k1_scalar n; + + if (secp256k1_scalar_is_zero(key) || secp256k1_scalar_is_zero(nonce)) { + return 0; + } + n = *nonce; + + secp256k1_ecmult_gen(ctx, &Rj, &n); + if (pubnonce != NULL) { + secp256k1_gej_add_ge(&Rj, &Rj, pubnonce); + } + secp256k1_ge_set_gej(&Ra, &Rj); + secp256k1_fe_normalize(&Ra.y); + if (secp256k1_fe_is_odd(&Ra.y)) { + /* R's y coordinate is odd, which is not allowed (see rationale above). + Force it to be even by negating the nonce. Note that this even works + for multiparty signing, as the R point is known to all participants, + which can all decide to flip the sign in unison, resulting in the + overall R point to be negated too. */ + secp256k1_scalar_negate(&n, &n); + } + secp256k1_fe_normalize(&Ra.x); + secp256k1_fe_get_b32(sig64, &Ra.x); + hash(h32, sig64, msg32); + overflow = 0; + secp256k1_scalar_set_b32(&h, h32, &overflow); + if (overflow || secp256k1_scalar_is_zero(&h)) { + secp256k1_scalar_clear(&n); + return 0; + } + secp256k1_scalar_mul(&s, &h, key); + secp256k1_scalar_negate(&s, &s); + secp256k1_scalar_add(&s, &s, &n); + secp256k1_scalar_clear(&n); + secp256k1_scalar_get_b32(sig64 + 32, &s); + return 1; +} + +static int secp256k1_schnorr_sig_verify(const secp256k1_ecmult_context* ctx, const unsigned char *sig64, const secp256k1_ge *pubkey, secp256k1_schnorr_msghash hash, const unsigned char *msg32) { + secp256k1_gej Qj, Rj; + secp256k1_ge Ra; + secp256k1_fe Rx; + secp256k1_scalar h, s; + unsigned char hh[32]; + int overflow; + + if (secp256k1_ge_is_infinity(pubkey)) { + return 0; + } + hash(hh, sig64, msg32); + overflow = 0; + secp256k1_scalar_set_b32(&h, hh, &overflow); + if (overflow || secp256k1_scalar_is_zero(&h)) { + return 0; + } + overflow = 0; + secp256k1_scalar_set_b32(&s, sig64 + 32, &overflow); + if (overflow) { + return 0; + } + if (!secp256k1_fe_set_b32(&Rx, sig64)) { + return 0; + } + secp256k1_gej_set_ge(&Qj, pubkey); + secp256k1_ecmult(ctx, &Rj, &Qj, &h, &s); + if (secp256k1_gej_is_infinity(&Rj)) { + return 0; + } + secp256k1_ge_set_gej_var(&Ra, &Rj); + secp256k1_fe_normalize_var(&Ra.y); + if (secp256k1_fe_is_odd(&Ra.y)) { + return 0; + } + return secp256k1_fe_equal_var(&Rx, &Ra.x); +} + +static int secp256k1_schnorr_sig_recover(const secp256k1_ecmult_context* ctx, const unsigned char *sig64, secp256k1_ge *pubkey, secp256k1_schnorr_msghash hash, const unsigned char *msg32) { + secp256k1_gej Qj, Rj; + secp256k1_ge Ra; + secp256k1_fe Rx; + secp256k1_scalar h, s; + unsigned char hh[32]; + int overflow; + + hash(hh, sig64, msg32); + overflow = 0; + secp256k1_scalar_set_b32(&h, hh, &overflow); + if (overflow || secp256k1_scalar_is_zero(&h)) { + return 0; + } + overflow = 0; + secp256k1_scalar_set_b32(&s, sig64 + 32, &overflow); + if (overflow) { + return 0; + } + if (!secp256k1_fe_set_b32(&Rx, sig64)) { + return 0; + } + if (!secp256k1_ge_set_xo_var(&Ra, &Rx, 0)) { + return 0; + } + secp256k1_gej_set_ge(&Rj, &Ra); + secp256k1_scalar_inverse_var(&h, &h); + secp256k1_scalar_negate(&s, &s); + secp256k1_scalar_mul(&s, &s, &h); + secp256k1_ecmult(ctx, &Qj, &Rj, &h, &s); + if (secp256k1_gej_is_infinity(&Qj)) { + return 0; + } + secp256k1_ge_set_gej(pubkey, &Qj); + return 1; +} + +static int secp256k1_schnorr_sig_combine(unsigned char *sig64, size_t n, const unsigned char * const *sig64ins) { + secp256k1_scalar s = SECP256K1_SCALAR_CONST(0, 0, 0, 0, 0, 0, 0, 0); + size_t i; + for (i = 0; i < n; i++) { + secp256k1_scalar si; + int overflow; + secp256k1_scalar_set_b32(&si, sig64ins[i] + 32, &overflow); + if (overflow) { + return -1; + } + if (i) { + if (memcmp(sig64ins[i - 1], sig64ins[i], 32) != 0) { + return -1; + } + } + secp256k1_scalar_add(&s, &s, &si); + } + if (secp256k1_scalar_is_zero(&s)) { + return 0; + } + memcpy(sig64, sig64ins[0], 32); + secp256k1_scalar_get_b32(sig64 + 32, &s); + secp256k1_scalar_clear(&s); + return 1; +} + +#endif diff --git a/src/secp256k1/src/modules/schnorr/tests_impl.h b/src/secp256k1/src/modules/schnorr/tests_impl.h new file mode 100644 index 00000000..5bd14a03 --- /dev/null +++ b/src/secp256k1/src/modules/schnorr/tests_impl.h @@ -0,0 +1,175 @@ +/********************************************************************** + * Copyright (c) 2014-2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef SECP256K1_MODULE_SCHNORR_TESTS +#define SECP256K1_MODULE_SCHNORR_TESTS + +#include "include/secp256k1_schnorr.h" + +void test_schnorr_end_to_end(void) { + unsigned char privkey[32]; + unsigned char message[32]; + unsigned char schnorr_signature[64]; + secp256k1_pubkey pubkey, recpubkey; + + /* Generate a random key and message. */ + { + secp256k1_scalar key; + random_scalar_order_test(&key); + secp256k1_scalar_get_b32(privkey, &key); + secp256k1_rand256_test(message); + } + + /* Construct and verify corresponding public key. */ + CHECK(secp256k1_ec_seckey_verify(ctx, privkey) == 1); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, privkey) == 1); + + /* Schnorr sign. */ + CHECK(secp256k1_schnorr_sign(ctx, schnorr_signature, message, privkey, NULL, NULL) == 1); + CHECK(secp256k1_schnorr_verify(ctx, schnorr_signature, message, &pubkey) == 1); + CHECK(secp256k1_schnorr_recover(ctx, &recpubkey, schnorr_signature, message) == 1); + CHECK(memcmp(&pubkey, &recpubkey, sizeof(pubkey)) == 0); + /* Destroy signature and verify again. */ + schnorr_signature[secp256k1_rand_bits(6)] += 1 + secp256k1_rand_int(255); + CHECK(secp256k1_schnorr_verify(ctx, schnorr_signature, message, &pubkey) == 0); + CHECK(secp256k1_schnorr_recover(ctx, &recpubkey, schnorr_signature, message) != 1 || + memcmp(&pubkey, &recpubkey, sizeof(pubkey)) != 0); +} + +/** Horribly broken hash function. Do not use for anything but tests. */ +void test_schnorr_hash(unsigned char *h32, const unsigned char *r32, const unsigned char *msg32) { + int i; + for (i = 0; i < 32; i++) { + h32[i] = r32[i] ^ msg32[i]; + } +} + +void test_schnorr_sign_verify(void) { + unsigned char msg32[32]; + unsigned char sig64[3][64]; + secp256k1_gej pubkeyj[3]; + secp256k1_ge pubkey[3]; + secp256k1_scalar nonce[3], key[3]; + int i = 0; + int k; + + secp256k1_rand256_test(msg32); + + for (k = 0; k < 3; k++) { + random_scalar_order_test(&key[k]); + + do { + random_scalar_order_test(&nonce[k]); + if (secp256k1_schnorr_sig_sign(&ctx->ecmult_gen_ctx, sig64[k], &key[k], &nonce[k], NULL, &test_schnorr_hash, msg32)) { + break; + } + } while(1); + + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &pubkeyj[k], &key[k]); + secp256k1_ge_set_gej_var(&pubkey[k], &pubkeyj[k]); + CHECK(secp256k1_schnorr_sig_verify(&ctx->ecmult_ctx, sig64[k], &pubkey[k], &test_schnorr_hash, msg32)); + + for (i = 0; i < 4; i++) { + int pos = secp256k1_rand_bits(6); + int mod = 1 + secp256k1_rand_int(255); + sig64[k][pos] ^= mod; + CHECK(secp256k1_schnorr_sig_verify(&ctx->ecmult_ctx, sig64[k], &pubkey[k], &test_schnorr_hash, msg32) == 0); + sig64[k][pos] ^= mod; + } + } +} + +void test_schnorr_threshold(void) { + unsigned char msg[32]; + unsigned char sec[5][32]; + secp256k1_pubkey pub[5]; + unsigned char nonce[5][32]; + secp256k1_pubkey pubnonce[5]; + unsigned char sig[5][64]; + const unsigned char* sigs[5]; + unsigned char allsig[64]; + const secp256k1_pubkey* pubs[5]; + secp256k1_pubkey allpub; + int n, i; + int damage; + int ret = 0; + + damage = secp256k1_rand_bits(1) ? (1 + secp256k1_rand_int(4)) : 0; + secp256k1_rand256_test(msg); + n = 2 + secp256k1_rand_int(4); + for (i = 0; i < n; i++) { + do { + secp256k1_rand256_test(sec[i]); + } while (!secp256k1_ec_seckey_verify(ctx, sec[i])); + CHECK(secp256k1_ec_pubkey_create(ctx, &pub[i], sec[i])); + CHECK(secp256k1_schnorr_generate_nonce_pair(ctx, &pubnonce[i], nonce[i], msg, sec[i], NULL, NULL)); + pubs[i] = &pub[i]; + } + if (damage == 1) { + nonce[secp256k1_rand_int(n)][secp256k1_rand_int(32)] ^= 1 + secp256k1_rand_int(255); + } else if (damage == 2) { + sec[secp256k1_rand_int(n)][secp256k1_rand_int(32)] ^= 1 + secp256k1_rand_int(255); + } + for (i = 0; i < n; i++) { + secp256k1_pubkey allpubnonce; + const secp256k1_pubkey *pubnonces[4]; + int j; + for (j = 0; j < i; j++) { + pubnonces[j] = &pubnonce[j]; + } + for (j = i + 1; j < n; j++) { + pubnonces[j - 1] = &pubnonce[j]; + } + CHECK(secp256k1_ec_pubkey_combine(ctx, &allpubnonce, pubnonces, n - 1)); + ret |= (secp256k1_schnorr_partial_sign(ctx, sig[i], msg, sec[i], &allpubnonce, nonce[i]) != 1) * 1; + sigs[i] = sig[i]; + } + if (damage == 3) { + sig[secp256k1_rand_int(n)][secp256k1_rand_bits(6)] ^= 1 + secp256k1_rand_int(255); + } + ret |= (secp256k1_ec_pubkey_combine(ctx, &allpub, pubs, n) != 1) * 2; + if ((ret & 1) == 0) { + ret |= (secp256k1_schnorr_partial_combine(ctx, allsig, sigs, n) != 1) * 4; + } + if (damage == 4) { + allsig[secp256k1_rand_int(32)] ^= 1 + secp256k1_rand_int(255); + } + if ((ret & 7) == 0) { + ret |= (secp256k1_schnorr_verify(ctx, allsig, msg, &allpub) != 1) * 8; + } + CHECK((ret == 0) == (damage == 0)); +} + +void test_schnorr_recovery(void) { + unsigned char msg32[32]; + unsigned char sig64[64]; + secp256k1_ge Q; + + secp256k1_rand256_test(msg32); + secp256k1_rand256_test(sig64); + secp256k1_rand256_test(sig64 + 32); + if (secp256k1_schnorr_sig_recover(&ctx->ecmult_ctx, sig64, &Q, &test_schnorr_hash, msg32) == 1) { + CHECK(secp256k1_schnorr_sig_verify(&ctx->ecmult_ctx, sig64, &Q, &test_schnorr_hash, msg32) == 1); + } +} + +void run_schnorr_tests(void) { + int i; + for (i = 0; i < 32*count; i++) { + test_schnorr_end_to_end(); + } + for (i = 0; i < 32 * count; i++) { + test_schnorr_sign_verify(); + } + for (i = 0; i < 16 * count; i++) { + test_schnorr_recovery(); + } + for (i = 0; i < 10 * count; i++) { + test_schnorr_threshold(); + } +} + +#endif diff --git a/src/secp256k1/src/num.h b/src/secp256k1/src/num.h new file mode 100644 index 00000000..ebfa71eb --- /dev/null +++ b/src/secp256k1/src/num.h @@ -0,0 +1,68 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_NUM_ +#define _SECP256K1_NUM_ + +#ifndef USE_NUM_NONE + +#if defined HAVE_CONFIG_H +#include "libsecp256k1-config.h" +#endif + +#if defined(USE_NUM_GMP) +#include "num_gmp.h" +#else +#error "Please select num implementation" +#endif + +/** Copy a number. */ +static void secp256k1_num_copy(secp256k1_num *r, const secp256k1_num *a); + +/** Convert a number's absolute value to a binary big-endian string. + * There must be enough place. */ +static void secp256k1_num_get_bin(unsigned char *r, unsigned int rlen, const secp256k1_num *a); + +/** Set a number to the value of a binary big-endian string. */ +static void secp256k1_num_set_bin(secp256k1_num *r, const unsigned char *a, unsigned int alen); + +/** Compute a modular inverse. The input must be less than the modulus. */ +static void secp256k1_num_mod_inverse(secp256k1_num *r, const secp256k1_num *a, const secp256k1_num *m); + +/** Compare the absolute value of two numbers. */ +static int secp256k1_num_cmp(const secp256k1_num *a, const secp256k1_num *b); + +/** Test whether two number are equal (including sign). */ +static int secp256k1_num_eq(const secp256k1_num *a, const secp256k1_num *b); + +/** Add two (signed) numbers. */ +static void secp256k1_num_add(secp256k1_num *r, const secp256k1_num *a, const secp256k1_num *b); + +/** Subtract two (signed) numbers. */ +static void secp256k1_num_sub(secp256k1_num *r, const secp256k1_num *a, const secp256k1_num *b); + +/** Multiply two (signed) numbers. */ +static void secp256k1_num_mul(secp256k1_num *r, const secp256k1_num *a, const secp256k1_num *b); + +/** Replace a number by its remainder modulo m. M's sign is ignored. The result is a number between 0 and m-1, + even if r was negative. */ +static void secp256k1_num_mod(secp256k1_num *r, const secp256k1_num *m); + +/** Right-shift the passed number by bits bits. */ +static void secp256k1_num_shift(secp256k1_num *r, int bits); + +/** Check whether a number is zero. */ +static int secp256k1_num_is_zero(const secp256k1_num *a); + +/** Check whether a number is strictly negative. */ +static int secp256k1_num_is_neg(const secp256k1_num *a); + +/** Change a number's sign. */ +static void secp256k1_num_negate(secp256k1_num *r); + +#endif + +#endif diff --git a/src/secp256k1/src/num_gmp.h b/src/secp256k1/src/num_gmp.h new file mode 100644 index 00000000..7dd81308 --- /dev/null +++ b/src/secp256k1/src/num_gmp.h @@ -0,0 +1,20 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_NUM_REPR_ +#define _SECP256K1_NUM_REPR_ + +#include + +#define NUM_LIMBS ((256+GMP_NUMB_BITS-1)/GMP_NUMB_BITS) + +typedef struct { + mp_limb_t data[2*NUM_LIMBS]; + int neg; + int limbs; +} secp256k1_num; + +#endif diff --git a/src/secp256k1/src/num_gmp_impl.h b/src/secp256k1/src/num_gmp_impl.h new file mode 100644 index 00000000..7b6a8971 --- /dev/null +++ b/src/secp256k1/src/num_gmp_impl.h @@ -0,0 +1,262 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_NUM_REPR_IMPL_H_ +#define _SECP256K1_NUM_REPR_IMPL_H_ + +#include +#include +#include + +#include "util.h" +#include "num.h" + +#ifdef VERIFY +static void secp256k1_num_sanity(const secp256k1_num *a) { + VERIFY_CHECK(a->limbs == 1 || (a->limbs > 1 && a->data[a->limbs-1] != 0)); +} +#else +#define secp256k1_num_sanity(a) do { } while(0) +#endif + +static void secp256k1_num_copy(secp256k1_num *r, const secp256k1_num *a) { + *r = *a; +} + +static void secp256k1_num_get_bin(unsigned char *r, unsigned int rlen, const secp256k1_num *a) { + unsigned char tmp[65]; + int len = 0; + int shift = 0; + if (a->limbs>1 || a->data[0] != 0) { + len = mpn_get_str(tmp, 256, (mp_limb_t*)a->data, a->limbs); + } + while (shift < len && tmp[shift] == 0) shift++; + VERIFY_CHECK(len-shift <= (int)rlen); + memset(r, 0, rlen - len + shift); + if (len > shift) { + memcpy(r + rlen - len + shift, tmp + shift, len - shift); + } + memset(tmp, 0, sizeof(tmp)); +} + +static void secp256k1_num_set_bin(secp256k1_num *r, const unsigned char *a, unsigned int alen) { + int len; + VERIFY_CHECK(alen > 0); + VERIFY_CHECK(alen <= 64); + len = mpn_set_str(r->data, a, alen, 256); + if (len == 0) { + r->data[0] = 0; + len = 1; + } + VERIFY_CHECK(len <= NUM_LIMBS*2); + r->limbs = len; + r->neg = 0; + while (r->limbs > 1 && r->data[r->limbs-1]==0) { + r->limbs--; + } +} + +static void secp256k1_num_add_abs(secp256k1_num *r, const secp256k1_num *a, const secp256k1_num *b) { + mp_limb_t c = mpn_add(r->data, a->data, a->limbs, b->data, b->limbs); + r->limbs = a->limbs; + if (c != 0) { + VERIFY_CHECK(r->limbs < 2*NUM_LIMBS); + r->data[r->limbs++] = c; + } +} + +static void secp256k1_num_sub_abs(secp256k1_num *r, const secp256k1_num *a, const secp256k1_num *b) { + mp_limb_t c = mpn_sub(r->data, a->data, a->limbs, b->data, b->limbs); + (void)c; + VERIFY_CHECK(c == 0); + r->limbs = a->limbs; + while (r->limbs > 1 && r->data[r->limbs-1]==0) { + r->limbs--; + } +} + +static void secp256k1_num_mod(secp256k1_num *r, const secp256k1_num *m) { + secp256k1_num_sanity(r); + secp256k1_num_sanity(m); + + if (r->limbs >= m->limbs) { + mp_limb_t t[2*NUM_LIMBS]; + mpn_tdiv_qr(t, r->data, 0, r->data, r->limbs, m->data, m->limbs); + memset(t, 0, sizeof(t)); + r->limbs = m->limbs; + while (r->limbs > 1 && r->data[r->limbs-1]==0) { + r->limbs--; + } + } + + if (r->neg && (r->limbs > 1 || r->data[0] != 0)) { + secp256k1_num_sub_abs(r, m, r); + r->neg = 0; + } +} + +static void secp256k1_num_mod_inverse(secp256k1_num *r, const secp256k1_num *a, const secp256k1_num *m) { + int i; + mp_limb_t g[NUM_LIMBS+1]; + mp_limb_t u[NUM_LIMBS+1]; + mp_limb_t v[NUM_LIMBS+1]; + mp_size_t sn; + mp_size_t gn; + secp256k1_num_sanity(a); + secp256k1_num_sanity(m); + + /** mpn_gcdext computes: (G,S) = gcdext(U,V), where + * * G = gcd(U,V) + * * G = U*S + V*T + * * U has equal or more limbs than V, and V has no padding + * If we set U to be (a padded version of) a, and V = m: + * G = a*S + m*T + * G = a*S mod m + * Assuming G=1: + * S = 1/a mod m + */ + VERIFY_CHECK(m->limbs <= NUM_LIMBS); + VERIFY_CHECK(m->data[m->limbs-1] != 0); + for (i = 0; i < m->limbs; i++) { + u[i] = (i < a->limbs) ? a->data[i] : 0; + v[i] = m->data[i]; + } + sn = NUM_LIMBS+1; + gn = mpn_gcdext(g, r->data, &sn, u, m->limbs, v, m->limbs); + (void)gn; + VERIFY_CHECK(gn == 1); + VERIFY_CHECK(g[0] == 1); + r->neg = a->neg ^ m->neg; + if (sn < 0) { + mpn_sub(r->data, m->data, m->limbs, r->data, -sn); + r->limbs = m->limbs; + while (r->limbs > 1 && r->data[r->limbs-1]==0) { + r->limbs--; + } + } else { + r->limbs = sn; + } + memset(g, 0, sizeof(g)); + memset(u, 0, sizeof(u)); + memset(v, 0, sizeof(v)); +} + +static int secp256k1_num_is_zero(const secp256k1_num *a) { + return (a->limbs == 1 && a->data[0] == 0); +} + +static int secp256k1_num_is_neg(const secp256k1_num *a) { + return (a->limbs > 1 || a->data[0] != 0) && a->neg; +} + +static int secp256k1_num_cmp(const secp256k1_num *a, const secp256k1_num *b) { + if (a->limbs > b->limbs) { + return 1; + } + if (a->limbs < b->limbs) { + return -1; + } + return mpn_cmp(a->data, b->data, a->limbs); +} + +static int secp256k1_num_eq(const secp256k1_num *a, const secp256k1_num *b) { + if (a->limbs > b->limbs) { + return 0; + } + if (a->limbs < b->limbs) { + return 0; + } + if ((a->neg && !secp256k1_num_is_zero(a)) != (b->neg && !secp256k1_num_is_zero(b))) { + return 0; + } + return mpn_cmp(a->data, b->data, a->limbs) == 0; +} + +static void secp256k1_num_subadd(secp256k1_num *r, const secp256k1_num *a, const secp256k1_num *b, int bneg) { + if (!(b->neg ^ bneg ^ a->neg)) { /* a and b have the same sign */ + r->neg = a->neg; + if (a->limbs >= b->limbs) { + secp256k1_num_add_abs(r, a, b); + } else { + secp256k1_num_add_abs(r, b, a); + } + } else { + if (secp256k1_num_cmp(a, b) > 0) { + r->neg = a->neg; + secp256k1_num_sub_abs(r, a, b); + } else { + r->neg = b->neg ^ bneg; + secp256k1_num_sub_abs(r, b, a); + } + } +} + +static void secp256k1_num_add(secp256k1_num *r, const secp256k1_num *a, const secp256k1_num *b) { + secp256k1_num_sanity(a); + secp256k1_num_sanity(b); + secp256k1_num_subadd(r, a, b, 0); +} + +static void secp256k1_num_sub(secp256k1_num *r, const secp256k1_num *a, const secp256k1_num *b) { + secp256k1_num_sanity(a); + secp256k1_num_sanity(b); + secp256k1_num_subadd(r, a, b, 1); +} + +static void secp256k1_num_mul(secp256k1_num *r, const secp256k1_num *a, const secp256k1_num *b) { + mp_limb_t tmp[2*NUM_LIMBS+1]; + secp256k1_num_sanity(a); + secp256k1_num_sanity(b); + + VERIFY_CHECK(a->limbs + b->limbs <= 2*NUM_LIMBS+1); + if ((a->limbs==1 && a->data[0]==0) || (b->limbs==1 && b->data[0]==0)) { + r->limbs = 1; + r->neg = 0; + r->data[0] = 0; + return; + } + if (a->limbs >= b->limbs) { + mpn_mul(tmp, a->data, a->limbs, b->data, b->limbs); + } else { + mpn_mul(tmp, b->data, b->limbs, a->data, a->limbs); + } + r->limbs = a->limbs + b->limbs; + if (r->limbs > 1 && tmp[r->limbs - 1]==0) { + r->limbs--; + } + VERIFY_CHECK(r->limbs <= 2*NUM_LIMBS); + mpn_copyi(r->data, tmp, r->limbs); + r->neg = a->neg ^ b->neg; + memset(tmp, 0, sizeof(tmp)); +} + +static void secp256k1_num_shift(secp256k1_num *r, int bits) { + if (bits % GMP_NUMB_BITS) { + /* Shift within limbs. */ + mpn_rshift(r->data, r->data, r->limbs, bits % GMP_NUMB_BITS); + } + if (bits >= GMP_NUMB_BITS) { + int i; + /* Shift full limbs. */ + for (i = 0; i < r->limbs; i++) { + int index = i + (bits / GMP_NUMB_BITS); + if (index < r->limbs && index < 2*NUM_LIMBS) { + r->data[i] = r->data[index]; + } else { + r->data[i] = 0; + } + } + } + while (r->limbs>1 && r->data[r->limbs-1]==0) { + r->limbs--; + } +} + +static void secp256k1_num_negate(secp256k1_num *r) { + r->neg ^= 1; +} + +#endif diff --git a/src/secp256k1/src/num_impl.h b/src/secp256k1/src/num_impl.h new file mode 100644 index 00000000..0b0e3a07 --- /dev/null +++ b/src/secp256k1/src/num_impl.h @@ -0,0 +1,24 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_NUM_IMPL_H_ +#define _SECP256K1_NUM_IMPL_H_ + +#if defined HAVE_CONFIG_H +#include "libsecp256k1-config.h" +#endif + +#include "num.h" + +#if defined(USE_NUM_GMP) +#include "num_gmp_impl.h" +#elif defined(USE_NUM_NONE) +/* Nothing. */ +#else +#error "Please select num implementation" +#endif + +#endif diff --git a/src/secp256k1/src/scalar.h b/src/secp256k1/src/scalar.h new file mode 100644 index 00000000..b590ccd6 --- /dev/null +++ b/src/secp256k1/src/scalar.h @@ -0,0 +1,104 @@ +/********************************************************************** + * Copyright (c) 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_SCALAR_ +#define _SECP256K1_SCALAR_ + +#include "num.h" + +#if defined HAVE_CONFIG_H +#include "libsecp256k1-config.h" +#endif + +#if defined(USE_SCALAR_4X64) +#include "scalar_4x64.h" +#elif defined(USE_SCALAR_8X32) +#include "scalar_8x32.h" +#else +#error "Please select scalar implementation" +#endif + +/** Clear a scalar to prevent the leak of sensitive data. */ +static void secp256k1_scalar_clear(secp256k1_scalar *r); + +/** Access bits from a scalar. All requested bits must belong to the same 32-bit limb. */ +static unsigned int secp256k1_scalar_get_bits(const secp256k1_scalar *a, unsigned int offset, unsigned int count); + +/** Access bits from a scalar. Not constant time. */ +static unsigned int secp256k1_scalar_get_bits_var(const secp256k1_scalar *a, unsigned int offset, unsigned int count); + +/** Set a scalar from a big endian byte array. */ +static void secp256k1_scalar_set_b32(secp256k1_scalar *r, const unsigned char *bin, int *overflow); + +/** Set a scalar to an unsigned integer. */ +static void secp256k1_scalar_set_int(secp256k1_scalar *r, unsigned int v); + +/** Convert a scalar to a byte array. */ +static void secp256k1_scalar_get_b32(unsigned char *bin, const secp256k1_scalar* a); + +/** Add two scalars together (modulo the group order). Returns whether it overflowed. */ +static int secp256k1_scalar_add(secp256k1_scalar *r, const secp256k1_scalar *a, const secp256k1_scalar *b); + +/** Conditionally add a power of two to a scalar. The result is not allowed to overflow. */ +static void secp256k1_scalar_cadd_bit(secp256k1_scalar *r, unsigned int bit, int flag); + +/** Multiply two scalars (modulo the group order). */ +static void secp256k1_scalar_mul(secp256k1_scalar *r, const secp256k1_scalar *a, const secp256k1_scalar *b); + +/** Shift a scalar right by some amount strictly between 0 and 16, returning + * the low bits that were shifted off */ +static int secp256k1_scalar_shr_int(secp256k1_scalar *r, int n); + +/** Compute the square of a scalar (modulo the group order). */ +static void secp256k1_scalar_sqr(secp256k1_scalar *r, const secp256k1_scalar *a); + +/** Compute the inverse of a scalar (modulo the group order). */ +static void secp256k1_scalar_inverse(secp256k1_scalar *r, const secp256k1_scalar *a); + +/** Compute the inverse of a scalar (modulo the group order), without constant-time guarantee. */ +static void secp256k1_scalar_inverse_var(secp256k1_scalar *r, const secp256k1_scalar *a); + +/** Compute the complement of a scalar (modulo the group order). */ +static void secp256k1_scalar_negate(secp256k1_scalar *r, const secp256k1_scalar *a); + +/** Check whether a scalar equals zero. */ +static int secp256k1_scalar_is_zero(const secp256k1_scalar *a); + +/** Check whether a scalar equals one. */ +static int secp256k1_scalar_is_one(const secp256k1_scalar *a); + +/** Check whether a scalar, considered as an nonnegative integer, is even. */ +static int secp256k1_scalar_is_even(const secp256k1_scalar *a); + +/** Check whether a scalar is higher than the group order divided by 2. */ +static int secp256k1_scalar_is_high(const secp256k1_scalar *a); + +/** Conditionally negate a number, in constant time. + * Returns -1 if the number was negated, 1 otherwise */ +static int secp256k1_scalar_cond_negate(secp256k1_scalar *a, int flag); + +#ifndef USE_NUM_NONE +/** Convert a scalar to a number. */ +static void secp256k1_scalar_get_num(secp256k1_num *r, const secp256k1_scalar *a); + +/** Get the order of the group as a number. */ +static void secp256k1_scalar_order_get_num(secp256k1_num *r); +#endif + +/** Compare two scalars. */ +static int secp256k1_scalar_eq(const secp256k1_scalar *a, const secp256k1_scalar *b); + +#ifdef USE_ENDOMORPHISM +/** Find r1 and r2 such that r1+r2*2^128 = a. */ +static void secp256k1_scalar_split_128(secp256k1_scalar *r1, secp256k1_scalar *r2, const secp256k1_scalar *a); +/** Find r1 and r2 such that r1+r2*lambda = a, and r1 and r2 are maximum 128 bits long (see secp256k1_gej_mul_lambda). */ +static void secp256k1_scalar_split_lambda(secp256k1_scalar *r1, secp256k1_scalar *r2, const secp256k1_scalar *a); +#endif + +/** Multiply a and b (without taking the modulus!), divide by 2**shift, and round to the nearest integer. Shift must be at least 256. */ +static void secp256k1_scalar_mul_shift_var(secp256k1_scalar *r, const secp256k1_scalar *a, const secp256k1_scalar *b, unsigned int shift); + +#endif diff --git a/src/secp256k1/src/scalar_4x64.h b/src/secp256k1/src/scalar_4x64.h new file mode 100644 index 00000000..cff40603 --- /dev/null +++ b/src/secp256k1/src/scalar_4x64.h @@ -0,0 +1,19 @@ +/********************************************************************** + * Copyright (c) 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_SCALAR_REPR_ +#define _SECP256K1_SCALAR_REPR_ + +#include + +/** A scalar modulo the group order of the secp256k1 curve. */ +typedef struct { + uint64_t d[4]; +} secp256k1_scalar; + +#define SECP256K1_SCALAR_CONST(d7, d6, d5, d4, d3, d2, d1, d0) {{((uint64_t)(d1)) << 32 | (d0), ((uint64_t)(d3)) << 32 | (d2), ((uint64_t)(d5)) << 32 | (d4), ((uint64_t)(d7)) << 32 | (d6)}} + +#endif diff --git a/src/secp256k1/src/scalar_4x64_impl.h b/src/secp256k1/src/scalar_4x64_impl.h new file mode 100644 index 00000000..aa2703dd --- /dev/null +++ b/src/secp256k1/src/scalar_4x64_impl.h @@ -0,0 +1,949 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_SCALAR_REPR_IMPL_H_ +#define _SECP256K1_SCALAR_REPR_IMPL_H_ + +/* Limbs of the secp256k1 order. */ +#define SECP256K1_N_0 ((uint64_t)0xBFD25E8CD0364141ULL) +#define SECP256K1_N_1 ((uint64_t)0xBAAEDCE6AF48A03BULL) +#define SECP256K1_N_2 ((uint64_t)0xFFFFFFFFFFFFFFFEULL) +#define SECP256K1_N_3 ((uint64_t)0xFFFFFFFFFFFFFFFFULL) + +/* Limbs of 2^256 minus the secp256k1 order. */ +#define SECP256K1_N_C_0 (~SECP256K1_N_0 + 1) +#define SECP256K1_N_C_1 (~SECP256K1_N_1) +#define SECP256K1_N_C_2 (1) + +/* Limbs of half the secp256k1 order. */ +#define SECP256K1_N_H_0 ((uint64_t)0xDFE92F46681B20A0ULL) +#define SECP256K1_N_H_1 ((uint64_t)0x5D576E7357A4501DULL) +#define SECP256K1_N_H_2 ((uint64_t)0xFFFFFFFFFFFFFFFFULL) +#define SECP256K1_N_H_3 ((uint64_t)0x7FFFFFFFFFFFFFFFULL) + +SECP256K1_INLINE static void secp256k1_scalar_clear(secp256k1_scalar *r) { + r->d[0] = 0; + r->d[1] = 0; + r->d[2] = 0; + r->d[3] = 0; +} + +SECP256K1_INLINE static void secp256k1_scalar_set_int(secp256k1_scalar *r, unsigned int v) { + r->d[0] = v; + r->d[1] = 0; + r->d[2] = 0; + r->d[3] = 0; +} + +SECP256K1_INLINE static unsigned int secp256k1_scalar_get_bits(const secp256k1_scalar *a, unsigned int offset, unsigned int count) { + VERIFY_CHECK((offset + count - 1) >> 6 == offset >> 6); + return (a->d[offset >> 6] >> (offset & 0x3F)) & ((((uint64_t)1) << count) - 1); +} + +SECP256K1_INLINE static unsigned int secp256k1_scalar_get_bits_var(const secp256k1_scalar *a, unsigned int offset, unsigned int count) { + VERIFY_CHECK(count < 32); + VERIFY_CHECK(offset + count <= 256); + if ((offset + count - 1) >> 6 == offset >> 6) { + return secp256k1_scalar_get_bits(a, offset, count); + } else { + VERIFY_CHECK((offset >> 6) + 1 < 4); + return ((a->d[offset >> 6] >> (offset & 0x3F)) | (a->d[(offset >> 6) + 1] << (64 - (offset & 0x3F)))) & ((((uint64_t)1) << count) - 1); + } +} + +SECP256K1_INLINE static int secp256k1_scalar_check_overflow(const secp256k1_scalar *a) { + int yes = 0; + int no = 0; + no |= (a->d[3] < SECP256K1_N_3); /* No need for a > check. */ + no |= (a->d[2] < SECP256K1_N_2); + yes |= (a->d[2] > SECP256K1_N_2) & ~no; + no |= (a->d[1] < SECP256K1_N_1); + yes |= (a->d[1] > SECP256K1_N_1) & ~no; + yes |= (a->d[0] >= SECP256K1_N_0) & ~no; + return yes; +} + +SECP256K1_INLINE static int secp256k1_scalar_reduce(secp256k1_scalar *r, unsigned int overflow) { + uint128_t t; + VERIFY_CHECK(overflow <= 1); + t = (uint128_t)r->d[0] + overflow * SECP256K1_N_C_0; + r->d[0] = t & 0xFFFFFFFFFFFFFFFFULL; t >>= 64; + t += (uint128_t)r->d[1] + overflow * SECP256K1_N_C_1; + r->d[1] = t & 0xFFFFFFFFFFFFFFFFULL; t >>= 64; + t += (uint128_t)r->d[2] + overflow * SECP256K1_N_C_2; + r->d[2] = t & 0xFFFFFFFFFFFFFFFFULL; t >>= 64; + t += (uint64_t)r->d[3]; + r->d[3] = t & 0xFFFFFFFFFFFFFFFFULL; + return overflow; +} + +static int secp256k1_scalar_add(secp256k1_scalar *r, const secp256k1_scalar *a, const secp256k1_scalar *b) { + int overflow; + uint128_t t = (uint128_t)a->d[0] + b->d[0]; + r->d[0] = t & 0xFFFFFFFFFFFFFFFFULL; t >>= 64; + t += (uint128_t)a->d[1] + b->d[1]; + r->d[1] = t & 0xFFFFFFFFFFFFFFFFULL; t >>= 64; + t += (uint128_t)a->d[2] + b->d[2]; + r->d[2] = t & 0xFFFFFFFFFFFFFFFFULL; t >>= 64; + t += (uint128_t)a->d[3] + b->d[3]; + r->d[3] = t & 0xFFFFFFFFFFFFFFFFULL; t >>= 64; + overflow = t + secp256k1_scalar_check_overflow(r); + VERIFY_CHECK(overflow == 0 || overflow == 1); + secp256k1_scalar_reduce(r, overflow); + return overflow; +} + +static void secp256k1_scalar_cadd_bit(secp256k1_scalar *r, unsigned int bit, int flag) { + uint128_t t; + VERIFY_CHECK(bit < 256); + bit += ((uint32_t) flag - 1) & 0x100; /* forcing (bit >> 6) > 3 makes this a noop */ + t = (uint128_t)r->d[0] + (((uint64_t)((bit >> 6) == 0)) << (bit & 0x3F)); + r->d[0] = t & 0xFFFFFFFFFFFFFFFFULL; t >>= 64; + t += (uint128_t)r->d[1] + (((uint64_t)((bit >> 6) == 1)) << (bit & 0x3F)); + r->d[1] = t & 0xFFFFFFFFFFFFFFFFULL; t >>= 64; + t += (uint128_t)r->d[2] + (((uint64_t)((bit >> 6) == 2)) << (bit & 0x3F)); + r->d[2] = t & 0xFFFFFFFFFFFFFFFFULL; t >>= 64; + t += (uint128_t)r->d[3] + (((uint64_t)((bit >> 6) == 3)) << (bit & 0x3F)); + r->d[3] = t & 0xFFFFFFFFFFFFFFFFULL; +#ifdef VERIFY + VERIFY_CHECK((t >> 64) == 0); + VERIFY_CHECK(secp256k1_scalar_check_overflow(r) == 0); +#endif +} + +static void secp256k1_scalar_set_b32(secp256k1_scalar *r, const unsigned char *b32, int *overflow) { + int over; + r->d[0] = (uint64_t)b32[31] | (uint64_t)b32[30] << 8 | (uint64_t)b32[29] << 16 | (uint64_t)b32[28] << 24 | (uint64_t)b32[27] << 32 | (uint64_t)b32[26] << 40 | (uint64_t)b32[25] << 48 | (uint64_t)b32[24] << 56; + r->d[1] = (uint64_t)b32[23] | (uint64_t)b32[22] << 8 | (uint64_t)b32[21] << 16 | (uint64_t)b32[20] << 24 | (uint64_t)b32[19] << 32 | (uint64_t)b32[18] << 40 | (uint64_t)b32[17] << 48 | (uint64_t)b32[16] << 56; + r->d[2] = (uint64_t)b32[15] | (uint64_t)b32[14] << 8 | (uint64_t)b32[13] << 16 | (uint64_t)b32[12] << 24 | (uint64_t)b32[11] << 32 | (uint64_t)b32[10] << 40 | (uint64_t)b32[9] << 48 | (uint64_t)b32[8] << 56; + r->d[3] = (uint64_t)b32[7] | (uint64_t)b32[6] << 8 | (uint64_t)b32[5] << 16 | (uint64_t)b32[4] << 24 | (uint64_t)b32[3] << 32 | (uint64_t)b32[2] << 40 | (uint64_t)b32[1] << 48 | (uint64_t)b32[0] << 56; + over = secp256k1_scalar_reduce(r, secp256k1_scalar_check_overflow(r)); + if (overflow) { + *overflow = over; + } +} + +static void secp256k1_scalar_get_b32(unsigned char *bin, const secp256k1_scalar* a) { + bin[0] = a->d[3] >> 56; bin[1] = a->d[3] >> 48; bin[2] = a->d[3] >> 40; bin[3] = a->d[3] >> 32; bin[4] = a->d[3] >> 24; bin[5] = a->d[3] >> 16; bin[6] = a->d[3] >> 8; bin[7] = a->d[3]; + bin[8] = a->d[2] >> 56; bin[9] = a->d[2] >> 48; bin[10] = a->d[2] >> 40; bin[11] = a->d[2] >> 32; bin[12] = a->d[2] >> 24; bin[13] = a->d[2] >> 16; bin[14] = a->d[2] >> 8; bin[15] = a->d[2]; + bin[16] = a->d[1] >> 56; bin[17] = a->d[1] >> 48; bin[18] = a->d[1] >> 40; bin[19] = a->d[1] >> 32; bin[20] = a->d[1] >> 24; bin[21] = a->d[1] >> 16; bin[22] = a->d[1] >> 8; bin[23] = a->d[1]; + bin[24] = a->d[0] >> 56; bin[25] = a->d[0] >> 48; bin[26] = a->d[0] >> 40; bin[27] = a->d[0] >> 32; bin[28] = a->d[0] >> 24; bin[29] = a->d[0] >> 16; bin[30] = a->d[0] >> 8; bin[31] = a->d[0]; +} + +SECP256K1_INLINE static int secp256k1_scalar_is_zero(const secp256k1_scalar *a) { + return (a->d[0] | a->d[1] | a->d[2] | a->d[3]) == 0; +} + +static void secp256k1_scalar_negate(secp256k1_scalar *r, const secp256k1_scalar *a) { + uint64_t nonzero = 0xFFFFFFFFFFFFFFFFULL * (secp256k1_scalar_is_zero(a) == 0); + uint128_t t = (uint128_t)(~a->d[0]) + SECP256K1_N_0 + 1; + r->d[0] = t & nonzero; t >>= 64; + t += (uint128_t)(~a->d[1]) + SECP256K1_N_1; + r->d[1] = t & nonzero; t >>= 64; + t += (uint128_t)(~a->d[2]) + SECP256K1_N_2; + r->d[2] = t & nonzero; t >>= 64; + t += (uint128_t)(~a->d[3]) + SECP256K1_N_3; + r->d[3] = t & nonzero; +} + +SECP256K1_INLINE static int secp256k1_scalar_is_one(const secp256k1_scalar *a) { + return ((a->d[0] ^ 1) | a->d[1] | a->d[2] | a->d[3]) == 0; +} + +static int secp256k1_scalar_is_high(const secp256k1_scalar *a) { + int yes = 0; + int no = 0; + no |= (a->d[3] < SECP256K1_N_H_3); + yes |= (a->d[3] > SECP256K1_N_H_3) & ~no; + no |= (a->d[2] < SECP256K1_N_H_2) & ~yes; /* No need for a > check. */ + no |= (a->d[1] < SECP256K1_N_H_1) & ~yes; + yes |= (a->d[1] > SECP256K1_N_H_1) & ~no; + yes |= (a->d[0] > SECP256K1_N_H_0) & ~no; + return yes; +} + +static int secp256k1_scalar_cond_negate(secp256k1_scalar *r, int flag) { + /* If we are flag = 0, mask = 00...00 and this is a no-op; + * if we are flag = 1, mask = 11...11 and this is identical to secp256k1_scalar_negate */ + uint64_t mask = !flag - 1; + uint64_t nonzero = (secp256k1_scalar_is_zero(r) != 0) - 1; + uint128_t t = (uint128_t)(r->d[0] ^ mask) + ((SECP256K1_N_0 + 1) & mask); + r->d[0] = t & nonzero; t >>= 64; + t += (uint128_t)(r->d[1] ^ mask) + (SECP256K1_N_1 & mask); + r->d[1] = t & nonzero; t >>= 64; + t += (uint128_t)(r->d[2] ^ mask) + (SECP256K1_N_2 & mask); + r->d[2] = t & nonzero; t >>= 64; + t += (uint128_t)(r->d[3] ^ mask) + (SECP256K1_N_3 & mask); + r->d[3] = t & nonzero; + return 2 * (mask == 0) - 1; +} + +/* Inspired by the macros in OpenSSL's crypto/bn/asm/x86_64-gcc.c. */ + +/** Add a*b to the number defined by (c0,c1,c2). c2 must never overflow. */ +#define muladd(a,b) { \ + uint64_t tl, th; \ + { \ + uint128_t t = (uint128_t)a * b; \ + th = t >> 64; /* at most 0xFFFFFFFFFFFFFFFE */ \ + tl = t; \ + } \ + c0 += tl; /* overflow is handled on the next line */ \ + th += (c0 < tl) ? 1 : 0; /* at most 0xFFFFFFFFFFFFFFFF */ \ + c1 += th; /* overflow is handled on the next line */ \ + c2 += (c1 < th) ? 1 : 0; /* never overflows by contract (verified in the next line) */ \ + VERIFY_CHECK((c1 >= th) || (c2 != 0)); \ +} + +/** Add a*b to the number defined by (c0,c1). c1 must never overflow. */ +#define muladd_fast(a,b) { \ + uint64_t tl, th; \ + { \ + uint128_t t = (uint128_t)a * b; \ + th = t >> 64; /* at most 0xFFFFFFFFFFFFFFFE */ \ + tl = t; \ + } \ + c0 += tl; /* overflow is handled on the next line */ \ + th += (c0 < tl) ? 1 : 0; /* at most 0xFFFFFFFFFFFFFFFF */ \ + c1 += th; /* never overflows by contract (verified in the next line) */ \ + VERIFY_CHECK(c1 >= th); \ +} + +/** Add 2*a*b to the number defined by (c0,c1,c2). c2 must never overflow. */ +#define muladd2(a,b) { \ + uint64_t tl, th, th2, tl2; \ + { \ + uint128_t t = (uint128_t)a * b; \ + th = t >> 64; /* at most 0xFFFFFFFFFFFFFFFE */ \ + tl = t; \ + } \ + th2 = th + th; /* at most 0xFFFFFFFFFFFFFFFE (in case th was 0x7FFFFFFFFFFFFFFF) */ \ + c2 += (th2 < th) ? 1 : 0; /* never overflows by contract (verified the next line) */ \ + VERIFY_CHECK((th2 >= th) || (c2 != 0)); \ + tl2 = tl + tl; /* at most 0xFFFFFFFFFFFFFFFE (in case the lowest 63 bits of tl were 0x7FFFFFFFFFFFFFFF) */ \ + th2 += (tl2 < tl) ? 1 : 0; /* at most 0xFFFFFFFFFFFFFFFF */ \ + c0 += tl2; /* overflow is handled on the next line */ \ + th2 += (c0 < tl2) ? 1 : 0; /* second overflow is handled on the next line */ \ + c2 += (c0 < tl2) & (th2 == 0); /* never overflows by contract (verified the next line) */ \ + VERIFY_CHECK((c0 >= tl2) || (th2 != 0) || (c2 != 0)); \ + c1 += th2; /* overflow is handled on the next line */ \ + c2 += (c1 < th2) ? 1 : 0; /* never overflows by contract (verified the next line) */ \ + VERIFY_CHECK((c1 >= th2) || (c2 != 0)); \ +} + +/** Add a to the number defined by (c0,c1,c2). c2 must never overflow. */ +#define sumadd(a) { \ + unsigned int over; \ + c0 += (a); /* overflow is handled on the next line */ \ + over = (c0 < (a)) ? 1 : 0; \ + c1 += over; /* overflow is handled on the next line */ \ + c2 += (c1 < over) ? 1 : 0; /* never overflows by contract */ \ +} + +/** Add a to the number defined by (c0,c1). c1 must never overflow, c2 must be zero. */ +#define sumadd_fast(a) { \ + c0 += (a); /* overflow is handled on the next line */ \ + c1 += (c0 < (a)) ? 1 : 0; /* never overflows by contract (verified the next line) */ \ + VERIFY_CHECK((c1 != 0) | (c0 >= (a))); \ + VERIFY_CHECK(c2 == 0); \ +} + +/** Extract the lowest 64 bits of (c0,c1,c2) into n, and left shift the number 64 bits. */ +#define extract(n) { \ + (n) = c0; \ + c0 = c1; \ + c1 = c2; \ + c2 = 0; \ +} + +/** Extract the lowest 64 bits of (c0,c1,c2) into n, and left shift the number 64 bits. c2 is required to be zero. */ +#define extract_fast(n) { \ + (n) = c0; \ + c0 = c1; \ + c1 = 0; \ + VERIFY_CHECK(c2 == 0); \ +} + +static void secp256k1_scalar_reduce_512(secp256k1_scalar *r, const uint64_t *l) { +#ifdef USE_ASM_X86_64 + /* Reduce 512 bits into 385. */ + uint64_t m0, m1, m2, m3, m4, m5, m6; + uint64_t p0, p1, p2, p3, p4; + uint64_t c; + + __asm__ __volatile__( + /* Preload. */ + "movq 32(%%rsi), %%r11\n" + "movq 40(%%rsi), %%r12\n" + "movq 48(%%rsi), %%r13\n" + "movq 56(%%rsi), %%r14\n" + /* Initialize r8,r9,r10 */ + "movq 0(%%rsi), %%r8\n" + "movq $0, %%r9\n" + "movq $0, %%r10\n" + /* (r8,r9) += n0 * c0 */ + "movq %8, %%rax\n" + "mulq %%r11\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + /* extract m0 */ + "movq %%r8, %q0\n" + "movq $0, %%r8\n" + /* (r9,r10) += l1 */ + "addq 8(%%rsi), %%r9\n" + "adcq $0, %%r10\n" + /* (r9,r10,r8) += n1 * c0 */ + "movq %8, %%rax\n" + "mulq %%r12\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* (r9,r10,r8) += n0 * c1 */ + "movq %9, %%rax\n" + "mulq %%r11\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* extract m1 */ + "movq %%r9, %q1\n" + "movq $0, %%r9\n" + /* (r10,r8,r9) += l2 */ + "addq 16(%%rsi), %%r10\n" + "adcq $0, %%r8\n" + "adcq $0, %%r9\n" + /* (r10,r8,r9) += n2 * c0 */ + "movq %8, %%rax\n" + "mulq %%r13\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + /* (r10,r8,r9) += n1 * c1 */ + "movq %9, %%rax\n" + "mulq %%r12\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + /* (r10,r8,r9) += n0 */ + "addq %%r11, %%r10\n" + "adcq $0, %%r8\n" + "adcq $0, %%r9\n" + /* extract m2 */ + "movq %%r10, %q2\n" + "movq $0, %%r10\n" + /* (r8,r9,r10) += l3 */ + "addq 24(%%rsi), %%r8\n" + "adcq $0, %%r9\n" + "adcq $0, %%r10\n" + /* (r8,r9,r10) += n3 * c0 */ + "movq %8, %%rax\n" + "mulq %%r14\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + /* (r8,r9,r10) += n2 * c1 */ + "movq %9, %%rax\n" + "mulq %%r13\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + /* (r8,r9,r10) += n1 */ + "addq %%r12, %%r8\n" + "adcq $0, %%r9\n" + "adcq $0, %%r10\n" + /* extract m3 */ + "movq %%r8, %q3\n" + "movq $0, %%r8\n" + /* (r9,r10,r8) += n3 * c1 */ + "movq %9, %%rax\n" + "mulq %%r14\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* (r9,r10,r8) += n2 */ + "addq %%r13, %%r9\n" + "adcq $0, %%r10\n" + "adcq $0, %%r8\n" + /* extract m4 */ + "movq %%r9, %q4\n" + /* (r10,r8) += n3 */ + "addq %%r14, %%r10\n" + "adcq $0, %%r8\n" + /* extract m5 */ + "movq %%r10, %q5\n" + /* extract m6 */ + "movq %%r8, %q6\n" + : "=g"(m0), "=g"(m1), "=g"(m2), "=g"(m3), "=g"(m4), "=g"(m5), "=g"(m6) + : "S"(l), "n"(SECP256K1_N_C_0), "n"(SECP256K1_N_C_1) + : "rax", "rdx", "r8", "r9", "r10", "r11", "r12", "r13", "r14", "cc"); + + /* Reduce 385 bits into 258. */ + __asm__ __volatile__( + /* Preload */ + "movq %q9, %%r11\n" + "movq %q10, %%r12\n" + "movq %q11, %%r13\n" + /* Initialize (r8,r9,r10) */ + "movq %q5, %%r8\n" + "movq $0, %%r9\n" + "movq $0, %%r10\n" + /* (r8,r9) += m4 * c0 */ + "movq %12, %%rax\n" + "mulq %%r11\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + /* extract p0 */ + "movq %%r8, %q0\n" + "movq $0, %%r8\n" + /* (r9,r10) += m1 */ + "addq %q6, %%r9\n" + "adcq $0, %%r10\n" + /* (r9,r10,r8) += m5 * c0 */ + "movq %12, %%rax\n" + "mulq %%r12\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* (r9,r10,r8) += m4 * c1 */ + "movq %13, %%rax\n" + "mulq %%r11\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* extract p1 */ + "movq %%r9, %q1\n" + "movq $0, %%r9\n" + /* (r10,r8,r9) += m2 */ + "addq %q7, %%r10\n" + "adcq $0, %%r8\n" + "adcq $0, %%r9\n" + /* (r10,r8,r9) += m6 * c0 */ + "movq %12, %%rax\n" + "mulq %%r13\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + /* (r10,r8,r9) += m5 * c1 */ + "movq %13, %%rax\n" + "mulq %%r12\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + /* (r10,r8,r9) += m4 */ + "addq %%r11, %%r10\n" + "adcq $0, %%r8\n" + "adcq $0, %%r9\n" + /* extract p2 */ + "movq %%r10, %q2\n" + /* (r8,r9) += m3 */ + "addq %q8, %%r8\n" + "adcq $0, %%r9\n" + /* (r8,r9) += m6 * c1 */ + "movq %13, %%rax\n" + "mulq %%r13\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + /* (r8,r9) += m5 */ + "addq %%r12, %%r8\n" + "adcq $0, %%r9\n" + /* extract p3 */ + "movq %%r8, %q3\n" + /* (r9) += m6 */ + "addq %%r13, %%r9\n" + /* extract p4 */ + "movq %%r9, %q4\n" + : "=&g"(p0), "=&g"(p1), "=&g"(p2), "=g"(p3), "=g"(p4) + : "g"(m0), "g"(m1), "g"(m2), "g"(m3), "g"(m4), "g"(m5), "g"(m6), "n"(SECP256K1_N_C_0), "n"(SECP256K1_N_C_1) + : "rax", "rdx", "r8", "r9", "r10", "r11", "r12", "r13", "cc"); + + /* Reduce 258 bits into 256. */ + __asm__ __volatile__( + /* Preload */ + "movq %q5, %%r10\n" + /* (rax,rdx) = p4 * c0 */ + "movq %7, %%rax\n" + "mulq %%r10\n" + /* (rax,rdx) += p0 */ + "addq %q1, %%rax\n" + "adcq $0, %%rdx\n" + /* extract r0 */ + "movq %%rax, 0(%q6)\n" + /* Move to (r8,r9) */ + "movq %%rdx, %%r8\n" + "movq $0, %%r9\n" + /* (r8,r9) += p1 */ + "addq %q2, %%r8\n" + "adcq $0, %%r9\n" + /* (r8,r9) += p4 * c1 */ + "movq %8, %%rax\n" + "mulq %%r10\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + /* Extract r1 */ + "movq %%r8, 8(%q6)\n" + "movq $0, %%r8\n" + /* (r9,r8) += p4 */ + "addq %%r10, %%r9\n" + "adcq $0, %%r8\n" + /* (r9,r8) += p2 */ + "addq %q3, %%r9\n" + "adcq $0, %%r8\n" + /* Extract r2 */ + "movq %%r9, 16(%q6)\n" + "movq $0, %%r9\n" + /* (r8,r9) += p3 */ + "addq %q4, %%r8\n" + "adcq $0, %%r9\n" + /* Extract r3 */ + "movq %%r8, 24(%q6)\n" + /* Extract c */ + "movq %%r9, %q0\n" + : "=g"(c) + : "g"(p0), "g"(p1), "g"(p2), "g"(p3), "g"(p4), "D"(r), "n"(SECP256K1_N_C_0), "n"(SECP256K1_N_C_1) + : "rax", "rdx", "r8", "r9", "r10", "cc", "memory"); +#else + uint128_t c; + uint64_t c0, c1, c2; + uint64_t n0 = l[4], n1 = l[5], n2 = l[6], n3 = l[7]; + uint64_t m0, m1, m2, m3, m4, m5; + uint32_t m6; + uint64_t p0, p1, p2, p3; + uint32_t p4; + + /* Reduce 512 bits into 385. */ + /* m[0..6] = l[0..3] + n[0..3] * SECP256K1_N_C. */ + c0 = l[0]; c1 = 0; c2 = 0; + muladd_fast(n0, SECP256K1_N_C_0); + extract_fast(m0); + sumadd_fast(l[1]); + muladd(n1, SECP256K1_N_C_0); + muladd(n0, SECP256K1_N_C_1); + extract(m1); + sumadd(l[2]); + muladd(n2, SECP256K1_N_C_0); + muladd(n1, SECP256K1_N_C_1); + sumadd(n0); + extract(m2); + sumadd(l[3]); + muladd(n3, SECP256K1_N_C_0); + muladd(n2, SECP256K1_N_C_1); + sumadd(n1); + extract(m3); + muladd(n3, SECP256K1_N_C_1); + sumadd(n2); + extract(m4); + sumadd_fast(n3); + extract_fast(m5); + VERIFY_CHECK(c0 <= 1); + m6 = c0; + + /* Reduce 385 bits into 258. */ + /* p[0..4] = m[0..3] + m[4..6] * SECP256K1_N_C. */ + c0 = m0; c1 = 0; c2 = 0; + muladd_fast(m4, SECP256K1_N_C_0); + extract_fast(p0); + sumadd_fast(m1); + muladd(m5, SECP256K1_N_C_0); + muladd(m4, SECP256K1_N_C_1); + extract(p1); + sumadd(m2); + muladd(m6, SECP256K1_N_C_0); + muladd(m5, SECP256K1_N_C_1); + sumadd(m4); + extract(p2); + sumadd_fast(m3); + muladd_fast(m6, SECP256K1_N_C_1); + sumadd_fast(m5); + extract_fast(p3); + p4 = c0 + m6; + VERIFY_CHECK(p4 <= 2); + + /* Reduce 258 bits into 256. */ + /* r[0..3] = p[0..3] + p[4] * SECP256K1_N_C. */ + c = p0 + (uint128_t)SECP256K1_N_C_0 * p4; + r->d[0] = c & 0xFFFFFFFFFFFFFFFFULL; c >>= 64; + c += p1 + (uint128_t)SECP256K1_N_C_1 * p4; + r->d[1] = c & 0xFFFFFFFFFFFFFFFFULL; c >>= 64; + c += p2 + (uint128_t)p4; + r->d[2] = c & 0xFFFFFFFFFFFFFFFFULL; c >>= 64; + c += p3; + r->d[3] = c & 0xFFFFFFFFFFFFFFFFULL; c >>= 64; +#endif + + /* Final reduction of r. */ + secp256k1_scalar_reduce(r, c + secp256k1_scalar_check_overflow(r)); +} + +static void secp256k1_scalar_mul_512(uint64_t l[8], const secp256k1_scalar *a, const secp256k1_scalar *b) { +#ifdef USE_ASM_X86_64 + const uint64_t *pb = b->d; + __asm__ __volatile__( + /* Preload */ + "movq 0(%%rdi), %%r15\n" + "movq 8(%%rdi), %%rbx\n" + "movq 16(%%rdi), %%rcx\n" + "movq 0(%%rdx), %%r11\n" + "movq 8(%%rdx), %%r12\n" + "movq 16(%%rdx), %%r13\n" + "movq 24(%%rdx), %%r14\n" + /* (rax,rdx) = a0 * b0 */ + "movq %%r15, %%rax\n" + "mulq %%r11\n" + /* Extract l0 */ + "movq %%rax, 0(%%rsi)\n" + /* (r8,r9,r10) = (rdx) */ + "movq %%rdx, %%r8\n" + "xorq %%r9, %%r9\n" + "xorq %%r10, %%r10\n" + /* (r8,r9,r10) += a0 * b1 */ + "movq %%r15, %%rax\n" + "mulq %%r12\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + /* (r8,r9,r10) += a1 * b0 */ + "movq %%rbx, %%rax\n" + "mulq %%r11\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + /* Extract l1 */ + "movq %%r8, 8(%%rsi)\n" + "xorq %%r8, %%r8\n" + /* (r9,r10,r8) += a0 * b2 */ + "movq %%r15, %%rax\n" + "mulq %%r13\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* (r9,r10,r8) += a1 * b1 */ + "movq %%rbx, %%rax\n" + "mulq %%r12\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* (r9,r10,r8) += a2 * b0 */ + "movq %%rcx, %%rax\n" + "mulq %%r11\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* Extract l2 */ + "movq %%r9, 16(%%rsi)\n" + "xorq %%r9, %%r9\n" + /* (r10,r8,r9) += a0 * b3 */ + "movq %%r15, %%rax\n" + "mulq %%r14\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + /* Preload a3 */ + "movq 24(%%rdi), %%r15\n" + /* (r10,r8,r9) += a1 * b2 */ + "movq %%rbx, %%rax\n" + "mulq %%r13\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + /* (r10,r8,r9) += a2 * b1 */ + "movq %%rcx, %%rax\n" + "mulq %%r12\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + /* (r10,r8,r9) += a3 * b0 */ + "movq %%r15, %%rax\n" + "mulq %%r11\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + /* Extract l3 */ + "movq %%r10, 24(%%rsi)\n" + "xorq %%r10, %%r10\n" + /* (r8,r9,r10) += a1 * b3 */ + "movq %%rbx, %%rax\n" + "mulq %%r14\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + /* (r8,r9,r10) += a2 * b2 */ + "movq %%rcx, %%rax\n" + "mulq %%r13\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + /* (r8,r9,r10) += a3 * b1 */ + "movq %%r15, %%rax\n" + "mulq %%r12\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + /* Extract l4 */ + "movq %%r8, 32(%%rsi)\n" + "xorq %%r8, %%r8\n" + /* (r9,r10,r8) += a2 * b3 */ + "movq %%rcx, %%rax\n" + "mulq %%r14\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* (r9,r10,r8) += a3 * b2 */ + "movq %%r15, %%rax\n" + "mulq %%r13\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* Extract l5 */ + "movq %%r9, 40(%%rsi)\n" + /* (r10,r8) += a3 * b3 */ + "movq %%r15, %%rax\n" + "mulq %%r14\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + /* Extract l6 */ + "movq %%r10, 48(%%rsi)\n" + /* Extract l7 */ + "movq %%r8, 56(%%rsi)\n" + : "+d"(pb) + : "S"(l), "D"(a->d) + : "rax", "rbx", "rcx", "r8", "r9", "r10", "r11", "r12", "r13", "r14", "r15", "cc", "memory"); +#else + /* 160 bit accumulator. */ + uint64_t c0 = 0, c1 = 0; + uint32_t c2 = 0; + + /* l[0..7] = a[0..3] * b[0..3]. */ + muladd_fast(a->d[0], b->d[0]); + extract_fast(l[0]); + muladd(a->d[0], b->d[1]); + muladd(a->d[1], b->d[0]); + extract(l[1]); + muladd(a->d[0], b->d[2]); + muladd(a->d[1], b->d[1]); + muladd(a->d[2], b->d[0]); + extract(l[2]); + muladd(a->d[0], b->d[3]); + muladd(a->d[1], b->d[2]); + muladd(a->d[2], b->d[1]); + muladd(a->d[3], b->d[0]); + extract(l[3]); + muladd(a->d[1], b->d[3]); + muladd(a->d[2], b->d[2]); + muladd(a->d[3], b->d[1]); + extract(l[4]); + muladd(a->d[2], b->d[3]); + muladd(a->d[3], b->d[2]); + extract(l[5]); + muladd_fast(a->d[3], b->d[3]); + extract_fast(l[6]); + VERIFY_CHECK(c1 == 0); + l[7] = c0; +#endif +} + +static void secp256k1_scalar_sqr_512(uint64_t l[8], const secp256k1_scalar *a) { +#ifdef USE_ASM_X86_64 + __asm__ __volatile__( + /* Preload */ + "movq 0(%%rdi), %%r11\n" + "movq 8(%%rdi), %%r12\n" + "movq 16(%%rdi), %%r13\n" + "movq 24(%%rdi), %%r14\n" + /* (rax,rdx) = a0 * a0 */ + "movq %%r11, %%rax\n" + "mulq %%r11\n" + /* Extract l0 */ + "movq %%rax, 0(%%rsi)\n" + /* (r8,r9,r10) = (rdx,0) */ + "movq %%rdx, %%r8\n" + "xorq %%r9, %%r9\n" + "xorq %%r10, %%r10\n" + /* (r8,r9,r10) += 2 * a0 * a1 */ + "movq %%r11, %%rax\n" + "mulq %%r12\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + /* Extract l1 */ + "movq %%r8, 8(%%rsi)\n" + "xorq %%r8, %%r8\n" + /* (r9,r10,r8) += 2 * a0 * a2 */ + "movq %%r11, %%rax\n" + "mulq %%r13\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* (r9,r10,r8) += a1 * a1 */ + "movq %%r12, %%rax\n" + "mulq %%r12\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* Extract l2 */ + "movq %%r9, 16(%%rsi)\n" + "xorq %%r9, %%r9\n" + /* (r10,r8,r9) += 2 * a0 * a3 */ + "movq %%r11, %%rax\n" + "mulq %%r14\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + /* (r10,r8,r9) += 2 * a1 * a2 */ + "movq %%r12, %%rax\n" + "mulq %%r13\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + "adcq $0, %%r9\n" + /* Extract l3 */ + "movq %%r10, 24(%%rsi)\n" + "xorq %%r10, %%r10\n" + /* (r8,r9,r10) += 2 * a1 * a3 */ + "movq %%r12, %%rax\n" + "mulq %%r14\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + /* (r8,r9,r10) += a2 * a2 */ + "movq %%r13, %%rax\n" + "mulq %%r13\n" + "addq %%rax, %%r8\n" + "adcq %%rdx, %%r9\n" + "adcq $0, %%r10\n" + /* Extract l4 */ + "movq %%r8, 32(%%rsi)\n" + "xorq %%r8, %%r8\n" + /* (r9,r10,r8) += 2 * a2 * a3 */ + "movq %%r13, %%rax\n" + "mulq %%r14\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + "addq %%rax, %%r9\n" + "adcq %%rdx, %%r10\n" + "adcq $0, %%r8\n" + /* Extract l5 */ + "movq %%r9, 40(%%rsi)\n" + /* (r10,r8) += a3 * a3 */ + "movq %%r14, %%rax\n" + "mulq %%r14\n" + "addq %%rax, %%r10\n" + "adcq %%rdx, %%r8\n" + /* Extract l6 */ + "movq %%r10, 48(%%rsi)\n" + /* Extract l7 */ + "movq %%r8, 56(%%rsi)\n" + : + : "S"(l), "D"(a->d) + : "rax", "rdx", "r8", "r9", "r10", "r11", "r12", "r13", "r14", "cc", "memory"); +#else + /* 160 bit accumulator. */ + uint64_t c0 = 0, c1 = 0; + uint32_t c2 = 0; + + /* l[0..7] = a[0..3] * b[0..3]. */ + muladd_fast(a->d[0], a->d[0]); + extract_fast(l[0]); + muladd2(a->d[0], a->d[1]); + extract(l[1]); + muladd2(a->d[0], a->d[2]); + muladd(a->d[1], a->d[1]); + extract(l[2]); + muladd2(a->d[0], a->d[3]); + muladd2(a->d[1], a->d[2]); + extract(l[3]); + muladd2(a->d[1], a->d[3]); + muladd(a->d[2], a->d[2]); + extract(l[4]); + muladd2(a->d[2], a->d[3]); + extract(l[5]); + muladd_fast(a->d[3], a->d[3]); + extract_fast(l[6]); + VERIFY_CHECK(c1 == 0); + l[7] = c0; +#endif +} + +#undef sumadd +#undef sumadd_fast +#undef muladd +#undef muladd_fast +#undef muladd2 +#undef extract +#undef extract_fast + +static void secp256k1_scalar_mul(secp256k1_scalar *r, const secp256k1_scalar *a, const secp256k1_scalar *b) { + uint64_t l[8]; + secp256k1_scalar_mul_512(l, a, b); + secp256k1_scalar_reduce_512(r, l); +} + +static int secp256k1_scalar_shr_int(secp256k1_scalar *r, int n) { + int ret; + VERIFY_CHECK(n > 0); + VERIFY_CHECK(n < 16); + ret = r->d[0] & ((1 << n) - 1); + r->d[0] = (r->d[0] >> n) + (r->d[1] << (64 - n)); + r->d[1] = (r->d[1] >> n) + (r->d[2] << (64 - n)); + r->d[2] = (r->d[2] >> n) + (r->d[3] << (64 - n)); + r->d[3] = (r->d[3] >> n); + return ret; +} + +static void secp256k1_scalar_sqr(secp256k1_scalar *r, const secp256k1_scalar *a) { + uint64_t l[8]; + secp256k1_scalar_sqr_512(l, a); + secp256k1_scalar_reduce_512(r, l); +} + +#ifdef USE_ENDOMORPHISM +static void secp256k1_scalar_split_128(secp256k1_scalar *r1, secp256k1_scalar *r2, const secp256k1_scalar *a) { + r1->d[0] = a->d[0]; + r1->d[1] = a->d[1]; + r1->d[2] = 0; + r1->d[3] = 0; + r2->d[0] = a->d[2]; + r2->d[1] = a->d[3]; + r2->d[2] = 0; + r2->d[3] = 0; +} +#endif + +SECP256K1_INLINE static int secp256k1_scalar_eq(const secp256k1_scalar *a, const secp256k1_scalar *b) { + return ((a->d[0] ^ b->d[0]) | (a->d[1] ^ b->d[1]) | (a->d[2] ^ b->d[2]) | (a->d[3] ^ b->d[3])) == 0; +} + +SECP256K1_INLINE static void secp256k1_scalar_mul_shift_var(secp256k1_scalar *r, const secp256k1_scalar *a, const secp256k1_scalar *b, unsigned int shift) { + uint64_t l[8]; + unsigned int shiftlimbs; + unsigned int shiftlow; + unsigned int shifthigh; + VERIFY_CHECK(shift >= 256); + secp256k1_scalar_mul_512(l, a, b); + shiftlimbs = shift >> 6; + shiftlow = shift & 0x3F; + shifthigh = 64 - shiftlow; + r->d[0] = shift < 512 ? (l[0 + shiftlimbs] >> shiftlow | (shift < 448 && shiftlow ? (l[1 + shiftlimbs] << shifthigh) : 0)) : 0; + r->d[1] = shift < 448 ? (l[1 + shiftlimbs] >> shiftlow | (shift < 384 && shiftlow ? (l[2 + shiftlimbs] << shifthigh) : 0)) : 0; + r->d[2] = shift < 384 ? (l[2 + shiftlimbs] >> shiftlow | (shift < 320 && shiftlow ? (l[3 + shiftlimbs] << shifthigh) : 0)) : 0; + r->d[3] = shift < 320 ? (l[3 + shiftlimbs] >> shiftlow) : 0; + secp256k1_scalar_cadd_bit(r, 0, (l[(shift - 1) >> 6] >> ((shift - 1) & 0x3f)) & 1); +} + +#endif diff --git a/src/secp256k1/src/scalar_8x32.h b/src/secp256k1/src/scalar_8x32.h new file mode 100644 index 00000000..1319664f --- /dev/null +++ b/src/secp256k1/src/scalar_8x32.h @@ -0,0 +1,19 @@ +/********************************************************************** + * Copyright (c) 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_SCALAR_REPR_ +#define _SECP256K1_SCALAR_REPR_ + +#include + +/** A scalar modulo the group order of the secp256k1 curve. */ +typedef struct { + uint32_t d[8]; +} secp256k1_scalar; + +#define SECP256K1_SCALAR_CONST(d7, d6, d5, d4, d3, d2, d1, d0) {{(d0), (d1), (d2), (d3), (d4), (d5), (d6), (d7)}} + +#endif diff --git a/src/secp256k1/src/scalar_8x32_impl.h b/src/secp256k1/src/scalar_8x32_impl.h new file mode 100644 index 00000000..aae4f35c --- /dev/null +++ b/src/secp256k1/src/scalar_8x32_impl.h @@ -0,0 +1,721 @@ +/********************************************************************** + * Copyright (c) 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_SCALAR_REPR_IMPL_H_ +#define _SECP256K1_SCALAR_REPR_IMPL_H_ + +/* Limbs of the secp256k1 order. */ +#define SECP256K1_N_0 ((uint32_t)0xD0364141UL) +#define SECP256K1_N_1 ((uint32_t)0xBFD25E8CUL) +#define SECP256K1_N_2 ((uint32_t)0xAF48A03BUL) +#define SECP256K1_N_3 ((uint32_t)0xBAAEDCE6UL) +#define SECP256K1_N_4 ((uint32_t)0xFFFFFFFEUL) +#define SECP256K1_N_5 ((uint32_t)0xFFFFFFFFUL) +#define SECP256K1_N_6 ((uint32_t)0xFFFFFFFFUL) +#define SECP256K1_N_7 ((uint32_t)0xFFFFFFFFUL) + +/* Limbs of 2^256 minus the secp256k1 order. */ +#define SECP256K1_N_C_0 (~SECP256K1_N_0 + 1) +#define SECP256K1_N_C_1 (~SECP256K1_N_1) +#define SECP256K1_N_C_2 (~SECP256K1_N_2) +#define SECP256K1_N_C_3 (~SECP256K1_N_3) +#define SECP256K1_N_C_4 (1) + +/* Limbs of half the secp256k1 order. */ +#define SECP256K1_N_H_0 ((uint32_t)0x681B20A0UL) +#define SECP256K1_N_H_1 ((uint32_t)0xDFE92F46UL) +#define SECP256K1_N_H_2 ((uint32_t)0x57A4501DUL) +#define SECP256K1_N_H_3 ((uint32_t)0x5D576E73UL) +#define SECP256K1_N_H_4 ((uint32_t)0xFFFFFFFFUL) +#define SECP256K1_N_H_5 ((uint32_t)0xFFFFFFFFUL) +#define SECP256K1_N_H_6 ((uint32_t)0xFFFFFFFFUL) +#define SECP256K1_N_H_7 ((uint32_t)0x7FFFFFFFUL) + +SECP256K1_INLINE static void secp256k1_scalar_clear(secp256k1_scalar *r) { + r->d[0] = 0; + r->d[1] = 0; + r->d[2] = 0; + r->d[3] = 0; + r->d[4] = 0; + r->d[5] = 0; + r->d[6] = 0; + r->d[7] = 0; +} + +SECP256K1_INLINE static void secp256k1_scalar_set_int(secp256k1_scalar *r, unsigned int v) { + r->d[0] = v; + r->d[1] = 0; + r->d[2] = 0; + r->d[3] = 0; + r->d[4] = 0; + r->d[5] = 0; + r->d[6] = 0; + r->d[7] = 0; +} + +SECP256K1_INLINE static unsigned int secp256k1_scalar_get_bits(const secp256k1_scalar *a, unsigned int offset, unsigned int count) { + VERIFY_CHECK((offset + count - 1) >> 5 == offset >> 5); + return (a->d[offset >> 5] >> (offset & 0x1F)) & ((1 << count) - 1); +} + +SECP256K1_INLINE static unsigned int secp256k1_scalar_get_bits_var(const secp256k1_scalar *a, unsigned int offset, unsigned int count) { + VERIFY_CHECK(count < 32); + VERIFY_CHECK(offset + count <= 256); + if ((offset + count - 1) >> 5 == offset >> 5) { + return secp256k1_scalar_get_bits(a, offset, count); + } else { + VERIFY_CHECK((offset >> 5) + 1 < 8); + return ((a->d[offset >> 5] >> (offset & 0x1F)) | (a->d[(offset >> 5) + 1] << (32 - (offset & 0x1F)))) & ((((uint32_t)1) << count) - 1); + } +} + +SECP256K1_INLINE static int secp256k1_scalar_check_overflow(const secp256k1_scalar *a) { + int yes = 0; + int no = 0; + no |= (a->d[7] < SECP256K1_N_7); /* No need for a > check. */ + no |= (a->d[6] < SECP256K1_N_6); /* No need for a > check. */ + no |= (a->d[5] < SECP256K1_N_5); /* No need for a > check. */ + no |= (a->d[4] < SECP256K1_N_4); + yes |= (a->d[4] > SECP256K1_N_4) & ~no; + no |= (a->d[3] < SECP256K1_N_3) & ~yes; + yes |= (a->d[3] > SECP256K1_N_3) & ~no; + no |= (a->d[2] < SECP256K1_N_2) & ~yes; + yes |= (a->d[2] > SECP256K1_N_2) & ~no; + no |= (a->d[1] < SECP256K1_N_1) & ~yes; + yes |= (a->d[1] > SECP256K1_N_1) & ~no; + yes |= (a->d[0] >= SECP256K1_N_0) & ~no; + return yes; +} + +SECP256K1_INLINE static int secp256k1_scalar_reduce(secp256k1_scalar *r, uint32_t overflow) { + uint64_t t; + VERIFY_CHECK(overflow <= 1); + t = (uint64_t)r->d[0] + overflow * SECP256K1_N_C_0; + r->d[0] = t & 0xFFFFFFFFUL; t >>= 32; + t += (uint64_t)r->d[1] + overflow * SECP256K1_N_C_1; + r->d[1] = t & 0xFFFFFFFFUL; t >>= 32; + t += (uint64_t)r->d[2] + overflow * SECP256K1_N_C_2; + r->d[2] = t & 0xFFFFFFFFUL; t >>= 32; + t += (uint64_t)r->d[3] + overflow * SECP256K1_N_C_3; + r->d[3] = t & 0xFFFFFFFFUL; t >>= 32; + t += (uint64_t)r->d[4] + overflow * SECP256K1_N_C_4; + r->d[4] = t & 0xFFFFFFFFUL; t >>= 32; + t += (uint64_t)r->d[5]; + r->d[5] = t & 0xFFFFFFFFUL; t >>= 32; + t += (uint64_t)r->d[6]; + r->d[6] = t & 0xFFFFFFFFUL; t >>= 32; + t += (uint64_t)r->d[7]; + r->d[7] = t & 0xFFFFFFFFUL; + return overflow; +} + +static int secp256k1_scalar_add(secp256k1_scalar *r, const secp256k1_scalar *a, const secp256k1_scalar *b) { + int overflow; + uint64_t t = (uint64_t)a->d[0] + b->d[0]; + r->d[0] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)a->d[1] + b->d[1]; + r->d[1] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)a->d[2] + b->d[2]; + r->d[2] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)a->d[3] + b->d[3]; + r->d[3] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)a->d[4] + b->d[4]; + r->d[4] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)a->d[5] + b->d[5]; + r->d[5] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)a->d[6] + b->d[6]; + r->d[6] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)a->d[7] + b->d[7]; + r->d[7] = t & 0xFFFFFFFFULL; t >>= 32; + overflow = t + secp256k1_scalar_check_overflow(r); + VERIFY_CHECK(overflow == 0 || overflow == 1); + secp256k1_scalar_reduce(r, overflow); + return overflow; +} + +static void secp256k1_scalar_cadd_bit(secp256k1_scalar *r, unsigned int bit, int flag) { + uint64_t t; + VERIFY_CHECK(bit < 256); + bit += ((uint32_t) flag - 1) & 0x100; /* forcing (bit >> 5) > 7 makes this a noop */ + t = (uint64_t)r->d[0] + (((uint32_t)((bit >> 5) == 0)) << (bit & 0x1F)); + r->d[0] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)r->d[1] + (((uint32_t)((bit >> 5) == 1)) << (bit & 0x1F)); + r->d[1] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)r->d[2] + (((uint32_t)((bit >> 5) == 2)) << (bit & 0x1F)); + r->d[2] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)r->d[3] + (((uint32_t)((bit >> 5) == 3)) << (bit & 0x1F)); + r->d[3] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)r->d[4] + (((uint32_t)((bit >> 5) == 4)) << (bit & 0x1F)); + r->d[4] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)r->d[5] + (((uint32_t)((bit >> 5) == 5)) << (bit & 0x1F)); + r->d[5] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)r->d[6] + (((uint32_t)((bit >> 5) == 6)) << (bit & 0x1F)); + r->d[6] = t & 0xFFFFFFFFULL; t >>= 32; + t += (uint64_t)r->d[7] + (((uint32_t)((bit >> 5) == 7)) << (bit & 0x1F)); + r->d[7] = t & 0xFFFFFFFFULL; +#ifdef VERIFY + VERIFY_CHECK((t >> 32) == 0); + VERIFY_CHECK(secp256k1_scalar_check_overflow(r) == 0); +#endif +} + +static void secp256k1_scalar_set_b32(secp256k1_scalar *r, const unsigned char *b32, int *overflow) { + int over; + r->d[0] = (uint32_t)b32[31] | (uint32_t)b32[30] << 8 | (uint32_t)b32[29] << 16 | (uint32_t)b32[28] << 24; + r->d[1] = (uint32_t)b32[27] | (uint32_t)b32[26] << 8 | (uint32_t)b32[25] << 16 | (uint32_t)b32[24] << 24; + r->d[2] = (uint32_t)b32[23] | (uint32_t)b32[22] << 8 | (uint32_t)b32[21] << 16 | (uint32_t)b32[20] << 24; + r->d[3] = (uint32_t)b32[19] | (uint32_t)b32[18] << 8 | (uint32_t)b32[17] << 16 | (uint32_t)b32[16] << 24; + r->d[4] = (uint32_t)b32[15] | (uint32_t)b32[14] << 8 | (uint32_t)b32[13] << 16 | (uint32_t)b32[12] << 24; + r->d[5] = (uint32_t)b32[11] | (uint32_t)b32[10] << 8 | (uint32_t)b32[9] << 16 | (uint32_t)b32[8] << 24; + r->d[6] = (uint32_t)b32[7] | (uint32_t)b32[6] << 8 | (uint32_t)b32[5] << 16 | (uint32_t)b32[4] << 24; + r->d[7] = (uint32_t)b32[3] | (uint32_t)b32[2] << 8 | (uint32_t)b32[1] << 16 | (uint32_t)b32[0] << 24; + over = secp256k1_scalar_reduce(r, secp256k1_scalar_check_overflow(r)); + if (overflow) { + *overflow = over; + } +} + +static void secp256k1_scalar_get_b32(unsigned char *bin, const secp256k1_scalar* a) { + bin[0] = a->d[7] >> 24; bin[1] = a->d[7] >> 16; bin[2] = a->d[7] >> 8; bin[3] = a->d[7]; + bin[4] = a->d[6] >> 24; bin[5] = a->d[6] >> 16; bin[6] = a->d[6] >> 8; bin[7] = a->d[6]; + bin[8] = a->d[5] >> 24; bin[9] = a->d[5] >> 16; bin[10] = a->d[5] >> 8; bin[11] = a->d[5]; + bin[12] = a->d[4] >> 24; bin[13] = a->d[4] >> 16; bin[14] = a->d[4] >> 8; bin[15] = a->d[4]; + bin[16] = a->d[3] >> 24; bin[17] = a->d[3] >> 16; bin[18] = a->d[3] >> 8; bin[19] = a->d[3]; + bin[20] = a->d[2] >> 24; bin[21] = a->d[2] >> 16; bin[22] = a->d[2] >> 8; bin[23] = a->d[2]; + bin[24] = a->d[1] >> 24; bin[25] = a->d[1] >> 16; bin[26] = a->d[1] >> 8; bin[27] = a->d[1]; + bin[28] = a->d[0] >> 24; bin[29] = a->d[0] >> 16; bin[30] = a->d[0] >> 8; bin[31] = a->d[0]; +} + +SECP256K1_INLINE static int secp256k1_scalar_is_zero(const secp256k1_scalar *a) { + return (a->d[0] | a->d[1] | a->d[2] | a->d[3] | a->d[4] | a->d[5] | a->d[6] | a->d[7]) == 0; +} + +static void secp256k1_scalar_negate(secp256k1_scalar *r, const secp256k1_scalar *a) { + uint32_t nonzero = 0xFFFFFFFFUL * (secp256k1_scalar_is_zero(a) == 0); + uint64_t t = (uint64_t)(~a->d[0]) + SECP256K1_N_0 + 1; + r->d[0] = t & nonzero; t >>= 32; + t += (uint64_t)(~a->d[1]) + SECP256K1_N_1; + r->d[1] = t & nonzero; t >>= 32; + t += (uint64_t)(~a->d[2]) + SECP256K1_N_2; + r->d[2] = t & nonzero; t >>= 32; + t += (uint64_t)(~a->d[3]) + SECP256K1_N_3; + r->d[3] = t & nonzero; t >>= 32; + t += (uint64_t)(~a->d[4]) + SECP256K1_N_4; + r->d[4] = t & nonzero; t >>= 32; + t += (uint64_t)(~a->d[5]) + SECP256K1_N_5; + r->d[5] = t & nonzero; t >>= 32; + t += (uint64_t)(~a->d[6]) + SECP256K1_N_6; + r->d[6] = t & nonzero; t >>= 32; + t += (uint64_t)(~a->d[7]) + SECP256K1_N_7; + r->d[7] = t & nonzero; +} + +SECP256K1_INLINE static int secp256k1_scalar_is_one(const secp256k1_scalar *a) { + return ((a->d[0] ^ 1) | a->d[1] | a->d[2] | a->d[3] | a->d[4] | a->d[5] | a->d[6] | a->d[7]) == 0; +} + +static int secp256k1_scalar_is_high(const secp256k1_scalar *a) { + int yes = 0; + int no = 0; + no |= (a->d[7] < SECP256K1_N_H_7); + yes |= (a->d[7] > SECP256K1_N_H_7) & ~no; + no |= (a->d[6] < SECP256K1_N_H_6) & ~yes; /* No need for a > check. */ + no |= (a->d[5] < SECP256K1_N_H_5) & ~yes; /* No need for a > check. */ + no |= (a->d[4] < SECP256K1_N_H_4) & ~yes; /* No need for a > check. */ + no |= (a->d[3] < SECP256K1_N_H_3) & ~yes; + yes |= (a->d[3] > SECP256K1_N_H_3) & ~no; + no |= (a->d[2] < SECP256K1_N_H_2) & ~yes; + yes |= (a->d[2] > SECP256K1_N_H_2) & ~no; + no |= (a->d[1] < SECP256K1_N_H_1) & ~yes; + yes |= (a->d[1] > SECP256K1_N_H_1) & ~no; + yes |= (a->d[0] > SECP256K1_N_H_0) & ~no; + return yes; +} + +static int secp256k1_scalar_cond_negate(secp256k1_scalar *r, int flag) { + /* If we are flag = 0, mask = 00...00 and this is a no-op; + * if we are flag = 1, mask = 11...11 and this is identical to secp256k1_scalar_negate */ + uint32_t mask = !flag - 1; + uint32_t nonzero = 0xFFFFFFFFUL * (secp256k1_scalar_is_zero(r) == 0); + uint64_t t = (uint64_t)(r->d[0] ^ mask) + ((SECP256K1_N_0 + 1) & mask); + r->d[0] = t & nonzero; t >>= 32; + t += (uint64_t)(r->d[1] ^ mask) + (SECP256K1_N_1 & mask); + r->d[1] = t & nonzero; t >>= 32; + t += (uint64_t)(r->d[2] ^ mask) + (SECP256K1_N_2 & mask); + r->d[2] = t & nonzero; t >>= 32; + t += (uint64_t)(r->d[3] ^ mask) + (SECP256K1_N_3 & mask); + r->d[3] = t & nonzero; t >>= 32; + t += (uint64_t)(r->d[4] ^ mask) + (SECP256K1_N_4 & mask); + r->d[4] = t & nonzero; t >>= 32; + t += (uint64_t)(r->d[5] ^ mask) + (SECP256K1_N_5 & mask); + r->d[5] = t & nonzero; t >>= 32; + t += (uint64_t)(r->d[6] ^ mask) + (SECP256K1_N_6 & mask); + r->d[6] = t & nonzero; t >>= 32; + t += (uint64_t)(r->d[7] ^ mask) + (SECP256K1_N_7 & mask); + r->d[7] = t & nonzero; + return 2 * (mask == 0) - 1; +} + + +/* Inspired by the macros in OpenSSL's crypto/bn/asm/x86_64-gcc.c. */ + +/** Add a*b to the number defined by (c0,c1,c2). c2 must never overflow. */ +#define muladd(a,b) { \ + uint32_t tl, th; \ + { \ + uint64_t t = (uint64_t)a * b; \ + th = t >> 32; /* at most 0xFFFFFFFE */ \ + tl = t; \ + } \ + c0 += tl; /* overflow is handled on the next line */ \ + th += (c0 < tl) ? 1 : 0; /* at most 0xFFFFFFFF */ \ + c1 += th; /* overflow is handled on the next line */ \ + c2 += (c1 < th) ? 1 : 0; /* never overflows by contract (verified in the next line) */ \ + VERIFY_CHECK((c1 >= th) || (c2 != 0)); \ +} + +/** Add a*b to the number defined by (c0,c1). c1 must never overflow. */ +#define muladd_fast(a,b) { \ + uint32_t tl, th; \ + { \ + uint64_t t = (uint64_t)a * b; \ + th = t >> 32; /* at most 0xFFFFFFFE */ \ + tl = t; \ + } \ + c0 += tl; /* overflow is handled on the next line */ \ + th += (c0 < tl) ? 1 : 0; /* at most 0xFFFFFFFF */ \ + c1 += th; /* never overflows by contract (verified in the next line) */ \ + VERIFY_CHECK(c1 >= th); \ +} + +/** Add 2*a*b to the number defined by (c0,c1,c2). c2 must never overflow. */ +#define muladd2(a,b) { \ + uint32_t tl, th, th2, tl2; \ + { \ + uint64_t t = (uint64_t)a * b; \ + th = t >> 32; /* at most 0xFFFFFFFE */ \ + tl = t; \ + } \ + th2 = th + th; /* at most 0xFFFFFFFE (in case th was 0x7FFFFFFF) */ \ + c2 += (th2 < th) ? 1 : 0; /* never overflows by contract (verified the next line) */ \ + VERIFY_CHECK((th2 >= th) || (c2 != 0)); \ + tl2 = tl + tl; /* at most 0xFFFFFFFE (in case the lowest 63 bits of tl were 0x7FFFFFFF) */ \ + th2 += (tl2 < tl) ? 1 : 0; /* at most 0xFFFFFFFF */ \ + c0 += tl2; /* overflow is handled on the next line */ \ + th2 += (c0 < tl2) ? 1 : 0; /* second overflow is handled on the next line */ \ + c2 += (c0 < tl2) & (th2 == 0); /* never overflows by contract (verified the next line) */ \ + VERIFY_CHECK((c0 >= tl2) || (th2 != 0) || (c2 != 0)); \ + c1 += th2; /* overflow is handled on the next line */ \ + c2 += (c1 < th2) ? 1 : 0; /* never overflows by contract (verified the next line) */ \ + VERIFY_CHECK((c1 >= th2) || (c2 != 0)); \ +} + +/** Add a to the number defined by (c0,c1,c2). c2 must never overflow. */ +#define sumadd(a) { \ + unsigned int over; \ + c0 += (a); /* overflow is handled on the next line */ \ + over = (c0 < (a)) ? 1 : 0; \ + c1 += over; /* overflow is handled on the next line */ \ + c2 += (c1 < over) ? 1 : 0; /* never overflows by contract */ \ +} + +/** Add a to the number defined by (c0,c1). c1 must never overflow, c2 must be zero. */ +#define sumadd_fast(a) { \ + c0 += (a); /* overflow is handled on the next line */ \ + c1 += (c0 < (a)) ? 1 : 0; /* never overflows by contract (verified the next line) */ \ + VERIFY_CHECK((c1 != 0) | (c0 >= (a))); \ + VERIFY_CHECK(c2 == 0); \ +} + +/** Extract the lowest 32 bits of (c0,c1,c2) into n, and left shift the number 32 bits. */ +#define extract(n) { \ + (n) = c0; \ + c0 = c1; \ + c1 = c2; \ + c2 = 0; \ +} + +/** Extract the lowest 32 bits of (c0,c1,c2) into n, and left shift the number 32 bits. c2 is required to be zero. */ +#define extract_fast(n) { \ + (n) = c0; \ + c0 = c1; \ + c1 = 0; \ + VERIFY_CHECK(c2 == 0); \ +} + +static void secp256k1_scalar_reduce_512(secp256k1_scalar *r, const uint32_t *l) { + uint64_t c; + uint32_t n0 = l[8], n1 = l[9], n2 = l[10], n3 = l[11], n4 = l[12], n5 = l[13], n6 = l[14], n7 = l[15]; + uint32_t m0, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, m12; + uint32_t p0, p1, p2, p3, p4, p5, p6, p7, p8; + + /* 96 bit accumulator. */ + uint32_t c0, c1, c2; + + /* Reduce 512 bits into 385. */ + /* m[0..12] = l[0..7] + n[0..7] * SECP256K1_N_C. */ + c0 = l[0]; c1 = 0; c2 = 0; + muladd_fast(n0, SECP256K1_N_C_0); + extract_fast(m0); + sumadd_fast(l[1]); + muladd(n1, SECP256K1_N_C_0); + muladd(n0, SECP256K1_N_C_1); + extract(m1); + sumadd(l[2]); + muladd(n2, SECP256K1_N_C_0); + muladd(n1, SECP256K1_N_C_1); + muladd(n0, SECP256K1_N_C_2); + extract(m2); + sumadd(l[3]); + muladd(n3, SECP256K1_N_C_0); + muladd(n2, SECP256K1_N_C_1); + muladd(n1, SECP256K1_N_C_2); + muladd(n0, SECP256K1_N_C_3); + extract(m3); + sumadd(l[4]); + muladd(n4, SECP256K1_N_C_0); + muladd(n3, SECP256K1_N_C_1); + muladd(n2, SECP256K1_N_C_2); + muladd(n1, SECP256K1_N_C_3); + sumadd(n0); + extract(m4); + sumadd(l[5]); + muladd(n5, SECP256K1_N_C_0); + muladd(n4, SECP256K1_N_C_1); + muladd(n3, SECP256K1_N_C_2); + muladd(n2, SECP256K1_N_C_3); + sumadd(n1); + extract(m5); + sumadd(l[6]); + muladd(n6, SECP256K1_N_C_0); + muladd(n5, SECP256K1_N_C_1); + muladd(n4, SECP256K1_N_C_2); + muladd(n3, SECP256K1_N_C_3); + sumadd(n2); + extract(m6); + sumadd(l[7]); + muladd(n7, SECP256K1_N_C_0); + muladd(n6, SECP256K1_N_C_1); + muladd(n5, SECP256K1_N_C_2); + muladd(n4, SECP256K1_N_C_3); + sumadd(n3); + extract(m7); + muladd(n7, SECP256K1_N_C_1); + muladd(n6, SECP256K1_N_C_2); + muladd(n5, SECP256K1_N_C_3); + sumadd(n4); + extract(m8); + muladd(n7, SECP256K1_N_C_2); + muladd(n6, SECP256K1_N_C_3); + sumadd(n5); + extract(m9); + muladd(n7, SECP256K1_N_C_3); + sumadd(n6); + extract(m10); + sumadd_fast(n7); + extract_fast(m11); + VERIFY_CHECK(c0 <= 1); + m12 = c0; + + /* Reduce 385 bits into 258. */ + /* p[0..8] = m[0..7] + m[8..12] * SECP256K1_N_C. */ + c0 = m0; c1 = 0; c2 = 0; + muladd_fast(m8, SECP256K1_N_C_0); + extract_fast(p0); + sumadd_fast(m1); + muladd(m9, SECP256K1_N_C_0); + muladd(m8, SECP256K1_N_C_1); + extract(p1); + sumadd(m2); + muladd(m10, SECP256K1_N_C_0); + muladd(m9, SECP256K1_N_C_1); + muladd(m8, SECP256K1_N_C_2); + extract(p2); + sumadd(m3); + muladd(m11, SECP256K1_N_C_0); + muladd(m10, SECP256K1_N_C_1); + muladd(m9, SECP256K1_N_C_2); + muladd(m8, SECP256K1_N_C_3); + extract(p3); + sumadd(m4); + muladd(m12, SECP256K1_N_C_0); + muladd(m11, SECP256K1_N_C_1); + muladd(m10, SECP256K1_N_C_2); + muladd(m9, SECP256K1_N_C_3); + sumadd(m8); + extract(p4); + sumadd(m5); + muladd(m12, SECP256K1_N_C_1); + muladd(m11, SECP256K1_N_C_2); + muladd(m10, SECP256K1_N_C_3); + sumadd(m9); + extract(p5); + sumadd(m6); + muladd(m12, SECP256K1_N_C_2); + muladd(m11, SECP256K1_N_C_3); + sumadd(m10); + extract(p6); + sumadd_fast(m7); + muladd_fast(m12, SECP256K1_N_C_3); + sumadd_fast(m11); + extract_fast(p7); + p8 = c0 + m12; + VERIFY_CHECK(p8 <= 2); + + /* Reduce 258 bits into 256. */ + /* r[0..7] = p[0..7] + p[8] * SECP256K1_N_C. */ + c = p0 + (uint64_t)SECP256K1_N_C_0 * p8; + r->d[0] = c & 0xFFFFFFFFUL; c >>= 32; + c += p1 + (uint64_t)SECP256K1_N_C_1 * p8; + r->d[1] = c & 0xFFFFFFFFUL; c >>= 32; + c += p2 + (uint64_t)SECP256K1_N_C_2 * p8; + r->d[2] = c & 0xFFFFFFFFUL; c >>= 32; + c += p3 + (uint64_t)SECP256K1_N_C_3 * p8; + r->d[3] = c & 0xFFFFFFFFUL; c >>= 32; + c += p4 + (uint64_t)p8; + r->d[4] = c & 0xFFFFFFFFUL; c >>= 32; + c += p5; + r->d[5] = c & 0xFFFFFFFFUL; c >>= 32; + c += p6; + r->d[6] = c & 0xFFFFFFFFUL; c >>= 32; + c += p7; + r->d[7] = c & 0xFFFFFFFFUL; c >>= 32; + + /* Final reduction of r. */ + secp256k1_scalar_reduce(r, c + secp256k1_scalar_check_overflow(r)); +} + +static void secp256k1_scalar_mul_512(uint32_t *l, const secp256k1_scalar *a, const secp256k1_scalar *b) { + /* 96 bit accumulator. */ + uint32_t c0 = 0, c1 = 0, c2 = 0; + + /* l[0..15] = a[0..7] * b[0..7]. */ + muladd_fast(a->d[0], b->d[0]); + extract_fast(l[0]); + muladd(a->d[0], b->d[1]); + muladd(a->d[1], b->d[0]); + extract(l[1]); + muladd(a->d[0], b->d[2]); + muladd(a->d[1], b->d[1]); + muladd(a->d[2], b->d[0]); + extract(l[2]); + muladd(a->d[0], b->d[3]); + muladd(a->d[1], b->d[2]); + muladd(a->d[2], b->d[1]); + muladd(a->d[3], b->d[0]); + extract(l[3]); + muladd(a->d[0], b->d[4]); + muladd(a->d[1], b->d[3]); + muladd(a->d[2], b->d[2]); + muladd(a->d[3], b->d[1]); + muladd(a->d[4], b->d[0]); + extract(l[4]); + muladd(a->d[0], b->d[5]); + muladd(a->d[1], b->d[4]); + muladd(a->d[2], b->d[3]); + muladd(a->d[3], b->d[2]); + muladd(a->d[4], b->d[1]); + muladd(a->d[5], b->d[0]); + extract(l[5]); + muladd(a->d[0], b->d[6]); + muladd(a->d[1], b->d[5]); + muladd(a->d[2], b->d[4]); + muladd(a->d[3], b->d[3]); + muladd(a->d[4], b->d[2]); + muladd(a->d[5], b->d[1]); + muladd(a->d[6], b->d[0]); + extract(l[6]); + muladd(a->d[0], b->d[7]); + muladd(a->d[1], b->d[6]); + muladd(a->d[2], b->d[5]); + muladd(a->d[3], b->d[4]); + muladd(a->d[4], b->d[3]); + muladd(a->d[5], b->d[2]); + muladd(a->d[6], b->d[1]); + muladd(a->d[7], b->d[0]); + extract(l[7]); + muladd(a->d[1], b->d[7]); + muladd(a->d[2], b->d[6]); + muladd(a->d[3], b->d[5]); + muladd(a->d[4], b->d[4]); + muladd(a->d[5], b->d[3]); + muladd(a->d[6], b->d[2]); + muladd(a->d[7], b->d[1]); + extract(l[8]); + muladd(a->d[2], b->d[7]); + muladd(a->d[3], b->d[6]); + muladd(a->d[4], b->d[5]); + muladd(a->d[5], b->d[4]); + muladd(a->d[6], b->d[3]); + muladd(a->d[7], b->d[2]); + extract(l[9]); + muladd(a->d[3], b->d[7]); + muladd(a->d[4], b->d[6]); + muladd(a->d[5], b->d[5]); + muladd(a->d[6], b->d[4]); + muladd(a->d[7], b->d[3]); + extract(l[10]); + muladd(a->d[4], b->d[7]); + muladd(a->d[5], b->d[6]); + muladd(a->d[6], b->d[5]); + muladd(a->d[7], b->d[4]); + extract(l[11]); + muladd(a->d[5], b->d[7]); + muladd(a->d[6], b->d[6]); + muladd(a->d[7], b->d[5]); + extract(l[12]); + muladd(a->d[6], b->d[7]); + muladd(a->d[7], b->d[6]); + extract(l[13]); + muladd_fast(a->d[7], b->d[7]); + extract_fast(l[14]); + VERIFY_CHECK(c1 == 0); + l[15] = c0; +} + +static void secp256k1_scalar_sqr_512(uint32_t *l, const secp256k1_scalar *a) { + /* 96 bit accumulator. */ + uint32_t c0 = 0, c1 = 0, c2 = 0; + + /* l[0..15] = a[0..7]^2. */ + muladd_fast(a->d[0], a->d[0]); + extract_fast(l[0]); + muladd2(a->d[0], a->d[1]); + extract(l[1]); + muladd2(a->d[0], a->d[2]); + muladd(a->d[1], a->d[1]); + extract(l[2]); + muladd2(a->d[0], a->d[3]); + muladd2(a->d[1], a->d[2]); + extract(l[3]); + muladd2(a->d[0], a->d[4]); + muladd2(a->d[1], a->d[3]); + muladd(a->d[2], a->d[2]); + extract(l[4]); + muladd2(a->d[0], a->d[5]); + muladd2(a->d[1], a->d[4]); + muladd2(a->d[2], a->d[3]); + extract(l[5]); + muladd2(a->d[0], a->d[6]); + muladd2(a->d[1], a->d[5]); + muladd2(a->d[2], a->d[4]); + muladd(a->d[3], a->d[3]); + extract(l[6]); + muladd2(a->d[0], a->d[7]); + muladd2(a->d[1], a->d[6]); + muladd2(a->d[2], a->d[5]); + muladd2(a->d[3], a->d[4]); + extract(l[7]); + muladd2(a->d[1], a->d[7]); + muladd2(a->d[2], a->d[6]); + muladd2(a->d[3], a->d[5]); + muladd(a->d[4], a->d[4]); + extract(l[8]); + muladd2(a->d[2], a->d[7]); + muladd2(a->d[3], a->d[6]); + muladd2(a->d[4], a->d[5]); + extract(l[9]); + muladd2(a->d[3], a->d[7]); + muladd2(a->d[4], a->d[6]); + muladd(a->d[5], a->d[5]); + extract(l[10]); + muladd2(a->d[4], a->d[7]); + muladd2(a->d[5], a->d[6]); + extract(l[11]); + muladd2(a->d[5], a->d[7]); + muladd(a->d[6], a->d[6]); + extract(l[12]); + muladd2(a->d[6], a->d[7]); + extract(l[13]); + muladd_fast(a->d[7], a->d[7]); + extract_fast(l[14]); + VERIFY_CHECK(c1 == 0); + l[15] = c0; +} + +#undef sumadd +#undef sumadd_fast +#undef muladd +#undef muladd_fast +#undef muladd2 +#undef extract +#undef extract_fast + +static void secp256k1_scalar_mul(secp256k1_scalar *r, const secp256k1_scalar *a, const secp256k1_scalar *b) { + uint32_t l[16]; + secp256k1_scalar_mul_512(l, a, b); + secp256k1_scalar_reduce_512(r, l); +} + +static int secp256k1_scalar_shr_int(secp256k1_scalar *r, int n) { + int ret; + VERIFY_CHECK(n > 0); + VERIFY_CHECK(n < 16); + ret = r->d[0] & ((1 << n) - 1); + r->d[0] = (r->d[0] >> n) + (r->d[1] << (32 - n)); + r->d[1] = (r->d[1] >> n) + (r->d[2] << (32 - n)); + r->d[2] = (r->d[2] >> n) + (r->d[3] << (32 - n)); + r->d[3] = (r->d[3] >> n) + (r->d[4] << (32 - n)); + r->d[4] = (r->d[4] >> n) + (r->d[5] << (32 - n)); + r->d[5] = (r->d[5] >> n) + (r->d[6] << (32 - n)); + r->d[6] = (r->d[6] >> n) + (r->d[7] << (32 - n)); + r->d[7] = (r->d[7] >> n); + return ret; +} + +static void secp256k1_scalar_sqr(secp256k1_scalar *r, const secp256k1_scalar *a) { + uint32_t l[16]; + secp256k1_scalar_sqr_512(l, a); + secp256k1_scalar_reduce_512(r, l); +} + +#ifdef USE_ENDOMORPHISM +static void secp256k1_scalar_split_128(secp256k1_scalar *r1, secp256k1_scalar *r2, const secp256k1_scalar *a) { + r1->d[0] = a->d[0]; + r1->d[1] = a->d[1]; + r1->d[2] = a->d[2]; + r1->d[3] = a->d[3]; + r1->d[4] = 0; + r1->d[5] = 0; + r1->d[6] = 0; + r1->d[7] = 0; + r2->d[0] = a->d[4]; + r2->d[1] = a->d[5]; + r2->d[2] = a->d[6]; + r2->d[3] = a->d[7]; + r2->d[4] = 0; + r2->d[5] = 0; + r2->d[6] = 0; + r2->d[7] = 0; +} +#endif + +SECP256K1_INLINE static int secp256k1_scalar_eq(const secp256k1_scalar *a, const secp256k1_scalar *b) { + return ((a->d[0] ^ b->d[0]) | (a->d[1] ^ b->d[1]) | (a->d[2] ^ b->d[2]) | (a->d[3] ^ b->d[3]) | (a->d[4] ^ b->d[4]) | (a->d[5] ^ b->d[5]) | (a->d[6] ^ b->d[6]) | (a->d[7] ^ b->d[7])) == 0; +} + +SECP256K1_INLINE static void secp256k1_scalar_mul_shift_var(secp256k1_scalar *r, const secp256k1_scalar *a, const secp256k1_scalar *b, unsigned int shift) { + uint32_t l[16]; + unsigned int shiftlimbs; + unsigned int shiftlow; + unsigned int shifthigh; + VERIFY_CHECK(shift >= 256); + secp256k1_scalar_mul_512(l, a, b); + shiftlimbs = shift >> 5; + shiftlow = shift & 0x1F; + shifthigh = 32 - shiftlow; + r->d[0] = shift < 512 ? (l[0 + shiftlimbs] >> shiftlow | (shift < 480 && shiftlow ? (l[1 + shiftlimbs] << shifthigh) : 0)) : 0; + r->d[1] = shift < 480 ? (l[1 + shiftlimbs] >> shiftlow | (shift < 448 && shiftlow ? (l[2 + shiftlimbs] << shifthigh) : 0)) : 0; + r->d[2] = shift < 448 ? (l[2 + shiftlimbs] >> shiftlow | (shift < 416 && shiftlow ? (l[3 + shiftlimbs] << shifthigh) : 0)) : 0; + r->d[3] = shift < 416 ? (l[3 + shiftlimbs] >> shiftlow | (shift < 384 && shiftlow ? (l[4 + shiftlimbs] << shifthigh) : 0)) : 0; + r->d[4] = shift < 384 ? (l[4 + shiftlimbs] >> shiftlow | (shift < 352 && shiftlow ? (l[5 + shiftlimbs] << shifthigh) : 0)) : 0; + r->d[5] = shift < 352 ? (l[5 + shiftlimbs] >> shiftlow | (shift < 320 && shiftlow ? (l[6 + shiftlimbs] << shifthigh) : 0)) : 0; + r->d[6] = shift < 320 ? (l[6 + shiftlimbs] >> shiftlow | (shift < 288 && shiftlow ? (l[7 + shiftlimbs] << shifthigh) : 0)) : 0; + r->d[7] = shift < 288 ? (l[7 + shiftlimbs] >> shiftlow) : 0; + secp256k1_scalar_cadd_bit(r, 0, (l[(shift - 1) >> 5] >> ((shift - 1) & 0x1f)) & 1); +} + +#endif diff --git a/src/secp256k1/src/scalar_impl.h b/src/secp256k1/src/scalar_impl.h new file mode 100644 index 00000000..88ea97de --- /dev/null +++ b/src/secp256k1/src/scalar_impl.h @@ -0,0 +1,337 @@ +/********************************************************************** + * Copyright (c) 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_SCALAR_IMPL_H_ +#define _SECP256K1_SCALAR_IMPL_H_ + +#include + +#include "group.h" +#include "scalar.h" + +#if defined HAVE_CONFIG_H +#include "libsecp256k1-config.h" +#endif + +#if defined(USE_SCALAR_4X64) +#include "scalar_4x64_impl.h" +#elif defined(USE_SCALAR_8X32) +#include "scalar_8x32_impl.h" +#else +#error "Please select scalar implementation" +#endif + +#ifndef USE_NUM_NONE +static void secp256k1_scalar_get_num(secp256k1_num *r, const secp256k1_scalar *a) { + unsigned char c[32]; + secp256k1_scalar_get_b32(c, a); + secp256k1_num_set_bin(r, c, 32); +} + +/** secp256k1 curve order, see secp256k1_ecdsa_const_order_as_fe in ecdsa_impl.h */ +static void secp256k1_scalar_order_get_num(secp256k1_num *r) { + static const unsigned char order[32] = { + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFE, + 0xBA,0xAE,0xDC,0xE6,0xAF,0x48,0xA0,0x3B, + 0xBF,0xD2,0x5E,0x8C,0xD0,0x36,0x41,0x41 + }; + secp256k1_num_set_bin(r, order, 32); +} +#endif + +static void secp256k1_scalar_inverse(secp256k1_scalar *r, const secp256k1_scalar *x) { + secp256k1_scalar *t; + int i; + /* First compute x ^ (2^N - 1) for some values of N. */ + secp256k1_scalar x2, x3, x4, x6, x7, x8, x15, x30, x60, x120, x127; + + secp256k1_scalar_sqr(&x2, x); + secp256k1_scalar_mul(&x2, &x2, x); + + secp256k1_scalar_sqr(&x3, &x2); + secp256k1_scalar_mul(&x3, &x3, x); + + secp256k1_scalar_sqr(&x4, &x3); + secp256k1_scalar_mul(&x4, &x4, x); + + secp256k1_scalar_sqr(&x6, &x4); + secp256k1_scalar_sqr(&x6, &x6); + secp256k1_scalar_mul(&x6, &x6, &x2); + + secp256k1_scalar_sqr(&x7, &x6); + secp256k1_scalar_mul(&x7, &x7, x); + + secp256k1_scalar_sqr(&x8, &x7); + secp256k1_scalar_mul(&x8, &x8, x); + + secp256k1_scalar_sqr(&x15, &x8); + for (i = 0; i < 6; i++) { + secp256k1_scalar_sqr(&x15, &x15); + } + secp256k1_scalar_mul(&x15, &x15, &x7); + + secp256k1_scalar_sqr(&x30, &x15); + for (i = 0; i < 14; i++) { + secp256k1_scalar_sqr(&x30, &x30); + } + secp256k1_scalar_mul(&x30, &x30, &x15); + + secp256k1_scalar_sqr(&x60, &x30); + for (i = 0; i < 29; i++) { + secp256k1_scalar_sqr(&x60, &x60); + } + secp256k1_scalar_mul(&x60, &x60, &x30); + + secp256k1_scalar_sqr(&x120, &x60); + for (i = 0; i < 59; i++) { + secp256k1_scalar_sqr(&x120, &x120); + } + secp256k1_scalar_mul(&x120, &x120, &x60); + + secp256k1_scalar_sqr(&x127, &x120); + for (i = 0; i < 6; i++) { + secp256k1_scalar_sqr(&x127, &x127); + } + secp256k1_scalar_mul(&x127, &x127, &x7); + + /* Then accumulate the final result (t starts at x127). */ + t = &x127; + for (i = 0; i < 2; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 4; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x3); /* 111 */ + for (i = 0; i < 2; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 2; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 2; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 4; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x3); /* 111 */ + for (i = 0; i < 3; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x2); /* 11 */ + for (i = 0; i < 4; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x3); /* 111 */ + for (i = 0; i < 5; i++) { /* 00 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x3); /* 111 */ + for (i = 0; i < 4; i++) { /* 00 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x2); /* 11 */ + for (i = 0; i < 2; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 2; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 5; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x4); /* 1111 */ + for (i = 0; i < 2; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 3; i++) { /* 00 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 4; i++) { /* 000 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 2; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 10; i++) { /* 0000000 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x3); /* 111 */ + for (i = 0; i < 4; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x3); /* 111 */ + for (i = 0; i < 9; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x8); /* 11111111 */ + for (i = 0; i < 2; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 3; i++) { /* 00 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 3; i++) { /* 00 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 5; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x4); /* 1111 */ + for (i = 0; i < 2; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 5; i++) { /* 000 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x2); /* 11 */ + for (i = 0; i < 4; i++) { /* 00 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x2); /* 11 */ + for (i = 0; i < 2; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 8; i++) { /* 000000 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x2); /* 11 */ + for (i = 0; i < 3; i++) { /* 0 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, &x2); /* 11 */ + for (i = 0; i < 3; i++) { /* 00 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 6; i++) { /* 00000 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(t, t, x); /* 1 */ + for (i = 0; i < 8; i++) { /* 00 */ + secp256k1_scalar_sqr(t, t); + } + secp256k1_scalar_mul(r, t, &x6); /* 111111 */ +} + +SECP256K1_INLINE static int secp256k1_scalar_is_even(const secp256k1_scalar *a) { + /* d[0] is present and is the lowest word for all representations */ + return !(a->d[0] & 1); +} + +static void secp256k1_scalar_inverse_var(secp256k1_scalar *r, const secp256k1_scalar *x) { +#if defined(USE_SCALAR_INV_BUILTIN) + secp256k1_scalar_inverse(r, x); +#elif defined(USE_SCALAR_INV_NUM) + unsigned char b[32]; + secp256k1_num n, m; + secp256k1_scalar t = *x; + secp256k1_scalar_get_b32(b, &t); + secp256k1_num_set_bin(&n, b, 32); + secp256k1_scalar_order_get_num(&m); + secp256k1_num_mod_inverse(&n, &n, &m); + secp256k1_num_get_bin(b, 32, &n); + secp256k1_scalar_set_b32(r, b, NULL); + /* Verify that the inverse was computed correctly, without GMP code. */ + secp256k1_scalar_mul(&t, &t, r); + CHECK(secp256k1_scalar_is_one(&t)); +#else +#error "Please select scalar inverse implementation" +#endif +} + +#ifdef USE_ENDOMORPHISM +/** + * The Secp256k1 curve has an endomorphism, where lambda * (x, y) = (beta * x, y), where + * lambda is {0x53,0x63,0xad,0x4c,0xc0,0x5c,0x30,0xe0,0xa5,0x26,0x1c,0x02,0x88,0x12,0x64,0x5a, + * 0x12,0x2e,0x22,0xea,0x20,0x81,0x66,0x78,0xdf,0x02,0x96,0x7c,0x1b,0x23,0xbd,0x72} + * + * "Guide to Elliptic Curve Cryptography" (Hankerson, Menezes, Vanstone) gives an algorithm + * (algorithm 3.74) to find k1 and k2 given k, such that k1 + k2 * lambda == k mod n, and k1 + * and k2 have a small size. + * It relies on constants a1, b1, a2, b2. These constants for the value of lambda above are: + * + * - a1 = {0x30,0x86,0xd2,0x21,0xa7,0xd4,0x6b,0xcd,0xe8,0x6c,0x90,0xe4,0x92,0x84,0xeb,0x15} + * - b1 = -{0xe4,0x43,0x7e,0xd6,0x01,0x0e,0x88,0x28,0x6f,0x54,0x7f,0xa9,0x0a,0xbf,0xe4,0xc3} + * - a2 = {0x01,0x14,0xca,0x50,0xf7,0xa8,0xe2,0xf3,0xf6,0x57,0xc1,0x10,0x8d,0x9d,0x44,0xcf,0xd8} + * - b2 = {0x30,0x86,0xd2,0x21,0xa7,0xd4,0x6b,0xcd,0xe8,0x6c,0x90,0xe4,0x92,0x84,0xeb,0x15} + * + * The algorithm then computes c1 = round(b1 * k / n) and c2 = round(b2 * k / n), and gives + * k1 = k - (c1*a1 + c2*a2) and k2 = -(c1*b1 + c2*b2). Instead, we use modular arithmetic, and + * compute k1 as k - k2 * lambda, avoiding the need for constants a1 and a2. + * + * g1, g2 are precomputed constants used to replace division with a rounded multiplication + * when decomposing the scalar for an endomorphism-based point multiplication. + * + * The possibility of using precomputed estimates is mentioned in "Guide to Elliptic Curve + * Cryptography" (Hankerson, Menezes, Vanstone) in section 3.5. + * + * The derivation is described in the paper "Efficient Software Implementation of Public-Key + * Cryptography on Sensor Networks Using the MSP430X Microcontroller" (Gouvea, Oliveira, Lopez), + * Section 4.3 (here we use a somewhat higher-precision estimate): + * d = a1*b2 - b1*a2 + * g1 = round((2^272)*b2/d) + * g2 = round((2^272)*b1/d) + * + * (Note that 'd' is also equal to the curve order here because [a1,b1] and [a2,b2] are found + * as outputs of the Extended Euclidean Algorithm on inputs 'order' and 'lambda'). + * + * The function below splits a in r1 and r2, such that r1 + lambda * r2 == a (mod order). + */ + +static void secp256k1_scalar_split_lambda(secp256k1_scalar *r1, secp256k1_scalar *r2, const secp256k1_scalar *a) { + secp256k1_scalar c1, c2; + static const secp256k1_scalar minus_lambda = SECP256K1_SCALAR_CONST( + 0xAC9C52B3UL, 0x3FA3CF1FUL, 0x5AD9E3FDUL, 0x77ED9BA4UL, + 0xA880B9FCUL, 0x8EC739C2UL, 0xE0CFC810UL, 0xB51283CFUL + ); + static const secp256k1_scalar minus_b1 = SECP256K1_SCALAR_CONST( + 0x00000000UL, 0x00000000UL, 0x00000000UL, 0x00000000UL, + 0xE4437ED6UL, 0x010E8828UL, 0x6F547FA9UL, 0x0ABFE4C3UL + ); + static const secp256k1_scalar minus_b2 = SECP256K1_SCALAR_CONST( + 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFEUL, + 0x8A280AC5UL, 0x0774346DUL, 0xD765CDA8UL, 0x3DB1562CUL + ); + static const secp256k1_scalar g1 = SECP256K1_SCALAR_CONST( + 0x00000000UL, 0x00000000UL, 0x00000000UL, 0x00003086UL, + 0xD221A7D4UL, 0x6BCDE86CUL, 0x90E49284UL, 0xEB153DABUL + ); + static const secp256k1_scalar g2 = SECP256K1_SCALAR_CONST( + 0x00000000UL, 0x00000000UL, 0x00000000UL, 0x0000E443UL, + 0x7ED6010EUL, 0x88286F54UL, 0x7FA90ABFUL, 0xE4C42212UL + ); + VERIFY_CHECK(r1 != a); + VERIFY_CHECK(r2 != a); + /* these _var calls are constant time since the shift amount is constant */ + secp256k1_scalar_mul_shift_var(&c1, a, &g1, 272); + secp256k1_scalar_mul_shift_var(&c2, a, &g2, 272); + secp256k1_scalar_mul(&c1, &c1, &minus_b1); + secp256k1_scalar_mul(&c2, &c2, &minus_b2); + secp256k1_scalar_add(r2, &c1, &c2); + secp256k1_scalar_mul(r1, r2, &minus_lambda); + secp256k1_scalar_add(r1, r1, a); +} +#endif + +#endif diff --git a/src/secp256k1/src/secp256k1.c b/src/secp256k1/src/secp256k1.c new file mode 100644 index 00000000..62d192ba --- /dev/null +++ b/src/secp256k1/src/secp256k1.c @@ -0,0 +1,568 @@ +/********************************************************************** + * Copyright (c) 2013-2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#define SECP256K1_BUILD (1) + +#include "include/secp256k1.h" + +#include "util.h" +#include "num_impl.h" +#include "field_impl.h" +#include "scalar_impl.h" +#include "group_impl.h" +#include "ecmult_impl.h" +#include "ecmult_const_impl.h" +#include "ecmult_gen_impl.h" +#include "ecdsa_impl.h" +#include "eckey_impl.h" +#include "hash_impl.h" + +#define ARG_CHECK(cond) do { \ + if (EXPECT(!(cond), 0)) { \ + secp256k1_callback_call(&ctx->illegal_callback, #cond); \ + return 0; \ + } \ +} while(0) + +static void default_illegal_callback_fn(const char* str, void* data) { + (void)data; + fprintf(stderr, "[libsecp256k1] illegal argument: %s\n", str); + abort(); +} + +static const secp256k1_callback default_illegal_callback = { + default_illegal_callback_fn, + NULL +}; + +static void default_error_callback_fn(const char* str, void* data) { + (void)data; + fprintf(stderr, "[libsecp256k1] internal consistency check failed: %s\n", str); + abort(); +} + +static const secp256k1_callback default_error_callback = { + default_error_callback_fn, + NULL +}; + + +struct secp256k1_context_struct { + secp256k1_ecmult_context ecmult_ctx; + secp256k1_ecmult_gen_context ecmult_gen_ctx; + secp256k1_callback illegal_callback; + secp256k1_callback error_callback; +}; + +secp256k1_context* secp256k1_context_create(unsigned int flags) { + secp256k1_context* ret = (secp256k1_context*)checked_malloc(&default_error_callback, sizeof(secp256k1_context)); + ret->illegal_callback = default_illegal_callback; + ret->error_callback = default_error_callback; + + if (EXPECT((flags & SECP256K1_FLAGS_TYPE_MASK) != SECP256K1_FLAGS_TYPE_CONTEXT, 0)) { + secp256k1_callback_call(&ret->illegal_callback, + "Invalid flags"); + free(ret); + return NULL; + } + + secp256k1_ecmult_context_init(&ret->ecmult_ctx); + secp256k1_ecmult_gen_context_init(&ret->ecmult_gen_ctx); + + if (flags & SECP256K1_FLAGS_BIT_CONTEXT_SIGN) { + secp256k1_ecmult_gen_context_build(&ret->ecmult_gen_ctx, &ret->error_callback); + } + if (flags & SECP256K1_FLAGS_BIT_CONTEXT_VERIFY) { + secp256k1_ecmult_context_build(&ret->ecmult_ctx, &ret->error_callback); + } + + return ret; +} + +secp256k1_context* secp256k1_context_clone(const secp256k1_context* ctx) { + secp256k1_context* ret = (secp256k1_context*)checked_malloc(&ctx->error_callback, sizeof(secp256k1_context)); + ret->illegal_callback = ctx->illegal_callback; + ret->error_callback = ctx->error_callback; + secp256k1_ecmult_context_clone(&ret->ecmult_ctx, &ctx->ecmult_ctx, &ctx->error_callback); + secp256k1_ecmult_gen_context_clone(&ret->ecmult_gen_ctx, &ctx->ecmult_gen_ctx, &ctx->error_callback); + return ret; +} + +void secp256k1_context_destroy(secp256k1_context* ctx) { + if (ctx != NULL) { + secp256k1_ecmult_context_clear(&ctx->ecmult_ctx); + secp256k1_ecmult_gen_context_clear(&ctx->ecmult_gen_ctx); + + free(ctx); + } +} + +void secp256k1_context_set_illegal_callback(secp256k1_context* ctx, void (*fun)(const char* message, void* data), const void* data) { + if (fun == NULL) { + fun = default_illegal_callback_fn; + } + ctx->illegal_callback.fn = fun; + ctx->illegal_callback.data = data; +} + +void secp256k1_context_set_error_callback(secp256k1_context* ctx, void (*fun)(const char* message, void* data), const void* data) { + if (fun == NULL) { + fun = default_error_callback_fn; + } + ctx->error_callback.fn = fun; + ctx->error_callback.data = data; +} + +static int secp256k1_pubkey_load(const secp256k1_context* ctx, secp256k1_ge* ge, const secp256k1_pubkey* pubkey) { + if (sizeof(secp256k1_ge_storage) == 64) { + /* When the secp256k1_ge_storage type is exactly 64 byte, use its + * representation inside secp256k1_pubkey, as conversion is very fast. + * Note that secp256k1_pubkey_save must use the same representation. */ + secp256k1_ge_storage s; + memcpy(&s, &pubkey->data[0], 64); + secp256k1_ge_from_storage(ge, &s); + } else { + /* Otherwise, fall back to 32-byte big endian for X and Y. */ + secp256k1_fe x, y; + secp256k1_fe_set_b32(&x, pubkey->data); + secp256k1_fe_set_b32(&y, pubkey->data + 32); + secp256k1_ge_set_xy(ge, &x, &y); + } + ARG_CHECK(!secp256k1_fe_is_zero(&ge->x)); + return 1; +} + +static void secp256k1_pubkey_save(secp256k1_pubkey* pubkey, secp256k1_ge* ge) { + if (sizeof(secp256k1_ge_storage) == 64) { + secp256k1_ge_storage s; + secp256k1_ge_to_storage(&s, ge); + memcpy(&pubkey->data[0], &s, 64); + } else { + VERIFY_CHECK(!secp256k1_ge_is_infinity(ge)); + secp256k1_fe_normalize_var(&ge->x); + secp256k1_fe_normalize_var(&ge->y); + secp256k1_fe_get_b32(pubkey->data, &ge->x); + secp256k1_fe_get_b32(pubkey->data + 32, &ge->y); + } +} + +int secp256k1_ec_pubkey_parse(const secp256k1_context* ctx, secp256k1_pubkey* pubkey, const unsigned char *input, size_t inputlen) { + secp256k1_ge Q; + + (void)ctx; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(pubkey != NULL); + memset(pubkey, 0, sizeof(*pubkey)); + ARG_CHECK(input != NULL); + if (!secp256k1_eckey_pubkey_parse(&Q, input, inputlen)) { + return 0; + } + secp256k1_pubkey_save(pubkey, &Q); + secp256k1_ge_clear(&Q); + return 1; +} + +int secp256k1_ec_pubkey_serialize(const secp256k1_context* ctx, unsigned char *output, size_t *outputlen, const secp256k1_pubkey* pubkey, unsigned int flags) { + secp256k1_ge Q; + size_t len; + int ret = 0; + + (void)ctx; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(outputlen != NULL); + ARG_CHECK(*outputlen >= ((flags & SECP256K1_FLAGS_BIT_COMPRESSION) ? 33 : 65)); + len = *outputlen; + *outputlen = 0; + ARG_CHECK(output != NULL); + memset(output, 0, len); + ARG_CHECK(pubkey != NULL); + ARG_CHECK((flags & SECP256K1_FLAGS_TYPE_MASK) == SECP256K1_FLAGS_TYPE_COMPRESSION); + if (secp256k1_pubkey_load(ctx, &Q, pubkey)) { + ret = secp256k1_eckey_pubkey_serialize(&Q, output, &len, flags & SECP256K1_FLAGS_BIT_COMPRESSION); + if (ret) { + *outputlen = len; + } + } + return ret; +} + +static void secp256k1_ecdsa_signature_load(const secp256k1_context* ctx, secp256k1_scalar* r, secp256k1_scalar* s, const secp256k1_ecdsa_signature* sig) { + (void)ctx; + if (sizeof(secp256k1_scalar) == 32) { + /* When the secp256k1_scalar type is exactly 32 byte, use its + * representation inside secp256k1_ecdsa_signature, as conversion is very fast. + * Note that secp256k1_ecdsa_signature_save must use the same representation. */ + memcpy(r, &sig->data[0], 32); + memcpy(s, &sig->data[32], 32); + } else { + secp256k1_scalar_set_b32(r, &sig->data[0], NULL); + secp256k1_scalar_set_b32(s, &sig->data[32], NULL); + } +} + +static void secp256k1_ecdsa_signature_save(secp256k1_ecdsa_signature* sig, const secp256k1_scalar* r, const secp256k1_scalar* s) { + if (sizeof(secp256k1_scalar) == 32) { + memcpy(&sig->data[0], r, 32); + memcpy(&sig->data[32], s, 32); + } else { + secp256k1_scalar_get_b32(&sig->data[0], r); + secp256k1_scalar_get_b32(&sig->data[32], s); + } +} + +int secp256k1_ecdsa_signature_parse_der(const secp256k1_context* ctx, secp256k1_ecdsa_signature* sig, const unsigned char *input, size_t inputlen) { + secp256k1_scalar r, s; + + (void)ctx; + ARG_CHECK(sig != NULL); + ARG_CHECK(input != NULL); + + if (secp256k1_ecdsa_sig_parse(&r, &s, input, inputlen)) { + secp256k1_ecdsa_signature_save(sig, &r, &s); + return 1; + } else { + memset(sig, 0, sizeof(*sig)); + return 0; + } +} + +int secp256k1_ecdsa_signature_parse_compact(const secp256k1_context* ctx, secp256k1_ecdsa_signature* sig, const unsigned char *input64) { + secp256k1_scalar r, s; + int ret = 1; + int overflow = 0; + + (void)ctx; + ARG_CHECK(sig != NULL); + ARG_CHECK(input64 != NULL); + + secp256k1_scalar_set_b32(&r, &input64[0], &overflow); + ret &= !overflow; + secp256k1_scalar_set_b32(&s, &input64[32], &overflow); + ret &= !overflow; + if (ret) { + secp256k1_ecdsa_signature_save(sig, &r, &s); + } else { + memset(sig, 0, sizeof(*sig)); + } + return ret; +} + +int secp256k1_ecdsa_signature_serialize_der(const secp256k1_context* ctx, unsigned char *output, size_t *outputlen, const secp256k1_ecdsa_signature* sig) { + secp256k1_scalar r, s; + + (void)ctx; + ARG_CHECK(output != NULL); + ARG_CHECK(outputlen != NULL); + ARG_CHECK(sig != NULL); + + secp256k1_ecdsa_signature_load(ctx, &r, &s, sig); + return secp256k1_ecdsa_sig_serialize(output, outputlen, &r, &s); +} + +int secp256k1_ecdsa_signature_serialize_compact(const secp256k1_context* ctx, unsigned char *output64, const secp256k1_ecdsa_signature* sig) { + secp256k1_scalar r, s; + + (void)ctx; + ARG_CHECK(output64 != NULL); + ARG_CHECK(sig != NULL); + + secp256k1_ecdsa_signature_load(ctx, &r, &s, sig); + secp256k1_scalar_get_b32(&output64[0], &r); + secp256k1_scalar_get_b32(&output64[32], &s); + return 1; +} + +int secp256k1_ecdsa_signature_normalize(const secp256k1_context* ctx, secp256k1_ecdsa_signature *sigout, const secp256k1_ecdsa_signature *sigin) { + secp256k1_scalar r, s; + int ret = 0; + + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(sigin != NULL); + + secp256k1_ecdsa_signature_load(ctx, &r, &s, sigin); + ret = secp256k1_scalar_is_high(&s); + if (sigout != NULL) { + if (ret) { + secp256k1_scalar_negate(&s, &s); + } + secp256k1_ecdsa_signature_save(sigout, &r, &s); + } + + return ret; +} + +int secp256k1_ecdsa_verify(const secp256k1_context* ctx, const secp256k1_ecdsa_signature *sig, const unsigned char *msg32, const secp256k1_pubkey *pubkey) { + secp256k1_ge q; + secp256k1_scalar r, s; + secp256k1_scalar m; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_context_is_built(&ctx->ecmult_ctx)); + ARG_CHECK(msg32 != NULL); + ARG_CHECK(sig != NULL); + ARG_CHECK(pubkey != NULL); + + secp256k1_scalar_set_b32(&m, msg32, NULL); + secp256k1_ecdsa_signature_load(ctx, &r, &s, sig); + return (!secp256k1_scalar_is_high(&s) && + secp256k1_pubkey_load(ctx, &q, pubkey) && + secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &r, &s, &q, &m)); +} + +static int nonce_function_rfc6979(unsigned char *nonce32, const unsigned char *msg32, const unsigned char *key32, const unsigned char *algo16, void *data, unsigned int counter) { + unsigned char keydata[112]; + int keylen = 64; + secp256k1_rfc6979_hmac_sha256_t rng; + unsigned int i; + /* We feed a byte array to the PRNG as input, consisting of: + * - the private key (32 bytes) and message (32 bytes), see RFC 6979 3.2d. + * - optionally 32 extra bytes of data, see RFC 6979 3.6 Additional Data. + * - optionally 16 extra bytes with the algorithm name. + * Because the arguments have distinct fixed lengths it is not possible for + * different argument mixtures to emulate each other and result in the same + * nonces. + */ + memcpy(keydata, key32, 32); + memcpy(keydata + 32, msg32, 32); + if (data != NULL) { + memcpy(keydata + 64, data, 32); + keylen = 96; + } + if (algo16 != NULL) { + memcpy(keydata + keylen, algo16, 16); + keylen += 16; + } + secp256k1_rfc6979_hmac_sha256_initialize(&rng, keydata, keylen); + memset(keydata, 0, sizeof(keydata)); + for (i = 0; i <= counter; i++) { + secp256k1_rfc6979_hmac_sha256_generate(&rng, nonce32, 32); + } + secp256k1_rfc6979_hmac_sha256_finalize(&rng); + return 1; +} + +const secp256k1_nonce_function secp256k1_nonce_function_rfc6979 = nonce_function_rfc6979; +const secp256k1_nonce_function secp256k1_nonce_function_default = nonce_function_rfc6979; + +int secp256k1_ecdsa_sign(const secp256k1_context* ctx, secp256k1_ecdsa_signature *signature, const unsigned char *msg32, const unsigned char *seckey, secp256k1_nonce_function noncefp, const void* noncedata) { + secp256k1_scalar r, s; + secp256k1_scalar sec, non, msg; + int ret = 0; + int overflow = 0; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_gen_context_is_built(&ctx->ecmult_gen_ctx)); + ARG_CHECK(msg32 != NULL); + ARG_CHECK(signature != NULL); + ARG_CHECK(seckey != NULL); + if (noncefp == NULL) { + noncefp = secp256k1_nonce_function_default; + } + + secp256k1_scalar_set_b32(&sec, seckey, &overflow); + /* Fail if the secret key is invalid. */ + if (!overflow && !secp256k1_scalar_is_zero(&sec)) { + unsigned int count = 0; + secp256k1_scalar_set_b32(&msg, msg32, NULL); + while (1) { + unsigned char nonce32[32]; + ret = noncefp(nonce32, msg32, seckey, NULL, (void*)noncedata, count); + if (!ret) { + break; + } + secp256k1_scalar_set_b32(&non, nonce32, &overflow); + memset(nonce32, 0, 32); + if (!overflow && !secp256k1_scalar_is_zero(&non)) { + if (secp256k1_ecdsa_sig_sign(&ctx->ecmult_gen_ctx, &r, &s, &sec, &msg, &non, NULL)) { + break; + } + } + count++; + } + secp256k1_scalar_clear(&msg); + secp256k1_scalar_clear(&non); + secp256k1_scalar_clear(&sec); + } + if (ret) { + secp256k1_ecdsa_signature_save(signature, &r, &s); + } else { + memset(signature, 0, sizeof(*signature)); + } + return ret; +} + +int secp256k1_ec_seckey_verify(const secp256k1_context* ctx, const unsigned char *seckey) { + secp256k1_scalar sec; + int ret; + int overflow; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(seckey != NULL); + (void)ctx; + + secp256k1_scalar_set_b32(&sec, seckey, &overflow); + ret = !overflow && !secp256k1_scalar_is_zero(&sec); + secp256k1_scalar_clear(&sec); + return ret; +} + +int secp256k1_ec_pubkey_create(const secp256k1_context* ctx, secp256k1_pubkey *pubkey, const unsigned char *seckey) { + secp256k1_gej pj; + secp256k1_ge p; + secp256k1_scalar sec; + int overflow; + int ret = 0; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(pubkey != NULL); + memset(pubkey, 0, sizeof(*pubkey)); + ARG_CHECK(secp256k1_ecmult_gen_context_is_built(&ctx->ecmult_gen_ctx)); + ARG_CHECK(seckey != NULL); + + secp256k1_scalar_set_b32(&sec, seckey, &overflow); + ret = (!overflow) & (!secp256k1_scalar_is_zero(&sec)); + if (ret) { + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &pj, &sec); + secp256k1_ge_set_gej(&p, &pj); + secp256k1_pubkey_save(pubkey, &p); + } + secp256k1_scalar_clear(&sec); + return ret; +} + +int secp256k1_ec_privkey_tweak_add(const secp256k1_context* ctx, unsigned char *seckey, const unsigned char *tweak) { + secp256k1_scalar term; + secp256k1_scalar sec; + int ret = 0; + int overflow = 0; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(seckey != NULL); + ARG_CHECK(tweak != NULL); + (void)ctx; + + secp256k1_scalar_set_b32(&term, tweak, &overflow); + secp256k1_scalar_set_b32(&sec, seckey, NULL); + + ret = !overflow && secp256k1_eckey_privkey_tweak_add(&sec, &term); + memset(seckey, 0, 32); + if (ret) { + secp256k1_scalar_get_b32(seckey, &sec); + } + + secp256k1_scalar_clear(&sec); + secp256k1_scalar_clear(&term); + return ret; +} + +int secp256k1_ec_pubkey_tweak_add(const secp256k1_context* ctx, secp256k1_pubkey *pubkey, const unsigned char *tweak) { + secp256k1_ge p; + secp256k1_scalar term; + int ret = 0; + int overflow = 0; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_context_is_built(&ctx->ecmult_ctx)); + ARG_CHECK(pubkey != NULL); + ARG_CHECK(tweak != NULL); + + secp256k1_scalar_set_b32(&term, tweak, &overflow); + ret = !overflow && secp256k1_pubkey_load(ctx, &p, pubkey); + memset(pubkey, 0, sizeof(*pubkey)); + if (ret) { + if (secp256k1_eckey_pubkey_tweak_add(&ctx->ecmult_ctx, &p, &term)) { + secp256k1_pubkey_save(pubkey, &p); + } else { + ret = 0; + } + } + + return ret; +} + +int secp256k1_ec_privkey_tweak_mul(const secp256k1_context* ctx, unsigned char *seckey, const unsigned char *tweak) { + secp256k1_scalar factor; + secp256k1_scalar sec; + int ret = 0; + int overflow = 0; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(seckey != NULL); + ARG_CHECK(tweak != NULL); + (void)ctx; + + secp256k1_scalar_set_b32(&factor, tweak, &overflow); + secp256k1_scalar_set_b32(&sec, seckey, NULL); + ret = !overflow && secp256k1_eckey_privkey_tweak_mul(&sec, &factor); + memset(seckey, 0, 32); + if (ret) { + secp256k1_scalar_get_b32(seckey, &sec); + } + + secp256k1_scalar_clear(&sec); + secp256k1_scalar_clear(&factor); + return ret; +} + +int secp256k1_ec_pubkey_tweak_mul(const secp256k1_context* ctx, secp256k1_pubkey *pubkey, const unsigned char *tweak) { + secp256k1_ge p; + secp256k1_scalar factor; + int ret = 0; + int overflow = 0; + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_context_is_built(&ctx->ecmult_ctx)); + ARG_CHECK(pubkey != NULL); + ARG_CHECK(tweak != NULL); + + secp256k1_scalar_set_b32(&factor, tweak, &overflow); + ret = !overflow && secp256k1_pubkey_load(ctx, &p, pubkey); + memset(pubkey, 0, sizeof(*pubkey)); + if (ret) { + if (secp256k1_eckey_pubkey_tweak_mul(&ctx->ecmult_ctx, &p, &factor)) { + secp256k1_pubkey_save(pubkey, &p); + } else { + ret = 0; + } + } + + return ret; +} + +int secp256k1_context_randomize(secp256k1_context* ctx, const unsigned char *seed32) { + VERIFY_CHECK(ctx != NULL); + ARG_CHECK(secp256k1_ecmult_gen_context_is_built(&ctx->ecmult_gen_ctx)); + secp256k1_ecmult_gen_blind(&ctx->ecmult_gen_ctx, seed32); + return 1; +} + +int secp256k1_ec_pubkey_combine(const secp256k1_context* ctx, secp256k1_pubkey *pubnonce, const secp256k1_pubkey * const *pubnonces, size_t n) { + size_t i; + secp256k1_gej Qj; + secp256k1_ge Q; + + ARG_CHECK(pubnonce != NULL); + memset(pubnonce, 0, sizeof(*pubnonce)); + ARG_CHECK(n >= 1); + ARG_CHECK(pubnonces != NULL); + + secp256k1_gej_set_infinity(&Qj); + + for (i = 0; i < n; i++) { + secp256k1_pubkey_load(ctx, &Q, pubnonces[i]); + secp256k1_gej_add_ge(&Qj, &Qj, &Q); + } + if (secp256k1_gej_is_infinity(&Qj)) { + return 0; + } + secp256k1_ge_set_gej(&Q, &Qj); + secp256k1_pubkey_save(pubnonce, &Q); + return 1; +} + +#ifdef ENABLE_MODULE_ECDH +# include "modules/ecdh/main_impl.h" +#endif + +#ifdef ENABLE_MODULE_SCHNORR +# include "modules/schnorr/main_impl.h" +#endif + +#ifdef ENABLE_MODULE_RECOVERY +# include "modules/recovery/main_impl.h" +#endif diff --git a/src/secp256k1/src/testrand.h b/src/secp256k1/src/testrand.h new file mode 100644 index 00000000..f8efa93c --- /dev/null +++ b/src/secp256k1/src/testrand.h @@ -0,0 +1,38 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_TESTRAND_H_ +#define _SECP256K1_TESTRAND_H_ + +#if defined HAVE_CONFIG_H +#include "libsecp256k1-config.h" +#endif + +/* A non-cryptographic RNG used only for test infrastructure. */ + +/** Seed the pseudorandom number generator for testing. */ +SECP256K1_INLINE static void secp256k1_rand_seed(const unsigned char *seed16); + +/** Generate a pseudorandom number in the range [0..2**32-1]. */ +static uint32_t secp256k1_rand32(void); + +/** Generate a pseudorandom number in the range [0..2**bits-1]. Bits must be 1 or + * more. */ +static uint32_t secp256k1_rand_bits(int bits); + +/** Generate a pseudorandom number in the range [0..range-1]. */ +static uint32_t secp256k1_rand_int(uint32_t range); + +/** Generate a pseudorandom 32-byte array. */ +static void secp256k1_rand256(unsigned char *b32); + +/** Generate a pseudorandom 32-byte array with long sequences of zero and one bits. */ +static void secp256k1_rand256_test(unsigned char *b32); + +/** Generate pseudorandom bytes with long sequences of zero and one bits. */ +static void secp256k1_rand_bytes_test(unsigned char *bytes, size_t len); + +#endif diff --git a/src/secp256k1/src/testrand_impl.h b/src/secp256k1/src/testrand_impl.h new file mode 100644 index 00000000..15c7b9f1 --- /dev/null +++ b/src/secp256k1/src/testrand_impl.h @@ -0,0 +1,110 @@ +/********************************************************************** + * Copyright (c) 2013-2015 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_TESTRAND_IMPL_H_ +#define _SECP256K1_TESTRAND_IMPL_H_ + +#include +#include + +#include "testrand.h" +#include "hash.h" + +static secp256k1_rfc6979_hmac_sha256_t secp256k1_test_rng; +static uint32_t secp256k1_test_rng_precomputed[8]; +static int secp256k1_test_rng_precomputed_used = 8; +static uint64_t secp256k1_test_rng_integer; +static int secp256k1_test_rng_integer_bits_left = 0; + +SECP256K1_INLINE static void secp256k1_rand_seed(const unsigned char *seed16) { + secp256k1_rfc6979_hmac_sha256_initialize(&secp256k1_test_rng, seed16, 16); +} + +SECP256K1_INLINE static uint32_t secp256k1_rand32(void) { + if (secp256k1_test_rng_precomputed_used == 8) { + secp256k1_rfc6979_hmac_sha256_generate(&secp256k1_test_rng, (unsigned char*)(&secp256k1_test_rng_precomputed[0]), sizeof(secp256k1_test_rng_precomputed)); + secp256k1_test_rng_precomputed_used = 0; + } + return secp256k1_test_rng_precomputed[secp256k1_test_rng_precomputed_used++]; +} + +static uint32_t secp256k1_rand_bits(int bits) { + uint32_t ret; + if (secp256k1_test_rng_integer_bits_left < bits) { + secp256k1_test_rng_integer |= (((uint64_t)secp256k1_rand32()) << secp256k1_test_rng_integer_bits_left); + secp256k1_test_rng_integer_bits_left += 32; + } + ret = secp256k1_test_rng_integer; + secp256k1_test_rng_integer >>= bits; + secp256k1_test_rng_integer_bits_left -= bits; + ret &= ((~((uint32_t)0)) >> (32 - bits)); + return ret; +} + +static uint32_t secp256k1_rand_int(uint32_t range) { + /* We want a uniform integer between 0 and range-1, inclusive. + * B is the smallest number such that range <= 2**B. + * two mechanisms implemented here: + * - generate B bits numbers until one below range is found, and return it + * - find the largest multiple M of range that is <= 2**(B+A), generate B+A + * bits numbers until one below M is found, and return it modulo range + * The second mechanism consumes A more bits of entropy in every iteration, + * but may need fewer iterations due to M being closer to 2**(B+A) then + * range is to 2**B. The array below (indexed by B) contains a 0 when the + * first mechanism is to be used, and the number A otherwise. + */ + static const int addbits[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 1, 0}; + uint32_t trange, mult; + int bits = 0; + if (range <= 1) { + return 0; + } + trange = range - 1; + while (trange > 0) { + trange >>= 1; + bits++; + } + if (addbits[bits]) { + bits = bits + addbits[bits]; + mult = ((~((uint32_t)0)) >> (32 - bits)) / range; + trange = range * mult; + } else { + trange = range; + mult = 1; + } + while(1) { + uint32_t x = secp256k1_rand_bits(bits); + if (x < trange) { + return (mult == 1) ? x : (x % range); + } + } +} + +static void secp256k1_rand256(unsigned char *b32) { + secp256k1_rfc6979_hmac_sha256_generate(&secp256k1_test_rng, b32, 32); +} + +static void secp256k1_rand_bytes_test(unsigned char *bytes, size_t len) { + size_t bits = 0; + memset(bytes, 0, len); + while (bits < len * 8) { + int now; + uint32_t val; + now = 1 + (secp256k1_rand_bits(6) * secp256k1_rand_bits(5) + 16) / 31; + val = secp256k1_rand_bits(1); + while (now > 0 && bits < len * 8) { + bytes[bits / 8] |= val << (bits % 8); + now--; + bits++; + } + } +} + +static void secp256k1_rand256_test(unsigned char *b32) { + secp256k1_rand_bytes_test(b32, 32); +} + +#endif diff --git a/src/secp256k1/src/tests.c b/src/secp256k1/src/tests.c new file mode 100644 index 00000000..687a5f2f --- /dev/null +++ b/src/secp256k1/src/tests.c @@ -0,0 +1,4383 @@ +/********************************************************************** + * Copyright (c) 2013, 2014, 2015 Pieter Wuille, Gregory Maxwell * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#if defined HAVE_CONFIG_H +#include "libsecp256k1-config.h" +#endif + +#include +#include + +#include + +#include "secp256k1.c" +#include "include/secp256k1.h" +#include "testrand_impl.h" + +#ifdef ENABLE_OPENSSL_TESTS +#include "openssl/bn.h" +#include "openssl/ec.h" +#include "openssl/ecdsa.h" +#include "openssl/obj_mac.h" +#endif + +#include "contrib/lax_der_parsing.c" +#include "contrib/lax_der_privatekey_parsing.c" + +#if !defined(VG_CHECK) +# if defined(VALGRIND) +# include +# define VG_UNDEF(x,y) VALGRIND_MAKE_MEM_UNDEFINED((x),(y)) +# define VG_CHECK(x,y) VALGRIND_CHECK_MEM_IS_DEFINED((x),(y)) +# else +# define VG_UNDEF(x,y) +# define VG_CHECK(x,y) +# endif +#endif + +static int count = 64; +static secp256k1_context *ctx = NULL; + +static void counting_illegal_callback_fn(const char* str, void* data) { + /* Dummy callback function that just counts. */ + int32_t *p; + (void)str; + p = data; + (*p)++; +} + +static void uncounting_illegal_callback_fn(const char* str, void* data) { + /* Dummy callback function that just counts (backwards). */ + int32_t *p; + (void)str; + p = data; + (*p)--; +} + +void random_field_element_test(secp256k1_fe *fe) { + do { + unsigned char b32[32]; + secp256k1_rand256_test(b32); + if (secp256k1_fe_set_b32(fe, b32)) { + break; + } + } while(1); +} + +void random_field_element_magnitude(secp256k1_fe *fe) { + secp256k1_fe zero; + int n = secp256k1_rand_int(9); + secp256k1_fe_normalize(fe); + if (n == 0) { + return; + } + secp256k1_fe_clear(&zero); + secp256k1_fe_negate(&zero, &zero, 0); + secp256k1_fe_mul_int(&zero, n - 1); + secp256k1_fe_add(fe, &zero); + VERIFY_CHECK(fe->magnitude == n); +} + +void random_group_element_test(secp256k1_ge *ge) { + secp256k1_fe fe; + do { + random_field_element_test(&fe); + if (secp256k1_ge_set_xo_var(ge, &fe, secp256k1_rand_bits(1))) { + secp256k1_fe_normalize(&ge->y); + break; + } + } while(1); +} + +void random_group_element_jacobian_test(secp256k1_gej *gej, const secp256k1_ge *ge) { + secp256k1_fe z2, z3; + do { + random_field_element_test(&gej->z); + if (!secp256k1_fe_is_zero(&gej->z)) { + break; + } + } while(1); + secp256k1_fe_sqr(&z2, &gej->z); + secp256k1_fe_mul(&z3, &z2, &gej->z); + secp256k1_fe_mul(&gej->x, &ge->x, &z2); + secp256k1_fe_mul(&gej->y, &ge->y, &z3); + gej->infinity = ge->infinity; +} + +void random_scalar_order_test(secp256k1_scalar *num) { + do { + unsigned char b32[32]; + int overflow = 0; + secp256k1_rand256_test(b32); + secp256k1_scalar_set_b32(num, b32, &overflow); + if (overflow || secp256k1_scalar_is_zero(num)) { + continue; + } + break; + } while(1); +} + +void random_scalar_order(secp256k1_scalar *num) { + do { + unsigned char b32[32]; + int overflow = 0; + secp256k1_rand256(b32); + secp256k1_scalar_set_b32(num, b32, &overflow); + if (overflow || secp256k1_scalar_is_zero(num)) { + continue; + } + break; + } while(1); +} + +void run_context_tests(void) { + secp256k1_pubkey pubkey; + secp256k1_ecdsa_signature sig; + unsigned char ctmp[32]; + int32_t ecount; + int32_t ecount2; + secp256k1_context *none = secp256k1_context_create(SECP256K1_CONTEXT_NONE); + secp256k1_context *sign = secp256k1_context_create(SECP256K1_CONTEXT_SIGN); + secp256k1_context *vrfy = secp256k1_context_create(SECP256K1_CONTEXT_VERIFY); + secp256k1_context *both = secp256k1_context_create(SECP256K1_CONTEXT_SIGN | SECP256K1_CONTEXT_VERIFY); + + secp256k1_gej pubj; + secp256k1_ge pub; + secp256k1_scalar msg, key, nonce; + secp256k1_scalar sigr, sigs; + + ecount = 0; + ecount2 = 10; + secp256k1_context_set_illegal_callback(vrfy, counting_illegal_callback_fn, &ecount); + secp256k1_context_set_illegal_callback(sign, counting_illegal_callback_fn, &ecount2); + secp256k1_context_set_error_callback(sign, counting_illegal_callback_fn, NULL); + CHECK(vrfy->error_callback.fn != sign->error_callback.fn); + + /*** clone and destroy all of them to make sure cloning was complete ***/ + { + secp256k1_context *ctx_tmp; + + ctx_tmp = none; none = secp256k1_context_clone(none); secp256k1_context_destroy(ctx_tmp); + ctx_tmp = sign; sign = secp256k1_context_clone(sign); secp256k1_context_destroy(ctx_tmp); + ctx_tmp = vrfy; vrfy = secp256k1_context_clone(vrfy); secp256k1_context_destroy(ctx_tmp); + ctx_tmp = both; both = secp256k1_context_clone(both); secp256k1_context_destroy(ctx_tmp); + } + + /* Verify that the error callback makes it across the clone. */ + CHECK(vrfy->error_callback.fn != sign->error_callback.fn); + /* And that it resets back to default. */ + secp256k1_context_set_error_callback(sign, NULL, NULL); + CHECK(vrfy->error_callback.fn == sign->error_callback.fn); + + /*** attempt to use them ***/ + random_scalar_order_test(&msg); + random_scalar_order_test(&key); + secp256k1_ecmult_gen(&both->ecmult_gen_ctx, &pubj, &key); + secp256k1_ge_set_gej(&pub, &pubj); + + /* Verify context-type checking illegal-argument errors. */ + memset(ctmp, 1, 32); + CHECK(secp256k1_ec_pubkey_create(vrfy, &pubkey, ctmp) == 0); + CHECK(ecount == 1); + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_create(sign, &pubkey, ctmp) == 1); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ecdsa_sign(vrfy, &sig, ctmp, ctmp, NULL, NULL) == 0); + CHECK(ecount == 2); + VG_UNDEF(&sig, sizeof(sig)); + CHECK(secp256k1_ecdsa_sign(sign, &sig, ctmp, ctmp, NULL, NULL) == 1); + VG_CHECK(&sig, sizeof(sig)); + CHECK(ecount2 == 10); + CHECK(secp256k1_ecdsa_verify(sign, &sig, ctmp, &pubkey) == 0); + CHECK(ecount2 == 11); + CHECK(secp256k1_ecdsa_verify(vrfy, &sig, ctmp, &pubkey) == 1); + CHECK(ecount == 2); + CHECK(secp256k1_ec_pubkey_tweak_add(sign, &pubkey, ctmp) == 0); + CHECK(ecount2 == 12); + CHECK(secp256k1_ec_pubkey_tweak_add(vrfy, &pubkey, ctmp) == 1); + CHECK(ecount == 2); + CHECK(secp256k1_ec_pubkey_tweak_mul(sign, &pubkey, ctmp) == 0); + CHECK(ecount2 == 13); + CHECK(secp256k1_ec_pubkey_tweak_mul(vrfy, &pubkey, ctmp) == 1); + CHECK(ecount == 2); + CHECK(secp256k1_context_randomize(vrfy, ctmp) == 0); + CHECK(ecount == 3); + CHECK(secp256k1_context_randomize(sign, NULL) == 1); + CHECK(ecount2 == 13); + secp256k1_context_set_illegal_callback(vrfy, NULL, NULL); + secp256k1_context_set_illegal_callback(sign, NULL, NULL); + + /* This shouldn't leak memory, due to already-set tests. */ + secp256k1_ecmult_gen_context_build(&sign->ecmult_gen_ctx, NULL); + secp256k1_ecmult_context_build(&vrfy->ecmult_ctx, NULL); + + /* obtain a working nonce */ + do { + random_scalar_order_test(&nonce); + } while(!secp256k1_ecdsa_sig_sign(&both->ecmult_gen_ctx, &sigr, &sigs, &key, &msg, &nonce, NULL)); + + /* try signing */ + CHECK(secp256k1_ecdsa_sig_sign(&sign->ecmult_gen_ctx, &sigr, &sigs, &key, &msg, &nonce, NULL)); + CHECK(secp256k1_ecdsa_sig_sign(&both->ecmult_gen_ctx, &sigr, &sigs, &key, &msg, &nonce, NULL)); + + /* try verifying */ + CHECK(secp256k1_ecdsa_sig_verify(&vrfy->ecmult_ctx, &sigr, &sigs, &pub, &msg)); + CHECK(secp256k1_ecdsa_sig_verify(&both->ecmult_ctx, &sigr, &sigs, &pub, &msg)); + + /* cleanup */ + secp256k1_context_destroy(none); + secp256k1_context_destroy(sign); + secp256k1_context_destroy(vrfy); + secp256k1_context_destroy(both); + /* Defined as no-op. */ + secp256k1_context_destroy(NULL); +} + +/***** HASH TESTS *****/ + +void run_sha256_tests(void) { + static const char *inputs[8] = { + "", "abc", "message digest", "secure hash algorithm", "SHA256 is considered to be safe", + "abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq", + "For this sample, this 63-byte string will be used as input data", + "This is exactly 64 bytes long, not counting the terminating byte" + }; + static const unsigned char outputs[8][32] = { + {0xe3, 0xb0, 0xc4, 0x42, 0x98, 0xfc, 0x1c, 0x14, 0x9a, 0xfb, 0xf4, 0xc8, 0x99, 0x6f, 0xb9, 0x24, 0x27, 0xae, 0x41, 0xe4, 0x64, 0x9b, 0x93, 0x4c, 0xa4, 0x95, 0x99, 0x1b, 0x78, 0x52, 0xb8, 0x55}, + {0xba, 0x78, 0x16, 0xbf, 0x8f, 0x01, 0xcf, 0xea, 0x41, 0x41, 0x40, 0xde, 0x5d, 0xae, 0x22, 0x23, 0xb0, 0x03, 0x61, 0xa3, 0x96, 0x17, 0x7a, 0x9c, 0xb4, 0x10, 0xff, 0x61, 0xf2, 0x00, 0x15, 0xad}, + {0xf7, 0x84, 0x6f, 0x55, 0xcf, 0x23, 0xe1, 0x4e, 0xeb, 0xea, 0xb5, 0xb4, 0xe1, 0x55, 0x0c, 0xad, 0x5b, 0x50, 0x9e, 0x33, 0x48, 0xfb, 0xc4, 0xef, 0xa3, 0xa1, 0x41, 0x3d, 0x39, 0x3c, 0xb6, 0x50}, + {0xf3, 0x0c, 0xeb, 0x2b, 0xb2, 0x82, 0x9e, 0x79, 0xe4, 0xca, 0x97, 0x53, 0xd3, 0x5a, 0x8e, 0xcc, 0x00, 0x26, 0x2d, 0x16, 0x4c, 0xc0, 0x77, 0x08, 0x02, 0x95, 0x38, 0x1c, 0xbd, 0x64, 0x3f, 0x0d}, + {0x68, 0x19, 0xd9, 0x15, 0xc7, 0x3f, 0x4d, 0x1e, 0x77, 0xe4, 0xe1, 0xb5, 0x2d, 0x1f, 0xa0, 0xf9, 0xcf, 0x9b, 0xea, 0xea, 0xd3, 0x93, 0x9f, 0x15, 0x87, 0x4b, 0xd9, 0x88, 0xe2, 0xa2, 0x36, 0x30}, + {0x24, 0x8d, 0x6a, 0x61, 0xd2, 0x06, 0x38, 0xb8, 0xe5, 0xc0, 0x26, 0x93, 0x0c, 0x3e, 0x60, 0x39, 0xa3, 0x3c, 0xe4, 0x59, 0x64, 0xff, 0x21, 0x67, 0xf6, 0xec, 0xed, 0xd4, 0x19, 0xdb, 0x06, 0xc1}, + {0xf0, 0x8a, 0x78, 0xcb, 0xba, 0xee, 0x08, 0x2b, 0x05, 0x2a, 0xe0, 0x70, 0x8f, 0x32, 0xfa, 0x1e, 0x50, 0xc5, 0xc4, 0x21, 0xaa, 0x77, 0x2b, 0xa5, 0xdb, 0xb4, 0x06, 0xa2, 0xea, 0x6b, 0xe3, 0x42}, + {0xab, 0x64, 0xef, 0xf7, 0xe8, 0x8e, 0x2e, 0x46, 0x16, 0x5e, 0x29, 0xf2, 0xbc, 0xe4, 0x18, 0x26, 0xbd, 0x4c, 0x7b, 0x35, 0x52, 0xf6, 0xb3, 0x82, 0xa9, 0xe7, 0xd3, 0xaf, 0x47, 0xc2, 0x45, 0xf8} + }; + int i; + for (i = 0; i < 8; i++) { + unsigned char out[32]; + secp256k1_sha256_t hasher; + secp256k1_sha256_initialize(&hasher); + secp256k1_sha256_write(&hasher, (const unsigned char*)(inputs[i]), strlen(inputs[i])); + secp256k1_sha256_finalize(&hasher, out); + CHECK(memcmp(out, outputs[i], 32) == 0); + if (strlen(inputs[i]) > 0) { + int split = secp256k1_rand_int(strlen(inputs[i])); + secp256k1_sha256_initialize(&hasher); + secp256k1_sha256_write(&hasher, (const unsigned char*)(inputs[i]), split); + secp256k1_sha256_write(&hasher, (const unsigned char*)(inputs[i] + split), strlen(inputs[i]) - split); + secp256k1_sha256_finalize(&hasher, out); + CHECK(memcmp(out, outputs[i], 32) == 0); + } + } +} + +void run_hmac_sha256_tests(void) { + static const char *keys[6] = { + "\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b", + "\x4a\x65\x66\x65", + "\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa", + "\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19", + "\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa", + "\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa" + }; + static const char *inputs[6] = { + "\x48\x69\x20\x54\x68\x65\x72\x65", + "\x77\x68\x61\x74\x20\x64\x6f\x20\x79\x61\x20\x77\x61\x6e\x74\x20\x66\x6f\x72\x20\x6e\x6f\x74\x68\x69\x6e\x67\x3f", + "\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd\xdd", + "\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd", + "\x54\x65\x73\x74\x20\x55\x73\x69\x6e\x67\x20\x4c\x61\x72\x67\x65\x72\x20\x54\x68\x61\x6e\x20\x42\x6c\x6f\x63\x6b\x2d\x53\x69\x7a\x65\x20\x4b\x65\x79\x20\x2d\x20\x48\x61\x73\x68\x20\x4b\x65\x79\x20\x46\x69\x72\x73\x74", + "\x54\x68\x69\x73\x20\x69\x73\x20\x61\x20\x74\x65\x73\x74\x20\x75\x73\x69\x6e\x67\x20\x61\x20\x6c\x61\x72\x67\x65\x72\x20\x74\x68\x61\x6e\x20\x62\x6c\x6f\x63\x6b\x2d\x73\x69\x7a\x65\x20\x6b\x65\x79\x20\x61\x6e\x64\x20\x61\x20\x6c\x61\x72\x67\x65\x72\x20\x74\x68\x61\x6e\x20\x62\x6c\x6f\x63\x6b\x2d\x73\x69\x7a\x65\x20\x64\x61\x74\x61\x2e\x20\x54\x68\x65\x20\x6b\x65\x79\x20\x6e\x65\x65\x64\x73\x20\x74\x6f\x20\x62\x65\x20\x68\x61\x73\x68\x65\x64\x20\x62\x65\x66\x6f\x72\x65\x20\x62\x65\x69\x6e\x67\x20\x75\x73\x65\x64\x20\x62\x79\x20\x74\x68\x65\x20\x48\x4d\x41\x43\x20\x61\x6c\x67\x6f\x72\x69\x74\x68\x6d\x2e" + }; + static const unsigned char outputs[6][32] = { + {0xb0, 0x34, 0x4c, 0x61, 0xd8, 0xdb, 0x38, 0x53, 0x5c, 0xa8, 0xaf, 0xce, 0xaf, 0x0b, 0xf1, 0x2b, 0x88, 0x1d, 0xc2, 0x00, 0xc9, 0x83, 0x3d, 0xa7, 0x26, 0xe9, 0x37, 0x6c, 0x2e, 0x32, 0xcf, 0xf7}, + {0x5b, 0xdc, 0xc1, 0x46, 0xbf, 0x60, 0x75, 0x4e, 0x6a, 0x04, 0x24, 0x26, 0x08, 0x95, 0x75, 0xc7, 0x5a, 0x00, 0x3f, 0x08, 0x9d, 0x27, 0x39, 0x83, 0x9d, 0xec, 0x58, 0xb9, 0x64, 0xec, 0x38, 0x43}, + {0x77, 0x3e, 0xa9, 0x1e, 0x36, 0x80, 0x0e, 0x46, 0x85, 0x4d, 0xb8, 0xeb, 0xd0, 0x91, 0x81, 0xa7, 0x29, 0x59, 0x09, 0x8b, 0x3e, 0xf8, 0xc1, 0x22, 0xd9, 0x63, 0x55, 0x14, 0xce, 0xd5, 0x65, 0xfe}, + {0x82, 0x55, 0x8a, 0x38, 0x9a, 0x44, 0x3c, 0x0e, 0xa4, 0xcc, 0x81, 0x98, 0x99, 0xf2, 0x08, 0x3a, 0x85, 0xf0, 0xfa, 0xa3, 0xe5, 0x78, 0xf8, 0x07, 0x7a, 0x2e, 0x3f, 0xf4, 0x67, 0x29, 0x66, 0x5b}, + {0x60, 0xe4, 0x31, 0x59, 0x1e, 0xe0, 0xb6, 0x7f, 0x0d, 0x8a, 0x26, 0xaa, 0xcb, 0xf5, 0xb7, 0x7f, 0x8e, 0x0b, 0xc6, 0x21, 0x37, 0x28, 0xc5, 0x14, 0x05, 0x46, 0x04, 0x0f, 0x0e, 0xe3, 0x7f, 0x54}, + {0x9b, 0x09, 0xff, 0xa7, 0x1b, 0x94, 0x2f, 0xcb, 0x27, 0x63, 0x5f, 0xbc, 0xd5, 0xb0, 0xe9, 0x44, 0xbf, 0xdc, 0x63, 0x64, 0x4f, 0x07, 0x13, 0x93, 0x8a, 0x7f, 0x51, 0x53, 0x5c, 0x3a, 0x35, 0xe2} + }; + int i; + for (i = 0; i < 6; i++) { + secp256k1_hmac_sha256_t hasher; + unsigned char out[32]; + secp256k1_hmac_sha256_initialize(&hasher, (const unsigned char*)(keys[i]), strlen(keys[i])); + secp256k1_hmac_sha256_write(&hasher, (const unsigned char*)(inputs[i]), strlen(inputs[i])); + secp256k1_hmac_sha256_finalize(&hasher, out); + CHECK(memcmp(out, outputs[i], 32) == 0); + if (strlen(inputs[i]) > 0) { + int split = secp256k1_rand_int(strlen(inputs[i])); + secp256k1_hmac_sha256_initialize(&hasher, (const unsigned char*)(keys[i]), strlen(keys[i])); + secp256k1_hmac_sha256_write(&hasher, (const unsigned char*)(inputs[i]), split); + secp256k1_hmac_sha256_write(&hasher, (const unsigned char*)(inputs[i] + split), strlen(inputs[i]) - split); + secp256k1_hmac_sha256_finalize(&hasher, out); + CHECK(memcmp(out, outputs[i], 32) == 0); + } + } +} + +void run_rfc6979_hmac_sha256_tests(void) { + static const unsigned char key1[65] = {0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x00, 0x4b, 0xf5, 0x12, 0x2f, 0x34, 0x45, 0x54, 0xc5, 0x3b, 0xde, 0x2e, 0xbb, 0x8c, 0xd2, 0xb7, 0xe3, 0xd1, 0x60, 0x0a, 0xd6, 0x31, 0xc3, 0x85, 0xa5, 0xd7, 0xcc, 0xe2, 0x3c, 0x77, 0x85, 0x45, 0x9a, 0}; + static const unsigned char out1[3][32] = { + {0x4f, 0xe2, 0x95, 0x25, 0xb2, 0x08, 0x68, 0x09, 0x15, 0x9a, 0xcd, 0xf0, 0x50, 0x6e, 0xfb, 0x86, 0xb0, 0xec, 0x93, 0x2c, 0x7b, 0xa4, 0x42, 0x56, 0xab, 0x32, 0x1e, 0x42, 0x1e, 0x67, 0xe9, 0xfb}, + {0x2b, 0xf0, 0xff, 0xf1, 0xd3, 0xc3, 0x78, 0xa2, 0x2d, 0xc5, 0xde, 0x1d, 0x85, 0x65, 0x22, 0x32, 0x5c, 0x65, 0xb5, 0x04, 0x49, 0x1a, 0x0c, 0xbd, 0x01, 0xcb, 0x8f, 0x3a, 0xa6, 0x7f, 0xfd, 0x4a}, + {0xf5, 0x28, 0xb4, 0x10, 0xcb, 0x54, 0x1f, 0x77, 0x00, 0x0d, 0x7a, 0xfb, 0x6c, 0x5b, 0x53, 0xc5, 0xc4, 0x71, 0xea, 0xb4, 0x3e, 0x46, 0x6d, 0x9a, 0xc5, 0x19, 0x0c, 0x39, 0xc8, 0x2f, 0xd8, 0x2e} + }; + + static const unsigned char key2[64] = {0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xe3, 0xb0, 0xc4, 0x42, 0x98, 0xfc, 0x1c, 0x14, 0x9a, 0xfb, 0xf4, 0xc8, 0x99, 0x6f, 0xb9, 0x24, 0x27, 0xae, 0x41, 0xe4, 0x64, 0x9b, 0x93, 0x4c, 0xa4, 0x95, 0x99, 0x1b, 0x78, 0x52, 0xb8, 0x55}; + static const unsigned char out2[3][32] = { + {0x9c, 0x23, 0x6c, 0x16, 0x5b, 0x82, 0xae, 0x0c, 0xd5, 0x90, 0x65, 0x9e, 0x10, 0x0b, 0x6b, 0xab, 0x30, 0x36, 0xe7, 0xba, 0x8b, 0x06, 0x74, 0x9b, 0xaf, 0x69, 0x81, 0xe1, 0x6f, 0x1a, 0x2b, 0x95}, + {0xdf, 0x47, 0x10, 0x61, 0x62, 0x5b, 0xc0, 0xea, 0x14, 0xb6, 0x82, 0xfe, 0xee, 0x2c, 0x9c, 0x02, 0xf2, 0x35, 0xda, 0x04, 0x20, 0x4c, 0x1d, 0x62, 0xa1, 0x53, 0x6c, 0x6e, 0x17, 0xae, 0xd7, 0xa9}, + {0x75, 0x97, 0x88, 0x7c, 0xbd, 0x76, 0x32, 0x1f, 0x32, 0xe3, 0x04, 0x40, 0x67, 0x9a, 0x22, 0xcf, 0x7f, 0x8d, 0x9d, 0x2e, 0xac, 0x39, 0x0e, 0x58, 0x1f, 0xea, 0x09, 0x1c, 0xe2, 0x02, 0xba, 0x94} + }; + + secp256k1_rfc6979_hmac_sha256_t rng; + unsigned char out[32]; + int i; + + secp256k1_rfc6979_hmac_sha256_initialize(&rng, key1, 64); + for (i = 0; i < 3; i++) { + secp256k1_rfc6979_hmac_sha256_generate(&rng, out, 32); + CHECK(memcmp(out, out1[i], 32) == 0); + } + secp256k1_rfc6979_hmac_sha256_finalize(&rng); + + secp256k1_rfc6979_hmac_sha256_initialize(&rng, key1, 65); + for (i = 0; i < 3; i++) { + secp256k1_rfc6979_hmac_sha256_generate(&rng, out, 32); + CHECK(memcmp(out, out1[i], 32) != 0); + } + secp256k1_rfc6979_hmac_sha256_finalize(&rng); + + secp256k1_rfc6979_hmac_sha256_initialize(&rng, key2, 64); + for (i = 0; i < 3; i++) { + secp256k1_rfc6979_hmac_sha256_generate(&rng, out, 32); + CHECK(memcmp(out, out2[i], 32) == 0); + } + secp256k1_rfc6979_hmac_sha256_finalize(&rng); +} + +/***** RANDOM TESTS *****/ + +void test_rand_bits(int rand32, int bits) { + /* (1-1/2^B)^rounds[B] < 1/10^9, so rounds is the number of iterations to + * get a false negative chance below once in a billion */ + static const unsigned int rounds[7] = {1, 30, 73, 156, 322, 653, 1316}; + /* We try multiplying the results with various odd numbers, which shouldn't + * influence the uniform distribution modulo a power of 2. */ + static const uint32_t mults[6] = {1, 3, 21, 289, 0x9999, 0x80402011}; + /* We only select up to 6 bits from the output to analyse */ + unsigned int usebits = bits > 6 ? 6 : bits; + unsigned int maxshift = bits - usebits; + /* For each of the maxshift+1 usebits-bit sequences inside a bits-bit + number, track all observed outcomes, one per bit in a uint64_t. */ + uint64_t x[6][27] = {{0}}; + unsigned int i, shift, m; + /* Multiply the output of all rand calls with the odd number m, which + should not change the uniformity of its distribution. */ + for (i = 0; i < rounds[usebits]; i++) { + uint32_t r = (rand32 ? secp256k1_rand32() : secp256k1_rand_bits(bits)); + CHECK((((uint64_t)r) >> bits) == 0); + for (m = 0; m < sizeof(mults) / sizeof(mults[0]); m++) { + uint32_t rm = r * mults[m]; + for (shift = 0; shift <= maxshift; shift++) { + x[m][shift] |= (((uint64_t)1) << ((rm >> shift) & ((1 << usebits) - 1))); + } + } + } + for (m = 0; m < sizeof(mults) / sizeof(mults[0]); m++) { + for (shift = 0; shift <= maxshift; shift++) { + /* Test that the lower usebits bits of x[shift] are 1 */ + CHECK(((~x[m][shift]) << (64 - (1 << usebits))) == 0); + } + } +} + +/* Subrange must be a whole divisor of range, and at most 64 */ +void test_rand_int(uint32_t range, uint32_t subrange) { + /* (1-1/subrange)^rounds < 1/10^9 */ + int rounds = (subrange * 2073) / 100; + int i; + uint64_t x = 0; + CHECK((range % subrange) == 0); + for (i = 0; i < rounds; i++) { + uint32_t r = secp256k1_rand_int(range); + CHECK(r < range); + r = r % subrange; + x |= (((uint64_t)1) << r); + } + /* Test that the lower subrange bits of x are 1. */ + CHECK(((~x) << (64 - subrange)) == 0); +} + +void run_rand_bits(void) { + size_t b; + test_rand_bits(1, 32); + for (b = 1; b <= 32; b++) { + test_rand_bits(0, b); + } +} + +void run_rand_int(void) { + static const uint32_t ms[] = {1, 3, 17, 1000, 13771, 999999, 33554432}; + static const uint32_t ss[] = {1, 3, 6, 9, 13, 31, 64}; + unsigned int m, s; + for (m = 0; m < sizeof(ms) / sizeof(ms[0]); m++) { + for (s = 0; s < sizeof(ss) / sizeof(ss[0]); s++) { + test_rand_int(ms[m] * ss[s], ss[s]); + } + } +} + +/***** NUM TESTS *****/ + +#ifndef USE_NUM_NONE +void random_num_negate(secp256k1_num *num) { + if (secp256k1_rand_bits(1)) { + secp256k1_num_negate(num); + } +} + +void random_num_order_test(secp256k1_num *num) { + secp256k1_scalar sc; + random_scalar_order_test(&sc); + secp256k1_scalar_get_num(num, &sc); +} + +void random_num_order(secp256k1_num *num) { + secp256k1_scalar sc; + random_scalar_order(&sc); + secp256k1_scalar_get_num(num, &sc); +} + +void test_num_negate(void) { + secp256k1_num n1; + secp256k1_num n2; + random_num_order_test(&n1); /* n1 = R */ + random_num_negate(&n1); + secp256k1_num_copy(&n2, &n1); /* n2 = R */ + secp256k1_num_sub(&n1, &n2, &n1); /* n1 = n2-n1 = 0 */ + CHECK(secp256k1_num_is_zero(&n1)); + secp256k1_num_copy(&n1, &n2); /* n1 = R */ + secp256k1_num_negate(&n1); /* n1 = -R */ + CHECK(!secp256k1_num_is_zero(&n1)); + secp256k1_num_add(&n1, &n2, &n1); /* n1 = n2+n1 = 0 */ + CHECK(secp256k1_num_is_zero(&n1)); + secp256k1_num_copy(&n1, &n2); /* n1 = R */ + secp256k1_num_negate(&n1); /* n1 = -R */ + CHECK(secp256k1_num_is_neg(&n1) != secp256k1_num_is_neg(&n2)); + secp256k1_num_negate(&n1); /* n1 = R */ + CHECK(secp256k1_num_eq(&n1, &n2)); +} + +void test_num_add_sub(void) { + secp256k1_num n1; + secp256k1_num n2; + secp256k1_num n1p2, n2p1, n1m2, n2m1; + random_num_order_test(&n1); /* n1 = R1 */ + if (secp256k1_rand_bits(1)) { + random_num_negate(&n1); + } + random_num_order_test(&n2); /* n2 = R2 */ + if (secp256k1_rand_bits(1)) { + random_num_negate(&n2); + } + secp256k1_num_add(&n1p2, &n1, &n2); /* n1p2 = R1 + R2 */ + secp256k1_num_add(&n2p1, &n2, &n1); /* n2p1 = R2 + R1 */ + secp256k1_num_sub(&n1m2, &n1, &n2); /* n1m2 = R1 - R2 */ + secp256k1_num_sub(&n2m1, &n2, &n1); /* n2m1 = R2 - R1 */ + CHECK(secp256k1_num_eq(&n1p2, &n2p1)); + CHECK(!secp256k1_num_eq(&n1p2, &n1m2)); + secp256k1_num_negate(&n2m1); /* n2m1 = -R2 + R1 */ + CHECK(secp256k1_num_eq(&n2m1, &n1m2)); + CHECK(!secp256k1_num_eq(&n2m1, &n1)); + secp256k1_num_add(&n2m1, &n2m1, &n2); /* n2m1 = -R2 + R1 + R2 = R1 */ + CHECK(secp256k1_num_eq(&n2m1, &n1)); + CHECK(!secp256k1_num_eq(&n2p1, &n1)); + secp256k1_num_sub(&n2p1, &n2p1, &n2); /* n2p1 = R2 + R1 - R2 = R1 */ + CHECK(secp256k1_num_eq(&n2p1, &n1)); +} + +void run_num_smalltests(void) { + int i; + for (i = 0; i < 100*count; i++) { + test_num_negate(); + test_num_add_sub(); + } +} +#endif + +/***** SCALAR TESTS *****/ + +void scalar_test(void) { + secp256k1_scalar s; + secp256k1_scalar s1; + secp256k1_scalar s2; +#ifndef USE_NUM_NONE + secp256k1_num snum, s1num, s2num; + secp256k1_num order, half_order; +#endif + unsigned char c[32]; + + /* Set 's' to a random scalar, with value 'snum'. */ + random_scalar_order_test(&s); + + /* Set 's1' to a random scalar, with value 's1num'. */ + random_scalar_order_test(&s1); + + /* Set 's2' to a random scalar, with value 'snum2', and byte array representation 'c'. */ + random_scalar_order_test(&s2); + secp256k1_scalar_get_b32(c, &s2); + +#ifndef USE_NUM_NONE + secp256k1_scalar_get_num(&snum, &s); + secp256k1_scalar_get_num(&s1num, &s1); + secp256k1_scalar_get_num(&s2num, &s2); + + secp256k1_scalar_order_get_num(&order); + half_order = order; + secp256k1_num_shift(&half_order, 1); +#endif + + { + int i; + /* Test that fetching groups of 4 bits from a scalar and recursing n(i)=16*n(i-1)+p(i) reconstructs it. */ + secp256k1_scalar n; + secp256k1_scalar_set_int(&n, 0); + for (i = 0; i < 256; i += 4) { + secp256k1_scalar t; + int j; + secp256k1_scalar_set_int(&t, secp256k1_scalar_get_bits(&s, 256 - 4 - i, 4)); + for (j = 0; j < 4; j++) { + secp256k1_scalar_add(&n, &n, &n); + } + secp256k1_scalar_add(&n, &n, &t); + } + CHECK(secp256k1_scalar_eq(&n, &s)); + } + + { + /* Test that fetching groups of randomly-sized bits from a scalar and recursing n(i)=b*n(i-1)+p(i) reconstructs it. */ + secp256k1_scalar n; + int i = 0; + secp256k1_scalar_set_int(&n, 0); + while (i < 256) { + secp256k1_scalar t; + int j; + int now = secp256k1_rand_int(15) + 1; + if (now + i > 256) { + now = 256 - i; + } + secp256k1_scalar_set_int(&t, secp256k1_scalar_get_bits_var(&s, 256 - now - i, now)); + for (j = 0; j < now; j++) { + secp256k1_scalar_add(&n, &n, &n); + } + secp256k1_scalar_add(&n, &n, &t); + i += now; + } + CHECK(secp256k1_scalar_eq(&n, &s)); + } + +#ifndef USE_NUM_NONE + { + /* Test that adding the scalars together is equal to adding their numbers together modulo the order. */ + secp256k1_num rnum; + secp256k1_num r2num; + secp256k1_scalar r; + secp256k1_num_add(&rnum, &snum, &s2num); + secp256k1_num_mod(&rnum, &order); + secp256k1_scalar_add(&r, &s, &s2); + secp256k1_scalar_get_num(&r2num, &r); + CHECK(secp256k1_num_eq(&rnum, &r2num)); + } + + { + /* Test that multiplying the scalars is equal to multiplying their numbers modulo the order. */ + secp256k1_scalar r; + secp256k1_num r2num; + secp256k1_num rnum; + secp256k1_num_mul(&rnum, &snum, &s2num); + secp256k1_num_mod(&rnum, &order); + secp256k1_scalar_mul(&r, &s, &s2); + secp256k1_scalar_get_num(&r2num, &r); + CHECK(secp256k1_num_eq(&rnum, &r2num)); + /* The result can only be zero if at least one of the factors was zero. */ + CHECK(secp256k1_scalar_is_zero(&r) == (secp256k1_scalar_is_zero(&s) || secp256k1_scalar_is_zero(&s2))); + /* The results can only be equal to one of the factors if that factor was zero, or the other factor was one. */ + CHECK(secp256k1_num_eq(&rnum, &snum) == (secp256k1_scalar_is_zero(&s) || secp256k1_scalar_is_one(&s2))); + CHECK(secp256k1_num_eq(&rnum, &s2num) == (secp256k1_scalar_is_zero(&s2) || secp256k1_scalar_is_one(&s))); + } + + { + secp256k1_scalar neg; + secp256k1_num negnum; + secp256k1_num negnum2; + /* Check that comparison with zero matches comparison with zero on the number. */ + CHECK(secp256k1_num_is_zero(&snum) == secp256k1_scalar_is_zero(&s)); + /* Check that comparison with the half order is equal to testing for high scalar. */ + CHECK(secp256k1_scalar_is_high(&s) == (secp256k1_num_cmp(&snum, &half_order) > 0)); + secp256k1_scalar_negate(&neg, &s); + secp256k1_num_sub(&negnum, &order, &snum); + secp256k1_num_mod(&negnum, &order); + /* Check that comparison with the half order is equal to testing for high scalar after negation. */ + CHECK(secp256k1_scalar_is_high(&neg) == (secp256k1_num_cmp(&negnum, &half_order) > 0)); + /* Negating should change the high property, unless the value was already zero. */ + CHECK((secp256k1_scalar_is_high(&s) == secp256k1_scalar_is_high(&neg)) == secp256k1_scalar_is_zero(&s)); + secp256k1_scalar_get_num(&negnum2, &neg); + /* Negating a scalar should be equal to (order - n) mod order on the number. */ + CHECK(secp256k1_num_eq(&negnum, &negnum2)); + secp256k1_scalar_add(&neg, &neg, &s); + /* Adding a number to its negation should result in zero. */ + CHECK(secp256k1_scalar_is_zero(&neg)); + secp256k1_scalar_negate(&neg, &neg); + /* Negating zero should still result in zero. */ + CHECK(secp256k1_scalar_is_zero(&neg)); + } + + { + /* Test secp256k1_scalar_mul_shift_var. */ + secp256k1_scalar r; + secp256k1_num one; + secp256k1_num rnum; + secp256k1_num rnum2; + unsigned char cone[1] = {0x01}; + unsigned int shift = 256 + secp256k1_rand_int(257); + secp256k1_scalar_mul_shift_var(&r, &s1, &s2, shift); + secp256k1_num_mul(&rnum, &s1num, &s2num); + secp256k1_num_shift(&rnum, shift - 1); + secp256k1_num_set_bin(&one, cone, 1); + secp256k1_num_add(&rnum, &rnum, &one); + secp256k1_num_shift(&rnum, 1); + secp256k1_scalar_get_num(&rnum2, &r); + CHECK(secp256k1_num_eq(&rnum, &rnum2)); + } + + { + /* test secp256k1_scalar_shr_int */ + secp256k1_scalar r; + int i; + random_scalar_order_test(&r); + for (i = 0; i < 100; ++i) { + int low; + int shift = 1 + secp256k1_rand_int(15); + int expected = r.d[0] % (1 << shift); + low = secp256k1_scalar_shr_int(&r, shift); + CHECK(expected == low); + } + } +#endif + + { + /* Test that scalar inverses are equal to the inverse of their number modulo the order. */ + if (!secp256k1_scalar_is_zero(&s)) { + secp256k1_scalar inv; +#ifndef USE_NUM_NONE + secp256k1_num invnum; + secp256k1_num invnum2; +#endif + secp256k1_scalar_inverse(&inv, &s); +#ifndef USE_NUM_NONE + secp256k1_num_mod_inverse(&invnum, &snum, &order); + secp256k1_scalar_get_num(&invnum2, &inv); + CHECK(secp256k1_num_eq(&invnum, &invnum2)); +#endif + secp256k1_scalar_mul(&inv, &inv, &s); + /* Multiplying a scalar with its inverse must result in one. */ + CHECK(secp256k1_scalar_is_one(&inv)); + secp256k1_scalar_inverse(&inv, &inv); + /* Inverting one must result in one. */ + CHECK(secp256k1_scalar_is_one(&inv)); + } + } + + { + /* Test commutativity of add. */ + secp256k1_scalar r1, r2; + secp256k1_scalar_add(&r1, &s1, &s2); + secp256k1_scalar_add(&r2, &s2, &s1); + CHECK(secp256k1_scalar_eq(&r1, &r2)); + } + + { + secp256k1_scalar r1, r2; + secp256k1_scalar b; + int i; + /* Test add_bit. */ + int bit = secp256k1_rand_bits(8); + secp256k1_scalar_set_int(&b, 1); + CHECK(secp256k1_scalar_is_one(&b)); + for (i = 0; i < bit; i++) { + secp256k1_scalar_add(&b, &b, &b); + } + r1 = s1; + r2 = s1; + if (!secp256k1_scalar_add(&r1, &r1, &b)) { + /* No overflow happened. */ + secp256k1_scalar_cadd_bit(&r2, bit, 1); + CHECK(secp256k1_scalar_eq(&r1, &r2)); + /* cadd is a noop when flag is zero */ + secp256k1_scalar_cadd_bit(&r2, bit, 0); + CHECK(secp256k1_scalar_eq(&r1, &r2)); + } + } + + { + /* Test commutativity of mul. */ + secp256k1_scalar r1, r2; + secp256k1_scalar_mul(&r1, &s1, &s2); + secp256k1_scalar_mul(&r2, &s2, &s1); + CHECK(secp256k1_scalar_eq(&r1, &r2)); + } + + { + /* Test associativity of add. */ + secp256k1_scalar r1, r2; + secp256k1_scalar_add(&r1, &s1, &s2); + secp256k1_scalar_add(&r1, &r1, &s); + secp256k1_scalar_add(&r2, &s2, &s); + secp256k1_scalar_add(&r2, &s1, &r2); + CHECK(secp256k1_scalar_eq(&r1, &r2)); + } + + { + /* Test associativity of mul. */ + secp256k1_scalar r1, r2; + secp256k1_scalar_mul(&r1, &s1, &s2); + secp256k1_scalar_mul(&r1, &r1, &s); + secp256k1_scalar_mul(&r2, &s2, &s); + secp256k1_scalar_mul(&r2, &s1, &r2); + CHECK(secp256k1_scalar_eq(&r1, &r2)); + } + + { + /* Test distributitivity of mul over add. */ + secp256k1_scalar r1, r2, t; + secp256k1_scalar_add(&r1, &s1, &s2); + secp256k1_scalar_mul(&r1, &r1, &s); + secp256k1_scalar_mul(&r2, &s1, &s); + secp256k1_scalar_mul(&t, &s2, &s); + secp256k1_scalar_add(&r2, &r2, &t); + CHECK(secp256k1_scalar_eq(&r1, &r2)); + } + + { + /* Test square. */ + secp256k1_scalar r1, r2; + secp256k1_scalar_sqr(&r1, &s1); + secp256k1_scalar_mul(&r2, &s1, &s1); + CHECK(secp256k1_scalar_eq(&r1, &r2)); + } + + { + /* Test multiplicative identity. */ + secp256k1_scalar r1, v1; + secp256k1_scalar_set_int(&v1,1); + secp256k1_scalar_mul(&r1, &s1, &v1); + CHECK(secp256k1_scalar_eq(&r1, &s1)); + } + + { + /* Test additive identity. */ + secp256k1_scalar r1, v0; + secp256k1_scalar_set_int(&v0,0); + secp256k1_scalar_add(&r1, &s1, &v0); + CHECK(secp256k1_scalar_eq(&r1, &s1)); + } + + { + /* Test zero product property. */ + secp256k1_scalar r1, v0; + secp256k1_scalar_set_int(&v0,0); + secp256k1_scalar_mul(&r1, &s1, &v0); + CHECK(secp256k1_scalar_eq(&r1, &v0)); + } + +} + +void run_scalar_tests(void) { + int i; + for (i = 0; i < 128 * count; i++) { + scalar_test(); + } + + { + /* (-1)+1 should be zero. */ + secp256k1_scalar s, o; + secp256k1_scalar_set_int(&s, 1); + CHECK(secp256k1_scalar_is_one(&s)); + secp256k1_scalar_negate(&o, &s); + secp256k1_scalar_add(&o, &o, &s); + CHECK(secp256k1_scalar_is_zero(&o)); + secp256k1_scalar_negate(&o, &o); + CHECK(secp256k1_scalar_is_zero(&o)); + } + +#ifndef USE_NUM_NONE + { + /* A scalar with value of the curve order should be 0. */ + secp256k1_num order; + secp256k1_scalar zero; + unsigned char bin[32]; + int overflow = 0; + secp256k1_scalar_order_get_num(&order); + secp256k1_num_get_bin(bin, 32, &order); + secp256k1_scalar_set_b32(&zero, bin, &overflow); + CHECK(overflow == 1); + CHECK(secp256k1_scalar_is_zero(&zero)); + } +#endif + + { + /* Does check_overflow check catch all ones? */ + static const secp256k1_scalar overflowed = SECP256K1_SCALAR_CONST( + 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFFUL, + 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFFUL + ); + CHECK(secp256k1_scalar_check_overflow(&overflowed)); + } + + { + /* Static test vectors. + * These were reduced from ~10^12 random vectors based on comparison-decision + * and edge-case coverage on 32-bit and 64-bit implementations. + * The responses were generated with Sage 5.9. + */ + secp256k1_scalar x; + secp256k1_scalar y; + secp256k1_scalar z; + secp256k1_scalar zz; + secp256k1_scalar one; + secp256k1_scalar r1; + secp256k1_scalar r2; +#if defined(USE_SCALAR_INV_NUM) + secp256k1_scalar zzv; +#endif + int overflow; + unsigned char chal[32][2][32] = { + {{0xff, 0xff, 0x03, 0x07, 0x00, 0x00, 0x00, 0x00, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x03, + 0x00, 0x00, 0x00, 0x00, 0x00, 0xf8, 0xff, 0xff, + 0xff, 0xff, 0x03, 0x00, 0xc0, 0xff, 0xff, 0xff}, + {0xff, 0xff, 0xff, 0xff, 0xff, 0x0f, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf8, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0x03, 0x00, 0x00, 0x00, 0x00, 0xe0, 0xff}}, + {{0xef, 0xff, 0x1f, 0x00, 0x00, 0x00, 0x00, 0x00, + 0xfe, 0xff, 0xff, 0xff, 0xff, 0xff, 0x3f, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}, + {0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe0, + 0xff, 0xff, 0xff, 0xff, 0xfc, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0x7f, 0x00, 0x80, 0xff}}, + {{0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, + 0x80, 0x00, 0x00, 0x80, 0xff, 0x3f, 0x00, 0x00, + 0x00, 0x00, 0x00, 0xf8, 0xff, 0xff, 0xff, 0x00}, + {0x00, 0x00, 0xfc, 0xff, 0xff, 0xff, 0xff, 0x80, + 0xff, 0xff, 0xff, 0xff, 0xff, 0x0f, 0x00, 0xe0, + 0xff, 0xff, 0xff, 0xff, 0xff, 0x7f, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x7f, 0xff, 0xff, 0xff}}, + {{0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x80, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, + 0x00, 0x1e, 0xf8, 0xff, 0xff, 0xff, 0xfd, 0xff}, + {0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x1f, + 0x00, 0x00, 0x00, 0xf8, 0xff, 0x03, 0x00, 0xe0, + 0xff, 0x0f, 0x00, 0x00, 0x00, 0x00, 0xf0, 0xff, + 0xf3, 0xff, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00}}, + {{0x80, 0x00, 0x00, 0x80, 0xff, 0xff, 0xff, 0x00, + 0x00, 0x1c, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xe0, 0xff, 0xff, 0xff, 0x00, + 0x00, 0x00, 0x00, 0x00, 0xe0, 0xff, 0xff, 0xff}, + {0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x03, 0x00, + 0xf8, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0x1f, 0x00, 0x00, 0x80, 0xff, 0xff, 0x3f, + 0x00, 0xfe, 0xff, 0xff, 0xff, 0xdf, 0xff, 0xff}}, + {{0xff, 0xff, 0xff, 0xff, 0x00, 0x0f, 0xfc, 0x9f, + 0xff, 0xff, 0xff, 0x00, 0x80, 0x00, 0x00, 0x80, + 0xff, 0x0f, 0xfc, 0xff, 0x7f, 0x00, 0x00, 0x00, + 0x00, 0xf8, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00}, + {0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, + 0x00, 0x00, 0xf8, 0xff, 0x0f, 0xc0, 0xff, 0xff, + 0xff, 0x1f, 0x00, 0x00, 0x00, 0xc0, 0xff, 0xff, + 0xff, 0xff, 0xff, 0x07, 0x80, 0xff, 0xff, 0xff}}, + {{0xff, 0xff, 0xff, 0xff, 0xff, 0x3f, 0x00, 0x00, + 0x80, 0x00, 0x00, 0x80, 0xff, 0xff, 0xff, 0xff, + 0xf7, 0xff, 0xff, 0xef, 0xff, 0xff, 0xff, 0x00, + 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0xf0}, + {0x00, 0x00, 0x00, 0x00, 0xf8, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0x01, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x80, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}}, + {{0x00, 0xf8, 0xff, 0x03, 0xff, 0xff, 0xff, 0x00, + 0x00, 0xfe, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, + 0x80, 0x00, 0x00, 0x80, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0x03, 0xc0, 0xff, 0x0f, 0xfc, 0xff}, + {0xff, 0xff, 0xff, 0xff, 0xff, 0xe0, 0xff, 0xff, + 0xff, 0x01, 0x00, 0x00, 0x00, 0x3f, 0x00, 0xc0, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}}, + {{0x8f, 0x0f, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0xf8, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0x7f, 0x00, 0x00, 0x80, 0x00, 0x00, 0x80, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00}, + {0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0x0f, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}}, + {{0x00, 0x00, 0x00, 0xc0, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0x03, 0x00, 0x80, 0x00, 0x00, 0x80, + 0xff, 0xff, 0xff, 0x00, 0x00, 0x80, 0xff, 0x7f}, + {0xff, 0xcf, 0xff, 0xff, 0x01, 0x00, 0x00, 0x00, + 0x00, 0xc0, 0xff, 0xcf, 0xff, 0xff, 0xff, 0xff, + 0xbf, 0xff, 0x0e, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x80, 0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00}}, + {{0x00, 0x00, 0x00, 0x00, 0x00, 0x80, 0xff, 0xff, + 0xff, 0xff, 0x00, 0xfc, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0x00, 0x80, 0x00, 0x00, 0x80, + 0xff, 0x01, 0xfc, 0xff, 0x01, 0x00, 0xfe, 0xff}, + {0xff, 0xff, 0xff, 0x03, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x03, 0x00}}, + {{0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, + 0xe0, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0x00, 0xf8, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0x7f, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x80}, + {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0xf8, 0xff, 0x01, 0x00, 0xf0, 0xff, 0xff, + 0xe0, 0xff, 0x0f, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}}, + {{0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0xf8, 0xff, 0x00}, + {0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, 0x00, + 0xfc, 0xff, 0xff, 0x3f, 0xf0, 0xff, 0xff, 0x3f, + 0x00, 0x00, 0xf8, 0x07, 0x00, 0x00, 0x00, 0xff, + 0xff, 0xff, 0xff, 0xff, 0x0f, 0x7e, 0x00, 0x00}}, + {{0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x80, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0x1f, 0x00, 0x00, 0xfe, 0x07, 0x00}, + {0x00, 0x00, 0x00, 0xf0, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xfb, 0xff, 0x07, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x60}}, + {{0xff, 0x01, 0x00, 0xff, 0xff, 0xff, 0x0f, 0x00, + 0x80, 0x7f, 0xfe, 0xff, 0xff, 0xff, 0xff, 0x03, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x80, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}, + {0xff, 0xff, 0x1f, 0x00, 0xf0, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0x3f, 0x00, 0x00, 0x00, 0x00}}, + {{0x80, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}, + {0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xf1, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x03, + 0x00, 0x00, 0x00, 0xe0, 0xff, 0xff, 0xff, 0xff}}, + {{0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, + 0x7e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0xc0, 0xff, 0xff, 0xcf, 0xff, 0x1f, 0x00, 0x00, + 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80}, + {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0xe0, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0x3f, 0x00, 0x7e, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}}, + {{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0xfc, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0x00}, + {0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, + 0xff, 0xff, 0x7f, 0x00, 0x80, 0x00, 0x00, 0x00, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, + 0x00, 0x00, 0xe0, 0xff, 0xff, 0xff, 0xff, 0xff}}, + {{0xff, 0xff, 0xff, 0xff, 0xff, 0x1f, 0x00, 0x80, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, + 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00}, + {0xf0, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0x3f, 0x00, 0x00, 0x80, + 0xff, 0x01, 0x00, 0x00, 0x00, 0x00, 0xff, 0xff, + 0xff, 0x7f, 0xf8, 0xff, 0xff, 0x1f, 0x00, 0xfe}}, + {{0xff, 0xff, 0xff, 0x3f, 0xf8, 0xff, 0xff, 0xff, + 0xff, 0x03, 0xfe, 0x01, 0x00, 0x00, 0x00, 0x00, + 0xf0, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x07}, + {0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, + 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, + 0xff, 0xff, 0xff, 0xff, 0x01, 0x80, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00}}, + {{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}, + {0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, + 0xba, 0xae, 0xdc, 0xe6, 0xaf, 0x48, 0xa0, 0x3b, + 0xbf, 0xd2, 0x5e, 0x8c, 0xd0, 0x36, 0x41, 0x40}}, + {{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}, + {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}}, + {{0x7f, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}, + {0x7f, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}}, + {{0xff, 0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0xc0, + 0xff, 0x0f, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0xf0, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x7f}, + {0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x01, 0x00, + 0xf0, 0xff, 0xff, 0xff, 0xff, 0x07, 0x00, 0x00, + 0x00, 0x00, 0x00, 0xfe, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0x01, 0xff, 0xff, 0xff}}, + {{0x7f, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}, + {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02}}, + {{0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, + 0xba, 0xae, 0xdc, 0xe6, 0xaf, 0x48, 0xa0, 0x3b, + 0xbf, 0xd2, 0x5e, 0x8c, 0xd0, 0x36, 0x41, 0x40}, + {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}}, + {{0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0x7e, 0x00, 0x00, 0xc0, 0xff, 0xff, 0x07, 0x00, + 0x80, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x00, + 0xfc, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}, + {0xff, 0x01, 0x00, 0x00, 0x00, 0xe0, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0x1f, 0x00, 0x80, + 0xff, 0xff, 0xff, 0xff, 0xff, 0x03, 0x00, 0x00, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}}, + {{0xff, 0xff, 0xf0, 0xff, 0xff, 0xff, 0xff, 0x00, + 0xf0, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, + 0x00, 0xe0, 0xff, 0xff, 0xff, 0xff, 0xff, 0x01, + 0x80, 0x00, 0x00, 0x80, 0xff, 0xff, 0xff, 0xff}, + {0x00, 0x00, 0x00, 0x00, 0x00, 0xe0, 0xff, 0xff, + 0xff, 0xff, 0x3f, 0x00, 0xf8, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0x3f, 0x00, 0x00, 0xc0, 0xf1, 0x7f, 0x00}}, + {{0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0xc0, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x80, 0x00, 0x00, 0x80, 0xff, 0xff, 0xff, 0x00}, + {0x00, 0xf8, 0xff, 0xff, 0xff, 0xff, 0xff, 0x01, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf8, 0xff, + 0xff, 0x7f, 0x00, 0x00, 0x00, 0x00, 0x80, 0x1f, + 0x00, 0x00, 0xfc, 0xff, 0xff, 0x01, 0xff, 0xff}}, + {{0x00, 0xfe, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, + 0x80, 0x00, 0x00, 0x80, 0xff, 0x03, 0xe0, 0x01, + 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0xfc, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00}, + {0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, + 0xfe, 0xff, 0xff, 0xf0, 0x07, 0x00, 0x3c, 0x80, + 0xff, 0xff, 0xff, 0xff, 0xfc, 0xff, 0xff, 0xff, + 0xff, 0xff, 0x07, 0xe0, 0xff, 0x00, 0x00, 0x00}}, + {{0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, + 0xfc, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x07, 0xf8, + 0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x80}, + {0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0x0c, 0x80, 0x00, + 0x00, 0x00, 0x00, 0xc0, 0x7f, 0xfe, 0xff, 0x1f, + 0x00, 0xfe, 0xff, 0x03, 0x00, 0x00, 0xfe, 0xff}}, + {{0xff, 0xff, 0x81, 0xff, 0xff, 0xff, 0xff, 0x00, + 0x80, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x83, + 0xff, 0xff, 0x00, 0x00, 0x80, 0x00, 0x00, 0x80, + 0xff, 0xff, 0x7f, 0x00, 0x00, 0x00, 0x00, 0xf0}, + {0xff, 0x01, 0x00, 0x00, 0x00, 0x00, 0xf8, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0x1f, 0x00, 0x00, + 0xf8, 0x07, 0x00, 0x80, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xc7, 0xff, 0xff, 0xe0, 0xff, 0xff, 0xff}} + }; + unsigned char res[32][2][32] = { + {{0x0c, 0x3b, 0x0a, 0xca, 0x8d, 0x1a, 0x2f, 0xb9, + 0x8a, 0x7b, 0x53, 0x5a, 0x1f, 0xc5, 0x22, 0xa1, + 0x07, 0x2a, 0x48, 0xea, 0x02, 0xeb, 0xb3, 0xd6, + 0x20, 0x1e, 0x86, 0xd0, 0x95, 0xf6, 0x92, 0x35}, + {0xdc, 0x90, 0x7a, 0x07, 0x2e, 0x1e, 0x44, 0x6d, + 0xf8, 0x15, 0x24, 0x5b, 0x5a, 0x96, 0x37, 0x9c, + 0x37, 0x7b, 0x0d, 0xac, 0x1b, 0x65, 0x58, 0x49, + 0x43, 0xb7, 0x31, 0xbb, 0xa7, 0xf4, 0x97, 0x15}}, + {{0xf1, 0xf7, 0x3a, 0x50, 0xe6, 0x10, 0xba, 0x22, + 0x43, 0x4d, 0x1f, 0x1f, 0x7c, 0x27, 0xca, 0x9c, + 0xb8, 0xb6, 0xa0, 0xfc, 0xd8, 0xc0, 0x05, 0x2f, + 0xf7, 0x08, 0xe1, 0x76, 0xdd, 0xd0, 0x80, 0xc8}, + {0xe3, 0x80, 0x80, 0xb8, 0xdb, 0xe3, 0xa9, 0x77, + 0x00, 0xb0, 0xf5, 0x2e, 0x27, 0xe2, 0x68, 0xc4, + 0x88, 0xe8, 0x04, 0xc1, 0x12, 0xbf, 0x78, 0x59, + 0xe6, 0xa9, 0x7c, 0xe1, 0x81, 0xdd, 0xb9, 0xd5}}, + {{0x96, 0xe2, 0xee, 0x01, 0xa6, 0x80, 0x31, 0xef, + 0x5c, 0xd0, 0x19, 0xb4, 0x7d, 0x5f, 0x79, 0xab, + 0xa1, 0x97, 0xd3, 0x7e, 0x33, 0xbb, 0x86, 0x55, + 0x60, 0x20, 0x10, 0x0d, 0x94, 0x2d, 0x11, 0x7c}, + {0xcc, 0xab, 0xe0, 0xe8, 0x98, 0x65, 0x12, 0x96, + 0x38, 0x5a, 0x1a, 0xf2, 0x85, 0x23, 0x59, 0x5f, + 0xf9, 0xf3, 0xc2, 0x81, 0x70, 0x92, 0x65, 0x12, + 0x9c, 0x65, 0x1e, 0x96, 0x00, 0xef, 0xe7, 0x63}}, + {{0xac, 0x1e, 0x62, 0xc2, 0x59, 0xfc, 0x4e, 0x5c, + 0x83, 0xb0, 0xd0, 0x6f, 0xce, 0x19, 0xf6, 0xbf, + 0xa4, 0xb0, 0xe0, 0x53, 0x66, 0x1f, 0xbf, 0xc9, + 0x33, 0x47, 0x37, 0xa9, 0x3d, 0x5d, 0xb0, 0x48}, + {0x86, 0xb9, 0x2a, 0x7f, 0x8e, 0xa8, 0x60, 0x42, + 0x26, 0x6d, 0x6e, 0x1c, 0xa2, 0xec, 0xe0, 0xe5, + 0x3e, 0x0a, 0x33, 0xbb, 0x61, 0x4c, 0x9f, 0x3c, + 0xd1, 0xdf, 0x49, 0x33, 0xcd, 0x72, 0x78, 0x18}}, + {{0xf7, 0xd3, 0xcd, 0x49, 0x5c, 0x13, 0x22, 0xfb, + 0x2e, 0xb2, 0x2f, 0x27, 0xf5, 0x8a, 0x5d, 0x74, + 0xc1, 0x58, 0xc5, 0xc2, 0x2d, 0x9f, 0x52, 0xc6, + 0x63, 0x9f, 0xba, 0x05, 0x76, 0x45, 0x7a, 0x63}, + {0x8a, 0xfa, 0x55, 0x4d, 0xdd, 0xa3, 0xb2, 0xc3, + 0x44, 0xfd, 0xec, 0x72, 0xde, 0xef, 0xc0, 0x99, + 0xf5, 0x9f, 0xe2, 0x52, 0xb4, 0x05, 0x32, 0x58, + 0x57, 0xc1, 0x8f, 0xea, 0xc3, 0x24, 0x5b, 0x94}}, + {{0x05, 0x83, 0xee, 0xdd, 0x64, 0xf0, 0x14, 0x3b, + 0xa0, 0x14, 0x4a, 0x3a, 0x41, 0x82, 0x7c, 0xa7, + 0x2c, 0xaa, 0xb1, 0x76, 0xbb, 0x59, 0x64, 0x5f, + 0x52, 0xad, 0x25, 0x29, 0x9d, 0x8f, 0x0b, 0xb0}, + {0x7e, 0xe3, 0x7c, 0xca, 0xcd, 0x4f, 0xb0, 0x6d, + 0x7a, 0xb2, 0x3e, 0xa0, 0x08, 0xb9, 0xa8, 0x2d, + 0xc2, 0xf4, 0x99, 0x66, 0xcc, 0xac, 0xd8, 0xb9, + 0x72, 0x2a, 0x4a, 0x3e, 0x0f, 0x7b, 0xbf, 0xf4}}, + {{0x8c, 0x9c, 0x78, 0x2b, 0x39, 0x61, 0x7e, 0xf7, + 0x65, 0x37, 0x66, 0x09, 0x38, 0xb9, 0x6f, 0x70, + 0x78, 0x87, 0xff, 0xcf, 0x93, 0xca, 0x85, 0x06, + 0x44, 0x84, 0xa7, 0xfe, 0xd3, 0xa4, 0xe3, 0x7e}, + {0xa2, 0x56, 0x49, 0x23, 0x54, 0xa5, 0x50, 0xe9, + 0x5f, 0xf0, 0x4d, 0xe7, 0xdc, 0x38, 0x32, 0x79, + 0x4f, 0x1c, 0xb7, 0xe4, 0xbb, 0xf8, 0xbb, 0x2e, + 0x40, 0x41, 0x4b, 0xcc, 0xe3, 0x1e, 0x16, 0x36}}, + {{0x0c, 0x1e, 0xd7, 0x09, 0x25, 0x40, 0x97, 0xcb, + 0x5c, 0x46, 0xa8, 0xda, 0xef, 0x25, 0xd5, 0xe5, + 0x92, 0x4d, 0xcf, 0xa3, 0xc4, 0x5d, 0x35, 0x4a, + 0xe4, 0x61, 0x92, 0xf3, 0xbf, 0x0e, 0xcd, 0xbe}, + {0xe4, 0xaf, 0x0a, 0xb3, 0x30, 0x8b, 0x9b, 0x48, + 0x49, 0x43, 0xc7, 0x64, 0x60, 0x4a, 0x2b, 0x9e, + 0x95, 0x5f, 0x56, 0xe8, 0x35, 0xdc, 0xeb, 0xdc, + 0xc7, 0xc4, 0xfe, 0x30, 0x40, 0xc7, 0xbf, 0xa4}}, + {{0xd4, 0xa0, 0xf5, 0x81, 0x49, 0x6b, 0xb6, 0x8b, + 0x0a, 0x69, 0xf9, 0xfe, 0xa8, 0x32, 0xe5, 0xe0, + 0xa5, 0xcd, 0x02, 0x53, 0xf9, 0x2c, 0xe3, 0x53, + 0x83, 0x36, 0xc6, 0x02, 0xb5, 0xeb, 0x64, 0xb8}, + {0x1d, 0x42, 0xb9, 0xf9, 0xe9, 0xe3, 0x93, 0x2c, + 0x4c, 0xee, 0x6c, 0x5a, 0x47, 0x9e, 0x62, 0x01, + 0x6b, 0x04, 0xfe, 0xa4, 0x30, 0x2b, 0x0d, 0x4f, + 0x71, 0x10, 0xd3, 0x55, 0xca, 0xf3, 0x5e, 0x80}}, + {{0x77, 0x05, 0xf6, 0x0c, 0x15, 0x9b, 0x45, 0xe7, + 0xb9, 0x11, 0xb8, 0xf5, 0xd6, 0xda, 0x73, 0x0c, + 0xda, 0x92, 0xea, 0xd0, 0x9d, 0xd0, 0x18, 0x92, + 0xce, 0x9a, 0xaa, 0xee, 0x0f, 0xef, 0xde, 0x30}, + {0xf1, 0xf1, 0xd6, 0x9b, 0x51, 0xd7, 0x77, 0x62, + 0x52, 0x10, 0xb8, 0x7a, 0x84, 0x9d, 0x15, 0x4e, + 0x07, 0xdc, 0x1e, 0x75, 0x0d, 0x0c, 0x3b, 0xdb, + 0x74, 0x58, 0x62, 0x02, 0x90, 0x54, 0x8b, 0x43}}, + {{0xa6, 0xfe, 0x0b, 0x87, 0x80, 0x43, 0x67, 0x25, + 0x57, 0x5d, 0xec, 0x40, 0x50, 0x08, 0xd5, 0x5d, + 0x43, 0xd7, 0xe0, 0xaa, 0xe0, 0x13, 0xb6, 0xb0, + 0xc0, 0xd4, 0xe5, 0x0d, 0x45, 0x83, 0xd6, 0x13}, + {0x40, 0x45, 0x0a, 0x92, 0x31, 0xea, 0x8c, 0x60, + 0x8c, 0x1f, 0xd8, 0x76, 0x45, 0xb9, 0x29, 0x00, + 0x26, 0x32, 0xd8, 0xa6, 0x96, 0x88, 0xe2, 0xc4, + 0x8b, 0xdb, 0x7f, 0x17, 0x87, 0xcc, 0xc8, 0xf2}}, + {{0xc2, 0x56, 0xe2, 0xb6, 0x1a, 0x81, 0xe7, 0x31, + 0x63, 0x2e, 0xbb, 0x0d, 0x2f, 0x81, 0x67, 0xd4, + 0x22, 0xe2, 0x38, 0x02, 0x25, 0x97, 0xc7, 0x88, + 0x6e, 0xdf, 0xbe, 0x2a, 0xa5, 0x73, 0x63, 0xaa}, + {0x50, 0x45, 0xe2, 0xc3, 0xbd, 0x89, 0xfc, 0x57, + 0xbd, 0x3c, 0xa3, 0x98, 0x7e, 0x7f, 0x36, 0x38, + 0x92, 0x39, 0x1f, 0x0f, 0x81, 0x1a, 0x06, 0x51, + 0x1f, 0x8d, 0x6a, 0xff, 0x47, 0x16, 0x06, 0x9c}}, + {{0x33, 0x95, 0xa2, 0x6f, 0x27, 0x5f, 0x9c, 0x9c, + 0x64, 0x45, 0xcb, 0xd1, 0x3c, 0xee, 0x5e, 0x5f, + 0x48, 0xa6, 0xaf, 0xe3, 0x79, 0xcf, 0xb1, 0xe2, + 0xbf, 0x55, 0x0e, 0xa2, 0x3b, 0x62, 0xf0, 0xe4}, + {0x14, 0xe8, 0x06, 0xe3, 0xbe, 0x7e, 0x67, 0x01, + 0xc5, 0x21, 0x67, 0xd8, 0x54, 0xb5, 0x7f, 0xa4, + 0xf9, 0x75, 0x70, 0x1c, 0xfd, 0x79, 0xdb, 0x86, + 0xad, 0x37, 0x85, 0x83, 0x56, 0x4e, 0xf0, 0xbf}}, + {{0xbc, 0xa6, 0xe0, 0x56, 0x4e, 0xef, 0xfa, 0xf5, + 0x1d, 0x5d, 0x3f, 0x2a, 0x5b, 0x19, 0xab, 0x51, + 0xc5, 0x8b, 0xdd, 0x98, 0x28, 0x35, 0x2f, 0xc3, + 0x81, 0x4f, 0x5c, 0xe5, 0x70, 0xb9, 0xeb, 0x62}, + {0xc4, 0x6d, 0x26, 0xb0, 0x17, 0x6b, 0xfe, 0x6c, + 0x12, 0xf8, 0xe7, 0xc1, 0xf5, 0x2f, 0xfa, 0x91, + 0x13, 0x27, 0xbd, 0x73, 0xcc, 0x33, 0x31, 0x1c, + 0x39, 0xe3, 0x27, 0x6a, 0x95, 0xcf, 0xc5, 0xfb}}, + {{0x30, 0xb2, 0x99, 0x84, 0xf0, 0x18, 0x2a, 0x6e, + 0x1e, 0x27, 0xed, 0xa2, 0x29, 0x99, 0x41, 0x56, + 0xe8, 0xd4, 0x0d, 0xef, 0x99, 0x9c, 0xf3, 0x58, + 0x29, 0x55, 0x1a, 0xc0, 0x68, 0xd6, 0x74, 0xa4}, + {0x07, 0x9c, 0xe7, 0xec, 0xf5, 0x36, 0x73, 0x41, + 0xa3, 0x1c, 0xe5, 0x93, 0x97, 0x6a, 0xfd, 0xf7, + 0x53, 0x18, 0xab, 0xaf, 0xeb, 0x85, 0xbd, 0x92, + 0x90, 0xab, 0x3c, 0xbf, 0x30, 0x82, 0xad, 0xf6}}, + {{0xc6, 0x87, 0x8a, 0x2a, 0xea, 0xc0, 0xa9, 0xec, + 0x6d, 0xd3, 0xdc, 0x32, 0x23, 0xce, 0x62, 0x19, + 0xa4, 0x7e, 0xa8, 0xdd, 0x1c, 0x33, 0xae, 0xd3, + 0x4f, 0x62, 0x9f, 0x52, 0xe7, 0x65, 0x46, 0xf4}, + {0x97, 0x51, 0x27, 0x67, 0x2d, 0xa2, 0x82, 0x87, + 0x98, 0xd3, 0xb6, 0x14, 0x7f, 0x51, 0xd3, 0x9a, + 0x0b, 0xd0, 0x76, 0x81, 0xb2, 0x4f, 0x58, 0x92, + 0xa4, 0x86, 0xa1, 0xa7, 0x09, 0x1d, 0xef, 0x9b}}, + {{0xb3, 0x0f, 0x2b, 0x69, 0x0d, 0x06, 0x90, 0x64, + 0xbd, 0x43, 0x4c, 0x10, 0xe8, 0x98, 0x1c, 0xa3, + 0xe1, 0x68, 0xe9, 0x79, 0x6c, 0x29, 0x51, 0x3f, + 0x41, 0xdc, 0xdf, 0x1f, 0xf3, 0x60, 0xbe, 0x33}, + {0xa1, 0x5f, 0xf7, 0x1d, 0xb4, 0x3e, 0x9b, 0x3c, + 0xe7, 0xbd, 0xb6, 0x06, 0xd5, 0x60, 0x06, 0x6d, + 0x50, 0xd2, 0xf4, 0x1a, 0x31, 0x08, 0xf2, 0xea, + 0x8e, 0xef, 0x5f, 0x7d, 0xb6, 0xd0, 0xc0, 0x27}}, + {{0x62, 0x9a, 0xd9, 0xbb, 0x38, 0x36, 0xce, 0xf7, + 0x5d, 0x2f, 0x13, 0xec, 0xc8, 0x2d, 0x02, 0x8a, + 0x2e, 0x72, 0xf0, 0xe5, 0x15, 0x9d, 0x72, 0xae, + 0xfc, 0xb3, 0x4f, 0x02, 0xea, 0xe1, 0x09, 0xfe}, + {0x00, 0x00, 0x00, 0x00, 0xfa, 0x0a, 0x3d, 0xbc, + 0xad, 0x16, 0x0c, 0xb6, 0xe7, 0x7c, 0x8b, 0x39, + 0x9a, 0x43, 0xbb, 0xe3, 0xc2, 0x55, 0x15, 0x14, + 0x75, 0xac, 0x90, 0x9b, 0x7f, 0x9a, 0x92, 0x00}}, + {{0x8b, 0xac, 0x70, 0x86, 0x29, 0x8f, 0x00, 0x23, + 0x7b, 0x45, 0x30, 0xaa, 0xb8, 0x4c, 0xc7, 0x8d, + 0x4e, 0x47, 0x85, 0xc6, 0x19, 0xe3, 0x96, 0xc2, + 0x9a, 0xa0, 0x12, 0xed, 0x6f, 0xd7, 0x76, 0x16}, + {0x45, 0xaf, 0x7e, 0x33, 0xc7, 0x7f, 0x10, 0x6c, + 0x7c, 0x9f, 0x29, 0xc1, 0xa8, 0x7e, 0x15, 0x84, + 0xe7, 0x7d, 0xc0, 0x6d, 0xab, 0x71, 0x5d, 0xd0, + 0x6b, 0x9f, 0x97, 0xab, 0xcb, 0x51, 0x0c, 0x9f}}, + {{0x9e, 0xc3, 0x92, 0xb4, 0x04, 0x9f, 0xc8, 0xbb, + 0xdd, 0x9e, 0xc6, 0x05, 0xfd, 0x65, 0xec, 0x94, + 0x7f, 0x2c, 0x16, 0xc4, 0x40, 0xac, 0x63, 0x7b, + 0x7d, 0xb8, 0x0c, 0xe4, 0x5b, 0xe3, 0xa7, 0x0e}, + {0x43, 0xf4, 0x44, 0xe8, 0xcc, 0xc8, 0xd4, 0x54, + 0x33, 0x37, 0x50, 0xf2, 0x87, 0x42, 0x2e, 0x00, + 0x49, 0x60, 0x62, 0x02, 0xfd, 0x1a, 0x7c, 0xdb, + 0x29, 0x6c, 0x6d, 0x54, 0x53, 0x08, 0xd1, 0xc8}}, + {{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}, + {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}}, + {{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}, + {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}}, + {{0x27, 0x59, 0xc7, 0x35, 0x60, 0x71, 0xa6, 0xf1, + 0x79, 0xa5, 0xfd, 0x79, 0x16, 0xf3, 0x41, 0xf0, + 0x57, 0xb4, 0x02, 0x97, 0x32, 0xe7, 0xde, 0x59, + 0xe2, 0x2d, 0x9b, 0x11, 0xea, 0x2c, 0x35, 0x92}, + {0x27, 0x59, 0xc7, 0x35, 0x60, 0x71, 0xa6, 0xf1, + 0x79, 0xa5, 0xfd, 0x79, 0x16, 0xf3, 0x41, 0xf0, + 0x57, 0xb4, 0x02, 0x97, 0x32, 0xe7, 0xde, 0x59, + 0xe2, 0x2d, 0x9b, 0x11, 0xea, 0x2c, 0x35, 0x92}}, + {{0x28, 0x56, 0xac, 0x0e, 0x4f, 0x98, 0x09, 0xf0, + 0x49, 0xfa, 0x7f, 0x84, 0xac, 0x7e, 0x50, 0x5b, + 0x17, 0x43, 0x14, 0x89, 0x9c, 0x53, 0xa8, 0x94, + 0x30, 0xf2, 0x11, 0x4d, 0x92, 0x14, 0x27, 0xe8}, + {0x39, 0x7a, 0x84, 0x56, 0x79, 0x9d, 0xec, 0x26, + 0x2c, 0x53, 0xc1, 0x94, 0xc9, 0x8d, 0x9e, 0x9d, + 0x32, 0x1f, 0xdd, 0x84, 0x04, 0xe8, 0xe2, 0x0a, + 0x6b, 0xbe, 0xbb, 0x42, 0x40, 0x67, 0x30, 0x6c}}, + {{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x45, 0x51, 0x23, 0x19, 0x50, 0xb7, 0x5f, 0xc4, + 0x40, 0x2d, 0xa1, 0x73, 0x2f, 0xc9, 0xbe, 0xbd}, + {0x27, 0x59, 0xc7, 0x35, 0x60, 0x71, 0xa6, 0xf1, + 0x79, 0xa5, 0xfd, 0x79, 0x16, 0xf3, 0x41, 0xf0, + 0x57, 0xb4, 0x02, 0x97, 0x32, 0xe7, 0xde, 0x59, + 0xe2, 0x2d, 0x9b, 0x11, 0xea, 0x2c, 0x35, 0x92}}, + {{0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, + 0xba, 0xae, 0xdc, 0xe6, 0xaf, 0x48, 0xa0, 0x3b, + 0xbf, 0xd2, 0x5e, 0x8c, 0xd0, 0x36, 0x41, 0x40}, + {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}}, + {{0x1c, 0xc4, 0xf7, 0xda, 0x0f, 0x65, 0xca, 0x39, + 0x70, 0x52, 0x92, 0x8e, 0xc3, 0xc8, 0x15, 0xea, + 0x7f, 0x10, 0x9e, 0x77, 0x4b, 0x6e, 0x2d, 0xdf, + 0xe8, 0x30, 0x9d, 0xda, 0xe8, 0x9a, 0x65, 0xae}, + {0x02, 0xb0, 0x16, 0xb1, 0x1d, 0xc8, 0x57, 0x7b, + 0xa2, 0x3a, 0xa2, 0xa3, 0x38, 0x5c, 0x8f, 0xeb, + 0x66, 0x37, 0x91, 0xa8, 0x5f, 0xef, 0x04, 0xf6, + 0x59, 0x75, 0xe1, 0xee, 0x92, 0xf6, 0x0e, 0x30}}, + {{0x8d, 0x76, 0x14, 0xa4, 0x14, 0x06, 0x9f, 0x9a, + 0xdf, 0x4a, 0x85, 0xa7, 0x6b, 0xbf, 0x29, 0x6f, + 0xbc, 0x34, 0x87, 0x5d, 0xeb, 0xbb, 0x2e, 0xa9, + 0xc9, 0x1f, 0x58, 0xd6, 0x9a, 0x82, 0xa0, 0x56}, + {0xd4, 0xb9, 0xdb, 0x88, 0x1d, 0x04, 0xe9, 0x93, + 0x8d, 0x3f, 0x20, 0xd5, 0x86, 0xa8, 0x83, 0x07, + 0xdb, 0x09, 0xd8, 0x22, 0x1f, 0x7f, 0xf1, 0x71, + 0xc8, 0xe7, 0x5d, 0x47, 0xaf, 0x8b, 0x72, 0xe9}}, + {{0x83, 0xb9, 0x39, 0xb2, 0xa4, 0xdf, 0x46, 0x87, + 0xc2, 0xb8, 0xf1, 0xe6, 0x4c, 0xd1, 0xe2, 0xa9, + 0xe4, 0x70, 0x30, 0x34, 0xbc, 0x52, 0x7c, 0x55, + 0xa6, 0xec, 0x80, 0xa4, 0xe5, 0xd2, 0xdc, 0x73}, + {0x08, 0xf1, 0x03, 0xcf, 0x16, 0x73, 0xe8, 0x7d, + 0xb6, 0x7e, 0x9b, 0xc0, 0xb4, 0xc2, 0xa5, 0x86, + 0x02, 0x77, 0xd5, 0x27, 0x86, 0xa5, 0x15, 0xfb, + 0xae, 0x9b, 0x8c, 0xa9, 0xf9, 0xf8, 0xa8, 0x4a}}, + {{0x8b, 0x00, 0x49, 0xdb, 0xfa, 0xf0, 0x1b, 0xa2, + 0xed, 0x8a, 0x9a, 0x7a, 0x36, 0x78, 0x4a, 0xc7, + 0xf7, 0xad, 0x39, 0xd0, 0x6c, 0x65, 0x7a, 0x41, + 0xce, 0xd6, 0xd6, 0x4c, 0x20, 0x21, 0x6b, 0xc7}, + {0xc6, 0xca, 0x78, 0x1d, 0x32, 0x6c, 0x6c, 0x06, + 0x91, 0xf2, 0x1a, 0xe8, 0x43, 0x16, 0xea, 0x04, + 0x3c, 0x1f, 0x07, 0x85, 0xf7, 0x09, 0x22, 0x08, + 0xba, 0x13, 0xfd, 0x78, 0x1e, 0x3f, 0x6f, 0x62}}, + {{0x25, 0x9b, 0x7c, 0xb0, 0xac, 0x72, 0x6f, 0xb2, + 0xe3, 0x53, 0x84, 0x7a, 0x1a, 0x9a, 0x98, 0x9b, + 0x44, 0xd3, 0x59, 0xd0, 0x8e, 0x57, 0x41, 0x40, + 0x78, 0xa7, 0x30, 0x2f, 0x4c, 0x9c, 0xb9, 0x68}, + {0xb7, 0x75, 0x03, 0x63, 0x61, 0xc2, 0x48, 0x6e, + 0x12, 0x3d, 0xbf, 0x4b, 0x27, 0xdf, 0xb1, 0x7a, + 0xff, 0x4e, 0x31, 0x07, 0x83, 0xf4, 0x62, 0x5b, + 0x19, 0xa5, 0xac, 0xa0, 0x32, 0x58, 0x0d, 0xa7}}, + {{0x43, 0x4f, 0x10, 0xa4, 0xca, 0xdb, 0x38, 0x67, + 0xfa, 0xae, 0x96, 0xb5, 0x6d, 0x97, 0xff, 0x1f, + 0xb6, 0x83, 0x43, 0xd3, 0xa0, 0x2d, 0x70, 0x7a, + 0x64, 0x05, 0x4c, 0xa7, 0xc1, 0xa5, 0x21, 0x51}, + {0xe4, 0xf1, 0x23, 0x84, 0xe1, 0xb5, 0x9d, 0xf2, + 0xb8, 0x73, 0x8b, 0x45, 0x2b, 0x35, 0x46, 0x38, + 0x10, 0x2b, 0x50, 0xf8, 0x8b, 0x35, 0xcd, 0x34, + 0xc8, 0x0e, 0xf6, 0xdb, 0x09, 0x35, 0xf0, 0xda}} + }; + secp256k1_scalar_set_int(&one, 1); + for (i = 0; i < 32; i++) { + secp256k1_scalar_set_b32(&x, chal[i][0], &overflow); + CHECK(!overflow); + secp256k1_scalar_set_b32(&y, chal[i][1], &overflow); + CHECK(!overflow); + secp256k1_scalar_set_b32(&r1, res[i][0], &overflow); + CHECK(!overflow); + secp256k1_scalar_set_b32(&r2, res[i][1], &overflow); + CHECK(!overflow); + secp256k1_scalar_mul(&z, &x, &y); + CHECK(!secp256k1_scalar_check_overflow(&z)); + CHECK(secp256k1_scalar_eq(&r1, &z)); + if (!secp256k1_scalar_is_zero(&y)) { + secp256k1_scalar_inverse(&zz, &y); + CHECK(!secp256k1_scalar_check_overflow(&zz)); +#if defined(USE_SCALAR_INV_NUM) + secp256k1_scalar_inverse_var(&zzv, &y); + CHECK(secp256k1_scalar_eq(&zzv, &zz)); +#endif + secp256k1_scalar_mul(&z, &z, &zz); + CHECK(!secp256k1_scalar_check_overflow(&z)); + CHECK(secp256k1_scalar_eq(&x, &z)); + secp256k1_scalar_mul(&zz, &zz, &y); + CHECK(!secp256k1_scalar_check_overflow(&zz)); + CHECK(secp256k1_scalar_eq(&one, &zz)); + } + secp256k1_scalar_mul(&z, &x, &x); + CHECK(!secp256k1_scalar_check_overflow(&z)); + secp256k1_scalar_sqr(&zz, &x); + CHECK(!secp256k1_scalar_check_overflow(&zz)); + CHECK(secp256k1_scalar_eq(&zz, &z)); + CHECK(secp256k1_scalar_eq(&r2, &zz)); + } + } +} + +/***** FIELD TESTS *****/ + +void random_fe(secp256k1_fe *x) { + unsigned char bin[32]; + do { + secp256k1_rand256(bin); + if (secp256k1_fe_set_b32(x, bin)) { + return; + } + } while(1); +} + +void random_fe_test(secp256k1_fe *x) { + unsigned char bin[32]; + do { + secp256k1_rand256_test(bin); + if (secp256k1_fe_set_b32(x, bin)) { + return; + } + } while(1); +} + +void random_fe_non_zero(secp256k1_fe *nz) { + int tries = 10; + while (--tries >= 0) { + random_fe(nz); + secp256k1_fe_normalize(nz); + if (!secp256k1_fe_is_zero(nz)) { + break; + } + } + /* Infinitesimal probability of spurious failure here */ + CHECK(tries >= 0); +} + +void random_fe_non_square(secp256k1_fe *ns) { + secp256k1_fe r; + random_fe_non_zero(ns); + if (secp256k1_fe_sqrt_var(&r, ns)) { + secp256k1_fe_negate(ns, ns, 1); + } +} + +int check_fe_equal(const secp256k1_fe *a, const secp256k1_fe *b) { + secp256k1_fe an = *a; + secp256k1_fe bn = *b; + secp256k1_fe_normalize_weak(&an); + secp256k1_fe_normalize_var(&bn); + return secp256k1_fe_equal_var(&an, &bn); +} + +int check_fe_inverse(const secp256k1_fe *a, const secp256k1_fe *ai) { + secp256k1_fe x; + secp256k1_fe one = SECP256K1_FE_CONST(0, 0, 0, 0, 0, 0, 0, 1); + secp256k1_fe_mul(&x, a, ai); + return check_fe_equal(&x, &one); +} + +void run_field_convert(void) { + static const unsigned char b32[32] = { + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, + 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, + 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, + 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x40 + }; + static const secp256k1_fe_storage fes = SECP256K1_FE_STORAGE_CONST( + 0x00010203UL, 0x04050607UL, 0x11121314UL, 0x15161718UL, + 0x22232425UL, 0x26272829UL, 0x33343536UL, 0x37383940UL + ); + static const secp256k1_fe fe = SECP256K1_FE_CONST( + 0x00010203UL, 0x04050607UL, 0x11121314UL, 0x15161718UL, + 0x22232425UL, 0x26272829UL, 0x33343536UL, 0x37383940UL + ); + secp256k1_fe fe2; + unsigned char b322[32]; + secp256k1_fe_storage fes2; + /* Check conversions to fe. */ + CHECK(secp256k1_fe_set_b32(&fe2, b32)); + CHECK(secp256k1_fe_equal_var(&fe, &fe2)); + secp256k1_fe_from_storage(&fe2, &fes); + CHECK(secp256k1_fe_equal_var(&fe, &fe2)); + /* Check conversion from fe. */ + secp256k1_fe_get_b32(b322, &fe); + CHECK(memcmp(b322, b32, 32) == 0); + secp256k1_fe_to_storage(&fes2, &fe); + CHECK(memcmp(&fes2, &fes, sizeof(fes)) == 0); +} + +int fe_memcmp(const secp256k1_fe *a, const secp256k1_fe *b) { + secp256k1_fe t = *b; +#ifdef VERIFY + t.magnitude = a->magnitude; + t.normalized = a->normalized; +#endif + return memcmp(a, &t, sizeof(secp256k1_fe)); +} + +void run_field_misc(void) { + secp256k1_fe x; + secp256k1_fe y; + secp256k1_fe z; + secp256k1_fe q; + secp256k1_fe fe5 = SECP256K1_FE_CONST(0, 0, 0, 0, 0, 0, 0, 5); + int i, j; + for (i = 0; i < 5*count; i++) { + secp256k1_fe_storage xs, ys, zs; + random_fe(&x); + random_fe_non_zero(&y); + /* Test the fe equality and comparison operations. */ + CHECK(secp256k1_fe_cmp_var(&x, &x) == 0); + CHECK(secp256k1_fe_equal_var(&x, &x)); + z = x; + secp256k1_fe_add(&z,&y); + /* Test fe conditional move; z is not normalized here. */ + q = x; + secp256k1_fe_cmov(&x, &z, 0); + VERIFY_CHECK(!x.normalized && x.magnitude == z.magnitude); + secp256k1_fe_cmov(&x, &x, 1); + CHECK(fe_memcmp(&x, &z) != 0); + CHECK(fe_memcmp(&x, &q) == 0); + secp256k1_fe_cmov(&q, &z, 1); + VERIFY_CHECK(!q.normalized && q.magnitude == z.magnitude); + CHECK(fe_memcmp(&q, &z) == 0); + secp256k1_fe_normalize_var(&x); + secp256k1_fe_normalize_var(&z); + CHECK(!secp256k1_fe_equal_var(&x, &z)); + secp256k1_fe_normalize_var(&q); + secp256k1_fe_cmov(&q, &z, (i&1)); + VERIFY_CHECK(q.normalized && q.magnitude == 1); + for (j = 0; j < 6; j++) { + secp256k1_fe_negate(&z, &z, j+1); + secp256k1_fe_normalize_var(&q); + secp256k1_fe_cmov(&q, &z, (j&1)); + VERIFY_CHECK(!q.normalized && q.magnitude == (j+2)); + } + secp256k1_fe_normalize_var(&z); + /* Test storage conversion and conditional moves. */ + secp256k1_fe_to_storage(&xs, &x); + secp256k1_fe_to_storage(&ys, &y); + secp256k1_fe_to_storage(&zs, &z); + secp256k1_fe_storage_cmov(&zs, &xs, 0); + secp256k1_fe_storage_cmov(&zs, &zs, 1); + CHECK(memcmp(&xs, &zs, sizeof(xs)) != 0); + secp256k1_fe_storage_cmov(&ys, &xs, 1); + CHECK(memcmp(&xs, &ys, sizeof(xs)) == 0); + secp256k1_fe_from_storage(&x, &xs); + secp256k1_fe_from_storage(&y, &ys); + secp256k1_fe_from_storage(&z, &zs); + /* Test that mul_int, mul, and add agree. */ + secp256k1_fe_add(&y, &x); + secp256k1_fe_add(&y, &x); + z = x; + secp256k1_fe_mul_int(&z, 3); + CHECK(check_fe_equal(&y, &z)); + secp256k1_fe_add(&y, &x); + secp256k1_fe_add(&z, &x); + CHECK(check_fe_equal(&z, &y)); + z = x; + secp256k1_fe_mul_int(&z, 5); + secp256k1_fe_mul(&q, &x, &fe5); + CHECK(check_fe_equal(&z, &q)); + secp256k1_fe_negate(&x, &x, 1); + secp256k1_fe_add(&z, &x); + secp256k1_fe_add(&q, &x); + CHECK(check_fe_equal(&y, &z)); + CHECK(check_fe_equal(&q, &y)); + } +} + +void run_field_inv(void) { + secp256k1_fe x, xi, xii; + int i; + for (i = 0; i < 10*count; i++) { + random_fe_non_zero(&x); + secp256k1_fe_inv(&xi, &x); + CHECK(check_fe_inverse(&x, &xi)); + secp256k1_fe_inv(&xii, &xi); + CHECK(check_fe_equal(&x, &xii)); + } +} + +void run_field_inv_var(void) { + secp256k1_fe x, xi, xii; + int i; + for (i = 0; i < 10*count; i++) { + random_fe_non_zero(&x); + secp256k1_fe_inv_var(&xi, &x); + CHECK(check_fe_inverse(&x, &xi)); + secp256k1_fe_inv_var(&xii, &xi); + CHECK(check_fe_equal(&x, &xii)); + } +} + +void run_field_inv_all_var(void) { + secp256k1_fe x[16], xi[16], xii[16]; + int i; + /* Check it's safe to call for 0 elements */ + secp256k1_fe_inv_all_var(0, xi, x); + for (i = 0; i < count; i++) { + size_t j; + size_t len = secp256k1_rand_int(15) + 1; + for (j = 0; j < len; j++) { + random_fe_non_zero(&x[j]); + } + secp256k1_fe_inv_all_var(len, xi, x); + for (j = 0; j < len; j++) { + CHECK(check_fe_inverse(&x[j], &xi[j])); + } + secp256k1_fe_inv_all_var(len, xii, xi); + for (j = 0; j < len; j++) { + CHECK(check_fe_equal(&x[j], &xii[j])); + } + } +} + +void run_sqr(void) { + secp256k1_fe x, s; + + { + int i; + secp256k1_fe_set_int(&x, 1); + secp256k1_fe_negate(&x, &x, 1); + + for (i = 1; i <= 512; ++i) { + secp256k1_fe_mul_int(&x, 2); + secp256k1_fe_normalize(&x); + secp256k1_fe_sqr(&s, &x); + } + } +} + +void test_sqrt(const secp256k1_fe *a, const secp256k1_fe *k) { + secp256k1_fe r1, r2; + int v = secp256k1_fe_sqrt_var(&r1, a); + CHECK((v == 0) == (k == NULL)); + + if (k != NULL) { + /* Check that the returned root is +/- the given known answer */ + secp256k1_fe_negate(&r2, &r1, 1); + secp256k1_fe_add(&r1, k); secp256k1_fe_add(&r2, k); + secp256k1_fe_normalize(&r1); secp256k1_fe_normalize(&r2); + CHECK(secp256k1_fe_is_zero(&r1) || secp256k1_fe_is_zero(&r2)); + } +} + +void run_sqrt(void) { + secp256k1_fe ns, x, s, t; + int i; + + /* Check sqrt(0) is 0 */ + secp256k1_fe_set_int(&x, 0); + secp256k1_fe_sqr(&s, &x); + test_sqrt(&s, &x); + + /* Check sqrt of small squares (and their negatives) */ + for (i = 1; i <= 100; i++) { + secp256k1_fe_set_int(&x, i); + secp256k1_fe_sqr(&s, &x); + test_sqrt(&s, &x); + secp256k1_fe_negate(&t, &s, 1); + test_sqrt(&t, NULL); + } + + /* Consistency checks for large random values */ + for (i = 0; i < 10; i++) { + int j; + random_fe_non_square(&ns); + for (j = 0; j < count; j++) { + random_fe(&x); + secp256k1_fe_sqr(&s, &x); + test_sqrt(&s, &x); + secp256k1_fe_negate(&t, &s, 1); + test_sqrt(&t, NULL); + secp256k1_fe_mul(&t, &s, &ns); + test_sqrt(&t, NULL); + } + } +} + +/***** GROUP TESTS *****/ + +void ge_equals_ge(const secp256k1_ge *a, const secp256k1_ge *b) { + CHECK(a->infinity == b->infinity); + if (a->infinity) { + return; + } + CHECK(secp256k1_fe_equal_var(&a->x, &b->x)); + CHECK(secp256k1_fe_equal_var(&a->y, &b->y)); +} + +/* This compares jacobian points including their Z, not just their geometric meaning. */ +int gej_xyz_equals_gej(const secp256k1_gej *a, const secp256k1_gej *b) { + secp256k1_gej a2; + secp256k1_gej b2; + int ret = 1; + ret &= a->infinity == b->infinity; + if (ret && !a->infinity) { + a2 = *a; + b2 = *b; + secp256k1_fe_normalize(&a2.x); + secp256k1_fe_normalize(&a2.y); + secp256k1_fe_normalize(&a2.z); + secp256k1_fe_normalize(&b2.x); + secp256k1_fe_normalize(&b2.y); + secp256k1_fe_normalize(&b2.z); + ret &= secp256k1_fe_cmp_var(&a2.x, &b2.x) == 0; + ret &= secp256k1_fe_cmp_var(&a2.y, &b2.y) == 0; + ret &= secp256k1_fe_cmp_var(&a2.z, &b2.z) == 0; + } + return ret; +} + +void ge_equals_gej(const secp256k1_ge *a, const secp256k1_gej *b) { + secp256k1_fe z2s; + secp256k1_fe u1, u2, s1, s2; + CHECK(a->infinity == b->infinity); + if (a->infinity) { + return; + } + /* Check a.x * b.z^2 == b.x && a.y * b.z^3 == b.y, to avoid inverses. */ + secp256k1_fe_sqr(&z2s, &b->z); + secp256k1_fe_mul(&u1, &a->x, &z2s); + u2 = b->x; secp256k1_fe_normalize_weak(&u2); + secp256k1_fe_mul(&s1, &a->y, &z2s); secp256k1_fe_mul(&s1, &s1, &b->z); + s2 = b->y; secp256k1_fe_normalize_weak(&s2); + CHECK(secp256k1_fe_equal_var(&u1, &u2)); + CHECK(secp256k1_fe_equal_var(&s1, &s2)); +} + +void test_ge(void) { + int i, i1; +#ifdef USE_ENDOMORPHISM + int runs = 6; +#else + int runs = 4; +#endif + /* Points: (infinity, p1, p1, -p1, -p1, p2, p2, -p2, -p2, p3, p3, -p3, -p3, p4, p4, -p4, -p4). + * The second in each pair of identical points uses a random Z coordinate in the Jacobian form. + * All magnitudes are randomized. + * All 17*17 combinations of points are added to each other, using all applicable methods. + * + * When the endomorphism code is compiled in, p5 = lambda*p1 and p6 = lambda^2*p1 are added as well. + */ + secp256k1_ge *ge = (secp256k1_ge *)malloc(sizeof(secp256k1_ge) * (1 + 4 * runs)); + secp256k1_gej *gej = (secp256k1_gej *)malloc(sizeof(secp256k1_gej) * (1 + 4 * runs)); + secp256k1_fe *zinv = (secp256k1_fe *)malloc(sizeof(secp256k1_fe) * (1 + 4 * runs)); + secp256k1_fe zf; + secp256k1_fe zfi2, zfi3; + + secp256k1_gej_set_infinity(&gej[0]); + secp256k1_ge_clear(&ge[0]); + secp256k1_ge_set_gej_var(&ge[0], &gej[0]); + for (i = 0; i < runs; i++) { + int j; + secp256k1_ge g; + random_group_element_test(&g); +#ifdef USE_ENDOMORPHISM + if (i >= runs - 2) { + secp256k1_ge_mul_lambda(&g, &ge[1]); + } + if (i >= runs - 1) { + secp256k1_ge_mul_lambda(&g, &g); + } +#endif + ge[1 + 4 * i] = g; + ge[2 + 4 * i] = g; + secp256k1_ge_neg(&ge[3 + 4 * i], &g); + secp256k1_ge_neg(&ge[4 + 4 * i], &g); + secp256k1_gej_set_ge(&gej[1 + 4 * i], &ge[1 + 4 * i]); + random_group_element_jacobian_test(&gej[2 + 4 * i], &ge[2 + 4 * i]); + secp256k1_gej_set_ge(&gej[3 + 4 * i], &ge[3 + 4 * i]); + random_group_element_jacobian_test(&gej[4 + 4 * i], &ge[4 + 4 * i]); + for (j = 0; j < 4; j++) { + random_field_element_magnitude(&ge[1 + j + 4 * i].x); + random_field_element_magnitude(&ge[1 + j + 4 * i].y); + random_field_element_magnitude(&gej[1 + j + 4 * i].x); + random_field_element_magnitude(&gej[1 + j + 4 * i].y); + random_field_element_magnitude(&gej[1 + j + 4 * i].z); + } + } + + /* Compute z inverses. */ + { + secp256k1_fe *zs = malloc(sizeof(secp256k1_fe) * (1 + 4 * runs)); + for (i = 0; i < 4 * runs + 1; i++) { + if (i == 0) { + /* The point at infinity does not have a meaningful z inverse. Any should do. */ + do { + random_field_element_test(&zs[i]); + } while(secp256k1_fe_is_zero(&zs[i])); + } else { + zs[i] = gej[i].z; + } + } + secp256k1_fe_inv_all_var(4 * runs + 1, zinv, zs); + free(zs); + } + + /* Generate random zf, and zfi2 = 1/zf^2, zfi3 = 1/zf^3 */ + do { + random_field_element_test(&zf); + } while(secp256k1_fe_is_zero(&zf)); + random_field_element_magnitude(&zf); + secp256k1_fe_inv_var(&zfi3, &zf); + secp256k1_fe_sqr(&zfi2, &zfi3); + secp256k1_fe_mul(&zfi3, &zfi3, &zfi2); + + for (i1 = 0; i1 < 1 + 4 * runs; i1++) { + int i2; + for (i2 = 0; i2 < 1 + 4 * runs; i2++) { + /* Compute reference result using gej + gej (var). */ + secp256k1_gej refj, resj; + secp256k1_ge ref; + secp256k1_fe zr; + secp256k1_gej_add_var(&refj, &gej[i1], &gej[i2], secp256k1_gej_is_infinity(&gej[i1]) ? NULL : &zr); + /* Check Z ratio. */ + if (!secp256k1_gej_is_infinity(&gej[i1]) && !secp256k1_gej_is_infinity(&refj)) { + secp256k1_fe zrz; secp256k1_fe_mul(&zrz, &zr, &gej[i1].z); + CHECK(secp256k1_fe_equal_var(&zrz, &refj.z)); + } + secp256k1_ge_set_gej_var(&ref, &refj); + + /* Test gej + ge with Z ratio result (var). */ + secp256k1_gej_add_ge_var(&resj, &gej[i1], &ge[i2], secp256k1_gej_is_infinity(&gej[i1]) ? NULL : &zr); + ge_equals_gej(&ref, &resj); + if (!secp256k1_gej_is_infinity(&gej[i1]) && !secp256k1_gej_is_infinity(&resj)) { + secp256k1_fe zrz; secp256k1_fe_mul(&zrz, &zr, &gej[i1].z); + CHECK(secp256k1_fe_equal_var(&zrz, &resj.z)); + } + + /* Test gej + ge (var, with additional Z factor). */ + { + secp256k1_ge ge2_zfi = ge[i2]; /* the second term with x and y rescaled for z = 1/zf */ + secp256k1_fe_mul(&ge2_zfi.x, &ge2_zfi.x, &zfi2); + secp256k1_fe_mul(&ge2_zfi.y, &ge2_zfi.y, &zfi3); + random_field_element_magnitude(&ge2_zfi.x); + random_field_element_magnitude(&ge2_zfi.y); + secp256k1_gej_add_zinv_var(&resj, &gej[i1], &ge2_zfi, &zf); + ge_equals_gej(&ref, &resj); + } + + /* Test gej + ge (const). */ + if (i2 != 0) { + /* secp256k1_gej_add_ge does not support its second argument being infinity. */ + secp256k1_gej_add_ge(&resj, &gej[i1], &ge[i2]); + ge_equals_gej(&ref, &resj); + } + + /* Test doubling (var). */ + if ((i1 == 0 && i2 == 0) || ((i1 + 3)/4 == (i2 + 3)/4 && ((i1 + 3)%4)/2 == ((i2 + 3)%4)/2)) { + secp256k1_fe zr2; + /* Normal doubling with Z ratio result. */ + secp256k1_gej_double_var(&resj, &gej[i1], &zr2); + ge_equals_gej(&ref, &resj); + /* Check Z ratio. */ + secp256k1_fe_mul(&zr2, &zr2, &gej[i1].z); + CHECK(secp256k1_fe_equal_var(&zr2, &resj.z)); + /* Normal doubling. */ + secp256k1_gej_double_var(&resj, &gej[i2], NULL); + ge_equals_gej(&ref, &resj); + } + + /* Test adding opposites. */ + if ((i1 == 0 && i2 == 0) || ((i1 + 3)/4 == (i2 + 3)/4 && ((i1 + 3)%4)/2 != ((i2 + 3)%4)/2)) { + CHECK(secp256k1_ge_is_infinity(&ref)); + } + + /* Test adding infinity. */ + if (i1 == 0) { + CHECK(secp256k1_ge_is_infinity(&ge[i1])); + CHECK(secp256k1_gej_is_infinity(&gej[i1])); + ge_equals_gej(&ref, &gej[i2]); + } + if (i2 == 0) { + CHECK(secp256k1_ge_is_infinity(&ge[i2])); + CHECK(secp256k1_gej_is_infinity(&gej[i2])); + ge_equals_gej(&ref, &gej[i1]); + } + } + } + + /* Test adding all points together in random order equals infinity. */ + { + secp256k1_gej sum = SECP256K1_GEJ_CONST_INFINITY; + secp256k1_gej *gej_shuffled = (secp256k1_gej *)malloc((4 * runs + 1) * sizeof(secp256k1_gej)); + for (i = 0; i < 4 * runs + 1; i++) { + gej_shuffled[i] = gej[i]; + } + for (i = 0; i < 4 * runs + 1; i++) { + int swap = i + secp256k1_rand_int(4 * runs + 1 - i); + if (swap != i) { + secp256k1_gej t = gej_shuffled[i]; + gej_shuffled[i] = gej_shuffled[swap]; + gej_shuffled[swap] = t; + } + } + for (i = 0; i < 4 * runs + 1; i++) { + secp256k1_gej_add_var(&sum, &sum, &gej_shuffled[i], NULL); + } + CHECK(secp256k1_gej_is_infinity(&sum)); + free(gej_shuffled); + } + + /* Test batch gej -> ge conversion with and without known z ratios. */ + { + secp256k1_fe *zr = (secp256k1_fe *)malloc((4 * runs + 1) * sizeof(secp256k1_fe)); + secp256k1_ge *ge_set_table = (secp256k1_ge *)malloc((4 * runs + 1) * sizeof(secp256k1_ge)); + secp256k1_ge *ge_set_all = (secp256k1_ge *)malloc((4 * runs + 1) * sizeof(secp256k1_ge)); + for (i = 0; i < 4 * runs + 1; i++) { + /* Compute gej[i + 1].z / gez[i].z (with gej[n].z taken to be 1). */ + if (i < 4 * runs) { + secp256k1_fe_mul(&zr[i + 1], &zinv[i], &gej[i + 1].z); + } + } + secp256k1_ge_set_table_gej_var(4 * runs + 1, ge_set_table, gej, zr); + secp256k1_ge_set_all_gej_var(4 * runs + 1, ge_set_all, gej, &ctx->error_callback); + for (i = 0; i < 4 * runs + 1; i++) { + secp256k1_fe s; + random_fe_non_zero(&s); + secp256k1_gej_rescale(&gej[i], &s); + ge_equals_gej(&ge_set_table[i], &gej[i]); + ge_equals_gej(&ge_set_all[i], &gej[i]); + } + free(ge_set_table); + free(ge_set_all); + free(zr); + } + + free(ge); + free(gej); + free(zinv); +} + +void test_add_neg_y_diff_x(void) { + /* The point of this test is to check that we can add two points + * whose y-coordinates are negatives of each other but whose x + * coordinates differ. If the x-coordinates were the same, these + * points would be negatives of each other and their sum is + * infinity. This is cool because it "covers up" any degeneracy + * in the addition algorithm that would cause the xy coordinates + * of the sum to be wrong (since infinity has no xy coordinates). + * HOWEVER, if the x-coordinates are different, infinity is the + * wrong answer, and such degeneracies are exposed. This is the + * root of https://github.com/bitcoin/secp256k1/issues/257 which + * this test is a regression test for. + * + * These points were generated in sage as + * # secp256k1 params + * F = FiniteField (0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F) + * C = EllipticCurve ([F (0), F (7)]) + * G = C.lift_x(0x79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798) + * N = FiniteField(G.order()) + * + * # endomorphism values (lambda is 1^{1/3} in N, beta is 1^{1/3} in F) + * x = polygen(N) + * lam = (1 - x^3).roots()[1][0] + * + * # random "bad pair" + * P = C.random_element() + * Q = -int(lam) * P + * print " P: %x %x" % P.xy() + * print " Q: %x %x" % Q.xy() + * print "P + Q: %x %x" % (P + Q).xy() + */ + secp256k1_gej aj = SECP256K1_GEJ_CONST( + 0x8d24cd95, 0x0a355af1, 0x3c543505, 0x44238d30, + 0x0643d79f, 0x05a59614, 0x2f8ec030, 0xd58977cb, + 0x001e337a, 0x38093dcd, 0x6c0f386d, 0x0b1293a8, + 0x4d72c879, 0xd7681924, 0x44e6d2f3, 0x9190117d + ); + secp256k1_gej bj = SECP256K1_GEJ_CONST( + 0xc7b74206, 0x1f788cd9, 0xabd0937d, 0x164a0d86, + 0x95f6ff75, 0xf19a4ce9, 0xd013bd7b, 0xbf92d2a7, + 0xffe1cc85, 0xc7f6c232, 0x93f0c792, 0xf4ed6c57, + 0xb28d3786, 0x2897e6db, 0xbb192d0b, 0x6e6feab2 + ); + secp256k1_gej sumj = SECP256K1_GEJ_CONST( + 0x671a63c0, 0x3efdad4c, 0x389a7798, 0x24356027, + 0xb3d69010, 0x278625c3, 0x5c86d390, 0x184a8f7a, + 0x5f6409c2, 0x2ce01f2b, 0x511fd375, 0x25071d08, + 0xda651801, 0x70e95caf, 0x8f0d893c, 0xbed8fbbe + ); + secp256k1_ge b; + secp256k1_gej resj; + secp256k1_ge res; + secp256k1_ge_set_gej(&b, &bj); + + secp256k1_gej_add_var(&resj, &aj, &bj, NULL); + secp256k1_ge_set_gej(&res, &resj); + ge_equals_gej(&res, &sumj); + + secp256k1_gej_add_ge(&resj, &aj, &b); + secp256k1_ge_set_gej(&res, &resj); + ge_equals_gej(&res, &sumj); + + secp256k1_gej_add_ge_var(&resj, &aj, &b, NULL); + secp256k1_ge_set_gej(&res, &resj); + ge_equals_gej(&res, &sumj); +} + +void run_ge(void) { + int i; + for (i = 0; i < count * 32; i++) { + test_ge(); + } + test_add_neg_y_diff_x(); +} + +void test_ec_combine(void) { + secp256k1_scalar sum = SECP256K1_SCALAR_CONST(0, 0, 0, 0, 0, 0, 0, 0); + secp256k1_pubkey data[6]; + const secp256k1_pubkey* d[6]; + secp256k1_pubkey sd; + secp256k1_pubkey sd2; + secp256k1_gej Qj; + secp256k1_ge Q; + int i; + for (i = 1; i <= 6; i++) { + secp256k1_scalar s; + random_scalar_order_test(&s); + secp256k1_scalar_add(&sum, &sum, &s); + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &Qj, &s); + secp256k1_ge_set_gej(&Q, &Qj); + secp256k1_pubkey_save(&data[i - 1], &Q); + d[i - 1] = &data[i - 1]; + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &Qj, &sum); + secp256k1_ge_set_gej(&Q, &Qj); + secp256k1_pubkey_save(&sd, &Q); + CHECK(secp256k1_ec_pubkey_combine(ctx, &sd2, d, i) == 1); + CHECK(memcmp(&sd, &sd2, sizeof(sd)) == 0); + } +} + +void run_ec_combine(void) { + int i; + for (i = 0; i < count * 8; i++) { + test_ec_combine(); + } +} + +void test_group_decompress(const secp256k1_fe* x) { + /* The input itself, normalized. */ + secp256k1_fe fex = *x; + secp256k1_fe tmp; + /* Results of set_xquad_var, set_xo_var(..., 0), set_xo_var(..., 1). */ + secp256k1_ge ge_quad, ge_even, ge_odd; + /* Return values of the above calls. */ + int res_quad, res_even, res_odd; + + secp256k1_fe_normalize_var(&fex); + + res_quad = secp256k1_ge_set_xquad_var(&ge_quad, &fex); + res_even = secp256k1_ge_set_xo_var(&ge_even, &fex, 0); + res_odd = secp256k1_ge_set_xo_var(&ge_odd, &fex, 1); + + CHECK(res_quad == res_even); + CHECK(res_quad == res_odd); + + if (res_quad) { + secp256k1_fe_normalize_var(&ge_quad.x); + secp256k1_fe_normalize_var(&ge_odd.x); + secp256k1_fe_normalize_var(&ge_even.x); + secp256k1_fe_normalize_var(&ge_quad.y); + secp256k1_fe_normalize_var(&ge_odd.y); + secp256k1_fe_normalize_var(&ge_even.y); + + /* No infinity allowed. */ + CHECK(!ge_quad.infinity); + CHECK(!ge_even.infinity); + CHECK(!ge_odd.infinity); + + /* Check that the x coordinates check out. */ + CHECK(secp256k1_fe_equal_var(&ge_quad.x, x)); + CHECK(secp256k1_fe_equal_var(&ge_even.x, x)); + CHECK(secp256k1_fe_equal_var(&ge_odd.x, x)); + + /* Check that the Y coordinate result in ge_quad is a square. */ + CHECK(secp256k1_fe_sqrt_var(&tmp, &ge_quad.y)); + secp256k1_fe_sqr(&tmp, &tmp); + CHECK(secp256k1_fe_equal_var(&tmp, &ge_quad.y)); + + /* Check odd/even Y in ge_odd, ge_even. */ + CHECK(secp256k1_fe_is_odd(&ge_odd.y)); + CHECK(!secp256k1_fe_is_odd(&ge_even.y)); + } +} + +void run_group_decompress(void) { + int i; + for (i = 0; i < count * 4; i++) { + secp256k1_fe fe; + random_fe_test(&fe); + test_group_decompress(&fe); + } +} + +/***** ECMULT TESTS *****/ + +void run_ecmult_chain(void) { + /* random starting point A (on the curve) */ + secp256k1_gej a = SECP256K1_GEJ_CONST( + 0x8b30bbe9, 0xae2a9906, 0x96b22f67, 0x0709dff3, + 0x727fd8bc, 0x04d3362c, 0x6c7bf458, 0xe2846004, + 0xa357ae91, 0x5c4a6528, 0x1309edf2, 0x0504740f, + 0x0eb33439, 0x90216b4f, 0x81063cb6, 0x5f2f7e0f + ); + /* two random initial factors xn and gn */ + secp256k1_scalar xn = SECP256K1_SCALAR_CONST( + 0x84cc5452, 0xf7fde1ed, 0xb4d38a8c, 0xe9b1b84c, + 0xcef31f14, 0x6e569be9, 0x705d357a, 0x42985407 + ); + secp256k1_scalar gn = SECP256K1_SCALAR_CONST( + 0xa1e58d22, 0x553dcd42, 0xb2398062, 0x5d4c57a9, + 0x6e9323d4, 0x2b3152e5, 0xca2c3990, 0xedc7c9de + ); + /* two small multipliers to be applied to xn and gn in every iteration: */ + static const secp256k1_scalar xf = SECP256K1_SCALAR_CONST(0, 0, 0, 0, 0, 0, 0, 0x1337); + static const secp256k1_scalar gf = SECP256K1_SCALAR_CONST(0, 0, 0, 0, 0, 0, 0, 0x7113); + /* accumulators with the resulting coefficients to A and G */ + secp256k1_scalar ae = SECP256K1_SCALAR_CONST(0, 0, 0, 0, 0, 0, 0, 1); + secp256k1_scalar ge = SECP256K1_SCALAR_CONST(0, 0, 0, 0, 0, 0, 0, 0); + /* actual points */ + secp256k1_gej x; + secp256k1_gej x2; + int i; + + /* the point being computed */ + x = a; + for (i = 0; i < 200*count; i++) { + /* in each iteration, compute X = xn*X + gn*G; */ + secp256k1_ecmult(&ctx->ecmult_ctx, &x, &x, &xn, &gn); + /* also compute ae and ge: the actual accumulated factors for A and G */ + /* if X was (ae*A+ge*G), xn*X + gn*G results in (xn*ae*A + (xn*ge+gn)*G) */ + secp256k1_scalar_mul(&ae, &ae, &xn); + secp256k1_scalar_mul(&ge, &ge, &xn); + secp256k1_scalar_add(&ge, &ge, &gn); + /* modify xn and gn */ + secp256k1_scalar_mul(&xn, &xn, &xf); + secp256k1_scalar_mul(&gn, &gn, &gf); + + /* verify */ + if (i == 19999) { + /* expected result after 19999 iterations */ + secp256k1_gej rp = SECP256K1_GEJ_CONST( + 0xD6E96687, 0xF9B10D09, 0x2A6F3543, 0x9D86CEBE, + 0xA4535D0D, 0x409F5358, 0x6440BD74, 0xB933E830, + 0xB95CBCA2, 0xC77DA786, 0x539BE8FD, 0x53354D2D, + 0x3B4F566A, 0xE6580454, 0x07ED6015, 0xEE1B2A88 + ); + + secp256k1_gej_neg(&rp, &rp); + secp256k1_gej_add_var(&rp, &rp, &x, NULL); + CHECK(secp256k1_gej_is_infinity(&rp)); + } + } + /* redo the computation, but directly with the resulting ae and ge coefficients: */ + secp256k1_ecmult(&ctx->ecmult_ctx, &x2, &a, &ae, &ge); + secp256k1_gej_neg(&x2, &x2); + secp256k1_gej_add_var(&x2, &x2, &x, NULL); + CHECK(secp256k1_gej_is_infinity(&x2)); +} + +void test_point_times_order(const secp256k1_gej *point) { + /* X * (point + G) + (order-X) * (pointer + G) = 0 */ + secp256k1_scalar x; + secp256k1_scalar nx; + secp256k1_scalar zero = SECP256K1_SCALAR_CONST(0, 0, 0, 0, 0, 0, 0, 0); + secp256k1_scalar one = SECP256K1_SCALAR_CONST(0, 0, 0, 0, 0, 0, 0, 1); + secp256k1_gej res1, res2; + secp256k1_ge res3; + unsigned char pub[65]; + size_t psize = 65; + random_scalar_order_test(&x); + secp256k1_scalar_negate(&nx, &x); + secp256k1_ecmult(&ctx->ecmult_ctx, &res1, point, &x, &x); /* calc res1 = x * point + x * G; */ + secp256k1_ecmult(&ctx->ecmult_ctx, &res2, point, &nx, &nx); /* calc res2 = (order - x) * point + (order - x) * G; */ + secp256k1_gej_add_var(&res1, &res1, &res2, NULL); + CHECK(secp256k1_gej_is_infinity(&res1)); + CHECK(secp256k1_gej_is_valid_var(&res1) == 0); + secp256k1_ge_set_gej(&res3, &res1); + CHECK(secp256k1_ge_is_infinity(&res3)); + CHECK(secp256k1_ge_is_valid_var(&res3) == 0); + CHECK(secp256k1_eckey_pubkey_serialize(&res3, pub, &psize, 0) == 0); + psize = 65; + CHECK(secp256k1_eckey_pubkey_serialize(&res3, pub, &psize, 1) == 0); + /* check zero/one edge cases */ + secp256k1_ecmult(&ctx->ecmult_ctx, &res1, point, &zero, &zero); + secp256k1_ge_set_gej(&res3, &res1); + CHECK(secp256k1_ge_is_infinity(&res3)); + secp256k1_ecmult(&ctx->ecmult_ctx, &res1, point, &one, &zero); + secp256k1_ge_set_gej(&res3, &res1); + ge_equals_gej(&res3, point); + secp256k1_ecmult(&ctx->ecmult_ctx, &res1, point, &zero, &one); + secp256k1_ge_set_gej(&res3, &res1); + ge_equals_ge(&res3, &secp256k1_ge_const_g); +} + +void run_point_times_order(void) { + int i; + secp256k1_fe x = SECP256K1_FE_CONST(0, 0, 0, 0, 0, 0, 0, 2); + static const secp256k1_fe xr = SECP256K1_FE_CONST( + 0x7603CB59, 0xB0EF6C63, 0xFE608479, 0x2A0C378C, + 0xDB3233A8, 0x0F8A9A09, 0xA877DEAD, 0x31B38C45 + ); + for (i = 0; i < 500; i++) { + secp256k1_ge p; + if (secp256k1_ge_set_xo_var(&p, &x, 1)) { + secp256k1_gej j; + CHECK(secp256k1_ge_is_valid_var(&p)); + secp256k1_gej_set_ge(&j, &p); + CHECK(secp256k1_gej_is_valid_var(&j)); + test_point_times_order(&j); + } + secp256k1_fe_sqr(&x, &x); + } + secp256k1_fe_normalize_var(&x); + CHECK(secp256k1_fe_equal_var(&x, &xr)); +} + +void ecmult_const_random_mult(void) { + /* random starting point A (on the curve) */ + secp256k1_ge a = SECP256K1_GE_CONST( + 0x6d986544, 0x57ff52b8, 0xcf1b8126, 0x5b802a5b, + 0xa97f9263, 0xb1e88044, 0x93351325, 0x91bc450a, + 0x535c59f7, 0x325e5d2b, 0xc391fbe8, 0x3c12787c, + 0x337e4a98, 0xe82a9011, 0x0123ba37, 0xdd769c7d + ); + /* random initial factor xn */ + secp256k1_scalar xn = SECP256K1_SCALAR_CONST( + 0x649d4f77, 0xc4242df7, 0x7f2079c9, 0x14530327, + 0xa31b876a, 0xd2d8ce2a, 0x2236d5c6, 0xd7b2029b + ); + /* expected xn * A (from sage) */ + secp256k1_ge expected_b = SECP256K1_GE_CONST( + 0x23773684, 0x4d209dc7, 0x098a786f, 0x20d06fcd, + 0x070a38bf, 0xc11ac651, 0x03004319, 0x1e2a8786, + 0xed8c3b8e, 0xc06dd57b, 0xd06ea66e, 0x45492b0f, + 0xb84e4e1b, 0xfb77e21f, 0x96baae2a, 0x63dec956 + ); + secp256k1_gej b; + secp256k1_ecmult_const(&b, &a, &xn); + + CHECK(secp256k1_ge_is_valid_var(&a)); + ge_equals_gej(&expected_b, &b); +} + +void ecmult_const_commutativity(void) { + secp256k1_scalar a; + secp256k1_scalar b; + secp256k1_gej res1; + secp256k1_gej res2; + secp256k1_ge mid1; + secp256k1_ge mid2; + random_scalar_order_test(&a); + random_scalar_order_test(&b); + + secp256k1_ecmult_const(&res1, &secp256k1_ge_const_g, &a); + secp256k1_ecmult_const(&res2, &secp256k1_ge_const_g, &b); + secp256k1_ge_set_gej(&mid1, &res1); + secp256k1_ge_set_gej(&mid2, &res2); + secp256k1_ecmult_const(&res1, &mid1, &b); + secp256k1_ecmult_const(&res2, &mid2, &a); + secp256k1_ge_set_gej(&mid1, &res1); + secp256k1_ge_set_gej(&mid2, &res2); + ge_equals_ge(&mid1, &mid2); +} + +void ecmult_const_mult_zero_one(void) { + secp256k1_scalar zero = SECP256K1_SCALAR_CONST(0, 0, 0, 0, 0, 0, 0, 0); + secp256k1_scalar one = SECP256K1_SCALAR_CONST(0, 0, 0, 0, 0, 0, 0, 1); + secp256k1_scalar negone; + secp256k1_gej res1; + secp256k1_ge res2; + secp256k1_ge point; + secp256k1_scalar_negate(&negone, &one); + + random_group_element_test(&point); + secp256k1_ecmult_const(&res1, &point, &zero); + secp256k1_ge_set_gej(&res2, &res1); + CHECK(secp256k1_ge_is_infinity(&res2)); + secp256k1_ecmult_const(&res1, &point, &one); + secp256k1_ge_set_gej(&res2, &res1); + ge_equals_ge(&res2, &point); + secp256k1_ecmult_const(&res1, &point, &negone); + secp256k1_gej_neg(&res1, &res1); + secp256k1_ge_set_gej(&res2, &res1); + ge_equals_ge(&res2, &point); +} + +void ecmult_const_chain_multiply(void) { + /* Check known result (randomly generated test problem from sage) */ + const secp256k1_scalar scalar = SECP256K1_SCALAR_CONST( + 0x4968d524, 0x2abf9b7a, 0x466abbcf, 0x34b11b6d, + 0xcd83d307, 0x827bed62, 0x05fad0ce, 0x18fae63b + ); + const secp256k1_gej expected_point = SECP256K1_GEJ_CONST( + 0x5494c15d, 0x32099706, 0xc2395f94, 0x348745fd, + 0x757ce30e, 0x4e8c90fb, 0xa2bad184, 0xf883c69f, + 0x5d195d20, 0xe191bf7f, 0x1be3e55f, 0x56a80196, + 0x6071ad01, 0xf1462f66, 0xc997fa94, 0xdb858435 + ); + secp256k1_gej point; + secp256k1_ge res; + int i; + + secp256k1_gej_set_ge(&point, &secp256k1_ge_const_g); + for (i = 0; i < 100; ++i) { + secp256k1_ge tmp; + secp256k1_ge_set_gej(&tmp, &point); + secp256k1_ecmult_const(&point, &tmp, &scalar); + } + secp256k1_ge_set_gej(&res, &point); + ge_equals_gej(&res, &expected_point); +} + +void run_ecmult_const_tests(void) { + ecmult_const_mult_zero_one(); + ecmult_const_random_mult(); + ecmult_const_commutativity(); + ecmult_const_chain_multiply(); +} + +void test_wnaf(const secp256k1_scalar *number, int w) { + secp256k1_scalar x, two, t; + int wnaf[256]; + int zeroes = -1; + int i; + int bits; + secp256k1_scalar_set_int(&x, 0); + secp256k1_scalar_set_int(&two, 2); + bits = secp256k1_ecmult_wnaf(wnaf, 256, number, w); + CHECK(bits <= 256); + for (i = bits-1; i >= 0; i--) { + int v = wnaf[i]; + secp256k1_scalar_mul(&x, &x, &two); + if (v) { + CHECK(zeroes == -1 || zeroes >= w-1); /* check that distance between non-zero elements is at least w-1 */ + zeroes=0; + CHECK((v & 1) == 1); /* check non-zero elements are odd */ + CHECK(v <= (1 << (w-1)) - 1); /* check range below */ + CHECK(v >= -(1 << (w-1)) - 1); /* check range above */ + } else { + CHECK(zeroes != -1); /* check that no unnecessary zero padding exists */ + zeroes++; + } + if (v >= 0) { + secp256k1_scalar_set_int(&t, v); + } else { + secp256k1_scalar_set_int(&t, -v); + secp256k1_scalar_negate(&t, &t); + } + secp256k1_scalar_add(&x, &x, &t); + } + CHECK(secp256k1_scalar_eq(&x, number)); /* check that wnaf represents number */ +} + +void test_constant_wnaf_negate(const secp256k1_scalar *number) { + secp256k1_scalar neg1 = *number; + secp256k1_scalar neg2 = *number; + int sign1 = 1; + int sign2 = 1; + + if (!secp256k1_scalar_get_bits(&neg1, 0, 1)) { + secp256k1_scalar_negate(&neg1, &neg1); + sign1 = -1; + } + sign2 = secp256k1_scalar_cond_negate(&neg2, secp256k1_scalar_is_even(&neg2)); + CHECK(sign1 == sign2); + CHECK(secp256k1_scalar_eq(&neg1, &neg2)); +} + +void test_constant_wnaf(const secp256k1_scalar *number, int w) { + secp256k1_scalar x, shift; + int wnaf[256] = {0}; + int i; +#ifdef USE_ENDOMORPHISM + int skew; +#endif + secp256k1_scalar num = *number; + + secp256k1_scalar_set_int(&x, 0); + secp256k1_scalar_set_int(&shift, 1 << w); + /* With USE_ENDOMORPHISM on we only consider 128-bit numbers */ +#ifdef USE_ENDOMORPHISM + for (i = 0; i < 16; ++i) { + secp256k1_scalar_shr_int(&num, 8); + } + skew = secp256k1_wnaf_const(wnaf, num, w); +#else + secp256k1_wnaf_const(wnaf, num, w); +#endif + + for (i = WNAF_SIZE(w); i >= 0; --i) { + secp256k1_scalar t; + int v = wnaf[i]; + CHECK(v != 0); /* check nonzero */ + CHECK(v & 1); /* check parity */ + CHECK(v > -(1 << w)); /* check range above */ + CHECK(v < (1 << w)); /* check range below */ + + secp256k1_scalar_mul(&x, &x, &shift); + if (v >= 0) { + secp256k1_scalar_set_int(&t, v); + } else { + secp256k1_scalar_set_int(&t, -v); + secp256k1_scalar_negate(&t, &t); + } + secp256k1_scalar_add(&x, &x, &t); + } +#ifdef USE_ENDOMORPHISM + /* Skew num because when encoding 128-bit numbers as odd we use an offset */ + secp256k1_scalar_cadd_bit(&num, skew == 2, 1); +#endif + CHECK(secp256k1_scalar_eq(&x, &num)); +} + +void run_wnaf(void) { + int i; + secp256k1_scalar n = {{0}}; + + /* Sanity check: 1 and 2 are the smallest odd and even numbers and should + * have easier-to-diagnose failure modes */ + n.d[0] = 1; + test_constant_wnaf(&n, 4); + n.d[0] = 2; + test_constant_wnaf(&n, 4); + /* Random tests */ + for (i = 0; i < count; i++) { + random_scalar_order(&n); + test_wnaf(&n, 4+(i%10)); + test_constant_wnaf_negate(&n); + test_constant_wnaf(&n, 4 + (i % 10)); + } + secp256k1_scalar_set_int(&n, 0); + CHECK(secp256k1_scalar_cond_negate(&n, 1) == -1); + CHECK(secp256k1_scalar_is_zero(&n)); + CHECK(secp256k1_scalar_cond_negate(&n, 0) == 1); + CHECK(secp256k1_scalar_is_zero(&n)); +} + +void test_ecmult_constants(void) { + /* Test ecmult_gen() for [0..36) and [order-36..0). */ + secp256k1_scalar x; + secp256k1_gej r; + secp256k1_ge ng; + int i; + int j; + secp256k1_ge_neg(&ng, &secp256k1_ge_const_g); + for (i = 0; i < 36; i++ ) { + secp256k1_scalar_set_int(&x, i); + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &r, &x); + for (j = 0; j < i; j++) { + if (j == i - 1) { + ge_equals_gej(&secp256k1_ge_const_g, &r); + } + secp256k1_gej_add_ge(&r, &r, &ng); + } + CHECK(secp256k1_gej_is_infinity(&r)); + } + for (i = 1; i <= 36; i++ ) { + secp256k1_scalar_set_int(&x, i); + secp256k1_scalar_negate(&x, &x); + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &r, &x); + for (j = 0; j < i; j++) { + if (j == i - 1) { + ge_equals_gej(&ng, &r); + } + secp256k1_gej_add_ge(&r, &r, &secp256k1_ge_const_g); + } + CHECK(secp256k1_gej_is_infinity(&r)); + } +} + +void run_ecmult_constants(void) { + test_ecmult_constants(); +} + +void test_ecmult_gen_blind(void) { + /* Test ecmult_gen() blinding and confirm that the blinding changes, the affine points match, and the z's don't match. */ + secp256k1_scalar key; + secp256k1_scalar b; + unsigned char seed32[32]; + secp256k1_gej pgej; + secp256k1_gej pgej2; + secp256k1_gej i; + secp256k1_ge pge; + random_scalar_order_test(&key); + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &pgej, &key); + secp256k1_rand256(seed32); + b = ctx->ecmult_gen_ctx.blind; + i = ctx->ecmult_gen_ctx.initial; + secp256k1_ecmult_gen_blind(&ctx->ecmult_gen_ctx, seed32); + CHECK(!secp256k1_scalar_eq(&b, &ctx->ecmult_gen_ctx.blind)); + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &pgej2, &key); + CHECK(!gej_xyz_equals_gej(&pgej, &pgej2)); + CHECK(!gej_xyz_equals_gej(&i, &ctx->ecmult_gen_ctx.initial)); + secp256k1_ge_set_gej(&pge, &pgej); + ge_equals_gej(&pge, &pgej2); +} + +void test_ecmult_gen_blind_reset(void) { + /* Test ecmult_gen() blinding reset and confirm that the blinding is consistent. */ + secp256k1_scalar b; + secp256k1_gej initial; + secp256k1_ecmult_gen_blind(&ctx->ecmult_gen_ctx, 0); + b = ctx->ecmult_gen_ctx.blind; + initial = ctx->ecmult_gen_ctx.initial; + secp256k1_ecmult_gen_blind(&ctx->ecmult_gen_ctx, 0); + CHECK(secp256k1_scalar_eq(&b, &ctx->ecmult_gen_ctx.blind)); + CHECK(gej_xyz_equals_gej(&initial, &ctx->ecmult_gen_ctx.initial)); +} + +void run_ecmult_gen_blind(void) { + int i; + test_ecmult_gen_blind_reset(); + for (i = 0; i < 10; i++) { + test_ecmult_gen_blind(); + } +} + +#ifdef USE_ENDOMORPHISM +/***** ENDOMORPHISH TESTS *****/ +void test_scalar_split(void) { + secp256k1_scalar full; + secp256k1_scalar s1, slam; + const unsigned char zero[32] = {0}; + unsigned char tmp[32]; + + random_scalar_order_test(&full); + secp256k1_scalar_split_lambda(&s1, &slam, &full); + + /* check that both are <= 128 bits in size */ + if (secp256k1_scalar_is_high(&s1)) { + secp256k1_scalar_negate(&s1, &s1); + } + if (secp256k1_scalar_is_high(&slam)) { + secp256k1_scalar_negate(&slam, &slam); + } + + secp256k1_scalar_get_b32(tmp, &s1); + CHECK(memcmp(zero, tmp, 16) == 0); + secp256k1_scalar_get_b32(tmp, &slam); + CHECK(memcmp(zero, tmp, 16) == 0); +} + +void run_endomorphism_tests(void) { + test_scalar_split(); +} +#endif + +void ec_pubkey_parse_pointtest(const unsigned char *input, int xvalid, int yvalid) { + unsigned char pubkeyc[65]; + secp256k1_pubkey pubkey; + secp256k1_ge ge; + size_t pubkeyclen; + int32_t ecount; + ecount = 0; + secp256k1_context_set_illegal_callback(ctx, counting_illegal_callback_fn, &ecount); + for (pubkeyclen = 3; pubkeyclen <= 65; pubkeyclen++) { + /* Smaller sizes are tested exhaustively elsewhere. */ + int32_t i; + memcpy(&pubkeyc[1], input, 64); + VG_UNDEF(&pubkeyc[pubkeyclen], 65 - pubkeyclen); + for (i = 0; i < 256; i++) { + /* Try all type bytes. */ + int xpass; + int ypass; + int ysign; + pubkeyc[0] = i; + /* What sign does this point have? */ + ysign = (input[63] & 1) + 2; + /* For the current type (i) do we expect parsing to work? Handled all of compressed/uncompressed/hybrid. */ + xpass = xvalid && (pubkeyclen == 33) && ((i & 254) == 2); + /* Do we expect a parse and re-serialize as uncompressed to give a matching y? */ + ypass = xvalid && yvalid && ((i & 4) == ((pubkeyclen == 65) << 2)) && + ((i == 4) || ((i & 251) == ysign)) && ((pubkeyclen == 33) || (pubkeyclen == 65)); + if (xpass || ypass) { + /* These cases must parse. */ + unsigned char pubkeyo[65]; + size_t outl; + memset(&pubkey, 0, sizeof(pubkey)); + VG_UNDEF(&pubkey, sizeof(pubkey)); + ecount = 0; + CHECK(secp256k1_ec_pubkey_parse(ctx, &pubkey, pubkeyc, pubkeyclen) == 1); + VG_CHECK(&pubkey, sizeof(pubkey)); + outl = 65; + VG_UNDEF(pubkeyo, 65); + CHECK(secp256k1_ec_pubkey_serialize(ctx, pubkeyo, &outl, &pubkey, SECP256K1_EC_COMPRESSED) == 1); + VG_CHECK(pubkeyo, outl); + CHECK(outl == 33); + CHECK(memcmp(&pubkeyo[1], &pubkeyc[1], 32) == 0); + CHECK((pubkeyclen != 33) || (pubkeyo[0] == pubkeyc[0])); + if (ypass) { + /* This test isn't always done because we decode with alternative signs, so the y won't match. */ + CHECK(pubkeyo[0] == ysign); + CHECK(secp256k1_pubkey_load(ctx, &ge, &pubkey) == 1); + memset(&pubkey, 0, sizeof(pubkey)); + VG_UNDEF(&pubkey, sizeof(pubkey)); + secp256k1_pubkey_save(&pubkey, &ge); + VG_CHECK(&pubkey, sizeof(pubkey)); + outl = 65; + VG_UNDEF(pubkeyo, 65); + CHECK(secp256k1_ec_pubkey_serialize(ctx, pubkeyo, &outl, &pubkey, SECP256K1_EC_UNCOMPRESSED) == 1); + VG_CHECK(pubkeyo, outl); + CHECK(outl == 65); + CHECK(pubkeyo[0] == 4); + CHECK(memcmp(&pubkeyo[1], input, 64) == 0); + } + CHECK(ecount == 0); + } else { + /* These cases must fail to parse. */ + memset(&pubkey, 0xfe, sizeof(pubkey)); + ecount = 0; + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_parse(ctx, &pubkey, pubkeyc, pubkeyclen) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(ecount == 0); + CHECK(secp256k1_pubkey_load(ctx, &ge, &pubkey) == 0); + CHECK(ecount == 1); + } + } + } + secp256k1_context_set_illegal_callback(ctx, NULL, NULL); +} + +void run_ec_pubkey_parse_test(void) { +#define SECP256K1_EC_PARSE_TEST_NVALID (12) + const unsigned char valid[SECP256K1_EC_PARSE_TEST_NVALID][64] = { + { + /* Point with leading and trailing zeros in x and y serialization. */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x42, 0x52, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x64, 0xef, 0xa1, 0x7b, 0x77, 0x61, 0xe1, 0xe4, 0x27, 0x06, 0x98, 0x9f, 0xb4, 0x83, + 0xb8, 0xd2, 0xd4, 0x9b, 0xf7, 0x8f, 0xae, 0x98, 0x03, 0xf0, 0x99, 0xb8, 0x34, 0xed, 0xeb, 0x00 + }, + { + /* Point with x equal to a 3rd root of unity.*/ + 0x7a, 0xe9, 0x6a, 0x2b, 0x65, 0x7c, 0x07, 0x10, 0x6e, 0x64, 0x47, 0x9e, 0xac, 0x34, 0x34, 0xe9, + 0x9c, 0xf0, 0x49, 0x75, 0x12, 0xf5, 0x89, 0x95, 0xc1, 0x39, 0x6c, 0x28, 0x71, 0x95, 0x01, 0xee, + 0x42, 0x18, 0xf2, 0x0a, 0xe6, 0xc6, 0x46, 0xb3, 0x63, 0xdb, 0x68, 0x60, 0x58, 0x22, 0xfb, 0x14, + 0x26, 0x4c, 0xa8, 0xd2, 0x58, 0x7f, 0xdd, 0x6f, 0xbc, 0x75, 0x0d, 0x58, 0x7e, 0x76, 0xa7, 0xee, + }, + { + /* Point with largest x. (1/2) */ + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x2c, + 0x0e, 0x99, 0x4b, 0x14, 0xea, 0x72, 0xf8, 0xc3, 0xeb, 0x95, 0xc7, 0x1e, 0xf6, 0x92, 0x57, 0x5e, + 0x77, 0x50, 0x58, 0x33, 0x2d, 0x7e, 0x52, 0xd0, 0x99, 0x5c, 0xf8, 0x03, 0x88, 0x71, 0xb6, 0x7d, + }, + { + /* Point with largest x. (2/2) */ + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x2c, + 0xf1, 0x66, 0xb4, 0xeb, 0x15, 0x8d, 0x07, 0x3c, 0x14, 0x6a, 0x38, 0xe1, 0x09, 0x6d, 0xa8, 0xa1, + 0x88, 0xaf, 0xa7, 0xcc, 0xd2, 0x81, 0xad, 0x2f, 0x66, 0xa3, 0x07, 0xfb, 0x77, 0x8e, 0x45, 0xb2, + }, + { + /* Point with smallest x. (1/2) */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x42, 0x18, 0xf2, 0x0a, 0xe6, 0xc6, 0x46, 0xb3, 0x63, 0xdb, 0x68, 0x60, 0x58, 0x22, 0xfb, 0x14, + 0x26, 0x4c, 0xa8, 0xd2, 0x58, 0x7f, 0xdd, 0x6f, 0xbc, 0x75, 0x0d, 0x58, 0x7e, 0x76, 0xa7, 0xee, + }, + { + /* Point with smallest x. (2/2) */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0xbd, 0xe7, 0x0d, 0xf5, 0x19, 0x39, 0xb9, 0x4c, 0x9c, 0x24, 0x97, 0x9f, 0xa7, 0xdd, 0x04, 0xeb, + 0xd9, 0xb3, 0x57, 0x2d, 0xa7, 0x80, 0x22, 0x90, 0x43, 0x8a, 0xf2, 0xa6, 0x81, 0x89, 0x54, 0x41, + }, + { + /* Point with largest y. (1/3) */ + 0x1f, 0xe1, 0xe5, 0xef, 0x3f, 0xce, 0xb5, 0xc1, 0x35, 0xab, 0x77, 0x41, 0x33, 0x3c, 0xe5, 0xa6, + 0xe8, 0x0d, 0x68, 0x16, 0x76, 0x53, 0xf6, 0xb2, 0xb2, 0x4b, 0xcb, 0xcf, 0xaa, 0xaf, 0xf5, 0x07, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x2e, + }, + { + /* Point with largest y. (2/3) */ + 0xcb, 0xb0, 0xde, 0xab, 0x12, 0x57, 0x54, 0xf1, 0xfd, 0xb2, 0x03, 0x8b, 0x04, 0x34, 0xed, 0x9c, + 0xb3, 0xfb, 0x53, 0xab, 0x73, 0x53, 0x91, 0x12, 0x99, 0x94, 0xa5, 0x35, 0xd9, 0x25, 0xf6, 0x73, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x2e, + }, + { + /* Point with largest y. (3/3) */ + 0x14, 0x6d, 0x3b, 0x65, 0xad, 0xd9, 0xf5, 0x4c, 0xcc, 0xa2, 0x85, 0x33, 0xc8, 0x8e, 0x2c, 0xbc, + 0x63, 0xf7, 0x44, 0x3e, 0x16, 0x58, 0x78, 0x3a, 0xb4, 0x1f, 0x8e, 0xf9, 0x7c, 0x2a, 0x10, 0xb5, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x2e, + }, + { + /* Point with smallest y. (1/3) */ + 0x1f, 0xe1, 0xe5, 0xef, 0x3f, 0xce, 0xb5, 0xc1, 0x35, 0xab, 0x77, 0x41, 0x33, 0x3c, 0xe5, 0xa6, + 0xe8, 0x0d, 0x68, 0x16, 0x76, 0x53, 0xf6, 0xb2, 0xb2, 0x4b, 0xcb, 0xcf, 0xaa, 0xaf, 0xf5, 0x07, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + }, + { + /* Point with smallest y. (2/3) */ + 0xcb, 0xb0, 0xde, 0xab, 0x12, 0x57, 0x54, 0xf1, 0xfd, 0xb2, 0x03, 0x8b, 0x04, 0x34, 0xed, 0x9c, + 0xb3, 0xfb, 0x53, 0xab, 0x73, 0x53, 0x91, 0x12, 0x99, 0x94, 0xa5, 0x35, 0xd9, 0x25, 0xf6, 0x73, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + }, + { + /* Point with smallest y. (3/3) */ + 0x14, 0x6d, 0x3b, 0x65, 0xad, 0xd9, 0xf5, 0x4c, 0xcc, 0xa2, 0x85, 0x33, 0xc8, 0x8e, 0x2c, 0xbc, + 0x63, 0xf7, 0x44, 0x3e, 0x16, 0x58, 0x78, 0x3a, 0xb4, 0x1f, 0x8e, 0xf9, 0x7c, 0x2a, 0x10, 0xb5, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01 + } + }; +#define SECP256K1_EC_PARSE_TEST_NXVALID (4) + const unsigned char onlyxvalid[SECP256K1_EC_PARSE_TEST_NXVALID][64] = { + { + /* Valid if y overflow ignored (y = 1 mod p). (1/3) */ + 0x1f, 0xe1, 0xe5, 0xef, 0x3f, 0xce, 0xb5, 0xc1, 0x35, 0xab, 0x77, 0x41, 0x33, 0x3c, 0xe5, 0xa6, + 0xe8, 0x0d, 0x68, 0x16, 0x76, 0x53, 0xf6, 0xb2, 0xb2, 0x4b, 0xcb, 0xcf, 0xaa, 0xaf, 0xf5, 0x07, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x30, + }, + { + /* Valid if y overflow ignored (y = 1 mod p). (2/3) */ + 0xcb, 0xb0, 0xde, 0xab, 0x12, 0x57, 0x54, 0xf1, 0xfd, 0xb2, 0x03, 0x8b, 0x04, 0x34, 0xed, 0x9c, + 0xb3, 0xfb, 0x53, 0xab, 0x73, 0x53, 0x91, 0x12, 0x99, 0x94, 0xa5, 0x35, 0xd9, 0x25, 0xf6, 0x73, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x30, + }, + { + /* Valid if y overflow ignored (y = 1 mod p). (3/3)*/ + 0x14, 0x6d, 0x3b, 0x65, 0xad, 0xd9, 0xf5, 0x4c, 0xcc, 0xa2, 0x85, 0x33, 0xc8, 0x8e, 0x2c, 0xbc, + 0x63, 0xf7, 0x44, 0x3e, 0x16, 0x58, 0x78, 0x3a, 0xb4, 0x1f, 0x8e, 0xf9, 0x7c, 0x2a, 0x10, 0xb5, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x30, + }, + { + /* x on curve, y is from y^2 = x^3 + 8. */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03 + } + }; +#define SECP256K1_EC_PARSE_TEST_NINVALID (7) + const unsigned char invalid[SECP256K1_EC_PARSE_TEST_NINVALID][64] = { + { + /* x is third root of -8, y is -1 * (x^3+7); also on the curve for y^2 = x^3 + 9. */ + 0x0a, 0x2d, 0x2b, 0xa9, 0x35, 0x07, 0xf1, 0xdf, 0x23, 0x37, 0x70, 0xc2, 0xa7, 0x97, 0x96, 0x2c, + 0xc6, 0x1f, 0x6d, 0x15, 0xda, 0x14, 0xec, 0xd4, 0x7d, 0x8d, 0x27, 0xae, 0x1c, 0xd5, 0xf8, 0x53, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + }, + { + /* Valid if x overflow ignored (x = 1 mod p). */ + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x30, + 0x42, 0x18, 0xf2, 0x0a, 0xe6, 0xc6, 0x46, 0xb3, 0x63, 0xdb, 0x68, 0x60, 0x58, 0x22, 0xfb, 0x14, + 0x26, 0x4c, 0xa8, 0xd2, 0x58, 0x7f, 0xdd, 0x6f, 0xbc, 0x75, 0x0d, 0x58, 0x7e, 0x76, 0xa7, 0xee, + }, + { + /* Valid if x overflow ignored (x = 1 mod p). */ + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x30, + 0xbd, 0xe7, 0x0d, 0xf5, 0x19, 0x39, 0xb9, 0x4c, 0x9c, 0x24, 0x97, 0x9f, 0xa7, 0xdd, 0x04, 0xeb, + 0xd9, 0xb3, 0x57, 0x2d, 0xa7, 0x80, 0x22, 0x90, 0x43, 0x8a, 0xf2, 0xa6, 0x81, 0x89, 0x54, 0x41, + }, + { + /* x is -1, y is the result of the sqrt ladder; also on the curve for y^2 = x^3 - 5. */ + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x2e, + 0xf4, 0x84, 0x14, 0x5c, 0xb0, 0x14, 0x9b, 0x82, 0x5d, 0xff, 0x41, 0x2f, 0xa0, 0x52, 0xa8, 0x3f, + 0xcb, 0x72, 0xdb, 0x61, 0xd5, 0x6f, 0x37, 0x70, 0xce, 0x06, 0x6b, 0x73, 0x49, 0xa2, 0xaa, 0x28, + }, + { + /* x is -1, y is the result of the sqrt ladder; also on the curve for y^2 = x^3 - 5. */ + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, 0xff, 0xff, 0xfc, 0x2e, + 0x0b, 0x7b, 0xeb, 0xa3, 0x4f, 0xeb, 0x64, 0x7d, 0xa2, 0x00, 0xbe, 0xd0, 0x5f, 0xad, 0x57, 0xc0, + 0x34, 0x8d, 0x24, 0x9e, 0x2a, 0x90, 0xc8, 0x8f, 0x31, 0xf9, 0x94, 0x8b, 0xb6, 0x5d, 0x52, 0x07, + }, + { + /* x is zero, y is the result of the sqrt ladder; also on the curve for y^2 = x^3 - 7. */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x8f, 0x53, 0x7e, 0xef, 0xdf, 0xc1, 0x60, 0x6a, 0x07, 0x27, 0xcd, 0x69, 0xb4, 0xa7, 0x33, 0x3d, + 0x38, 0xed, 0x44, 0xe3, 0x93, 0x2a, 0x71, 0x79, 0xee, 0xcb, 0x4b, 0x6f, 0xba, 0x93, 0x60, 0xdc, + }, + { + /* x is zero, y is the result of the sqrt ladder; also on the curve for y^2 = x^3 - 7. */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x70, 0xac, 0x81, 0x10, 0x20, 0x3e, 0x9f, 0x95, 0xf8, 0xd8, 0x32, 0x96, 0x4b, 0x58, 0xcc, 0xc2, + 0xc7, 0x12, 0xbb, 0x1c, 0x6c, 0xd5, 0x8e, 0x86, 0x11, 0x34, 0xb4, 0x8f, 0x45, 0x6c, 0x9b, 0x53 + } + }; + const unsigned char pubkeyc[66] = { + /* Serialization of G. */ + 0x04, 0x79, 0xBE, 0x66, 0x7E, 0xF9, 0xDC, 0xBB, 0xAC, 0x55, 0xA0, 0x62, 0x95, 0xCE, 0x87, 0x0B, + 0x07, 0x02, 0x9B, 0xFC, 0xDB, 0x2D, 0xCE, 0x28, 0xD9, 0x59, 0xF2, 0x81, 0x5B, 0x16, 0xF8, 0x17, + 0x98, 0x48, 0x3A, 0xDA, 0x77, 0x26, 0xA3, 0xC4, 0x65, 0x5D, 0xA4, 0xFB, 0xFC, 0x0E, 0x11, 0x08, + 0xA8, 0xFD, 0x17, 0xB4, 0x48, 0xA6, 0x85, 0x54, 0x19, 0x9C, 0x47, 0xD0, 0x8F, 0xFB, 0x10, 0xD4, + 0xB8, 0x00 + }; + unsigned char sout[65]; + unsigned char shortkey[2]; + secp256k1_ge ge; + secp256k1_pubkey pubkey; + size_t len; + int32_t i; + int32_t ecount; + int32_t ecount2; + ecount = 0; + /* Nothing should be reading this far into pubkeyc. */ + VG_UNDEF(&pubkeyc[65], 1); + secp256k1_context_set_illegal_callback(ctx, counting_illegal_callback_fn, &ecount); + /* Zero length claimed, fail, zeroize, no illegal arg error. */ + memset(&pubkey, 0xfe, sizeof(pubkey)); + ecount = 0; + VG_UNDEF(shortkey, 2); + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_parse(ctx, &pubkey, shortkey, 0) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(ecount == 0); + CHECK(secp256k1_pubkey_load(ctx, &ge, &pubkey) == 0); + CHECK(ecount == 1); + /* Length one claimed, fail, zeroize, no illegal arg error. */ + for (i = 0; i < 256 ; i++) { + memset(&pubkey, 0xfe, sizeof(pubkey)); + ecount = 0; + shortkey[0] = i; + VG_UNDEF(&shortkey[1], 1); + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_parse(ctx, &pubkey, shortkey, 1) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(ecount == 0); + CHECK(secp256k1_pubkey_load(ctx, &ge, &pubkey) == 0); + CHECK(ecount == 1); + } + /* Length two claimed, fail, zeroize, no illegal arg error. */ + for (i = 0; i < 65536 ; i++) { + memset(&pubkey, 0xfe, sizeof(pubkey)); + ecount = 0; + shortkey[0] = i & 255; + shortkey[1] = i >> 8; + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_parse(ctx, &pubkey, shortkey, 2) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(ecount == 0); + CHECK(secp256k1_pubkey_load(ctx, &ge, &pubkey) == 0); + CHECK(ecount == 1); + } + memset(&pubkey, 0xfe, sizeof(pubkey)); + ecount = 0; + VG_UNDEF(&pubkey, sizeof(pubkey)); + /* 33 bytes claimed on otherwise valid input starting with 0x04, fail, zeroize output, no illegal arg error. */ + CHECK(secp256k1_ec_pubkey_parse(ctx, &pubkey, pubkeyc, 33) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(ecount == 0); + CHECK(secp256k1_pubkey_load(ctx, &ge, &pubkey) == 0); + CHECK(ecount == 1); + /* NULL pubkey, illegal arg error. Pubkey isn't rewritten before this step, since it's NULL into the parser. */ + CHECK(secp256k1_ec_pubkey_parse(ctx, NULL, pubkeyc, 65) == 0); + CHECK(ecount == 2); + /* NULL input string. Illegal arg and zeroize output. */ + memset(&pubkey, 0xfe, sizeof(pubkey)); + ecount = 0; + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_parse(ctx, &pubkey, NULL, 65) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(ecount == 1); + CHECK(secp256k1_pubkey_load(ctx, &ge, &pubkey) == 0); + CHECK(ecount == 2); + /* 64 bytes claimed on input starting with 0x04, fail, zeroize output, no illegal arg error. */ + memset(&pubkey, 0xfe, sizeof(pubkey)); + ecount = 0; + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_parse(ctx, &pubkey, pubkeyc, 64) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(ecount == 0); + CHECK(secp256k1_pubkey_load(ctx, &ge, &pubkey) == 0); + CHECK(ecount == 1); + /* 66 bytes claimed, fail, zeroize output, no illegal arg error. */ + memset(&pubkey, 0xfe, sizeof(pubkey)); + ecount = 0; + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_parse(ctx, &pubkey, pubkeyc, 66) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(ecount == 0); + CHECK(secp256k1_pubkey_load(ctx, &ge, &pubkey) == 0); + CHECK(ecount == 1); + /* Valid parse. */ + memset(&pubkey, 0, sizeof(pubkey)); + ecount = 0; + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_parse(ctx, &pubkey, pubkeyc, 65) == 1); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(ecount == 0); + VG_UNDEF(&ge, sizeof(ge)); + CHECK(secp256k1_pubkey_load(ctx, &ge, &pubkey) == 1); + VG_CHECK(&ge.x, sizeof(ge.x)); + VG_CHECK(&ge.y, sizeof(ge.y)); + VG_CHECK(&ge.infinity, sizeof(ge.infinity)); + ge_equals_ge(&secp256k1_ge_const_g, &ge); + CHECK(ecount == 0); + /* secp256k1_ec_pubkey_serialize illegal args. */ + ecount = 0; + len = 65; + CHECK(secp256k1_ec_pubkey_serialize(ctx, NULL, &len, &pubkey, SECP256K1_EC_UNCOMPRESSED) == 0); + CHECK(ecount == 1); + CHECK(len == 0); + CHECK(secp256k1_ec_pubkey_serialize(ctx, sout, NULL, &pubkey, SECP256K1_EC_UNCOMPRESSED) == 0); + CHECK(ecount == 2); + len = 65; + VG_UNDEF(sout, 65); + CHECK(secp256k1_ec_pubkey_serialize(ctx, sout, &len, NULL, SECP256K1_EC_UNCOMPRESSED) == 0); + VG_CHECK(sout, 65); + CHECK(ecount == 3); + CHECK(len == 0); + len = 65; + CHECK(secp256k1_ec_pubkey_serialize(ctx, sout, &len, &pubkey, ~0) == 0); + CHECK(ecount == 4); + CHECK(len == 0); + len = 65; + VG_UNDEF(sout, 65); + CHECK(secp256k1_ec_pubkey_serialize(ctx, sout, &len, &pubkey, SECP256K1_EC_UNCOMPRESSED) == 1); + VG_CHECK(sout, 65); + CHECK(ecount == 4); + CHECK(len == 65); + /* Multiple illegal args. Should still set arg error only once. */ + ecount = 0; + ecount2 = 11; + CHECK(secp256k1_ec_pubkey_parse(ctx, NULL, NULL, 65) == 0); + CHECK(ecount == 1); + /* Does the illegal arg callback actually change the behavior? */ + secp256k1_context_set_illegal_callback(ctx, uncounting_illegal_callback_fn, &ecount2); + CHECK(secp256k1_ec_pubkey_parse(ctx, NULL, NULL, 65) == 0); + CHECK(ecount == 1); + CHECK(ecount2 == 10); + secp256k1_context_set_illegal_callback(ctx, NULL, NULL); + /* Try a bunch of prefabbed points with all possible encodings. */ + for (i = 0; i < SECP256K1_EC_PARSE_TEST_NVALID; i++) { + ec_pubkey_parse_pointtest(valid[i], 1, 1); + } + for (i = 0; i < SECP256K1_EC_PARSE_TEST_NXVALID; i++) { + ec_pubkey_parse_pointtest(onlyxvalid[i], 1, 0); + } + for (i = 0; i < SECP256K1_EC_PARSE_TEST_NINVALID; i++) { + ec_pubkey_parse_pointtest(invalid[i], 0, 0); + } +} + +void run_eckey_edge_case_test(void) { + const unsigned char orderc[32] = { + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, + 0xba, 0xae, 0xdc, 0xe6, 0xaf, 0x48, 0xa0, 0x3b, + 0xbf, 0xd2, 0x5e, 0x8c, 0xd0, 0x36, 0x41, 0x41 + }; + const unsigned char zeros[sizeof(secp256k1_pubkey)] = {0x00}; + unsigned char ctmp[33]; + unsigned char ctmp2[33]; + secp256k1_pubkey pubkey; + secp256k1_pubkey pubkey2; + secp256k1_pubkey pubkey_one; + secp256k1_pubkey pubkey_negone; + const secp256k1_pubkey *pubkeys[3]; + size_t len; + int32_t ecount; + /* Group order is too large, reject. */ + CHECK(secp256k1_ec_seckey_verify(ctx, orderc) == 0); + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, orderc) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) == 0); + /* Maximum value is too large, reject. */ + memset(ctmp, 255, 32); + CHECK(secp256k1_ec_seckey_verify(ctx, ctmp) == 0); + memset(&pubkey, 1, sizeof(pubkey)); + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, ctmp) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) == 0); + /* Zero is too small, reject. */ + memset(ctmp, 0, 32); + CHECK(secp256k1_ec_seckey_verify(ctx, ctmp) == 0); + memset(&pubkey, 1, sizeof(pubkey)); + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, ctmp) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) == 0); + /* One must be accepted. */ + ctmp[31] = 0x01; + CHECK(secp256k1_ec_seckey_verify(ctx, ctmp) == 1); + memset(&pubkey, 0, sizeof(pubkey)); + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, ctmp) == 1); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) > 0); + pubkey_one = pubkey; + /* Group order + 1 is too large, reject. */ + memcpy(ctmp, orderc, 32); + ctmp[31] = 0x42; + CHECK(secp256k1_ec_seckey_verify(ctx, ctmp) == 0); + memset(&pubkey, 1, sizeof(pubkey)); + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, ctmp) == 0); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) == 0); + /* -1 must be accepted. */ + ctmp[31] = 0x40; + CHECK(secp256k1_ec_seckey_verify(ctx, ctmp) == 1); + memset(&pubkey, 0, sizeof(pubkey)); + VG_UNDEF(&pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, ctmp) == 1); + VG_CHECK(&pubkey, sizeof(pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) > 0); + pubkey_negone = pubkey; + /* Tweak of zero leaves the value changed. */ + memset(ctmp2, 0, 32); + CHECK(secp256k1_ec_privkey_tweak_add(ctx, ctmp, ctmp2) == 1); + CHECK(memcmp(orderc, ctmp, 31) == 0 && ctmp[31] == 0x40); + memcpy(&pubkey2, &pubkey, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_tweak_add(ctx, &pubkey, ctmp2) == 1); + CHECK(memcmp(&pubkey, &pubkey2, sizeof(pubkey)) == 0); + /* Multiply tweak of zero zeroizes the output. */ + CHECK(secp256k1_ec_privkey_tweak_mul(ctx, ctmp, ctmp2) == 0); + CHECK(memcmp(zeros, ctmp, 32) == 0); + CHECK(secp256k1_ec_pubkey_tweak_mul(ctx, &pubkey, ctmp2) == 0); + CHECK(memcmp(&pubkey, zeros, sizeof(pubkey)) == 0); + memcpy(&pubkey, &pubkey2, sizeof(pubkey)); + /* Overflowing key tweak zeroizes. */ + memcpy(ctmp, orderc, 32); + ctmp[31] = 0x40; + CHECK(secp256k1_ec_privkey_tweak_add(ctx, ctmp, orderc) == 0); + CHECK(memcmp(zeros, ctmp, 32) == 0); + memcpy(ctmp, orderc, 32); + ctmp[31] = 0x40; + CHECK(secp256k1_ec_privkey_tweak_mul(ctx, ctmp, orderc) == 0); + CHECK(memcmp(zeros, ctmp, 32) == 0); + memcpy(ctmp, orderc, 32); + ctmp[31] = 0x40; + CHECK(secp256k1_ec_pubkey_tweak_add(ctx, &pubkey, orderc) == 0); + CHECK(memcmp(&pubkey, zeros, sizeof(pubkey)) == 0); + memcpy(&pubkey, &pubkey2, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_tweak_mul(ctx, &pubkey, orderc) == 0); + CHECK(memcmp(&pubkey, zeros, sizeof(pubkey)) == 0); + memcpy(&pubkey, &pubkey2, sizeof(pubkey)); + /* Private key tweaks results in a key of zero. */ + ctmp2[31] = 1; + CHECK(secp256k1_ec_privkey_tweak_add(ctx, ctmp2, ctmp) == 0); + CHECK(memcmp(zeros, ctmp2, 32) == 0); + ctmp2[31] = 1; + CHECK(secp256k1_ec_pubkey_tweak_add(ctx, &pubkey, ctmp2) == 0); + CHECK(memcmp(&pubkey, zeros, sizeof(pubkey)) == 0); + memcpy(&pubkey, &pubkey2, sizeof(pubkey)); + /* Tweak computation wraps and results in a key of 1. */ + ctmp2[31] = 2; + CHECK(secp256k1_ec_privkey_tweak_add(ctx, ctmp2, ctmp) == 1); + CHECK(memcmp(ctmp2, zeros, 31) == 0 && ctmp2[31] == 1); + ctmp2[31] = 2; + CHECK(secp256k1_ec_pubkey_tweak_add(ctx, &pubkey, ctmp2) == 1); + ctmp2[31] = 1; + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey2, ctmp2) == 1); + CHECK(memcmp(&pubkey, &pubkey2, sizeof(pubkey)) == 0); + /* Tweak mul * 2 = 1+1. */ + CHECK(secp256k1_ec_pubkey_tweak_add(ctx, &pubkey, ctmp2) == 1); + ctmp2[31] = 2; + CHECK(secp256k1_ec_pubkey_tweak_mul(ctx, &pubkey2, ctmp2) == 1); + CHECK(memcmp(&pubkey, &pubkey2, sizeof(pubkey)) == 0); + /* Test argument errors. */ + ecount = 0; + secp256k1_context_set_illegal_callback(ctx, counting_illegal_callback_fn, &ecount); + CHECK(ecount == 0); + /* Zeroize pubkey on parse error. */ + memset(&pubkey, 0, 32); + CHECK(secp256k1_ec_pubkey_tweak_add(ctx, &pubkey, ctmp2) == 0); + CHECK(ecount == 1); + CHECK(memcmp(&pubkey, zeros, sizeof(pubkey)) == 0); + memcpy(&pubkey, &pubkey2, sizeof(pubkey)); + memset(&pubkey2, 0, 32); + CHECK(secp256k1_ec_pubkey_tweak_mul(ctx, &pubkey2, ctmp2) == 0); + CHECK(ecount == 2); + CHECK(memcmp(&pubkey2, zeros, sizeof(pubkey2)) == 0); + /* Plain argument errors. */ + ecount = 0; + CHECK(secp256k1_ec_seckey_verify(ctx, ctmp) == 1); + CHECK(ecount == 0); + CHECK(secp256k1_ec_seckey_verify(ctx, NULL) == 0); + CHECK(ecount == 1); + ecount = 0; + memset(ctmp2, 0, 32); + ctmp2[31] = 4; + CHECK(secp256k1_ec_pubkey_tweak_add(ctx, NULL, ctmp2) == 0); + CHECK(ecount == 1); + CHECK(secp256k1_ec_pubkey_tweak_add(ctx, &pubkey, NULL) == 0); + CHECK(ecount == 2); + ecount = 0; + memset(ctmp2, 0, 32); + ctmp2[31] = 4; + CHECK(secp256k1_ec_pubkey_tweak_mul(ctx, NULL, ctmp2) == 0); + CHECK(ecount == 1); + CHECK(secp256k1_ec_pubkey_tweak_mul(ctx, &pubkey, NULL) == 0); + CHECK(ecount == 2); + ecount = 0; + memset(ctmp2, 0, 32); + CHECK(secp256k1_ec_privkey_tweak_add(ctx, NULL, ctmp2) == 0); + CHECK(ecount == 1); + CHECK(secp256k1_ec_privkey_tweak_add(ctx, ctmp, NULL) == 0); + CHECK(ecount == 2); + ecount = 0; + memset(ctmp2, 0, 32); + ctmp2[31] = 1; + CHECK(secp256k1_ec_privkey_tweak_mul(ctx, NULL, ctmp2) == 0); + CHECK(ecount == 1); + CHECK(secp256k1_ec_privkey_tweak_mul(ctx, ctmp, NULL) == 0); + CHECK(ecount == 2); + ecount = 0; + CHECK(secp256k1_ec_pubkey_create(ctx, NULL, ctmp) == 0); + CHECK(ecount == 1); + memset(&pubkey, 1, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, NULL) == 0); + CHECK(ecount == 2); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) == 0); + /* secp256k1_ec_pubkey_combine tests. */ + ecount = 0; + pubkeys[0] = &pubkey_one; + VG_UNDEF(&pubkeys[0], sizeof(secp256k1_pubkey *)); + VG_UNDEF(&pubkeys[1], sizeof(secp256k1_pubkey *)); + VG_UNDEF(&pubkeys[2], sizeof(secp256k1_pubkey *)); + memset(&pubkey, 255, sizeof(secp256k1_pubkey)); + VG_UNDEF(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(secp256k1_ec_pubkey_combine(ctx, &pubkey, pubkeys, 0) == 0); + VG_CHECK(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) == 0); + CHECK(ecount == 1); + CHECK(secp256k1_ec_pubkey_combine(ctx, NULL, pubkeys, 1) == 0); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) == 0); + CHECK(ecount == 2); + memset(&pubkey, 255, sizeof(secp256k1_pubkey)); + VG_UNDEF(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(secp256k1_ec_pubkey_combine(ctx, &pubkey, NULL, 1) == 0); + VG_CHECK(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) == 0); + CHECK(ecount == 3); + pubkeys[0] = &pubkey_negone; + memset(&pubkey, 255, sizeof(secp256k1_pubkey)); + VG_UNDEF(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(secp256k1_ec_pubkey_combine(ctx, &pubkey, pubkeys, 1) == 1); + VG_CHECK(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) > 0); + CHECK(ecount == 3); + len = 33; + CHECK(secp256k1_ec_pubkey_serialize(ctx, ctmp, &len, &pubkey, SECP256K1_EC_COMPRESSED) == 1); + CHECK(secp256k1_ec_pubkey_serialize(ctx, ctmp2, &len, &pubkey_negone, SECP256K1_EC_COMPRESSED) == 1); + CHECK(memcmp(ctmp, ctmp2, 33) == 0); + /* Result is infinity. */ + pubkeys[0] = &pubkey_one; + pubkeys[1] = &pubkey_negone; + memset(&pubkey, 255, sizeof(secp256k1_pubkey)); + VG_UNDEF(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(secp256k1_ec_pubkey_combine(ctx, &pubkey, pubkeys, 2) == 0); + VG_CHECK(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) == 0); + CHECK(ecount == 3); + /* Passes through infinity but comes out one. */ + pubkeys[2] = &pubkey_one; + memset(&pubkey, 255, sizeof(secp256k1_pubkey)); + VG_UNDEF(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(secp256k1_ec_pubkey_combine(ctx, &pubkey, pubkeys, 3) == 1); + VG_CHECK(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) > 0); + CHECK(ecount == 3); + len = 33; + CHECK(secp256k1_ec_pubkey_serialize(ctx, ctmp, &len, &pubkey, SECP256K1_EC_COMPRESSED) == 1); + CHECK(secp256k1_ec_pubkey_serialize(ctx, ctmp2, &len, &pubkey_one, SECP256K1_EC_COMPRESSED) == 1); + CHECK(memcmp(ctmp, ctmp2, 33) == 0); + /* Adds to two. */ + pubkeys[1] = &pubkey_one; + memset(&pubkey, 255, sizeof(secp256k1_pubkey)); + VG_UNDEF(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(secp256k1_ec_pubkey_combine(ctx, &pubkey, pubkeys, 2) == 1); + VG_CHECK(&pubkey, sizeof(secp256k1_pubkey)); + CHECK(memcmp(&pubkey, zeros, sizeof(secp256k1_pubkey)) > 0); + CHECK(ecount == 3); + secp256k1_context_set_illegal_callback(ctx, NULL, NULL); +} + +void random_sign(secp256k1_scalar *sigr, secp256k1_scalar *sigs, const secp256k1_scalar *key, const secp256k1_scalar *msg, int *recid) { + secp256k1_scalar nonce; + do { + random_scalar_order_test(&nonce); + } while(!secp256k1_ecdsa_sig_sign(&ctx->ecmult_gen_ctx, sigr, sigs, key, msg, &nonce, recid)); +} + +void test_ecdsa_sign_verify(void) { + secp256k1_gej pubj; + secp256k1_ge pub; + secp256k1_scalar one; + secp256k1_scalar msg, key; + secp256k1_scalar sigr, sigs; + int recid; + int getrec; + random_scalar_order_test(&msg); + random_scalar_order_test(&key); + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &pubj, &key); + secp256k1_ge_set_gej(&pub, &pubj); + getrec = secp256k1_rand_bits(1); + random_sign(&sigr, &sigs, &key, &msg, getrec?&recid:NULL); + if (getrec) { + CHECK(recid >= 0 && recid < 4); + } + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sigr, &sigs, &pub, &msg)); + secp256k1_scalar_set_int(&one, 1); + secp256k1_scalar_add(&msg, &msg, &one); + CHECK(!secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sigr, &sigs, &pub, &msg)); +} + +void run_ecdsa_sign_verify(void) { + int i; + for (i = 0; i < 10*count; i++) { + test_ecdsa_sign_verify(); + } +} + +/** Dummy nonce generation function that just uses a precomputed nonce, and fails if it is not accepted. Use only for testing. */ +static int precomputed_nonce_function(unsigned char *nonce32, const unsigned char *msg32, const unsigned char *key32, const unsigned char *algo16, void *data, unsigned int counter) { + (void)msg32; + (void)key32; + (void)algo16; + memcpy(nonce32, data, 32); + return (counter == 0); +} + +static int nonce_function_test_fail(unsigned char *nonce32, const unsigned char *msg32, const unsigned char *key32, const unsigned char *algo16, void *data, unsigned int counter) { + /* Dummy nonce generator that has a fatal error on the first counter value. */ + if (counter == 0) { + return 0; + } + return nonce_function_rfc6979(nonce32, msg32, key32, algo16, data, counter - 1); +} + +static int nonce_function_test_retry(unsigned char *nonce32, const unsigned char *msg32, const unsigned char *key32, const unsigned char *algo16, void *data, unsigned int counter) { + /* Dummy nonce generator that produces unacceptable nonces for the first several counter values. */ + if (counter < 3) { + memset(nonce32, counter==0 ? 0 : 255, 32); + if (counter == 2) { + nonce32[31]--; + } + return 1; + } + if (counter < 5) { + static const unsigned char order[] = { + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFE, + 0xBA,0xAE,0xDC,0xE6,0xAF,0x48,0xA0,0x3B, + 0xBF,0xD2,0x5E,0x8C,0xD0,0x36,0x41,0x41 + }; + memcpy(nonce32, order, 32); + if (counter == 4) { + nonce32[31]++; + } + return 1; + } + /* Retry rate of 6979 is negligible esp. as we only call this in deterministic tests. */ + /* If someone does fine a case where it retries for secp256k1, we'd like to know. */ + if (counter > 5) { + return 0; + } + return nonce_function_rfc6979(nonce32, msg32, key32, algo16, data, counter - 5); +} + +int is_empty_signature(const secp256k1_ecdsa_signature *sig) { + static const unsigned char res[sizeof(secp256k1_ecdsa_signature)] = {0}; + return memcmp(sig, res, sizeof(secp256k1_ecdsa_signature)) == 0; +} + +void test_ecdsa_end_to_end(void) { + unsigned char extra[32] = {0x00}; + unsigned char privkey[32]; + unsigned char message[32]; + unsigned char privkey2[32]; + secp256k1_ecdsa_signature signature[6]; + secp256k1_scalar r, s; + unsigned char sig[74]; + size_t siglen = 74; + unsigned char pubkeyc[65]; + size_t pubkeyclen = 65; + secp256k1_pubkey pubkey; + unsigned char seckey[300]; + size_t seckeylen = 300; + + /* Generate a random key and message. */ + { + secp256k1_scalar msg, key; + random_scalar_order_test(&msg); + random_scalar_order_test(&key); + secp256k1_scalar_get_b32(privkey, &key); + secp256k1_scalar_get_b32(message, &msg); + } + + /* Construct and verify corresponding public key. */ + CHECK(secp256k1_ec_seckey_verify(ctx, privkey) == 1); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, privkey) == 1); + + /* Verify exporting and importing public key. */ + CHECK(secp256k1_ec_pubkey_serialize(ctx, pubkeyc, &pubkeyclen, &pubkey, secp256k1_rand_bits(1) == 1 ? SECP256K1_EC_COMPRESSED : SECP256K1_EC_UNCOMPRESSED)); + memset(&pubkey, 0, sizeof(pubkey)); + CHECK(secp256k1_ec_pubkey_parse(ctx, &pubkey, pubkeyc, pubkeyclen) == 1); + + /* Verify private key import and export. */ + CHECK(ec_privkey_export_der(ctx, seckey, &seckeylen, privkey, secp256k1_rand_bits(1) == 1)); + CHECK(ec_privkey_import_der(ctx, privkey2, seckey, seckeylen) == 1); + CHECK(memcmp(privkey, privkey2, 32) == 0); + + /* Optionally tweak the keys using addition. */ + if (secp256k1_rand_int(3) == 0) { + int ret1; + int ret2; + unsigned char rnd[32]; + secp256k1_pubkey pubkey2; + secp256k1_rand256_test(rnd); + ret1 = secp256k1_ec_privkey_tweak_add(ctx, privkey, rnd); + ret2 = secp256k1_ec_pubkey_tweak_add(ctx, &pubkey, rnd); + CHECK(ret1 == ret2); + if (ret1 == 0) { + return; + } + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey2, privkey) == 1); + CHECK(memcmp(&pubkey, &pubkey2, sizeof(pubkey)) == 0); + } + + /* Optionally tweak the keys using multiplication. */ + if (secp256k1_rand_int(3) == 0) { + int ret1; + int ret2; + unsigned char rnd[32]; + secp256k1_pubkey pubkey2; + secp256k1_rand256_test(rnd); + ret1 = secp256k1_ec_privkey_tweak_mul(ctx, privkey, rnd); + ret2 = secp256k1_ec_pubkey_tweak_mul(ctx, &pubkey, rnd); + CHECK(ret1 == ret2); + if (ret1 == 0) { + return; + } + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey2, privkey) == 1); + CHECK(memcmp(&pubkey, &pubkey2, sizeof(pubkey)) == 0); + } + + /* Sign. */ + CHECK(secp256k1_ecdsa_sign(ctx, &signature[0], message, privkey, NULL, NULL) == 1); + CHECK(secp256k1_ecdsa_sign(ctx, &signature[4], message, privkey, NULL, NULL) == 1); + CHECK(secp256k1_ecdsa_sign(ctx, &signature[1], message, privkey, NULL, extra) == 1); + extra[31] = 1; + CHECK(secp256k1_ecdsa_sign(ctx, &signature[2], message, privkey, NULL, extra) == 1); + extra[31] = 0; + extra[0] = 1; + CHECK(secp256k1_ecdsa_sign(ctx, &signature[3], message, privkey, NULL, extra) == 1); + CHECK(memcmp(&signature[0], &signature[4], sizeof(signature[0])) == 0); + CHECK(memcmp(&signature[0], &signature[1], sizeof(signature[0])) != 0); + CHECK(memcmp(&signature[0], &signature[2], sizeof(signature[0])) != 0); + CHECK(memcmp(&signature[0], &signature[3], sizeof(signature[0])) != 0); + CHECK(memcmp(&signature[1], &signature[2], sizeof(signature[0])) != 0); + CHECK(memcmp(&signature[1], &signature[3], sizeof(signature[0])) != 0); + CHECK(memcmp(&signature[2], &signature[3], sizeof(signature[0])) != 0); + /* Verify. */ + CHECK(secp256k1_ecdsa_verify(ctx, &signature[0], message, &pubkey) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &signature[1], message, &pubkey) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &signature[2], message, &pubkey) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &signature[3], message, &pubkey) == 1); + /* Test lower-S form, malleate, verify and fail, test again, malleate again */ + CHECK(!secp256k1_ecdsa_signature_normalize(ctx, NULL, &signature[0])); + secp256k1_ecdsa_signature_load(ctx, &r, &s, &signature[0]); + secp256k1_scalar_negate(&s, &s); + secp256k1_ecdsa_signature_save(&signature[5], &r, &s); + CHECK(secp256k1_ecdsa_verify(ctx, &signature[5], message, &pubkey) == 0); + CHECK(secp256k1_ecdsa_signature_normalize(ctx, NULL, &signature[5])); + CHECK(secp256k1_ecdsa_signature_normalize(ctx, &signature[5], &signature[5])); + CHECK(!secp256k1_ecdsa_signature_normalize(ctx, NULL, &signature[5])); + CHECK(!secp256k1_ecdsa_signature_normalize(ctx, &signature[5], &signature[5])); + CHECK(secp256k1_ecdsa_verify(ctx, &signature[5], message, &pubkey) == 1); + secp256k1_scalar_negate(&s, &s); + secp256k1_ecdsa_signature_save(&signature[5], &r, &s); + CHECK(!secp256k1_ecdsa_signature_normalize(ctx, NULL, &signature[5])); + CHECK(secp256k1_ecdsa_verify(ctx, &signature[5], message, &pubkey) == 1); + CHECK(memcmp(&signature[5], &signature[0], 64) == 0); + + /* Serialize/parse DER and verify again */ + CHECK(secp256k1_ecdsa_signature_serialize_der(ctx, sig, &siglen, &signature[0]) == 1); + memset(&signature[0], 0, sizeof(signature[0])); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &signature[0], sig, siglen) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, &signature[0], message, &pubkey) == 1); + /* Serialize/destroy/parse DER and verify again. */ + siglen = 74; + CHECK(secp256k1_ecdsa_signature_serialize_der(ctx, sig, &siglen, &signature[0]) == 1); + sig[secp256k1_rand_int(siglen)] += 1 + secp256k1_rand_int(255); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &signature[0], sig, siglen) == 0 || + secp256k1_ecdsa_verify(ctx, &signature[0], message, &pubkey) == 0); +} + +void test_random_pubkeys(void) { + secp256k1_ge elem; + secp256k1_ge elem2; + unsigned char in[65]; + /* Generate some randomly sized pubkeys. */ + size_t len = secp256k1_rand_bits(2) == 0 ? 65 : 33; + if (secp256k1_rand_bits(2) == 0) { + len = secp256k1_rand_bits(6); + } + if (len == 65) { + in[0] = secp256k1_rand_bits(1) ? 4 : (secp256k1_rand_bits(1) ? 6 : 7); + } else { + in[0] = secp256k1_rand_bits(1) ? 2 : 3; + } + if (secp256k1_rand_bits(3) == 0) { + in[0] = secp256k1_rand_bits(8); + } + if (len > 1) { + secp256k1_rand256(&in[1]); + } + if (len > 33) { + secp256k1_rand256(&in[33]); + } + if (secp256k1_eckey_pubkey_parse(&elem, in, len)) { + unsigned char out[65]; + unsigned char firstb; + int res; + size_t size = len; + firstb = in[0]; + /* If the pubkey can be parsed, it should round-trip... */ + CHECK(secp256k1_eckey_pubkey_serialize(&elem, out, &size, len == 33)); + CHECK(size == len); + CHECK(memcmp(&in[1], &out[1], len-1) == 0); + /* ... except for the type of hybrid inputs. */ + if ((in[0] != 6) && (in[0] != 7)) { + CHECK(in[0] == out[0]); + } + size = 65; + CHECK(secp256k1_eckey_pubkey_serialize(&elem, in, &size, 0)); + CHECK(size == 65); + CHECK(secp256k1_eckey_pubkey_parse(&elem2, in, size)); + ge_equals_ge(&elem,&elem2); + /* Check that the X9.62 hybrid type is checked. */ + in[0] = secp256k1_rand_bits(1) ? 6 : 7; + res = secp256k1_eckey_pubkey_parse(&elem2, in, size); + if (firstb == 2 || firstb == 3) { + if (in[0] == firstb + 4) { + CHECK(res); + } else { + CHECK(!res); + } + } + if (res) { + ge_equals_ge(&elem,&elem2); + CHECK(secp256k1_eckey_pubkey_serialize(&elem, out, &size, 0)); + CHECK(memcmp(&in[1], &out[1], 64) == 0); + } + } +} + +void run_random_pubkeys(void) { + int i; + for (i = 0; i < 10*count; i++) { + test_random_pubkeys(); + } +} + +void run_ecdsa_end_to_end(void) { + int i; + for (i = 0; i < 64*count; i++) { + test_ecdsa_end_to_end(); + } +} + +int test_ecdsa_der_parse(const unsigned char *sig, size_t siglen, int certainly_der, int certainly_not_der) { + static const unsigned char zeroes[32] = {0}; + static const unsigned char max_scalar[32] = { + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, + 0xba, 0xae, 0xdc, 0xe6, 0xaf, 0x48, 0xa0, 0x3b, + 0xbf, 0xd2, 0x5e, 0x8c, 0xd0, 0x36, 0x41, 0x40 + }; + + int ret = 0; + + secp256k1_ecdsa_signature sig_der; + unsigned char roundtrip_der[2048]; + unsigned char compact_der[64]; + size_t len_der = 2048; + int parsed_der = 0, valid_der = 0, roundtrips_der = 0; + + secp256k1_ecdsa_signature sig_der_lax; + unsigned char roundtrip_der_lax[2048]; + unsigned char compact_der_lax[64]; + size_t len_der_lax = 2048; + int parsed_der_lax = 0, valid_der_lax = 0, roundtrips_der_lax = 0; + +#ifdef ENABLE_OPENSSL_TESTS + ECDSA_SIG *sig_openssl; + const unsigned char *sigptr; + unsigned char roundtrip_openssl[2048]; + int len_openssl = 2048; + int parsed_openssl, valid_openssl = 0, roundtrips_openssl = 0; +#endif + + parsed_der = secp256k1_ecdsa_signature_parse_der(ctx, &sig_der, sig, siglen); + if (parsed_der) { + ret |= (!secp256k1_ecdsa_signature_serialize_compact(ctx, compact_der, &sig_der)) << 0; + valid_der = (memcmp(compact_der, zeroes, 32) != 0) && (memcmp(compact_der + 32, zeroes, 32) != 0); + } + if (valid_der) { + ret |= (!secp256k1_ecdsa_signature_serialize_der(ctx, roundtrip_der, &len_der, &sig_der)) << 1; + roundtrips_der = (len_der == siglen) && memcmp(roundtrip_der, sig, siglen) == 0; + } + + parsed_der_lax = ecdsa_signature_parse_der_lax(ctx, &sig_der_lax, sig, siglen); + if (parsed_der_lax) { + ret |= (!secp256k1_ecdsa_signature_serialize_compact(ctx, compact_der_lax, &sig_der_lax)) << 10; + valid_der_lax = (memcmp(compact_der_lax, zeroes, 32) != 0) && (memcmp(compact_der_lax + 32, zeroes, 32) != 0); + } + if (valid_der_lax) { + ret |= (!secp256k1_ecdsa_signature_serialize_der(ctx, roundtrip_der_lax, &len_der_lax, &sig_der_lax)) << 11; + roundtrips_der_lax = (len_der_lax == siglen) && memcmp(roundtrip_der_lax, sig, siglen) == 0; + } + + if (certainly_der) { + ret |= (!parsed_der) << 2; + } + if (certainly_not_der) { + ret |= (parsed_der) << 17; + } + if (valid_der) { + ret |= (!roundtrips_der) << 3; + } + + if (valid_der) { + ret |= (!roundtrips_der_lax) << 12; + ret |= (len_der != len_der_lax) << 13; + ret |= (memcmp(roundtrip_der_lax, roundtrip_der, len_der) != 0) << 14; + } + ret |= (roundtrips_der != roundtrips_der_lax) << 15; + if (parsed_der) { + ret |= (!parsed_der_lax) << 16; + } + +#ifdef ENABLE_OPENSSL_TESTS + sig_openssl = ECDSA_SIG_new(); + sigptr = sig; + parsed_openssl = (d2i_ECDSA_SIG(&sig_openssl, &sigptr, siglen) != NULL); + if (parsed_openssl) { + valid_openssl = !BN_is_negative(sig_openssl->r) && !BN_is_negative(sig_openssl->s) && BN_num_bits(sig_openssl->r) > 0 && BN_num_bits(sig_openssl->r) <= 256 && BN_num_bits(sig_openssl->s) > 0 && BN_num_bits(sig_openssl->s) <= 256; + if (valid_openssl) { + unsigned char tmp[32] = {0}; + BN_bn2bin(sig_openssl->r, tmp + 32 - BN_num_bytes(sig_openssl->r)); + valid_openssl = memcmp(tmp, max_scalar, 32) < 0; + } + if (valid_openssl) { + unsigned char tmp[32] = {0}; + BN_bn2bin(sig_openssl->s, tmp + 32 - BN_num_bytes(sig_openssl->s)); + valid_openssl = memcmp(tmp, max_scalar, 32) < 0; + } + } + len_openssl = i2d_ECDSA_SIG(sig_openssl, NULL); + if (len_openssl <= 2048) { + unsigned char *ptr = roundtrip_openssl; + CHECK(i2d_ECDSA_SIG(sig_openssl, &ptr) == len_openssl); + roundtrips_openssl = valid_openssl && ((size_t)len_openssl == siglen) && (memcmp(roundtrip_openssl, sig, siglen) == 0); + } else { + len_openssl = 0; + } + ECDSA_SIG_free(sig_openssl); + + ret |= (parsed_der && !parsed_openssl) << 4; + ret |= (valid_der && !valid_openssl) << 5; + ret |= (roundtrips_openssl && !parsed_der) << 6; + ret |= (roundtrips_der != roundtrips_openssl) << 7; + if (roundtrips_openssl) { + ret |= (len_der != (size_t)len_openssl) << 8; + ret |= (memcmp(roundtrip_der, roundtrip_openssl, len_der) != 0) << 9; + } +#endif + return ret; +} + +static void assign_big_endian(unsigned char *ptr, size_t ptrlen, uint32_t val) { + size_t i; + for (i = 0; i < ptrlen; i++) { + int shift = ptrlen - 1 - i; + if (shift >= 4) { + ptr[i] = 0; + } else { + ptr[i] = (val >> shift) & 0xFF; + } + } +} + +static void damage_array(unsigned char *sig, size_t *len) { + int pos; + int action = secp256k1_rand_bits(3); + if (action < 1) { + /* Delete a byte. */ + pos = secp256k1_rand_int(*len); + memmove(sig + pos, sig + pos + 1, *len - pos - 1); + (*len)--; + return; + } else if (action < 2) { + /* Insert a byte. */ + pos = secp256k1_rand_int(1 + *len); + memmove(sig + pos + 1, sig + pos, *len - pos); + sig[pos] = secp256k1_rand_bits(8); + (*len)++; + return; + } else if (action < 4) { + /* Modify a byte. */ + sig[secp256k1_rand_int(*len)] += 1 + secp256k1_rand_int(255); + return; + } else { /* action < 8 */ + /* Modify a bit. */ + sig[secp256k1_rand_int(*len)] ^= 1 << secp256k1_rand_bits(3); + return; + } +} + +static void random_ber_signature(unsigned char *sig, size_t *len, int* certainly_der, int* certainly_not_der) { + int der; + int nlow[2], nlen[2], nlenlen[2], nhbit[2], nhbyte[2], nzlen[2]; + size_t tlen, elen, glen; + int indet; + int n; + + *len = 0; + der = secp256k1_rand_bits(2) == 0; + *certainly_der = der; + *certainly_not_der = 0; + indet = der ? 0 : secp256k1_rand_int(10) == 0; + + for (n = 0; n < 2; n++) { + /* We generate two classes of numbers: nlow==1 "low" ones (up to 32 bytes), nlow==0 "high" ones (32 bytes with 129 top bits set, or larger than 32 bytes) */ + nlow[n] = der ? 1 : (secp256k1_rand_bits(3) != 0); + /* The length of the number in bytes (the first byte of which will always be nonzero) */ + nlen[n] = nlow[n] ? secp256k1_rand_int(33) : 32 + secp256k1_rand_int(200) * secp256k1_rand_int(8) / 8; + CHECK(nlen[n] <= 232); + /* The top bit of the number. */ + nhbit[n] = (nlow[n] == 0 && nlen[n] == 32) ? 1 : (nlen[n] == 0 ? 0 : secp256k1_rand_bits(1)); + /* The top byte of the number (after the potential hardcoded 16 0xFF characters for "high" 32 bytes numbers) */ + nhbyte[n] = nlen[n] == 0 ? 0 : (nhbit[n] ? 128 + secp256k1_rand_bits(7) : 1 + secp256k1_rand_int(127)); + /* The number of zero bytes in front of the number (which is 0 or 1 in case of DER, otherwise we extend up to 300 bytes) */ + nzlen[n] = der ? ((nlen[n] == 0 || nhbit[n]) ? 1 : 0) : (nlow[n] ? secp256k1_rand_int(3) : secp256k1_rand_int(300 - nlen[n]) * secp256k1_rand_int(8) / 8); + if (nzlen[n] > ((nlen[n] == 0 || nhbit[n]) ? 1 : 0)) { + *certainly_not_der = 1; + } + CHECK(nlen[n] + nzlen[n] <= 300); + /* The length of the length descriptor for the number. 0 means short encoding, anything else is long encoding. */ + nlenlen[n] = nlen[n] + nzlen[n] < 128 ? 0 : (nlen[n] + nzlen[n] < 256 ? 1 : 2); + if (!der) { + /* nlenlen[n] max 127 bytes */ + int add = secp256k1_rand_int(127 - nlenlen[n]) * secp256k1_rand_int(16) * secp256k1_rand_int(16) / 256; + nlenlen[n] += add; + if (add != 0) { + *certainly_not_der = 1; + } + } + CHECK(nlen[n] + nzlen[n] + nlenlen[n] <= 427); + } + + /* The total length of the data to go, so far */ + tlen = 2 + nlenlen[0] + nlen[0] + nzlen[0] + 2 + nlenlen[1] + nlen[1] + nzlen[1]; + CHECK(tlen <= 856); + + /* The length of the garbage inside the tuple. */ + elen = (der || indet) ? 0 : secp256k1_rand_int(980 - tlen) * secp256k1_rand_int(8) / 8; + if (elen != 0) { + *certainly_not_der = 1; + } + tlen += elen; + CHECK(tlen <= 980); + + /* The length of the garbage after the end of the tuple. */ + glen = der ? 0 : secp256k1_rand_int(990 - tlen) * secp256k1_rand_int(8) / 8; + if (glen != 0) { + *certainly_not_der = 1; + } + CHECK(tlen + glen <= 990); + + /* Write the tuple header. */ + sig[(*len)++] = 0x30; + if (indet) { + /* Indeterminate length */ + sig[(*len)++] = 0x80; + *certainly_not_der = 1; + } else { + int tlenlen = tlen < 128 ? 0 : (tlen < 256 ? 1 : 2); + if (!der) { + int add = secp256k1_rand_int(127 - tlenlen) * secp256k1_rand_int(16) * secp256k1_rand_int(16) / 256; + tlenlen += add; + if (add != 0) { + *certainly_not_der = 1; + } + } + if (tlenlen == 0) { + /* Short length notation */ + sig[(*len)++] = tlen; + } else { + /* Long length notation */ + sig[(*len)++] = 128 + tlenlen; + assign_big_endian(sig + *len, tlenlen, tlen); + *len += tlenlen; + } + tlen += tlenlen; + } + tlen += 2; + CHECK(tlen + glen <= 1119); + + for (n = 0; n < 2; n++) { + /* Write the integer header. */ + sig[(*len)++] = 0x02; + if (nlenlen[n] == 0) { + /* Short length notation */ + sig[(*len)++] = nlen[n] + nzlen[n]; + } else { + /* Long length notation. */ + sig[(*len)++] = 128 + nlenlen[n]; + assign_big_endian(sig + *len, nlenlen[n], nlen[n] + nzlen[n]); + *len += nlenlen[n]; + } + /* Write zero padding */ + while (nzlen[n] > 0) { + sig[(*len)++] = 0x00; + nzlen[n]--; + } + if (nlen[n] == 32 && !nlow[n]) { + /* Special extra 16 0xFF bytes in "high" 32-byte numbers */ + int i; + for (i = 0; i < 16; i++) { + sig[(*len)++] = 0xFF; + } + nlen[n] -= 16; + } + /* Write first byte of number */ + if (nlen[n] > 0) { + sig[(*len)++] = nhbyte[n]; + nlen[n]--; + } + /* Generate remaining random bytes of number */ + secp256k1_rand_bytes_test(sig + *len, nlen[n]); + *len += nlen[n]; + nlen[n] = 0; + } + + /* Generate random garbage inside tuple. */ + secp256k1_rand_bytes_test(sig + *len, elen); + *len += elen; + + /* Generate end-of-contents bytes. */ + if (indet) { + sig[(*len)++] = 0; + sig[(*len)++] = 0; + tlen += 2; + } + CHECK(tlen + glen <= 1121); + + /* Generate random garbage outside tuple. */ + secp256k1_rand_bytes_test(sig + *len, glen); + *len += glen; + tlen += glen; + CHECK(tlen <= 1121); + CHECK(tlen == *len); +} + +void run_ecdsa_der_parse(void) { + int i,j; + for (i = 0; i < 200 * count; i++) { + unsigned char buffer[2048]; + size_t buflen = 0; + int certainly_der = 0; + int certainly_not_der = 0; + random_ber_signature(buffer, &buflen, &certainly_der, &certainly_not_der); + for (j = 0; j < 16; j++) { + int ret = 0; + if (j > 0) { + damage_array(buffer, &buflen); + /* We don't know anything anymore about the DERness of the result */ + certainly_der = 0; + certainly_not_der = 0; + } + ret = test_ecdsa_der_parse(buffer, buflen, certainly_der, certainly_not_der); + if (ret != 0) { + size_t k; + fprintf(stderr, "Failure %x on ", ret); + for (k = 0; k < buflen; k++) { + fprintf(stderr, "%02x ", buffer[k]); + } + fprintf(stderr, "\n"); + } + CHECK(ret == 0); + } + } +} + +/* Tests several edge cases. */ +void test_ecdsa_edge_cases(void) { + int t; + secp256k1_ecdsa_signature sig; + + /* Test the case where ECDSA recomputes a point that is infinity. */ + { + secp256k1_gej keyj; + secp256k1_ge key; + secp256k1_scalar msg; + secp256k1_scalar sr, ss; + secp256k1_scalar_set_int(&ss, 1); + secp256k1_scalar_negate(&ss, &ss); + secp256k1_scalar_inverse(&ss, &ss); + secp256k1_scalar_set_int(&sr, 1); + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &keyj, &sr); + secp256k1_ge_set_gej(&key, &keyj); + msg = ss; + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 0); + } + + /* Verify signature with r of zero fails. */ + { + const unsigned char pubkey_mods_zero[33] = { + 0x02, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xfe, 0xba, 0xae, 0xdc, 0xe6, 0xaf, 0x48, 0xa0, + 0x3b, 0xbf, 0xd2, 0x5e, 0x8c, 0xd0, 0x36, 0x41, + 0x41 + }; + secp256k1_ge key; + secp256k1_scalar msg; + secp256k1_scalar sr, ss; + secp256k1_scalar_set_int(&ss, 1); + secp256k1_scalar_set_int(&msg, 0); + secp256k1_scalar_set_int(&sr, 0); + CHECK(secp256k1_eckey_pubkey_parse(&key, pubkey_mods_zero, 33)); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 0); + } + + /* Verify signature with s of zero fails. */ + { + const unsigned char pubkey[33] = { + 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x01 + }; + secp256k1_ge key; + secp256k1_scalar msg; + secp256k1_scalar sr, ss; + secp256k1_scalar_set_int(&ss, 0); + secp256k1_scalar_set_int(&msg, 0); + secp256k1_scalar_set_int(&sr, 1); + CHECK(secp256k1_eckey_pubkey_parse(&key, pubkey, 33)); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 0); + } + + /* Verify signature with message 0 passes. */ + { + const unsigned char pubkey[33] = { + 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x02 + }; + const unsigned char pubkey2[33] = { + 0x02, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xfe, 0xba, 0xae, 0xdc, 0xe6, 0xaf, 0x48, 0xa0, + 0x3b, 0xbf, 0xd2, 0x5e, 0x8c, 0xd0, 0x36, 0x41, + 0x43 + }; + secp256k1_ge key; + secp256k1_ge key2; + secp256k1_scalar msg; + secp256k1_scalar sr, ss; + secp256k1_scalar_set_int(&ss, 2); + secp256k1_scalar_set_int(&msg, 0); + secp256k1_scalar_set_int(&sr, 2); + CHECK(secp256k1_eckey_pubkey_parse(&key, pubkey, 33)); + CHECK(secp256k1_eckey_pubkey_parse(&key2, pubkey2, 33)); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 1); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key2, &msg) == 1); + secp256k1_scalar_negate(&ss, &ss); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 1); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key2, &msg) == 1); + secp256k1_scalar_set_int(&ss, 1); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 0); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key2, &msg) == 0); + } + + /* Verify signature with message 1 passes. */ + { + const unsigned char pubkey[33] = { + 0x02, 0x14, 0x4e, 0x5a, 0x58, 0xef, 0x5b, 0x22, + 0x6f, 0xd2, 0xe2, 0x07, 0x6a, 0x77, 0xcf, 0x05, + 0xb4, 0x1d, 0xe7, 0x4a, 0x30, 0x98, 0x27, 0x8c, + 0x93, 0xe6, 0xe6, 0x3c, 0x0b, 0xc4, 0x73, 0x76, + 0x25 + }; + const unsigned char pubkey2[33] = { + 0x02, 0x8a, 0xd5, 0x37, 0xed, 0x73, 0xd9, 0x40, + 0x1d, 0xa0, 0x33, 0xd2, 0xdc, 0xf0, 0xaf, 0xae, + 0x34, 0xcf, 0x5f, 0x96, 0x4c, 0x73, 0x28, 0x0f, + 0x92, 0xc0, 0xf6, 0x9d, 0xd9, 0xb2, 0x09, 0x10, + 0x62 + }; + const unsigned char csr[32] = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x45, 0x51, 0x23, 0x19, 0x50, 0xb7, 0x5f, 0xc4, + 0x40, 0x2d, 0xa1, 0x72, 0x2f, 0xc9, 0xba, 0xeb + }; + secp256k1_ge key; + secp256k1_ge key2; + secp256k1_scalar msg; + secp256k1_scalar sr, ss; + secp256k1_scalar_set_int(&ss, 1); + secp256k1_scalar_set_int(&msg, 1); + secp256k1_scalar_set_b32(&sr, csr, NULL); + CHECK(secp256k1_eckey_pubkey_parse(&key, pubkey, 33)); + CHECK(secp256k1_eckey_pubkey_parse(&key2, pubkey2, 33)); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 1); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key2, &msg) == 1); + secp256k1_scalar_negate(&ss, &ss); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 1); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key2, &msg) == 1); + secp256k1_scalar_set_int(&ss, 2); + secp256k1_scalar_inverse_var(&ss, &ss); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 0); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key2, &msg) == 0); + } + + /* Verify signature with message -1 passes. */ + { + const unsigned char pubkey[33] = { + 0x03, 0xaf, 0x97, 0xff, 0x7d, 0x3a, 0xf6, 0xa0, + 0x02, 0x94, 0xbd, 0x9f, 0x4b, 0x2e, 0xd7, 0x52, + 0x28, 0xdb, 0x49, 0x2a, 0x65, 0xcb, 0x1e, 0x27, + 0x57, 0x9c, 0xba, 0x74, 0x20, 0xd5, 0x1d, 0x20, + 0xf1 + }; + const unsigned char csr[32] = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + 0x45, 0x51, 0x23, 0x19, 0x50, 0xb7, 0x5f, 0xc4, + 0x40, 0x2d, 0xa1, 0x72, 0x2f, 0xc9, 0xba, 0xee + }; + secp256k1_ge key; + secp256k1_scalar msg; + secp256k1_scalar sr, ss; + secp256k1_scalar_set_int(&ss, 1); + secp256k1_scalar_set_int(&msg, 1); + secp256k1_scalar_negate(&msg, &msg); + secp256k1_scalar_set_b32(&sr, csr, NULL); + CHECK(secp256k1_eckey_pubkey_parse(&key, pubkey, 33)); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 1); + secp256k1_scalar_negate(&ss, &ss); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 1); + secp256k1_scalar_set_int(&ss, 3); + secp256k1_scalar_inverse_var(&ss, &ss); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sr, &ss, &key, &msg) == 0); + } + + /* Signature where s would be zero. */ + { + secp256k1_pubkey pubkey; + size_t siglen; + int32_t ecount; + unsigned char signature[72]; + static const unsigned char nonce[32] = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + }; + static const unsigned char nonce2[32] = { + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF, + 0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFE, + 0xBA,0xAE,0xDC,0xE6,0xAF,0x48,0xA0,0x3B, + 0xBF,0xD2,0x5E,0x8C,0xD0,0x36,0x41,0x40 + }; + const unsigned char key[32] = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + }; + unsigned char msg[32] = { + 0x86, 0x41, 0x99, 0x81, 0x06, 0x23, 0x44, 0x53, + 0xaa, 0x5f, 0x9d, 0x6a, 0x31, 0x78, 0xf4, 0xf7, + 0xb8, 0x12, 0xe0, 0x0b, 0x81, 0x7a, 0x77, 0x62, + 0x65, 0xdf, 0xdd, 0x31, 0xb9, 0x3e, 0x29, 0xa9, + }; + ecount = 0; + secp256k1_context_set_illegal_callback(ctx, counting_illegal_callback_fn, &ecount); + CHECK(secp256k1_ecdsa_sign(ctx, &sig, msg, key, precomputed_nonce_function, nonce) == 0); + CHECK(secp256k1_ecdsa_sign(ctx, &sig, msg, key, precomputed_nonce_function, nonce2) == 0); + msg[31] = 0xaa; + CHECK(secp256k1_ecdsa_sign(ctx, &sig, msg, key, precomputed_nonce_function, nonce) == 1); + CHECK(ecount == 0); + CHECK(secp256k1_ecdsa_sign(ctx, NULL, msg, key, precomputed_nonce_function, nonce2) == 0); + CHECK(ecount == 1); + CHECK(secp256k1_ecdsa_sign(ctx, &sig, NULL, key, precomputed_nonce_function, nonce2) == 0); + CHECK(ecount == 2); + CHECK(secp256k1_ecdsa_sign(ctx, &sig, msg, NULL, precomputed_nonce_function, nonce2) == 0); + CHECK(ecount == 3); + CHECK(secp256k1_ecdsa_sign(ctx, &sig, msg, key, precomputed_nonce_function, nonce2) == 1); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, key) == 1); + CHECK(secp256k1_ecdsa_verify(ctx, NULL, msg, &pubkey) == 0); + CHECK(ecount == 4); + CHECK(secp256k1_ecdsa_verify(ctx, &sig, NULL, &pubkey) == 0); + CHECK(ecount == 5); + CHECK(secp256k1_ecdsa_verify(ctx, &sig, msg, NULL) == 0); + CHECK(ecount == 6); + CHECK(secp256k1_ecdsa_verify(ctx, &sig, msg, &pubkey) == 1); + CHECK(ecount == 6); + CHECK(secp256k1_ec_pubkey_create(ctx, &pubkey, NULL) == 0); + CHECK(ecount == 7); + /* That pubkeyload fails via an ARGCHECK is a little odd but makes sense because pubkeys are an opaque data type. */ + CHECK(secp256k1_ecdsa_verify(ctx, &sig, msg, &pubkey) == 0); + CHECK(ecount == 8); + siglen = 72; + CHECK(secp256k1_ecdsa_signature_serialize_der(ctx, NULL, &siglen, &sig) == 0); + CHECK(ecount == 9); + CHECK(secp256k1_ecdsa_signature_serialize_der(ctx, signature, NULL, &sig) == 0); + CHECK(ecount == 10); + CHECK(secp256k1_ecdsa_signature_serialize_der(ctx, signature, &siglen, NULL) == 0); + CHECK(ecount == 11); + CHECK(secp256k1_ecdsa_signature_serialize_der(ctx, signature, &siglen, &sig) == 1); + CHECK(ecount == 11); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, NULL, signature, siglen) == 0); + CHECK(ecount == 12); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, NULL, siglen) == 0); + CHECK(ecount == 13); + CHECK(secp256k1_ecdsa_signature_parse_der(ctx, &sig, signature, siglen) == 1); + CHECK(ecount == 13); + siglen = 10; + /* Too little room for a signature does not fail via ARGCHECK. */ + CHECK(secp256k1_ecdsa_signature_serialize_der(ctx, signature, &siglen, &sig) == 0); + CHECK(ecount == 13); + ecount = 0; + CHECK(secp256k1_ecdsa_signature_normalize(ctx, NULL, NULL) == 0); + CHECK(ecount == 1); + CHECK(secp256k1_ecdsa_signature_serialize_compact(ctx, NULL, &sig) == 0); + CHECK(ecount == 2); + CHECK(secp256k1_ecdsa_signature_serialize_compact(ctx, signature, NULL) == 0); + CHECK(ecount == 3); + CHECK(secp256k1_ecdsa_signature_serialize_compact(ctx, signature, &sig) == 1); + CHECK(ecount == 3); + CHECK(secp256k1_ecdsa_signature_parse_compact(ctx, NULL, signature) == 0); + CHECK(ecount == 4); + CHECK(secp256k1_ecdsa_signature_parse_compact(ctx, &sig, NULL) == 0); + CHECK(ecount == 5); + CHECK(secp256k1_ecdsa_signature_parse_compact(ctx, &sig, signature) == 1); + CHECK(ecount == 5); + memset(signature, 255, 64); + CHECK(secp256k1_ecdsa_signature_parse_compact(ctx, &sig, signature) == 0); + CHECK(ecount == 5); + secp256k1_context_set_illegal_callback(ctx, NULL, NULL); + } + + /* Nonce function corner cases. */ + for (t = 0; t < 2; t++) { + static const unsigned char zero[32] = {0x00}; + int i; + unsigned char key[32]; + unsigned char msg[32]; + secp256k1_ecdsa_signature sig2; + secp256k1_scalar sr[512], ss; + const unsigned char *extra; + extra = t == 0 ? NULL : zero; + memset(msg, 0, 32); + msg[31] = 1; + /* High key results in signature failure. */ + memset(key, 0xFF, 32); + CHECK(secp256k1_ecdsa_sign(ctx, &sig, msg, key, NULL, extra) == 0); + CHECK(is_empty_signature(&sig)); + /* Zero key results in signature failure. */ + memset(key, 0, 32); + CHECK(secp256k1_ecdsa_sign(ctx, &sig, msg, key, NULL, extra) == 0); + CHECK(is_empty_signature(&sig)); + /* Nonce function failure results in signature failure. */ + key[31] = 1; + CHECK(secp256k1_ecdsa_sign(ctx, &sig, msg, key, nonce_function_test_fail, extra) == 0); + CHECK(is_empty_signature(&sig)); + /* The retry loop successfully makes its way to the first good value. */ + CHECK(secp256k1_ecdsa_sign(ctx, &sig, msg, key, nonce_function_test_retry, extra) == 1); + CHECK(!is_empty_signature(&sig)); + CHECK(secp256k1_ecdsa_sign(ctx, &sig2, msg, key, nonce_function_rfc6979, extra) == 1); + CHECK(!is_empty_signature(&sig2)); + CHECK(memcmp(&sig, &sig2, sizeof(sig)) == 0); + /* The default nonce function is deterministic. */ + CHECK(secp256k1_ecdsa_sign(ctx, &sig2, msg, key, NULL, extra) == 1); + CHECK(!is_empty_signature(&sig2)); + CHECK(memcmp(&sig, &sig2, sizeof(sig)) == 0); + /* The default nonce function changes output with different messages. */ + for(i = 0; i < 256; i++) { + int j; + msg[0] = i; + CHECK(secp256k1_ecdsa_sign(ctx, &sig2, msg, key, NULL, extra) == 1); + CHECK(!is_empty_signature(&sig2)); + secp256k1_ecdsa_signature_load(ctx, &sr[i], &ss, &sig2); + for (j = 0; j < i; j++) { + CHECK(!secp256k1_scalar_eq(&sr[i], &sr[j])); + } + } + msg[0] = 0; + msg[31] = 2; + /* The default nonce function changes output with different keys. */ + for(i = 256; i < 512; i++) { + int j; + key[0] = i - 256; + CHECK(secp256k1_ecdsa_sign(ctx, &sig2, msg, key, NULL, extra) == 1); + CHECK(!is_empty_signature(&sig2)); + secp256k1_ecdsa_signature_load(ctx, &sr[i], &ss, &sig2); + for (j = 0; j < i; j++) { + CHECK(!secp256k1_scalar_eq(&sr[i], &sr[j])); + } + } + key[0] = 0; + } + + { + /* Check that optional nonce arguments do not have equivalent effect. */ + const unsigned char zeros[32] = {0}; + unsigned char nonce[32]; + unsigned char nonce2[32]; + unsigned char nonce3[32]; + unsigned char nonce4[32]; + VG_UNDEF(nonce,32); + VG_UNDEF(nonce2,32); + VG_UNDEF(nonce3,32); + VG_UNDEF(nonce4,32); + CHECK(nonce_function_rfc6979(nonce, zeros, zeros, NULL, NULL, 0) == 1); + VG_CHECK(nonce,32); + CHECK(nonce_function_rfc6979(nonce2, zeros, zeros, zeros, NULL, 0) == 1); + VG_CHECK(nonce2,32); + CHECK(nonce_function_rfc6979(nonce3, zeros, zeros, NULL, (void *)zeros, 0) == 1); + VG_CHECK(nonce3,32); + CHECK(nonce_function_rfc6979(nonce4, zeros, zeros, zeros, (void *)zeros, 0) == 1); + VG_CHECK(nonce4,32); + CHECK(memcmp(nonce, nonce2, 32) != 0); + CHECK(memcmp(nonce, nonce3, 32) != 0); + CHECK(memcmp(nonce, nonce4, 32) != 0); + CHECK(memcmp(nonce2, nonce3, 32) != 0); + CHECK(memcmp(nonce2, nonce4, 32) != 0); + CHECK(memcmp(nonce3, nonce4, 32) != 0); + } + + + /* Privkey export where pubkey is the point at infinity. */ + { + unsigned char privkey[300]; + unsigned char seckey[32] = { + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, + 0xba, 0xae, 0xdc, 0xe6, 0xaf, 0x48, 0xa0, 0x3b, + 0xbf, 0xd2, 0x5e, 0x8c, 0xd0, 0x36, 0x41, 0x41, + }; + size_t outlen = 300; + CHECK(!ec_privkey_export_der(ctx, privkey, &outlen, seckey, 0)); + outlen = 300; + CHECK(!ec_privkey_export_der(ctx, privkey, &outlen, seckey, 1)); + } +} + +void run_ecdsa_edge_cases(void) { + test_ecdsa_edge_cases(); +} + +#ifdef ENABLE_OPENSSL_TESTS +EC_KEY *get_openssl_key(const unsigned char *key32) { + unsigned char privkey[300]; + size_t privkeylen; + const unsigned char* pbegin = privkey; + int compr = secp256k1_rand_bits(1); + EC_KEY *ec_key = EC_KEY_new_by_curve_name(NID_secp256k1); + CHECK(ec_privkey_export_der(ctx, privkey, &privkeylen, key32, compr)); + CHECK(d2i_ECPrivateKey(&ec_key, &pbegin, privkeylen)); + CHECK(EC_KEY_check_key(ec_key)); + return ec_key; +} + +void test_ecdsa_openssl(void) { + secp256k1_gej qj; + secp256k1_ge q; + secp256k1_scalar sigr, sigs; + secp256k1_scalar one; + secp256k1_scalar msg2; + secp256k1_scalar key, msg; + EC_KEY *ec_key; + unsigned int sigsize = 80; + size_t secp_sigsize = 80; + unsigned char message[32]; + unsigned char signature[80]; + unsigned char key32[32]; + secp256k1_rand256_test(message); + secp256k1_scalar_set_b32(&msg, message, NULL); + random_scalar_order_test(&key); + secp256k1_scalar_get_b32(key32, &key); + secp256k1_ecmult_gen(&ctx->ecmult_gen_ctx, &qj, &key); + secp256k1_ge_set_gej(&q, &qj); + ec_key = get_openssl_key(key32); + CHECK(ec_key != NULL); + CHECK(ECDSA_sign(0, message, sizeof(message), signature, &sigsize, ec_key)); + CHECK(secp256k1_ecdsa_sig_parse(&sigr, &sigs, signature, sigsize)); + CHECK(secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sigr, &sigs, &q, &msg)); + secp256k1_scalar_set_int(&one, 1); + secp256k1_scalar_add(&msg2, &msg, &one); + CHECK(!secp256k1_ecdsa_sig_verify(&ctx->ecmult_ctx, &sigr, &sigs, &q, &msg2)); + + random_sign(&sigr, &sigs, &key, &msg, NULL); + CHECK(secp256k1_ecdsa_sig_serialize(signature, &secp_sigsize, &sigr, &sigs)); + CHECK(ECDSA_verify(0, message, sizeof(message), signature, secp_sigsize, ec_key) == 1); + + EC_KEY_free(ec_key); +} + +void run_ecdsa_openssl(void) { + int i; + for (i = 0; i < 10*count; i++) { + test_ecdsa_openssl(); + } +} +#endif + +#ifdef ENABLE_MODULE_ECDH +# include "modules/ecdh/tests_impl.h" +#endif + +#ifdef ENABLE_MODULE_SCHNORR +# include "modules/schnorr/tests_impl.h" +#endif + +#ifdef ENABLE_MODULE_RECOVERY +# include "modules/recovery/tests_impl.h" +#endif + +int main(int argc, char **argv) { + unsigned char seed16[16] = {0}; + unsigned char run32[32] = {0}; + /* find iteration count */ + if (argc > 1) { + count = strtol(argv[1], NULL, 0); + } + + /* find random seed */ + if (argc > 2) { + int pos = 0; + const char* ch = argv[2]; + while (pos < 16 && ch[0] != 0 && ch[1] != 0) { + unsigned short sh; + if (sscanf(ch, "%2hx", &sh)) { + seed16[pos] = sh; + } else { + break; + } + ch += 2; + pos++; + } + } else { + FILE *frand = fopen("/dev/urandom", "r"); + if ((frand == NULL) || !fread(&seed16, sizeof(seed16), 1, frand)) { + uint64_t t = time(NULL) * (uint64_t)1337; + seed16[0] ^= t; + seed16[1] ^= t >> 8; + seed16[2] ^= t >> 16; + seed16[3] ^= t >> 24; + seed16[4] ^= t >> 32; + seed16[5] ^= t >> 40; + seed16[6] ^= t >> 48; + seed16[7] ^= t >> 56; + } + fclose(frand); + } + secp256k1_rand_seed(seed16); + + printf("test count = %i\n", count); + printf("random seed = %02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x\n", seed16[0], seed16[1], seed16[2], seed16[3], seed16[4], seed16[5], seed16[6], seed16[7], seed16[8], seed16[9], seed16[10], seed16[11], seed16[12], seed16[13], seed16[14], seed16[15]); + + /* initialize */ + run_context_tests(); + ctx = secp256k1_context_create(SECP256K1_CONTEXT_SIGN | SECP256K1_CONTEXT_VERIFY); + if (secp256k1_rand_bits(1)) { + secp256k1_rand256(run32); + CHECK(secp256k1_context_randomize(ctx, secp256k1_rand_bits(1) ? run32 : NULL)); + } + + run_rand_bits(); + run_rand_int(); + + run_sha256_tests(); + run_hmac_sha256_tests(); + run_rfc6979_hmac_sha256_tests(); + +#ifndef USE_NUM_NONE + /* num tests */ + run_num_smalltests(); +#endif + + /* scalar tests */ + run_scalar_tests(); + + /* field tests */ + run_field_inv(); + run_field_inv_var(); + run_field_inv_all_var(); + run_field_misc(); + run_field_convert(); + run_sqr(); + run_sqrt(); + + /* group tests */ + run_ge(); + run_group_decompress(); + + /* ecmult tests */ + run_wnaf(); + run_point_times_order(); + run_ecmult_chain(); + run_ecmult_constants(); + run_ecmult_gen_blind(); + run_ecmult_const_tests(); + run_ec_combine(); + + /* endomorphism tests */ +#ifdef USE_ENDOMORPHISM + run_endomorphism_tests(); +#endif + + /* EC point parser test */ + run_ec_pubkey_parse_test(); + + /* EC key edge cases */ + run_eckey_edge_case_test(); + +#ifdef ENABLE_MODULE_ECDH + /* ecdh tests */ + run_ecdh_tests(); +#endif + + /* ecdsa tests */ + run_random_pubkeys(); + run_ecdsa_der_parse(); + run_ecdsa_sign_verify(); + run_ecdsa_end_to_end(); + run_ecdsa_edge_cases(); +#ifdef ENABLE_OPENSSL_TESTS + run_ecdsa_openssl(); +#endif + +#ifdef ENABLE_MODULE_SCHNORR + /* Schnorr tests */ + run_schnorr_tests(); +#endif + +#ifdef ENABLE_MODULE_RECOVERY + /* ECDSA pubkey recovery tests */ + run_recovery_tests(); +#endif + + secp256k1_rand256(run32); + printf("random run = %02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x\n", run32[0], run32[1], run32[2], run32[3], run32[4], run32[5], run32[6], run32[7], run32[8], run32[9], run32[10], run32[11], run32[12], run32[13], run32[14], run32[15]); + + /* shutdown */ + secp256k1_context_destroy(ctx); + + printf("no problems found\n"); + return 0; +} diff --git a/src/secp256k1/src/util.h b/src/secp256k1/src/util.h new file mode 100644 index 00000000..4eef4ded --- /dev/null +++ b/src/secp256k1/src/util.h @@ -0,0 +1,110 @@ +/********************************************************************** + * Copyright (c) 2013, 2014 Pieter Wuille * + * Distributed under the MIT software license, see the accompanying * + * file COPYING or http://www.opensource.org/licenses/mit-license.php.* + **********************************************************************/ + +#ifndef _SECP256K1_UTIL_H_ +#define _SECP256K1_UTIL_H_ + +#if defined HAVE_CONFIG_H +#include "libsecp256k1-config.h" +#endif + +#include +#include +#include + +typedef struct { + void (*fn)(const char *text, void* data); + const void* data; +} secp256k1_callback; + +static SECP256K1_INLINE void secp256k1_callback_call(const secp256k1_callback * const cb, const char * const text) { + cb->fn(text, (void*)cb->data); +} + +#ifdef DETERMINISTIC +#define TEST_FAILURE(msg) do { \ + fprintf(stderr, "%s\n", msg); \ + abort(); \ +} while(0); +#else +#define TEST_FAILURE(msg) do { \ + fprintf(stderr, "%s:%d: %s\n", __FILE__, __LINE__, msg); \ + abort(); \ +} while(0) +#endif + +#ifdef HAVE_BUILTIN_EXPECT +#define EXPECT(x,c) __builtin_expect((x),(c)) +#else +#define EXPECT(x,c) (x) +#endif + +#ifdef DETERMINISTIC +#define CHECK(cond) do { \ + if (EXPECT(!(cond), 0)) { \ + TEST_FAILURE("test condition failed"); \ + } \ +} while(0) +#else +#define CHECK(cond) do { \ + if (EXPECT(!(cond), 0)) { \ + TEST_FAILURE("test condition failed: " #cond); \ + } \ +} while(0) +#endif + +/* Like assert(), but when VERIFY is defined, and side-effect safe. */ +#ifdef VERIFY +#define VERIFY_CHECK CHECK +#define VERIFY_SETUP(stmt) do { stmt; } while(0) +#else +#define VERIFY_CHECK(cond) do { (void)(cond); } while(0) +#define VERIFY_SETUP(stmt) +#endif + +static SECP256K1_INLINE void *checked_malloc(const secp256k1_callback* cb, size_t size) { + void *ret = malloc(size); + if (ret == NULL) { + secp256k1_callback_call(cb, "Out of memory"); + } + return ret; +} + +/* Macro for restrict, when available and not in a VERIFY build. */ +#if defined(SECP256K1_BUILD) && defined(VERIFY) +# define SECP256K1_RESTRICT +#else +# if (!defined(__STDC_VERSION__) || (__STDC_VERSION__ < 199901L) ) +# if SECP256K1_GNUC_PREREQ(3,0) +# define SECP256K1_RESTRICT __restrict__ +# elif (defined(_MSC_VER) && _MSC_VER >= 1400) +# define SECP256K1_RESTRICT __restrict +# else +# define SECP256K1_RESTRICT +# endif +# else +# define SECP256K1_RESTRICT restrict +# endif +#endif + +#if defined(_WIN32) +# define I64FORMAT "I64d" +# define I64uFORMAT "I64u" +#else +# define I64FORMAT "lld" +# define I64uFORMAT "llu" +#endif + +#if defined(HAVE___INT128) +# if defined(__GNUC__) +# define SECP256K1_GNUC_EXT __extension__ +# else +# define SECP256K1_GNUC_EXT +# endif +SECP256K1_GNUC_EXT typedef unsigned __int128 uint128_t; +#endif + +#endif diff --git a/src/storage/addrman.cpp b/src/storage/addrman.cpp new file mode 100644 index 00000000..2afac08c --- /dev/null +++ b/src/storage/addrman.cpp @@ -0,0 +1,573 @@ +// Copyright (c) 2012 Pieter Wuille +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "storage/addrman.h" + +#include "structs/hash.h" +#include "utils/serialize.h" +#include "utils/streams.h" + +using namespace std; + +int CAddrInfo::GetTriedBucket(const std::vector& nKey) const +{ + CDataStream ss1(SER_GETHASH, 0); + std::vector vchKey = GetKey(); + ss1 << nKey << vchKey; + uint64_t hash1 = Hash(ss1.begin(), ss1.end()).GetLow64(); + + CDataStream ss2(SER_GETHASH, 0); + std::vector vchGroupKey = GetGroup(); + ss2 << nKey << vchGroupKey << (hash1 % ADDRMAN_TRIED_BUCKETS_PER_GROUP); + uint64_t hash2 = Hash(ss2.begin(), ss2.end()).GetLow64(); + return hash2 % ADDRMAN_TRIED_BUCKET_COUNT; +} + +int CAddrInfo::GetNewBucket(const std::vector& nKey, const CNetAddr& src) const +{ + CDataStream ss1(SER_GETHASH, 0); + std::vector vchGroupKey = GetGroup(); + std::vector vchSourceGroupKey = src.GetGroup(); + ss1 << nKey << vchGroupKey << vchSourceGroupKey; + uint64_t hash1 = Hash(ss1.begin(), ss1.end()).GetLow64(); + + CDataStream ss2(SER_GETHASH, 0); + ss2 << nKey << vchSourceGroupKey << (hash1 % ADDRMAN_NEW_BUCKETS_PER_SOURCE_GROUP); + uint64_t hash2 = Hash(ss2.begin(), ss2.end()).GetLow64(); + return hash2 % ADDRMAN_NEW_BUCKET_COUNT; +} + +bool CAddrInfo::IsTerrible(int64_t nNow) const +{ + if (nLastTry && nLastTry >= nNow - 60) // never remove things tried in the last minute + return false; + + if (nTime > nNow + 10 * 60) // came in a flying DeLorean + return true; + + if (nTime == 0 || nNow - nTime > ADDRMAN_HORIZON_DAYS * 24 * 60 * 60) // not seen in recent history + return true; + + if (nLastSuccess == 0 && nAttempts >= ADDRMAN_RETRIES) // tried N times and never a success + return true; + + if (nNow - nLastSuccess > ADDRMAN_MIN_FAIL_DAYS * 24 * 60 * 60 && nAttempts >= ADDRMAN_MAX_FAILURES) // N successive failures in the last week + return true; + + return false; +} + +double CAddrInfo::GetChance(int64_t nNow) const +{ + double fChance = 1.0; + + int64_t nSinceLastSeen = nNow - nTime; + int64_t nSinceLastTry = nNow - nLastTry; + + if (nSinceLastSeen < 0) + nSinceLastSeen = 0; + if (nSinceLastTry < 0) + nSinceLastTry = 0; + + fChance *= 600.0 / (600.0 + nSinceLastSeen); + + // deprioritize very recent attempts away + if (nSinceLastTry < 60 * 10) + fChance *= 0.01; + + // deprioritize 50% after each failed attempt + for (int n = 0; n < nAttempts; n++) + fChance /= 1.5; + +/* MCHN START */ + // Without this, if all nodes are down, fChance goes to 0 + // and it takes several seconds of 100% CPU to choose from 3 addresses. + if(fChance<0.00001) + { + fChance=0.00001; + } +/* MCHN END */ + + return fChance; +} + +CAddrInfo* CAddrMan::Find(const CNetAddr& addr, int* pnId) +{ + std::map::iterator it = mapAddr.find(addr); + if (it == mapAddr.end()) + return NULL; + if (pnId) + *pnId = (*it).second; + std::map::iterator it2 = mapInfo.find((*it).second); + if (it2 != mapInfo.end()) + return &(*it2).second; + return NULL; +} + +CAddrInfo* CAddrMan::FindWithPort(const CService& addr, int* pnId) +{ + std::map::iterator it = mapAddrWithPort.find(addr); + if (it == mapAddrWithPort.end()) + return NULL; + if (pnId) + *pnId = (*it).second; + std::map::iterator it2 = mapInfo.find((*it).second); + if (it2 != mapInfo.end()) + return &(*it2).second; + return NULL; +} + +CAddrInfo* CAddrMan::Create(const CAddress& addr, const CNetAddr& addrSource, int* pnId) +{ + int nId = nIdCount++; + mapInfo[nId] = CAddrInfo(addr, addrSource); + mapAddr[addr] = nId; + mapAddrWithPort[addr] = nId; + mapInfo[nId].nRandomPos = vRandom.size(); + vRandom.push_back(nId); + if (pnId) + *pnId = nId; + return &mapInfo[nId]; +} + +void CAddrMan::SwapRandom(unsigned int nRndPos1, unsigned int nRndPos2) +{ + if (nRndPos1 == nRndPos2) + return; + + assert(nRndPos1 < vRandom.size() && nRndPos2 < vRandom.size()); + + int nId1 = vRandom[nRndPos1]; + int nId2 = vRandom[nRndPos2]; + + assert(mapInfo.count(nId1) == 1); + assert(mapInfo.count(nId2) == 1); + + mapInfo[nId1].nRandomPos = nRndPos2; + mapInfo[nId2].nRandomPos = nRndPos1; + + vRandom[nRndPos1] = nId2; + vRandom[nRndPos2] = nId1; +} + +int CAddrMan::SelectTried(int nKBucket) +{ + std::vector& vTried = vvTried[nKBucket]; + + // randomly shuffle the first few elements (using the entire list) + // find the least recently tried among them + int64_t nOldest = -1; + int nOldestPos = -1; + for (unsigned int i = 0; i < ADDRMAN_TRIED_ENTRIES_INSPECT_ON_EVICT && i < vTried.size(); i++) { + int nPos = GetRandInt(vTried.size() - i) + i; + int nTemp = vTried[nPos]; + vTried[nPos] = vTried[i]; + vTried[i] = nTemp; + assert(nOldest == -1 || mapInfo.count(nTemp) == 1); + if (nOldest == -1 || mapInfo[nTemp].nLastSuccess < mapInfo[nOldest].nLastSuccess) { + nOldest = nTemp; + nOldestPos = nPos; + } + } + + return nOldestPos; +} + +int CAddrMan::ShrinkNew(int nUBucket) +{ + assert(nUBucket >= 0 && (unsigned int)nUBucket < vvNew.size()); + std::set& vNew = vvNew[nUBucket]; + + // first look for deletable items + for (std::set::iterator it = vNew.begin(); it != vNew.end(); it++) { + assert(mapInfo.count(*it)); + CAddrInfo& info = mapInfo[*it]; + if (info.IsTerrible()) { + if (--info.nRefCount == 0) { + SwapRandom(info.nRandomPos, vRandom.size() - 1); + vRandom.pop_back(); + mapAddr.erase(info); + mapAddrWithPort.erase(info); + mapInfo.erase(*it); + nNew--; + } + vNew.erase(it); + return 0; + } + } + + // otherwise, select four randomly, and pick the oldest of those to replace + int n[4] = {GetRandInt(vNew.size()), GetRandInt(vNew.size()), GetRandInt(vNew.size()), GetRandInt(vNew.size())}; + int nI = 0; + int nOldest = -1; + for (std::set::iterator it = vNew.begin(); it != vNew.end(); it++) { + if (nI == n[0] || nI == n[1] || nI == n[2] || nI == n[3]) { + assert(nOldest == -1 || mapInfo.count(*it) == 1); + if (nOldest == -1 || mapInfo[*it].nTime < mapInfo[nOldest].nTime) + nOldest = *it; + } + nI++; + } + assert(mapInfo.count(nOldest) == 1); + CAddrInfo& info = mapInfo[nOldest]; + if (--info.nRefCount == 0) { + SwapRandom(info.nRandomPos, vRandom.size() - 1); + vRandom.pop_back(); + mapAddr.erase(info); + mapAddrWithPort.erase(info); + mapInfo.erase(nOldest); + nNew--; + } + vNew.erase(nOldest); + + return 1; +} + +void CAddrMan::MakeTried(CAddrInfo& info, int nId, int nOrigin) +{ + assert(vvNew[nOrigin].count(nId) == 1); + + // remove the entry from all new buckets + for (std::vector >::iterator it = vvNew.begin(); it != vvNew.end(); it++) { + if ((*it).erase(nId)) + info.nRefCount--; + } + nNew--; + + assert(info.nRefCount == 0); + + // which tried bucket to move the entry to + int nKBucket = info.GetTriedBucket(nKey); + std::vector& vTried = vvTried[nKBucket]; + + // first check whether there is place to just add it + if (vTried.size() < ADDRMAN_TRIED_BUCKET_SIZE) { + vTried.push_back(nId); + nTried++; + info.fInTried = true; + return; + } + + // otherwise, find an item to evict + int nPos = SelectTried(nKBucket); + + // find which new bucket it belongs to + assert(mapInfo.count(vTried[nPos]) == 1); + int nUBucket = mapInfo[vTried[nPos]].GetNewBucket(nKey); + std::set& vNew = vvNew[nUBucket]; + + // remove the to-be-replaced tried entry from the tried set + CAddrInfo& infoOld = mapInfo[vTried[nPos]]; + infoOld.fInTried = false; + infoOld.nRefCount = 1; + // do not update nTried, as we are going to move something else there immediately + + // check whether there is place in that one, + if (vNew.size() < ADDRMAN_NEW_BUCKET_SIZE) { + // if so, move it back there + vNew.insert(vTried[nPos]); + } else { + // otherwise, move it to the new bucket nId came from (there is certainly place there) + vvNew[nOrigin].insert(vTried[nPos]); + } + nNew++; + + vTried[nPos] = nId; + // we just overwrote an entry in vTried; no need to update nTried + info.fInTried = true; + return; +} + +void CAddrMan::Good_(const CService& addr, int64_t nTime) +{ + int nId; +/* MCHN START */ +// CAddrInfo* pinfo = Find(addr, &nId); + CAddrInfo* pinfo = FindWithPort(addr, &nId); +/* MCHN END */ + // if not found, bail out + if (!pinfo) + return; + + CAddrInfo& info = *pinfo; + + // check whether we are talking about the exact same CService (including same port) + if (info != addr) + return; + + // update info + info.nLastSuccess = nTime; + info.nLastTry = nTime; + info.nTime = nTime; + info.nAttempts = 0; + + // if it is already in the tried set, don't do anything else + if (info.fInTried) + return; + + // find a bucket it is in now + int nRnd = GetRandInt(vvNew.size()); + int nUBucket = -1; + for (unsigned int n = 0; n < vvNew.size(); n++) { + int nB = (n + nRnd) % vvNew.size(); + std::set& vNew = vvNew[nB]; + if (vNew.count(nId)) { + nUBucket = nB; + break; + } + } + + // if no bucket is found, something bad happened; + // TODO: maybe re-add the node, but for now, just bail out + if (nUBucket == -1) + return; + + LogPrint("addrman", "Moving %s to tried\n", addr.ToString()); + + // move nId to the tried tables + MakeTried(info, nId, nUBucket); +} + +bool CAddrMan::Add_(const CAddress& addr, const CNetAddr& source, int64_t nTimePenalty) +{ + if (!addr.IsRoutable()) + return false; + + bool fNew = false; + int nId; + CAddrInfo* pinfo = FindWithPort(addr, &nId); + + if (pinfo) { + // periodically update nTime + bool fCurrentlyOnline = (GetAdjustedTime() - addr.nTime < 24 * 60 * 60); + int64_t nUpdateInterval = (fCurrentlyOnline ? 60 * 60 : 24 * 60 * 60); + if (addr.nTime && (!pinfo->nTime || pinfo->nTime < addr.nTime - nUpdateInterval - nTimePenalty)) + pinfo->nTime = max((int64_t)0, addr.nTime - nTimePenalty); + + // add services + pinfo->nServices |= addr.nServices; + + // do not update if no new information is present + if (!addr.nTime || (pinfo->nTime && addr.nTime <= pinfo->nTime)) + return false; + + // do not update if the entry was already in the "tried" table + if (pinfo->fInTried) + return false; + + // do not update if the max reference count is reached + if (pinfo->nRefCount == ADDRMAN_NEW_BUCKETS_PER_ADDRESS) + return false; + + // stochastic test: previous nRefCount == N: 2^N times harder to increase it + int nFactor = 1; + for (int n = 0; n < pinfo->nRefCount; n++) + nFactor *= 2; + if (nFactor > 1 && (GetRandInt(nFactor) != 0)) + return false; + } else { + pinfo = Create(addr, source, &nId); + pinfo->nTime = max((int64_t)0, (int64_t)pinfo->nTime - nTimePenalty); + nNew++; + fNew = true; + } + + int nUBucket = pinfo->GetNewBucket(nKey, source); + std::set& vNew = vvNew[nUBucket]; + if (!vNew.count(nId)) { + pinfo->nRefCount++; + if (vNew.size() == ADDRMAN_NEW_BUCKET_SIZE) + ShrinkNew(nUBucket); + vvNew[nUBucket].insert(nId); + } + return fNew; +} + +void CAddrMan::Attempt_(const CService& addr, int64_t nTime) +{ +// CAddrInfo* pinfo = Find(addr); + CAddrInfo* pinfo = FindWithPort(addr); + + // if not found, bail out + if (!pinfo) + return; + + CAddrInfo& info = *pinfo; + + // check whether we are talking about the exact same CService (including same port) + if (info != addr) + return; + + // update info + info.nLastTry = nTime; + info.nAttempts++; +} + +CAddress CAddrMan::Select_(int nUnkBias) +{ + if (size() == 0) + return CAddress(); + + double nCorTried = sqrt(nTried) * (100.0 - nUnkBias); + double nCorNew = sqrt(nNew) * nUnkBias; + if ((nCorTried + nCorNew) * GetRandInt(1 << 30) / (1 << 30) < nCorTried) { + // use a tried node + double fChanceFactor = 1.0; + while (1) { + int nKBucket = GetRandInt(vvTried.size()); + std::vector& vTried = vvTried[nKBucket]; + if (vTried.size() == 0) + continue; + int nPos = GetRandInt(vTried.size()); + assert(mapInfo.count(vTried[nPos]) == 1); + CAddrInfo& info = mapInfo[vTried[nPos]]; + if (GetRandInt(1 << 30) < fChanceFactor * info.GetChance() * (1 << 30)) + return info; + fChanceFactor *= 1.2; + } + } else { + // use a new node + double fChanceFactor = 1.0; + while (1) { + int nUBucket = GetRandInt(vvNew.size()); + std::set& vNew = vvNew[nUBucket]; + if (vNew.size() == 0) + continue; + int nPos = GetRandInt(vNew.size()); + std::set::iterator it = vNew.begin(); + while (nPos--) + it++; + assert(mapInfo.count(*it) == 1); + CAddrInfo& info = mapInfo[*it]; + if (GetRandInt(1 << 30) < fChanceFactor * info.GetChance() * (1 << 30)) + return info; + fChanceFactor *= 1.2; + } + } +} + +#ifdef DEBUG_ADDRMAN +int CAddrMan::Check_() +{ + std::set setTried; + std::map mapNew; + + if (vRandom.size() != nTried + nNew) + return -7; + + for (std::map::iterator it = mapInfo.begin(); it != mapInfo.end(); it++) { + int n = (*it).first; + CAddrInfo& info = (*it).second; + if (info.fInTried) { + if (!info.nLastSuccess) + return -1; + if (info.nRefCount) + return -2; + setTried.insert(n); + } else { + if (info.nRefCount < 0 || info.nRefCount > ADDRMAN_NEW_BUCKETS_PER_ADDRESS) + return -3; + if (!info.nRefCount) + return -4; + mapNew[n] = info.nRefCount; + } + if (mapAddr[info] != n) + return -5; + if (info.nRandomPos < 0 || info.nRandomPos >= vRandom.size() || vRandom[info.nRandomPos] != n) + return -14; + if (info.nLastTry < 0) + return -6; + if (info.nLastSuccess < 0) + return -8; + } + + if (setTried.size() != nTried) + return -9; + if (mapNew.size() != nNew) + return -10; + + for (int n = 0; n < vvTried.size(); n++) { + std::vector& vTried = vvTried[n]; + for (std::vector::iterator it = vTried.begin(); it != vTried.end(); it++) { + if (!setTried.count(*it)) + return -11; + setTried.erase(*it); + } + } + + for (int n = 0; n < vvNew.size(); n++) { + std::set& vNew = vvNew[n]; + for (std::set::iterator it = vNew.begin(); it != vNew.end(); it++) { + if (!mapNew.count(*it)) + return -12; + if (--mapNew[*it] == 0) + mapNew.erase(*it); + } + } + + if (setTried.size()) + return -13; + if (mapNew.size()) + return -15; + + return 0; +} +#endif + +void CAddrMan::GetAddr_(std::vector& vAddr) +{ + unsigned int nNodes = ADDRMAN_GETADDR_MAX_PCT * vRandom.size() / 100; + +/* MCHN START */ + bool addTerrible=false; + if(nNodes < ADDRMAN_GETADDR_MAX_PCT * ADDRMAN_GETADDR_MAX / 100) + { + addTerrible=true; + nNodes=ADDRMAN_GETADDR_MAX_PCT * ADDRMAN_GETADDR_MAX / 100; + if(nNodes > vRandom.size()) + { + nNodes=vRandom.size(); + } + } +/* MCHN END */ + + if (nNodes > ADDRMAN_GETADDR_MAX) + nNodes = ADDRMAN_GETADDR_MAX; + + + // gather a list of random nodes, skipping those of low quality + for (unsigned int n = 0; n < vRandom.size(); n++) { + if (vAddr.size() >= nNodes) + break; + + int nRndPos = GetRandInt(vRandom.size() - n) + n; + SwapRandom(n, nRndPos); + assert(mapInfo.count(vRandom[n]) == 1); + + const CAddrInfo& ai = mapInfo[vRandom[n]]; + if (addTerrible || !ai.IsTerrible()) // MCHN + vAddr.push_back(ai); + } +} + +void CAddrMan::Connected_(const CService& addr, int64_t nTime) +{ + CAddrInfo* pinfo = Find(addr); + + // if not found, bail out + if (!pinfo) + return; + + CAddrInfo& info = *pinfo; + + // check whether we are talking about the exact same CService (including same port) + if (info != addr) + return; + + // update info + int64_t nUpdateInterval = 20 * 60; + if (nTime - info.nTime > nUpdateInterval) + info.nTime = nTime; +} diff --git a/src/storage/addrman.h b/src/storage/addrman.h new file mode 100644 index 00000000..54cf3715 --- /dev/null +++ b/src/storage/addrman.h @@ -0,0 +1,524 @@ +// Copyright (c) 2012 Pieter Wuille +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_ADDRMAN_H +#define BITCOIN_ADDRMAN_H + +#include "net/netbase.h" +#include "protocol/netprotocol.h" +#include "utils/random.h" +#include "utils/sync.h" +#include "utils/timedata.h" +#include "utils/util.h" + +#include +#include +#include +#include + +/** + * Extended statistics about a CAddress + */ +class CAddrInfo : public CAddress +{ +private: + //! where knowledge about this address first came from + CNetAddr source; + + //! last successful connection by us + int64_t nLastSuccess; + + //! last try whatsoever by us: + // int64_t CAddress::nLastTry + + //! connection attempts since last successful attempt + int nAttempts; + + //! reference count in new sets (memory only) + int nRefCount; + + //! in tried set? (memory only) + bool fInTried; + + //! position in vRandom + int nRandomPos; + + friend class CAddrMan; + +public: + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(*(CAddress*)this); + READWRITE(source); + READWRITE(nLastSuccess); + READWRITE(nAttempts); + } + + void Init() + { + nLastSuccess = 0; + nLastTry = 0; + nAttempts = 0; + nRefCount = 0; + fInTried = false; + nRandomPos = -1; + } + + CAddrInfo(const CAddress &addrIn, const CNetAddr &addrSource) : CAddress(addrIn), source(addrSource) + { + Init(); + } + + CAddrInfo() : CAddress(), source() + { + Init(); + } + + //! Calculate in which "tried" bucket this entry belongs + int GetTriedBucket(const std::vector &nKey) const; + + //! Calculate in which "new" bucket this entry belongs, given a certain source + int GetNewBucket(const std::vector &nKey, const CNetAddr& src) const; + + //! Calculate in which "new" bucket this entry belongs, using its default source + int GetNewBucket(const std::vector &nKey) const + { + return GetNewBucket(nKey, source); + } + + //! Determine whether the statistics about this entry are bad enough so that it can just be deleted + bool IsTerrible(int64_t nNow = GetAdjustedTime()) const; + + //! Calculate the relative chance this entry should be given when selecting nodes to connect to + double GetChance(int64_t nNow = GetAdjustedTime()) const; + +}; + +/** Stochastic address manager + * + * Design goals: + * * Keep the address tables in-memory, and asynchronously dump the entire to able in peers.dat. + * * Make sure no (localized) attacker can fill the entire table with his nodes/addresses. + * + * To that end: + * * Addresses are organized into buckets. + * * Address that have not yet been tried go into 256 "new" buckets. + * * Based on the address range (/16 for IPv4) of source of the information, 32 buckets are selected at random + * * The actual bucket is chosen from one of these, based on the range the address itself is located. + * * One single address can occur in up to 4 different buckets, to increase selection chances for addresses that + * are seen frequently. The chance for increasing this multiplicity decreases exponentially. + * * When adding a new address to a full bucket, a randomly chosen entry (with a bias favoring less recently seen + * ones) is removed from it first. + * * Addresses of nodes that are known to be accessible go into 64 "tried" buckets. + * * Each address range selects at random 4 of these buckets. + * * The actual bucket is chosen from one of these, based on the full address. + * * When adding a new good address to a full bucket, a randomly chosen entry (with a bias favoring less recently + * tried ones) is evicted from it, back to the "new" buckets. + * * Bucket selection is based on cryptographic hashing, using a randomly-generated 256-bit key, which should not + * be observable by adversaries. + * * Several indexes are kept for high performance. Defining DEBUG_ADDRMAN will introduce frequent (and expensive) + * consistency checks for the entire data structure. + */ + +//! total number of buckets for tried addresses +#define ADDRMAN_TRIED_BUCKET_COUNT 64 + +//! maximum allowed number of entries in buckets for tried addresses +#define ADDRMAN_TRIED_BUCKET_SIZE 64 + +//! total number of buckets for new addresses +#define ADDRMAN_NEW_BUCKET_COUNT 256 + +//! maximum allowed number of entries in buckets for new addresses +#define ADDRMAN_NEW_BUCKET_SIZE 64 + +//! over how many buckets entries with tried addresses from a single group (/16 for IPv4) are spread +#define ADDRMAN_TRIED_BUCKETS_PER_GROUP 4 + +//! over how many buckets entries with new addresses originating from a single group are spread +#define ADDRMAN_NEW_BUCKETS_PER_SOURCE_GROUP 32 + +//! in how many buckets for entries with new addresses a single address may occur +#define ADDRMAN_NEW_BUCKETS_PER_ADDRESS 4 + +//! how many entries in a bucket with tried addresses are inspected, when selecting one to replace +#define ADDRMAN_TRIED_ENTRIES_INSPECT_ON_EVICT 4 + +//! how old addresses can maximally be +#define ADDRMAN_HORIZON_DAYS 30 + +//! after how many failed attempts we give up on a new node +#define ADDRMAN_RETRIES 3 + +//! how many successive failures are allowed ... +#define ADDRMAN_MAX_FAILURES 10 + +//! ... in at least this many days +#define ADDRMAN_MIN_FAIL_DAYS 7 + +//! the maximum percentage of nodes to return in a getaddr call +#define ADDRMAN_GETADDR_MAX_PCT 23 + +//! the maximum number of nodes to return in a getaddr call +#define ADDRMAN_GETADDR_MAX 2500 + +/** + * Stochastical (IP) address manager + */ +class CAddrMan +{ +private: + //! critical section to protect the inner data structures + mutable CCriticalSection cs; + + //! secret key to randomize bucket select with + std::vector nKey; + + //! last used nId + int nIdCount; + + //! table with information about all nIds + std::map mapInfo; + + //! find an nId based on its network address + std::map mapAddr; + std::map mapAddrWithPort; + + //! randomly-ordered vector of all nIds + std::vector vRandom; + + // number of "tried" entries + int nTried; + + //! list of "tried" buckets + std::vector > vvTried; + + //! number of (unique) "new" entries + int nNew; + + //! list of "new" buckets + std::vector > vvNew; + +protected: + + //! Find an entry. + CAddrInfo* Find(const CNetAddr& addr, int *pnId = NULL); +/* MCHN START */ + CAddrInfo* FindWithPort(const CService& addr, int *pnId = NULL); +/* MCHN END */ + //! find an entry, creating it if necessary. + //! nTime and nServices of the found node are updated, if necessary. + CAddrInfo* Create(const CAddress &addr, const CNetAddr &addrSource, int *pnId = NULL); + + //! Swap two elements in vRandom. + void SwapRandom(unsigned int nRandomPos1, unsigned int nRandomPos2); + + //! Return position in given bucket to replace. + int SelectTried(int nKBucket); + + //! Remove an element from a "new" bucket. + //! This is the only place where actual deletions occur. + //! Elements are never deleted while in the "tried" table, only possibly evicted back to the "new" table. + int ShrinkNew(int nUBucket); + + //! Move an entry from the "new" table(s) to the "tried" table + //! @pre vvUnkown[nOrigin].count(nId) != 0 + void MakeTried(CAddrInfo& info, int nId, int nOrigin); + + //! Mark an entry "good", possibly moving it from "new" to "tried". + void Good_(const CService &addr, int64_t nTime); + + //! Add an entry to the "new" table. + bool Add_(const CAddress &addr, const CNetAddr& source, int64_t nTimePenalty); + + //! Mark an entry as attempted to connect. + void Attempt_(const CService &addr, int64_t nTime); + + //! Select an address to connect to. + //! nUnkBias determines how much to favor new addresses over tried ones (min=0, max=100) + CAddress Select_(int nUnkBias); + +#ifdef DEBUG_ADDRMAN + //! Perform consistency check. Returns an error code or zero. + int Check_(); +#endif + + //! Select several addresses at once. + void GetAddr_(std::vector &vAddr); + + //! Mark an entry as currently-connected-to. + void Connected_(const CService &addr, int64_t nTime); + +public: + /** + * serialized format: + * * version byte (currently 0) + * * nKey + * * nNew + * * nTried + * * number of "new" buckets + * * all nNew addrinfos in vvNew + * * all nTried addrinfos in vvTried + * * for each bucket: + * * number of elements + * * for each element: index + * + * Notice that vvTried, mapAddr and vVector are never encoded explicitly; + * they are instead reconstructed from the other information. + * + * vvNew is serialized, but only used if ADDRMAN_UNKOWN_BUCKET_COUNT didn't change, + * otherwise it is reconstructed as well. + * + * This format is more complex, but significantly smaller (at most 1.5 MiB), and supports + * changes to the ADDRMAN_ parameters without breaking the on-disk structure. + * + * We don't use ADD_SERIALIZE_METHODS since the serialization and deserialization code has + * very little in common. + * + */ + template + void Serialize(Stream &s, int nType, int nVersionDummy) const + { + LOCK(cs); + + unsigned char nVersion = 0; + s << nVersion; + s << nKey; + s << nNew; + s << nTried; + + int nUBuckets = ADDRMAN_NEW_BUCKET_COUNT; + s << nUBuckets; + std::map mapUnkIds; + int nIds = 0; + for (std::map::const_iterator it = mapInfo.begin(); it != mapInfo.end(); it++) { + if (nIds == nNew) break; // this means nNew was wrong, oh ow + mapUnkIds[(*it).first] = nIds; + const CAddrInfo &info = (*it).second; + if (info.nRefCount) { + s << info; + nIds++; + } + } + nIds = 0; + for (std::map::const_iterator it = mapInfo.begin(); it != mapInfo.end(); it++) { + if (nIds == nTried) break; // this means nTried was wrong, oh ow + const CAddrInfo &info = (*it).second; + if (info.fInTried) { + s << info; + nIds++; + } + } + for (std::vector >::const_iterator it = vvNew.begin(); it != vvNew.end(); it++) { + const std::set &vNew = (*it); + int nSize = vNew.size(); + s << nSize; + for (std::set::const_iterator it2 = vNew.begin(); it2 != vNew.end(); it2++) { + int nIndex = mapUnkIds[*it2]; + s << nIndex; + } + } + } + + template + void Unserialize(Stream& s, int nType, int nVersionDummy) + { + LOCK(cs); + + unsigned char nVersion; + s >> nVersion; + s >> nKey; + s >> nNew; + s >> nTried; + + int nUBuckets = 0; + s >> nUBuckets; + nIdCount = 0; + mapInfo.clear(); + mapAddr.clear(); + mapAddrWithPort.clear(); + vRandom.clear(); + vvTried = std::vector >(ADDRMAN_TRIED_BUCKET_COUNT, std::vector(0)); + vvNew = std::vector >(ADDRMAN_NEW_BUCKET_COUNT, std::set()); + for (int n = 0; n < nNew; n++) { + CAddrInfo &info = mapInfo[n]; + s >> info; + mapAddr[info] = n; + mapAddrWithPort[info] = n; + info.nRandomPos = vRandom.size(); + vRandom.push_back(n); + if (nUBuckets != ADDRMAN_NEW_BUCKET_COUNT) { + vvNew[info.GetNewBucket(nKey)].insert(n); + info.nRefCount++; + } + } + nIdCount = nNew; + int nLost = 0; + for (int n = 0; n < nTried; n++) { + CAddrInfo info; + s >> info; + std::vector &vTried = vvTried[info.GetTriedBucket(nKey)]; + if (vTried.size() < ADDRMAN_TRIED_BUCKET_SIZE) { + info.nRandomPos = vRandom.size(); + info.fInTried = true; + vRandom.push_back(nIdCount); + mapInfo[nIdCount] = info; + mapAddr[info] = nIdCount; + mapAddrWithPort[info] = nIdCount; + vTried.push_back(nIdCount); + nIdCount++; + } else { + nLost++; + } + } + nTried -= nLost; + for (int b = 0; b < nUBuckets; b++) { + std::set &vNew = vvNew[b]; + int nSize = 0; + s >> nSize; + for (int n = 0; n < nSize; n++) { + int nIndex = 0; + s >> nIndex; + CAddrInfo &info = mapInfo[nIndex]; + if (nUBuckets == ADDRMAN_NEW_BUCKET_COUNT && info.nRefCount < ADDRMAN_NEW_BUCKETS_PER_ADDRESS) { + info.nRefCount++; + vNew.insert(nIndex); + } + } + } + } + + unsigned int GetSerializeSize(int nType, int nVersion) const + { + return (CSizeComputer(nType, nVersion) << *this).size(); + } + + CAddrMan() : vRandom(0), vvTried(ADDRMAN_TRIED_BUCKET_COUNT, std::vector(0)), vvNew(ADDRMAN_NEW_BUCKET_COUNT, std::set()) + { + nKey.resize(32); + GetRandBytes(&nKey[0], 32); + + nIdCount = 0; + nTried = 0; + nNew = 0; + } + + //! Return the number of (unique) addresses in all tables. + int size() + { + return vRandom.size(); + } + + //! Consistency check + void Check() + { +#ifdef DEBUG_ADDRMAN + { + LOCK(cs); + int err; + if ((err=Check_())) + LogPrintf("ADDRMAN CONSISTENCY CHECK FAILED!!! err=%i\n", err); + } +#endif + } + + //! Add a single address. + bool Add(const CAddress &addr, const CNetAddr& source, int64_t nTimePenalty = 0) + { + bool fRet = false; + { + LOCK(cs); + Check(); + fRet |= Add_(addr, source, nTimePenalty); + Check(); + } + if (fRet) + LogPrint("addrman", "Added %s from %s: %i tried, %i new\n", addr.ToStringIPPort(), source.ToString(), nTried, nNew); + return fRet; + } + + //! Add multiple addresses. + bool Add(const std::vector &vAddr, const CNetAddr& source, int64_t nTimePenalty = 0) + { + int nAdd = 0; + { + LOCK(cs); + Check(); + for (std::vector::const_iterator it = vAddr.begin(); it != vAddr.end(); it++) + nAdd += Add_(*it, source, nTimePenalty) ? 1 : 0; + Check(); + } + if (nAdd) + LogPrint("addrman", "Added %i addresses from %s: %i tried, %i new\n", nAdd, source.ToString(), nTried, nNew); + return nAdd > 0; + } + + //! Mark an entry as accessible. + void Good(const CService &addr, int64_t nTime = GetAdjustedTime()) + { + { + LOCK(cs); + Check(); + Good_(addr, nTime); + Check(); + } + } + + //! Mark an entry as connection attempted to. + void Attempt(const CService &addr, int64_t nTime = GetAdjustedTime()) + { + { + LOCK(cs); + Check(); + Attempt_(addr, nTime); + Check(); + } + } + + /** + * Choose an address to connect to. + * nUnkBias determines how much "new" entries are favored over "tried" ones (0-100). + */ + CAddress Select(int nUnkBias = 50) + { + CAddress addrRet; + { + LOCK(cs); + Check(); + addrRet = Select_(nUnkBias); + Check(); + } + return addrRet; + } + + //! Return a bunch of addresses, selected at random. + std::vector GetAddr() + { + Check(); + std::vector vAddr; + { + LOCK(cs); + GetAddr_(vAddr); + } + Check(); + return vAddr; + } + + //! Mark an entry as currently-connected-to. + void Connected(const CService &addr, int64_t nTime = GetAdjustedTime()) + { + { + LOCK(cs); + Check(); + Connected_(addr, nTime); + Check(); + } + } +}; + +#endif // BITCOIN_ADDRMAN_H diff --git a/src/storage/coins.cpp b/src/storage/coins.cpp new file mode 100644 index 00000000..a5dbb7b0 --- /dev/null +++ b/src/storage/coins.cpp @@ -0,0 +1,282 @@ +// Copyright (c) 2012-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "storage/coins.h" +#include "utils/util.h" + +#include "utils/random.h" + +#include + +/** + * calculate number of bytes for the bitmask, and its number of non-zero bytes + * each bit in the bitmask represents the availability of one output, but the + * availabilities of the first two outputs are encoded separately + */ +void CCoins::CalcMaskSize(unsigned int &nBytes, unsigned int &nNonzeroBytes) const { + unsigned int nLastUsedByte = 0; + for (unsigned int b = 0; 2+b*8 < vout.size(); b++) { + bool fZero = true; + for (unsigned int i = 0; i < 8 && 2+b*8+i < vout.size(); i++) { + if (!vout[2+b*8+i].IsNull()) { + fZero = false; + continue; + } + } + if (!fZero) { + nLastUsedByte = b + 1; + nNonzeroBytes++; + } + } + nBytes += nLastUsedByte; +} + +bool CCoins::Spend(const COutPoint &out, CTxInUndo &undo) { + if (out.n >= vout.size()) + return false; + if (vout[out.n].IsNull()) + return false; + undo = CTxInUndo(vout[out.n]); + vout[out.n].SetNull(); + Cleanup(); + if (vout.size() == 0) { + undo.nHeight = nHeight; + undo.fCoinBase = fCoinBase; + undo.nVersion = this->nVersion; + } + return true; +} + +bool CCoins::Spend(int nPos) { + CTxInUndo undo; + COutPoint out(0, nPos); + return Spend(out, undo); +} + + +bool CCoinsView::GetCoins(const uint256 &txid, CCoins &coins) const { return false; } +bool CCoinsView::HaveCoins(const uint256 &txid) const { return false; } +uint256 CCoinsView::GetBestBlock() const { return uint256(0); } +bool CCoinsView::BatchWrite(CCoinsMap &mapCoins, const uint256 &hashBlock) { return false; } +bool CCoinsView::GetStats(CCoinsStats &stats) const { return false; } + + +CCoinsViewBacked::CCoinsViewBacked(CCoinsView *viewIn) : base(viewIn) { } +bool CCoinsViewBacked::GetCoins(const uint256 &txid, CCoins &coins) const { return base->GetCoins(txid, coins); } +bool CCoinsViewBacked::HaveCoins(const uint256 &txid) const { return base->HaveCoins(txid); } +uint256 CCoinsViewBacked::GetBestBlock() const { return base->GetBestBlock(); } +void CCoinsViewBacked::SetBackend(CCoinsView &viewIn) { base = &viewIn; } +bool CCoinsViewBacked::BatchWrite(CCoinsMap &mapCoins, const uint256 &hashBlock) { return base->BatchWrite(mapCoins, hashBlock); } +bool CCoinsViewBacked::GetStats(CCoinsStats &stats) const { return base->GetStats(stats); } + +CCoinsKeyHasher::CCoinsKeyHasher() : salt(GetRandHash()) {} + +CCoinsViewCache::CCoinsViewCache(CCoinsView *baseIn) : CCoinsViewBacked(baseIn), hasModifier(false), hashBlock(0) { } + +CCoinsViewCache::~CCoinsViewCache() +{ + assert(!hasModifier); +} + +CCoinsMap::const_iterator CCoinsViewCache::FetchCoins(const uint256 &txid) const { + CCoinsMap::iterator it = cacheCoins.find(txid); + if (it != cacheCoins.end()) + return it; + CCoins tmp; + if (!base->GetCoins(txid, tmp)) + return cacheCoins.end(); + CCoinsMap::iterator ret = cacheCoins.insert(std::make_pair(txid, CCoinsCacheEntry())).first; + tmp.swap(ret->second.coins); + if (ret->second.coins.IsPruned()) { + // The parent only has an empty entry for this txid; we can consider our + // version as fresh. + ret->second.flags = CCoinsCacheEntry::FRESH; + } + return ret; +} + +bool CCoinsViewCache::GetCoins(const uint256 &txid, CCoins &coins) const { + CCoinsMap::const_iterator it = FetchCoins(txid); + if (it != cacheCoins.end()) { + coins = it->second.coins; + return true; + } + return false; +} + +CCoinsModifier CCoinsViewCache::ModifyCoins(const uint256 &txid) { + assert(!hasModifier); + std::pair ret = cacheCoins.insert(std::make_pair(txid, CCoinsCacheEntry())); + if (ret.second) { + if (!base->GetCoins(txid, ret.first->second.coins)) { + // The parent view does not have this entry; mark it as fresh. + ret.first->second.coins.Clear(); + ret.first->second.flags = CCoinsCacheEntry::FRESH; + } else if (ret.first->second.coins.IsPruned()) { + // The parent view only has a pruned entry for this; mark it as fresh. + ret.first->second.flags = CCoinsCacheEntry::FRESH; + } + } + // Assume that whenever ModifyCoins is called, the entry will be modified. + ret.first->second.flags |= CCoinsCacheEntry::DIRTY; +/* MCHN START */ + LogPrint("mccoin","COIN: CH Modify %s\n", txid.ToString().c_str()); +/* MCHN END */ + return CCoinsModifier(*this, ret.first); +} + +const CCoins* CCoinsViewCache::AccessCoins(const uint256 &txid) const { + CCoinsMap::const_iterator it = FetchCoins(txid); + if (it == cacheCoins.end()) { + return NULL; + } else { + return &it->second.coins; + } +} + +bool CCoinsViewCache::HaveCoins(const uint256 &txid) const { + CCoinsMap::const_iterator it = FetchCoins(txid); + // We're using vtx.empty() instead of IsPruned here for performance reasons, + // as we only care about the case where a transaction was replaced entirely + // in a reorganization (which wipes vout entirely, as opposed to spending + // which just cleans individual outputs). + return (it != cacheCoins.end() && !it->second.coins.vout.empty()); +} + +uint256 CCoinsViewCache::GetBestBlock() const { + if (hashBlock == uint256(0)) + hashBlock = base->GetBestBlock(); + return hashBlock; +} + +void CCoinsViewCache::SetBestBlock(const uint256 &hashBlockIn) { + hashBlock = hashBlockIn; +} + +bool CCoinsViewCache::BatchWrite(CCoinsMap &mapCoins, const uint256 &hashBlockIn) { + assert(!hasModifier); + for (CCoinsMap::iterator it = mapCoins.begin(); it != mapCoins.end();) { + if (it->second.flags & CCoinsCacheEntry::DIRTY) { // Ignore non-dirty entries (optimization). + CCoinsMap::iterator itUs = cacheCoins.find(it->first); + if (itUs == cacheCoins.end()) { + if (!it->second.coins.IsPruned()) { + // The parent cache does not have an entry, while the child + // cache does have (a non-pruned) one. Move the data up, and + // mark it as fresh (if the grandparent did have it, we + // would have pulled it in at first GetCoins). + assert(it->second.flags & CCoinsCacheEntry::FRESH); + CCoinsCacheEntry& entry = cacheCoins[it->first]; + entry.coins.swap(it->second.coins); + entry.flags = CCoinsCacheEntry::DIRTY | CCoinsCacheEntry::FRESH; +/* MCHN START */ + LogPrint("mccoin","COIN: CH Write %s\n", it->first.ToString().c_str()); +/* MCHN END */ + } + } else { + if ((itUs->second.flags & CCoinsCacheEntry::FRESH) && it->second.coins.IsPruned()) { + // The grandparent does not have an entry, and the child is + // modified and being pruned. This means we can just delete + // it from the parent. + cacheCoins.erase(itUs); +/* MCHN START */ + LogPrint("mccoin","COIN: CH Erase %s\n", it->first.ToString().c_str()); +/* MCHN END */ + } else { + // A normal modification. + itUs->second.coins.swap(it->second.coins); + itUs->second.flags |= CCoinsCacheEntry::DIRTY; +/* MCHN START */ + LogPrint("mccoin","COIN: CH Update %s\n", it->first.ToString().c_str()); +/* MCHN END */ + } + } + } + else + { +/* MCHN START */ + LogPrint("mccoin","COIN: CH Clean! %s\n", it->first.ToString().c_str()); +/* MCHN END */ + } + CCoinsMap::iterator itOld = it++; + mapCoins.erase(itOld); + } + hashBlock = hashBlockIn; + return true; +} + +bool CCoinsViewCache::Flush() { + bool fOk = base->BatchWrite(cacheCoins, hashBlock); + cacheCoins.clear(); + return fOk; +} + +unsigned int CCoinsViewCache::GetCacheSize() const { + return cacheCoins.size(); +} + +const CTxOut &CCoinsViewCache::GetOutputFor(const CTxIn& input) const +{ + const CCoins* coins = AccessCoins(input.prevout.hash); + assert(coins && coins->IsAvailable(input.prevout.n)); + return coins->vout[input.prevout.n]; +} + +CAmount CCoinsViewCache::GetValueIn(const CTransaction& tx) const +{ + if (tx.IsCoinBase()) + return 0; + + CAmount nResult = 0; + for (unsigned int i = 0; i < tx.vin.size(); i++) + nResult += GetOutputFor(tx.vin[i]).nValue; + + return nResult; +} + +bool CCoinsViewCache::HaveInputs(const CTransaction& tx) const +{ + if (!tx.IsCoinBase()) { + for (unsigned int i = 0; i < tx.vin.size(); i++) { + const COutPoint &prevout = tx.vin[i].prevout; + const CCoins* coins = AccessCoins(prevout.hash); + if (!coins || !coins->IsAvailable(prevout.n)) { + return false; + } + } + } + return true; +} + +double CCoinsViewCache::GetPriority(const CTransaction &tx, int nHeight) const +{ + if (tx.IsCoinBase()) + return 0.0; + double dResult = 0.0; + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + const CCoins* coins = AccessCoins(txin.prevout.hash); + assert(coins); + if (!coins->IsAvailable(txin.prevout.n)) continue; + if (coins->nHeight < nHeight) { + dResult += coins->vout[txin.prevout.n].nValue * (nHeight-coins->nHeight); + } + } + return tx.ComputePriority(dResult); +} + +CCoinsModifier::CCoinsModifier(CCoinsViewCache& cache_, CCoinsMap::iterator it_) : cache(cache_), it(it_) { + assert(!cache.hasModifier); + cache.hasModifier = true; +} + +CCoinsModifier::~CCoinsModifier() +{ + assert(cache.hasModifier); + cache.hasModifier = false; + it->second.coins.Cleanup(); + if ((it->second.flags & CCoinsCacheEntry::FRESH) && it->second.coins.IsPruned()) { + cache.cacheCoins.erase(it); + } +} diff --git a/src/storage/coins.h b/src/storage/coins.h new file mode 100644 index 00000000..878fda9e --- /dev/null +++ b/src/storage/coins.h @@ -0,0 +1,447 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_COINS_H +#define BITCOIN_COINS_H + +#include "utils/compressor.h" +#include "utils/serialize.h" +#include "structs/uint256.h" +#include "chain/undo.h" + +#include +#include + +#include +#include + +/** + * Pruned version of CTransaction: only retains metadata and unspent transaction outputs + * + * Serialized format: + * - VARINT(nVersion) + * - VARINT(nCode) + * - unspentness bitvector, for vout[2] and further; least significant byte first + * - the non-spent CTxOuts (via CTxOutCompressor) + * - VARINT(nHeight) + * + * The nCode value consists of: + * - bit 1: IsCoinBase() + * - bit 2: vout[0] is not spent + * - bit 4: vout[1] is not spent + * - The higher bits encode N, the number of non-zero bytes in the following bitvector. + * - In case both bit 2 and bit 4 are unset, they encode N-1, as there must be at + * least one non-spent output). + * + * Example: 0104835800816115944e077fe7c803cfa57f29b36bf87c1d358bb85e + * <><><--------------------------------------------><----> + * | \ | / + * version code vout[1] height + * + * - version = 1 + * - code = 4 (vout[1] is not spent, and 0 non-zero bytes of bitvector follow) + * - unspentness bitvector: as 0 non-zero bytes follow, it has length 0 + * - vout[1]: 835800816115944e077fe7c803cfa57f29b36bf87c1d35 + * * 8358: compact amount representation for 60000000000 (600 BTC) + * * 00: special txout type pay-to-pubkey-hash + * * 816115944e077fe7c803cfa57f29b36bf87c1d35: address uint160 + * - height = 203998 + * + * + * Example: 0109044086ef97d5790061b01caab50f1b8e9c50a5057eb43c2d9563a4eebbd123008c988f1a4a4de2161e0f50aac7f17e7f9555caa486af3b + * <><><--><--------------------------------------------------><----------------------------------------------><----> + * / \ \ | | / + * version code unspentness vout[4] vout[16] height + * + * - version = 1 + * - code = 9 (coinbase, neither vout[0] or vout[1] are unspent, + * 2 (1, +1 because both bit 2 and bit 4 are unset) non-zero bitvector bytes follow) + * - unspentness bitvector: bits 2 (0x04) and 14 (0x4000) are set, so vout[2+2] and vout[14+2] are unspent + * - vout[4]: 86ef97d5790061b01caab50f1b8e9c50a5057eb43c2d9563a4ee + * * 86ef97d579: compact amount representation for 234925952 (2.35 BTC) + * * 00: special txout type pay-to-pubkey-hash + * * 61b01caab50f1b8e9c50a5057eb43c2d9563a4ee: address uint160 + * - vout[16]: bbd123008c988f1a4a4de2161e0f50aac7f17e7f9555caa4 + * * bbd123: compact amount representation for 110397 (0.001 BTC) + * * 00: special txout type pay-to-pubkey-hash + * * 8c988f1a4a4de2161e0f50aac7f17e7f9555caa4: address uint160 + * - height = 120891 + */ +class CCoins +{ +public: + //! whether transaction is a coinbase + bool fCoinBase; + + //! unspent transaction outputs; spent outputs are .IsNull(); spent outputs at the end of the array are dropped + std::vector vout; + + //! at which height this transaction was included in the active block chain + int nHeight; + + //! version of the CTransaction; accesses to this value should probably check for nHeight as well, + //! as new tx version will probably only be introduced at certain heights + int nVersion; + + void FromTx(const CTransaction &tx, int nHeightIn) { + fCoinBase = tx.IsCoinBase(); + vout = tx.vout; + nHeight = nHeightIn; + nVersion = tx.nVersion; + ClearUnspendable(); + } + + //! construct a CCoins from a CTransaction, at a given height + CCoins(const CTransaction &tx, int nHeightIn) { + FromTx(tx, nHeightIn); + } + + void Clear() { + fCoinBase = false; + std::vector().swap(vout); + nHeight = 0; + nVersion = 0; + } + + //! empty constructor + CCoins() : fCoinBase(false), vout(0), nHeight(0), nVersion(0) { } + + //!remove spent outputs at the end of vout + void Cleanup() { + while (vout.size() > 0 && vout.back().IsNull()) + vout.pop_back(); + if (vout.empty()) + std::vector().swap(vout); + } + + void ClearUnspendable() { + BOOST_FOREACH(CTxOut &txout, vout) { + if (txout.scriptPubKey.IsUnspendable()) + txout.SetNull(); + } + Cleanup(); + } + + void swap(CCoins &to) { + std::swap(to.fCoinBase, fCoinBase); + to.vout.swap(vout); + std::swap(to.nHeight, nHeight); + std::swap(to.nVersion, nVersion); + } + + //! equality test + friend bool operator==(const CCoins &a, const CCoins &b) { + // Empty CCoins objects are always equal. + if (a.IsPruned() && b.IsPruned()) + return true; + return a.fCoinBase == b.fCoinBase && + a.nHeight == b.nHeight && + a.nVersion == b.nVersion && + a.vout == b.vout; + } + friend bool operator!=(const CCoins &a, const CCoins &b) { + return !(a == b); + } + + void CalcMaskSize(unsigned int &nBytes, unsigned int &nNonzeroBytes) const; + + bool IsCoinBase() const { + return fCoinBase; + } + + unsigned int GetSerializeSize(int nType, int nVersion) const { + unsigned int nSize = 0; + unsigned int nMaskSize = 0, nMaskCode = 0; + CalcMaskSize(nMaskSize, nMaskCode); + bool fFirst = vout.size() > 0 && !vout[0].IsNull(); + bool fSecond = vout.size() > 1 && !vout[1].IsNull(); + assert(fFirst || fSecond || nMaskCode); + unsigned int nCode = 8*(nMaskCode - (fFirst || fSecond ? 0 : 1)) + (fCoinBase ? 1 : 0) + (fFirst ? 2 : 0) + (fSecond ? 4 : 0); + // version + nSize += ::GetSerializeSize(VARINT(this->nVersion), nType, nVersion); + // size of header code + nSize += ::GetSerializeSize(VARINT(nCode), nType, nVersion); + // spentness bitmask + nSize += nMaskSize; + // txouts themself + for (unsigned int i = 0; i < vout.size(); i++) + if (!vout[i].IsNull()) + nSize += ::GetSerializeSize(CTxOutCompressor(REF(vout[i])), nType, nVersion); + // height + nSize += ::GetSerializeSize(VARINT(nHeight), nType, nVersion); + return nSize; + } + + template + void Serialize(Stream &s, int nType, int nVersion) const { + unsigned int nMaskSize = 0, nMaskCode = 0; + CalcMaskSize(nMaskSize, nMaskCode); + bool fFirst = vout.size() > 0 && !vout[0].IsNull(); + bool fSecond = vout.size() > 1 && !vout[1].IsNull(); + assert(fFirst || fSecond || nMaskCode); + unsigned int nCode = 8*(nMaskCode - (fFirst || fSecond ? 0 : 1)) + (fCoinBase ? 1 : 0) + (fFirst ? 2 : 0) + (fSecond ? 4 : 0); + // version + ::Serialize(s, VARINT(this->nVersion), nType, nVersion); + // header code + ::Serialize(s, VARINT(nCode), nType, nVersion); + // spentness bitmask + for (unsigned int b = 0; b + void Unserialize(Stream &s, int nType, int nVersion) { + unsigned int nCode = 0; + // version + ::Unserialize(s, VARINT(this->nVersion), nType, nVersion); + // header code + ::Unserialize(s, VARINT(nCode), nType, nVersion); + fCoinBase = nCode & 1; + std::vector vAvail(2, false); + vAvail[0] = (nCode & 2) != 0; + vAvail[1] = (nCode & 4) != 0; + unsigned int nMaskCode = (nCode / 8) + ((nCode & 6) != 0 ? 0 : 1); + // spentness bitmask + while (nMaskCode > 0) { + unsigned char chAvail = 0; + ::Unserialize(s, chAvail, nType, nVersion); + for (unsigned int p = 0; p < 8; p++) { + bool f = (chAvail & (1 << p)) != 0; + vAvail.push_back(f); + } + if (chAvail != 0) + nMaskCode--; + } + // txouts themself + vout.assign(vAvail.size(), CTxOut()); + for (unsigned int i = 0; i < vAvail.size(); i++) { + if (vAvail[i]) + ::Unserialize(s, REF(CTxOutCompressor(vout[i])), nType, nVersion); + } + // coinbase height + ::Unserialize(s, VARINT(nHeight), nType, nVersion); + Cleanup(); + } + + //! mark an outpoint spent, and construct undo information + bool Spend(const COutPoint &out, CTxInUndo &undo); + + //! mark a vout spent + bool Spend(int nPos); + + //! check whether a particular output is still available + bool IsAvailable(unsigned int nPos) const { + return (nPos < vout.size() && !vout[nPos].IsNull()); + } + + //! check whether the entire CCoins is spent + //! note that only !IsPruned() CCoins can be serialized + bool IsPruned() const { + BOOST_FOREACH(const CTxOut &out, vout) + if (!out.IsNull()) + return false; + return true; + } +}; + +class CCoinsKeyHasher +{ +private: + uint256 salt; + +public: + CCoinsKeyHasher(); + + /** + * This *must* return size_t. With Boost 1.46 on 32-bit systems the + * unordered_map will behave unpredictably if the custom hasher returns a + * uint64_t, resulting in failures when syncing the chain (#4634). + */ + size_t operator()(const uint256& key) const { + return key.GetHash(salt); + } +}; + +struct CCoinsCacheEntry +{ + CCoins coins; // The actual cached data. + unsigned char flags; + + enum Flags { + DIRTY = (1 << 0), // This cache entry is potentially different from the version in the parent view. + FRESH = (1 << 1), // The parent view does not have this entry (or it is pruned). + }; + + CCoinsCacheEntry() : coins(), flags(0) {} +}; + +typedef boost::unordered_map CCoinsMap; + +struct CCoinsStats +{ + int nHeight; + uint256 hashBlock; + uint64_t nTransactions; + uint64_t nTransactionOutputs; + uint64_t nSerializedSize; + uint256 hashSerialized; + CAmount nTotalAmount; + + CCoinsStats() : nHeight(0), hashBlock(0), nTransactions(0), nTransactionOutputs(0), nSerializedSize(0), hashSerialized(0), nTotalAmount(0) {} +}; + + +/** Abstract view on the open txout dataset. */ +class CCoinsView +{ +public: + //! Retrieve the CCoins (unspent transaction outputs) for a given txid + virtual bool GetCoins(const uint256 &txid, CCoins &coins) const; + + //! Just check whether we have data for a given txid. + //! This may (but cannot always) return true for fully spent transactions + virtual bool HaveCoins(const uint256 &txid) const; + + //! Retrieve the block hash whose state this CCoinsView currently represents + virtual uint256 GetBestBlock() const; + + //! Do a bulk modification (multiple CCoins changes + BestBlock change). + //! The passed mapCoins can be modified. + virtual bool BatchWrite(CCoinsMap &mapCoins, const uint256 &hashBlock); + + //! Calculate statistics about the unspent transaction output set + virtual bool GetStats(CCoinsStats &stats) const; + + //! As we use CCoinsViews polymorphically, have a virtual destructor + virtual ~CCoinsView() {} +}; + + +/** CCoinsView backed by another CCoinsView */ +class CCoinsViewBacked : public CCoinsView +{ +protected: + CCoinsView *base; + +public: + CCoinsViewBacked(CCoinsView *viewIn); + bool GetCoins(const uint256 &txid, CCoins &coins) const; + bool HaveCoins(const uint256 &txid) const; + uint256 GetBestBlock() const; + void SetBackend(CCoinsView &viewIn); + bool BatchWrite(CCoinsMap &mapCoins, const uint256 &hashBlock); + bool GetStats(CCoinsStats &stats) const; +}; + + +class CCoinsViewCache; + +/** + * A reference to a mutable cache entry. Encapsulating it allows us to run + * cleanup code after the modification is finished, and keeping track of + * concurrent modifications. + */ +class CCoinsModifier +{ +private: + CCoinsViewCache& cache; + CCoinsMap::iterator it; + CCoinsModifier(CCoinsViewCache& cache_, CCoinsMap::iterator it_); + +public: + CCoins* operator->() { return &it->second.coins; } + CCoins& operator*() { return it->second.coins; } + ~CCoinsModifier(); + friend class CCoinsViewCache; +}; + +/** CCoinsView that adds a memory cache for transactions to another CCoinsView */ +class CCoinsViewCache : public CCoinsViewBacked +{ +protected: + /* Whether this cache has an active modifier. */ + bool hasModifier; + + /** + * Make mutable so that we can "fill the cache" even from Get-methods + * declared as "const". + */ + mutable uint256 hashBlock; + mutable CCoinsMap cacheCoins; + +public: + CCoinsViewCache(CCoinsView *baseIn); + ~CCoinsViewCache(); + + // Standard CCoinsView methods + bool GetCoins(const uint256 &txid, CCoins &coins) const; + bool HaveCoins(const uint256 &txid) const; + uint256 GetBestBlock() const; + void SetBestBlock(const uint256 &hashBlock); + bool BatchWrite(CCoinsMap &mapCoins, const uint256 &hashBlock); + + /** + * Return a pointer to CCoins in the cache, or NULL if not found. This is + * more efficient than GetCoins. Modifications to other cache entries are + * allowed while accessing the returned pointer. + */ + const CCoins* AccessCoins(const uint256 &txid) const; + + /** + * Return a modifiable reference to a CCoins. If no entry with the given + * txid exists, a new one is created. Simultaneous modifications are not + * allowed. + */ + CCoinsModifier ModifyCoins(const uint256 &txid); + + /** + * Push the modifications applied to this cache to its base. + * Failure to call this method before destruction will cause the changes to be forgotten. + * If false is returned, the state of this cache (and its backing view) will be undefined. + */ + bool Flush(); + + //! Calculate the size of the cache (in number of transactions) + unsigned int GetCacheSize() const; + + /** + * Amount of bitcoins coming in to a transaction + * Note that lightweight clients may not know anything besides the hash of previous transactions, + * so may not be able to calculate this. + * + * @param[in] tx transaction for which we are checking input total + * @return Sum of value of all inputs (scriptSigs) + */ + CAmount GetValueIn(const CTransaction& tx) const; + + //! Check whether all prevouts of the transaction are present in the UTXO set represented by this view + bool HaveInputs(const CTransaction& tx) const; + + //! Return priority of tx at height nHeight + double GetPriority(const CTransaction &tx, int nHeight) const; + + const CTxOut &GetOutputFor(const CTxIn& input) const; + + friend class CCoinsModifier; + +private: + CCoinsMap::iterator FetchCoins(const uint256 &txid); + CCoinsMap::const_iterator FetchCoins(const uint256 &txid) const; +}; + +#endif // BITCOIN_COINS_H diff --git a/src/storage/leveldbwrapper.cpp b/src/storage/leveldbwrapper.cpp new file mode 100644 index 00000000..8c5a9236 --- /dev/null +++ b/src/storage/leveldbwrapper.cpp @@ -0,0 +1,89 @@ +// Copyright (c) 2012-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "storage/leveldbwrapper.h" + +#include "utils/util.h" + +#include + +#include +#include +#include +#include + +void HandleError(const leveldb::Status& status) throw(leveldb_error) +{ + if (status.ok()) + return; + LogPrintf("%s\n", status.ToString()); + if (status.IsCorruption()) + throw leveldb_error("Database corrupted"); + if (status.IsIOError()) + throw leveldb_error("Database I/O error"); + if (status.IsNotFound()) + throw leveldb_error("Database entry missing"); + throw leveldb_error("Unknown database error"); +} + +static leveldb::Options GetOptions(size_t nCacheSize) +{ + leveldb::Options options; + options.block_cache = leveldb::NewLRUCache(nCacheSize / 2); + options.write_buffer_size = nCacheSize / 4; // up to two write buffers may be held in memory simultaneously + options.filter_policy = leveldb::NewBloomFilterPolicy(10); + options.compression = leveldb::kNoCompression; + options.max_open_files = 64; + if (leveldb::kMajorVersion > 1 || (leveldb::kMajorVersion == 1 && leveldb::kMinorVersion >= 16)) { + // LevelDB versions before 1.16 consider short writes to be corruption. Only trigger error + // on corruption in later versions. + options.paranoid_checks = true; + } + return options; +} + +CLevelDBWrapper::CLevelDBWrapper(const boost::filesystem::path& path, size_t nCacheSize, bool fMemory, bool fWipe) +{ + penv = NULL; + readoptions.verify_checksums = true; + iteroptions.verify_checksums = true; + iteroptions.fill_cache = false; + syncoptions.sync = true; + options = GetOptions(nCacheSize); + options.create_if_missing = true; + if (fMemory) { + penv = leveldb::NewMemEnv(leveldb::Env::Default()); + options.env = penv; + } else { + if (fWipe) { + LogPrintf("Wiping LevelDB in %s\n", path.string()); + leveldb::DestroyDB(path.string(), options); + } + TryCreateDirectory(path); + LogPrintf("Opening LevelDB in %s\n", path.string()); + } + leveldb::Status status = leveldb::DB::Open(options, path.string(), &pdb); + HandleError(status); + LogPrintf("Opened LevelDB successfully\n"); +} + +CLevelDBWrapper::~CLevelDBWrapper() +{ + delete pdb; + pdb = NULL; + delete options.filter_policy; + options.filter_policy = NULL; + delete options.block_cache; + options.block_cache = NULL; + delete penv; + options.env = NULL; +} + +bool CLevelDBWrapper::WriteBatch(CLevelDBBatch& batch, bool fSync) throw(leveldb_error) +{ + leveldb::Status status = pdb->Write(fSync ? syncoptions : writeoptions, &batch.batch); + HandleError(status); + return true; +} diff --git a/src/storage/leveldbwrapper.h b/src/storage/leveldbwrapper.h new file mode 100644 index 00000000..7cd7a58d --- /dev/null +++ b/src/storage/leveldbwrapper.h @@ -0,0 +1,174 @@ +// Copyright (c) 2012-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_LEVELDBWRAPPER_H +#define BITCOIN_LEVELDBWRAPPER_H + +#include "version/clientversion.h" +#include "utils/serialize.h" +#include "utils/streams.h" +#include "utils/util.h" +#include "version/bcversion.h" + +#include + +#include +#include + +class leveldb_error : public std::runtime_error +{ +public: + leveldb_error(const std::string& msg) : std::runtime_error(msg) {} +}; + +void HandleError(const leveldb::Status& status) throw(leveldb_error); + +/** Batch of changes queued to be written to a CLevelDBWrapper */ +class CLevelDBBatch +{ + friend class CLevelDBWrapper; + +private: + leveldb::WriteBatch batch; + +public: + template + void Write(const K& key, const V& value) + { + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + ssKey.reserve(ssKey.GetSerializeSize(key)); + ssKey << key; + leveldb::Slice slKey(&ssKey[0], ssKey.size()); + + CDataStream ssValue(SER_DISK, CLIENT_VERSION); + ssValue.reserve(ssValue.GetSerializeSize(value)); + ssValue << value; + leveldb::Slice slValue(&ssValue[0], ssValue.size()); + + batch.Put(slKey, slValue); + } + + template + void Erase(const K& key) + { + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + ssKey.reserve(ssKey.GetSerializeSize(key)); + ssKey << key; + leveldb::Slice slKey(&ssKey[0], ssKey.size()); + + batch.Delete(slKey); + } +}; + +class CLevelDBWrapper +{ +private: + //! custom environment this database is using (may be NULL in case of default environment) + leveldb::Env* penv; + + //! database options used + leveldb::Options options; + + //! options used when reading from the database + leveldb::ReadOptions readoptions; + + //! options used when iterating over values of the database + leveldb::ReadOptions iteroptions; + + //! options used when writing to the database + leveldb::WriteOptions writeoptions; + + //! options used when sync writing to the database + leveldb::WriteOptions syncoptions; + + //! the database itself + leveldb::DB* pdb; + +public: + CLevelDBWrapper(const boost::filesystem::path& path, size_t nCacheSize, bool fMemory = false, bool fWipe = false); + ~CLevelDBWrapper(); + + template + bool Read(const K& key, V& value) const throw(leveldb_error) + { + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + ssKey.reserve(ssKey.GetSerializeSize(key)); + ssKey << key; + leveldb::Slice slKey(&ssKey[0], ssKey.size()); + + std::string strValue; + leveldb::Status status = pdb->Get(readoptions, slKey, &strValue); + if (!status.ok()) { + if (status.IsNotFound()) + return false; + LogPrintf("LevelDB read failure: %s\n", status.ToString()); + HandleError(status); + } + try { + CDataStream ssValue(strValue.data(), strValue.data() + strValue.size(), SER_DISK, CLIENT_VERSION); + ssValue >> value; + } catch (const std::exception&) { + return false; + } + return true; + } + + template + bool Write(const K& key, const V& value, bool fSync = false) throw(leveldb_error) + { + CLevelDBBatch batch; + batch.Write(key, value); + return WriteBatch(batch, fSync); + } + + template + bool Exists(const K& key) const throw(leveldb_error) + { + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + ssKey.reserve(ssKey.GetSerializeSize(key)); + ssKey << key; + leveldb::Slice slKey(&ssKey[0], ssKey.size()); + + std::string strValue; + leveldb::Status status = pdb->Get(readoptions, slKey, &strValue); + if (!status.ok()) { + if (status.IsNotFound()) + return false; + LogPrintf("LevelDB read failure: %s\n", status.ToString()); + HandleError(status); + } + return true; + } + + template + bool Erase(const K& key, bool fSync = false) throw(leveldb_error) + { + CLevelDBBatch batch; + batch.Erase(key); + return WriteBatch(batch, fSync); + } + + bool WriteBatch(CLevelDBBatch& batch, bool fSync = false) throw(leveldb_error); + + // not available for LevelDB; provide for compatibility with BDB + bool Flush() + { + return true; + } + + bool Sync() throw(leveldb_error) + { + CLevelDBBatch batch; + return WriteBatch(batch, true); + } + + // not exactly clean encapsulation, but it's easiest for now + leveldb::Iterator* NewIterator() + { + return pdb->NewIterator(iteroptions); + } +}; + +#endif // BITCOIN_LEVELDBWRAPPER_H diff --git a/src/storage/txdb.cpp b/src/storage/txdb.cpp new file mode 100644 index 00000000..adf78c26 --- /dev/null +++ b/src/storage/txdb.cpp @@ -0,0 +1,239 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "storage/txdb.h" + +#include "chain/pow.h" +#include "structs/uint256.h" + +#include + +#include + +using namespace std; + +void static BatchWriteCoins(CLevelDBBatch &batch, const uint256 &hash, const CCoins &coins) { + if (coins.IsPruned()) + { +/* MCHN START */ + LogPrint("mccoin", "COIN: DB Erase %s\n", hash.ToString().c_str()); +/* MCHN END */ + batch.Erase(make_pair('c', hash)); + } + else + { +/* MCHN START */ + LogPrint("mccoin", "COIN: DB Write %s\n", hash.ToString().c_str()); +/* MCHN END */ + batch.Write(make_pair('c', hash), coins); + } +} + +void static BatchWriteHashBestChain(CLevelDBBatch &batch, const uint256 &hash) { + batch.Write('B', hash); +} + +CCoinsViewDB::CCoinsViewDB(size_t nCacheSize, bool fMemory, bool fWipe) : db(GetDataDir() / "chainstate", nCacheSize, fMemory, fWipe) { +} + +bool CCoinsViewDB::GetCoins(const uint256 &txid, CCoins &coins) const { + return db.Read(make_pair('c', txid), coins); +} + +bool CCoinsViewDB::HaveCoins(const uint256 &txid) const { + return db.Exists(make_pair('c', txid)); +} + +uint256 CCoinsViewDB::GetBestBlock() const { + uint256 hashBestChain; + if (!db.Read('B', hashBestChain)) + return uint256(0); + return hashBestChain; +} + +bool CCoinsViewDB::BatchWrite(CCoinsMap &mapCoins, const uint256 &hashBlock) { + CLevelDBBatch batch; + size_t count = 0; + size_t changed = 0; + for (CCoinsMap::iterator it = mapCoins.begin(); it != mapCoins.end();) { + if (it->second.flags & CCoinsCacheEntry::DIRTY) { + BatchWriteCoins(batch, it->first, it->second.coins); + changed++; + } + count++; + CCoinsMap::iterator itOld = it++; + mapCoins.erase(itOld); + } + if (hashBlock != uint256(0)) + BatchWriteHashBestChain(batch, hashBlock); + + LogPrint("coindb", "Committing %u changed transactions (out of %u) to coin database...\n", (unsigned int)changed, (unsigned int)count); + return db.WriteBatch(batch); +} + +CBlockTreeDB::CBlockTreeDB(size_t nCacheSize, bool fMemory, bool fWipe) : CLevelDBWrapper(GetDataDir() / "blocks" / "index", nCacheSize, fMemory, fWipe) { +} + +bool CBlockTreeDB::WriteBlockIndex(const CDiskBlockIndex& blockindex) +{ + return Write(make_pair('b', blockindex.GetBlockHash()), blockindex); +} + +bool CBlockTreeDB::WriteBlockFileInfo(int nFile, const CBlockFileInfo &info) { + return Write(make_pair('f', nFile), info); +} + +bool CBlockTreeDB::ReadBlockFileInfo(int nFile, CBlockFileInfo &info) { + return Read(make_pair('f', nFile), info); +} + +bool CBlockTreeDB::WriteLastBlockFile(int nFile) { + return Write('l', nFile); +} + +bool CBlockTreeDB::WriteReindexing(bool fReindexing) { + if (fReindexing) + return Write('R', '1'); + else + return Erase('R'); +} + +bool CBlockTreeDB::ReadReindexing(bool &fReindexing) { + fReindexing = Exists('R'); + return true; +} + +bool CBlockTreeDB::ReadLastBlockFile(int &nFile) { + return Read('l', nFile); +} + +bool CCoinsViewDB::GetStats(CCoinsStats &stats) const { + /* It seems that there are no "const iterators" for LevelDB. Since we + only need read operations on it, use a const-cast to get around + that restriction. */ + boost::scoped_ptr pcursor(const_cast(&db)->NewIterator()); + pcursor->SeekToFirst(); + + CHashWriter ss(SER_GETHASH, PROTOCOL_VERSION); + stats.hashBlock = GetBestBlock(); + ss << stats.hashBlock; + CAmount nTotalAmount = 0; + while (pcursor->Valid()) { + boost::this_thread::interruption_point(); + try { + leveldb::Slice slKey = pcursor->key(); + CDataStream ssKey(slKey.data(), slKey.data()+slKey.size(), SER_DISK, CLIENT_VERSION); + char chType; + ssKey >> chType; + if (chType == 'c') { + leveldb::Slice slValue = pcursor->value(); + CDataStream ssValue(slValue.data(), slValue.data()+slValue.size(), SER_DISK, CLIENT_VERSION); + CCoins coins; + ssValue >> coins; + uint256 txhash; + ssKey >> txhash; + ss << txhash; + ss << VARINT(coins.nVersion); + ss << (coins.fCoinBase ? 'c' : 'n'); + ss << VARINT(coins.nHeight); + stats.nTransactions++; + for (unsigned int i=0; iNext(); + } catch (std::exception &e) { + return error("%s : Deserialize or I/O error - %s", __func__, e.what()); + } + } + stats.nHeight = mapBlockIndex.find(GetBestBlock())->second->nHeight; + stats.hashSerialized = ss.GetHash(); + stats.nTotalAmount = nTotalAmount; + return true; +} + +bool CBlockTreeDB::ReadTxIndex(const uint256 &txid, CDiskTxPos &pos) { + return Read(make_pair('t', txid), pos); +} + +bool CBlockTreeDB::WriteTxIndex(const std::vector >&vect) { + CLevelDBBatch batch; + for (std::vector >::const_iterator it=vect.begin(); it!=vect.end(); it++) + batch.Write(make_pair('t', it->first), it->second); + return WriteBatch(batch); +} + +bool CBlockTreeDB::WriteFlag(const std::string &name, bool fValue) { + return Write(std::make_pair('F', name), fValue ? '1' : '0'); +} + +bool CBlockTreeDB::ReadFlag(const std::string &name, bool &fValue) { + char ch; + if (!Read(std::make_pair('F', name), ch)) + return false; + fValue = ch == '1'; + return true; +} + +bool CBlockTreeDB::LoadBlockIndexGuts() +{ + boost::scoped_ptr pcursor(NewIterator()); + + CDataStream ssKeySet(SER_DISK, CLIENT_VERSION); + ssKeySet << make_pair('b', uint256(0)); + pcursor->Seek(ssKeySet.str()); + + // Load mapBlockIndex + while (pcursor->Valid()) { + boost::this_thread::interruption_point(); + try { + leveldb::Slice slKey = pcursor->key(); + CDataStream ssKey(slKey.data(), slKey.data()+slKey.size(), SER_DISK, CLIENT_VERSION); + char chType; + ssKey >> chType; + if (chType == 'b') { + leveldb::Slice slValue = pcursor->value(); + CDataStream ssValue(slValue.data(), slValue.data()+slValue.size(), SER_DISK, CLIENT_VERSION); + CDiskBlockIndex diskindex; + ssValue >> diskindex; + + // Construct block index object + CBlockIndex* pindexNew = InsertBlockIndex(diskindex.GetBlockHash()); + pindexNew->pprev = InsertBlockIndex(diskindex.hashPrev); + pindexNew->nHeight = diskindex.nHeight; + pindexNew->nFile = diskindex.nFile; + pindexNew->nDataPos = diskindex.nDataPos; + pindexNew->nUndoPos = diskindex.nUndoPos; + pindexNew->nVersion = diskindex.nVersion; + pindexNew->hashMerkleRoot = diskindex.hashMerkleRoot; + pindexNew->nTime = diskindex.nTime; + pindexNew->nBits = diskindex.nBits; + pindexNew->nNonce = diskindex.nNonce; + pindexNew->nStatus = diskindex.nStatus; + pindexNew->nTx = diskindex.nTx; + + if (!CheckProofOfWork(pindexNew->GetBlockHash(), pindexNew->nBits)) + return error("LoadBlockIndex() : CheckProofOfWork failed: %s", pindexNew->ToString()); + + pcursor->Next(); + } else { + break; // if shutdown requested or finished loading block index + } + } catch (std::exception &e) { + return error("%s : Deserialize or I/O error - %s", __func__, e.what()); + } + } + + return true; +} diff --git a/src/storage/txdb.h b/src/storage/txdb.h new file mode 100644 index 00000000..aa9d5ad6 --- /dev/null +++ b/src/storage/txdb.h @@ -0,0 +1,66 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_TXDB_H +#define BITCOIN_TXDB_H + +#include "storage/leveldbwrapper.h" +#include "core/main.h" + +#include +#include +#include +#include + +class CCoins; +class uint256; + +//! -dbcache default (MiB) +static const int64_t nDefaultDbCache = 100; +//! max. -dbcache in (MiB) +static const int64_t nMaxDbCache = sizeof(void*) > 4 ? 4096 : 1024; +//! min. -dbcache in (MiB) +static const int64_t nMinDbCache = 4; + +/** CCoinsView backed by the LevelDB coin database (chainstate/) */ +class CCoinsViewDB : public CCoinsView +{ +protected: + CLevelDBWrapper db; +public: + CCoinsViewDB(size_t nCacheSize, bool fMemory = false, bool fWipe = false); + + bool GetCoins(const uint256 &txid, CCoins &coins) const; + bool HaveCoins(const uint256 &txid) const; + uint256 GetBestBlock() const; + bool BatchWrite(CCoinsMap &mapCoins, const uint256 &hashBlock); + bool GetStats(CCoinsStats &stats) const; +}; + +/** Access to the block database (blocks/index/) */ +class CBlockTreeDB : public CLevelDBWrapper +{ +public: + CBlockTreeDB(size_t nCacheSize, bool fMemory = false, bool fWipe = false); +private: + CBlockTreeDB(const CBlockTreeDB&); + void operator=(const CBlockTreeDB&); +public: + bool WriteBlockIndex(const CDiskBlockIndex& blockindex); + bool ReadBlockFileInfo(int nFile, CBlockFileInfo &fileinfo); + bool WriteBlockFileInfo(int nFile, const CBlockFileInfo &fileinfo); + bool ReadLastBlockFile(int &nFile); + bool WriteLastBlockFile(int nFile); + bool WriteReindexing(bool fReindex); + bool ReadReindexing(bool &fReindex); + bool ReadTxIndex(const uint256 &txid, CDiskTxPos &pos); + bool WriteTxIndex(const std::vector > &list); + bool WriteFlag(const std::string &name, bool fValue); + bool ReadFlag(const std::string &name, bool &fValue); + bool LoadBlockIndexGuts(); +}; + +#endif // BITCOIN_TXDB_H diff --git a/src/structs/alert.cpp b/src/structs/alert.cpp new file mode 100644 index 00000000..aad90188 --- /dev/null +++ b/src/structs/alert.cpp @@ -0,0 +1,267 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "structs/alert.h" + +#include "chainparams/chainparams.h" +#include "version/clientversion.h" +#include "net/net.h" +#include "keys/pubkey.h" +#include "utils/timedata.h" +#include "ui/ui_interface.h" +#include "utils/util.h" + +#include +#include +#include + +#include +#include +#include +#include + +using namespace std; + +map mapAlerts; +CCriticalSection cs_mapAlerts; + +void CUnsignedAlert::SetNull() +{ + nVersion = 1; + nRelayUntil = 0; + nExpiration = 0; + nID = 0; + nCancel = 0; + setCancel.clear(); + nMinVer = 0; + nMaxVer = 0; + setSubVer.clear(); + nPriority = 0; + + strComment.clear(); + strStatusBar.clear(); + strReserved.clear(); +} + +std::string CUnsignedAlert::ToString() const +{ + std::string strSetCancel; + BOOST_FOREACH(int n, setCancel) + strSetCancel += strprintf("%d ", n); + std::string strSetSubVer; + BOOST_FOREACH(std::string str, setSubVer) + strSetSubVer += "\"" + str + "\" "; + return strprintf( + "CAlert(\n" + " nVersion = %d\n" + " nRelayUntil = %d\n" + " nExpiration = %d\n" + " nID = %d\n" + " nCancel = %d\n" + " setCancel = %s\n" + " nMinVer = %d\n" + " nMaxVer = %d\n" + " setSubVer = %s\n" + " nPriority = %d\n" + " strComment = \"%s\"\n" + " strStatusBar = \"%s\"\n" + ")\n", + nVersion, + nRelayUntil, + nExpiration, + nID, + nCancel, + strSetCancel, + nMinVer, + nMaxVer, + strSetSubVer, + nPriority, + strComment, + strStatusBar); +} + +void CAlert::SetNull() +{ + CUnsignedAlert::SetNull(); + vchMsg.clear(); + vchSig.clear(); +} + +bool CAlert::IsNull() const +{ + return (nExpiration == 0); +} + +uint256 CAlert::GetHash() const +{ + return Hash(this->vchMsg.begin(), this->vchMsg.end()); +} + +bool CAlert::IsInEffect() const +{ + return (GetAdjustedTime() < nExpiration); +} + +bool CAlert::Cancels(const CAlert& alert) const +{ + if (!IsInEffect()) + return false; // this was a no-op before 31403 + return (alert.nID <= nCancel || setCancel.count(alert.nID)); +} + +bool CAlert::AppliesTo(int nVersion, std::string strSubVerIn) const +{ + // TODO: rework for client-version-embedded-in-strSubVer ? + return (IsInEffect() && + nMinVer <= nVersion && nVersion <= nMaxVer && + (setSubVer.empty() || setSubVer.count(strSubVerIn))); +} + +bool CAlert::AppliesToMe() const +{ + return AppliesTo(PROTOCOL_VERSION, FormatSubVersion(CLIENT_NAME, CLIENT_VERSION, std::vector())); +} + +bool CAlert::RelayTo(CNode* pnode) const +{ + if (!IsInEffect()) + return false; + // don't relay to nodes which haven't sent their version message + if (pnode->nVersion == 0) + return false; + // returns true if wasn't already contained in the set + if (pnode->setKnown.insert(GetHash()).second) + { + if (AppliesTo(pnode->nVersion, pnode->strSubVer) || + AppliesToMe() || + GetAdjustedTime() < nRelayUntil) + { + pnode->PushMessage("alert", *this); + return true; + } + } + return false; +} + +bool CAlert::CheckSignature() const +{ + CPubKey key(Params().AlertKey()); + if (!key.Verify(Hash(vchMsg.begin(), vchMsg.end()), vchSig)) + return error("CAlert::CheckSignature() : verify signature failed"); + + // Now unserialize the data + CDataStream sMsg(vchMsg, SER_NETWORK, PROTOCOL_VERSION); + sMsg >> *(CUnsignedAlert*)this; + return true; +} + +CAlert CAlert::getAlertByHash(const uint256 &hash) +{ + CAlert retval; + { + LOCK(cs_mapAlerts); + map::iterator mi = mapAlerts.find(hash); + if(mi != mapAlerts.end()) + retval = mi->second; + } + return retval; +} + +bool CAlert::ProcessAlert(bool fThread) +{ + if (!CheckSignature()) + return false; + if (!IsInEffect()) + return false; + + // alert.nID=max is reserved for if the alert key is + // compromised. It must have a pre-defined message, + // must never expire, must apply to all versions, + // and must cancel all previous + // alerts or it will be ignored (so an attacker can't + // send an "everything is OK, don't panic" version that + // cannot be overridden): + int maxInt = std::numeric_limits::max(); + if (nID == maxInt) + { + if (!( + nExpiration == maxInt && + nCancel == (maxInt-1) && + nMinVer == 0 && + nMaxVer == maxInt && + setSubVer.empty() && + nPriority == maxInt && + strStatusBar == "URGENT: Alert key compromised, upgrade required" + )) + return false; + } + + { + LOCK(cs_mapAlerts); + // Cancel previous alerts + for (map::iterator mi = mapAlerts.begin(); mi != mapAlerts.end();) + { + const CAlert& alert = (*mi).second; + if (Cancels(alert)) + { + LogPrint("alert", "cancelling alert %d\n", alert.nID); + uiInterface.NotifyAlertChanged((*mi).first, CT_DELETED); + mapAlerts.erase(mi++); + } + else if (!alert.IsInEffect()) + { + LogPrint("alert", "expiring alert %d\n", alert.nID); + uiInterface.NotifyAlertChanged((*mi).first, CT_DELETED); + mapAlerts.erase(mi++); + } + else + mi++; + } + + // Check if this alert has been cancelled + BOOST_FOREACH(PAIRTYPE(const uint256, CAlert)& item, mapAlerts) + { + const CAlert& alert = item.second; + if (alert.Cancels(*this)) + { + LogPrint("alert", "alert already cancelled by %d\n", alert.nID); + return false; + } + } + + // Add to mapAlerts + mapAlerts.insert(make_pair(GetHash(), *this)); + // Notify UI and -alertnotify if it applies to me + if(AppliesToMe()) + { + uiInterface.NotifyAlertChanged(GetHash(), CT_NEW); + Notify(strStatusBar, fThread); + } + } + + LogPrint("alert", "accepted alert %d, AppliesToMe()=%d\n", nID, AppliesToMe()); + return true; +} + +void +CAlert::Notify(const std::string& strMessage, bool fThread) +{ + std::string strCmd = GetArg("-alertnotify", ""); + if (strCmd.empty()) return; + + // Alert text should be plain ascii coming from a trusted source, but to + // be safe we first strip anything not in safeChars, then add single quotes around + // the whole string before passing it to the shell: + std::string singleQuote("'"); + std::string safeStatus = SanitizeString(strMessage); + safeStatus = singleQuote+safeStatus+singleQuote; + boost::replace_all(strCmd, "%s", safeStatus); + + if (fThread) + boost::thread t(runCommand, strCmd); // thread runs free + else + runCommand(strCmd); +} diff --git a/src/structs/alert.h b/src/structs/alert.h new file mode 100644 index 00000000..59b543ab --- /dev/null +++ b/src/structs/alert.h @@ -0,0 +1,114 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2009-2013 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_ALERT_H +#define BITCOIN_ALERT_H + +#include "utils/serialize.h" +#include "utils/sync.h" + +#include +#include +#include +#include + +class CAlert; +class CNode; +class uint256; + +extern std::map mapAlerts; +extern CCriticalSection cs_mapAlerts; + +/** Alerts are for notifying old versions if they become too obsolete and + * need to upgrade. The message is displayed in the status bar. + * Alert messages are broadcast as a vector of signed data. Unserializing may + * not read the entire buffer if the alert is for a newer version, but older + * versions can still relay the original data. + */ +class CUnsignedAlert +{ +public: + int nVersion; + int64_t nRelayUntil; // when newer nodes stop relaying to newer nodes + int64_t nExpiration; + int nID; + int nCancel; + std::set setCancel; + int nMinVer; // lowest version inclusive + int nMaxVer; // highest version inclusive + std::set setSubVer; // empty matches all + int nPriority; + + // Actions + std::string strComment; + std::string strStatusBar; + std::string strReserved; + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(this->nVersion); + nVersion = this->nVersion; + READWRITE(nRelayUntil); + READWRITE(nExpiration); + READWRITE(nID); + READWRITE(nCancel); + READWRITE(setCancel); + READWRITE(nMinVer); + READWRITE(nMaxVer); + READWRITE(setSubVer); + READWRITE(nPriority); + + READWRITE(LIMITED_STRING(strComment, 65536)); + READWRITE(LIMITED_STRING(strStatusBar, 256)); + READWRITE(LIMITED_STRING(strReserved, 256)); + } + + void SetNull(); + + std::string ToString() const; +}; + +/** An alert is a combination of a serialized CUnsignedAlert and a signature. */ +class CAlert : public CUnsignedAlert +{ +public: + std::vector vchMsg; + std::vector vchSig; + + CAlert() + { + SetNull(); + } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(vchMsg); + READWRITE(vchSig); + } + + void SetNull(); + bool IsNull() const; + uint256 GetHash() const; + bool IsInEffect() const; + bool Cancels(const CAlert& alert) const; + bool AppliesTo(int nVersion, std::string strSubVerIn) const; + bool AppliesToMe() const; + bool RelayTo(CNode* pnode) const; + bool CheckSignature() const; + bool ProcessAlert(bool fThread = true); // fThread means run -alertnotify in a free-running thread + static void Notify(const std::string& strMessage, bool fThread); + + /* + * Get copy of (active) alert object by hash. Returns a null alert if it is not found. + */ + static CAlert getAlertByHash(const uint256 &hash); +}; + +#endif // BITCOIN_ALERT_H diff --git a/src/structs/amount.cpp b/src/structs/amount.cpp new file mode 100644 index 00000000..0503637b --- /dev/null +++ b/src/structs/amount.cpp @@ -0,0 +1,38 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "structs/amount.h" + +#include "utils/tinyformat.h" + +CFeeRate::CFeeRate(const CAmount& nFeePaid, size_t nSize) +{ + if (nSize > 0) + nSatoshisPerK = nFeePaid*1000/nSize; + else + nSatoshisPerK = 0; +} + +CAmount CFeeRate::GetFee(size_t nSize) const +{ + CAmount nFee = nSatoshisPerK*nSize / 1000; + + if (nFee == 0 && nSatoshisPerK > 0) + nFee = nSatoshisPerK; + + return nFee; +} + +std::string CFeeRate::ToString() const +{ +/* MCHN START */ + if(COIN == 0) + { + return strprintf("%d.%08d BTC/kB", nSatoshisPerK, nSatoshisPerK); + } +/* MCHN END */ + return strprintf("%d.%08d BTC/kB", nSatoshisPerK / COIN, nSatoshisPerK % COIN); +} diff --git a/src/structs/amount.h b/src/structs/amount.h new file mode 100644 index 00000000..b894f3fb --- /dev/null +++ b/src/structs/amount.h @@ -0,0 +1,61 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_AMOUNT_H +#define BITCOIN_AMOUNT_H + +#include "utils/serialize.h" + +#include +#include + +typedef int64_t CAmount; + +/* MCHN START */ +//static const CAmount COIN = 100000000; +//static const CAmount CENT = 1000000; + +/** No amount larger than this (in satoshi) is valid */ +//static const CAmount MAX_MONEY = 21000000 * COIN; +extern int64_t COIN; +extern int64_t CENT; +extern int64_t MAX_MONEY; +/* MCHN END */ + +inline bool MoneyRange(const CAmount& nValue) { return (nValue >= 0 && nValue <= MAX_MONEY); } + +/** Type-safe wrapper class to for fee rates + * (how much to pay based on transaction size) + */ +class CFeeRate +{ +private: + CAmount nSatoshisPerK; // unit is satoshis-per-1,000-bytes +public: + CFeeRate() : nSatoshisPerK(0) { } + explicit CFeeRate(const CAmount& _nSatoshisPerK): nSatoshisPerK(_nSatoshisPerK) { } + CFeeRate(const CAmount& nFeePaid, size_t nSize); + CFeeRate(const CFeeRate& other) { nSatoshisPerK = other.nSatoshisPerK; } + + CAmount GetFee(size_t size) const; // unit returned is satoshis + CAmount GetFeePerK() const { return GetFee(1000); } // satoshis-per-1000-bytes + + friend bool operator<(const CFeeRate& a, const CFeeRate& b) { return a.nSatoshisPerK < b.nSatoshisPerK; } + friend bool operator>(const CFeeRate& a, const CFeeRate& b) { return a.nSatoshisPerK > b.nSatoshisPerK; } + friend bool operator==(const CFeeRate& a, const CFeeRate& b) { return a.nSatoshisPerK == b.nSatoshisPerK; } + friend bool operator<=(const CFeeRate& a, const CFeeRate& b) { return a.nSatoshisPerK <= b.nSatoshisPerK; } + friend bool operator>=(const CFeeRate& a, const CFeeRate& b) { return a.nSatoshisPerK >= b.nSatoshisPerK; } + std::string ToString() const; + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(nSatoshisPerK); + } +}; + +#endif // BITCOIN_AMOUNT_H diff --git a/src/structs/base58.cpp b/src/structs/base58.cpp new file mode 100644 index 00000000..25f93c77 --- /dev/null +++ b/src/structs/base58.cpp @@ -0,0 +1,488 @@ +// Copyright (c) 2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "structs/base58.h" + +#include "structs/hash.h" +#include "structs/uint256.h" + +#include +#include +#include +#include +#include +#include +#include + +#include "multichain/multichain.h" + + +/** All alphanumeric characters except for "0", "I", "O", and "l" */ +static const char* pszBase58 = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz"; + +bool DecodeBase58(const char* psz, std::vector& vch) +{ + // Skip leading spaces. + while (*psz && isspace(*psz)) + psz++; + // Skip and count leading '1's. + int zeroes = 0; + while (*psz == '1') { + zeroes++; + psz++; + } + // Allocate enough space in big-endian base256 representation. + std::vector b256(strlen(psz) * 733 / 1000 + 1); // log(58) / log(256), rounded up. + // Process the characters. + while (*psz && !isspace(*psz)) { + // Decode base58 character + const char* ch = strchr(pszBase58, *psz); + if (ch == NULL) + return false; + // Apply "b256 = b256 * 58 + ch". + int carry = ch - pszBase58; + for (std::vector::reverse_iterator it = b256.rbegin(); it != b256.rend(); it++) { + carry += 58 * (*it); + *it = carry % 256; + carry /= 256; + } + assert(carry == 0); + psz++; + } + // Skip trailing spaces. + while (isspace(*psz)) + psz++; + if (*psz != 0) + return false; + // Skip leading zeroes in b256. + std::vector::iterator it = b256.begin(); + while (it != b256.end() && *it == 0) + it++; + // Copy result into output vector. + vch.reserve(zeroes + (b256.end() - it)); + vch.assign(zeroes, 0x00); + while (it != b256.end()) + vch.push_back(*(it++)); + return true; +} + +std::string EncodeBase58(const unsigned char* pbegin, const unsigned char* pend) +{ + // Skip & count leading zeroes. + int zeroes = 0; + while (pbegin != pend && *pbegin == 0) { + pbegin++; + zeroes++; + } + // Allocate enough space in big-endian base58 representation. + std::vector b58((pend - pbegin) * 138 / 100 + 1); // log(256) / log(58), rounded up. + // Process the bytes. + while (pbegin != pend) { + int carry = *pbegin; + // Apply "b58 = b58 * 256 + ch". + for (std::vector::reverse_iterator it = b58.rbegin(); it != b58.rend(); it++) { + carry += 256 * (*it); + *it = carry % 58; + carry /= 58; + } + assert(carry == 0); + pbegin++; + } + // Skip leading zeroes in base58 result. + std::vector::iterator it = b58.begin(); + while (it != b58.end() && *it == 0) + it++; + // Translate the result into a string. + std::string str; + str.reserve(zeroes + (b58.end() - it)); + str.assign(zeroes, '1'); + while (it != b58.end()) + str += pszBase58[*(it++)]; + return str; +} + +std::string EncodeBase58(const std::vector& vch) +{ + return EncodeBase58(&vch[0], &vch[0] + vch.size()); +} + +bool DecodeBase58(const std::string& str, std::vector& vchRet) +{ + return DecodeBase58(str.c_str(), vchRet); +} + +std::string EncodeBase58Check(const std::vector& vchIn) +{ + // add 4-byte hash check to the end + std::vector vch(vchIn); + uint256 hash = Hash(vch.begin(), vch.end()); + +/* MCHN START */ + int32_t checksum=(int32_t)mc_GetLE(&hash,4); + checksum ^= (int32_t)mc_gState->m_NetworkParams->GetInt64Param("addresschecksumvalue"); + vch.insert(vch.end(), (unsigned char*)&checksum, (unsigned char*)&checksum + 4); +// vch.insert(vch.end(), (unsigned char*)&hash, (unsigned char*)&hash + 4); +/* MCHN END */ +// mc_DumpSize("TS",&vch[0],vch.size(),vch.size()); + return EncodeBase58(vch); +} + +bool DecodeBase58Check(const char* psz, std::vector& vchRet) +{ + if (!DecodeBase58(psz, vchRet) || + (vchRet.size() < 4)) { + vchRet.clear(); + return false; + } + // re-calculate the checksum, insure it matches the included 4-byte checksum + uint256 hash = Hash(vchRet.begin(), vchRet.end() - 4); + +/* MCHN START */ + int32_t checksum=(int32_t)mc_GetLE(&hash,4); + checksum ^= (int32_t)mc_gState->m_NetworkParams->GetInt64Param("addresschecksumvalue"); + + if (memcmp((unsigned char*)&checksum, &vchRet.end()[-4], 4) != 0) { +// if (memcmp(&hash, &vchRet.end()[-4], 4) != 0) { + vchRet.clear(); + return false; + } +/* MCHN END */ + vchRet.resize(vchRet.size() - 4); + return true; +} + +bool DecodeBase58Check(const std::string& str, std::vector& vchRet) +{ + return DecodeBase58Check(str.c_str(), vchRet); +} + +CBase58Data::CBase58Data() +{ + vchVersion.clear(); + vchData.clear(); +} + +void CBase58Data::SetData(const std::vector& vchVersionIn, const void* pdata, size_t nSize) +{ + vchVersion = vchVersionIn; + vchData.resize(nSize); + if (!vchData.empty()) + memcpy(&vchData[0], pdata, nSize); +} + +void CBase58Data::SetData(const std::vector& vchVersionIn, const unsigned char* pbegin, const unsigned char* pend) +{ + SetData(vchVersionIn, (void*)pbegin, pend - pbegin); +} + +bool CBase58Data::SetString(const char* psz, unsigned int nVersionBytes) +{ + std::vector vchTemp; + bool rc58 = DecodeBase58Check(psz, vchTemp); + if ((!rc58) || (vchTemp.size() < nVersionBytes)) { + vchData.clear(); + vchVersion.clear(); + return false; + } + +/* MCHN START */ + +/* + vchVersion.assign(vchTemp.begin(), vchTemp.begin() + nVersionBytes); + vchData.resize(vchTemp.size() - nVersionBytes); + if (!vchData.empty()) + memcpy(&vchData[0], &vchTemp[nVersionBytes], vchData.size()); +*/ + + int shift=(vchTemp.size() - nVersionBytes) / nVersionBytes; + vchVersion.resize(nVersionBytes); + vchData.resize(vchTemp.size() - nVersionBytes); + for(int i=0;i<(int)nVersionBytes;i++) + { + vchVersion[i]=vchTemp[i*(shift+1)]; + int size=shift; + if(i == (int)(nVersionBytes-1)) + { + size=(vchTemp.size() - nVersionBytes)-i*shift; + } + memcpy(&vchData[i*shift],&vchTemp[i*(shift+1)+1],size); + } + + +/* MCHN END */ + OPENSSL_cleanse(&vchTemp[0], vchData.size()); + return true; +} + +bool CBase58Data::SetString(const std::string& str) +{ + return SetString(str.c_str(),Params().Base58Prefix(CChainParams::PUBKEY_ADDRESS).size());// MCHN, 1 (default) in original code +} + +std::string CBase58Data::ToString() const +{ +/* MCHN START */ +/* + std::vector vch = vchVersion; + vch.insert(vch.end(), vchData.begin(), vchData.end()); + */ + + std::vector vch; + int nVersionBytes=vchVersion.size(); + + int shift=vchData.size() / nVersionBytes; + vch.resize(vchData.size() + nVersionBytes); + for(int i=0;i& vchVersion) +{ + std::vector vch; + int nVersionBytes=vchVersion.size(); + int nDataBytes=20; + int nHashBytes=4; + unsigned char data[20]; + char test[100]; + char res[100]; + + int shift=nDataBytes / nVersionBytes; + vch.resize(nDataBytes + nVersionBytes + nHashBytes); + int p; + for(int i=2;i nDataBytes + nVersionBytes + nHashBytes) + { + res[strlen(res)-1]=0x00; + DecodeBase58(res,vch); + } + + p=0; + while(p strlen(res)) + { + res[strlen(res)+1]=0x00; + res[strlen(res)]='X'; + } + if(strlen(test) < strlen(res)) + { + res[strlen(res)-1]=0x00; + } + } + int j=0; + while( (j<(int)strlen(res)) && (res[j] == test[j]) ) + { + j++; + } + int k=0; + while( (k<3) && (j<(int)strlen(res)) && ((vch[p] != vchVersion[p / (shift+1)]) || (k ==0))) + { + res[j]=test[j]; + DecodeBase58(res,vch); + k++; + j++; + } + } + } + p++; + } + +// strcpy(test,EncodeBase58(vch).c_str()); + + + for(int i=0;i<(int)nVersionBytes;i++) + { + int size=shift; + if(i == (int)(nVersionBytes-1)) + { + size=nDataBytes-i*shift; + } + memcpy(data+i*shift,&vch[i*(shift+1)+1],size); + } + + CKeyID kBurn; + memcpy(&kBurn,data,nDataBytes); + + CBitcoinAddress ba; + ba.Set(kBurn,vchVersion); + return ba.ToString(); +} + +int CBase58Data::CompareTo(const CBase58Data& b58) const +{ + if (vchVersion < b58.vchVersion) + return -1; + if (vchVersion > b58.vchVersion) + return 1; + if (vchData < b58.vchData) + return -1; + if (vchData > b58.vchData) + return 1; + return 0; +} + +namespace +{ +class CBitcoinAddressVisitor : public boost::static_visitor +{ +private: + CBitcoinAddress* addr; + +public: + CBitcoinAddressVisitor(CBitcoinAddress* addrIn) : addr(addrIn) {} + + bool operator()(const CKeyID& id) const { return addr->Set(id); } + bool operator()(const CScriptID& id) const { return addr->Set(id); } + bool operator()(const CNoDestination& no) const { return false; } +}; + +} // anon namespace + +/* MCHN START */ +bool CBitcoinAddress::Set(const CKeyID &id,const std::vector& vchV) +{ + SetData(vchV, &id, 20); + return true; +} +/* MCHN END */ + + +bool CBitcoinAddress::Set(const CKeyID& id) +{ + SetData(Params().Base58Prefix(CChainParams::PUBKEY_ADDRESS), &id, 20); + return true; +} + +bool CBitcoinAddress::Set(const CScriptID& id) +{ + SetData(Params().Base58Prefix(CChainParams::SCRIPT_ADDRESS), &id, 20); + return true; +} + +bool CBitcoinAddress::Set(const CTxDestination& dest) +{ + return boost::apply_visitor(CBitcoinAddressVisitor(this), dest); +} + +bool CBitcoinAddress::IsValid() const +{ + return IsValid(Params()); +} + +bool CBitcoinAddress::IsValid(const CChainParams& params) const +{ + bool fCorrectSize = vchData.size() == 20; + bool fKnownVersion = vchVersion == params.Base58Prefix(CChainParams::PUBKEY_ADDRESS) || + vchVersion == params.Base58Prefix(CChainParams::SCRIPT_ADDRESS); + return fCorrectSize && fKnownVersion; +} + +CTxDestination CBitcoinAddress::Get() const +{ + if (!IsValid()) + return CNoDestination(); + uint160 id; + memcpy(&id, &vchData[0], 20); + if (vchVersion == Params().Base58Prefix(CChainParams::PUBKEY_ADDRESS)) + return CKeyID(id); + else if (vchVersion == Params().Base58Prefix(CChainParams::SCRIPT_ADDRESS)) + return CScriptID(id); + else + return CNoDestination(); +} + +bool CBitcoinAddress::GetKeyID(CKeyID& keyID) const +{ + if (!IsValid() || vchVersion != Params().Base58Prefix(CChainParams::PUBKEY_ADDRESS)) + return false; + uint160 id; + memcpy(&id, &vchData[0], 20); + keyID = CKeyID(id); + return true; +} + +bool CBitcoinAddress::GetScriptID(CScriptID& scriptID) const +{ + if (!IsValid() || vchVersion != Params().Base58Prefix(CChainParams::SCRIPT_ADDRESS)) + return false; + uint160 id; + memcpy(&id, &vchData[0], 20); + scriptID = CScriptID(id); + return true; +} + +bool CBitcoinAddress::IsScript() const +{ + return IsValid() && vchVersion == Params().Base58Prefix(CChainParams::SCRIPT_ADDRESS); +} + +void CBitcoinSecret::SetKey(const CKey& vchSecret) +{ + assert(vchSecret.IsValid()); + SetData(Params().Base58Prefix(CChainParams::SECRET_KEY), vchSecret.begin(), vchSecret.size()); + if (vchSecret.IsCompressed()) + vchData.push_back(1); +} + +CKey CBitcoinSecret::GetKey() +{ + CKey ret; + assert(vchData.size() >= 32); + ret.Set(vchData.begin(), vchData.begin() + 32, vchData.size() > 32 && vchData[32] == 1); + return ret; +} + +bool CBitcoinSecret::IsValid() const +{ + bool fExpectedFormat = vchData.size() == 32 || (vchData.size() == 33 && vchData[32] == 1); + bool fCorrectVersion = vchVersion == Params().Base58Prefix(CChainParams::SECRET_KEY); + return fExpectedFormat && fCorrectVersion; +} + +bool CBitcoinSecret::SetString(const char* pszSecret) +{ + return CBase58Data::SetString(pszSecret,Params().Base58Prefix(CChainParams::SECRET_KEY).size()) && IsValid(); // MCHN 1 (default) in original code +} + +bool CBitcoinSecret::SetString(const std::string& strSecret) +{ + return SetString(strSecret.c_str()); +} diff --git a/src/structs/base58.h b/src/structs/base58.h new file mode 100644 index 00000000..4e3c379b --- /dev/null +++ b/src/structs/base58.h @@ -0,0 +1,165 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +/** + * Why base-58 instead of standard base-64 encoding? + * - Don't want 0OIl characters that look the same in some fonts and + * could be used to create visually identical looking account numbers. + * - A string with non-alphanumeric characters is not as easily accepted as an account number. + * - E-mail usually won't line-break if there's no punctuation to break at. + * - Double-clicking selects the whole number as one word if it's all alphanumeric. + */ +#ifndef BITCOIN_BASE58_H +#define BITCOIN_BASE58_H + +#include "chainparams/chainparams.h" +#include "keys/key.h" +#include "keys/pubkey.h" +#include "script/script.h" +#include "script/standard.h" + +#include +#include + +/** + * Encode a byte sequence as a base58-encoded string. + * pbegin and pend cannot be NULL, unless both are. + */ +std::string EncodeBase58(const unsigned char* pbegin, const unsigned char* pend); + +/** + * Encode a byte vector as a base58-encoded string + */ +std::string EncodeBase58(const std::vector& vch); + +/** + * Decode a base58-encoded string (psz) into a byte vector (vchRet). + * return true if decoding is successful. + * psz cannot be NULL. + */ +bool DecodeBase58(const char* psz, std::vector& vchRet); + +/** + * Decode a base58-encoded string (str) into a byte vector (vchRet). + * return true if decoding is successful. + */ +bool DecodeBase58(const std::string& str, std::vector& vchRet); + +/** + * Encode a byte vector into a base58-encoded string, including checksum + */ +std::string EncodeBase58Check(const std::vector& vchIn); + +/** + * Decode a base58-encoded string (psz) that includes a checksum into a byte + * vector (vchRet), return true if decoding is successful + */ +inline bool DecodeBase58Check(const char* psz, std::vector& vchRet); + +/** + * Decode a base58-encoded string (str) that includes a checksum into a byte + * vector (vchRet), return true if decoding is successful + */ +inline bool DecodeBase58Check(const std::string& str, std::vector& vchRet); + +/** + * Base class for all base58-encoded data + */ +class CBase58Data +{ +protected: + //! the version byte(s) + std::vector vchVersion; + + //! the actually encoded data + typedef std::vector > vector_uchar; + vector_uchar vchData; + + CBase58Data(); + void SetData(const std::vector &vchVersionIn, const void* pdata, size_t nSize); + void SetData(const std::vector &vchVersionIn, const unsigned char *pbegin, const unsigned char *pend); + +public: + bool SetString(const char* psz, unsigned int nVersionBytes = 1); + bool SetString(const std::string& str); + std::string ToString() const; + int CompareTo(const CBase58Data& b58) const; + + bool operator==(const CBase58Data& b58) const { return CompareTo(b58) == 0; } + bool operator<=(const CBase58Data& b58) const { return CompareTo(b58) <= 0; } + bool operator>=(const CBase58Data& b58) const { return CompareTo(b58) >= 0; } + bool operator< (const CBase58Data& b58) const { return CompareTo(b58) < 0; } + bool operator> (const CBase58Data& b58) const { return CompareTo(b58) > 0; } +}; + +/** base58-encoded Bitcoin addresses. + * Public-key-hash-addresses have version 0 (or 111 testnet). + * The data vector contains RIPEMD160(SHA256(pubkey)), where pubkey is the serialized public key. + * Script-hash-addresses have version 5 (or 196 testnet). + * The data vector contains RIPEMD160(SHA256(cscript)), where cscript is the serialized redemption script. + */ +class CBitcoinAddress : public CBase58Data { +public: + bool Set(const CKeyID &id); + bool Set(const CKeyID &id,const std::vector& vchVersion); + bool Set(const CScriptID &id); + bool Set(const CTxDestination &dest); + bool IsValid() const; + bool IsValid(const CChainParams ¶ms) const; + + CBitcoinAddress() {} + CBitcoinAddress(const CTxDestination &dest) { Set(dest); } + CBitcoinAddress(const std::string& strAddress) { SetString(strAddress); } + CBitcoinAddress(const char* pszAddress) { SetString(pszAddress); } + + CTxDestination Get() const; + bool GetKeyID(CKeyID &keyID) const; + bool GetScriptID(CScriptID &scriptID) const; + bool IsScript() const; +}; + +/** + * A base58-encoded secret key + */ +class CBitcoinSecret : public CBase58Data +{ +public: + void SetKey(const CKey& vchSecret); + CKey GetKey(); + bool IsValid() const; + bool SetString(const char* pszSecret); + bool SetString(const std::string& strSecret); + + CBitcoinSecret(const CKey& vchSecret) { SetKey(vchSecret); } + CBitcoinSecret() {} +}; + +template class CBitcoinExtKeyBase : public CBase58Data +{ +public: + void SetKey(const K &key) { + unsigned char vch[Size]; + key.Encode(vch); + SetData(Params().Base58Prefix(Type), vch, vch+Size); + } + + K GetKey() { + K ret; + ret.Decode(&vchData[0], &vchData[Size]); + return ret; + } + + CBitcoinExtKeyBase(const K &key) { + SetKey(key); + } + + CBitcoinExtKeyBase() {} +}; + +typedef CBitcoinExtKeyBase CBitcoinExtKey; +typedef CBitcoinExtKeyBase CBitcoinExtPubKey; + +#endif // BITCOIN_BASE58_H diff --git a/src/structs/bloom.cpp b/src/structs/bloom.cpp new file mode 100644 index 00000000..6a073abf --- /dev/null +++ b/src/structs/bloom.cpp @@ -0,0 +1,200 @@ +// Copyright (c) 2012-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "structs/bloom.h" + +#include "primitives/transaction.h" +#include "structs/hash.h" +#include "script/script.h" +#include "script/standard.h" +#include "utils/streams.h" + +#include +#include + +#include + +#define LN2SQUARED 0.4804530139182014246671025263266649717305529515945455 +#define LN2 0.6931471805599453094172321214581765680755001343602552 + +using namespace std; + +CBloomFilter::CBloomFilter(unsigned int nElements, double nFPRate, unsigned int nTweakIn, unsigned char nFlagsIn) : +/** + * The ideal size for a bloom filter with a given number of elements and false positive rate is: + * - nElements * log(fp rate) / ln(2)^2 + * We ignore filter parameters which will create a bloom filter larger than the protocol limits + */ +vData(min((unsigned int)(-1 / LN2SQUARED * nElements * log(nFPRate)), MAX_BLOOM_FILTER_SIZE * 8) / 8), +/** + * The ideal number of hash functions is filter size * ln(2) / number of elements + * Again, we ignore filter parameters which will create a bloom filter with more hash functions than the protocol limits + * See https://en.wikipedia.org/wiki/Bloom_filter for an explanation of these formulas + */ +isFull(false), +isEmpty(false), +nHashFuncs(min((unsigned int)(vData.size() * 8 / nElements * LN2), MAX_HASH_FUNCS)), +nTweak(nTweakIn), +nFlags(nFlagsIn) +{ +} + +inline unsigned int CBloomFilter::Hash(unsigned int nHashNum, const std::vector& vDataToHash) const +{ + // 0xFBA4C795 chosen as it guarantees a reasonable bit difference between nHashNum values. + return MurmurHash3(nHashNum * 0xFBA4C795 + nTweak, vDataToHash) % (vData.size() * 8); +} + +void CBloomFilter::insert(const vector& vKey) +{ + if (isFull) + return; + for (unsigned int i = 0; i < nHashFuncs; i++) + { + unsigned int nIndex = Hash(i, vKey); + // Sets bit nIndex of vData + vData[nIndex >> 3] |= (1 << (7 & nIndex)); + } + isEmpty = false; +} + +void CBloomFilter::insert(const COutPoint& outpoint) +{ + CDataStream stream(SER_NETWORK, PROTOCOL_VERSION); + stream << outpoint; + vector data(stream.begin(), stream.end()); + insert(data); +} + +void CBloomFilter::insert(const uint256& hash) +{ + vector data(hash.begin(), hash.end()); + insert(data); +} + +bool CBloomFilter::contains(const vector& vKey) const +{ + if (isFull) + return true; + if (isEmpty) + return false; + for (unsigned int i = 0; i < nHashFuncs; i++) + { + unsigned int nIndex = Hash(i, vKey); + // Checks bit nIndex of vData + if (!(vData[nIndex >> 3] & (1 << (7 & nIndex)))) + return false; + } + return true; +} + +bool CBloomFilter::contains(const COutPoint& outpoint) const +{ + CDataStream stream(SER_NETWORK, PROTOCOL_VERSION); + stream << outpoint; + vector data(stream.begin(), stream.end()); + return contains(data); +} + +bool CBloomFilter::contains(const uint256& hash) const +{ + vector data(hash.begin(), hash.end()); + return contains(data); +} + +void CBloomFilter::clear() +{ + vData.assign(vData.size(),0); + isFull = false; + isEmpty = true; +} + +bool CBloomFilter::IsWithinSizeConstraints() const +{ + return vData.size() <= MAX_BLOOM_FILTER_SIZE && nHashFuncs <= MAX_HASH_FUNCS; +} + +bool CBloomFilter::IsRelevantAndUpdate(const CTransaction& tx) +{ + bool fFound = false; + // Match if the filter contains the hash of tx + // for finding tx when they appear in a block + if (isFull) + return true; + if (isEmpty) + return false; + const uint256& hash = tx.GetHash(); + if (contains(hash)) + fFound = true; + + for (unsigned int i = 0; i < tx.vout.size(); i++) + { + const CTxOut& txout = tx.vout[i]; + // Match if the filter contains any arbitrary script data element in any scriptPubKey in tx + // If this matches, also add the specific output that was matched. + // This means clients don't have to update the filter themselves when a new relevant tx + // is discovered in order to find spending transactions, which avoids round-tripping and race conditions. + CScript::const_iterator pc = txout.scriptPubKey.begin(); + vector data; + while (pc < txout.scriptPubKey.end()) + { + opcodetype opcode; + if (!txout.scriptPubKey.GetOp(pc, opcode, data)) + break; + if (data.size() != 0 && contains(data)) + { + fFound = true; + if ((nFlags & BLOOM_UPDATE_MASK) == BLOOM_UPDATE_ALL) + insert(COutPoint(hash, i)); + else if ((nFlags & BLOOM_UPDATE_MASK) == BLOOM_UPDATE_P2PUBKEY_ONLY) + { + txnouttype type; + vector > vSolutions; + if (Solver(txout.scriptPubKey, type, vSolutions) && + (type == TX_PUBKEY || type == TX_MULTISIG)) + insert(COutPoint(hash, i)); + } + break; + } + } + } + + if (fFound) + return true; + + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + // Match if the filter contains an outpoint tx spends + if (contains(txin.prevout)) + return true; + + // Match if the filter contains any arbitrary script data element in any scriptSig in tx + CScript::const_iterator pc = txin.scriptSig.begin(); + vector data; + while (pc < txin.scriptSig.end()) + { + opcodetype opcode; + if (!txin.scriptSig.GetOp(pc, opcode, data)) + break; + if (data.size() != 0 && contains(data)) + return true; + } + } + + return false; +} + +void CBloomFilter::UpdateEmptyFull() +{ + bool full = true; + bool empty = true; + for (unsigned int i = 0; i < vData.size(); i++) + { + full &= vData[i] == 0xff; + empty &= vData[i] == 0; + } + isFull = full; + isEmpty = empty; +} diff --git a/src/structs/bloom.h b/src/structs/bloom.h new file mode 100644 index 00000000..29b85f51 --- /dev/null +++ b/src/structs/bloom.h @@ -0,0 +1,101 @@ +// Copyright (c) 2012-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_BLOOM_H +#define BITCOIN_BLOOM_H + +#include "utils/serialize.h" + +#include + +class COutPoint; +class CTransaction; +class uint256; + +//! 20,000 items with fp rate < 0.1% or 10,000 items and <0.0001% +static const unsigned int MAX_BLOOM_FILTER_SIZE = 36000; // bytes +static const unsigned int MAX_HASH_FUNCS = 50; + +/** + * First two bits of nFlags control how much IsRelevantAndUpdate actually updates + * The remaining bits are reserved + */ +enum bloomflags +{ + BLOOM_UPDATE_NONE = 0, + BLOOM_UPDATE_ALL = 1, + // Only adds outpoints to the filter if the output is a pay-to-pubkey/pay-to-multisig script + BLOOM_UPDATE_P2PUBKEY_ONLY = 2, + BLOOM_UPDATE_MASK = 3, +}; + +/** + * BloomFilter is a probabilistic filter which SPV clients provide + * so that we can filter the transactions we sends them. + * + * This allows for significantly more efficient transaction and block downloads. + * + * Because bloom filters are probabilistic, an SPV node can increase the false- + * positive rate, making us send them transactions which aren't actually theirs, + * allowing clients to trade more bandwidth for more privacy by obfuscating which + * keys are owned by them. + */ +class CBloomFilter +{ +private: + std::vector vData; + bool isFull; + bool isEmpty; + unsigned int nHashFuncs; + unsigned int nTweak; + unsigned char nFlags; + + unsigned int Hash(unsigned int nHashNum, const std::vector& vDataToHash) const; + +public: + /** + * Creates a new bloom filter which will provide the given fp rate when filled with the given number of elements + * Note that if the given parameters will result in a filter outside the bounds of the protocol limits, + * the filter created will be as close to the given parameters as possible within the protocol limits. + * This will apply if nFPRate is very low or nElements is unreasonably high. + * nTweak is a constant which is added to the seed value passed to the hash function + * It should generally always be a random value (and is largely only exposed for unit testing) + * nFlags should be one of the BLOOM_UPDATE_* enums (not _MASK) + */ + CBloomFilter(unsigned int nElements, double nFPRate, unsigned int nTweak, unsigned char nFlagsIn); + CBloomFilter() : isFull(true), isEmpty(false), nHashFuncs(0), nTweak(0), nFlags(0) {} + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(vData); + READWRITE(nHashFuncs); + READWRITE(nTweak); + READWRITE(nFlags); + } + + void insert(const std::vector& vKey); + void insert(const COutPoint& outpoint); + void insert(const uint256& hash); + + bool contains(const std::vector& vKey) const; + bool contains(const COutPoint& outpoint) const; + bool contains(const uint256& hash) const; + + void clear(); + + //! True if the size is <= MAX_BLOOM_FILTER_SIZE and the number of hash functions is <= MAX_HASH_FUNCS + //! (catch a filter which was just deserialized which was too big) + bool IsWithinSizeConstraints() const; + + //! Also adds any outputs which match the filter to the filter (to match their spending txes) + bool IsRelevantAndUpdate(const CTransaction& tx); + + //! Checks for empty and full filters to avoid wasting cpu + void UpdateEmptyFull(); +}; + +#endif // BITCOIN_BLOOM_H diff --git a/src/structs/hash.cpp b/src/structs/hash.cpp new file mode 100644 index 00000000..b1a5887a --- /dev/null +++ b/src/structs/hash.cpp @@ -0,0 +1,81 @@ +// Copyright (c) 2013-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "structs/hash.h" +#include "crypto/hmac_sha512.h" + +inline uint32_t ROTL32(uint32_t x, int8_t r) +{ + return (x << r) | (x >> (32 - r)); +} + +unsigned int MurmurHash3(unsigned int nHashSeed, const std::vector& vDataToHash) +{ + // The following is MurmurHash3 (x86_32), see http://code.google.com/p/smhasher/source/browse/trunk/MurmurHash3.cpp + uint32_t h1 = nHashSeed; + if (vDataToHash.size() > 0) + { + const uint32_t c1 = 0xcc9e2d51; + const uint32_t c2 = 0x1b873593; + + const int nblocks = vDataToHash.size() / 4; + + //---------- + // body + const uint32_t* blocks = (const uint32_t*)(&vDataToHash[0] + nblocks * 4); + + for (int i = -nblocks; i; i++) { + uint32_t k1 = blocks[i]; + + k1 *= c1; + k1 = ROTL32(k1, 15); + k1 *= c2; + + h1 ^= k1; + h1 = ROTL32(h1, 13); + h1 = h1 * 5 + 0xe6546b64; + } + + //---------- + // tail + const uint8_t* tail = (const uint8_t*)(&vDataToHash[0] + nblocks * 4); + + uint32_t k1 = 0; + + switch (vDataToHash.size() & 3) { + case 3: + k1 ^= tail[2] << 16; + case 2: + k1 ^= tail[1] << 8; + case 1: + k1 ^= tail[0]; + k1 *= c1; + k1 = ROTL32(k1, 15); + k1 *= c2; + h1 ^= k1; + }; + } + + //---------- + // finalization + h1 ^= vDataToHash.size(); + h1 ^= h1 >> 16; + h1 *= 0x85ebca6b; + h1 ^= h1 >> 13; + h1 *= 0xc2b2ae35; + h1 ^= h1 >> 16; + + return h1; +} + +void BIP32Hash(const ChainCode &chainCode, unsigned int nChild, unsigned char header, const unsigned char data[32], unsigned char output[64]) +{ + unsigned char num[4]; + num[0] = (nChild >> 24) & 0xFF; + num[1] = (nChild >> 16) & 0xFF; + num[2] = (nChild >> 8) & 0xFF; + num[3] = (nChild >> 0) & 0xFF; + CHMAC_SHA512(chainCode.begin(), chainCode.size()).Write(&header, 1).Write(data, 32).Write(num, 4).Finalize(output); +} diff --git a/src/structs/hash.h b/src/structs/hash.h new file mode 100644 index 00000000..6b394d6a --- /dev/null +++ b/src/structs/hash.h @@ -0,0 +1,167 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2013 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_HASH_H +#define BITCOIN_HASH_H + +#include "crypto/ripemd160.h" +#include "crypto/sha256.h" +#include "utils/serialize.h" +#include "structs/uint256.h" +#include "version/bcversion.h" + +#include + +typedef uint256 ChainCode; + + /** A hasher class for Bitcoin's 256-bit hash (double SHA-256). */ +class CHash256 { +private: + CSHA256 sha; +public: + static const size_t OUTPUT_SIZE = CSHA256::OUTPUT_SIZE; + + void Finalize(unsigned char hash[OUTPUT_SIZE]) { + unsigned char buf[sha.OUTPUT_SIZE]; + sha.Finalize(buf); + sha.Reset().Write(buf, sha.OUTPUT_SIZE).Finalize(hash); + } + + CHash256& Write(const unsigned char *data, size_t len) { + sha.Write(data, len); + return *this; + } + + CHash256& Reset() { + sha.Reset(); + return *this; + } +}; + +/** A hasher class for Bitcoin's 160-bit hash (SHA-256 + RIPEMD-160). */ +class CHash160 { +private: + CSHA256 sha; +public: + static const size_t OUTPUT_SIZE = CRIPEMD160::OUTPUT_SIZE; + + void Finalize(unsigned char hash[OUTPUT_SIZE]) { + unsigned char buf[sha.OUTPUT_SIZE]; + sha.Finalize(buf); + CRIPEMD160().Write(buf, sha.OUTPUT_SIZE).Finalize(hash); + } + + CHash160& Write(const unsigned char *data, size_t len) { + sha.Write(data, len); + return *this; + } + + CHash160& Reset() { + sha.Reset(); + return *this; + } +}; + +/** Compute the 256-bit hash of an object. */ +template +inline uint256 Hash(const T1 pbegin, const T1 pend) +{ + static const unsigned char pblank[1] = {}; + uint256 result; + CHash256().Write(pbegin == pend ? pblank : (const unsigned char*)&pbegin[0], (pend - pbegin) * sizeof(pbegin[0])) + .Finalize((unsigned char*)&result); + return result; +} + +/** Compute the 256-bit hash of the concatenation of two objects. */ +template +inline uint256 Hash(const T1 p1begin, const T1 p1end, + const T2 p2begin, const T2 p2end) { + static const unsigned char pblank[1] = {}; + uint256 result; + CHash256().Write(p1begin == p1end ? pblank : (const unsigned char*)&p1begin[0], (p1end - p1begin) * sizeof(p1begin[0])) + .Write(p2begin == p2end ? pblank : (const unsigned char*)&p2begin[0], (p2end - p2begin) * sizeof(p2begin[0])) + .Finalize((unsigned char*)&result); + return result; +} + +/** Compute the 256-bit hash of the concatenation of three objects. */ +template +inline uint256 Hash(const T1 p1begin, const T1 p1end, + const T2 p2begin, const T2 p2end, + const T3 p3begin, const T3 p3end) { + static const unsigned char pblank[1] = {}; + uint256 result; + CHash256().Write(p1begin == p1end ? pblank : (const unsigned char*)&p1begin[0], (p1end - p1begin) * sizeof(p1begin[0])) + .Write(p2begin == p2end ? pblank : (const unsigned char*)&p2begin[0], (p2end - p2begin) * sizeof(p2begin[0])) + .Write(p3begin == p3end ? pblank : (const unsigned char*)&p3begin[0], (p3end - p3begin) * sizeof(p3begin[0])) + .Finalize((unsigned char*)&result); + return result; +} + +/** Compute the 160-bit hash an object. */ +template +inline uint160 Hash160(const T1 pbegin, const T1 pend) +{ + static unsigned char pblank[1] = {}; + uint160 result; + CHash160().Write(pbegin == pend ? pblank : (const unsigned char*)&pbegin[0], (pend - pbegin) * sizeof(pbegin[0])) + .Finalize((unsigned char*)&result); + return result; +} + +/** Compute the 160-bit hash of a vector. */ +inline uint160 Hash160(const std::vector& vch) +{ + return Hash160(vch.begin(), vch.end()); +} + +/** A writer stream (for serialization) that computes a 256-bit hash. */ +class CHashWriter +{ +private: + CHash256 ctx; + +public: + int nType; + int nVersion; + + CHashWriter(int nTypeIn, int nVersionIn) : nType(nTypeIn), nVersion(nVersionIn) {} + + CHashWriter& write(const char *pch, size_t size) { + ctx.Write((const unsigned char*)pch, size); + return (*this); + } + + // invalidates the object + uint256 GetHash() { + uint256 result; + ctx.Finalize((unsigned char*)&result); + return result; + } + + template + CHashWriter& operator<<(const T& obj) { + // Serialize to this stream + ::Serialize(*this, obj, nType, nVersion); + return (*this); + } +}; + +/** Compute the 256-bit hash of an object's serialization. */ +template +uint256 SerializeHash(const T& obj, int nType=SER_GETHASH, int nVersion=PROTOCOL_VERSION) +{ + CHashWriter ss(nType, nVersion); + ss << obj; + return ss.GetHash(); +} + +unsigned int MurmurHash3(unsigned int nHashSeed, const std::vector& vDataToHash); + +void BIP32Hash(const ChainCode &chainCode, unsigned int nChild, unsigned char header, const unsigned char data[32], unsigned char output[64]); + +#endif // BITCOIN_HASH_H diff --git a/src/structs/limitedmap.h b/src/structs/limitedmap.h new file mode 100644 index 00000000..73270204 --- /dev/null +++ b/src/structs/limitedmap.h @@ -0,0 +1,95 @@ +// Copyright (c) 2012-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_LIMITEDMAP_H +#define BITCOIN_LIMITEDMAP_H + +#include +#include + +/** STL-like map container that only keeps the N elements with the highest value. */ +template +class limitedmap +{ +public: + typedef K key_type; + typedef V mapped_type; + typedef std::pair value_type; + typedef typename std::map::const_iterator const_iterator; + typedef typename std::map::size_type size_type; + +protected: + std::map map; + typedef typename std::map::iterator iterator; + std::multimap rmap; + typedef typename std::multimap::iterator rmap_iterator; + size_type nMaxSize; + +public: + limitedmap(size_type nMaxSizeIn = 0) { nMaxSize = nMaxSizeIn; } + const_iterator begin() const { return map.begin(); } + const_iterator end() const { return map.end(); } + size_type size() const { return map.size(); } + bool empty() const { return map.empty(); } + const_iterator find(const key_type& k) const { return map.find(k); } + size_type count(const key_type& k) const { return map.count(k); } + void insert(const value_type& x) + { + std::pair ret = map.insert(x); + if (ret.second) { + if (nMaxSize && map.size() == nMaxSize) { + map.erase(rmap.begin()->second); + rmap.erase(rmap.begin()); + } + rmap.insert(make_pair(x.second, ret.first)); + } + return; + } + void erase(const key_type& k) + { + iterator itTarget = map.find(k); + if (itTarget == map.end()) + return; + std::pair itPair = rmap.equal_range(itTarget->second); + for (rmap_iterator it = itPair.first; it != itPair.second; ++it) + if (it->second == itTarget) { + rmap.erase(it); + map.erase(itTarget); + return; + } + // Shouldn't ever get here + assert(0); + } + void update(const_iterator itIn, const mapped_type& v) + { + // TODO: When we switch to C++11, use map.erase(itIn, itIn) to get the non-const iterator. + iterator itTarget = map.find(itIn->first); + if (itTarget == map.end()) + return; + std::pair itPair = rmap.equal_range(itTarget->second); + for (rmap_iterator it = itPair.first; it != itPair.second; ++it) + if (it->second == itTarget) { + rmap.erase(it); + itTarget->second = v; + rmap.insert(make_pair(v, itTarget)); + return; + } + // Shouldn't ever get here + assert(0); + } + size_type max_size() const { return nMaxSize; } + size_type max_size(size_type s) + { + if (s) + while (map.size() > s) { + map.erase(rmap.begin()->second); + rmap.erase(rmap.begin()); + } + nMaxSize = s; + return nMaxSize; + } +}; + +#endif // BITCOIN_LIMITEDMAP_H diff --git a/src/structs/uint256.cpp b/src/structs/uint256.cpp new file mode 100644 index 00000000..cc5d9154 --- /dev/null +++ b/src/structs/uint256.cpp @@ -0,0 +1,358 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "structs/uint256.h" + +#include "utils/utilstrencodings.h" + +#include +#include + +template +base_uint::base_uint(const std::string& str) +{ + SetHex(str); +} + +template +base_uint::base_uint(const std::vector& vch) +{ + if (vch.size() != sizeof(pn)) + throw uint_error("Converting vector of wrong size to base_uint"); + memcpy(pn, &vch[0], sizeof(pn)); +} + +template +base_uint& base_uint::operator<<=(unsigned int shift) +{ + base_uint a(*this); + for (int i = 0; i < WIDTH; i++) + pn[i] = 0; + int k = shift / 32; + shift = shift % 32; + for (int i = 0; i < WIDTH; i++) { + if (i + k + 1 < WIDTH && shift != 0) + pn[i + k + 1] |= (a.pn[i] >> (32 - shift)); + if (i + k < WIDTH) + pn[i + k] |= (a.pn[i] << shift); + } + return *this; +} + +template +base_uint& base_uint::operator>>=(unsigned int shift) +{ + base_uint a(*this); + for (int i = 0; i < WIDTH; i++) + pn[i] = 0; + int k = shift / 32; + shift = shift % 32; + for (int i = 0; i < WIDTH; i++) { + if (i - k - 1 >= 0 && shift != 0) + pn[i - k - 1] |= (a.pn[i] << (32 - shift)); + if (i - k >= 0) + pn[i - k] |= (a.pn[i] >> shift); + } + return *this; +} + +template +base_uint& base_uint::operator*=(uint32_t b32) +{ + uint64_t carry = 0; + for (int i = 0; i < WIDTH; i++) { + uint64_t n = carry + (uint64_t)b32 * pn[i]; + pn[i] = n & 0xffffffff; + carry = n >> 32; + } + return *this; +} + +template +base_uint& base_uint::operator*=(const base_uint& b) +{ + base_uint a = *this; + *this = 0; + for (int j = 0; j < WIDTH; j++) { + uint64_t carry = 0; + for (int i = 0; i + j < WIDTH; i++) { + uint64_t n = carry + pn[i + j] + (uint64_t)a.pn[j] * b.pn[i]; + pn[i + j] = n & 0xffffffff; + carry = n >> 32; + } + } + return *this; +} + +template +base_uint& base_uint::operator/=(const base_uint& b) +{ + base_uint div = b; // make a copy, so we can shift. + base_uint num = *this; // make a copy, so we can subtract. + *this = 0; // the quotient. + int num_bits = num.bits(); + int div_bits = div.bits(); + if (div_bits == 0) + throw uint_error("Division by zero"); + if (div_bits > num_bits) // the result is certainly 0. + return *this; + int shift = num_bits - div_bits; + div <<= shift; // shift so that div and nun align. + while (shift >= 0) { + if (num >= div) { + num -= div; + pn[shift / 32] |= (1 << (shift & 31)); // set a bit of the result. + } + div >>= 1; // shift back. + shift--; + } + // num now contains the remainder of the division. + return *this; +} + +template +int base_uint::CompareTo(const base_uint& b) const +{ + for (int i = WIDTH - 1; i >= 0; i--) { + if (pn[i] < b.pn[i]) + return -1; + if (pn[i] > b.pn[i]) + return 1; + } + return 0; +} + +template +bool base_uint::EqualTo(uint64_t b) const +{ + for (int i = WIDTH - 1; i >= 2; i--) { + if (pn[i]) + return false; + } + if (pn[1] != (b >> 32)) + return false; + if (pn[0] != (b & 0xfffffffful)) + return false; + return true; +} + +template +double base_uint::getdouble() const +{ + double ret = 0.0; + double fact = 1.0; + for (int i = 0; i < WIDTH; i++) { + ret += fact * pn[i]; + fact *= 4294967296.0; + } + return ret; +} + +template +std::string base_uint::GetHex() const +{ + char psz[sizeof(pn) * 2 + 1]; + for (unsigned int i = 0; i < sizeof(pn); i++) + sprintf(psz + i * 2, "%02x", ((unsigned char*)pn)[sizeof(pn) - i - 1]); + return std::string(psz, psz + sizeof(pn) * 2); +} + +template +void base_uint::SetHex(const char* psz) +{ + memset(pn, 0, sizeof(pn)); + + // skip leading spaces + while (isspace(*psz)) + psz++; + + // skip 0x + if (psz[0] == '0' && tolower(psz[1]) == 'x') + psz += 2; + + // hex string to uint + const char* pbegin = psz; + while (::HexDigit(*psz) != -1) + psz++; + psz--; + unsigned char* p1 = (unsigned char*)pn; + unsigned char* pend = p1 + WIDTH * 4; + while (psz >= pbegin && p1 < pend) { + *p1 = ::HexDigit(*psz--); + if (psz >= pbegin) { + *p1 |= ((unsigned char)::HexDigit(*psz--) << 4); + p1++; + } + } +} + +template +void base_uint::SetHex(const std::string& str) +{ + SetHex(str.c_str()); +} + +template +std::string base_uint::ToString() const +{ + return (GetHex()); +} + +template +unsigned int base_uint::bits() const +{ + for (int pos = WIDTH - 1; pos >= 0; pos--) { + if (pn[pos]) { + for (int bits = 31; bits > 0; bits--) { + if (pn[pos] & 1 << bits) + return 32 * pos + bits + 1; + } + return 32 * pos + 1; + } + } + return 0; +} + +// Explicit instantiations for base_uint<160> +template base_uint<160>::base_uint(const std::string&); +template base_uint<160>::base_uint(const std::vector&); +template base_uint<160>& base_uint<160>::operator<<=(unsigned int); +template base_uint<160>& base_uint<160>::operator>>=(unsigned int); +template base_uint<160>& base_uint<160>::operator*=(uint32_t b32); +template base_uint<160>& base_uint<160>::operator*=(const base_uint<160>& b); +template base_uint<160>& base_uint<160>::operator/=(const base_uint<160>& b); +template int base_uint<160>::CompareTo(const base_uint<160>&) const; +template bool base_uint<160>::EqualTo(uint64_t) const; +template double base_uint<160>::getdouble() const; +template std::string base_uint<160>::GetHex() const; +template std::string base_uint<160>::ToString() const; +template void base_uint<160>::SetHex(const char*); +template void base_uint<160>::SetHex(const std::string&); +template unsigned int base_uint<160>::bits() const; + +// Explicit instantiations for base_uint<256> +template base_uint<256>::base_uint(const std::string&); +template base_uint<256>::base_uint(const std::vector&); +template base_uint<256>& base_uint<256>::operator<<=(unsigned int); +template base_uint<256>& base_uint<256>::operator>>=(unsigned int); +template base_uint<256>& base_uint<256>::operator*=(uint32_t b32); +template base_uint<256>& base_uint<256>::operator*=(const base_uint<256>& b); +template base_uint<256>& base_uint<256>::operator/=(const base_uint<256>& b); +template int base_uint<256>::CompareTo(const base_uint<256>&) const; +template bool base_uint<256>::EqualTo(uint64_t) const; +template double base_uint<256>::getdouble() const; +template std::string base_uint<256>::GetHex() const; +template std::string base_uint<256>::ToString() const; +template void base_uint<256>::SetHex(const char*); +template void base_uint<256>::SetHex(const std::string&); +template unsigned int base_uint<256>::bits() const; + +// This implementation directly uses shifts instead of going +// through an intermediate MPI representation. +uint256& uint256::SetCompact(uint32_t nCompact, bool* pfNegative, bool* pfOverflow) +{ + int nSize = nCompact >> 24; + uint32_t nWord = nCompact & 0x007fffff; + if (nSize <= 3) { + nWord >>= 8 * (3 - nSize); + *this = nWord; + } else { + *this = nWord; + *this <<= 8 * (nSize - 3); + } + if (pfNegative) + *pfNegative = nWord != 0 && (nCompact & 0x00800000) != 0; + if (pfOverflow) + *pfOverflow = nWord != 0 && ((nSize > 34) || + (nWord > 0xff && nSize > 33) || + (nWord > 0xffff && nSize > 32)); + return *this; +} + +uint32_t uint256::GetCompact(bool fNegative) const +{ + int nSize = (bits() + 7) / 8; + uint32_t nCompact = 0; + if (nSize <= 3) { + nCompact = GetLow64() << 8 * (3 - nSize); + } else { + uint256 bn = *this >> 8 * (nSize - 3); + nCompact = bn.GetLow64(); + } + // The 0x00800000 bit denotes the sign. + // Thus, if it is already set, divide the mantissa by 256 and increase the exponent. + if (nCompact & 0x00800000) { + nCompact >>= 8; + nSize++; + } + assert((nCompact & ~0x007fffff) == 0); + assert(nSize < 256); + nCompact |= nSize << 24; + nCompact |= (fNegative && (nCompact & 0x007fffff) ? 0x00800000 : 0); + return nCompact; +} + +static void inline HashMix(uint32_t& a, uint32_t& b, uint32_t& c) +{ + // Taken from lookup3, by Bob Jenkins. + a -= c; + a ^= ((c << 4) | (c >> 28)); + c += b; + b -= a; + b ^= ((a << 6) | (a >> 26)); + a += c; + c -= b; + c ^= ((b << 8) | (b >> 24)); + b += a; + a -= c; + a ^= ((c << 16) | (c >> 16)); + c += b; + b -= a; + b ^= ((a << 19) | (a >> 13)); + a += c; + c -= b; + c ^= ((b << 4) | (b >> 28)); + b += a; +} + +static void inline HashFinal(uint32_t& a, uint32_t& b, uint32_t& c) +{ + // Taken from lookup3, by Bob Jenkins. + c ^= b; + c -= ((b << 14) | (b >> 18)); + a ^= c; + a -= ((c << 11) | (c >> 21)); + b ^= a; + b -= ((a << 25) | (a >> 7)); + c ^= b; + c -= ((b << 16) | (b >> 16)); + a ^= c; + a -= ((c << 4) | (c >> 28)); + b ^= a; + b -= ((a << 14) | (a >> 18)); + c ^= b; + c -= ((b << 24) | (b >> 8)); +} + +uint64_t uint256::GetHash(const uint256& salt) const +{ + uint32_t a, b, c; + a = b = c = 0xdeadbeef + (WIDTH << 2); + + a += pn[0] ^ salt.pn[0]; + b += pn[1] ^ salt.pn[1]; + c += pn[2] ^ salt.pn[2]; + HashMix(a, b, c); + a += pn[3] ^ salt.pn[3]; + b += pn[4] ^ salt.pn[4]; + c += pn[5] ^ salt.pn[5]; + HashMix(a, b, c); + a += pn[6] ^ salt.pn[6]; + b += pn[7] ^ salt.pn[7]; + HashFinal(a, b, c); + + return ((((uint64_t)b) << 32) | c); +} diff --git a/src/structs/uint256.h b/src/structs/uint256.h new file mode 100644 index 00000000..7f39dd75 --- /dev/null +++ b/src/structs/uint256.h @@ -0,0 +1,334 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_UINT256_H +#define BITCOIN_UINT256_H + +#include +#include +#include +#include +#include +#include + +class uint_error : public std::runtime_error { +public: + explicit uint_error(const std::string& str) : std::runtime_error(str) {} +}; + +/** Template base class for unsigned big integers. */ +template +class base_uint +{ +protected: + enum { WIDTH=BITS/32 }; + uint32_t pn[WIDTH]; +public: + + base_uint() + { + for (int i = 0; i < WIDTH; i++) + pn[i] = 0; + } + + base_uint(const base_uint& b) + { + for (int i = 0; i < WIDTH; i++) + pn[i] = b.pn[i]; + } + + base_uint& operator=(const base_uint& b) + { + for (int i = 0; i < WIDTH; i++) + pn[i] = b.pn[i]; + return *this; + } + + base_uint(uint64_t b) + { + pn[0] = (unsigned int)b; + pn[1] = (unsigned int)(b >> 32); + for (int i = 2; i < WIDTH; i++) + pn[i] = 0; + } + + explicit base_uint(const std::string& str); + explicit base_uint(const std::vector& vch); + + bool operator!() const + { + for (int i = 0; i < WIDTH; i++) + if (pn[i] != 0) + return false; + return true; + } + + const base_uint operator~() const + { + base_uint ret; + for (int i = 0; i < WIDTH; i++) + ret.pn[i] = ~pn[i]; + return ret; + } + + const base_uint operator-() const + { + base_uint ret; + for (int i = 0; i < WIDTH; i++) + ret.pn[i] = ~pn[i]; + ret++; + return ret; + } + + double getdouble() const; + + base_uint& operator=(uint64_t b) + { + pn[0] = (unsigned int)b; + pn[1] = (unsigned int)(b >> 32); + for (int i = 2; i < WIDTH; i++) + pn[i] = 0; + return *this; + } + + base_uint& operator^=(const base_uint& b) + { + for (int i = 0; i < WIDTH; i++) + pn[i] ^= b.pn[i]; + return *this; + } + + base_uint& operator&=(const base_uint& b) + { + for (int i = 0; i < WIDTH; i++) + pn[i] &= b.pn[i]; + return *this; + } + + base_uint& operator|=(const base_uint& b) + { + for (int i = 0; i < WIDTH; i++) + pn[i] |= b.pn[i]; + return *this; + } + + base_uint& operator^=(uint64_t b) + { + pn[0] ^= (unsigned int)b; + pn[1] ^= (unsigned int)(b >> 32); + return *this; + } + + base_uint& operator|=(uint64_t b) + { + pn[0] |= (unsigned int)b; + pn[1] |= (unsigned int)(b >> 32); + return *this; + } + + base_uint& operator<<=(unsigned int shift); + base_uint& operator>>=(unsigned int shift); + + base_uint& operator+=(const base_uint& b) + { + uint64_t carry = 0; + for (int i = 0; i < WIDTH; i++) + { + uint64_t n = carry + pn[i] + b.pn[i]; + pn[i] = n & 0xffffffff; + carry = n >> 32; + } + return *this; + } + + base_uint& operator-=(const base_uint& b) + { + *this += -b; + return *this; + } + + base_uint& operator+=(uint64_t b64) + { + base_uint b; + b = b64; + *this += b; + return *this; + } + + base_uint& operator-=(uint64_t b64) + { + base_uint b; + b = b64; + *this += -b; + return *this; + } + + base_uint& operator*=(uint32_t b32); + base_uint& operator*=(const base_uint& b); + base_uint& operator/=(const base_uint& b); + + base_uint& operator++() + { + // prefix operator + int i = 0; + while (++pn[i] == 0 && i < WIDTH-1) + i++; + return *this; + } + + const base_uint operator++(int) + { + // postfix operator + const base_uint ret = *this; + ++(*this); + return ret; + } + + base_uint& operator--() + { + // prefix operator + int i = 0; + while (--pn[i] == (uint32_t)-1 && i < WIDTH-1) + i++; + return *this; + } + + const base_uint operator--(int) + { + // postfix operator + const base_uint ret = *this; + --(*this); + return ret; + } + + int CompareTo(const base_uint& b) const; + bool EqualTo(uint64_t b) const; + + friend inline const base_uint operator+(const base_uint& a, const base_uint& b) { return base_uint(a) += b; } + friend inline const base_uint operator-(const base_uint& a, const base_uint& b) { return base_uint(a) -= b; } + friend inline const base_uint operator*(const base_uint& a, const base_uint& b) { return base_uint(a) *= b; } + friend inline const base_uint operator/(const base_uint& a, const base_uint& b) { return base_uint(a) /= b; } + friend inline const base_uint operator|(const base_uint& a, const base_uint& b) { return base_uint(a) |= b; } + friend inline const base_uint operator&(const base_uint& a, const base_uint& b) { return base_uint(a) &= b; } + friend inline const base_uint operator^(const base_uint& a, const base_uint& b) { return base_uint(a) ^= b; } + friend inline const base_uint operator>>(const base_uint& a, int shift) { return base_uint(a) >>= shift; } + friend inline const base_uint operator<<(const base_uint& a, int shift) { return base_uint(a) <<= shift; } + friend inline const base_uint operator*(const base_uint& a, uint32_t b) { return base_uint(a) *= b; } + friend inline bool operator==(const base_uint& a, const base_uint& b) { return memcmp(a.pn, b.pn, sizeof(a.pn)) == 0; } + friend inline bool operator!=(const base_uint& a, const base_uint& b) { return memcmp(a.pn, b.pn, sizeof(a.pn)) != 0; } + friend inline bool operator>(const base_uint& a, const base_uint& b) { return a.CompareTo(b) > 0; } + friend inline bool operator<(const base_uint& a, const base_uint& b) { return a.CompareTo(b) < 0; } + friend inline bool operator>=(const base_uint& a, const base_uint& b) { return a.CompareTo(b) >= 0; } + friend inline bool operator<=(const base_uint& a, const base_uint& b) { return a.CompareTo(b) <= 0; } + friend inline bool operator==(const base_uint& a, uint64_t b) { return a.EqualTo(b); } + friend inline bool operator!=(const base_uint& a, uint64_t b) { return !a.EqualTo(b); } + + std::string GetHex() const; + void SetHex(const char* psz); + void SetHex(const std::string& str); + std::string ToString() const; + + unsigned char* begin() + { + return (unsigned char*)&pn[0]; + } + + unsigned char* end() + { + return (unsigned char*)&pn[WIDTH]; + } + + const unsigned char* begin() const + { + return (unsigned char*)&pn[0]; + } + + const unsigned char* end() const + { + return (unsigned char*)&pn[WIDTH]; + } + + unsigned int size() const + { + return sizeof(pn); + } + + /** + * Returns the position of the highest bit set plus one, or zero if the + * value is zero. + */ + unsigned int bits() const; + + uint64_t GetLow64() const + { + assert(WIDTH >= 2); + return pn[0] | (uint64_t)pn[1] << 32; + } + + unsigned int GetSerializeSize(int nType, int nVersion) const + { + return sizeof(pn); + } + + template + void Serialize(Stream& s, int nType, int nVersion) const + { + s.write((char*)pn, sizeof(pn)); + } + + template + void Unserialize(Stream& s, int nType, int nVersion) + { + s.read((char*)pn, sizeof(pn)); + } +}; + +/** 160-bit unsigned big integer. */ +class uint160 : public base_uint<160> { +public: + uint160() {} + uint160(const base_uint<160>& b) : base_uint<160>(b) {} + uint160(uint64_t b) : base_uint<160>(b) {} + explicit uint160(const std::string& str) : base_uint<160>(str) {} + explicit uint160(const std::vector& vch) : base_uint<160>(vch) {} +}; + +/** 256-bit unsigned big integer. */ +class uint256 : public base_uint<256> { +public: + uint256() {} + uint256(const base_uint<256>& b) : base_uint<256>(b) {} + uint256(uint64_t b) : base_uint<256>(b) {} + explicit uint256(const std::string& str) : base_uint<256>(str) {} + explicit uint256(const std::vector& vch) : base_uint<256>(vch) {} + + /** + * The "compact" format is a representation of a whole + * number N using an unsigned 32bit number similar to a + * floating point format. + * The most significant 8 bits are the unsigned exponent of base 256. + * This exponent can be thought of as "number of bytes of N". + * The lower 23 bits are the mantissa. + * Bit number 24 (0x800000) represents the sign of N. + * N = (-1^sign) * mantissa * 256^(exponent-3) + * + * Satoshi's original implementation used BN_bn2mpi() and BN_mpi2bn(). + * MPI uses the most significant bit of the first byte as sign. + * Thus 0x1234560000 is compact (0x05123456) + * and 0xc0de000000 is compact (0x0600c0de) + * + * Bitcoin only uses this "compact" format for encoding difficulty + * targets, which are unsigned 256bit quantities. Thus, all the + * complexities of the sign bit and using base 256 are probably an + * implementation accident. + */ + uint256& SetCompact(uint32_t nCompact, bool *pfNegative = NULL, bool *pfOverflow = NULL); + uint32_t GetCompact(bool fNegative = false) const; + + uint64_t GetHash(const uint256& salt) const; +}; + +#endif // BITCOIN_UINT256_H diff --git a/src/ui/noui.cpp b/src/ui/noui.cpp new file mode 100644 index 00000000..5d0e2b47 --- /dev/null +++ b/src/ui/noui.cpp @@ -0,0 +1,53 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "ui/noui.h" + +#include "ui/ui_interface.h" +#include "utils/util.h" + +#include +#include +#include + +static bool noui_ThreadSafeMessageBox(const std::string& message, const std::string& caption, unsigned int style) +{ + bool fSecure = style & CClientUIInterface::SECURE; + style &= ~CClientUIInterface::SECURE; + + std::string strCaption; + // Check for usage of predefined caption + switch (style) { + case CClientUIInterface::MSG_ERROR: + strCaption += _("Error"); + break; + case CClientUIInterface::MSG_WARNING: + strCaption += _("Warning"); + break; + case CClientUIInterface::MSG_INFORMATION: + strCaption += _("Information"); + break; + default: + strCaption += caption; // Use supplied caption (can be empty) + } + + if (!fSecure) + LogPrintf("%s: %s\n", strCaption, message); + fprintf(stderr, "%s: %s\n", strCaption.c_str(), message.c_str()); + return false; +} + +static void noui_InitMessage(const std::string& message) +{ + LogPrintf("init message: %s\n", message); +} + +void noui_connect() +{ + // Connect bitcoind signal handlers + uiInterface.ThreadSafeMessageBox.connect(noui_ThreadSafeMessageBox); + uiInterface.InitMessage.connect(noui_InitMessage); +} diff --git a/src/ui/noui.h b/src/ui/noui.h new file mode 100644 index 00000000..0dc69dbb --- /dev/null +++ b/src/ui/noui.h @@ -0,0 +1,11 @@ +// Copyright (c) 2013 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_NOUI_H +#define BITCOIN_NOUI_H + +extern void noui_connect(); + +#endif // BITCOIN_NOUI_H diff --git a/src/ui/ui_interface.h b/src/ui/ui_interface.h new file mode 100644 index 00000000..4e2cea19 --- /dev/null +++ b/src/ui/ui_interface.h @@ -0,0 +1,116 @@ +// Copyright (c) 2010 Satoshi Nakamoto +// Copyright (c) 2012 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_UI_INTERFACE_H +#define BITCOIN_UI_INTERFACE_H + +#include +#include + +#include +#include + +class CBasicKeyStore; +class CWallet; +class uint256; + +/** General change type (added, updated, removed). */ +enum ChangeType +{ + CT_NEW, + CT_UPDATED, + CT_DELETED +}; + +/** Signals for UI communication. */ +class CClientUIInterface +{ +public: + /** Flags for CClientUIInterface::ThreadSafeMessageBox */ + enum MessageBoxFlags + { + ICON_INFORMATION = 0, + ICON_WARNING = (1U << 0), + ICON_ERROR = (1U << 1), + /** + * Mask of all available icons in CClientUIInterface::MessageBoxFlags + * This needs to be updated, when icons are changed there! + */ + ICON_MASK = (ICON_INFORMATION | ICON_WARNING | ICON_ERROR), + + /** These values are taken from qmessagebox.h "enum StandardButton" to be directly usable */ + BTN_OK = 0x00000400U, // QMessageBox::Ok + BTN_YES = 0x00004000U, // QMessageBox::Yes + BTN_NO = 0x00010000U, // QMessageBox::No + BTN_ABORT = 0x00040000U, // QMessageBox::Abort + BTN_RETRY = 0x00080000U, // QMessageBox::Retry + BTN_IGNORE = 0x00100000U, // QMessageBox::Ignore + BTN_CLOSE = 0x00200000U, // QMessageBox::Close + BTN_CANCEL = 0x00400000U, // QMessageBox::Cancel + BTN_DISCARD = 0x00800000U, // QMessageBox::Discard + BTN_HELP = 0x01000000U, // QMessageBox::Help + BTN_APPLY = 0x02000000U, // QMessageBox::Apply + BTN_RESET = 0x04000000U, // QMessageBox::Reset + /** + * Mask of all available buttons in CClientUIInterface::MessageBoxFlags + * This needs to be updated, when buttons are changed there! + */ + BTN_MASK = (BTN_OK | BTN_YES | BTN_NO | BTN_ABORT | BTN_RETRY | BTN_IGNORE | + BTN_CLOSE | BTN_CANCEL | BTN_DISCARD | BTN_HELP | BTN_APPLY | BTN_RESET), + + /** Force blocking, modal message box dialog (not just OS notification) */ + MODAL = 0x10000000U, + + /** Do not print contents of message to debug log */ + SECURE = 0x40000000U, + + /** Predefined combinations for certain default usage cases */ + MSG_INFORMATION = ICON_INFORMATION, + MSG_WARNING = (ICON_WARNING | BTN_OK | MODAL), + MSG_ERROR = (ICON_ERROR | BTN_OK | MODAL) + }; + + /** Show message box. */ + boost::signals2::signal > ThreadSafeMessageBox; + + /** Progress message during initialization. */ + boost::signals2::signal InitMessage; + + /** Translate a message to the native language of the user. */ + boost::signals2::signal Translate; + + /** Number of network connections changed. */ + boost::signals2::signal NotifyNumConnectionsChanged; + + /** + * New, updated or cancelled alert. + * @note called with lock cs_mapAlerts held. + */ + boost::signals2::signal NotifyAlertChanged; + + /** A wallet has been loaded. */ + boost::signals2::signal LoadWallet; + + /** Show progress e.g. for verifychain */ + boost::signals2::signal ShowProgress; + + /** New block has been accepted */ + boost::signals2::signal NotifyBlockTip; +}; + +extern CClientUIInterface uiInterface; + +/** + * Translation function: Call Translate signal on UI interface, which returns a boost::optional result. + * If no translation slot is registered, nothing is returned, and simply return the input. + */ +inline std::string _(const char* psz) +{ + boost::optional rv = uiInterface.Translate(psz); + return rv ? (*rv) : psz; +} + +#endif // BITCOIN_UI_INTERFACE_H diff --git a/src/univalue/gen.cpp b/src/univalue/gen.cpp new file mode 100644 index 00000000..f0b352ee --- /dev/null +++ b/src/univalue/gen.cpp @@ -0,0 +1,78 @@ +// Copyright 2014 BitPay Inc. +// Distributed under the MIT/X11 software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +// +// To re-create univalue_escapes.h: +// $ g++ -o gen gen.cpp +// $ ./gen > univalue_escapes.h +// + +#include +#include +#include +#include "univalue.h" + +using namespace std; + +static bool initEscapes; +static const char *escapes[256]; + +static void initJsonEscape() +{ + escapes[(int)'"'] = "\\\""; + escapes[(int)'\\'] = "\\\\"; + escapes[(int)'/'] = "\\/"; + escapes[(int)'\b'] = "\\b"; + escapes[(int)'\f'] = "\\f"; + escapes[(int)'\n'] = "\\n"; + escapes[(int)'\r'] = "\\r"; + escapes[(int)'\t'] = "\\t"; + + initEscapes = true; +} + +static void outputEscape() +{ + printf( "// Automatically generated file. Do not modify.\n" + "#ifndef BITCOIN_UNIVALUE_UNIVALUE_ESCAPES_H\n" + "#define BITCOIN_UNIVALUE_UNIVALUE_ESCAPES_H\n" + "static const char *escapes[256] = {\n"); + + for (unsigned int i = 0; i < 256; i++) { + if (!escapes[i]) { + printf("\tNULL,\n"); + } else { + printf("\t\""); + + unsigned int si; + for (si = 0; si < strlen(escapes[i]); si++) { + char ch = escapes[i][si]; + switch (ch) { + case '"': + printf("\\\""); + break; + case '\\': + printf("\\\\"); + break; + default: + printf("%c", escapes[i][si]); + break; + } + } + + printf("\",\n"); + } + } + + printf( "};\n" + "#endif // BITCOIN_UNIVALUE_UNIVALUE_ESCAPES_H\n"); +} + +int main (int argc, char *argv[]) +{ + initJsonEscape(); + outputEscape(); + return 0; +} + diff --git a/src/univalue/univalue.cpp b/src/univalue/univalue.cpp new file mode 100644 index 00000000..b0171e48 --- /dev/null +++ b/src/univalue/univalue.cpp @@ -0,0 +1,211 @@ +// Copyright 2014 BitPay Inc. +// Distributed under the MIT/X11 software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include +#include +#include +#include "univalue.h" + +using namespace std; + +static const UniValue nullValue; + +void UniValue::clear() +{ + typ = VNULL; + val.clear(); + keys.clear(); + values.clear(); +} + +bool UniValue::setNull() +{ + clear(); + return true; +} + +bool UniValue::setBool(bool val_) +{ + clear(); + typ = VBOOL; + if (val_) + val = "1"; + return true; +} + +static bool validNumStr(const string& s) +{ + string tokenVal; + unsigned int consumed; + enum jtokentype tt = getJsonToken(tokenVal, consumed, s.c_str()); + return (tt == JTOK_NUMBER); +} + +bool UniValue::setNumStr(const string& val_) +{ + if (!validNumStr(val_)) + return false; + + clear(); + typ = VNUM; + val = val_; + return true; +} + +bool UniValue::setInt(uint64_t val) +{ + string s; + ostringstream oss; + + oss << val; + + return setNumStr(oss.str()); +} + +bool UniValue::setInt(int64_t val) +{ + string s; + ostringstream oss; + + oss << val; + + return setNumStr(oss.str()); +} + +bool UniValue::setFloat(double val) +{ + string s; + ostringstream oss; + + oss << val; + + return setNumStr(oss.str()); +} + +bool UniValue::setStr(const string& val_) +{ + clear(); + typ = VSTR; + val = val_; + return true; +} + +bool UniValue::setArray() +{ + clear(); + typ = VARR; + return true; +} + +bool UniValue::setObject() +{ + clear(); + typ = VOBJ; + return true; +} + +bool UniValue::push_back(const UniValue& val) +{ + if (typ != VARR) + return false; + + values.push_back(val); + return true; +} + +bool UniValue::push_backV(const std::vector& vec) +{ + if (typ != VARR) + return false; + + values.insert(values.end(), vec.begin(), vec.end()); + + return true; +} + +bool UniValue::pushKV(const std::string& key, const UniValue& val) +{ + if (typ != VOBJ) + return false; + + keys.push_back(key); + values.push_back(val); + return true; +} + +bool UniValue::pushKVs(const UniValue& obj) +{ + if (typ != VOBJ || obj.typ != VOBJ) + return false; + + for (unsigned int i = 0; i < obj.keys.size(); i++) { + keys.push_back(obj.keys[i]); + values.push_back(obj.values[i]); + } + + return true; +} + +int UniValue::findKey(const std::string& key) const +{ + for (unsigned int i = 0; i < keys.size(); i++) { + if (keys[i] == key) + return (int) i; + } + + return -1; +} + +bool UniValue::checkObject(const std::map& t) +{ + for (std::map::const_iterator it = t.begin(); + it != t.end(); it++) { + int idx = findKey(it->first); + if (idx < 0) + return false; + + if (values[idx].getType() != it->second) + return false; + } + + return true; +} + +const UniValue& UniValue::operator[](const std::string& key) const +{ + if (typ != VOBJ) + return nullValue; + + int index = findKey(key); + if (index < 0) + return nullValue; + + return values[index]; +} + +const UniValue& UniValue::operator[](unsigned int index) const +{ + if (typ != VOBJ && typ != VARR) + return nullValue; + if (index >= values.size()) + return nullValue; + + return values[index]; +} + +const char *uvTypeName(UniValue::VType t) +{ + switch (t) { + case UniValue::VNULL: return "null"; + case UniValue::VBOOL: return "bool"; + case UniValue::VOBJ: return "object"; + case UniValue::VARR: return "array"; + case UniValue::VSTR: return "string"; + case UniValue::VNUM: return "number"; + } + + // not reached + return NULL; +} + diff --git a/src/univalue/univalue.h b/src/univalue/univalue.h new file mode 100644 index 00000000..5ac301d9 --- /dev/null +++ b/src/univalue/univalue.h @@ -0,0 +1,155 @@ +// Copyright 2014 BitPay Inc. +// Distributed under the MIT/X11 software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#ifndef BITCOIN_UNIVALUE_UNIVALUE_H +#define BITCOIN_UNIVALUE_UNIVALUE_H + +#include +#include +#include +#include +#include + +class UniValue { +public: + enum VType { VNULL, VOBJ, VARR, VSTR, VNUM, VBOOL, }; + + UniValue() { typ = VNULL; } + UniValue(UniValue::VType initialType, const std::string& initialStr = "") { + typ = initialType; + val = initialStr; + } + UniValue(uint64_t val_) { + setInt(val_); + } + UniValue(int64_t val_) { + setInt(val_); + } + UniValue(int val_) { + setInt(val_); + } + UniValue(double val_) { + setFloat(val_); + } + UniValue(const std::string& val_) { + setStr(val_); + } + UniValue(const char *val_) { + std::string s(val_); + setStr(s); + } + ~UniValue() {} + + void clear(); + + bool setNull(); + bool setBool(bool val); + bool setNumStr(const std::string& val); + bool setInt(uint64_t val); + bool setInt(int64_t val); + bool setInt(int val) { return setInt((int64_t)val); } + bool setFloat(double val); + bool setStr(const std::string& val); + bool setArray(); + bool setObject(); + + enum VType getType() const { return typ; } + std::string getValStr() const { return val; } + bool empty() const { return (values.size() == 0); } + + size_t count() const { return values.size(); } + + bool getBool() const { return isTrue(); } + bool checkObject(const std::map& memberTypes); + const UniValue& operator[](const std::string& key) const; + const UniValue& operator[](unsigned int index) const; + bool exists(const std::string& key) const { return (findKey(key) >= 0); } + + bool isNull() const { return (typ == VNULL); } + bool isTrue() const { return (typ == VBOOL) && (val == "1"); } + bool isFalse() const { return (!isTrue()); } + bool isBool() const { return (typ == VBOOL); } + bool isStr() const { return (typ == VSTR); } + bool isNum() const { return (typ == VNUM); } + bool isArray() const { return (typ == VARR); } + bool isObject() const { return (typ == VOBJ); } + + bool push_back(const UniValue& val); + bool push_back(const std::string& val_) { + UniValue tmpVal(VSTR, val_); + return push_back(tmpVal); + } + bool push_back(const char *val_) { + std::string s(val_); + return push_back(s); + } + bool push_backV(const std::vector& vec); + + bool pushKV(const std::string& key, const UniValue& val); + bool pushKV(const std::string& key, const std::string& val) { + UniValue tmpVal(VSTR, val); + return pushKV(key, tmpVal); + } + bool pushKV(const std::string& key, const char *val_) { + std::string val(val_); + return pushKV(key, val); + } + bool pushKV(const std::string& key, int64_t val) { + UniValue tmpVal(val); + return pushKV(key, tmpVal); + } + bool pushKV(const std::string& key, uint64_t val) { + UniValue tmpVal(val); + return pushKV(key, tmpVal); + } + bool pushKV(const std::string& key, int val) { + UniValue tmpVal((int64_t)val); + return pushKV(key, tmpVal); + } + bool pushKV(const std::string& key, double val) { + UniValue tmpVal(val); + return pushKV(key, tmpVal); + } + bool pushKVs(const UniValue& obj); + + std::string write(unsigned int prettyIndent = 0, + unsigned int indentLevel = 0) const; + + bool read(const char *raw); + bool read(const std::string& rawStr) { + return read(rawStr.c_str()); + } + +private: + UniValue::VType typ; + std::string val; // numbers are stored as C++ strings + std::vector keys; + std::vector values; + + int findKey(const std::string& key) const; + void writeArray(unsigned int prettyIndent, unsigned int indentLevel, std::string& s) const; + void writeObject(unsigned int prettyIndent, unsigned int indentLevel, std::string& s) const; +}; + +enum jtokentype { + JTOK_ERR = -1, + JTOK_NONE = 0, // eof + JTOK_OBJ_OPEN, + JTOK_OBJ_CLOSE, + JTOK_ARR_OPEN, + JTOK_ARR_CLOSE, + JTOK_COLON, + JTOK_COMMA, + JTOK_KW_NULL, + JTOK_KW_TRUE, + JTOK_KW_FALSE, + JTOK_NUMBER, + JTOK_STRING, +}; + +extern enum jtokentype getJsonToken(std::string& tokenVal, + unsigned int& consumed, const char *raw); +extern const char *uvTypeName(UniValue::VType t); + +#endif // BITCOIN_UNIVALUE_UNIVALUE_H diff --git a/src/univalue/univalue_escapes.h b/src/univalue/univalue_escapes.h new file mode 100644 index 00000000..05141182 --- /dev/null +++ b/src/univalue/univalue_escapes.h @@ -0,0 +1,262 @@ +// Automatically generated file. Do not modify. +#ifndef BITCOIN_UNIVALUE_UNIVALUE_ESCAPES_H +#define BITCOIN_UNIVALUE_UNIVALUE_ESCAPES_H +static const char *escapes[256] = { + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + "\\b", + "\\t", + "\\n", + NULL, + "\\f", + "\\r", + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + "\\\"", + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + "\\/", + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + "\\\\", + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, +}; +#endif // BITCOIN_UNIVALUE_UNIVALUE_ESCAPES_H diff --git a/src/univalue/univalue_read.cpp b/src/univalue/univalue_read.cpp new file mode 100644 index 00000000..405be3e8 --- /dev/null +++ b/src/univalue/univalue_read.cpp @@ -0,0 +1,390 @@ +// Copyright 2014 BitPay Inc. +// Distributed under the MIT/X11 software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include +#include +#include +#include "univalue.h" + +using namespace std; + +// convert hexadecimal string to unsigned integer +static const char *hatoui(const char *first, const char *last, + unsigned int& out) +{ + unsigned int result = 0; + for (; first != last; ++first) + { + int digit; + if (isdigit(*first)) + digit = *first - '0'; + + else if (*first >= 'a' && *first <= 'f') + digit = *first - 'a' + 10; + + else if (*first >= 'A' && *first <= 'F') + digit = *first - 'A' + 10; + + else + break; + + result = 16 * result + digit; + } + out = result; + + return first; +} + +enum jtokentype getJsonToken(string& tokenVal, unsigned int& consumed, + const char *raw) +{ + tokenVal.clear(); + consumed = 0; + + const char *rawStart = raw; + + while ((*raw) && (isspace(*raw))) // skip whitespace + raw++; + + switch (*raw) { + + case 0: + return JTOK_NONE; + + case '{': + raw++; + consumed = (raw - rawStart); + return JTOK_OBJ_OPEN; + case '}': + raw++; + consumed = (raw - rawStart); + return JTOK_OBJ_CLOSE; + case '[': + raw++; + consumed = (raw - rawStart); + return JTOK_ARR_OPEN; + case ']': + raw++; + consumed = (raw - rawStart); + return JTOK_ARR_CLOSE; + + case ':': + raw++; + consumed = (raw - rawStart); + return JTOK_COLON; + case ',': + raw++; + consumed = (raw - rawStart); + return JTOK_COMMA; + + case 'n': + case 't': + case 'f': + if (!strncmp(raw, "null", 4)) { + raw += 4; + consumed = (raw - rawStart); + return JTOK_KW_NULL; + } else if (!strncmp(raw, "true", 4)) { + raw += 4; + consumed = (raw - rawStart); + return JTOK_KW_TRUE; + } else if (!strncmp(raw, "false", 5)) { + raw += 5; + consumed = (raw - rawStart); + return JTOK_KW_FALSE; + } else + return JTOK_ERR; + + case '-': + case '0': + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': { + // part 1: int + string numStr; + + const char *first = raw; + + const char *firstDigit = first; + if (!isdigit(*firstDigit)) + firstDigit++; + if ((*firstDigit == '0') && isdigit(firstDigit[1])) + return JTOK_ERR; + + numStr += *raw; // copy first char + raw++; + + if ((*first == '-') && (!isdigit(*raw))) + return JTOK_ERR; + + while ((*raw) && isdigit(*raw)) { // copy digits + numStr += *raw; + raw++; + } + + // part 2: frac + if (*raw == '.') { + numStr += *raw; // copy . + raw++; + + if (!isdigit(*raw)) + return JTOK_ERR; + while ((*raw) && isdigit(*raw)) { // copy digits + numStr += *raw; + raw++; + } + } + + // part 3: exp + if (*raw == 'e' || *raw == 'E') { + numStr += *raw; // copy E + raw++; + + if (*raw == '-' || *raw == '+') { // copy +/- + numStr += *raw; + raw++; + } + + if (!isdigit(*raw)) + return JTOK_ERR; + while ((*raw) && isdigit(*raw)) { // copy digits + numStr += *raw; + raw++; + } + } + + tokenVal = numStr; + consumed = (raw - rawStart); + return JTOK_NUMBER; + } + + case '"': { + raw++; // skip " + + string valStr; + + while (*raw) { + if (*raw < 0x20) + return JTOK_ERR; + + else if (*raw == '\\') { + raw++; // skip backslash + + switch (*raw) { + case '"': valStr += "\""; break; + case '\\': valStr += "\\"; break; + case '/': valStr += "/"; break; + case 'b': valStr += "\b"; break; + case 'f': valStr += "\f"; break; + case 'n': valStr += "\n"; break; + case 'r': valStr += "\r"; break; + case 't': valStr += "\t"; break; + + case 'u': { + char buf[4] = {0,0,0,0}; + char *last = &buf[0]; + unsigned int codepoint; + if (hatoui(raw + 1, raw + 1 + 4, codepoint) != + raw + 1 + 4) + return JTOK_ERR; + + if (codepoint <= 0x7f) + *last = (char)codepoint; + else if (codepoint <= 0x7FF) { + *last++ = (char)(0xC0 | (codepoint >> 6)); + *last = (char)(0x80 | (codepoint & 0x3F)); + } else if (codepoint <= 0xFFFF) { + *last++ = (char)(0xE0 | (codepoint >> 12)); + *last++ = (char)(0x80 | ((codepoint >> 6) & 0x3F)); + *last = (char)(0x80 | (codepoint & 0x3F)); + } + + valStr += buf; + raw += 4; + break; + } + default: + return JTOK_ERR; + + } + + raw++; // skip esc'd char + } + + else if (*raw == '"') { + raw++; // skip " + break; // stop scanning + } + + else { + valStr += *raw; + raw++; + } + } + + tokenVal = valStr; + consumed = (raw - rawStart); + return JTOK_STRING; + } + + default: + return JTOK_ERR; + } +} + +bool UniValue::read(const char *raw) +{ + clear(); + + bool expectName = false; + bool expectColon = false; + vector stack; + + enum jtokentype tok = JTOK_NONE; + enum jtokentype last_tok = JTOK_NONE; + while (1) { + last_tok = tok; + + string tokenVal; + unsigned int consumed; + tok = getJsonToken(tokenVal, consumed, raw); + if (tok == JTOK_NONE || tok == JTOK_ERR) + break; + raw += consumed; + + switch (tok) { + + case JTOK_OBJ_OPEN: + case JTOK_ARR_OPEN: { + VType utyp = (tok == JTOK_OBJ_OPEN ? VOBJ : VARR); + if (!stack.size()) { + if (utyp == VOBJ) + setObject(); + else + setArray(); + stack.push_back(this); + } else { + UniValue tmpVal(utyp); + UniValue *top = stack.back(); + top->values.push_back(tmpVal); + + UniValue *newTop = &(top->values.back()); + stack.push_back(newTop); + } + + if (utyp == VOBJ) + expectName = true; + break; + } + + case JTOK_OBJ_CLOSE: + case JTOK_ARR_CLOSE: { + if (!stack.size() || expectColon || (last_tok == JTOK_COMMA)) + return false; + + VType utyp = (tok == JTOK_OBJ_CLOSE ? VOBJ : VARR); + UniValue *top = stack.back(); + if (utyp != top->getType()) + return false; + + stack.pop_back(); + expectName = false; + break; + } + + case JTOK_COLON: { + if (!stack.size() || expectName || !expectColon) + return false; + + UniValue *top = stack.back(); + if (top->getType() != VOBJ) + return false; + + expectColon = false; + break; + } + + case JTOK_COMMA: { + if (!stack.size() || expectName || expectColon || + (last_tok == JTOK_COMMA) || (last_tok == JTOK_ARR_OPEN)) + return false; + + UniValue *top = stack.back(); + if (top->getType() == VOBJ) + expectName = true; + break; + } + + case JTOK_KW_NULL: + case JTOK_KW_TRUE: + case JTOK_KW_FALSE: { + if (!stack.size() || expectName || expectColon) + return false; + + UniValue tmpVal; + switch (tok) { + case JTOK_KW_NULL: + // do nothing more + break; + case JTOK_KW_TRUE: + tmpVal.setBool(true); + break; + case JTOK_KW_FALSE: + tmpVal.setBool(false); + break; + default: /* impossible */ break; + } + + UniValue *top = stack.back(); + top->values.push_back(tmpVal); + + break; + } + + case JTOK_NUMBER: { + if (!stack.size() || expectName || expectColon) + return false; + + UniValue tmpVal(VNUM, tokenVal); + UniValue *top = stack.back(); + top->values.push_back(tmpVal); + + break; + } + + case JTOK_STRING: { + if (!stack.size()) + return false; + + UniValue *top = stack.back(); + + if (expectName) { + top->keys.push_back(tokenVal); + expectName = false; + expectColon = true; + } else { + UniValue tmpVal(VSTR, tokenVal); + top->values.push_back(tmpVal); + } + + break; + } + + default: + return false; + } + } + + if (stack.size() != 0) + return false; + + return true; +} + diff --git a/src/univalue/univalue_write.cpp b/src/univalue/univalue_write.cpp new file mode 100644 index 00000000..9565cfa1 --- /dev/null +++ b/src/univalue/univalue_write.cpp @@ -0,0 +1,125 @@ +// Copyright 2014 BitPay Inc. +// Distributed under the MIT/X11 software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include +#include +#include "univalue.h" +#include "univalue_escapes.h" + +// TODO: Using UTF8 + +using namespace std; + +static string json_escape(const string& inS) +{ + string outS; + outS.reserve(inS.size() * 2); + + for (unsigned int i = 0; i < inS.size(); i++) { + unsigned char ch = inS[i]; + const char *escStr = escapes[ch]; + + if (escStr) + outS += escStr; + + else if (isprint(ch)) + outS += ch; + + else { + char tmpesc[16]; + sprintf(tmpesc, "\\u%04x", ch); + outS += tmpesc; + } + } + + return outS; +} + +string UniValue::write(unsigned int prettyIndent, + unsigned int indentLevel) const +{ + string s; + s.reserve(1024); + + unsigned int modIndent = indentLevel; + if (modIndent == 0) + modIndent = 1; + + switch (typ) { + case VNULL: + s += "null"; + break; + case VOBJ: + writeObject(prettyIndent, modIndent, s); + break; + case VARR: + writeArray(prettyIndent, modIndent, s); + break; + case VSTR: + s += "\"" + json_escape(val) + "\""; + break; + case VNUM: + s += val; + break; + case VBOOL: + s += (val == "1" ? "true" : "false"); + break; + } + + return s; +} + +static void indentStr(unsigned int prettyIndent, unsigned int indentLevel, string& s) +{ + s.append(prettyIndent * indentLevel, ' '); +} + +void UniValue::writeArray(unsigned int prettyIndent, unsigned int indentLevel, string& s) const +{ + s += "["; + if (prettyIndent) + s += "\n"; + + for (unsigned int i = 0; i < values.size(); i++) { + if (prettyIndent) + indentStr(prettyIndent, indentLevel, s); + s += values[i].write(prettyIndent, indentLevel + 1); + if (i != (values.size() - 1)) { + s += ","; + if (prettyIndent) + s += " "; + } + if (prettyIndent) + s += "\n"; + } + + if (prettyIndent) + indentStr(prettyIndent, indentLevel - 1, s); + s += "]"; +} + +void UniValue::writeObject(unsigned int prettyIndent, unsigned int indentLevel, string& s) const +{ + s += "{"; + if (prettyIndent) + s += "\n"; + + for (unsigned int i = 0; i < keys.size(); i++) { + if (prettyIndent) + indentStr(prettyIndent, indentLevel, s); + s += "\"" + json_escape(keys[i]) + "\":"; + if (prettyIndent) + s += " "; + s += values[i].write(prettyIndent, indentLevel + 1); + if (i != (values.size() - 1)) + s += ","; + if (prettyIndent) + s += "\n"; + } + + if (prettyIndent) + indentStr(prettyIndent, indentLevel - 1, s); + s += "}"; +} + diff --git a/src/utils/allocators.cpp b/src/utils/allocators.cpp new file mode 100644 index 00000000..55a96f78 --- /dev/null +++ b/src/utils/allocators.cpp @@ -0,0 +1,67 @@ +// Copyright (c) 2009-2013 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "utils/allocators.h" + +#ifdef WIN32 +#ifdef _WIN32_WINNT +#undef _WIN32_WINNT +#endif +#define _WIN32_WINNT 0x0501 +#define WIN32_LEAN_AND_MEAN 1 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +// This is used to attempt to keep keying material out of swap +// Note that VirtualLock does not provide this as a guarantee on Windows, +// but, in practice, memory that has been VirtualLock'd almost never gets written to +// the pagefile except in rare circumstances where memory is extremely low. +#else +#include +#include // for PAGESIZE +#include // for sysconf +#endif + +LockedPageManager* LockedPageManager::_instance = NULL; +boost::once_flag LockedPageManager::init_flag = BOOST_ONCE_INIT; + +/** Determine system page size in bytes */ +static inline size_t GetSystemPageSize() +{ + size_t page_size; +#if defined(WIN32) + SYSTEM_INFO sSysInfo; + GetSystemInfo(&sSysInfo); + page_size = sSysInfo.dwPageSize; +#elif defined(PAGESIZE) // defined in limits.h + page_size = PAGESIZE; +#else // assume some POSIX OS + page_size = sysconf(_SC_PAGESIZE); +#endif + return page_size; +} + +bool MemoryPageLocker::Lock(const void* addr, size_t len) +{ +#ifdef WIN32 + return VirtualLock(const_cast(addr), len) != 0; +#else + return mlock(addr, len) == 0; +#endif +} + +bool MemoryPageLocker::Unlock(const void* addr, size_t len) +{ +#ifdef WIN32 + return VirtualUnlock(const_cast(addr), len) != 0; +#else + return munlock(addr, len) == 0; +#endif +} + +LockedPageManager::LockedPageManager() : LockedPageManagerBase(GetSystemPageSize()) +{ +} diff --git a/src/utils/allocators.h b/src/utils/allocators.h new file mode 100644 index 00000000..90e44fdc --- /dev/null +++ b/src/utils/allocators.h @@ -0,0 +1,269 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2013 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_ALLOCATORS_H +#define BITCOIN_ALLOCATORS_H + +#include +#include +#include +#include + +#include +#include + +#include // for OPENSSL_cleanse() + +/** + * Thread-safe class to keep track of locked (ie, non-swappable) memory pages. + * + * Memory locks do not stack, that is, pages which have been locked several times by calls to mlock() + * will be unlocked by a single call to munlock(). This can result in keying material ending up in swap when + * those functions are used naively. This class simulates stacking memory locks by keeping a counter per page. + * + * @note By using a map from each page base address to lock count, this class is optimized for + * small objects that span up to a few pages, mostly smaller than a page. To support large allocations, + * something like an interval tree would be the preferred data structure. + */ +template +class LockedPageManagerBase +{ +public: + LockedPageManagerBase(size_t page_size) : page_size(page_size) + { + // Determine bitmask for extracting page from address + assert(!(page_size & (page_size - 1))); // size must be power of two + page_mask = ~(page_size - 1); + } + + ~LockedPageManagerBase() + { + assert(this->GetLockedPageCount() == 0); + } + + + // For all pages in affected range, increase lock count + void LockRange(void* p, size_t size) + { + boost::mutex::scoped_lock lock(mutex); + if (!size) + return; + const size_t base_addr = reinterpret_cast(p); + const size_t start_page = base_addr & page_mask; + const size_t end_page = (base_addr + size - 1) & page_mask; + for (size_t page = start_page; page <= end_page; page += page_size) { + Histogram::iterator it = histogram.find(page); + if (it == histogram.end()) // Newly locked page + { + locker.Lock(reinterpret_cast(page), page_size); + histogram.insert(std::make_pair(page, 1)); + } else // Page was already locked; increase counter + { + it->second += 1; + } + } + } + + // For all pages in affected range, decrease lock count + void UnlockRange(void* p, size_t size) + { + boost::mutex::scoped_lock lock(mutex); + if (!size) + return; + const size_t base_addr = reinterpret_cast(p); + const size_t start_page = base_addr & page_mask; + const size_t end_page = (base_addr + size - 1) & page_mask; + for (size_t page = start_page; page <= end_page; page += page_size) { + Histogram::iterator it = histogram.find(page); + assert(it != histogram.end()); // Cannot unlock an area that was not locked + // Decrease counter for page, when it is zero, the page will be unlocked + it->second -= 1; + if (it->second == 0) // Nothing on the page anymore that keeps it locked + { + // Unlock page and remove the count from histogram + locker.Unlock(reinterpret_cast(page), page_size); + histogram.erase(it); + } + } + } + + // Get number of locked pages for diagnostics + int GetLockedPageCount() + { + boost::mutex::scoped_lock lock(mutex); + return histogram.size(); + } + +private: + Locker locker; + boost::mutex mutex; + size_t page_size, page_mask; + // map of page base address to lock count + typedef std::map Histogram; + Histogram histogram; +}; + + +/** + * OS-dependent memory page locking/unlocking. + * Defined as policy class to make stubbing for test possible. + */ +class MemoryPageLocker +{ +public: + /** Lock memory pages. + * addr and len must be a multiple of the system page size + */ + bool Lock(const void* addr, size_t len); + /** Unlock memory pages. + * addr and len must be a multiple of the system page size + */ + bool Unlock(const void* addr, size_t len); +}; + +/** + * Singleton class to keep track of locked (ie, non-swappable) memory pages, for use in + * std::allocator templates. + * + * Some implementations of the STL allocate memory in some constructors (i.e., see + * MSVC's vector implementation where it allocates 1 byte of memory in the allocator.) + * Due to the unpredictable order of static initializers, we have to make sure the + * LockedPageManager instance exists before any other STL-based objects that use + * secure_allocator are created. So instead of having LockedPageManager also be + * static-initialized, it is created on demand. + */ +class LockedPageManager : public LockedPageManagerBase +{ +public: + static LockedPageManager& Instance() + { + boost::call_once(LockedPageManager::CreateInstance, LockedPageManager::init_flag); + return *LockedPageManager::_instance; + } + +private: + LockedPageManager(); + + static void CreateInstance() + { + // Using a local static instance guarantees that the object is initialized + // when it's first needed and also deinitialized after all objects that use + // it are done with it. I can think of one unlikely scenario where we may + // have a static deinitialization order/problem, but the check in + // LockedPageManagerBase's destructor helps us detect if that ever happens. + static LockedPageManager instance; + LockedPageManager::_instance = &instance; + } + + static LockedPageManager* _instance; + static boost::once_flag init_flag; +}; + +// +// Functions for directly locking/unlocking memory objects. +// Intended for non-dynamically allocated structures. +// +template +void LockObject(const T& t) +{ + LockedPageManager::Instance().LockRange((void*)(&t), sizeof(T)); +} + +template +void UnlockObject(const T& t) +{ + OPENSSL_cleanse((void*)(&t), sizeof(T)); + LockedPageManager::Instance().UnlockRange((void*)(&t), sizeof(T)); +} + +// +// Allocator that locks its contents from being paged +// out of memory and clears its contents before deletion. +// +template +struct secure_allocator : public std::allocator { + // MSVC8 default copy constructor is broken + typedef std::allocator base; + typedef typename base::size_type size_type; + typedef typename base::difference_type difference_type; + typedef typename base::pointer pointer; + typedef typename base::const_pointer const_pointer; + typedef typename base::reference reference; + typedef typename base::const_reference const_reference; + typedef typename base::value_type value_type; + secure_allocator() throw() {} + secure_allocator(const secure_allocator& a) throw() : base(a) {} + template + secure_allocator(const secure_allocator& a) throw() : base(a) + { + } + ~secure_allocator() throw() {} + template + struct rebind { + typedef secure_allocator<_Other> other; + }; + + T* allocate(std::size_t n, const void* hint = 0) + { + T* p; + p = std::allocator::allocate(n, hint); + if (p != NULL) + LockedPageManager::Instance().LockRange(p, sizeof(T) * n); + return p; + } + + void deallocate(T* p, std::size_t n) + { + if (p != NULL) { + OPENSSL_cleanse(p, sizeof(T) * n); + LockedPageManager::Instance().UnlockRange(p, sizeof(T) * n); + } + std::allocator::deallocate(p, n); + } +}; + + +// +// Allocator that clears its contents before deletion. +// +template +struct zero_after_free_allocator : public std::allocator { + // MSVC8 default copy constructor is broken + typedef std::allocator base; + typedef typename base::size_type size_type; + typedef typename base::difference_type difference_type; + typedef typename base::pointer pointer; + typedef typename base::const_pointer const_pointer; + typedef typename base::reference reference; + typedef typename base::const_reference const_reference; + typedef typename base::value_type value_type; + zero_after_free_allocator() throw() {} + zero_after_free_allocator(const zero_after_free_allocator& a) throw() : base(a) {} + template + zero_after_free_allocator(const zero_after_free_allocator& a) throw() : base(a) + { + } + ~zero_after_free_allocator() throw() {} + template + struct rebind { + typedef zero_after_free_allocator<_Other> other; + }; + + void deallocate(T* p, std::size_t n) + { + if (p != NULL) + OPENSSL_cleanse(p, sizeof(T) * n); + std::allocator::deallocate(p, n); + } +}; + +// This is exactly like std::string, but with a custom allocator. +typedef std::basic_string, secure_allocator > SecureString; + +// Byte-vector that clears its contents before deletion. +typedef std::vector > CSerializeData; + +#endif // BITCOIN_ALLOCATORS_H diff --git a/src/utils/compat.h b/src/utils/compat.h new file mode 100644 index 00000000..4bdfb866 --- /dev/null +++ b/src/utils/compat.h @@ -0,0 +1,96 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_COMPAT_H +#define BITCOIN_COMPAT_H + +#if defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" +#endif + +#ifdef WIN32 +#ifdef _WIN32_WINNT +#undef _WIN32_WINNT +#endif +#define _WIN32_WINNT 0x0501 +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN 1 +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif +#ifdef FD_SETSIZE +#undef FD_SETSIZE // prevent redefinition compiler warning +#endif +#define FD_SETSIZE 1024 // max number of fds in fd_set + +#include // Must be included before mswsock.h and windows.h + +#include +#include +#include +#else +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#endif + +#ifdef WIN32 +#define MSG_DONTWAIT 0 +#else +typedef u_int SOCKET; +#include "errno.h" +#define WSAGetLastError() errno +#define WSAEINVAL EINVAL +#define WSAEALREADY EALREADY +#define WSAEWOULDBLOCK EWOULDBLOCK +#define WSAEMSGSIZE EMSGSIZE +#define WSAEINTR EINTR +#define WSAEINPROGRESS EINPROGRESS +#define WSAEADDRINUSE EADDRINUSE +#define WSAENOTSOCK EBADF +#define INVALID_SOCKET (SOCKET)(~0) +#define SOCKET_ERROR -1 +#endif + +#ifdef WIN32 +#ifndef S_IRUSR +#define S_IRUSR 0400 +#define S_IWUSR 0200 +#endif +#else +#define MAX_PATH 1024 +#endif + +// As Solaris does not have the MSG_NOSIGNAL flag for send(2) syscall, it is defined as 0 +#if !defined(HAVE_MSG_NOSIGNAL) && !defined(MSG_NOSIGNAL) +#define MSG_NOSIGNAL 0 +#endif + +#ifndef WIN32 +// PRIO_MAX is not defined on Solaris +#ifndef PRIO_MAX +#define PRIO_MAX 20 +#endif +#define THREAD_PRIORITY_LOWEST PRIO_MAX +#define THREAD_PRIORITY_BELOW_NORMAL 2 +#define THREAD_PRIORITY_NORMAL 0 +#define THREAD_PRIORITY_ABOVE_NORMAL (-2) +#endif + +#if HAVE_DECL_STRNLEN == 0 +size_t strnlen( const char *start, size_t max_len); +#endif // HAVE_DECL_STRNLEN + +#endif // BITCOIN_COMPAT_H diff --git a/src/utils/compressor.cpp b/src/utils/compressor.cpp new file mode 100644 index 00000000..93c24e39 --- /dev/null +++ b/src/utils/compressor.cpp @@ -0,0 +1,186 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "utils/compressor.h" + +#include "structs/hash.h" +#include "keys/pubkey.h" +#include "script/standard.h" + +bool CScriptCompressor::IsToKeyID(CKeyID &hash) const +{ + if (script.size() == 25 && script[0] == OP_DUP && script[1] == OP_HASH160 + && script[2] == 20 && script[23] == OP_EQUALVERIFY + && script[24] == OP_CHECKSIG) { + memcpy(&hash, &script[3], 20); + return true; + } + return false; +} + +bool CScriptCompressor::IsToScriptID(CScriptID &hash) const +{ + if (script.size() == 23 && script[0] == OP_HASH160 && script[1] == 20 + && script[22] == OP_EQUAL) { + memcpy(&hash, &script[2], 20); + return true; + } + return false; +} + +bool CScriptCompressor::IsToPubKey(CPubKey &pubkey) const +{ + if (script.size() == 35 && script[0] == 33 && script[34] == OP_CHECKSIG + && (script[1] == 0x02 || script[1] == 0x03)) { + pubkey.Set(&script[1], &script[34]); + return true; + } + if (script.size() == 67 && script[0] == 65 && script[66] == OP_CHECKSIG + && script[1] == 0x04) { + pubkey.Set(&script[1], &script[66]); + return pubkey.IsFullyValid(); // if not fully valid, a case that would not be compressible + } + return false; +} + +bool CScriptCompressor::Compress(std::vector &out) const +{ + CKeyID keyID; + if (IsToKeyID(keyID)) { + out.resize(21); + out[0] = 0x00; + memcpy(&out[1], &keyID, 20); + return true; + } + CScriptID scriptID; + if (IsToScriptID(scriptID)) { + out.resize(21); + out[0] = 0x01; + memcpy(&out[1], &scriptID, 20); + return true; + } + CPubKey pubkey; + if (IsToPubKey(pubkey)) { + out.resize(33); + memcpy(&out[1], &pubkey[1], 32); + if (pubkey[0] == 0x02 || pubkey[0] == 0x03) { + out[0] = pubkey[0]; + return true; + } else if (pubkey[0] == 0x04) { + out[0] = 0x04 | (pubkey[64] & 0x01); + return true; + } + } + return false; +} + +unsigned int CScriptCompressor::GetSpecialSize(unsigned int nSize) const +{ + if (nSize == 0 || nSize == 1) + return 20; + if (nSize == 2 || nSize == 3 || nSize == 4 || nSize == 5) + return 32; + return 0; +} + +bool CScriptCompressor::Decompress(unsigned int nSize, const std::vector &in) +{ + switch(nSize) { + case 0x00: + script.resize(25); + script[0] = OP_DUP; + script[1] = OP_HASH160; + script[2] = 20; + memcpy(&script[3], &in[0], 20); + script[23] = OP_EQUALVERIFY; + script[24] = OP_CHECKSIG; + return true; + case 0x01: + script.resize(23); + script[0] = OP_HASH160; + script[1] = 20; + memcpy(&script[2], &in[0], 20); + script[22] = OP_EQUAL; + return true; + case 0x02: + case 0x03: + script.resize(35); + script[0] = 33; + script[1] = nSize; + memcpy(&script[2], &in[0], 32); + script[34] = OP_CHECKSIG; + return true; + case 0x04: + case 0x05: + unsigned char vch[33] = {}; + vch[0] = nSize - 2; + memcpy(&vch[1], &in[0], 32); + CPubKey pubkey(&vch[0], &vch[33]); + if (!pubkey.Decompress()) + return false; + assert(pubkey.size() == 65); + script.resize(67); + script[0] = 65; + memcpy(&script[1], pubkey.begin(), 65); + script[66] = OP_CHECKSIG; + return true; + } + return false; +} + +// Amount compression: +// * If the amount is 0, output 0 +// * first, divide the amount (in base units) by the largest power of 10 possible; call the exponent e (e is max 9) +// * if e<9, the last digit of the resulting number cannot be 0; store it as d, and drop it (divide by 10) +// * call the result n +// * output 1 + 10*(9*n + d - 1) + e +// * if e==9, we only know the resulting number is not zero, so output 1 + 10*(n - 1) + 9 +// (this is decodable, as d is in [1-9] and e is in [0-9]) + +uint64_t CTxOutCompressor::CompressAmount(uint64_t n) +{ + if (n == 0) + return 0; + int e = 0; + while (((n % 10) == 0) && e < 9) { + n /= 10; + e++; + } + if (e < 9) { + int d = (n % 10); + assert(d >= 1 && d <= 9); + n /= 10; + return 1 + (n*9 + d - 1)*10 + e; + } else { + return 1 + (n - 1)*10 + 9; + } +} + +uint64_t CTxOutCompressor::DecompressAmount(uint64_t x) +{ + // x = 0 OR x = 1+10*(9*n + d - 1) + e OR x = 1+10*(n - 1) + 9 + if (x == 0) + return 0; + x--; + // x = 10*(9*n + d - 1) + e + int e = x % 10; + x /= 10; + uint64_t n = 0; + if (e < 9) { + // x = 9*n + d - 1 + int d = (x % 9) + 1; + x /= 9; + // x = n + n = x*10 + d; + } else { + n = x+1; + } + while (e) { + n *= 10; + e--; + } + return n; +} diff --git a/src/utils/compressor.h b/src/utils/compressor.h new file mode 100644 index 00000000..2fed78bd --- /dev/null +++ b/src/utils/compressor.h @@ -0,0 +1,124 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_COMPRESSOR_H +#define BITCOIN_COMPRESSOR_H + +#include "primitives/transaction.h" +#include "script/script.h" +#include "utils/serialize.h" + +class CKeyID; +class CPubKey; +class CScriptID; + +/** Compact serializer for scripts. + * + * It detects common cases and encodes them much more efficiently. + * 3 special cases are defined: + * * Pay to pubkey hash (encoded as 21 bytes) + * * Pay to script hash (encoded as 21 bytes) + * * Pay to pubkey starting with 0x02, 0x03 or 0x04 (encoded as 33 bytes) + * + * Other scripts up to 121 bytes require 1 byte + script length. Above + * that, scripts up to 16505 bytes require 2 bytes + script length. + */ +class CScriptCompressor +{ +private: + /** + * make this static for now (there are only 6 special scripts defined) + * this can potentially be extended together with a new nVersion for + * transactions, in which case this value becomes dependent on nVersion + * and nHeight of the enclosing transaction. + */ + static const unsigned int nSpecialScripts = 6; + + CScript &script; +protected: + /** + * These check for scripts for which a special case with a shorter encoding is defined. + * They are implemented separately from the CScript test, as these test for exact byte + * sequence correspondences, and are more strict. For example, IsToPubKey also verifies + * whether the public key is valid (as invalid ones cannot be represented in compressed + * form). + */ + bool IsToKeyID(CKeyID &hash) const; + bool IsToScriptID(CScriptID &hash) const; + bool IsToPubKey(CPubKey &pubkey) const; + + bool Compress(std::vector &out) const; + unsigned int GetSpecialSize(unsigned int nSize) const; + bool Decompress(unsigned int nSize, const std::vector &out); +public: + CScriptCompressor(CScript &scriptIn) : script(scriptIn) { } + + unsigned int GetSerializeSize(int nType, int nVersion) const { + std::vector compr; + if (Compress(compr)) + return compr.size(); + unsigned int nSize = script.size() + nSpecialScripts; + return script.size() + VARINT(nSize).GetSerializeSize(nType, nVersion); + } + + template + void Serialize(Stream &s, int nType, int nVersion) const { + std::vector compr; + if (Compress(compr)) { + s << CFlatData(compr); + return; + } + unsigned int nSize = script.size() + nSpecialScripts; + s << VARINT(nSize); + s << CFlatData(script); + } + + template + void Unserialize(Stream &s, int nType, int nVersion) { + unsigned int nSize = 0; + s >> VARINT(nSize); + if (nSize < nSpecialScripts) { + std::vector vch(GetSpecialSize(nSize), 0x00); + s >> REF(CFlatData(vch)); + Decompress(nSize, vch); + return; + } + nSize -= nSpecialScripts; + script.resize(nSize); + s >> REF(CFlatData(script)); + } +}; + +/** wrapper for CTxOut that provides a more compact serialization */ +class CTxOutCompressor +{ +private: + CTxOut &txout; + +public: + static uint64_t CompressAmount(uint64_t nAmount); + static uint64_t DecompressAmount(uint64_t nAmount); + + CTxOutCompressor(CTxOut &txoutIn) : txout(txoutIn) { } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + if (!ser_action.ForRead()) { + uint64_t nVal = CompressAmount(txout.nValue); + READWRITE(VARINT(nVal)); + } else { + uint64_t nVal = 0; + READWRITE(VARINT(nVal)); + txout.nValue = DecompressAmount(nVal); + } + CScriptCompressor cscript(REF(txout.scriptPubKey)); + READWRITE(cscript); + } +}; + +#endif // BITCOIN_COMPRESSOR_H diff --git a/src/utils/core_io.h b/src/utils/core_io.h new file mode 100644 index 00000000..dcd7c6e3 --- /dev/null +++ b/src/utils/core_io.h @@ -0,0 +1,33 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_CORE_IO_H +#define BITCOIN_CORE_IO_H + +#include +#include + +class CBlock; +class CScript; +class CTransaction; +class uint256; +class UniValue; + +// core_read.cpp +extern CScript ParseScript(std::string s); +extern bool DecodeHexTx(CTransaction& tx, const std::string& strHexTx); +extern bool DecodeHexBlk(CBlock&, const std::string& strHexBlk); +extern uint256 ParseHashUV(const UniValue& v, const std::string& strName); +extern uint256 ParseHashStr(const std::string&, const std::string& strName); +extern std::vector ParseHexUV(const UniValue& v, const std::string& strName); + +// core_write.cpp +extern std::string FormatScript(const CScript& script); +extern std::string EncodeHexTx(const CTransaction& tx); +extern void ScriptPubKeyToUniv(const CScript& scriptPubKey, + UniValue& out, bool fIncludeHex); +extern void TxToUniv(const CTransaction& tx, const uint256& hashBlock, UniValue& entry); + +#endif // BITCOIN_CORE_IO_H diff --git a/src/utils/core_read.cpp b/src/utils/core_read.cpp new file mode 100644 index 00000000..190a5d15 --- /dev/null +++ b/src/utils/core_read.cpp @@ -0,0 +1,156 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "utils/core_io.h" + +#include "primitives/block.h" +#include "primitives/transaction.h" +#include "script/script.h" +#include "utils/serialize.h" +#include "utils/streams.h" +#include "univalue/univalue.h" +#include "utils/util.h" +#include "utils/utilstrencodings.h" +#include "version/bcversion.h" + +#include +#include +#include +#include +#include + +using namespace boost; +using namespace boost::algorithm; +using namespace std; + +CScript ParseScript(std::string s) +{ + CScript result; + + static map mapOpNames; + + if (mapOpNames.empty()) + { + for (int op = 0; op <= OP_NOP10; op++) + { + // Allow OP_RESERVED to get into mapOpNames + if (op < OP_NOP && op != OP_RESERVED) + continue; + + const char* name = GetOpName((opcodetype)op); + if (strcmp(name, "OP_UNKNOWN") == 0) + continue; + string strName(name); + mapOpNames[strName] = (opcodetype)op; + // Convenience: OP_ADD and just ADD are both recognized: + replace_first(strName, "OP_", ""); + mapOpNames[strName] = (opcodetype)op; + } + } + + vector words; + split(words, s, is_any_of(" \t\n"), token_compress_on); + + for (std::vector::const_iterator w = words.begin(); w != words.end(); ++w) + { + if (w->empty()) + { + // Empty string, ignore. (boost::split given '' will return one word) + } + else if (all(*w, is_digit()) || + (starts_with(*w, "-") && all(string(w->begin()+1, w->end()), is_digit()))) + { + // Number + int64_t n = atoi64(*w); + result << n; + } + else if (starts_with(*w, "0x") && (w->begin()+2 != w->end()) && IsHex(string(w->begin()+2, w->end()))) + { + // Raw hex data, inserted NOT pushed onto stack: + std::vector raw = ParseHex(string(w->begin()+2, w->end())); + result.insert(result.end(), raw.begin(), raw.end()); + } + else if (w->size() >= 2 && starts_with(*w, "'") && ends_with(*w, "'")) + { + // Single-quoted string, pushed as data. NOTE: this is poor-man's + // parsing, spaces/tabs/newlines in single-quoted strings won't work. + std::vector value(w->begin()+1, w->end()-1); + result << value; + } + else if (mapOpNames.count(*w)) + { + // opcode, e.g. OP_ADD or ADD: + result << mapOpNames[*w]; + } + else + { + throw runtime_error("script parse error"); + } + } + + return result; +} + +bool DecodeHexTx(CTransaction& tx, const std::string& strHexTx) +{ + if (!IsHex(strHexTx)) + return false; + + vector txData(ParseHex(strHexTx)); + CDataStream ssData(txData, SER_NETWORK, PROTOCOL_VERSION); + try { + ssData >> tx; + } + catch (const std::exception &) { + return false; + } + + return true; +} + +bool DecodeHexBlk(CBlock& block, const std::string& strHexBlk) +{ + if (!IsHex(strHexBlk)) + return false; + + std::vector blockData(ParseHex(strHexBlk)); + CDataStream ssBlock(blockData, SER_NETWORK, PROTOCOL_VERSION); + try { + ssBlock >> block; + } + catch (const std::exception &) { + return false; + } + + return true; +} + +uint256 ParseHashUV(const UniValue& v, const string& strName) +{ + string strHex; + if (v.isStr()) + strHex = v.getValStr(); + return ParseHashStr(strHex, strName); // Note: ParseHashStr("") throws a runtime_error +} + +uint256 ParseHashStr(const std::string& strHex, const std::string& strName) +{ + if (!IsHex(strHex)) // Note: IsHex("") is false + throw runtime_error(strName+" must be hexadecimal string (not '"+strHex+"')"); + + uint256 result; + result.SetHex(strHex); + return result; +} + +vector ParseHexUV(const UniValue& v, const string& strName) +{ + string strHex; + if (v.isStr()) + strHex = v.getValStr(); + if (!IsHex(strHex)) + throw runtime_error(strName+" must be hexadecimal string (not '"+strHex+"')"); + return ParseHex(strHex); +} diff --git a/src/utils/core_write.cpp b/src/utils/core_write.cpp new file mode 100644 index 00000000..450df404 --- /dev/null +++ b/src/utils/core_write.cpp @@ -0,0 +1,135 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "utils/core_io.h" + +#include "structs/base58.h" +#include "primitives/transaction.h" +#include "script/script.h" +#include "script/standard.h" +#include "utils/serialize.h" +#include "utils/streams.h" +#include "univalue/univalue.h" +#include "utils/util.h" +#include "utils/utilmoneystr.h" +#include "utils/utilstrencodings.h" + +#include + +using namespace std; + +string FormatScript(const CScript& script) +{ + string ret; + CScript::const_iterator it = script.begin(); + opcodetype op; + while (it != script.end()) { + CScript::const_iterator it2 = it; + vector vch; + if (script.GetOp2(it, op, &vch)) { + if (op == OP_0) { + ret += "0 "; + continue; + } else if ((op >= OP_1 && op <= OP_16) || op == OP_1NEGATE) { + ret += strprintf("%i ", op - OP_1NEGATE - 1); + continue; + } else if (op >= OP_NOP && op <= OP_CHECKMULTISIGVERIFY) { + string str(GetOpName(op)); + if (str.substr(0, 3) == string("OP_")) { + ret += str.substr(3, string::npos) + " "; + continue; + } + } + if (vch.size() > 0) { + ret += strprintf("0x%x 0x%x ", HexStr(it2, it - vch.size()), HexStr(it - vch.size(), it)); + } else { + ret += strprintf("0x%x", HexStr(it2, it)); + } + continue; + } + ret += strprintf("0x%x ", HexStr(it2, script.end())); + break; + } + return ret.substr(0, ret.size() - 1); +} + +string EncodeHexTx(const CTransaction& tx) +{ + CDataStream ssTx(SER_NETWORK, PROTOCOL_VERSION); + ssTx << tx; + return HexStr(ssTx.begin(), ssTx.end()); +} + +void ScriptPubKeyToUniv(const CScript& scriptPubKey, + UniValue& out, bool fIncludeHex) +{ + txnouttype type; + vector addresses; + int nRequired; + + out.pushKV("asm", scriptPubKey.ToString()); + if (fIncludeHex) + out.pushKV("hex", HexStr(scriptPubKey.begin(), scriptPubKey.end())); + + if (!ExtractDestinations(scriptPubKey, type, addresses, nRequired)) { + out.pushKV("type", GetTxnOutputType(type)); + return; + } + + out.pushKV("reqSigs", nRequired); + out.pushKV("type", GetTxnOutputType(type)); + + UniValue a(UniValue::VARR); + BOOST_FOREACH(const CTxDestination& addr, addresses) + a.push_back(CBitcoinAddress(addr).ToString()); + out.pushKV("addresses", a); +} + +void TxToUniv(const CTransaction& tx, const uint256& hashBlock, UniValue& entry) +{ + entry.pushKV("txid", tx.GetHash().GetHex()); + entry.pushKV("version", tx.nVersion); + entry.pushKV("locktime", (int64_t)tx.nLockTime); + + UniValue vin(UniValue::VARR); + BOOST_FOREACH(const CTxIn& txin, tx.vin) { + UniValue in(UniValue::VOBJ); + if (tx.IsCoinBase()) + in.pushKV("coinbase", HexStr(txin.scriptSig.begin(), txin.scriptSig.end())); + else { + in.pushKV("txid", txin.prevout.hash.GetHex()); + in.pushKV("vout", (int64_t)txin.prevout.n); + UniValue o(UniValue::VOBJ); + o.pushKV("asm", txin.scriptSig.ToString()); + o.pushKV("hex", HexStr(txin.scriptSig.begin(), txin.scriptSig.end())); + in.pushKV("scriptSig", o); + } + in.pushKV("sequence", (int64_t)txin.nSequence); + vin.push_back(in); + } + entry.pushKV("vin", vin); + + UniValue vout(UniValue::VARR); + for (unsigned int i = 0; i < tx.vout.size(); i++) { + const CTxOut& txout = tx.vout[i]; + + UniValue out(UniValue::VOBJ); + + UniValue outValue(UniValue::VNUM, FormatMoney(txout.nValue)); + out.pushKV("value", outValue); + out.pushKV("n", (int64_t)i); + + UniValue o(UniValue::VOBJ); + ScriptPubKeyToUniv(txout.scriptPubKey, o, true); + out.pushKV("scriptPubKey", o); + vout.push_back(out); + } + entry.pushKV("vout", vout); + + if (hashBlock != 0) + entry.pushKV("blockhash", hashBlock.GetHex()); + + entry.pushKV("hex", EncodeHexTx(tx)); // the hex-encoded transaction. used the name "hex" to be consistent with the verbose output of "getrawtransaction". +} diff --git a/src/utils/dbwrapper.cpp b/src/utils/dbwrapper.cpp new file mode 100644 index 00000000..4086530e --- /dev/null +++ b/src/utils/dbwrapper.cpp @@ -0,0 +1,657 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +//#include "utils/declare.h" +//#include "utils/dbwrapper.h" + +#include "leveldb/include/leveldb/c.h" +#include "multichain/multichain.h" + +int cs_Database::Zero() +{ + + m_DB=NULL; + m_OpenOptions=NULL; + m_ReadOptions=NULL; + m_IterOptions=NULL; + m_WriteOptions=NULL; + m_SyncOptions=NULL; + m_Cache=NULL; + m_Iterator=NULL; + m_WriteBatch=NULL; + + m_ReadBuffer=NULL; + m_ReadBufferSize=0; + + m_ShMemKey=0; + m_MaxClients=MC_DCT_DB_DEFAULT_MAX_CLIENTS; + m_KeySize=MC_DCT_DB_DEFAULT_MAX_KEY_SIZE; + m_ValueSize=MC_DCT_DB_DEFAULT_MAX_VALUE_SIZE; + m_MaxRows=MC_DCT_DB_DEFAULT_MAX_ROWS; + m_Semaphore=NULL; + m_Signal=0; + m_LastError=0; + + m_hShMem=NULL; + m_lpShMem=NULL; + m_lpShMemSlot=NULL; + + m_Options=MC_OPT_DB_DATABASE_DEFAULT; + m_Status=MC_STT_DB_DATABASE_CLOSED; + m_Log=NULL; + + m_WriteCount=0; + m_DeleteCount=0; + + m_Name[0]=0; + + return MC_ERR_NOERROR; +} + +int cs_Database::Destroy() +{ + if(m_lpShMem) + { +// __US_ShMemUnmap(m_lpShMem); + m_lpShMem=NULL; + } + + if(m_hShMem) + { +// __US_ShMemClose(m_hShMem); + m_hShMem=NULL; + } + + + if(m_ReadBuffer) + { + mc_Delete(m_ReadBuffer); + m_ReadBuffer=NULL; + } + + Zero(); + + return MC_ERR_NOERROR; +} + +int cs_Database::Open(char *name,int Options) +{ + char *err = NULL; +// uint32_t pid; +// double start_time; + + + if(m_ReadBuffer == NULL) + { + m_ReadBuffer=(char*)mc_New(MC_DCT_DB_READ_BUFFER_SIZE); + if(m_ReadBuffer == NULL) + { + return MC_ERR_ALLOCATION; + } + } + + strcpy(m_Name,name); + + m_Options=Options; + + switch(m_Options & MC_OPT_DB_DATABASE_TYPE_MASK) + { + case MC_OPT_DB_DATABASE_LEVELDB: +// return MC_ERR_NOT_SUPPORTED; + + if((m_Options & MC_OPT_DB_DATABASE_DELAYED_OPEN) == 0) + { + m_OpenOptions = (void*)leveldb_options_create(); + m_ReadOptions = (void*)leveldb_readoptions_create(); + m_IterOptions = (void*)leveldb_readoptions_create(); + m_WriteOptions = (void*)leveldb_writeoptions_create(); + m_SyncOptions = (void*)leveldb_writeoptions_create(); + m_Cache = (void*)leveldb_cache_create_lru(128 << 20); + + leveldb_readoptions_set_fill_cache((leveldb_readoptions_t*)m_ReadOptions,0); + leveldb_readoptions_set_fill_cache((leveldb_readoptions_t*)m_IterOptions,0); + leveldb_writeoptions_set_sync((leveldb_writeoptions_t*)m_SyncOptions,1); + + leveldb_options_set_cache((leveldb_options_t*)m_OpenOptions,(leveldb_cache_t*)m_Cache); + leveldb_options_set_max_open_files((leveldb_options_t*)m_OpenOptions,128); + if(Options & MC_OPT_DB_DATABASE_CREATE_IF_MISSING) + { + leveldb_options_set_create_if_missing((leveldb_options_t*)m_OpenOptions, 1); + } + + if(Options & MC_OPT_DB_DATABASE_TRANSACTIONAL) + { + m_WriteBatch=(void*)leveldb_writebatch_create(); + if(m_WriteBatch == NULL) + { + return MC_ERR_DBOPEN_ERROR; + } + leveldb_writebatch_clear((leveldb_writebatch_t*)m_WriteBatch); + } + + m_DB = leveldb_open((leveldb_options_t*)m_OpenOptions, name, &err); + + if (err != NULL) + { + Destroy(); + printf("%s\n",err); + leveldb_free(err); + + return MC_ERR_DBOPEN_ERROR; + } +// m_Iterator=(void*)leveldb_create_iterator((leveldb_t*)m_DB,(leveldb_readoptions_t*)m_IterOptions); + } + + break; + + case MC_OPT_DB_DATABASE_REMOTE_SHMEM: + + return MC_ERR_DBOPEN_ERROR; +/* + if(m_ShMemKey == 0) + { + ierr=MC_ERR_DB_SHMEM_KEY_NOT_SET; + return ierr; + } + + m_hShMem=NULL; + m_lpShMem=NULL; + + if(__US_ShMemOpen(&m_hShMem,m_ShMemKey)) + { + ierr=MC_ERR_DB_SHMEM_CANNOT_OPEN; + return ierr; + } + + m_lpShMem=(uint64_t*)__US_ShMemMap(m_hShMem); + if(m_lpShMem == NULL) + { + __US_ShMemClose(m_hShMem); + m_hShMem=NULL; + ierr=MC_ERR_DB_SHMEM_CANNOT_OPEN; + return ierr; + } + + pid=__US_GetProcessId(); + + m_lpShMemSlot=NULL; + start_time=cs_TimeNow(); + while((m_lpShMemSlot==NULL) && (cs_TimeNow()-start_timem_ReadBufferSize) + { + NewSize=((keylen+vallen)/MC_DCT_DB_READ_BUFFER_SIZE + 1) * MC_DCT_DB_READ_BUFFER_SIZE; + lpNewBuffer=(char*)mc_New(NewSize); + if(lpNewBuffer == NULL) + { + *error=MC_ERR_ALLOCATION; + return NULL; + } + + mc_Delete(m_ReadBuffer); + m_ReadBuffer=lpNewBuffer; + m_ReadBufferSize=NewSize; + } + + memcpy(m_ReadBuffer,lpKey,keylen); + memcpy(m_ReadBuffer+keylen,lpValue,vallen); + m_ReadBuffer[keylen+vallen]=0; + + return m_ReadBuffer; +} + +char *cs_Database::MoveNextKeyLevels(int key_levels,int *error) +{ + *error=MC_ERR_OPERATION_NOT_SUPPORTED; + return NULL; +} + +char *cs_Database::Read(char *key,int key_len,int *value_len,int Options,int *error) +{ + char *err = NULL; + const char *lpIterRead; + char *lpRead; + char *lpNewBuffer; + int NewSize; + size_t vallen; + + int klen=key_len; + + *value_len=0; + *error=MC_ERR_NOERROR; + + if(key == NULL) + { + *error=MC_ERR_INTERNAL_ERROR; + return NULL; + } + if(klen<0)klen=strlen(key); + + if(m_DB == NULL) + { + *error=MC_ERR_DBOPEN_ERROR; + return NULL; + } + + lpRead=NULL; + lpIterRead=NULL; + + switch(m_Options & MC_OPT_DB_DATABASE_TYPE_MASK) + { + case MC_OPT_DB_DATABASE_LEVELDB: + +// *error=MC_ERR_NOT_SUPPORTED; +// return NULL; + + if(Options & MC_OPT_DB_DATABASE_SEEK_ON_READ) + { +// *error=MC_ERR_OPERATION_NOT_SUPPORTED; +// return NULL; + lpIterRead=NULL; + + if(m_Iterator) + { + leveldb_iter_destroy((leveldb_iterator_t*)m_Iterator); + m_Iterator=NULL; + } + m_Iterator=(void*)leveldb_create_iterator((leveldb_t*)m_DB,(leveldb_readoptions_t*)m_IterOptions); + + leveldb_iter_seek((leveldb_iterator_t*)m_Iterator, key, klen); + if(leveldb_iter_valid((leveldb_iterator_t*)m_Iterator)) + { + lpIterRead=leveldb_iter_value((leveldb_iterator_t*)m_Iterator, &vallen); + } + } + else + { + lpRead = leveldb_get((leveldb_t*)m_DB, (leveldb_readoptions_t*)m_ReadOptions, key, klen, &vallen, &err); + } + + if (err != NULL) + { + leveldb_free(err); + *error=MC_ERR_INTERNAL_ERROR; + return NULL; + } + + if((lpRead == NULL) && (lpIterRead == NULL)) + { + return NULL; + } + + *value_len=vallen; + + break; + default: + *error=MC_ERR_DBOPEN_ERROR; + return NULL; + break; + } + + if(*value_len+1>m_ReadBufferSize) + { + NewSize=((*value_len)/MC_DCT_DB_READ_BUFFER_SIZE + 1) * MC_DCT_DB_READ_BUFFER_SIZE; + lpNewBuffer=(char*)mc_New(NewSize); + if(lpNewBuffer == NULL) + { + *value_len=0; + *error=MC_ERR_ALLOCATION; + return NULL; + } + + mc_Delete(m_ReadBuffer); + m_ReadBuffer=lpNewBuffer; + m_ReadBufferSize=NewSize; + } + + if(lpRead) + { + memcpy(m_ReadBuffer,lpRead,*value_len); + } + if(lpIterRead) + { + memcpy(m_ReadBuffer,lpIterRead,*value_len); + } + m_ReadBuffer[*value_len]=0; + + switch(m_Options & MC_OPT_DB_DATABASE_TYPE_MASK) + { + case MC_OPT_DB_DATABASE_LEVELDB: + + if(lpRead) + { + leveldb_free(lpRead); + } + + break; + } + + return m_ReadBuffer; +} + +int cs_Database::Delete(char *key,int key_len,int Options) +{ + char *err = NULL; + int klen=key_len; + + m_DeleteCount++; + + if(key == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + if(klen<0)klen=strlen(key); + + if(m_DB == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + + switch(m_Options & MC_OPT_DB_DATABASE_TYPE_MASK) + { + case MC_OPT_DB_DATABASE_LEVELDB: +// return MC_ERR_NOT_SUPPORTED; + + + if(Options & MC_OPT_DB_DATABASE_TRANSACTIONAL) + { + leveldb_writebatch_delete((leveldb_writebatch_t*)m_WriteBatch, key, klen); + } + else + { + leveldb_delete((leveldb_t*)m_DB, (leveldb_writeoptions_t*)m_SyncOptions, key, klen, &err); + } + + if (err != NULL) + { + leveldb_free(err); + + return MC_ERR_INTERNAL_ERROR; + } + + break; + } + + return MC_ERR_NOERROR; + +} + +int cs_Database::Commit(int Options) +{ + char *err = NULL; + char msg[100]; + + sprintf(msg,"Writes: %6d; Deletes: %6d;",m_WriteCount,m_DeleteCount); +// cs_LogMessage(m_Log,MC_LOG_REPORT,"C-0083","Commit",msg); + + m_WriteCount=0; + m_DeleteCount=0; + + if(m_DB == NULL) + { + return MC_ERR_DBOPEN_ERROR; + } + + switch(m_Options & MC_OPT_DB_DATABASE_TYPE_MASK) + { + case MC_OPT_DB_DATABASE_LEVELDB: +// return MC_ERR_NOT_SUPPORTED; + + + if(Options & MC_OPT_DB_DATABASE_TRANSACTIONAL) + { + leveldb_write((leveldb_t*)m_DB, (leveldb_writeoptions_t*)m_WriteOptions, (leveldb_writebatch_t*)m_WriteBatch, &err); + leveldb_writebatch_clear((leveldb_writebatch_t*)m_WriteBatch); + } + + if (err != NULL) + { + leveldb_free(err); + + return MC_ERR_INTERNAL_ERROR; + } + + break; + } + return MC_ERR_NOERROR; +} + +int cs_Database::Synchronize() +{ + Close(); + + return Open(m_Name,m_Options); +} + + +int cs_Database::SetOption(const char* option, int suboption, int value) +{ + if(strcmp(option,"ShMemKey") == 0) + { + m_ShMemKey=value; + } + + if(strcmp(option,"KeySize") == 0) + { + m_KeySize=value; + } + + if(strcmp(option,"ValueSize") == 0) + { + m_ValueSize=value; + } + + if(strcmp(option,"MaxRows") == 0) + { + m_ValueSize=value; + } + + return MC_ERR_NOERROR; + +} + + + + + diff --git a/src/utils/dbwrapper.h b/src/utils/dbwrapper.h new file mode 100644 index 00000000..2339365d --- /dev/null +++ b/src/utils/dbwrapper.h @@ -0,0 +1,212 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAINDB_H +#define MULTICHAINDB_H + +#define MC_DCT_DB_MAX_PATH 1024 + +#define MC_DCT_DB_READ_BUFFER_SIZE 4096 +#define MC_DCT_DB_DEFAULT_MAX_CLIENTS 256 +#define MC_DCT_DB_DEFAULT_MAX_ROWS 256 +#define MC_DCT_DB_DEFAULT_MAX_KEY_SIZE 256 +#define MC_DCT_DB_DEFAULT_MAX_VALUE_SIZE 256 +#define MC_DCT_DB_DEFAULT_MAX_TIME_PER_SWEEP 1.f +#define MC_DCT_DB_DEFAULT_MAX_SHMEM_TIMEOUT 3.f + +#define MC_STT_DB_DATABASE_CLOSED 0x00000000 +#define MC_STT_DB_DATABASE_OPENED 0x00000001 +#define MC_STT_DB_DATABASE_SHMEM_OPENED 0x00000002 + + +#define MC_OPT_DB_DATABASE_DEFAULT 0x00000000 +#define MC_OPT_DB_DATABASE_CREATE_IF_MISSING 0x00000001 +#define MC_OPT_DB_DATABASE_TRANSACTIONAL 0x00000002 +#define MC_OPT_DB_DATABASE_READONLY 0x00000004 +#define MC_OPT_DB_DATABASE_SEEK_ON_READ 0x00000010 +#define MC_OPT_DB_DATABASE_DELAYED_OPEN 0x00000020 +#define MC_OPT_DB_DATABASE_OPTION_MASK 0x000FFFFF +#define MC_OPT_DB_DATABASE_LEVELDB 0x00100000 +#define MC_OPT_DB_DATABASE_FSR_UTXOC_BLOCKS 0x00200000 +#define MC_OPT_DB_DATABASE_REMOTE_SHMEM 0x01000000 +#define MC_OPT_DB_DATABASE_TYPE_MASK 0x0FF00000 + +#define MC_OFF_DB_SHMEM_HEAD_STATUS 0 +#define MC_OFF_DB_SHMEM_HEAD_SIGNAL 1 +#define MC_OFF_DB_SHMEM_HEAD_CLIENT_COUNT 2 +#define MC_OFF_DB_SHMEM_HEAD_MAX_CLIENT_COUNT 3 +#define MC_OFF_DB_SHMEM_HEAD_REQUEST_PROCESS_ID 4 +#define MC_OFF_DB_SHMEM_HEAD_RESPONSE_PROCESS_ID 5 +#define MC_OFF_DB_SHMEM_HEAD_RESPONSE_SLOT_ID 6 +#define MC_OFF_DB_SHMEM_HEAD_KEY_SIZE 8 +#define MC_OFF_DB_SHMEM_HEAD_VALUE_SIZE 9 +#define MC_OFF_DB_SHMEM_HEAD_ROW_SIZE 10 +#define MC_OFF_DB_SHMEM_HEAD_SLOT_SIZE 11 +#define MC_OFF_DB_SHMEM_HEAD_MAX_ROWS 12 +#define MC_OFF_DB_SHMEM_HEAD_DATA_OFFSET 16 +#define MC_OFF_DB_SHMEM_HEAD_SLOT_DATA_OFFSET 17 +#define MC_OFF_DB_SHMEM_HEAD_ROW_DATA_OFFSET 18 +#define MC_OFF_DB_SHMEM_HEAD_FIELD_COUNT 32 + +#define MC_OFF_DB_SHMEM_SLOT_PROCESS_ID 0 +#define MC_OFF_DB_SHMEM_SLOT_ROW_COUNT 1 +#define MC_OFF_DB_SHMEM_SLOT_RESULT 2 +#define MC_OFF_DB_SHMEM_SLOT_TIMESTAMP 3 +#define MC_OFF_DB_SHMEM_SLOT_FIXEDKEYSIZE 4 +#define MC_OFF_DB_SHMEM_SLOT_DATA 8 + +#define MC_OFF_DB_SHMEM_ROW_REQUEST 0 +#define MC_OFF_DB_SHMEM_ROW_RESPONSE 1 +#define MC_OFF_DB_SHMEM_ROW_KEY_SIZE 2 +#define MC_OFF_DB_SHMEM_ROW_VALUE_SIZE 3 +#define MC_OFF_DB_SHMEM_ROW_DATA 4 + +#define MC_PRT_DB_SHMEM_SLOT_RESULT_UNDEFINED 0 +#define MC_PRT_DB_SHMEM_SLOT_RESULT_SUCCESS 1 +#define MC_PRT_DB_SHMEM_SLOT_RESULT_ERROR 16 + +#define MC_PRT_DB_SHMEM_ROW_REQUEST_READ 1 +#define MC_PRT_DB_SHMEM_ROW_REQUEST_WRITE 2 +#define MC_PRT_DB_SHMEM_ROW_REQUEST_DELETE 3 +#define MC_PRT_DB_SHMEM_ROW_REQUEST_COMMIT 4 +#define MC_PRT_DB_SHMEM_ROW_REQUEST_READALL 5 +#define MC_PRT_DB_SHMEM_ROW_REQUEST_CLOSE 16 + +#define MC_PRT_DB_SHMEM_ROW_RESPONSE_UNDEFINED 0 +#define MC_PRT_DB_SHMEM_ROW_RESPONSE_SUCCESS 1 +#define MC_PRT_DB_SHMEM_ROW_RESPONSE_NULL 1 +#define MC_PRT_DB_SHMEM_ROW_RESPONSE_NOT_NULL 2 +#define MC_PRT_DB_SHMEM_ROW_RESPONSE_ERROR 16 + + + +#ifdef __cplusplus +extern "C" { +#endif + +typedef struct mc_Database +{ + mc_Database() + { + Zero(); + } + + ~mc_Database() + { + Destroy(); + } + + void * m_DB; + void * m_OpenOptions; + void * m_ReadOptions; + void * m_IterOptions; + void * m_WriteOptions; + void * m_SyncOptions; + void * m_Cache; + void * m_WriteBatch; + void * m_Iterator; + + uint32_t m_ShMemKey; + int m_MaxClients; + int m_KeySize; + int m_ValueSize; + int m_MaxRows; + void * m_Semaphore; + int m_Signal; + int m_LastError; + uint64_t *m_lpHandlerShMem; + + void * m_hShMem; + uint64_t *m_lpShMem; + uint64_t *m_lpShMemSlot; + + char *m_ReadBuffer; + int m_ReadBufferSize; + int m_Status; /* Database status - MC_STT_DB_DATABASE constants*/ + int m_Options; + void * m_Log; + int m_WriteCount; + int m_DeleteCount; + + char m_Name[MC_DCT_DB_MAX_PATH]; + + char m_LogFile[MC_DCT_DB_MAX_PATH]; + + char *m_LogBuffer; + int m_LogSize; + + int LogWrite(int op, + char *key, /* key */ + int key_len, /* key length, -1 if strlen is should be used */ + char *value, /* value */ + int value_len /* value length, -1 if strlen is used */ + ); + int LogFlush(); + + int Destroy(); + int Zero(); + + int SetOption(const char *option,int suboption,int value); + + int Open( /* Open database */ + char *name, /* Database name, path in case of leveldb */ + int Options /* Open options, MC_OPT_DB_DATABASE constants */ + ); + int Close(); /* Close databse */ + int Write( /* Writes (key,value) to database */ + char *key, /* key */ + int key_len, /* key length, -1 if strlen is should be used */ + char *value, /* value */ + int value_len, /* value length, -1 if strlen is used */ + int Options /* Options - not used */ + ); + char *Read( /* Reads value for specified key, no freeing required, pointer is valid until next read */ + char *key, /* key */ + int key_len, /* key length, -1 if strlen is should be used */ + int *value_len, /* value length */ + int Options, /* Options - not used */ + int *error /* Error */ + ); + int BatchRead( + char *Data, + int *results, + int count, + int Options + ); + int Delete( /* Delete key from database */ + char *key, /* key */ + int key_len, /* key length, -1 if strlen is should be used */ + int Options /* Options - not used */ + ); + int Commit( + int Options /* Options - not used */ + ); + + void Lock(int write_mode); + void UnLock(); + int Synchronize(); + + char *MoveNext( + int *error + ); + + char *MoveNextKeyLevels( + int key_levels, + int *error + ); + +} cs_Database; + + + + +#ifdef __cplusplus +} +#endif + + + + + +#endif /* MULTICHAINDB_H */ + diff --git a/src/utils/declare.h b/src/utils/declare.h new file mode 100644 index 00000000..f15bbf2c --- /dev/null +++ b/src/utils/declare.h @@ -0,0 +1,280 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAIN_DECLARE_H +#define MULTICHAIN_DECLARE_H + +#include "utils/define.h" + +#define MC_DCT_TERM_BUFFER_SIZE 25165824 + +/* Classes */ + + +#ifdef __cplusplus +extern "C" { +#endif + + +typedef struct mc_MapStringIndex +{ + void *mapObject; + mc_MapStringIndex() + { + mapObject=NULL; + Init(); + } + + ~mc_MapStringIndex() + { + Destroy(); + } + void Init(); + void Add(const char* key,int value); + void Add(const unsigned char* key,int size,int value); + void Remove(const char* key,int size); + int Get(const char* key); + int Get(const unsigned char* key,int size); + void Destroy(); + void Clear(); +} mc_MapStringIndex; + +typedef struct mc_MapStringString +{ + void *mapObject; + mc_MapStringString() + { + Init(); + } + + ~mc_MapStringString() + { + Destroy(); + } + void Init(); + void Add(const char* key,const char* value); + const char * Get(const char* key); + void Destroy(); + int GetCount(); +} mc_MapStringString; + +typedef struct mc_Buffer +{ + mc_Buffer() + { + Zero(); + } + + ~mc_Buffer() + { + Destroy(); + } + + mc_MapStringIndex *m_lpIndex; + unsigned char *m_lpData; + int m_AllocSize; + int m_Size; + int m_KeySize; + int m_RowSize; + int m_Count; + uint32_t m_Mode; + + void Zero(); + int Destroy(); + int Initialize(int KeySize,int TotalSize,uint32_t Mode); + + int Clear(); + int Realloc(int Rows); + int Add(const void *lpKey,const void *lpValue); + int Add(const void *lpKeyValue); + int Seek(void *lpKey); + unsigned char *GetRow(int RowID); + int PutRow(int RowID,const void *lpKey,const void *lpValue); + int GetCount(); + int SetCount(int count); + + int Sort(); + + void CopyFrom(mc_Buffer *source); + +} mc_Buffer; + + +typedef struct mc_List +{ + mc_List() + { + Zero(); + } + + ~mc_List() + { + Destroy(); + } + + unsigned char *m_lpData; + int m_AllocSize; + int m_Size; + int m_Pos; + int m_ItemSize; + + void Zero(); + int Destroy(); + + void Clear(); + int Put(unsigned char *ptr, int size); + unsigned char *First(); + unsigned char *Next(); + +} mc_List; + + +typedef struct mc_SHA256 +{ + void *m_HashObject; + void Init(); + void Destroy(); + void Reset(); + void Write(const void *lpData,int size); + void GetHash(unsigned char *hash); + + mc_SHA256() + { + Init(); + } + + ~mc_SHA256() + { + Destroy(); + } +} mc_SHA256; + + +struct mc_TerminalInput +{ + char m_Prompt[64]; + char m_Data[MC_DCT_TERM_BUFFER_SIZE]; + char m_Line[MC_DCT_TERM_BUFFER_SIZE]; + char m_Cache[MC_DCT_TERM_BUFFER_SIZE]; + int m_Offsets[100]; + + int m_HistoryLines; + int m_BufferSize; + int m_ThisLine; + int m_FirstLine; + int m_LoadedLine; + int m_TerminalCols; + int m_TerminalRows; + + char *GetLine(); + int SetPrompt(const char *prompt); + int Prompt(); + void AddLine(); + int LoadLine(int line); + void SaveLine(); + void MoveBack(int offset); + int TerminalCols(); + int IsAlphaNumeric(char c); + int LoadDataFromLog(const char *fileName); + + mc_TerminalInput() + { + int i; + strcpy(m_Prompt,""); + m_HistoryLines=100; + m_BufferSize=MC_DCT_TERM_BUFFER_SIZE; + m_ThisLine=0; + m_FirstLine=0; + m_LoadedLine=0; + memset(m_Line,0,m_BufferSize); + memset(m_Data,0,m_BufferSize); + for(i=0;i +#include +#include + +#ifndef WIN32 + +#include +#define _O_BINARY 0 +#include + +#else + +#include + +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* Error codes */ + +#define MC_ERR_NOERROR 0x00000000 + +/* General errors */ + +#define MC_ERR_ALLOCATION 0x00000001 +#define MC_ERR_TOO_FEW_PARAMETERS 0x00000002 +#define MC_ERR_MISSING_PARAMETER 0x00000003 +#define MC_ERR_OPERATION_NOT_SUPPORTED 0x00000004 +#define MC_ERR_INVALID_PARAMETER_VALUE 0x00000005 +#define MC_ERR_INTERNAL_ERROR 0x00000006 +#define MC_ERR_FILE_READ_ERROR 0x00000007 +#define MC_ERR_FILE_WRITE_ERROR 0x00000008 +#define MC_ERR_CONNECTION_ERROR 0x00000009 +#define MC_ERR_DBOPEN_ERROR 0x0000000A +#define MC_ERR_CORRUPTED 0x0000000B +#define MC_ERR_NOT_ALLOWED 0x0000000C +#define MC_ERR_WRONG_SCRIPT 0x0000000D +#define MC_ERR_FOUND 0x0000000E +#define MC_ERR_NOT_FOUND 0x0000000F +#define MC_ERR_NOT_SUPPORTED 0x00000010 +#define MC_ERR_ERROR_IN_SCRIPT 0x00000011 + + +#define MC_FOM_NONE 0x00000000 +#define MC_FOM_RELATIVE_TO_DATADIR 0x00000001 +#define MC_FOM_RELATIVE_MASK 0x0000000F +#define MC_FOM_CREATE_DIR 0x00000100 + +#define MC_BUF_MODE_DEFAULT 0x00000000 +#define MC_BUF_MODE_MAP 0x00000001 + + +#define MC_PRM_NETWORK_NAME_MAX_SIZE 32 + +#endif /* MULTICHAIN_DEFINE_H */ + diff --git a/src/utils/mruset.h b/src/utils/mruset.h new file mode 100644 index 00000000..cd55abc2 --- /dev/null +++ b/src/utils/mruset.h @@ -0,0 +1,70 @@ +// Copyright (c) 2012 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_MRUSET_H +#define BITCOIN_MRUSET_H + +#include +#include +#include + +/** STL-like set container that only keeps the most recent N elements. */ +template +class mruset +{ +public: + typedef T key_type; + typedef T value_type; + typedef typename std::set::iterator iterator; + typedef typename std::set::const_iterator const_iterator; + typedef typename std::set::size_type size_type; + +protected: + std::set set; + std::deque queue; + size_type nMaxSize; + +public: + mruset(size_type nMaxSizeIn = 0) { nMaxSize = nMaxSizeIn; } + iterator begin() const { return set.begin(); } + iterator end() const { return set.end(); } + size_type size() const { return set.size(); } + bool empty() const { return set.empty(); } + iterator find(const key_type& k) const { return set.find(k); } + size_type count(const key_type& k) const { return set.count(k); } + void clear() + { + set.clear(); + queue.clear(); + } + bool inline friend operator==(const mruset& a, const mruset& b) { return a.set == b.set; } + bool inline friend operator==(const mruset& a, const std::set& b) { return a.set == b; } + bool inline friend operator<(const mruset& a, const mruset& b) { return a.set < b.set; } + std::pair insert(const key_type& x) + { + std::pair ret = set.insert(x); + if (ret.second) { + if (nMaxSize && queue.size() == nMaxSize) { + set.erase(queue.front()); + queue.pop_front(); + } + queue.push_back(x); + } + return ret; + } + size_type max_size() const { return nMaxSize; } + size_type max_size(size_type s) + { + if (s) + while (queue.size() > s) { + set.erase(queue.front()); + queue.pop_front(); + } + nMaxSize = s; + return nMaxSize; + } +}; + +#endif // BITCOIN_MRUSET_H diff --git a/src/utils/random.cpp b/src/utils/random.cpp new file mode 100644 index 00000000..54453fd8 --- /dev/null +++ b/src/utils/random.cpp @@ -0,0 +1,139 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "utils/random.h" + +#ifdef WIN32 +#include "utils/compat.h" // for Windows API +#endif +#include "utils/serialize.h" // for begin_ptr(vec) +#include "utils/util.h" // for LogPrint() +#include "utils/utilstrencodings.h" // for GetTime() + +#include + +#ifndef WIN32 +#include +#endif + +#include +#include +#include + +static inline int64_t GetPerformanceCounter() +{ + int64_t nCounter = 0; +#ifdef WIN32 + QueryPerformanceCounter((LARGE_INTEGER*)&nCounter); +#else + timeval t; + gettimeofday(&t, NULL); + nCounter = (int64_t)(t.tv_sec * 1000000 + t.tv_usec); +#endif + return nCounter; +} + +void RandAddSeed() +{ + // Seed with CPU performance counter + int64_t nCounter = GetPerformanceCounter(); + RAND_add(&nCounter, sizeof(nCounter), 1.5); + OPENSSL_cleanse((void*)&nCounter, sizeof(nCounter)); +} + +void RandAddSeedPerfmon() +{ + RandAddSeed(); + + // This can take up to 2 seconds, so only do it every 10 minutes + static int64_t nLastPerfmon; + if (GetTime() < nLastPerfmon + 10 * 60) + return; + nLastPerfmon = GetTime(); + +#ifdef WIN32 + // Don't need this on Linux, OpenSSL automatically uses /dev/urandom + // Seed with the entire set of perfmon data + std::vector vData(250000, 0); + long ret = 0; + unsigned long nSize = 0; + const size_t nMaxSize = 10000000; // Bail out at more than 10MB of performance data + while (true) { + nSize = vData.size(); + ret = RegQueryValueExA(HKEY_PERFORMANCE_DATA, "Global", NULL, NULL, begin_ptr(vData), &nSize); + if (ret != ERROR_MORE_DATA || vData.size() >= nMaxSize) + break; + vData.resize(std::max((vData.size() * 3) / 2, nMaxSize)); // Grow size of buffer exponentially + } + RegCloseKey(HKEY_PERFORMANCE_DATA); + if (ret == ERROR_SUCCESS) { + RAND_add(begin_ptr(vData), nSize, nSize / 100.0); + OPENSSL_cleanse(begin_ptr(vData), nSize); + LogPrint("rand", "%s: %lu bytes\n", __func__, nSize); + } else { + static bool warned = false; // Warn only once + if (!warned) { + LogPrintf("%s: Warning: RegQueryValueExA(HKEY_PERFORMANCE_DATA) failed with code %i\n", __func__, ret); + warned = true; + } + } +#endif +} + +void GetRandBytes(unsigned char* buf, int num) +{ + if (RAND_bytes(buf, num) != 1) { + LogPrintf("%s: OpenSSL RAND_bytes() failed with error: %s\n", __func__, ERR_error_string(ERR_get_error(), NULL)); + assert(false); + } +} + +uint64_t GetRand(uint64_t nMax) +{ + if (nMax == 0) + return 0; + + // The range of the random source must be a multiple of the modulus + // to give every possible output value an equal possibility + uint64_t nRange = (std::numeric_limits::max() / nMax) * nMax; + uint64_t nRand = 0; + do { + GetRandBytes((unsigned char*)&nRand, sizeof(nRand)); + } while (nRand >= nRange); + return (nRand % nMax); +} + +int GetRandInt(int nMax) +{ + return GetRand(nMax); +} + +uint256 GetRandHash() +{ + uint256 hash; + GetRandBytes((unsigned char*)&hash, sizeof(hash)); + return hash; +} + +uint32_t insecure_rand_Rz = 11; +uint32_t insecure_rand_Rw = 11; +void seed_insecure_rand(bool fDeterministic) +{ + // The seed values have some unlikely fixed points which we avoid. + if (fDeterministic) { + insecure_rand_Rz = insecure_rand_Rw = 11; + } else { + uint32_t tmp; + do { + GetRandBytes((unsigned char*)&tmp, 4); + } while (tmp == 0 || tmp == 0x9068ffffU); + insecure_rand_Rz = tmp; + do { + GetRandBytes((unsigned char*)&tmp, 4); + } while (tmp == 0 || tmp == 0x464fffffU); + insecure_rand_Rw = tmp; + } +} diff --git a/src/utils/random.h b/src/utils/random.h new file mode 100644 index 00000000..a71040d4 --- /dev/null +++ b/src/utils/random.h @@ -0,0 +1,50 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_RANDOM_H +#define BITCOIN_RANDOM_H + +#include "structs/uint256.h" + +#include + +/** + * Seed OpenSSL PRNG with additional entropy data + */ +void RandAddSeed(); +void RandAddSeedPerfmon(); + +/** + * Functions to gather random data via the OpenSSL PRNG + */ +void GetRandBytes(unsigned char* buf, int num); +uint64_t GetRand(uint64_t nMax); +int GetRandInt(int nMax); +uint256 GetRandHash(); + +/** + * Seed insecure_rand using the random pool. + * @param Deterministic Use a deterministic seed + */ +void seed_insecure_rand(bool fDeterministic = false); + +/** + * MWC RNG of George Marsaglia + * This is intended to be fast. It has a period of 2^59.3, though the + * least significant 16 bits only have a period of about 2^30.1. + * + * @return random value + */ +extern uint32_t insecure_rand_Rz; +extern uint32_t insecure_rand_Rw; +static inline uint32_t insecure_rand(void) +{ + insecure_rand_Rz = 36969 * (insecure_rand_Rz & 65535) + (insecure_rand_Rz >> 16); + insecure_rand_Rw = 18000 * (insecure_rand_Rw & 65535) + (insecure_rand_Rw >> 16); + return (insecure_rand_Rw << 16) + insecure_rand_Rz; +} + +#endif // BITCOIN_RANDOM_H diff --git a/src/utils/serialize.h b/src/utils/serialize.h new file mode 100644 index 00000000..a30fa7fc --- /dev/null +++ b/src/utils/serialize.h @@ -0,0 +1,812 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_SERIALIZE_H +#define BITCOIN_SERIALIZE_H + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +class CScript; + +static const unsigned int MAX_SIZE = 0x02000000; + +/** + * Used to bypass the rule against non-const reference to temporary + * where it makes sense with wrappers such as CFlatData or CTxDB + */ +template +inline T& REF(const T& val) +{ + return const_cast(val); +} + +/** + * Used to acquire a non-const pointer "this" to generate bodies + * of const serialization operations from a template + */ +template +inline T* NCONST_PTR(const T* val) +{ + return const_cast(val); +} + +/** + * Get begin pointer of vector (non-const version). + * @note These functions avoid the undefined case of indexing into an empty + * vector, as well as that of indexing after the end of the vector. + */ +template +inline T* begin_ptr(std::vector& v) +{ + return v.empty() ? NULL : &v[0]; +} +/** Get begin pointer of vector (const version) */ +template +inline const T* begin_ptr(const std::vector& v) +{ + return v.empty() ? NULL : &v[0]; +} +/** Get end pointer of vector (non-const version) */ +template +inline T* end_ptr(std::vector& v) +{ + return v.empty() ? NULL : (&v[0] + v.size()); +} +/** Get end pointer of vector (const version) */ +template +inline const T* end_ptr(const std::vector& v) +{ + return v.empty() ? NULL : (&v[0] + v.size()); +} + +///////////////////////////////////////////////////////////////// +// +// Templates for serializing to anything that looks like a stream, +// i.e. anything that supports .read(char*, size_t) and .write(char*, size_t) +// + +enum +{ + // primary actions + SER_NETWORK = (1 << 0), + SER_DISK = (1 << 1), + SER_GETHASH = (1 << 2), +}; + +#define READWRITE(obj) (::SerReadWrite(s, (obj), nType, nVersion, ser_action)) + +/** + * Implement three methods for serializable objects. These are actually wrappers over + * "SerializationOp" template, which implements the body of each class' serialization + * code. Adding "ADD_SERIALIZE_METHODS" in the body of the class causes these wrappers to be + * added as members. + */ +#define ADD_SERIALIZE_METHODS \ + size_t GetSerializeSize(int nType, int nVersion) const { \ + CSizeComputer s(nType, nVersion); \ + NCONST_PTR(this)->SerializationOp(s, CSerActionSerialize(), nType, nVersion);\ + return s.size(); \ + } \ + template \ + void Serialize(Stream& s, int nType, int nVersion) const { \ + NCONST_PTR(this)->SerializationOp(s, CSerActionSerialize(), nType, nVersion);\ + } \ + template \ + void Unserialize(Stream& s, int nType, int nVersion) { \ + SerializationOp(s, CSerActionUnserialize(), nType, nVersion); \ + } + + + +/* + * Basic Types + */ +#define WRITEDATA(s, obj) s.write((char*)&(obj), sizeof(obj)) +#define READDATA(s, obj) s.read((char*)&(obj), sizeof(obj)) + +inline unsigned int GetSerializeSize(char a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(signed char a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(unsigned char a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(signed short a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(unsigned short a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(signed int a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(unsigned int a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(signed long a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(unsigned long a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(signed long long a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(unsigned long long a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(float a, int, int=0) { return sizeof(a); } +inline unsigned int GetSerializeSize(double a, int, int=0) { return sizeof(a); } + +template inline void Serialize(Stream& s, char a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, signed char a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, unsigned char a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, signed short a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, unsigned short a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, signed int a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, unsigned int a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, signed long a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, unsigned long a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, signed long long a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, unsigned long long a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, float a, int, int=0) { WRITEDATA(s, a); } +template inline void Serialize(Stream& s, double a, int, int=0) { WRITEDATA(s, a); } + +template inline void Unserialize(Stream& s, char& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, signed char& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, unsigned char& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, signed short& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, unsigned short& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, signed int& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, unsigned int& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, signed long& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, unsigned long& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, signed long long& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, unsigned long long& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, float& a, int, int=0) { READDATA(s, a); } +template inline void Unserialize(Stream& s, double& a, int, int=0) { READDATA(s, a); } + +inline unsigned int GetSerializeSize(bool a, int, int=0) { return sizeof(char); } +template inline void Serialize(Stream& s, bool a, int, int=0) { char f=a; WRITEDATA(s, f); } +template inline void Unserialize(Stream& s, bool& a, int, int=0) { char f; READDATA(s, f); a=f; } + + + + + + +/** + * Compact Size + * size < 253 -- 1 byte + * size <= USHRT_MAX -- 3 bytes (253 + 2 bytes) + * size <= UINT_MAX -- 5 bytes (254 + 4 bytes) + * size > UINT_MAX -- 9 bytes (255 + 8 bytes) + */ +inline unsigned int GetSizeOfCompactSize(uint64_t nSize) +{ + if (nSize < 253) return sizeof(unsigned char); + else if (nSize <= std::numeric_limits::max()) return sizeof(unsigned char) + sizeof(unsigned short); + else if (nSize <= std::numeric_limits::max()) return sizeof(unsigned char) + sizeof(unsigned int); + else return sizeof(unsigned char) + sizeof(uint64_t); +} + +template +void WriteCompactSize(Stream& os, uint64_t nSize) +{ + if (nSize < 253) + { + unsigned char chSize = nSize; + WRITEDATA(os, chSize); + } + else if (nSize <= std::numeric_limits::max()) + { + unsigned char chSize = 253; + unsigned short xSize = nSize; + WRITEDATA(os, chSize); + WRITEDATA(os, xSize); + } + else if (nSize <= std::numeric_limits::max()) + { + unsigned char chSize = 254; + unsigned int xSize = nSize; + WRITEDATA(os, chSize); + WRITEDATA(os, xSize); + } + else + { + unsigned char chSize = 255; + uint64_t xSize = nSize; + WRITEDATA(os, chSize); + WRITEDATA(os, xSize); + } + return; +} + +template +uint64_t ReadCompactSize(Stream& is) +{ + unsigned char chSize; + READDATA(is, chSize); + uint64_t nSizeRet = 0; + if (chSize < 253) + { + nSizeRet = chSize; + } + else if (chSize == 253) + { + unsigned short xSize; + READDATA(is, xSize); + nSizeRet = xSize; + if (nSizeRet < 253) + throw std::ios_base::failure("non-canonical ReadCompactSize()"); + } + else if (chSize == 254) + { + unsigned int xSize; + READDATA(is, xSize); + nSizeRet = xSize; + if (nSizeRet < 0x10000u) + throw std::ios_base::failure("non-canonical ReadCompactSize()"); + } + else + { + uint64_t xSize; + READDATA(is, xSize); + nSizeRet = xSize; + if (nSizeRet < 0x100000000ULL) + throw std::ios_base::failure("non-canonical ReadCompactSize()"); + } + if (nSizeRet > (uint64_t)MAX_SIZE) + throw std::ios_base::failure("ReadCompactSize() : size too large"); + return nSizeRet; +} + +/** + * Variable-length integers: bytes are a MSB base-128 encoding of the number. + * The high bit in each byte signifies whether another digit follows. To make + * sure the encoding is one-to-one, one is subtracted from all but the last digit. + * Thus, the byte sequence a[] with length len, where all but the last byte + * has bit 128 set, encodes the number: + * + * (a[len-1] & 0x7F) + sum(i=1..len-1, 128^i*((a[len-i-1] & 0x7F)+1)) + * + * Properties: + * * Very small (0-127: 1 byte, 128-16511: 2 bytes, 16512-2113663: 3 bytes) + * * Every integer has exactly one encoding + * * Encoding does not depend on size of original integer type + * * No redundancy: every (infinite) byte sequence corresponds to a list + * of encoded integers. + * + * 0: [0x00] 256: [0x81 0x00] + * 1: [0x01] 16383: [0xFE 0x7F] + * 127: [0x7F] 16384: [0xFF 0x00] + * 128: [0x80 0x00] 16511: [0x80 0xFF 0x7F] + * 255: [0x80 0x7F] 65535: [0x82 0xFD 0x7F] + * 2^32: [0x8E 0xFE 0xFE 0xFF 0x00] + */ + +template +inline unsigned int GetSizeOfVarInt(I n) +{ + int nRet = 0; + while(true) { + nRet++; + if (n <= 0x7F) + break; + n = (n >> 7) - 1; + } + return nRet; +} + +template +void WriteVarInt(Stream& os, I n) +{ + unsigned char tmp[(sizeof(n)*8+6)/7]; + int len=0; + while(true) { + tmp[len] = (n & 0x7F) | (len ? 0x80 : 0x00); + if (n <= 0x7F) + break; + n = (n >> 7) - 1; + len++; + } + do { + WRITEDATA(os, tmp[len]); + } while(len--); +} + +template +I ReadVarInt(Stream& is) +{ + I n = 0; + while(true) { + unsigned char chData; + READDATA(is, chData); + n = (n << 7) | (chData & 0x7F); + if (chData & 0x80) + n++; + else + return n; + } +} + +#define FLATDATA(obj) REF(CFlatData((char*)&(obj), (char*)&(obj) + sizeof(obj))) +#define VARINT(obj) REF(WrapVarInt(REF(obj))) +#define LIMITED_STRING(obj,n) REF(LimitedString< n >(REF(obj))) + +/** + * Wrapper for serializing arrays and POD. + */ +class CFlatData +{ +protected: + char* pbegin; + char* pend; +public: + CFlatData(void* pbeginIn, void* pendIn) : pbegin((char*)pbeginIn), pend((char*)pendIn) { } + template + explicit CFlatData(std::vector &v) + { + pbegin = (char*)begin_ptr(v); + pend = (char*)end_ptr(v); + } + char* begin() { return pbegin; } + const char* begin() const { return pbegin; } + char* end() { return pend; } + const char* end() const { return pend; } + + unsigned int GetSerializeSize(int, int=0) const + { + return pend - pbegin; + } + + template + void Serialize(Stream& s, int, int=0) const + { + s.write(pbegin, pend - pbegin); + } + + template + void Unserialize(Stream& s, int, int=0) + { + s.read(pbegin, pend - pbegin); + } +}; + +template +class CVarInt +{ +protected: + I &n; +public: + CVarInt(I& nIn) : n(nIn) { } + + unsigned int GetSerializeSize(int, int) const { + return GetSizeOfVarInt(n); + } + + template + void Serialize(Stream &s, int, int) const { + WriteVarInt(s, n); + } + + template + void Unserialize(Stream& s, int, int) { + n = ReadVarInt(s); + } +}; + +template +class LimitedString +{ +protected: + std::string& string; +public: + LimitedString(std::string& string) : string(string) {} + + template + void Unserialize(Stream& s, int, int=0) + { + size_t size = ReadCompactSize(s); + if (size > Limit) { + throw std::ios_base::failure("String length limit exceeded"); + } + string.resize(size); + if (size != 0) + s.read((char*)&string[0], size); + } + + template + void Serialize(Stream& s, int, int=0) const + { + WriteCompactSize(s, string.size()); + if (!string.empty()) + s.write((char*)&string[0], string.size()); + } + + unsigned int GetSerializeSize(int, int=0) const + { + return GetSizeOfCompactSize(string.size()) + string.size(); + } +}; + +template +CVarInt WrapVarInt(I& n) { return CVarInt(n); } + +/** + * Forward declarations + */ + +/** + * string + */ +template unsigned int GetSerializeSize(const std::basic_string& str, int, int=0); +template void Serialize(Stream& os, const std::basic_string& str, int, int=0); +template void Unserialize(Stream& is, std::basic_string& str, int, int=0); + +/** + * vector + * vectors of unsigned char are a special case and are intended to be serialized as a single opaque blob. + */ +template unsigned int GetSerializeSize_impl(const std::vector& v, int nType, int nVersion, const unsigned char&); +template unsigned int GetSerializeSize_impl(const std::vector& v, int nType, int nVersion, const V&); +template inline unsigned int GetSerializeSize(const std::vector& v, int nType, int nVersion); +template void Serialize_impl(Stream& os, const std::vector& v, int nType, int nVersion, const unsigned char&); +template void Serialize_impl(Stream& os, const std::vector& v, int nType, int nVersion, const V&); +template inline void Serialize(Stream& os, const std::vector& v, int nType, int nVersion); +template void Unserialize_impl(Stream& is, std::vector& v, int nType, int nVersion, const unsigned char&); +template void Unserialize_impl(Stream& is, std::vector& v, int nType, int nVersion, const V&); +template inline void Unserialize(Stream& is, std::vector& v, int nType, int nVersion); + +/** + * others derived from vector + */ +extern inline unsigned int GetSerializeSize(const CScript& v, int nType, int nVersion); +template void Serialize(Stream& os, const CScript& v, int nType, int nVersion); +template void Unserialize(Stream& is, CScript& v, int nType, int nVersion); + +/** + * pair + */ +template unsigned int GetSerializeSize(const std::pair& item, int nType, int nVersion); +template void Serialize(Stream& os, const std::pair& item, int nType, int nVersion); +template void Unserialize(Stream& is, std::pair& item, int nType, int nVersion); + +/** + * map + */ +template unsigned int GetSerializeSize(const std::map& m, int nType, int nVersion); +template void Serialize(Stream& os, const std::map& m, int nType, int nVersion); +template void Unserialize(Stream& is, std::map& m, int nType, int nVersion); + +/** + * set + */ +template unsigned int GetSerializeSize(const std::set& m, int nType, int nVersion); +template void Serialize(Stream& os, const std::set& m, int nType, int nVersion); +template void Unserialize(Stream& is, std::set& m, int nType, int nVersion); + + + + + +/** + * If none of the specialized versions above matched, default to calling member function. + * "int nType" is changed to "long nType" to keep from getting an ambiguous overload error. + * The compiler will only cast int to long if none of the other templates matched. + * Thanks to Boost serialization for this idea. + */ +template +inline unsigned int GetSerializeSize(const T& a, long nType, int nVersion) +{ + return a.GetSerializeSize((int)nType, nVersion); +} + +template +inline void Serialize(Stream& os, const T& a, long nType, int nVersion) +{ + a.Serialize(os, (int)nType, nVersion); +} + +template +inline void Unserialize(Stream& is, T& a, long nType, int nVersion) +{ + a.Unserialize(is, (int)nType, nVersion); +} + + + + + +/** + * string + */ +template +unsigned int GetSerializeSize(const std::basic_string& str, int, int) +{ + return GetSizeOfCompactSize(str.size()) + str.size() * sizeof(str[0]); +} + +template +void Serialize(Stream& os, const std::basic_string& str, int, int) +{ + WriteCompactSize(os, str.size()); + if (!str.empty()) + os.write((char*)&str[0], str.size() * sizeof(str[0])); +} + +template +void Unserialize(Stream& is, std::basic_string& str, int, int) +{ + unsigned int nSize = ReadCompactSize(is); + str.resize(nSize); + if (nSize != 0) + is.read((char*)&str[0], nSize * sizeof(str[0])); +} + + + +/** + * vector + */ +template +unsigned int GetSerializeSize_impl(const std::vector& v, int nType, int nVersion, const unsigned char&) +{ + return (GetSizeOfCompactSize(v.size()) + v.size() * sizeof(T)); +} + +template +unsigned int GetSerializeSize_impl(const std::vector& v, int nType, int nVersion, const V&) +{ + unsigned int nSize = GetSizeOfCompactSize(v.size()); + for (typename std::vector::const_iterator vi = v.begin(); vi != v.end(); ++vi) + nSize += GetSerializeSize((*vi), nType, nVersion); + return nSize; +} + +template +inline unsigned int GetSerializeSize(const std::vector& v, int nType, int nVersion) +{ + return GetSerializeSize_impl(v, nType, nVersion, T()); +} + + +template +void Serialize_impl(Stream& os, const std::vector& v, int nType, int nVersion, const unsigned char&) +{ + WriteCompactSize(os, v.size()); + if (!v.empty()) + os.write((char*)&v[0], v.size() * sizeof(T)); +} + +template +void Serialize_impl(Stream& os, const std::vector& v, int nType, int nVersion, const V&) +{ + WriteCompactSize(os, v.size()); + for (typename std::vector::const_iterator vi = v.begin(); vi != v.end(); ++vi) + ::Serialize(os, (*vi), nType, nVersion); +} + +template +inline void Serialize(Stream& os, const std::vector& v, int nType, int nVersion) +{ + Serialize_impl(os, v, nType, nVersion, T()); +} + + +template +void Unserialize_impl(Stream& is, std::vector& v, int nType, int nVersion, const unsigned char&) +{ + // Limit size per read so bogus size value won't cause out of memory + v.clear(); + unsigned int nSize = ReadCompactSize(is); + unsigned int i = 0; + while (i < nSize) + { + unsigned int blk = std::min(nSize - i, (unsigned int)(1 + 4999999 / sizeof(T))); + v.resize(i + blk); + is.read((char*)&v[i], blk * sizeof(T)); + i += blk; + } +} + +template +void Unserialize_impl(Stream& is, std::vector& v, int nType, int nVersion, const V&) +{ + v.clear(); + unsigned int nSize = ReadCompactSize(is); + unsigned int i = 0; + unsigned int nMid = 0; + while (nMid < nSize) + { + nMid += 5000000 / sizeof(T); + if (nMid > nSize) + nMid = nSize; + v.resize(nMid); + for (; i < nMid; i++) + Unserialize(is, v[i], nType, nVersion); + } +} + +template +inline void Unserialize(Stream& is, std::vector& v, int nType, int nVersion) +{ + Unserialize_impl(is, v, nType, nVersion, T()); +} + + + +/** + * others derived from vector + */ +inline unsigned int GetSerializeSize(const CScript& v, int nType, int nVersion) +{ + return GetSerializeSize((const std::vector&)v, nType, nVersion); +} + +template +void Serialize(Stream& os, const CScript& v, int nType, int nVersion) +{ + Serialize(os, (const std::vector&)v, nType, nVersion); +} + +template +void Unserialize(Stream& is, CScript& v, int nType, int nVersion) +{ + Unserialize(is, (std::vector&)v, nType, nVersion); +} + + + +/** + * pair + */ +template +unsigned int GetSerializeSize(const std::pair& item, int nType, int nVersion) +{ + return GetSerializeSize(item.first, nType, nVersion) + GetSerializeSize(item.second, nType, nVersion); +} + +template +void Serialize(Stream& os, const std::pair& item, int nType, int nVersion) +{ + Serialize(os, item.first, nType, nVersion); + Serialize(os, item.second, nType, nVersion); +} + +template +void Unserialize(Stream& is, std::pair& item, int nType, int nVersion) +{ + Unserialize(is, item.first, nType, nVersion); + Unserialize(is, item.second, nType, nVersion); +} + + + +/** + * map + */ +template +unsigned int GetSerializeSize(const std::map& m, int nType, int nVersion) +{ + unsigned int nSize = GetSizeOfCompactSize(m.size()); + for (typename std::map::const_iterator mi = m.begin(); mi != m.end(); ++mi) + nSize += GetSerializeSize((*mi), nType, nVersion); + return nSize; +} + +template +void Serialize(Stream& os, const std::map& m, int nType, int nVersion) +{ + WriteCompactSize(os, m.size()); + for (typename std::map::const_iterator mi = m.begin(); mi != m.end(); ++mi) + Serialize(os, (*mi), nType, nVersion); +} + +template +void Unserialize(Stream& is, std::map& m, int nType, int nVersion) +{ + m.clear(); + unsigned int nSize = ReadCompactSize(is); + typename std::map::iterator mi = m.begin(); + for (unsigned int i = 0; i < nSize; i++) + { + std::pair item; + Unserialize(is, item, nType, nVersion); + mi = m.insert(mi, item); + } +} + + + +/** + * set + */ +template +unsigned int GetSerializeSize(const std::set& m, int nType, int nVersion) +{ + unsigned int nSize = GetSizeOfCompactSize(m.size()); + for (typename std::set::const_iterator it = m.begin(); it != m.end(); ++it) + nSize += GetSerializeSize((*it), nType, nVersion); + return nSize; +} + +template +void Serialize(Stream& os, const std::set& m, int nType, int nVersion) +{ + WriteCompactSize(os, m.size()); + for (typename std::set::const_iterator it = m.begin(); it != m.end(); ++it) + Serialize(os, (*it), nType, nVersion); +} + +template +void Unserialize(Stream& is, std::set& m, int nType, int nVersion) +{ + m.clear(); + unsigned int nSize = ReadCompactSize(is); + typename std::set::iterator it = m.begin(); + for (unsigned int i = 0; i < nSize; i++) + { + K key; + Unserialize(is, key, nType, nVersion); + it = m.insert(it, key); + } +} + + + +/** + * Support for ADD_SERIALIZE_METHODS and READWRITE macro + */ +struct CSerActionSerialize +{ + bool ForRead() const { return false; } +}; +struct CSerActionUnserialize +{ + bool ForRead() const { return true; } +}; + +template +inline void SerReadWrite(Stream& s, const T& obj, int nType, int nVersion, CSerActionSerialize ser_action) +{ + ::Serialize(s, obj, nType, nVersion); +} + +template +inline void SerReadWrite(Stream& s, T& obj, int nType, int nVersion, CSerActionUnserialize ser_action) +{ + ::Unserialize(s, obj, nType, nVersion); +} + + + + + + + + + +class CSizeComputer +{ +protected: + size_t nSize; + +public: + int nType; + int nVersion; + + CSizeComputer(int nTypeIn, int nVersionIn) : nSize(0), nType(nTypeIn), nVersion(nVersionIn) {} + + CSizeComputer& write(const char *psz, size_t nSize) + { + this->nSize += nSize; + return *this; + } + + template + CSizeComputer& operator<<(const T& obj) + { + ::Serialize(*this, obj, nType, nVersion); + return (*this); + } + + size_t size() const { + return nSize; + } +}; + +#endif // BITCOIN_SERIALIZE_H diff --git a/src/utils/streams.h b/src/utils/streams.h new file mode 100644 index 00000000..246a2026 --- /dev/null +++ b/src/utils/streams.h @@ -0,0 +1,572 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2013 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_STREAMS_H +#define BITCOIN_STREAMS_H + +#include "utils/allocators.h" +#include "utils/serialize.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/** Double ended buffer combining vector and stream-like interfaces. + * + * >> and << read and write unformatted data using the above serialization templates. + * Fills with data in linear time; some stringstream implementations take N^2 time. + */ +class CDataStream +{ +protected: + typedef CSerializeData vector_type; + vector_type vch; + unsigned int nReadPos; +public: + int nType; + int nVersion; + + typedef vector_type::allocator_type allocator_type; + typedef vector_type::size_type size_type; + typedef vector_type::difference_type difference_type; + typedef vector_type::reference reference; + typedef vector_type::const_reference const_reference; + typedef vector_type::value_type value_type; + typedef vector_type::iterator iterator; + typedef vector_type::const_iterator const_iterator; + typedef vector_type::reverse_iterator reverse_iterator; + + explicit CDataStream(int nTypeIn, int nVersionIn) + { + Init(nTypeIn, nVersionIn); + } + + CDataStream(const_iterator pbegin, const_iterator pend, int nTypeIn, int nVersionIn) : vch(pbegin, pend) + { + Init(nTypeIn, nVersionIn); + } + +#if !defined(_MSC_VER) || _MSC_VER >= 1300 + CDataStream(const char* pbegin, const char* pend, int nTypeIn, int nVersionIn) : vch(pbegin, pend) + { + Init(nTypeIn, nVersionIn); + } +#endif + + CDataStream(const vector_type& vchIn, int nTypeIn, int nVersionIn) : vch(vchIn.begin(), vchIn.end()) + { + Init(nTypeIn, nVersionIn); + } + + CDataStream(const std::vector& vchIn, int nTypeIn, int nVersionIn) : vch(vchIn.begin(), vchIn.end()) + { + Init(nTypeIn, nVersionIn); + } + + CDataStream(const std::vector& vchIn, int nTypeIn, int nVersionIn) : vch(vchIn.begin(), vchIn.end()) + { + Init(nTypeIn, nVersionIn); + } + + void Init(int nTypeIn, int nVersionIn) + { + nReadPos = 0; + nType = nTypeIn; + nVersion = nVersionIn; + } + + CDataStream& operator+=(const CDataStream& b) + { + vch.insert(vch.end(), b.begin(), b.end()); + return *this; + } + + friend CDataStream operator+(const CDataStream& a, const CDataStream& b) + { + CDataStream ret = a; + ret += b; + return (ret); + } + + std::string str() const + { + return (std::string(begin(), end())); + } + + + // + // Vector subset + // + const_iterator begin() const { return vch.begin() + nReadPos; } + iterator begin() { return vch.begin() + nReadPos; } + const_iterator end() const { return vch.end(); } + iterator end() { return vch.end(); } + size_type size() const { return vch.size() - nReadPos; } + bool empty() const { return vch.size() == nReadPos; } + void resize(size_type n, value_type c=0) { vch.resize(n + nReadPos, c); } + void reserve(size_type n) { vch.reserve(n + nReadPos); } + const_reference operator[](size_type pos) const { return vch[pos + nReadPos]; } + reference operator[](size_type pos) { return vch[pos + nReadPos]; } + void clear() { vch.clear(); nReadPos = 0; } + iterator insert(iterator it, const char& x=char()) { return vch.insert(it, x); } + void insert(iterator it, size_type n, const char& x) { vch.insert(it, n, x); } + + void insert(iterator it, std::vector::const_iterator first, std::vector::const_iterator last) + { + assert(last - first >= 0); + if (it == vch.begin() + nReadPos && (unsigned int)(last - first) <= nReadPos) + { + // special case for inserting at the front when there's room + nReadPos -= (last - first); + memcpy(&vch[nReadPos], &first[0], last - first); + } + else + vch.insert(it, first, last); + } + +#if !defined(_MSC_VER) || _MSC_VER >= 1300 + void insert(iterator it, const char* first, const char* last) + { + assert(last - first >= 0); + if (it == vch.begin() + nReadPos && (unsigned int)(last - first) <= nReadPos) + { + // special case for inserting at the front when there's room + nReadPos -= (last - first); + memcpy(&vch[nReadPos], &first[0], last - first); + } + else + vch.insert(it, first, last); + } +#endif + + iterator erase(iterator it) + { + if (it == vch.begin() + nReadPos) + { + // special case for erasing from the front + if (++nReadPos >= vch.size()) + { + // whenever we reach the end, we take the opportunity to clear the buffer + nReadPos = 0; + return vch.erase(vch.begin(), vch.end()); + } + return vch.begin() + nReadPos; + } + else + return vch.erase(it); + } + + iterator erase(iterator first, iterator last) + { + if (first == vch.begin() + nReadPos) + { + // special case for erasing from the front + if (last == vch.end()) + { + nReadPos = 0; + return vch.erase(vch.begin(), vch.end()); + } + else + { + nReadPos = (last - vch.begin()); + return last; + } + } + else + return vch.erase(first, last); + } + + inline void Compact() + { + vch.erase(vch.begin(), vch.begin() + nReadPos); + nReadPos = 0; + } + + bool Rewind(size_type n) + { + // Rewind by n characters if the buffer hasn't been compacted yet + if (n > nReadPos) + return false; + nReadPos -= n; + return true; + } + + + // + // Stream subset + // + bool eof() const { return size() == 0; } + CDataStream* rdbuf() { return this; } + int in_avail() { return size(); } + + void SetType(int n) { nType = n; } + int GetType() { return nType; } + void SetVersion(int n) { nVersion = n; } + int GetVersion() { return nVersion; } + void ReadVersion() { *this >> nVersion; } + void WriteVersion() { *this << nVersion; } + + CDataStream& read(char* pch, size_t nSize) + { + // Read from the beginning of the buffer + unsigned int nReadPosNext = nReadPos + nSize; + if (nReadPosNext >= vch.size()) + { + if (nReadPosNext > vch.size()) + { + throw std::ios_base::failure("CDataStream::read() : end of data"); + } + memcpy(pch, &vch[nReadPos], nSize); + nReadPos = 0; + vch.clear(); + return (*this); + } + memcpy(pch, &vch[nReadPos], nSize); + nReadPos = nReadPosNext; + return (*this); + } + + CDataStream& ignore(int nSize) + { + // Ignore from the beginning of the buffer + assert(nSize >= 0); + unsigned int nReadPosNext = nReadPos + nSize; + if (nReadPosNext >= vch.size()) + { + if (nReadPosNext > vch.size()) + throw std::ios_base::failure("CDataStream::ignore() : end of data"); + nReadPos = 0; + vch.clear(); + return (*this); + } + nReadPos = nReadPosNext; + return (*this); + } + + CDataStream& write(const char* pch, size_t nSize) + { + // Write to the end of the buffer + vch.insert(vch.end(), pch, pch + nSize); + return (*this); + } + + template + void Serialize(Stream& s, int nType, int nVersion) const + { + // Special case: stream << stream concatenates like stream += stream + if (!vch.empty()) + s.write((char*)&vch[0], vch.size() * sizeof(vch[0])); + } + + template + unsigned int GetSerializeSize(const T& obj) + { + // Tells the size of the object if serialized to this stream + return ::GetSerializeSize(obj, nType, nVersion); + } + + template + CDataStream& operator<<(const T& obj) + { + // Serialize to this stream + ::Serialize(*this, obj, nType, nVersion); + return (*this); + } + + template + CDataStream& operator>>(T& obj) + { + // Unserialize from this stream + ::Unserialize(*this, obj, nType, nVersion); + return (*this); + } + + void GetAndClear(CSerializeData &data) { + data.insert(data.end(), begin(), end()); + clear(); + } +}; + + + + + + + + + + +/** Non-refcounted RAII wrapper for FILE* + * + * Will automatically close the file when it goes out of scope if not null. + * If you're returning the file pointer, return file.release(). + * If you need to close the file early, use file.fclose() instead of fclose(file). + */ +class CAutoFile +{ +private: + // Disallow copies + CAutoFile(const CAutoFile&); + CAutoFile& operator=(const CAutoFile&); + + int nType; + int nVersion; + + FILE* file; + +public: + CAutoFile(FILE* filenew, int nTypeIn, int nVersionIn) + { + file = filenew; + nType = nTypeIn; + nVersion = nVersionIn; + } + + ~CAutoFile() + { + fclose(); + } + + void fclose() + { + if (file) { + ::fclose(file); + file = NULL; + } + } + + /** Get wrapped FILE* with transfer of ownership. + * @note This will invalidate the CAutoFile object, and makes it the responsibility of the caller + * of this function to clean up the returned FILE*. + */ + FILE* release() { FILE* ret = file; file = NULL; return ret; } + + /** Get wrapped FILE* without transfer of ownership. + * @note Ownership of the FILE* will remain with this class. Use this only if the scope of the + * CAutoFile outlives use of the passed pointer. + */ + FILE* Get() const { return file; } + + /** Return true if the wrapped FILE* is NULL, false otherwise. + */ + bool IsNull() const { return (file == NULL); } + + // + // Stream subset + // + void SetType(int n) { nType = n; } + int GetType() { return nType; } + void SetVersion(int n) { nVersion = n; } + int GetVersion() { return nVersion; } + void ReadVersion() { *this >> nVersion; } + void WriteVersion() { *this << nVersion; } + + CAutoFile& read(char* pch, size_t nSize) + { + if (!file) + throw std::ios_base::failure("CAutoFile::read : file handle is NULL"); + if (fread(pch, 1, nSize, file) != nSize) + throw std::ios_base::failure(feof(file) ? "CAutoFile::read : end of file" : "CAutoFile::read : fread failed"); + return (*this); + } + + CAutoFile& write(const char* pch, size_t nSize) + { + if (!file) + throw std::ios_base::failure("CAutoFile::write : file handle is NULL"); + if (fwrite(pch, 1, nSize, file) != nSize) + throw std::ios_base::failure("CAutoFile::write : write failed"); + return (*this); + } + + template + unsigned int GetSerializeSize(const T& obj) + { + // Tells the size of the object if serialized to this stream + return ::GetSerializeSize(obj, nType, nVersion); + } + + template + CAutoFile& operator<<(const T& obj) + { + // Serialize to this stream + if (!file) + throw std::ios_base::failure("CAutoFile::operator<< : file handle is NULL"); + ::Serialize(*this, obj, nType, nVersion); + return (*this); + } + + template + CAutoFile& operator>>(T& obj) + { + // Unserialize from this stream + if (!file) + throw std::ios_base::failure("CAutoFile::operator>> : file handle is NULL"); + ::Unserialize(*this, obj, nType, nVersion); + return (*this); + } +}; + +/** Non-refcounted RAII wrapper around a FILE* that implements a ring buffer to + * deserialize from. It guarantees the ability to rewind a given number of bytes. + * + * Will automatically close the file when it goes out of scope if not null. + * If you need to close the file early, use file.fclose() instead of fclose(file). + */ +class CBufferedFile +{ +private: + // Disallow copies + CBufferedFile(const CBufferedFile&); + CBufferedFile& operator=(const CBufferedFile&); + + int nType; + int nVersion; + + FILE *src; // source file + uint64_t nSrcPos; // how many bytes have been read from source + uint64_t nReadPos; // how many bytes have been read from this + uint64_t nReadLimit; // up to which position we're allowed to read + uint64_t nRewind; // how many bytes we guarantee to rewind + std::vector vchBuf; // the buffer + +protected: + // read data from the source to fill the buffer + bool Fill() { + unsigned int pos = nSrcPos % vchBuf.size(); + unsigned int readNow = vchBuf.size() - pos; + unsigned int nAvail = vchBuf.size() - (nSrcPos - nReadPos) - nRewind; + if (nAvail < readNow) + readNow = nAvail; + if (readNow == 0) + return false; + size_t read = fread((void*)&vchBuf[pos], 1, readNow, src); + if (read == 0) { + throw std::ios_base::failure(feof(src) ? "CBufferedFile::Fill : end of file" : "CBufferedFile::Fill : fread failed"); + } else { + nSrcPos += read; + return true; + } + } + +public: + CBufferedFile(FILE *fileIn, uint64_t nBufSize, uint64_t nRewindIn, int nTypeIn, int nVersionIn) : + nSrcPos(0), nReadPos(0), nReadLimit((uint64_t)(-1)), nRewind(nRewindIn), vchBuf(nBufSize, 0) + { + src = fileIn; + nType = nTypeIn; + nVersion = nVersionIn; + } + + ~CBufferedFile() + { + fclose(); + } + + void fclose() + { + if (src) { + ::fclose(src); + src = NULL; + } + } + + // check whether we're at the end of the source file + bool eof() const { + return nReadPos == nSrcPos && feof(src); + } + + // read a number of bytes + CBufferedFile& read(char *pch, size_t nSize) { + if (nSize + nReadPos > nReadLimit) + throw std::ios_base::failure("Read attempted past buffer limit"); + if (nSize + nRewind > vchBuf.size()) + throw std::ios_base::failure("Read larger than buffer size"); + while (nSize > 0) { + if (nReadPos == nSrcPos) + Fill(); + unsigned int pos = nReadPos % vchBuf.size(); + size_t nNow = nSize; + if (nNow + pos > vchBuf.size()) + nNow = vchBuf.size() - pos; + if (nNow + nReadPos > nSrcPos) + nNow = nSrcPos - nReadPos; + memcpy(pch, &vchBuf[pos], nNow); + nReadPos += nNow; + pch += nNow; + nSize -= nNow; + } + return (*this); + } + + // return the current reading position + uint64_t GetPos() { + return nReadPos; + } + + // rewind to a given reading position + bool SetPos(uint64_t nPos) { + nReadPos = nPos; + if (nReadPos + nRewind < nSrcPos) { + nReadPos = nSrcPos - nRewind; + return false; + } else if (nReadPos > nSrcPos) { + nReadPos = nSrcPos; + return false; + } else { + return true; + } + } + + bool Seek(uint64_t nPos) { + long nLongPos = nPos; + if (nPos != (uint64_t)nLongPos) + return false; + if (fseek(src, nLongPos, SEEK_SET)) + return false; + nLongPos = ftell(src); + nSrcPos = nLongPos; + nReadPos = nLongPos; + return true; + } + + // prevent reading beyond a certain position + // no argument removes the limit + bool SetLimit(uint64_t nPos = (uint64_t)(-1)) { + if (nPos < nReadPos) + return false; + nReadLimit = nPos; + return true; + } + + template + CBufferedFile& operator>>(T& obj) { + // Unserialize from this stream + ::Unserialize(*this, obj, nType, nVersion); + return (*this); + } + + // search for a given byte in the stream, and remain positioned on it + void FindByte(char ch) { + while (true) { + if (nReadPos == nSrcPos) + Fill(); + if (vchBuf[nReadPos % vchBuf.size()] == ch) + break; + nReadPos++; + } + } +}; + +#endif // BITCOIN_STREAMS_H diff --git a/src/utils/sync.cpp b/src/utils/sync.cpp new file mode 100644 index 00000000..1e29a673 --- /dev/null +++ b/src/utils/sync.cpp @@ -0,0 +1,153 @@ +// Copyright (c) 2011-2012 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "utils/sync.h" + +#include "utils/util.h" +#include "utils/utilstrencodings.h" + +#include + +#include +#include + +#ifdef DEBUG_LOCKCONTENTION +void PrintLockContention(const char* pszName, const char* pszFile, int nLine) +{ + LogPrintf("LOCKCONTENTION: %s\n", pszName); + LogPrintf("Locker: %s:%d\n", pszFile, nLine); +} +#endif /* DEBUG_LOCKCONTENTION */ + +#ifdef DEBUG_LOCKORDER +// +// Early deadlock detection. +// Problem being solved: +// Thread 1 locks A, then B, then C +// Thread 2 locks D, then C, then A +// --> may result in deadlock between the two threads, depending on when they run. +// Solution implemented here: +// Keep track of pairs of locks: (A before B), (A before C), etc. +// Complain if any thread tries to lock in a different order. +// + +struct CLockLocation { + CLockLocation(const char* pszName, const char* pszFile, int nLine) + { + mutexName = pszName; + sourceFile = pszFile; + sourceLine = nLine; + } + + std::string ToString() const + { + return mutexName + " " + sourceFile + ":" + itostr(sourceLine); + } + + std::string MutexName() const { return mutexName; } + +private: + std::string mutexName; + std::string sourceFile; + int sourceLine; +}; + +typedef std::vector > LockStack; + +static boost::mutex dd_mutex; +static std::map, LockStack> lockorders; +static boost::thread_specific_ptr lockstack; + + +static void potential_deadlock_detected(const std::pair& mismatch, const LockStack& s1, const LockStack& s2) +{ + LogPrintf("POTENTIAL DEADLOCK DETECTED\n"); + LogPrintf("Previous lock order was:\n"); + BOOST_FOREACH (const PAIRTYPE(void*, CLockLocation) & i, s2) { + if (i.first == mismatch.first) + LogPrintf(" (1)"); + if (i.first == mismatch.second) + LogPrintf(" (2)"); + LogPrintf(" %s\n", i.second.ToString()); + } + LogPrintf("Current lock order is:\n"); + BOOST_FOREACH (const PAIRTYPE(void*, CLockLocation) & i, s1) { + if (i.first == mismatch.first) + LogPrintf(" (1)"); + if (i.first == mismatch.second) + LogPrintf(" (2)"); + LogPrintf(" %s\n", i.second.ToString()); + } +} + +static void push_lock(void* c, const CLockLocation& locklocation, bool fTry) +{ + if (lockstack.get() == NULL) + lockstack.reset(new LockStack); + + LogPrint("lock", "Locking: %s\n", locklocation.ToString()); + dd_mutex.lock(); + + (*lockstack).push_back(std::make_pair(c, locklocation)); + + if (!fTry) { + BOOST_FOREACH (const PAIRTYPE(void*, CLockLocation) & i, (*lockstack)) { + if (i.first == c) + break; + + std::pair p1 = std::make_pair(i.first, c); + if (lockorders.count(p1)) + continue; + lockorders[p1] = (*lockstack); + + std::pair p2 = std::make_pair(c, i.first); + if (lockorders.count(p2)) { + potential_deadlock_detected(p1, lockorders[p2], lockorders[p1]); + break; + } + } + } + dd_mutex.unlock(); +} + +static void pop_lock() +{ + if (fDebug) { + const CLockLocation& locklocation = (*lockstack).rbegin()->second; + LogPrint("lock", "Unlocked: %s\n", locklocation.ToString()); + } + dd_mutex.lock(); + (*lockstack).pop_back(); + dd_mutex.unlock(); +} + +void EnterCritical(const char* pszName, const char* pszFile, int nLine, void* cs, bool fTry) +{ + push_lock(cs, CLockLocation(pszName, pszFile, nLine), fTry); +} + +void LeaveCritical() +{ + pop_lock(); +} + +std::string LocksHeld() +{ + std::string result; + BOOST_FOREACH (const PAIRTYPE(void*, CLockLocation) & i, *lockstack) + result += i.second.ToString() + std::string("\n"); + return result; +} + +void AssertLockHeldInternal(const char* pszName, const char* pszFile, int nLine, void* cs) +{ + BOOST_FOREACH (const PAIRTYPE(void*, CLockLocation) & i, *lockstack) + if (i.first == cs) + return; + fprintf(stderr, "Assertion failed: lock %s not held in %s:%i; locks held:\n%s", pszName, pszFile, nLine, LocksHeld().c_str()); + abort(); +} + +#endif /* DEBUG_LOCKORDER */ diff --git a/src/utils/sync.h b/src/utils/sync.h new file mode 100644 index 00000000..1edd9f47 --- /dev/null +++ b/src/utils/sync.h @@ -0,0 +1,275 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2013 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_SYNC_H +#define BITCOIN_SYNC_H + +#include "utils/threadsafety.h" + +#include +#include +#include +#include + + +//////////////////////////////////////////////// +// // +// THE SIMPLE DEFINITON, EXCLUDING DEBUG CODE // +// // +//////////////////////////////////////////////// + +/* + + + +CCriticalSection mutex; + boost::recursive_mutex mutex; + +LOCK(mutex); + boost::unique_lock criticalblock(mutex); + +LOCK2(mutex1, mutex2); + boost::unique_lock criticalblock1(mutex1); + boost::unique_lock criticalblock2(mutex2); + +TRY_LOCK(mutex, name); + boost::unique_lock name(mutex, boost::try_to_lock_t); + +ENTER_CRITICAL_SECTION(mutex); // no RAII + mutex.lock(); + +LEAVE_CRITICAL_SECTION(mutex); // no RAII + mutex.unlock(); + + + + */ + + +/////////////////////////////// +// // +// THE ACTUAL IMPLEMENTATION // +// // +/////////////////////////////// + +// Template mixin that adds -Wthread-safety locking annotations to a +// subset of the mutex API. +template +class LOCKABLE AnnotatedMixin : public PARENT +{ +public: + void lock() EXCLUSIVE_LOCK_FUNCTION() + { + PARENT::lock(); + } + + void unlock() UNLOCK_FUNCTION() + { + PARENT::unlock(); + } + + bool try_lock() EXCLUSIVE_TRYLOCK_FUNCTION(true) + { + return PARENT::try_lock(); + } +}; + +/** Wrapped boost mutex: supports recursive locking, but no waiting */ +// TODO: We should move away from using the recursive lock by default. +typedef AnnotatedMixin CCriticalSection; + +/** Wrapped boost mutex: supports waiting but not recursive locking */ +typedef AnnotatedMixin CWaitableCriticalSection; + +/** Just a typedef for boost::condition_variable, can be wrapped later if desired */ +typedef boost::condition_variable CConditionVariable; + +#ifdef DEBUG_LOCKORDER +void EnterCritical(const char* pszName, const char* pszFile, int nLine, void* cs, bool fTry = false); +void LeaveCritical(); +std::string LocksHeld(); +void AssertLockHeldInternal(const char* pszName, const char* pszFile, int nLine, void* cs); +#else +void static inline EnterCritical(const char* pszName, const char* pszFile, int nLine, void* cs, bool fTry = false) +{ +} +void static inline LeaveCritical() {} +void static inline AssertLockHeldInternal(const char* pszName, const char* pszFile, int nLine, void* cs) {} +#endif +#define AssertLockHeld(cs) AssertLockHeldInternal(#cs, __FILE__, __LINE__, &cs) + +#ifdef DEBUG_LOCKCONTENTION +void PrintLockContention(const char* pszName, const char* pszFile, int nLine); +#endif + +/** Wrapper around boost::unique_lock */ +template +class CMutexLock +{ +private: + boost::unique_lock lock; + + void Enter(const char* pszName, const char* pszFile, int nLine) + { + EnterCritical(pszName, pszFile, nLine, (void*)(lock.mutex())); +#ifdef DEBUG_LOCKCONTENTION + if (!lock.try_lock()) { + PrintLockContention(pszName, pszFile, nLine); +#endif + lock.lock(); +#ifdef DEBUG_LOCKCONTENTION + } +#endif + } + + bool TryEnter(const char* pszName, const char* pszFile, int nLine) + { + EnterCritical(pszName, pszFile, nLine, (void*)(lock.mutex()), true); + lock.try_lock(); + if (!lock.owns_lock()) + LeaveCritical(); + return lock.owns_lock(); + } + +public: + CMutexLock(Mutex& mutexIn, const char* pszName, const char* pszFile, int nLine, bool fTry = false) : lock(mutexIn, boost::defer_lock) + { + if (fTry) + TryEnter(pszName, pszFile, nLine); + else + Enter(pszName, pszFile, nLine); + } + + ~CMutexLock() + { + if (lock.owns_lock()) + LeaveCritical(); + } + + operator bool() + { + return lock.owns_lock(); + } +}; + +typedef CMutexLock CCriticalBlock; + +#define LOCK(cs) CCriticalBlock criticalblock(cs, #cs, __FILE__, __LINE__) +#define LOCK2(cs1, cs2) CCriticalBlock criticalblock1(cs1, #cs1, __FILE__, __LINE__), criticalblock2(cs2, #cs2, __FILE__, __LINE__) +#define TRY_LOCK(cs, name) CCriticalBlock name(cs, #cs, __FILE__, __LINE__, true) + +#define ENTER_CRITICAL_SECTION(cs) \ + { \ + EnterCritical(#cs, __FILE__, __LINE__, (void*)(&cs)); \ + (cs).lock(); \ + } + +#define LEAVE_CRITICAL_SECTION(cs) \ + { \ + (cs).unlock(); \ + LeaveCritical(); \ + } + +class CSemaphore +{ +private: + boost::condition_variable condition; + boost::mutex mutex; + int value; + +public: + CSemaphore(int init) : value(init) {} + + void wait() + { + boost::unique_lock lock(mutex); + while (value < 1) { + condition.wait(lock); + } + value--; + } + + bool try_wait() + { + boost::unique_lock lock(mutex); + if (value < 1) + return false; + value--; + return true; + } + + void post() + { + { + boost::unique_lock lock(mutex); + value++; + } + condition.notify_one(); + } +}; + +/** RAII-style semaphore lock */ +class CSemaphoreGrant +{ +private: + CSemaphore* sem; + bool fHaveGrant; + +public: + void Acquire() + { + if (fHaveGrant) + return; + sem->wait(); + fHaveGrant = true; + } + + void Release() + { + if (!fHaveGrant) + return; + sem->post(); + fHaveGrant = false; + } + + bool TryAcquire() + { + if (!fHaveGrant && sem->try_wait()) + fHaveGrant = true; + return fHaveGrant; + } + + void MoveTo(CSemaphoreGrant& grant) + { + grant.Release(); + grant.sem = sem; + grant.fHaveGrant = fHaveGrant; + sem = NULL; + fHaveGrant = false; + } + + CSemaphoreGrant() : sem(NULL), fHaveGrant(false) {} + + CSemaphoreGrant(CSemaphore& sema, bool fTry = false) : sem(&sema), fHaveGrant(false) + { + if (fTry) + TryAcquire(); + else + Acquire(); + } + + ~CSemaphoreGrant() + { + Release(); + } + + operator bool() + { + return fHaveGrant; + } +}; + +#endif // BITCOIN_SYNC_H diff --git a/src/utils/systemdependent.cpp b/src/utils/systemdependent.cpp new file mode 100644 index 00000000..559b1b54 --- /dev/null +++ b/src/utils/systemdependent.cpp @@ -0,0 +1,225 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" + +#ifndef WIN32 + +#ifndef _AIX +#include +#endif +#include +#include +#include +#include +#include +#include + +#include + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +//#include "us_define.h" +#include +#include + + +#include // We need if of dlopen +#include // We need it for sleep + + +#include "utils/define.h" +#include "utils/declare.h" + +#define SHM_FAILED (void *)-1L +#define WAIT_FAILED ((cs_int32)0xFFFFFFFF) +#define WAIT_OBJECT_0 0 + + +void __US_Dummy() +{ + +} + +void __US_Sleep (int dwMilliseconds) +{ + timespec ts; + + ts.tv_sec = (time_t)(dwMilliseconds/1000); + ts.tv_nsec = (int) ((dwMilliseconds % 1000) * 1000000); + + nanosleep(&ts, NULL); +} + +int __US_Fork(char *lpCommandLine,int WinState,int *lpChildPID) +{ + int res; + int statloc; + + statloc=0; + res=fork(); + if(res==0) + { + res=fork(); + if(res) + { + *lpChildPID=res; + exit(0); //Child exits + } + } + else + { + if(res>0) + { + while(*lpChildPID==0) + { + __US_Sleep(1); + } + waitpid(res, &statloc, 0); + res=*lpChildPID; + } + } + + return res;//Parent exits with PID of grandchild , GrandChild exits with 0 +} + +int __US_BecomeDaemon() +{ + int ChildPID; + int res; + int i,fd; + + res=__US_Fork(NULL,0,&ChildPID); + + if(res) + exit(0); + + for (i = getdtablesize()-1; i > 2; --i) + close(i); + + fd = open("/dev/tty", O_RDWR); + ioctl(fd, TIOCNOTTY, 0); + close (fd); + + return res; +} + +void* __US_SemCreate() +{ + sem_t *lpsem; + + lpsem=NULL; + + lpsem=new sem_t; + if(sem_init(lpsem,0666,1)) + { + delete lpsem; + return NULL; + } + + return (void*)lpsem; +} + +void __US_SemWait(void* sem) +{ + if(sem) + { + sem_wait((sem_t *)sem); + } +} + +void __US_SemPost(void* sem) +{ + if(sem) + { + sem_post((sem_t*)sem); + } +} + +void __US_SemDestroy(void* sem) +{ + sem_t *lpsem; + if(sem) + { + sem_close((sem_t*)sem); + lpsem=(sem_t*)sem; + delete lpsem; + } +} + +uint64_t __US_ThreadID() +{ + return (uint64_t)pthread_self(); +} + +#else + +#include "windows.h" + +void __US_Dummy() +{ + +} + +int __US_Fork(char *lpCommandLine,int WinState,int *lpChildPID) +{ + return 0; +} + +int __US_BecomeDaemon() +{ + return 0; +} + +void __US_Sleep (int dwMilliseconds) +{ + Sleep(dwMilliseconds); +} + + +void* __US_SemCreate() +{ + return (void*)CreateSemaphore(NULL,1,1,NULL); +} + +void __US_SemWait(void* sem) +{ + if(sem) + { + WaitForSingleObject(sem, 0x7FFFFFFF); + } +} + +void __US_SemPost(void* sem) +{ + if(sem) + { + ReleaseSemaphore(sem,1,NULL); + } +} + +void __US_SemDestroy(void* sem) +{ + if(sem) + { + CloseHandle(sem); + } +} + +uint64_t __US_ThreadID() +{ + return (uint64_t)GetCurrentThreadId (); +} + +#endif diff --git a/src/utils/threadsafety.h b/src/utils/threadsafety.h new file mode 100644 index 00000000..e772e4c2 --- /dev/null +++ b/src/utils/threadsafety.h @@ -0,0 +1,56 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2012 The Bitcoin developers +// Original code was distributed under the MIT/X11 software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_THREADSAFETY_H +#define BITCOIN_THREADSAFETY_H + +#ifdef __clang__ +// TL;DR Add GUARDED_BY(mutex) to member variables. The others are +// rarely necessary. Ex: int nFoo GUARDED_BY(cs_foo); +// +// See http://clang.llvm.org/docs/LanguageExtensions.html#threadsafety +// for documentation. The clang compiler can do advanced static analysis +// of locking when given the -Wthread-safety option. +#define LOCKABLE __attribute__((lockable)) +#define SCOPED_LOCKABLE __attribute__((scoped_lockable)) +#define GUARDED_BY(x) __attribute__((guarded_by(x))) +#define GUARDED_VAR __attribute__((guarded_var)) +#define PT_GUARDED_BY(x) __attribute__((pt_guarded_by(x))) +#define PT_GUARDED_VAR __attribute__((pt_guarded_var)) +#define ACQUIRED_AFTER(...) __attribute__((acquired_after(__VA_ARGS__))) +#define ACQUIRED_BEFORE(...) __attribute__((acquired_before(__VA_ARGS__))) +#define EXCLUSIVE_LOCK_FUNCTION(...) __attribute__((exclusive_lock_function(__VA_ARGS__))) +#define SHARED_LOCK_FUNCTION(...) __attribute__((shared_lock_function(__VA_ARGS__))) +#define EXCLUSIVE_TRYLOCK_FUNCTION(...) __attribute__((exclusive_trylock_function(__VA_ARGS__))) +#define SHARED_TRYLOCK_FUNCTION(...) __attribute__((shared_trylock_function(__VA_ARGS__))) +#define UNLOCK_FUNCTION(...) __attribute__((unlock_function(__VA_ARGS__))) +#define LOCK_RETURNED(x) __attribute__((lock_returned(x))) +#define LOCKS_EXCLUDED(...) __attribute__((locks_excluded(__VA_ARGS__))) +#define EXCLUSIVE_LOCKS_REQUIRED(...) __attribute__((exclusive_locks_required(__VA_ARGS__))) +#define SHARED_LOCKS_REQUIRED(...) __attribute__((shared_locks_required(__VA_ARGS__))) +#define NO_THREAD_SAFETY_ANALYSIS __attribute__((no_thread_safety_analysis)) +#else +#define LOCKABLE +#define SCOPED_LOCKABLE +#define GUARDED_BY(x) +#define GUARDED_VAR +#define PT_GUARDED_BY(x) +#define PT_GUARDED_VAR +#define ACQUIRED_AFTER(...) +#define ACQUIRED_BEFORE(...) +#define EXCLUSIVE_LOCK_FUNCTION(...) +#define SHARED_LOCK_FUNCTION(...) +#define EXCLUSIVE_TRYLOCK_FUNCTION(...) +#define SHARED_TRYLOCK_FUNCTION(...) +#define UNLOCK_FUNCTION(...) +#define LOCK_RETURNED(x) +#define LOCKS_EXCLUDED(...) +#define EXCLUSIVE_LOCKS_REQUIRED(...) +#define SHARED_LOCKS_REQUIRED(...) +#define NO_THREAD_SAFETY_ANALYSIS +#endif // __GNUC__ + +#endif // BITCOIN_THREADSAFETY_H diff --git a/src/utils/timedata.cpp b/src/utils/timedata.cpp new file mode 100644 index 00000000..623bbfb8 --- /dev/null +++ b/src/utils/timedata.cpp @@ -0,0 +1,115 @@ +// Copyright (c) 2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "utils/timedata.h" + +#include "net/netbase.h" +#include "utils/sync.h" +#include "ui/ui_interface.h" +#include "utils/util.h" +#include "utils/utilstrencodings.h" + +#include + +using namespace std; + +static CCriticalSection cs_nTimeOffset; +static int64_t nTimeOffset = 0; + +/** + * "Never go to sea with two chronometers; take one or three." + * Our three time sources are: + * - System clock + * - Median of other nodes clocks + * - The user (asking the user to fix the system clock if the first two disagree) + */ +int64_t GetTimeOffset() +{ + LOCK(cs_nTimeOffset); + return nTimeOffset; +} + +int64_t GetAdjustedTime() +{ + return GetTime() + GetTimeOffset(); +} + +static int64_t abs64(int64_t n) +{ + return (n >= 0 ? n : -n); +} + +void AddTimeData(const CNetAddr& ip, int64_t nTime) +{ + int64_t nOffsetSample = nTime - GetTime(); + + LOCK(cs_nTimeOffset); + // Ignore duplicates + static set setKnown; + if (!setKnown.insert(ip).second) + return; + + // Add data + static CMedianFilter vTimeOffsets(200,0); + vTimeOffsets.input(nOffsetSample); + LogPrintf("Added time data, samples %d, offset %+d (%+d minutes)\n", vTimeOffsets.size(), nOffsetSample, nOffsetSample/60); + + // There is a known issue here (see issue #4521): + // + // - The structure vTimeOffsets contains up to 200 elements, after which + // any new element added to it will not increase its size, replacing the + // oldest element. + // + // - The condition to update nTimeOffset includes checking whether the + // number of elements in vTimeOffsets is odd, which will never happen after + // there are 200 elements. + // + // But in this case the 'bug' is protective against some attacks, and may + // actually explain why we've never seen attacks which manipulate the + // clock offset. + // + // So we should hold off on fixing this and clean it up as part of + // a timing cleanup that strengthens it in a number of other ways. + // + if (vTimeOffsets.size() >= 5 && vTimeOffsets.size() % 2 == 1) + { + int64_t nMedian = vTimeOffsets.median(); + std::vector vSorted = vTimeOffsets.sorted(); + // Only let other nodes change our time by so much + if (abs64(nMedian) < 70 * 60) + { + nTimeOffset = nMedian; + } + else + { + nTimeOffset = 0; + + static bool fDone; + if (!fDone) + { + // If nobody has a time different than ours but within 5 minutes of ours, give a warning + bool fMatch = false; + BOOST_FOREACH(int64_t nOffset, vSorted) + if (nOffset != 0 && abs64(nOffset) < 5 * 60) + fMatch = true; + + if (!fMatch) + { + fDone = true; + string strMessage = _("Warning: Please check that your computer's date and time are correct! If your clock is wrong MultiChain Core will not work properly."); + strMiscWarning = strMessage; + LogPrintf("*** %s\n", strMessage); + uiInterface.ThreadSafeMessageBox(strMessage, "", CClientUIInterface::MSG_WARNING); + } + } + } + if (fDebug) { + BOOST_FOREACH(int64_t n, vSorted) + LogPrintf("%+d ", n); + LogPrintf("| "); + } + LogPrintf("nTimeOffset = %+d (%+d minutes)\n", nTimeOffset, nTimeOffset/60); + } +} diff --git a/src/utils/timedata.h b/src/utils/timedata.h new file mode 100644 index 00000000..7c40b4a8 --- /dev/null +++ b/src/utils/timedata.h @@ -0,0 +1,77 @@ +// Copyright (c) 2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_TIMEDATA_H +#define BITCOIN_TIMEDATA_H + +#include +#include +#include +#include + +class CNetAddr; + +/** + * Median filter over a stream of values. + * Returns the median of the last N numbers + */ +template +class CMedianFilter +{ +private: + std::vector vValues; + std::vector vSorted; + unsigned int nSize; + +public: + CMedianFilter(unsigned int size, T initial_value) : nSize(size) + { + vValues.reserve(size); + vValues.push_back(initial_value); + vSorted = vValues; + } + + void input(T value) + { + if (vValues.size() == nSize) { + vValues.erase(vValues.begin()); + } + vValues.push_back(value); + + vSorted.resize(vValues.size()); + std::copy(vValues.begin(), vValues.end(), vSorted.begin()); + std::sort(vSorted.begin(), vSorted.end()); + } + + T median() const + { + int size = vSorted.size(); + assert(size > 0); + if (size & 1) // Odd number of elements + { + return vSorted[size / 2]; + } else // Even number of elements + { + return (vSorted[size / 2 - 1] + vSorted[size / 2]) / 2; + } + } + + int size() const + { + return vValues.size(); + } + + std::vector sorted() const + { + return vSorted; + } +}; + +/** Functions to keep track of adjusted P2P time */ +int64_t GetTimeOffset(); +int64_t GetAdjustedTime(); +void AddTimeData(const CNetAddr& ip, int64_t nTime); + +#endif // BITCOIN_TIMEDATA_H diff --git a/src/utils/tinyformat.h b/src/utils/tinyformat.h new file mode 100644 index 00000000..73d49a1f --- /dev/null +++ b/src/utils/tinyformat.h @@ -0,0 +1,1013 @@ +// tinyformat.h +// Copyright (C) 2011, Chris Foster [chris42f (at) gmail (d0t) com] +// +// Boost Software License - Version 1.0 +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +//------------------------------------------------------------------------------ +// Tinyformat: A minimal type safe printf replacement +// +// tinyformat.h is a type safe printf replacement library in a single C++ +// header file. Design goals include: +// +// * Type safety and extensibility for user defined types. +// * C99 printf() compatibility, to the extent possible using std::ostream +// * Simplicity and minimalism. A single header file to include and distribute +// with your projects. +// * Augment rather than replace the standard stream formatting mechanism +// * C++98 support, with optional C++11 niceties +// +// +// Main interface example usage +// ---------------------------- +// +// To print a date to std::cout: +// +// std::string weekday = "Wednesday"; +// const char* month = "July"; +// size_t day = 27; +// long hour = 14; +// int min = 44; +// +// tfm::printf("%s, %s %d, %.2d:%.2d\n", weekday, month, day, hour, min); +// +// The strange types here emphasize the type safety of the interface; it is +// possible to print a std::string using the "%s" conversion, and a +// size_t using the "%d" conversion. A similar result could be achieved +// using either of the tfm::format() functions. One prints on a user provided +// stream: +// +// tfm::format(std::cerr, "%s, %s %d, %.2d:%.2d\n", +// weekday, month, day, hour, min); +// +// The other returns a std::string: +// +// std::string date = tfm::format("%s, %s %d, %.2d:%.2d\n", +// weekday, month, day, hour, min); +// std::cout << date; +// +// These are the three primary interface functions. +// +// +// User defined format functions +// ----------------------------- +// +// Simulating variadic templates in C++98 is pretty painful since it requires +// writing out the same function for each desired number of arguments. To make +// this bearable tinyformat comes with a set of macros which are used +// internally to generate the API, but which may also be used in user code. +// +// The three macros TINYFORMAT_ARGTYPES(n), TINYFORMAT_VARARGS(n) and +// TINYFORMAT_PASSARGS(n) will generate a list of n argument types, +// type/name pairs and argument names respectively when called with an integer +// n between 1 and 16. We can use these to define a macro which generates the +// desired user defined function with n arguments. To generate all 16 user +// defined function bodies, use the macro TINYFORMAT_FOREACH_ARGNUM. For an +// example, see the implementation of printf() at the end of the source file. +// +// +// Additional API information +// -------------------------- +// +// Error handling: Define TINYFORMAT_ERROR to customize the error handling for +// format strings which are unsupported or have the wrong number of format +// specifiers (calls assert() by default). +// +// User defined types: Uses operator<< for user defined types by default. +// Overload formatValue() for more control. + + +#ifndef TINYFORMAT_H_INCLUDED +#define TINYFORMAT_H_INCLUDED + +namespace tinyformat {} +//------------------------------------------------------------------------------ +// Config section. Customize to your liking! + +// Namespace alias to encourage brevity +namespace tfm = tinyformat; + +// Error handling; calls assert() by default. +#define TINYFORMAT_ERROR(reasonString) throw std::runtime_error(reasonString) + +// Define for C++11 variadic templates which make the code shorter & more +// general. If you don't define this, C++11 support is autodetected below. +// #define TINYFORMAT_USE_VARIADIC_TEMPLATES + + +//------------------------------------------------------------------------------ +// Implementation details. +#include +#include +#include +#include + +#ifndef TINYFORMAT_ERROR +# define TINYFORMAT_ERROR(reason) assert(0 && reason) +#endif + +#if !defined(TINYFORMAT_USE_VARIADIC_TEMPLATES) && !defined(TINYFORMAT_NO_VARIADIC_TEMPLATES) +# ifdef __GXX_EXPERIMENTAL_CXX0X__ +# define TINYFORMAT_USE_VARIADIC_TEMPLATES +# endif +#endif + +#ifdef __GNUC__ +# define TINYFORMAT_NOINLINE __attribute__((noinline)) +#elif defined(_MSC_VER) +# define TINYFORMAT_NOINLINE __declspec(noinline) +#else +# define TINYFORMAT_NOINLINE +#endif + +#if defined(__GLIBCXX__) && __GLIBCXX__ < 20080201 +// std::showpos is broken on old libstdc++ as provided with OSX. See +// http://gcc.gnu.org/ml/libstdc++/2007-11/msg00075.html +# define TINYFORMAT_OLD_LIBSTDCPLUSPLUS_WORKAROUND +#endif + +namespace tinyformat { + +//------------------------------------------------------------------------------ +namespace detail { + +// Test whether type T1 is convertible to type T2 +template +struct is_convertible +{ + private: + // two types of different size + struct fail { char dummy[2]; }; + struct succeed { char dummy; }; + // Try to convert a T1 to a T2 by plugging into tryConvert + static fail tryConvert(...); + static succeed tryConvert(const T2&); + static const T1& makeT1(); + public: +# ifdef _MSC_VER + // Disable spurious loss of precision warnings in tryConvert(makeT1()) +# pragma warning(push) +# pragma warning(disable:4244) +# pragma warning(disable:4267) +# endif + // Standard trick: the (...) version of tryConvert will be chosen from + // the overload set only if the version taking a T2 doesn't match. + // Then we compare the sizes of the return types to check which + // function matched. Very neat, in a disgusting kind of way :) + static const bool value = + sizeof(tryConvert(makeT1())) == sizeof(succeed); +# ifdef _MSC_VER +# pragma warning(pop) +# endif +}; + + +// Detect when a type is not a wchar_t string +template struct is_wchar { typedef int tinyformat_wchar_is_not_supported; }; +template<> struct is_wchar {}; +template<> struct is_wchar {}; +template struct is_wchar {}; +template struct is_wchar {}; + + +// Format the value by casting to type fmtT. This default implementation +// should never be called. +template::value> +struct formatValueAsType +{ + static void invoke(std::ostream& /*out*/, const T& /*value*/) { assert(0); } +}; +// Specialized version for types that can actually be converted to fmtT, as +// indicated by the "convertible" template parameter. +template +struct formatValueAsType +{ + static void invoke(std::ostream& out, const T& value) + { out << static_cast(value); } +}; + +#ifdef TINYFORMAT_OLD_LIBSTDCPLUSPLUS_WORKAROUND +template::value> +struct formatZeroIntegerWorkaround +{ + static bool invoke(std::ostream& /**/, const T& /**/) { return false; } +}; +template +struct formatZeroIntegerWorkaround +{ + static bool invoke(std::ostream& out, const T& value) + { + if (static_cast(value) == 0 && out.flags() & std::ios::showpos) + { + out << "+0"; + return true; + } + return false; + } +}; +#endif // TINYFORMAT_OLD_LIBSTDCPLUSPLUS_WORKAROUND + +// Convert an arbitrary type to integer. The version with convertible=false +// throws an error. +template::value> +struct convertToInt +{ + static int invoke(const T& /*value*/) + { + TINYFORMAT_ERROR("tinyformat: Cannot convert from argument type to " + "integer for use as variable width or precision"); + return 0; + } +}; +// Specialization for convertToInt when conversion is possible +template +struct convertToInt +{ + static int invoke(const T& value) { return static_cast(value); } +}; + +} // namespace detail + + +//------------------------------------------------------------------------------ +// Variable formatting functions. May be overridden for user-defined types if +// desired. + + +// Format a value into a stream. Called from format() for all types by default. +// +// Users may override this for their own types. When this function is called, +// the stream flags will have been modified according to the format string. +// The format specification is provided in the range [fmtBegin, fmtEnd). +// +// By default, formatValue() uses the usual stream insertion operator +// operator<< to format the type T, with special cases for the %c and %p +// conversions. +template +inline void formatValue(std::ostream& out, const char* /*fmtBegin*/, + const char* fmtEnd, const T& value) +{ +#ifndef TINYFORMAT_ALLOW_WCHAR_STRINGS + // Since we don't support printing of wchar_t using "%ls", make it fail at + // compile time in preference to printing as a void* at runtime. + typedef typename detail::is_wchar::tinyformat_wchar_is_not_supported DummyType; + (void) DummyType(); // avoid unused type warning with gcc-4.8 +#endif + // The mess here is to support the %c and %p conversions: if these + // conversions are active we try to convert the type to a char or const + // void* respectively and format that instead of the value itself. For the + // %p conversion it's important to avoid dereferencing the pointer, which + // could otherwise lead to a crash when printing a dangling (const char*). + const bool canConvertToChar = detail::is_convertible::value; + const bool canConvertToVoidPtr = detail::is_convertible::value; + if(canConvertToChar && *(fmtEnd-1) == 'c') + detail::formatValueAsType::invoke(out, value); + else if(canConvertToVoidPtr && *(fmtEnd-1) == 'p') + detail::formatValueAsType::invoke(out, value); +#ifdef TINYFORMAT_OLD_LIBSTDCPLUSPLUS_WORKAROUND + else if(detail::formatZeroIntegerWorkaround::invoke(out, value)) /**/; +#endif + else + out << value; +} + + +// Overloaded version for char types to support printing as an integer +#define TINYFORMAT_DEFINE_FORMATVALUE_CHAR(charType) \ +inline void formatValue(std::ostream& out, const char* /*fmtBegin*/, \ + const char* fmtEnd, charType value) \ +{ \ + switch(*(fmtEnd-1)) \ + { \ + case 'u': case 'd': case 'i': case 'o': case 'X': case 'x': \ + out << static_cast(value); break; \ + default: \ + out << value; break; \ + } \ +} +// per 3.9.1: char, signed char and unsigned char are all distinct types +TINYFORMAT_DEFINE_FORMATVALUE_CHAR(char) +TINYFORMAT_DEFINE_FORMATVALUE_CHAR(signed char) +TINYFORMAT_DEFINE_FORMATVALUE_CHAR(unsigned char) +#undef TINYFORMAT_DEFINE_FORMATVALUE_CHAR + + +//------------------------------------------------------------------------------ +// Tools for emulating variadic templates in C++98. The basic idea here is +// stolen from the boost preprocessor metaprogramming library and cut down to +// be just general enough for what we need. + +#define TINYFORMAT_ARGTYPES(n) TINYFORMAT_ARGTYPES_ ## n +#define TINYFORMAT_VARARGS(n) TINYFORMAT_VARARGS_ ## n +#define TINYFORMAT_PASSARGS(n) TINYFORMAT_PASSARGS_ ## n +#define TINYFORMAT_PASSARGS_TAIL(n) TINYFORMAT_PASSARGS_TAIL_ ## n + +// To keep it as transparent as possible, the macros below have been generated +// using python via the excellent cog.py code generation script. This avoids +// the need for a bunch of complex (but more general) preprocessor tricks as +// used in boost.preprocessor. +// +// To rerun the code generation in place, use `cog.py -r tinyformat.h` +// (see http://nedbatchelder.com/code/cog). Alternatively you can just create +// extra versions by hand. + +/*[[[cog +maxParams = 16 + +def makeCommaSepLists(lineTemplate, elemTemplate, startInd=1): + for j in range(startInd,maxParams+1): + list = ', '.join([elemTemplate % {'i':i} for i in range(startInd,j+1)]) + cog.outl(lineTemplate % {'j':j, 'list':list}) + +makeCommaSepLists('#define TINYFORMAT_ARGTYPES_%(j)d %(list)s', + 'class T%(i)d') + +cog.outl() +makeCommaSepLists('#define TINYFORMAT_VARARGS_%(j)d %(list)s', + 'const T%(i)d& v%(i)d') + +cog.outl() +makeCommaSepLists('#define TINYFORMAT_PASSARGS_%(j)d %(list)s', 'v%(i)d') + +cog.outl() +cog.outl('#define TINYFORMAT_PASSARGS_TAIL_1') +makeCommaSepLists('#define TINYFORMAT_PASSARGS_TAIL_%(j)d , %(list)s', + 'v%(i)d', startInd = 2) + +cog.outl() +cog.outl('#define TINYFORMAT_FOREACH_ARGNUM(m) \\\n ' + + ' '.join(['m(%d)' % (j,) for j in range(1,maxParams+1)])) +]]]*/ +#define TINYFORMAT_ARGTYPES_1 class T1 +#define TINYFORMAT_ARGTYPES_2 class T1, class T2 +#define TINYFORMAT_ARGTYPES_3 class T1, class T2, class T3 +#define TINYFORMAT_ARGTYPES_4 class T1, class T2, class T3, class T4 +#define TINYFORMAT_ARGTYPES_5 class T1, class T2, class T3, class T4, class T5 +#define TINYFORMAT_ARGTYPES_6 class T1, class T2, class T3, class T4, class T5, class T6 +#define TINYFORMAT_ARGTYPES_7 class T1, class T2, class T3, class T4, class T5, class T6, class T7 +#define TINYFORMAT_ARGTYPES_8 class T1, class T2, class T3, class T4, class T5, class T6, class T7, class T8 +#define TINYFORMAT_ARGTYPES_9 class T1, class T2, class T3, class T4, class T5, class T6, class T7, class T8, class T9 +#define TINYFORMAT_ARGTYPES_10 class T1, class T2, class T3, class T4, class T5, class T6, class T7, class T8, class T9, class T10 +#define TINYFORMAT_ARGTYPES_11 class T1, class T2, class T3, class T4, class T5, class T6, class T7, class T8, class T9, class T10, class T11 +#define TINYFORMAT_ARGTYPES_12 class T1, class T2, class T3, class T4, class T5, class T6, class T7, class T8, class T9, class T10, class T11, class T12 +#define TINYFORMAT_ARGTYPES_13 class T1, class T2, class T3, class T4, class T5, class T6, class T7, class T8, class T9, class T10, class T11, class T12, class T13 +#define TINYFORMAT_ARGTYPES_14 class T1, class T2, class T3, class T4, class T5, class T6, class T7, class T8, class T9, class T10, class T11, class T12, class T13, class T14 +#define TINYFORMAT_ARGTYPES_15 class T1, class T2, class T3, class T4, class T5, class T6, class T7, class T8, class T9, class T10, class T11, class T12, class T13, class T14, class T15 +#define TINYFORMAT_ARGTYPES_16 class T1, class T2, class T3, class T4, class T5, class T6, class T7, class T8, class T9, class T10, class T11, class T12, class T13, class T14, class T15, class T16 + +#define TINYFORMAT_VARARGS_1 const T1& v1 +#define TINYFORMAT_VARARGS_2 const T1& v1, const T2& v2 +#define TINYFORMAT_VARARGS_3 const T1& v1, const T2& v2, const T3& v3 +#define TINYFORMAT_VARARGS_4 const T1& v1, const T2& v2, const T3& v3, const T4& v4 +#define TINYFORMAT_VARARGS_5 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5 +#define TINYFORMAT_VARARGS_6 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5, const T6& v6 +#define TINYFORMAT_VARARGS_7 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5, const T6& v6, const T7& v7 +#define TINYFORMAT_VARARGS_8 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5, const T6& v6, const T7& v7, const T8& v8 +#define TINYFORMAT_VARARGS_9 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5, const T6& v6, const T7& v7, const T8& v8, const T9& v9 +#define TINYFORMAT_VARARGS_10 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5, const T6& v6, const T7& v7, const T8& v8, const T9& v9, const T10& v10 +#define TINYFORMAT_VARARGS_11 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5, const T6& v6, const T7& v7, const T8& v8, const T9& v9, const T10& v10, const T11& v11 +#define TINYFORMAT_VARARGS_12 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5, const T6& v6, const T7& v7, const T8& v8, const T9& v9, const T10& v10, const T11& v11, const T12& v12 +#define TINYFORMAT_VARARGS_13 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5, const T6& v6, const T7& v7, const T8& v8, const T9& v9, const T10& v10, const T11& v11, const T12& v12, const T13& v13 +#define TINYFORMAT_VARARGS_14 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5, const T6& v6, const T7& v7, const T8& v8, const T9& v9, const T10& v10, const T11& v11, const T12& v12, const T13& v13, const T14& v14 +#define TINYFORMAT_VARARGS_15 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5, const T6& v6, const T7& v7, const T8& v8, const T9& v9, const T10& v10, const T11& v11, const T12& v12, const T13& v13, const T14& v14, const T15& v15 +#define TINYFORMAT_VARARGS_16 const T1& v1, const T2& v2, const T3& v3, const T4& v4, const T5& v5, const T6& v6, const T7& v7, const T8& v8, const T9& v9, const T10& v10, const T11& v11, const T12& v12, const T13& v13, const T14& v14, const T15& v15, const T16& v16 + +#define TINYFORMAT_PASSARGS_1 v1 +#define TINYFORMAT_PASSARGS_2 v1, v2 +#define TINYFORMAT_PASSARGS_3 v1, v2, v3 +#define TINYFORMAT_PASSARGS_4 v1, v2, v3, v4 +#define TINYFORMAT_PASSARGS_5 v1, v2, v3, v4, v5 +#define TINYFORMAT_PASSARGS_6 v1, v2, v3, v4, v5, v6 +#define TINYFORMAT_PASSARGS_7 v1, v2, v3, v4, v5, v6, v7 +#define TINYFORMAT_PASSARGS_8 v1, v2, v3, v4, v5, v6, v7, v8 +#define TINYFORMAT_PASSARGS_9 v1, v2, v3, v4, v5, v6, v7, v8, v9 +#define TINYFORMAT_PASSARGS_10 v1, v2, v3, v4, v5, v6, v7, v8, v9, v10 +#define TINYFORMAT_PASSARGS_11 v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11 +#define TINYFORMAT_PASSARGS_12 v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12 +#define TINYFORMAT_PASSARGS_13 v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13 +#define TINYFORMAT_PASSARGS_14 v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14 +#define TINYFORMAT_PASSARGS_15 v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15 +#define TINYFORMAT_PASSARGS_16 v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16 + +#define TINYFORMAT_PASSARGS_TAIL_1 +#define TINYFORMAT_PASSARGS_TAIL_2 , v2 +#define TINYFORMAT_PASSARGS_TAIL_3 , v2, v3 +#define TINYFORMAT_PASSARGS_TAIL_4 , v2, v3, v4 +#define TINYFORMAT_PASSARGS_TAIL_5 , v2, v3, v4, v5 +#define TINYFORMAT_PASSARGS_TAIL_6 , v2, v3, v4, v5, v6 +#define TINYFORMAT_PASSARGS_TAIL_7 , v2, v3, v4, v5, v6, v7 +#define TINYFORMAT_PASSARGS_TAIL_8 , v2, v3, v4, v5, v6, v7, v8 +#define TINYFORMAT_PASSARGS_TAIL_9 , v2, v3, v4, v5, v6, v7, v8, v9 +#define TINYFORMAT_PASSARGS_TAIL_10 , v2, v3, v4, v5, v6, v7, v8, v9, v10 +#define TINYFORMAT_PASSARGS_TAIL_11 , v2, v3, v4, v5, v6, v7, v8, v9, v10, v11 +#define TINYFORMAT_PASSARGS_TAIL_12 , v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12 +#define TINYFORMAT_PASSARGS_TAIL_13 , v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13 +#define TINYFORMAT_PASSARGS_TAIL_14 , v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14 +#define TINYFORMAT_PASSARGS_TAIL_15 , v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15 +#define TINYFORMAT_PASSARGS_TAIL_16 , v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16 + +#define TINYFORMAT_FOREACH_ARGNUM(m) \ + m(1) m(2) m(3) m(4) m(5) m(6) m(7) m(8) m(9) m(10) m(11) m(12) m(13) m(14) m(15) m(16) +//[[[end]]] + + + +namespace detail { + +// Class holding current position in format string and an output stream into +// which arguments are formatted. +class FormatIterator +{ + public: + // Flags for features not representable with standard stream state + enum ExtraFormatFlags + { + Flag_None = 0, + Flag_TruncateToPrecision = 1<<0, // truncate length to stream precision() + Flag_SpacePadPositive = 1<<1, // pad positive values with spaces + Flag_VariableWidth = 1<<2, // variable field width in arg list + Flag_VariablePrecision = 1<<3 // variable field precision in arg list + }; + + // out is the output stream, fmt is the full format string + FormatIterator(std::ostream& out, const char* fmt) + : m_out(out), + m_fmt(fmt), + m_extraFlags(Flag_None), + m_wantWidth(false), + m_wantPrecision(false), + m_variableWidth(0), + m_variablePrecision(0), + m_origWidth(out.width()), + m_origPrecision(out.precision()), + m_origFlags(out.flags()), + m_origFill(out.fill()) + { } + + // Print remaining part of format string. + void finish() + { + // It would be nice if we could do this from the destructor, but we + // can't if TINFORMAT_ERROR is used to throw an exception! + m_fmt = printFormatStringLiteral(m_out, m_fmt); + if(*m_fmt != '\0') + TINYFORMAT_ERROR("tinyformat: Too many conversion specifiers in format string"); + } + + ~FormatIterator() + { + // Restore stream state + m_out.width(m_origWidth); + m_out.precision(m_origPrecision); + m_out.flags(m_origFlags); + m_out.fill(m_origFill); + } + + template + void accept(const T& value); + + private: + // Parse and return an integer from the string c, as atoi() + // On return, c is set to one past the end of the integer. + static int parseIntAndAdvance(const char*& c) + { + int i = 0; + for(;*c >= '0' && *c <= '9'; ++c) + i = 10*i + (*c - '0'); + return i; + } + + // Format at most truncLen characters of a C string to the given + // stream. Return true if formatting proceeded (generic version always + // returns false) + template + static bool formatCStringTruncate(std::ostream& /*out*/, const T& /*value*/, + std::streamsize /*truncLen*/) + { + return false; + } +# define TINYFORMAT_DEFINE_FORMAT_C_STRING_TRUNCATE(type) \ + static bool formatCStringTruncate(std::ostream& out, type* value, \ + std::streamsize truncLen) \ + { \ + std::streamsize len = 0; \ + while(len < truncLen && value[len] != 0) \ + ++len; \ + out.write(value, len); \ + return true; \ + } + // Overload for const char* and char*. Could overload for signed & + // unsigned char too, but these are technically unneeded for printf + // compatibility. + TINYFORMAT_DEFINE_FORMAT_C_STRING_TRUNCATE(const char) + TINYFORMAT_DEFINE_FORMAT_C_STRING_TRUNCATE(char) +# undef TINYFORMAT_DEFINE_FORMAT_C_STRING_TRUNCATE + + // Print literal part of format string and return next format spec + // position. + // + // Skips over any occurrences of '%%', printing a literal '%' to the + // output. The position of the first % character of the next + // nontrivial format spec is returned, or the end of string. + static const char* printFormatStringLiteral(std::ostream& out, + const char* fmt) + { + const char* c = fmt; + for(; true; ++c) + { + switch(*c) + { + case '\0': + out.write(fmt, static_cast(c - fmt)); + return c; + case '%': + out.write(fmt, static_cast(c - fmt)); + if(*(c+1) != '%') + return c; + // for "%%", tack trailing % onto next literal section. + fmt = ++c; + break; + } + } + } + + static const char* streamStateFromFormat(std::ostream& out, + unsigned int& extraFlags, + const char* fmtStart, + int variableWidth, + int variablePrecision); + + // Private copy & assign: Kill gcc warnings with -Weffc++ + FormatIterator(const FormatIterator&); + FormatIterator& operator=(const FormatIterator&); + + // Stream, current format string & state + std::ostream& m_out; + const char* m_fmt; + unsigned int m_extraFlags; + // State machine info for handling of variable width & precision + bool m_wantWidth; + bool m_wantPrecision; + int m_variableWidth; + int m_variablePrecision; + // Saved stream state + std::streamsize m_origWidth; + std::streamsize m_origPrecision; + std::ios::fmtflags m_origFlags; + char m_origFill; +}; + + +// Accept a value for formatting into the internal stream. +template +TINYFORMAT_NOINLINE // < greatly reduces bloat in optimized builds +void FormatIterator::accept(const T& value) +{ + // Parse the format string + const char* fmtEnd = 0; + if(m_extraFlags == Flag_None && !m_wantWidth && !m_wantPrecision) + { + m_fmt = printFormatStringLiteral(m_out, m_fmt); + fmtEnd = streamStateFromFormat(m_out, m_extraFlags, m_fmt, 0, 0); + m_wantWidth = (m_extraFlags & Flag_VariableWidth) != 0; + m_wantPrecision = (m_extraFlags & Flag_VariablePrecision) != 0; + } + // Consume value as variable width and precision specifier if necessary + if(m_extraFlags & (Flag_VariableWidth | Flag_VariablePrecision)) + { + if(m_wantWidth || m_wantPrecision) + { + int v = convertToInt::invoke(value); + if(m_wantWidth) + { + m_variableWidth = v; + m_wantWidth = false; + } + else if(m_wantPrecision) + { + m_variablePrecision = v; + m_wantPrecision = false; + } + return; + } + // If we get here, we've set both the variable precision and width as + // required and we need to rerun the stream state setup to insert these. + fmtEnd = streamStateFromFormat(m_out, m_extraFlags, m_fmt, + m_variableWidth, m_variablePrecision); + } + + // Format the value into the stream. + if(!(m_extraFlags & (Flag_SpacePadPositive | Flag_TruncateToPrecision))) + formatValue(m_out, m_fmt, fmtEnd, value); + else + { + // The following are special cases where there's no direct + // correspondence between stream formatting and the printf() behaviour. + // Instead, we simulate the behaviour crudely by formatting into a + // temporary string stream and munging the resulting string. + std::ostringstream tmpStream; + tmpStream.copyfmt(m_out); + if(m_extraFlags & Flag_SpacePadPositive) + tmpStream.setf(std::ios::showpos); + // formatCStringTruncate is required for truncating conversions like + // "%.4s" where at most 4 characters of the c-string should be read. + // If we didn't include this special case, we might read off the end. + if(!( (m_extraFlags & Flag_TruncateToPrecision) && + formatCStringTruncate(tmpStream, value, m_out.precision()) )) + { + // Not a truncated c-string; just format normally. + formatValue(tmpStream, m_fmt, fmtEnd, value); + } + std::string result = tmpStream.str(); // allocates... yuck. + if(m_extraFlags & Flag_SpacePadPositive) + { + for(size_t i = 0, iend = result.size(); i < iend; ++i) + if(result[i] == '+') + result[i] = ' '; + } + if((m_extraFlags & Flag_TruncateToPrecision) && + (int)result.size() > (int)m_out.precision()) + m_out.write(result.c_str(), m_out.precision()); + else + m_out << result; + } + m_extraFlags = Flag_None; + m_fmt = fmtEnd; +} + + +// Parse a format string and set the stream state accordingly. +// +// The format mini-language recognized here is meant to be the one from C99, +// with the form "%[flags][width][.precision][length]type". +// +// Formatting options which can't be natively represented using the ostream +// state are returned in the extraFlags parameter which is a bitwise +// combination of values from the ExtraFormatFlags enum. +inline const char* FormatIterator::streamStateFromFormat(std::ostream& out, + unsigned int& extraFlags, + const char* fmtStart, + int variableWidth, + int variablePrecision) +{ + if(*fmtStart != '%') + { + TINYFORMAT_ERROR("tinyformat: Not enough conversion specifiers in format string"); + return fmtStart; + } + // Reset stream state to defaults. + out.width(0); + out.precision(6); + out.fill(' '); + // Reset most flags; ignore irrelevant unitbuf & skipws. + out.unsetf(std::ios::adjustfield | std::ios::basefield | + std::ios::floatfield | std::ios::showbase | std::ios::boolalpha | + std::ios::showpoint | std::ios::showpos | std::ios::uppercase); + extraFlags = Flag_None; + bool precisionSet = false; + bool widthSet = false; + const char* c = fmtStart + 1; + // 1) Parse flags + for(;; ++c) + { + switch(*c) + { + case '#': + out.setf(std::ios::showpoint | std::ios::showbase); + continue; + case '0': + // overridden by left alignment ('-' flag) + if(!(out.flags() & std::ios::left)) + { + // Use internal padding so that numeric values are + // formatted correctly, eg -00010 rather than 000-10 + out.fill('0'); + out.setf(std::ios::internal, std::ios::adjustfield); + } + continue; + case '-': + out.fill(' '); + out.setf(std::ios::left, std::ios::adjustfield); + continue; + case ' ': + // overridden by show positive sign, '+' flag. + if(!(out.flags() & std::ios::showpos)) + extraFlags |= Flag_SpacePadPositive; + continue; + case '+': + out.setf(std::ios::showpos); + extraFlags &= ~Flag_SpacePadPositive; + continue; + } + break; + } + // 2) Parse width + if(*c >= '0' && *c <= '9') + { + widthSet = true; + out.width(parseIntAndAdvance(c)); + } + if(*c == '*') + { + widthSet = true; + if(variableWidth < 0) + { + // negative widths correspond to '-' flag set + out.fill(' '); + out.setf(std::ios::left, std::ios::adjustfield); + variableWidth = -variableWidth; + } + out.width(variableWidth); + extraFlags |= Flag_VariableWidth; + ++c; + } + // 3) Parse precision + if(*c == '.') + { + ++c; + int precision = 0; + if(*c == '*') + { + ++c; + extraFlags |= Flag_VariablePrecision; + precision = variablePrecision; + } + else + { + if(*c >= '0' && *c <= '9') + precision = parseIntAndAdvance(c); + else if(*c == '-') // negative precisions ignored, treated as zero. + parseIntAndAdvance(++c); + } + out.precision(precision); + precisionSet = true; + } + // 4) Ignore any C99 length modifier + while(*c == 'l' || *c == 'h' || *c == 'L' || + *c == 'j' || *c == 'z' || *c == 't') + ++c; + // 5) We're up to the conversion specifier character. + // Set stream flags based on conversion specifier (thanks to the + // boost::format class for forging the way here). + bool intConversion = false; + switch(*c) + { + case 'u': case 'd': case 'i': + out.setf(std::ios::dec, std::ios::basefield); + intConversion = true; + break; + case 'o': + out.setf(std::ios::oct, std::ios::basefield); + intConversion = true; + break; + case 'X': + out.setf(std::ios::uppercase); + case 'x': case 'p': + out.setf(std::ios::hex, std::ios::basefield); + intConversion = true; + break; + case 'E': + out.setf(std::ios::uppercase); + case 'e': + out.setf(std::ios::scientific, std::ios::floatfield); + out.setf(std::ios::dec, std::ios::basefield); + break; + case 'F': + out.setf(std::ios::uppercase); + case 'f': + out.setf(std::ios::fixed, std::ios::floatfield); + break; + case 'G': + out.setf(std::ios::uppercase); + case 'g': + out.setf(std::ios::dec, std::ios::basefield); + // As in boost::format, let stream decide float format. + out.flags(out.flags() & ~std::ios::floatfield); + break; + case 'a': case 'A': + TINYFORMAT_ERROR("tinyformat: the %a and %A conversion specs " + "are not supported"); + break; + case 'c': + // Handled as special case inside formatValue() + break; + case 's': + if(precisionSet) + extraFlags |= Flag_TruncateToPrecision; + // Make %s print booleans as "true" and "false" + out.setf(std::ios::boolalpha); + break; + case 'n': + // Not supported - will cause problems! + TINYFORMAT_ERROR("tinyformat: %n conversion spec not supported"); + break; + case '\0': + TINYFORMAT_ERROR("tinyformat: Conversion spec incorrectly " + "terminated by end of string"); + return c; + } + if(intConversion && precisionSet && !widthSet) + { + // "precision" for integers gives the minimum number of digits (to be + // padded with zeros on the left). This isn't really supported by the + // iostreams, but we can approximately simulate it with the width if + // the width isn't otherwise used. + out.width(out.precision()); + out.setf(std::ios::internal, std::ios::adjustfield); + out.fill('0'); + } + return c+1; +} + + + +//------------------------------------------------------------------------------ +// Private format function on top of which the public interface is implemented. +// We enforce a mimimum of one value to be formatted to prevent bugs looking like +// +// const char* myStr = "100% broken"; +// printf(myStr); // Parses % as a format specifier +#ifdef TINYFORMAT_USE_VARIADIC_TEMPLATES + +template +void format(FormatIterator& fmtIter, const T1& value1) +{ + fmtIter.accept(value1); + fmtIter.finish(); +} + +// General version for C++11 +template +void format(FormatIterator& fmtIter, const T1& value1, const Args&... args) +{ + fmtIter.accept(value1); + format(fmtIter, args...); +} + +#else + +inline void format(FormatIterator& fmtIter) +{ + fmtIter.finish(); +} + +// General version for C++98 +#define TINYFORMAT_MAKE_FORMAT_DETAIL(n) \ +template \ +void format(detail::FormatIterator& fmtIter, TINYFORMAT_VARARGS(n)) \ +{ \ + fmtIter.accept(v1); \ + format(fmtIter TINYFORMAT_PASSARGS_TAIL(n)); \ +} + +TINYFORMAT_FOREACH_ARGNUM(TINYFORMAT_MAKE_FORMAT_DETAIL) +#undef TINYFORMAT_MAKE_FORMAT_DETAIL + +#endif // End C++98 variadic template emulation for format() + +} // namespace detail + + +//------------------------------------------------------------------------------ +// Implement all the main interface functions here in terms of detail::format() + +#ifdef TINYFORMAT_USE_VARIADIC_TEMPLATES + +// C++11 - the simple case +template +void format(std::ostream& out, const char* fmt, const T1& v1, const Args&... args) +{ + detail::FormatIterator fmtIter(out, fmt); + format(fmtIter, v1, args...); +} + +template +std::string format(const char* fmt, const T1& v1, const Args&... args) +{ + std::ostringstream oss; + format(oss, fmt, v1, args...); + return oss.str(); +} + +template +std::string format(const std::string &fmt, const T1& v1, const Args&... args) +{ + std::ostringstream oss; + format(oss, fmt.c_str(), v1, args...); + return oss.str(); +} + +template +void printf(const char* fmt, const T1& v1, const Args&... args) +{ + format(std::cout, fmt, v1, args...); +} + +#else + +// C++98 - define the interface functions using the wrapping macros +#define TINYFORMAT_MAKE_FORMAT_FUNCS(n) \ + \ +template \ +void format(std::ostream& out, const char* fmt, TINYFORMAT_VARARGS(n)) \ +{ \ + tinyformat::detail::FormatIterator fmtIter(out, fmt); \ + tinyformat::detail::format(fmtIter, TINYFORMAT_PASSARGS(n)); \ +} \ + \ +template \ +std::string format(const char* fmt, TINYFORMAT_VARARGS(n)) \ +{ \ + std::ostringstream oss; \ + tinyformat::format(oss, fmt, TINYFORMAT_PASSARGS(n)); \ + return oss.str(); \ +} \ + \ +template \ +std::string format(const std::string &fmt, TINYFORMAT_VARARGS(n)) \ +{ \ + std::ostringstream oss; \ + tinyformat::format(oss, fmt.c_str(), TINYFORMAT_PASSARGS(n)); \ + return oss.str(); \ +} \ + \ +template \ +void printf(const char* fmt, TINYFORMAT_VARARGS(n)) \ +{ \ + tinyformat::format(std::cout, fmt, TINYFORMAT_PASSARGS(n)); \ +} + +TINYFORMAT_FOREACH_ARGNUM(TINYFORMAT_MAKE_FORMAT_FUNCS) +#undef TINYFORMAT_MAKE_FORMAT_FUNCS +#endif + + +//------------------------------------------------------------------------------ +// Define deprecated wrapping macro for backward compatibility in tinyformat +// 1.x. Will be removed in version 2! +#define TINYFORMAT_WRAP_FORMAT_EXTRA_ARGS +#define TINYFORMAT_WRAP_FORMAT_N(n, returnType, funcName, funcDeclSuffix, \ + bodyPrefix, streamName, bodySuffix) \ +template \ +returnType funcName(TINYFORMAT_WRAP_FORMAT_EXTRA_ARGS const char* fmt, \ + TINYFORMAT_VARARGS(n)) funcDeclSuffix \ +{ \ + bodyPrefix \ + tinyformat::format(streamName, fmt, TINYFORMAT_PASSARGS(n)); \ + bodySuffix \ +} \ + +#define TINYFORMAT_WRAP_FORMAT(returnType, funcName, funcDeclSuffix, \ + bodyPrefix, streamName, bodySuffix) \ +inline \ +returnType funcName(TINYFORMAT_WRAP_FORMAT_EXTRA_ARGS const char* fmt \ + ) funcDeclSuffix \ +{ \ + bodyPrefix \ + tinyformat::detail::FormatIterator(streamName, fmt).finish(); \ + bodySuffix \ +} \ +TINYFORMAT_WRAP_FORMAT_N(1 , returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(2 , returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(3 , returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(4 , returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(5 , returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(6 , returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(7 , returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(8 , returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(9 , returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(10, returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(11, returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(12, returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(13, returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(14, returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(15, returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ +TINYFORMAT_WRAP_FORMAT_N(16, returnType, funcName, funcDeclSuffix, bodyPrefix, streamName, bodySuffix) \ + + +} // namespace tinyformat + +#define strprintf tfm::format + +#endif // TINYFORMAT_H_INCLUDED diff --git a/src/utils/tools.cpp b/src/utils/tools.cpp new file mode 100644 index 00000000..770194d8 --- /dev/null +++ b/src/utils/tools.cpp @@ -0,0 +1,103 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" + +#include +#include +#include + +using namespace std; + + +void mc_MapStringIndex::Init() +{ + Destroy(); + mapObject=new std::map; +} + +void mc_MapStringIndex::Destroy() +{ + if(mapObject) + { + delete (std::map*)mapObject; + } +} + +void mc_MapStringIndex::Clear() +{ + Init(); +} + +void mc_MapStringIndex::Add(const char* key, int value) +{ + ((std::map*)mapObject)->insert(std::pair(string(key), value)); +} + +void mc_MapStringIndex::Add(const unsigned char* key, int size, int value) +{ + ((std::map*)mapObject)->insert(std::pair(string(key,key+size), value)); +} + +void mc_MapStringIndex::Remove(const char* key,int size) +{ + ((std::map*)mapObject)->erase(string(key,key+size)); +} + +int mc_MapStringIndex::Get(const char* key) +{ + std::map::iterator it; + int value=-1; + it=((std::map*)mapObject)->find(string(key)); + if (it != ((std::map*)mapObject)->end()) + { + value=it->second; + } + return value; +} + +int mc_MapStringIndex::Get(const unsigned char* key,int size) +{ + std::map::iterator it; + int value=-1; + it=((std::map*)mapObject)->find(string(key,key+size)); + if (it != ((std::map*)mapObject)->end()) + { + value=it->second; + } + return value; +} + +void mc_MapStringString::Init() +{ + mapObject=new std::map; +} + +void mc_MapStringString::Destroy() +{ + if(mapObject) + { + delete (std::map*)mapObject; + } +} + +void mc_MapStringString::Add(const char* key, const char* value) +{ + ((std::map*)mapObject)->insert(std::pair(string(key), value)); +} + +const char* mc_MapStringString::Get(const char* key) +{ + std::map::iterator it; + it=((std::map*)mapObject)->find(string(key)); + if (it != ((std::map*)mapObject)->end()) + { + return it->second.c_str(); + } + return NULL; +} + +int mc_MapStringString::GetCount() +{ + return ((std::map*)mapObject)->size(); +} \ No newline at end of file diff --git a/src/utils/util.cpp b/src/utils/util.cpp new file mode 100644 index 00000000..8d6266b7 --- /dev/null +++ b/src/utils/util.cpp @@ -0,0 +1,854 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#if defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" +#endif + +#include "utils/util.h" + +#include "chainparams/chainparamsbase.h" +#include "utils/random.h" +#include "utils/serialize.h" +#include "utils/sync.h" +#include "utils/utilstrencodings.h" +#include "utils/utiltime.h" + +#include "multichain/multichain.h" // MCHN + +#include + +#ifndef WIN32 +// for posix_fallocate +#ifdef __linux__ + +#ifdef _POSIX_C_SOURCE +#undef _POSIX_C_SOURCE +#endif + +#define _POSIX_C_SOURCE 200112L + +#endif // __linux__ + +#include +#include +#include +#include + +#else + +#ifdef _MSC_VER +#pragma warning(disable:4786) +#pragma warning(disable:4804) +#pragma warning(disable:4805) +#pragma warning(disable:4717) +#endif + +#ifdef _WIN32_WINNT +#undef _WIN32_WINNT +#endif +#define _WIN32_WINNT 0x0501 + +#ifdef _WIN32_IE +#undef _WIN32_IE +#endif +#define _WIN32_IE 0x0501 + +#define WIN32_LEAN_AND_MEAN 1 +#ifndef NOMINMAX +#define NOMINMAX +#endif + +#include /* for _commit */ +#include +#endif + +#ifdef HAVE_SYS_PRCTL_H +#include +#endif + +#include // for to_lower() +#include +#include // for startswith() and endswith() +#include +#include +#include +#include +#include +#include +#include +#include + +// Work around clang compilation problem in Boost 1.46: +// /usr/include/boost/program_options/detail/config_file.hpp:163:17: error: call to function 'to_internal' that is neither visible in the template definition nor found by argument-dependent lookup +// See also: http://stackoverflow.com/questions/10020179/compilation-fail-in-boost-librairies-program-options +// http://clang.debian.net/status.php?version=3.0&key=CANNOT_FIND_FUNCTION +namespace boost { + + namespace program_options { + std::string to_internal(const std::string&); + } + +} // namespace boost + +using namespace std; + +map mapArgs; +map > mapMultiArgs; +bool fDebug = false; +bool fPrintToConsole = false; +bool fPrintToDebugLog = true; +bool fDaemon = false; +bool fServer = false; +string strMiscWarning; +bool fLogTimestamps = false; +bool fLogIPs = false; +bool fLogTimeMillis = false; +volatile bool fReopenDebugLog = false; + +/** Init OpenSSL library multithreading support */ +static CCriticalSection** ppmutexOpenSSL; +void locking_callback(int mode, int i, const char* file, int line) +{ + if (mode & CRYPTO_LOCK) { + ENTER_CRITICAL_SECTION(*ppmutexOpenSSL[i]); + } else { + LEAVE_CRITICAL_SECTION(*ppmutexOpenSSL[i]); + } +} + +// Init +class CInit +{ +public: + CInit() + { + // Init OpenSSL library multithreading support + ppmutexOpenSSL = (CCriticalSection**)OPENSSL_malloc(CRYPTO_num_locks() * sizeof(CCriticalSection*)); + for (int i = 0; i < CRYPTO_num_locks(); i++) + ppmutexOpenSSL[i] = new CCriticalSection(); + CRYPTO_set_locking_callback(locking_callback); + +#ifdef WIN32 + // Seed OpenSSL PRNG with current contents of the screen + RAND_screen(); +#endif + + // Seed OpenSSL PRNG with performance counter + RandAddSeed(); + } + ~CInit() + { + // Securely erase the memory used by the PRNG + RAND_cleanup(); + // Shutdown OpenSSL library multithreading support + CRYPTO_set_locking_callback(NULL); + for (int i = 0; i < CRYPTO_num_locks(); i++) + delete ppmutexOpenSSL[i]; + OPENSSL_free(ppmutexOpenSSL); + } +} +instance_of_cinit; + +/** + * LogPrintf() has been broken a couple of times now + * by well-meaning people adding mutexes in the most straightforward way. + * It breaks because it may be called by global destructors during shutdown. + * Since the order of destruction of static/global objects is undefined, + * defining a mutex as a global object doesn't work (the mutex gets + * destroyed, and then some later destructor calls OutputDebugStringF, + * maybe indirectly, and you get a core dump at shutdown trying to lock + * the mutex). + */ + +static boost::once_flag debugPrintInitFlag = BOOST_ONCE_INIT; +/** + * We use boost::call_once() to make sure these are initialized + * in a thread-safe manner the first time called: + */ +static FILE* fileout = NULL; +static boost::mutex* mutexDebugLog = NULL; + +static void DebugPrintInit() +{ + assert(fileout == NULL); + assert(mutexDebugLog == NULL); + + boost::filesystem::path pathDebug = GetDataDir() / "debug.log"; + fileout = fopen(pathDebug.string().c_str(), "a"); + if (fileout) setbuf(fileout, NULL); // unbuffered + + mutexDebugLog = new boost::mutex(); +} + +/* MCHN START */ + +void DebugPrintClose() +{ + if(fileout) + { + fclose(fileout); + fileout=NULL; + } +} + +/* MCHN END */ + +bool LogAcceptCategory(const char* category) +{ + if (category != NULL) + { + if (!fDebug) + return false; + + // Give each thread quick access to -debug settings. + // This helps prevent issues debugging global destructors, + // where mapMultiArgs might be deleted before another + // global destructor calls LogPrint() + static boost::thread_specific_ptr > ptrCategory; + if (ptrCategory.get() == NULL) + { + const vector& categories = mapMultiArgs["-debug"]; + ptrCategory.reset(new set(categories.begin(), categories.end())); + // thread_specific_ptr automatically deletes the set when the thread ends. + } + const set& setCategories = *ptrCategory.get(); + + // if not debugging everything and not debugging specific category, LogPrint does nothing. + if (setCategories.count(string("")) == 0 && + setCategories.count(string(category)) == 0) + return false; + } + return true; +} + +int LogPrintStr(const std::string &str) +{ + int ret = 0; // Returns total number of characters written + if (fPrintToConsole) + { + // print to console + ret = fwrite(str.data(), 1, str.size(), stdout); + fflush(stdout); + } + else if (fPrintToDebugLog && AreBaseParamsConfigured()) + { + static bool fStartedNewLine = true; + boost::call_once(&DebugPrintInit, debugPrintInitFlag); + + if (fileout == NULL) + return ret; + + boost::mutex::scoped_lock scoped_lock(*mutexDebugLog); + + // reopen the log file, if requested + if (fReopenDebugLog) { + fReopenDebugLog = false; + boost::filesystem::path pathDebug = GetDataDir() / "debug.log"; + if (freopen(pathDebug.string().c_str(),"a",fileout) != NULL) + setbuf(fileout, NULL); // unbuffered + } + + // Debug print useful for profiling + if (fLogTimestamps && fStartedNewLine) +/* MCHN PRMF */ + ret += fprintf(fileout, "%s", DateTimeStrFormat("%Y-%m-%d %H:%M:%S", GetTime()).c_str()); +// ret += fprintf(fileout, "%s ", DateTimeStrFormat("%Y-%m-%d %H:%M:%S", GetTime()).c_str()); + if (fLogTimestamps && fStartedNewLine) + { + if (fLogTimeMillis) + { + ret += fprintf(fileout, ".%03d ",(int)(GetTimeMillis()%1000)); + } + else + { + ret += fprintf(fileout, " "); + } + } +/* MCHN END */ + if (!str.empty() && str[str.size()-1] == '\n') + fStartedNewLine = true; + else + fStartedNewLine = false; + + ret = fwrite(str.data(), 1, str.size(), fileout); + } + + return ret; +} + +static void InterpretNegativeSetting(string name, map& mapSettingsRet) +{ + // interpret -nofoo as -foo=0 (and -nofoo=0 as -foo=1) as long as -foo not set + if (name.find("-no") == 0) + { + std::string positive("-"); + positive.append(name.begin()+3, name.end()); + if (mapSettingsRet.count(positive) == 0) + { + bool value = !GetBoolArg(name, false); + mapSettingsRet[positive] = (value ? "1" : "0"); + } + } +} + +void ParseParameters(int argc, const char* const argv[]) +{ + mapArgs.clear(); + mapMultiArgs.clear(); + + for (int i = 1; i < argc; i++) + { + std::string str(argv[i]); + std::string strValue; + size_t is_index = str.find('='); + if (is_index != std::string::npos) + { + strValue = str.substr(is_index+1); + str = str.substr(0, is_index); + } +#ifdef WIN32 + boost::to_lower(str); + if (boost::algorithm::starts_with(str, "/")) + str = "-" + str.substr(1); +#endif + +/* MCHN START */ +// Ignoring arguments not starting with "-". original bitcoin code discard everything after that + +/* + if (str[0] != '-') + break; +*/ + if (str[0] == '-') + { + // Interpret --foo as -foo. + // If both --foo and -foo are set, the last takes effect. + if (str.length() > 1 && str[1] == '-') + str = str.substr(1); + mapArgs[str] = strValue; + mapMultiArgs[str].push_back(strValue); + } + +/* MCHN END */ + + + } + + // New 0.6 features: + BOOST_FOREACH(const PAIRTYPE(string,string)& entry, mapArgs) + { + // interpret -nofoo as -foo=0 (and -nofoo=0 as -foo=1) as long as -foo not set + InterpretNegativeSetting(entry.first, mapArgs); + } +} + +std::string GetArg(const std::string& strArg, const std::string& strDefault) +{ + if (mapArgs.count(strArg)) + return mapArgs[strArg]; + return strDefault; +} + +int64_t GetArg(const std::string& strArg, int64_t nDefault) +{ + if (mapArgs.count(strArg)) + return atoi64(mapArgs[strArg]); + return nDefault; +} + +bool GetBoolArg(const std::string& strArg, bool fDefault) +{ + if (mapArgs.count(strArg)) + { + if (mapArgs[strArg].empty()) + return true; + return (atoi(mapArgs[strArg]) != 0); + } + return fDefault; +} + +bool SoftSetArg(const std::string& strArg, const std::string& strValue) +{ + if (mapArgs.count(strArg)) + return false; + mapArgs[strArg] = strValue; + return true; +} + +bool SoftSetBoolArg(const std::string& strArg, bool fValue) +{ + if (fValue) + return SoftSetArg(strArg, std::string("1")); + else + return SoftSetArg(strArg, std::string("0")); +} + +static std::string FormatException(std::exception* pex, const char* pszThread) +{ +#ifdef WIN32 + char pszModule[MAX_PATH] = ""; + GetModuleFileNameA(NULL, pszModule, sizeof(pszModule)); +#else + const char* pszModule = "bitcoin"; +#endif + if (pex) + return strprintf( + "EXCEPTION: %s \n%s \n%s in %s \n", typeid(*pex).name(), pex->what(), pszModule, pszThread); + else + return strprintf( + "UNKNOWN EXCEPTION \n%s in %s \n", pszModule, pszThread); +} + +void PrintExceptionContinue(std::exception* pex, const char* pszThread) +{ + std::string message = FormatException(pex, pszThread); + LogPrintf("\n\n************************\n%s\n", message); + fprintf(stderr, "\n\n************************\n%s\n", message.c_str()); + strMiscWarning = message; +} + +/* MCHN START */ +static boost::filesystem::path pathCachedMultiChain; +static CCriticalSection csPathCached; +/* MCHN END */ + +boost::filesystem::path GetDefaultDataDir() +{ + namespace fs = boost::filesystem; + // Windows < Vista: C:\Documents and Settings\Username\Application Data\Bitcoin + // Windows >= Vista: C:\Users\Username\AppData\Roaming\Bitcoin + // Mac: ~/Library/Application Support/Bitcoin + // Unix: ~/.bitcoin +/* MCHN START */ + LOCK(csPathCached); + + pathCachedMultiChain=fs::path(string(mc_gState->m_Params->DataDir())); + fs::path &path =pathCachedMultiChain; + return path; +// fs::path &path = fNetSpecific ? pathCachedNetSpecific : pathCached; +/* MCHN END */ +#ifdef WIN32 + // Windows + return GetSpecialFolderPath(CSIDL_APPDATA) / "MultiChain"; +#else + fs::path pathRet; + char* pszHome = getenv("HOME"); + if (pszHome == NULL || strlen(pszHome) == 0) + pathRet = fs::path("/"); + else + pathRet = fs::path(pszHome); +#ifdef MAC_OSX + // Mac + pathRet /= "Library/Application Support"; + TryCreateDirectory(pathRet); + return pathRet / "MultiChain"; +#else + // Unix + return pathRet / ".multichain"; +#endif +#endif +} + +static boost::filesystem::path pathCached; +static boost::filesystem::path pathCachedNetSpecific; +//static CCriticalSection csPathCached; + + +const boost::filesystem::path &GetDataDir(bool fNetSpecific) +{ + namespace fs = boost::filesystem; + + LOCK(csPathCached); + +/* MCHN START */ + fs::path &path =pathCachedMultiChain; + if (!path.empty()) + return path; + path=fs::path(string(mc_gState->m_Params->DataDir())); + return pathCachedMultiChain; +// pathCachedMultiChain=fs::path(string(mc_gState->m_Params->DataDir())); +// fs::path &path =pathCachedMultiChain; +// return path; +// fs::path &path = fNetSpecific ? pathCachedNetSpecific : pathCached; +/* MCHN END */ + // This can be called during exceptions by LogPrintf(), so we cache the + // value so we don't have to do memory allocations after that. + if (!path.empty()) + return path; + + if (mapArgs.count("-datadir")) { + path = fs::system_complete(mapArgs["-datadir"]); + if (!fs::is_directory(path)) { + path = ""; + return path; + } + } else { + path = GetDefaultDataDir(); + } + if (fNetSpecific) + path /= BaseParams().DataDir(); + + fs::create_directories(path); + + return path; +} + +void ClearDatadirCache() +{ + pathCached = boost::filesystem::path(); + pathCachedNetSpecific = boost::filesystem::path(); +} + +boost::filesystem::path GetConfigFile() +{ + boost::filesystem::path pathConfigFile(GetArg("-conf", string(mc_gState->m_Params->NetworkName()) + ".conf")); // MCHN + if (!pathConfigFile.is_complete()) + pathConfigFile = GetDataDir(false) / pathConfigFile; + + return pathConfigFile; +} + +void ReadConfigFile(map& mapSettingsRet, + map >& mapMultiSettingsRet) +{ + boost::filesystem::ifstream streamConfig(GetConfigFile()); + if (!streamConfig.good()) + return; // No bitcoin.conf file is OK + + set setOptions; + setOptions.insert("*"); + + for (boost::program_options::detail::config_file_iterator it(streamConfig, setOptions), end; it != end; ++it) + { + // Don't overwrite existing settings so command line settings override bitcoin.conf + string strKey = string("-") + it->string_key; + if (mapSettingsRet.count(strKey) == 0) + { + mapSettingsRet[strKey] = it->value[0]; + // interpret nofoo=1 as foo=0 (and nofoo=0 as foo=1) as long as foo not set) + InterpretNegativeSetting(strKey, mapSettingsRet); + } + mapMultiSettingsRet[strKey].push_back(it->value[0]); + } + // If datadir is changed in .conf file: + ClearDatadirCache(); +} + +#ifndef WIN32 +boost::filesystem::path GetPidFile() +{ + boost::filesystem::path pathPidFile(GetArg("-pid", "multichain.pid")); + if (!pathPidFile.is_complete()) pathPidFile = GetDataDir() / pathPidFile; + return pathPidFile; +} + +void CreatePidFile(const boost::filesystem::path &path, pid_t pid) +{ + FILE* file = fopen(path.string().c_str(), "w"); + if (file) + { + fprintf(file, "%d\n", pid); + fclose(file); + } +} +#endif + +bool RenameOver(boost::filesystem::path src, boost::filesystem::path dest) +{ +#ifdef WIN32 + return MoveFileExA(src.string().c_str(), dest.string().c_str(), + MOVEFILE_REPLACE_EXISTING) != 0; +#else + int rc = std::rename(src.string().c_str(), dest.string().c_str()); + return (rc == 0); +#endif /* WIN32 */ +} + +/** + * Ignores exceptions thrown by Boost's create_directory if the requested directory exists. + * Specifically handles case where path p exists, but it wasn't possible for the user to + * write to the parent directory. + */ +bool TryCreateDirectory(const boost::filesystem::path& p) +{ + try + { + return boost::filesystem::create_directory(p); + } catch (boost::filesystem::filesystem_error) { + if (!boost::filesystem::exists(p) || !boost::filesystem::is_directory(p)) + throw; + } + + // create_directory didn't create the directory, it had to have existed already + return false; +} + +void FileCommit(FILE *fileout) +{ + fflush(fileout); // harmless if redundantly called +#ifdef WIN32 + HANDLE hFile = (HANDLE)_get_osfhandle(_fileno(fileout)); + FlushFileBuffers(hFile); +#else + #if defined(__linux__) || defined(__NetBSD__) + fdatasync(fileno(fileout)); + #elif defined(__APPLE__) && defined(F_FULLFSYNC) + fcntl(fileno(fileout), F_FULLFSYNC, 0); + #else + fsync(fileno(fileout)); + #endif +#endif +} + +bool TruncateFile(FILE *file, unsigned int length) { +#if defined(WIN32) + return _chsize(_fileno(file), length) == 0; +#else + return ftruncate(fileno(file), length) == 0; +#endif +} + +/** + * this function tries to raise the file descriptor limit to the requested number. + * It returns the actual file descriptor limit (which may be more or less than nMinFD) + */ +int RaiseFileDescriptorLimit(int nMinFD) { +#if defined(WIN32) + return 2048; +#else + struct rlimit limitFD; + if (getrlimit(RLIMIT_NOFILE, &limitFD) != -1) { + if (limitFD.rlim_cur < (rlim_t)nMinFD) { + limitFD.rlim_cur = nMinFD; + if (limitFD.rlim_cur > limitFD.rlim_max) + limitFD.rlim_cur = limitFD.rlim_max; + setrlimit(RLIMIT_NOFILE, &limitFD); + getrlimit(RLIMIT_NOFILE, &limitFD); + } + return limitFD.rlim_cur; + } + return nMinFD; // getrlimit failed, assume it's fine +#endif +} + +/** + * this function tries to make a particular range of a file allocated (corresponding to disk space) + * it is advisory, and the range specified in the arguments will never contain live data + */ +void AllocateFileRange(FILE *file, unsigned int offset, unsigned int length) { +#if defined(WIN32) + // Windows-specific version + HANDLE hFile = (HANDLE)_get_osfhandle(_fileno(file)); + LARGE_INTEGER nFileSize; + int64_t nEndPos = (int64_t)offset + length; + nFileSize.u.LowPart = nEndPos & 0xFFFFFFFF; + nFileSize.u.HighPart = nEndPos >> 32; + SetFilePointerEx(hFile, nFileSize, 0, FILE_BEGIN); + SetEndOfFile(hFile); +#elif defined(MAC_OSX) + // OSX specific version + fstore_t fst; + fst.fst_flags = F_ALLOCATECONTIG; + fst.fst_posmode = F_PEOFPOSMODE; + fst.fst_offset = 0; + fst.fst_length = (off_t)offset + length; + fst.fst_bytesalloc = 0; + if (fcntl(fileno(file), F_PREALLOCATE, &fst) == -1) { + fst.fst_flags = F_ALLOCATEALL; + fcntl(fileno(file), F_PREALLOCATE, &fst); + } + ftruncate(fileno(file), fst.fst_length); +#elif defined(__linux__) + // Version using posix_fallocate + off_t nEndPos = (off_t)offset + length; + posix_fallocate(fileno(file), 0, nEndPos); +#else + // Fallback version + // TODO: just write one byte per block + static const char buf[65536] = {}; + fseek(file, offset, SEEK_SET); + while (length > 0) { + unsigned int now = 65536; + if (length < now) + now = length; + fwrite(buf, 1, now, file); // allowed to fail; this function is advisory anyway + length -= now; + } +#endif +} + +void ShrinkDebugFile(const char* FileName) +{ + // Scroll debug.log if it's getting too big + boost::filesystem::path pathLog = GetDataDir() / string(FileName); + FILE* file = fopen(pathLog.string().c_str(), "r"); +/* MCHN START */ + size_t bytes_written; + uint64_t shrink_size=GetArg("-shrinkdebugfilesize",200000); + if(shrink_size > 67108864) + { + shrink_size = 67108864; + } +/* MCHN END */ + if (file && boost::filesystem::file_size(pathLog) > 5 * shrink_size)// 10 * 1000000) + { + // Restart the file with some of the end + std::vector vch(shrink_size,0);//200000,0); + fseek(file, -((long)vch.size()), SEEK_END); + int nBytes = fread(begin_ptr(vch), 1, vch.size(), file); + fclose(file); + + file = fopen(pathLog.string().c_str(), "w"); + if (file) + { + bytes_written=fwrite(begin_ptr(vch), 1, nBytes, file); + if((int)bytes_written != nBytes) + { + fclose(file); + } + else + { + fclose(file); + } + } + } + else if (file != NULL) + fclose(file); +} + +void ShrinkDebugFile() +{ + ShrinkDebugFile("debug.log"); + ShrinkDebugFile("permissions.log"); + // Scroll debug.log if it's getting too big +/* + boost::filesystem::path pathLog = GetDataDir() / "debug.log"; + FILE* file = fopen(pathLog.string().c_str(), "r"); + int64_t shrink_size=GetArg("-shrinkdebugfilesize",200000); + if(shrink_size > 67108864) + { + shrink_size = 67108864; + } + if (file && boost::filesystem::file_size(pathLog) > 5 * shrink_size)// 10 * 1000000) + { + // Restart the file with some of the end + std::vector vch(shrink_size,0);//200000,0); + fseek(file, -((long)vch.size()), SEEK_END); + int nBytes = fread(begin_ptr(vch), 1, vch.size(), file); + fclose(file); + + file = fopen(pathLog.string().c_str(), "w"); + if (file) + { + fwrite(begin_ptr(vch), 1, nBytes, file); + fclose(file); + } + } + else if (file != NULL) + fclose(file); + */ +} + +#ifdef WIN32 +boost::filesystem::path GetSpecialFolderPath(int nFolder, bool fCreate) +{ + namespace fs = boost::filesystem; + + char pszPath[MAX_PATH] = ""; + + if(SHGetSpecialFolderPathA(NULL, pszPath, nFolder, fCreate)) + { + return fs::path(pszPath); + } + + LogPrintf("SHGetSpecialFolderPathA() failed, could not obtain requested path.\n"); + return fs::path(""); +} +#endif + +boost::filesystem::path GetTempPath() { +#if BOOST_FILESYSTEM_VERSION == 3 + return boost::filesystem::temp_directory_path(); +#else + // TODO: remove when we don't support filesystem v2 anymore + boost::filesystem::path path; +#ifdef WIN32 + char pszPath[MAX_PATH] = ""; + + if (GetTempPathA(MAX_PATH, pszPath)) + path = boost::filesystem::path(pszPath); +#else + path = boost::filesystem::path("/tmp"); +#endif + if (path.empty() || !boost::filesystem::is_directory(path)) { + LogPrintf("GetTempPath(): failed to find temp path\n"); + return boost::filesystem::path(""); + } + return path; +#endif +} + +void runCommand(std::string strCommand) +{ + int nErr = ::system(strCommand.c_str()); + if (nErr) + LogPrintf("runCommand error: system(%s) returned %d\n", strCommand, nErr); +} + +void RenameThread(const char* name) +{ +#if defined(PR_SET_NAME) + // Only the first 15 characters are used (16 - NUL terminator) + ::prctl(PR_SET_NAME, name, 0, 0, 0); +#elif 0 && (defined(__FreeBSD__) || defined(__OpenBSD__)) + // TODO: This is currently disabled because it needs to be verified to work + // on FreeBSD or OpenBSD first. When verified the '0 &&' part can be + // removed. + pthread_set_name_np(pthread_self(), name); + +#elif defined(MAC_OSX) && defined(__MAC_OS_X_VERSION_MAX_ALLOWED) + +// pthread_setname_np is XCode 10.6-and-later +#if __MAC_OS_X_VERSION_MAX_ALLOWED >= 1060 + pthread_setname_np(name); +#endif + +#else + // Prevent warnings for unused parameters... + (void)name; +#endif +} + +void SetupEnvironment() +{ +#ifndef WIN32 + try + { +#if BOOST_FILESYSTEM_VERSION == 3 + boost::filesystem::path::codecvt(); // Raises runtime error if current locale is invalid +#else // boost filesystem v2 + std::locale(); // Raises runtime error if current locale is invalid +#endif + } catch(std::runtime_error &e) + { + setenv("LC_ALL", "C", 1); // Force C locale + } +#endif +} + +void SetThreadPriority(int nPriority) +{ +#ifdef WIN32 + SetThreadPriority(GetCurrentThread(), nPriority); +#else // WIN32 +#ifdef PRIO_THREAD + setpriority(PRIO_THREAD, 0, nPriority); +#else // PRIO_THREAD + setpriority(PRIO_PROCESS, 0, nPriority); +#endif // PRIO_THREAD +#endif // WIN32 +} diff --git a/src/utils/util.h b/src/utils/util.h new file mode 100644 index 00000000..f3e89df9 --- /dev/null +++ b/src/utils/util.h @@ -0,0 +1,234 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +/** + * Server/client environment: argument handling, config file parsing, + * logging, thread wrappers + */ +#ifndef BITCOIN_UTIL_H +#define BITCOIN_UTIL_H + +#if defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" +#endif + +#include "utils/compat.h" +#include "utils/tinyformat.h" +#include "utils/utiltime.h" + +#include +#include +#include +#include +#include + +#include +#include + +extern std::map mapArgs; +extern std::map > mapMultiArgs; +extern bool fDebug; +extern bool fPrintToConsole; +extern bool fPrintToDebugLog; +extern bool fServer; +extern std::string strMiscWarning; +extern bool fLogTimestamps; +extern bool fLogIPs; +extern bool fLogTimeMillis; +extern volatile bool fReopenDebugLog; + +void SetupEnvironment(); + +/** Return true if log accepts specified category */ +bool LogAcceptCategory(const char* category); +/** Send a string to the log output */ +int LogPrintStr(const std::string &str); + +#define LogPrintf(...) LogPrint(NULL, __VA_ARGS__) + +/** + * When we switch to C++11, this can be switched to variadic templates instead + * of this macro-based construction (see tinyformat.h). + */ +#define MAKE_ERROR_AND_LOG_FUNC(n) \ + /** Print to debug.log if -debug=category switch is given OR category is NULL. */ \ + template \ + static inline int LogPrint(const char* category, const char* format, TINYFORMAT_VARARGS(n)) \ + { \ + if(!LogAcceptCategory(category)) return 0; \ + return LogPrintStr(tfm::format(format, TINYFORMAT_PASSARGS(n))); \ + } \ + /** Log error and return false */ \ + template \ + static inline bool error(const char* format, TINYFORMAT_VARARGS(n)) \ + { \ + LogPrintStr("ERROR: " + tfm::format(format, TINYFORMAT_PASSARGS(n)) + "\n"); \ + return false; \ + } + +TINYFORMAT_FOREACH_ARGNUM(MAKE_ERROR_AND_LOG_FUNC) + +/** + * Zero-arg versions of logging and error, these are not covered by + * TINYFORMAT_FOREACH_ARGNUM + */ +static inline int LogPrint(const char* category, const char* format) +{ + if(!LogAcceptCategory(category)) return 0; + return LogPrintStr(format); +} +static inline bool error(const char* format) +{ + LogPrintStr(std::string("ERROR: ") + format + "\n"); + return false; +} + +void PrintExceptionContinue(std::exception* pex, const char* pszThread); +void ParseParameters(int argc, const char*const argv[]); +void FileCommit(FILE *fileout); +bool TruncateFile(FILE *file, unsigned int length); +int RaiseFileDescriptorLimit(int nMinFD); +void AllocateFileRange(FILE *file, unsigned int offset, unsigned int length); +bool RenameOver(boost::filesystem::path src, boost::filesystem::path dest); +bool TryCreateDirectory(const boost::filesystem::path& p); +boost::filesystem::path GetDefaultDataDir(); +const boost::filesystem::path &GetDataDir(bool fNetSpecific = true); +boost::filesystem::path GetConfigFile(); +#ifndef WIN32 +boost::filesystem::path GetPidFile(); +void CreatePidFile(const boost::filesystem::path &path, pid_t pid); +#endif +void ReadConfigFile(std::map& mapSettingsRet, std::map >& mapMultiSettingsRet); +#ifdef WIN32 +boost::filesystem::path GetSpecialFolderPath(int nFolder, bool fCreate = true); +#endif +boost::filesystem::path GetTempPath(); +void ShrinkDebugFile(); +void runCommand(std::string strCommand); + +inline bool IsSwitchChar(char c) +{ +#ifdef WIN32 + return c == '-' || c == '/'; +#else + return c == '-'; +#endif +} + +/** + * Return string argument or default value + * + * @param strArg Argument to get (e.g. "-foo") + * @param default (e.g. "1") + * @return command-line argument or default value + */ +std::string GetArg(const std::string& strArg, const std::string& strDefault); + +/** + * Return integer argument or default value + * + * @param strArg Argument to get (e.g. "-foo") + * @param default (e.g. 1) + * @return command-line argument (0 if invalid number) or default value + */ +int64_t GetArg(const std::string& strArg, int64_t nDefault); + +/** + * Return boolean argument or default value + * + * @param strArg Argument to get (e.g. "-foo") + * @param default (true or false) + * @return command-line argument or default value + */ +bool GetBoolArg(const std::string& strArg, bool fDefault); + +/** + * Set an argument if it doesn't already have a value + * + * @param strArg Argument to set (e.g. "-foo") + * @param strValue Value (e.g. "1") + * @return true if argument gets set, false if it already had a value + */ +bool SoftSetArg(const std::string& strArg, const std::string& strValue); + +/** + * Set a boolean argument if it doesn't already have a value + * + * @param strArg Argument to set (e.g. "-foo") + * @param fValue Value (e.g. false) + * @return true if argument gets set, false if it already had a value + */ +bool SoftSetBoolArg(const std::string& strArg, bool fValue); + +void SetThreadPriority(int nPriority); +void RenameThread(const char* name); + +/** + * Standard wrapper for do-something-forever thread functions. + * "Forever" really means until the thread is interrupted. + * Use it like: + * new boost::thread(boost::bind(&LoopForever, "dumpaddr", &DumpAddresses, 900000)); + * or maybe: + * boost::function f = boost::bind(&FunctionWithArg, argument); + * threadGroup.create_thread(boost::bind(&LoopForever >, "nothing", f, milliseconds)); + */ +template void LoopForever(const char* name, Callable func, int64_t msecs) +{ + std::string s = strprintf("bitcoin-%s", name); + RenameThread(s.c_str()); + LogPrintf("%s thread start\n", name); + try + { + while (1) + { + MilliSleep(msecs); + func(); + } + } + catch (boost::thread_interrupted) + { + LogPrintf("%s thread stop\n", name); + throw; + } + catch (std::exception& e) { + PrintExceptionContinue(&e, name); + throw; + } + catch (...) { + PrintExceptionContinue(NULL, name); + throw; + } +} + +/** + * .. and a wrapper that just calls func once + */ +template void TraceThread(const char* name, Callable func) +{ + std::string s = strprintf("bitcoin-%s", name); + RenameThread(s.c_str()); + try + { + LogPrintf("%s thread start\n", name); + func(); + LogPrintf("%s thread exit\n", name); + } + catch (boost::thread_interrupted) + { + LogPrintf("%s thread interrupt\n", name); + throw; + } + catch (std::exception& e) { + PrintExceptionContinue(&e, name); + throw; + } + catch (...) { + PrintExceptionContinue(NULL, name); + throw; + } +} + +#endif // BITCOIN_UTIL_H diff --git a/src/utils/utility.cpp b/src/utils/utility.cpp new file mode 100644 index 00000000..e520383d --- /dev/null +++ b/src/utils/utility.cpp @@ -0,0 +1,1860 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" +#ifndef WIN32 +#include +#include +#include +#endif + +#define MC_DCT_BUF_ALLOC_ITEMS 256 +#define MC_DCT_LIST_ALLOC_MIN_SIZE 32768 +#define MC_DCT_LIST_ALLOC_MAX_SIZE 268435456 + + +int c_IsHexNumeric[256]={ + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 1,2,3,4,5,6,7,8,9,10,0,0,0,0,0,0, + 0,11,12,13,14,15,16,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,11,12,13,14,15,16,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0}; + +int mc_AllocSize(int items,int chunk_size,int item_size) +{ + if(items<=0) + { + return 0; + } + return ((items-1)/chunk_size+1)*chunk_size*item_size; +} + +void *mc_New(int Size) +{ + int TSize; + int64_t *ptr; + + TSize=(Size-1)/sizeof(int64_t)+1; + ptr=new int64_t[(size_t)TSize]; + + if(ptr) + { + memset(ptr,0,(size_t)Size); + } + + return ptr; +} + + +void mc_Delete(void *ptr) +{ + delete [] (int64_t*)ptr; +} + +void mc_PutLE(void *dest,void *src,int dest_size) +{ + memcpy(dest,src,dest_size); // Assuming all systems are little endian +} + +int64_t mc_GetLE(void *src,int size) +{ + int64_t result; + unsigned char *ptr; + unsigned char *ptrEnd; + int shift; + + ptr=(unsigned char*)src; + ptrEnd=ptr+size; + + result=0; + shift=0; + while(ptr=0x20) && (*dptr<=0x7F)) + { + printf("%c",*dptr); + } + else + { + printf("."); + } + } + dptr++; + k++; + } + printf("\n"); + } +} + +void mc_MemoryDumpChar(const void *ptr, + int from, + int len) +{ + mc_MemoryDumpCharSize(ptr,from,len,16); +} + + +void mc_DumpSize(const char * message,const void *ptr,int size,int row_size) +{ + printf("%s\n",message); + mc_MemoryDumpCharSize(ptr,0,size,row_size); + printf("\n"); +} + +void mc_Dump(const char * message,const void *ptr,int size) +{ + mc_DumpSize(message,ptr,size,16); +} + +void mc_RandomSeed(unsigned int seed) +{ + srand(seed); +} + +unsigned int mc_RandomInRange(unsigned int min,unsigned int max) +{ + double scaled = (double)rand()/RAND_MAX; + unsigned int result; + + result=(unsigned int)((max-min+1)*scaled) + min; + if(result>max) + { + result=max; + } + return result; +} + +unsigned int mc_TimeNowAsUInt() +{ + struct timeval time_now; + + gettimeofday(&time_now,NULL); + + return time_now.tv_sec; +} + +double mc_TimeNowAsDouble() +{ + struct timeval time_now; + + gettimeofday(&time_now,NULL); + + return (double)(time_now.tv_sec)+0.000001f*((double)(time_now.tv_usec)); +} + + +void mc_Buffer::Zero() +{ + m_lpData=NULL; + m_lpIndex=NULL; + m_AllocSize=0; + m_Size=0; + m_KeySize=0; + m_RowSize=0; + m_Count=0; + m_Mode=0; +} + +int mc_Buffer::Destroy() +{ + if(m_lpIndex) + { + delete m_lpIndex; + } + if(m_lpData) + { + mc_Delete(m_lpData); + } + + Zero(); + + return MC_ERR_NOERROR; +} + +int mc_Buffer::Initialize(int KeySize,int RowSize,uint32_t Mode) +{ + int err; + + err=MC_ERR_NOERROR; + + Destroy(); + + m_Mode=Mode; + m_KeySize=KeySize; + m_RowSize=RowSize; + + if(m_Mode & MC_BUF_MODE_MAP) + { + m_lpIndex=new mc_MapStringIndex; + } + + + m_AllocSize=mc_AllocSize(1,MC_DCT_BUF_ALLOC_ITEMS,m_RowSize); + + m_lpData=(unsigned char*)mc_New(m_AllocSize); + if(m_lpData==NULL) + { + Zero(); + err=MC_ERR_ALLOCATION; + return err; + } + + return err; +} + +int mc_Buffer::Clear() +{ + m_Size=0; + m_Count=0; + + if(m_lpIndex) + { + m_lpIndex->Clear(); + } + + + return MC_ERR_NOERROR; +} + +int mc_Buffer::Realloc(int Rows) +{ + unsigned char *lpNewBuffer; + int NewSize; + int err; + + err=MC_ERR_NOERROR; + + if(m_Size+m_RowSize*Rows>m_AllocSize) + { + NewSize=mc_AllocSize(m_Count+Rows,MC_DCT_BUF_ALLOC_ITEMS,m_RowSize); + lpNewBuffer=(unsigned char*)mc_New(NewSize); + + if(lpNewBuffer==NULL) + { + err=MC_ERR_ALLOCATION; + return err; + } + + memcpy(lpNewBuffer,m_lpData,m_AllocSize); + mc_Delete(m_lpData); + + m_AllocSize=NewSize; + m_lpData=lpNewBuffer; + } + + return err; +} + +int mc_Buffer::Add(const void *lpKey,const void *lpValue) +{ + int err; + + err=Realloc(1); + if(err) + { + return err; + } + + if(m_KeySize) + { + memcpy(m_lpData+m_Size,lpKey,m_KeySize); + } + + if(m_KeySizeAdd((unsigned char*)lpKey,m_KeySize,m_Count); + } + + m_Count++; + + return err; +} + +int mc_Buffer::Add(const void *lpKeyValue) +{ + return Add(lpKeyValue,(unsigned char*)lpKeyValue+m_KeySize); +} + +int mc_Buffer::PutRow(int RowID,const void *lpKey,const void *lpValue) +{ + unsigned char *ptr; + + if(RowID>=m_Count) + { + return MC_ERR_INTERNAL_ERROR; + } + + ptr=m_lpData+m_RowSize*RowID; + + if(m_KeySize) + { + memcpy(ptr,lpKey,m_KeySize); + } + + if(m_KeySizeAdd((unsigned char*)lpKey,m_KeySize,RowID); + } + + return MC_ERR_NOERROR; +} + + +int mc_Buffer::Seek(void *lpKey) +{ + unsigned char *ptr; + int row; + + if(m_lpIndex) + { + row=m_lpIndex->Get((unsigned char*)lpKey,m_KeySize); + if(row >= 0) + { + ptr=GetRow(row); + if(memcmp(ptr,lpKey,m_KeySize)==0) + { + return row; + } + } + return -1; + } + + ptr=m_lpData; + row=0; + + while(rowcount) + { + if(m_lpIndex) + { + m_lpIndex->Clear(); + m_Count=0; + m_Size=0; + for(i=0;iGetCount();i++) + { + ptr=source->GetRow(i); + Add(ptr,ptr+m_KeySize); + } +} + +int mc_Buffer::Sort() +{ + if(m_lpIndex) + { + return MC_ERR_NOT_SUPPORTED; + } + + if(m_Count <= 1) + { + return MC_ERR_NOERROR; + } + + int i,j,t; + int err; + + err=Realloc(1); + if(err) + { + return err; + } + + t=m_AllocSize/m_RowSize-1; + + for(i=0;i=0;j--) + { + if(memcmp(GetRow(j),GetRow(j+1),m_RowSize) > 0) + { + memcpy(GetRow(t),GetRow(j+1),m_RowSize); + memcpy(GetRow(j+1),GetRow(j),m_RowSize); + memcpy(GetRow(j),GetRow(t),m_RowSize); + } + } + } + + return MC_ERR_NOERROR; +} + + + + +void mc_List::Zero() +{ + m_lpData=NULL; + m_AllocSize=0; + m_Size=0; + m_Pos=0; + m_ItemSize=0; +} + + +int mc_List::Destroy() +{ + if(m_lpData) + { + mc_Delete(m_lpData); + } + + Zero(); + + return MC_ERR_NOERROR; +} + +void mc_List::Clear() +{ + m_Size=0; + m_Pos=0; + m_ItemSize=0; +} + +int mc_List::Put(unsigned char *ptr, int size) +{ + unsigned char *NewBuffer; + int NewSize; + int true_size; + + true_size=size; + if(size<0) + { + true_size=0; + } + if(ptr == NULL) + { + true_size=0; + } + + while(m_Size+true_size+(int)sizeof(int)>m_AllocSize) + { + if(m_AllocSize>0) + { + NewSize=m_AllocSize*2; + if(NewSize> MC_DCT_LIST_ALLOC_MAX_SIZE) + { + return MC_ERR_ALLOCATION; + } + } + else + { + NewSize=MC_DCT_LIST_ALLOC_MIN_SIZE; + } + + NewBuffer=(unsigned char*)mc_New(NewSize); + if(NewBuffer == NULL) + { + return MC_ERR_ALLOCATION; + } + else + { + if(m_lpData) + { + if(m_Size) + { + memcpy(NewBuffer,m_lpData,m_Size); + } + mc_Delete(m_lpData); + } + m_lpData=NULL; + } + + m_AllocSize=NewSize; + m_lpData=NewBuffer; + } + + *(int *)(m_lpData+m_Size)=true_size; + m_Size+=sizeof(int); + if(true_size) + { + memcpy(m_lpData+m_Size,ptr,true_size); + } + m_Size+=true_size; + + return MC_ERR_NOERROR; +} + +unsigned char *mc_List::First() +{ + m_Pos=0; + return Next(); +} + +unsigned char *mc_List::Next() +{ + unsigned char *ptr; + if(m_Pos>=m_Size) + { + m_ItemSize=0; + return NULL; + } + + m_ItemSize=*(int *)(m_lpData+m_Pos); + m_Pos+=sizeof(int); + ptr=m_lpData+m_Pos; + + m_Pos+=m_ItemSize; + return ptr; +} + + + +int mc_VarIntSize(unsigned char byte) +{ + if(byte<0xfd)return 0; + if(byte==0xfd)return 2; + if(byte==0xfe)return 4; + return 8; +} + +int64_t mc_GetVarInt(const unsigned char *buf,int max_size,int64_t default_value,int* shift) +{ + int size; + if(max_size<=0) + { + return default_value; + } + + size=mc_VarIntSize(buf[0]); + + if(max_size < size+1) + { + return default_value; + } + + if(shift) + { + *shift=size+1; + } + + if(size == 0) + { + return buf[0]; + } + + return mc_GetLE((void*)(buf+1),size); +} + +int mc_PutVarInt(unsigned char *buf,int max_size,int64_t value) +{ + int varint_size,shift; + + if(max_size<=0) + { + return -1; + } + + varint_size=1; + shift=0; + if(value>=0xfd) + { + shift=1; + if(value>=0xffff) + { + if(value>=0xffffffff) + { + buf[0]=0xff; + varint_size=8; + } + else + { + buf[0]=0xfe; + varint_size=4; + } + } + else + { + buf[0]=0xfd; + varint_size=2; + } + } + + if(max_size < shift+varint_size) + { + return -1; + } + + mc_PutLE(buf+shift,&value,varint_size); + return shift+varint_size; +} + +#ifndef WIN32 + +static struct termios oldtc , newtc; + +/* Initialize new terminal i/o settings */ +void initTermios(int echo) +{ + tcgetattr(0, &oldtc); /* grab old terminal i/o settings */ + + newtc = oldtc; /* make new settings same as old settings */ + + newtc.c_lflag &= ~ICANON; /* disable buffered i/o */ + newtc.c_lflag &= echo ? ECHO : ~ECHO; /* set echo mode */ + + tcsetattr(0, TCSANOW, &newtc); /* use these new terminal i/o settings now */ +} + +/* Restore old terminal i/o settings */ +void resetTermios(void) +{ + tcsetattr(0, TCSANOW, &oldtc); +} + +/* Read 1 character - echo defines echo mode */ +char getch_(int echo) +{ + char ch; +// initTermios(echo); + ch = getchar(); +// resetTermios(); + return ch; +} + +/* Read 1 character without echo */ +char getch(void) +{ + return getch_(0); +} + +int mc_TerminalInput::LoadDataFromLog(const char* fileName) +{ + int fHan; + char *raw; + + int64_t offset,size; + int err; + int start,pos; + + fHan=open(fileName,_O_BINARY | O_RDONLY); + if(fHan<0) + { + return MC_ERR_FILE_READ_ERROR; + } + raw=(char*)mc_New(MC_DCT_TERM_BUFFER_SIZE); + err=MC_ERR_NOERROR; + + size=lseek64(fHan,0,SEEK_END); + offset=size-m_BufferSize; + if(offset<0) + { + offset=0; + } + + if(lseek64(fHan,offset,SEEK_SET) != offset) + { + err=MC_ERR_FILE_READ_ERROR; + goto exitlbl; + } + + if(size>m_BufferSize) + { + size=m_BufferSize; + } + + if(read(fHan,raw,size) != size) + { + err=MC_ERR_FILE_READ_ERROR; + goto exitlbl; + } + + pos=0; + if(size==m_BufferSize) + { + while( (pos= 0x30) && (c <= 0x39) ) + { + return 1; + } + if( (c >= 0x41) && (c <= 0x5a) ) + { + return 1; + } + if( (c >= 0x61) && (c <= 0x7a) ) + { + return 1; + } + return 0; +} + +int mc_TerminalInput::SetPrompt(const char* prompt) +{ + strcpy(m_Prompt,prompt); + return MC_ERR_NOERROR; +} + +int mc_TerminalInput::Prompt() +{ + printf("%s: ",m_Prompt); + fflush(stdout); + return MC_ERR_NOERROR; +} + +int mc_TerminalInput::TerminalCols() +{ + struct winsize max; + ioctl(0, TIOCGWINSZ , &max); + m_TerminalCols=max.ws_col; + m_TerminalRows=max.ws_row; + return m_TerminalCols; +} + +void mc_TerminalInput::MoveBack(int offset) +{ + int p,q,i; + p=(offset+strlen(m_Prompt)+2)%m_TerminalCols; + if(p==0) + { + q=offset-m_TerminalCols; + printf("%c[A",0x1b); + if(q<0) + { + while(q<0) + { + printf(" "); + q++; + } + q+=strlen(m_Prompt)+2; + for(i=0;im_FirstLine) + { + pos=(m_ThisLine-1)%m_HistoryLines; + dataend=m_Offsets[pos]+strlen(m_Data+m_Offsets[pos])+1; + } + if(dataend+len>m_BufferSize) + { + dataend=0; + } + + while( (m_FirstLine=dataend) && (m_Offsets[m_FirstLine%m_HistoryLines]m_ThisLine) + { + return -1; + } + len=strlen(m_Line); + if(line==m_ThisLine) + { + strcpy(m_Line,m_Cache); + } + else + { + strcpy(m_Line,m_Data+m_Offsets[line%m_HistoryLines]); + } + m_LoadedLine=line; + return len; +} + + +char *mc_TerminalInput::GetLine() +{ + char c,l; + int offset,arrow_mode,arrow_prefix,len,i,cols,pos,oldcol,oldrow,totrow; + initTermios(0); + memset(m_Line,0,m_BufferSize); + offset=0; + len=0; + arrow_mode=0; + arrow_prefix=0; + m_LoadedLine=m_ThisLine; + cols=TerminalCols(); + + c=getch(); + while(c != '\n') + { + if(TerminalCols() != cols) + { + oldcol=(offset+strlen(m_Prompt)+2-1) % cols + 1; + oldrow=(offset+strlen(m_Prompt)+2-1) / cols; + totrow=(len+strlen(m_Prompt)+2-1) / cols; + for(i=0;ioffset;i--) + { + MoveBack(i); + } + fflush(stdout); + cols=TerminalCols(); + } + + switch(arrow_mode) + { + case 0: + if(((c >= 0x20) && (c<=0x7e)) || (c==0x09) ) + { + if(c == 0x09) + { + c=0x20; + } + if(offsetoffset;i--) + { + MoveBack(i); + } + fflush(stdout); + c=0; + } + } + if(c == 0x7f) + { + if(offset) + { + MoveBack(offset); + for(i=offset;ioffset;i--) + { + MoveBack(i); + } + MoveBack(offset); + offset--; + len--; + fflush(stdout); + c=0; + } + } + if(c == 0x01) + { + if(offset>0) + { + for(i=offset;i>0;i--) + { + MoveBack(i); + } + offset=0; + c=0; + } + } + if(c == 0x05) + { + if(offset < len) + { + for(i=offset;i0;i--) + { + MoveBack(i); + } + for(i=0;i0;i--) + { + MoveBack(i); + } + len=0; + offset=len; + c=0; + } + + if(c == 0x1b) + { + arrow_mode=1; + c=0; + } + + + break; + case 1: + arrow_mode=2; + arrow_prefix=c; + c=0; + break; + case 2: + if( arrow_prefix == 0x4f ) + { + switch(c) + { + case 0x48: + if(offset > 0) + { + for(i=offset;i>0;i--) + { + MoveBack(i); + } + offset=0; + c=0; + } + break; + case 0x46: + if(offset < len) + { + for(i=offset;i=m_FirstLine) && (m_LoadedLine-1<=m_ThisLine) ) + { + SaveLine(); + for(i=offset;i>0;i--) + { + MoveBack(i); + } + for(i=0;i0;i--) + { + MoveBack(i); + } + LoadLine(m_LoadedLine-1); + len=strlen(m_Line); + offset=len; + for(i=0;i=m_FirstLine) && (m_LoadedLine+1<=m_ThisLine) ) + { + SaveLine(); + for(i=offset;i>0;i--) + { + MoveBack(i); + } + for(i=0;i0;i--) + { + MoveBack(i); + } + LoadLine(m_LoadedLine+1); + len=strlen(m_Line); + offset=len; + for(i=0;i 0) ) + { + pos++; + } + if(pos != offset) + { + for(i=offset;i=0) && (IsAlphaNumeric(m_Line[pos]) == 0) ) + { + pos--; + } + while( (pos>=0) && (IsAlphaNumeric(m_Line[pos]) > 0) ) + { + pos--; + } + pos++; + if(pos <= (offset-1)) + { + for(i=offset;i>pos;i--) + { + MoveBack(i); + } + fflush(stdout); + offset=pos; + c=0; + } + } + arrow_mode=0; + break; + } + if(c) + { + printf("%c",0x07); + fflush(stdout); + } + c=getch(); + } + + m_Line[len]=0; + + if(len) + { + if((m_FirstLine>=m_ThisLine) || (strcmp(m_Data+m_Offsets[(m_ThisLine-1)%m_HistoryLines],m_Line) != 0)) + { + if((strcmp(m_Line,"exit")==0) || + (strcmp(m_Line,"quit")==0) || + (strcmp(m_Line,"bye")==0)) + { + ; + } + else + { + AddLine(); + m_ThisLine++; + } + } + } + + printf("%c[H",0x1b); + for(i=0;itm_year, + bdt->tm_mon+1, + bdt->tm_mday, + bdt->tm_hour, + bdt->tm_min, + bdt->tm_sec, + (int32_t)(time_now.tv_usec/1000)); + + + p=0; + for(a=1;a 0) + { + if(c) + { + fprintf(fHan,"%c",'\''); + } + for(i=0;i<(int)strlen(argv[a]);i++) + { + if(argv[a][i] == '\'') + { + fprintf(fHan,"%c%c%c%c",'\'','\\','\'','\''); + } + else + { + fprintf(fHan,"%c",argv[a][i]); + } + } + if(c) + { + fprintf(fHan,"%c",'\''); + } + if(a0x40&&*ptr<0x5b) + { + *ptr=*ptr+0x20; + } + ptr++; + } +} + +int mc_StringCompareCaseInsensitive(const char *str1,const char *str2, int len) +{ + int i,res; + + res=0; + + for(i=0;i0x60&&str1[i]<0x7b) + if(str1[i]-0x20==str2[i]) + res=0; + if(res) + { + if(str2[i]>0x60&&str2[i]<0x7b) + if(str2[i]-0x20==str1[i]) + res=0; + } + } + } + + return res; +} + +void mc_LogString(FILE *fHan, const char* message) +{ + struct tm *bdt; + +#ifndef WIN32 + struct timeval time_now; + gettimeofday(&time_now,NULL); + bdt=localtime(&(time_now.tv_sec)); + fprintf(fHan,"%04d-%02d-%02d %02d:%02d:%02d.%03d\t%s\n",1900+bdt->tm_year, + bdt->tm_mon+1, + bdt->tm_mday, + bdt->tm_hour, + bdt->tm_min, + bdt->tm_sec, + (int32_t)(time_now.tv_usec/1000), + message); +#else + time_t dt; + struct tm dc; + time(&dt); + dc=*localtime(&dt); + + fprintf(fHan,"%04d-%02d-%02d %02d:%02d:%02d\t%s\n",1900+dc.tm_year, + dc.tm_mon+1, + dc.tm_mday, + dc.tm_hour, + dc.tm_min, + dc.tm_sec, + message); +#endif + +} + +void mc_AdjustStartAndCount(int *count,int *start,int size) +{ + if(*count>size) + { + *count=size; + } + if(*start<0) + { + *start+=size; + if(*start<0) + { + *start=0; + } + } + + if(*start > 0) + { + if(*start+*count>size) + { + *count=size-*start; + } + } +} + diff --git a/src/utils/utilmoneystr.cpp b/src/utils/utilmoneystr.cpp new file mode 100644 index 00000000..964b8cb3 --- /dev/null +++ b/src/utils/utilmoneystr.cpp @@ -0,0 +1,92 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "utilmoneystr.h" + +#include "primitives/transaction.h" +#include "utils/tinyformat.h" +#include "utils/utilstrencodings.h" + +using namespace std; + +string FormatMoney(const CAmount& n, bool fPlus) +{ + // Note: not using straight sprintf here because we do NOT want + // localized number formatting. + int64_t n_abs = (n > 0 ? n : -n); +/* MCHN START */ + int64_t quotient = n_abs; + int64_t remainder = n_abs; + if(COIN > 0) + { + quotient=n_abs/COIN; + remainder = n_abs%COIN; + } + +// int64_t quotient = n_abs/COIN; +// int64_t remainder = n_abs%COIN; +/* MCHN END */ + string str = strprintf("%d.%08d", quotient, remainder); + + // Right-trim excess zeros before the decimal point: + int nTrim = 0; + for (int i = str.size()-1; (str[i] == '0' && isdigit(str[i-2])); --i) + ++nTrim; + if (nTrim) + str.erase(str.size()-nTrim, nTrim); + + if (n < 0) + str.insert((unsigned int)0, 1, '-'); + else if (fPlus && n > 0) + str.insert((unsigned int)0, 1, '+'); + return str; +} + + +bool ParseMoney(const string& str, CAmount& nRet) +{ + return ParseMoney(str.c_str(), nRet); +} + +bool ParseMoney(const char* pszIn, CAmount& nRet) +{ + string strWhole; + int64_t nUnits = 0; + const char* p = pszIn; + while (isspace(*p)) + p++; + for (; *p; p++) + { + if (*p == '.') + { + p++; + int64_t nMult = CENT*10; + while (isdigit(*p) && (nMult > 0)) + { + nUnits += nMult * (*p++ - '0'); + nMult /= 10; + } + break; + } + if (isspace(*p)) + break; + if (!isdigit(*p)) + return false; + strWhole.insert(strWhole.end(), *p); + } + for (; *p; p++) + if (!isspace(*p)) + return false; + if (strWhole.size() > 10) // guard against 63 bit overflow + return false; + if (nUnits < 0 || nUnits > COIN) + return false; + int64_t nWhole = atoi64(strWhole); + CAmount nValue = nWhole*COIN + nUnits; + + nRet = nValue; + return true; +} diff --git a/src/utils/utilmoneystr.h b/src/utils/utilmoneystr.h new file mode 100644 index 00000000..21b8caa7 --- /dev/null +++ b/src/utils/utilmoneystr.h @@ -0,0 +1,22 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +/** + * Money parsing/formatting utilities. + */ +#ifndef BITCOIN_UTILMONEYSTR_H +#define BITCOIN_UTILMONEYSTR_H + +#include +#include + +#include "structs/amount.h" + +std::string FormatMoney(const CAmount& n, bool fPlus=false); +bool ParseMoney(const std::string& str, CAmount& nRet); +bool ParseMoney(const char* pszIn, CAmount& nRet); + +#endif // BITCOIN_UTILMONEYSTR_H diff --git a/src/utils/utilparse.cpp b/src/utils/utilparse.cpp new file mode 100644 index 00000000..f8d3a7d5 --- /dev/null +++ b/src/utils/utilparse.cpp @@ -0,0 +1,881 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "utils/utilparse.h" +#include "util.h" + +using namespace std; + +const unsigned char* GetAddressIDPtr(const CTxDestination& address) +{ + const CKeyID *lpKeyID=boost::get (&address); + const CScriptID *lpScriptID=boost::get (&address); + unsigned char *aptr; + aptr=NULL; + if(lpKeyID) + { + aptr=(unsigned char*)(lpKeyID); + } + else + { + if(lpScriptID) + { + aptr=(unsigned char*)(lpScriptID); + } + } + + return aptr; +} + +/* + * Parses txout script into asset-quantity buffer + * Use it only with unspent or not yet created outputs + */ + + +bool ParseMultichainTxOutToBuffer(uint256 hash, // IN, tx hash, if !=0 genesis asset reference is retrieved from asset DB + const CTxOut& txout, // IN, tx to be parsed + mc_Buffer *amounts, // OUT, output amount buffer + mc_Script *lpScript, // TMP, temporary script object + int *allowed, // IN/OUT/NULL returns permissions of output address ANDed with input value + int *required, // IN/OUT/NULL returns permission required by this output, adds special rows to output buffer according to input value + map* mapSpecialEntity, + string& strFailReason) // OUT error +{ + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + int err,row,disallow_if_assets_found; + int64_t quantity,total,last; + int expected_allowed,expected_required; + uint32_t type,from,to,timestamp,type_ored; + bool issue_found; + uint32_t new_entity_type; + + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + + expected_allowed=0; + expected_required=0; + + strFailReason=""; + + if(allowed) + { + expected_allowed=*allowed; + *allowed=MC_PTP_ALL; + } + if(required) + { + expected_required=*required; + *required=0; + } + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + disallow_if_assets_found=0; + const CScript& script1 = txout.scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + lpScript->Clear(); + lpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + if(allowed) // Checking permissions this output address have + { + *allowed=0; + CTxDestination addressRet; + if(ExtractDestinationScriptValid(script1, addressRet)) + { + const unsigned char *aptr; + + aptr=GetAddressIDPtr(addressRet); + if(aptr) + { + if(expected_allowed & MC_PTP_SEND) + { + if(mc_gState->m_Permissions->CanSend(NULL,aptr)) + { + if(mc_gState->m_Permissions->CanReceive(NULL,aptr)) + { + *allowed |= MC_PTP_SEND; + } + else + { + if(mc_gState->m_Features->AnyoneCanReceiveEmpty()) + { + if(txout.nValue == 0) + { + disallow_if_assets_found=1; + *allowed |= MC_PTP_SEND; + } + } + } + } + } + if(expected_allowed & MC_PTP_WRITE) + { + unsigned char *lpEntity=NULL; + if(mapSpecialEntity) + { + std::map::const_iterator it = mapSpecialEntity->find(MC_PTP_WRITE); + if (it != mapSpecialEntity->end()) + { + lpEntity=(unsigned char*)(&(it->second)); + } + } + + if(mc_gState->m_Permissions->CanWrite(lpEntity,aptr)) + { + *allowed |= MC_PTP_WRITE; + } + } + if(expected_allowed & MC_PTP_ISSUE) + { + unsigned char *lpEntity=NULL; + + if(mapSpecialEntity) + { + std::map::const_iterator it = mapSpecialEntity->find(MC_PTP_ISSUE); + if (it != mapSpecialEntity->end()) + { + lpEntity=(unsigned char*)(&(it->second)); + } + } + + if(mc_gState->m_Permissions->CanIssue(lpEntity,aptr)) + { + *allowed |= MC_PTP_ISSUE; + } + } + if(expected_allowed & MC_PTP_CREATE) + { + if(mc_gState->m_Permissions->CanCreate(NULL,aptr)) + { + *allowed |= MC_PTP_CREATE; + } + } + if(expected_allowed & MC_PTP_ADMIN) + { + unsigned char *lpEntity=NULL; + + if(mapSpecialEntity) + { + std::map::const_iterator it = mapSpecialEntity->find(MC_PTP_ADMIN); + if (it != mapSpecialEntity->end()) + { + lpEntity=(unsigned char*)(&(it->second)); + } + } + + if(mc_gState->m_Permissions->CanAdmin(lpEntity,aptr)) + { + *allowed |= MC_PTP_ADMIN; + } + } + if(expected_allowed & MC_PTP_ACTIVATE) + { + unsigned char *lpEntity=NULL; + + if(mapSpecialEntity) + { + std::map::const_iterator it = mapSpecialEntity->find(MC_PTP_ACTIVATE); + if (it != mapSpecialEntity->end()) + { + lpEntity=(unsigned char*)(&(it->second)); + } + } + + if(mc_gState->m_Permissions->CanActivate(lpEntity,aptr)) + { + *allowed |= MC_PTP_ACTIVATE; + } + } + } + } + } + + issue_found=false; + total=0; + for (int e = 0; e < lpScript->GetNumElements(); e++) + { + lpScript->SetElement(e); + err=lpScript->GetAssetGenesis(&quantity); + if(err == 0) + { + issue_found=true; + if(quantity+total<0) + { + strFailReason="Invalid asset genesis script"; + return false; + } + + total+=quantity; + } + else + { + if(err != MC_ERR_WRONG_SCRIPT) + { + strFailReason="Invalid asset genesis script"; + return false; + } + } + } + + if(issue_found) + { + if(mc_gState->m_Features->FollowOnIssues() == 0) + { + if(total == 0) + { + issue_found=false; + } + } + } + + if(issue_found) + { + if(hash != 0) + { + mc_EntityDetails entity; + if(mc_gState->m_Assets->FindEntityByTxID(&entity,(unsigned char*)&hash)) + { + if(disallow_if_assets_found) + { + disallow_if_assets_found=0; + *allowed -= MC_PTP_SEND; + } + + if((mc_gState->m_Features->ShortTxIDAsAssetRef() == 0) && (entity.IsUnconfirmedGenesis() != 0) ) + { + if(required) // Unconfirmed genesis in protocol < 10007, cannot be spent + { + memset(buf,0,MC_AST_ASSET_FULLREF_SIZE); + mc_SetABRefType(buf,MC_AST_ASSET_REF_TYPE_GENESIS); + mc_SetABQuantity(buf,total); + amounts->Add(buf); + *required |= MC_PTP_ISSUE; + } + } + else + { + memcpy(buf,entity.GetFullRef(),MC_AST_ASSET_FULLREF_SIZE); + row=amounts->Seek(buf); + last=0; + if(row >= 0) + { + last=mc_GetABQuantity(amounts->GetRow(row)); + total+=last; + mc_SetABQuantity(amounts->GetRow(row),total); + } + else + { + mc_SetABQuantity(buf,total); + amounts->Add(buf); + } + + if(required) + { + if(expected_required == 0) + { + *required |= MC_PTP_ISSUE; + } + } + } + } + else // Asset not found, no error but the caller should check required field + { + if(required) + { + *required |= MC_PTP_ISSUE; + } + } + } + else // New unconfirmed genesis + { + memset(buf,0,MC_AST_ASSET_FULLREF_SIZE); + mc_SetABRefType(buf,MC_AST_ASSET_REF_TYPE_GENESIS); + mc_SetABQuantity(buf,total); + amounts->Add(buf); + if(required) + { + *required |= MC_PTP_ISSUE; + } + } + } + + if(lpScript->IsOpReturnScript() == 0) + { + for (int e = 0; e < lpScript->GetNumElements(); e++) // Parsing asset quantities + { + lpScript->SetElement(e); + err=lpScript->GetAssetQuantities(amounts,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER); // Buffer is updated, new rows are added if needed + if((err != MC_ERR_NOERROR) && (err != MC_ERR_WRONG_SCRIPT)) + { + strFailReason="Invalid asset transfer script"; + return false; + } + + if(disallow_if_assets_found) // Cannot use in coin selection as this is non-empty output without receive permission + { + if(err != MC_ERR_WRONG_SCRIPT) + { + disallow_if_assets_found=0; + *allowed -= MC_PTP_SEND; + } + } + + if(hash != 0) // Follow-ons + { + err=lpScript->GetAssetQuantities(amounts,MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON); + if((err != MC_ERR_NOERROR) && (err != MC_ERR_WRONG_SCRIPT)) + { + strFailReason="Invalid asset followon script"; + return false; + } + if(disallow_if_assets_found) + { + if(err != MC_ERR_WRONG_SCRIPT) + { + disallow_if_assets_found=0; + *allowed -= MC_PTP_SEND; + } + } + } + else + { + uint32_t script_type=0; + unsigned char ref[MC_AST_ASSET_FULLREF_SIZE]; + err=lpScript->GetFullRef(ref,&script_type); + + switch(script_type) + { + case MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON: + mc_EntityDetails entity; + if(mc_gState->m_Assets->FindEntityByFullRef(&entity,ref)) + { + if(required) + { + *required |= MC_PTP_ISSUE; + } + if(mapSpecialEntity) + { + std::map::const_iterator it = mapSpecialEntity->find(MC_PTP_ISSUE); + if (it == mapSpecialEntity->end()) + { + mapSpecialEntity->insert(make_pair(MC_PTP_ISSUE,*(uint256*)(entity.GetTxID()))); + } + else + { + if(it->second != *(uint256*)(entity.GetTxID())) + { + strFailReason="Invalid asset follow-on script, multiple assets"; + return false; + } + } + } + } + else + { + strFailReason="Invalid asset follow-on script, asset not found"; + return false; + } + + break; + } + } + + } + } + else // OP_RETURN outputs + { + if(required) + { + if(hash == 0) + { + if(lpScript->GetNumElements() == 2) // Create entity + { + lpScript->SetElement(0); + if(lpScript->GetNewEntityType(&new_entity_type) == 0) + { + if(new_entity_type == MC_ENT_TYPE_STREAM) + { + *required |= MC_PTP_CREATE; + } + } + } + if(lpScript->GetNumElements() == 3) // Publish + { + unsigned char short_txid[MC_AST_SHORT_TXID_SIZE]; + lpScript->SetElement(0); + if(lpScript->GetEntity(short_txid) == 0) + { + mc_EntityDetails entity; + if(mc_gState->m_Assets->FindEntityByShortTxID(&entity,short_txid)) + { + if(entity.GetEntityType() == MC_ENT_TYPE_STREAM) + { + if(entity.AnyoneCanWrite() == 0) + { + if(mapSpecialEntity) + { + if(required) + { + *required |= MC_PTP_WRITE; + } + std::map::const_iterator it = mapSpecialEntity->find(MC_PTP_WRITE); + if (it == mapSpecialEntity->end()) + { + mapSpecialEntity->insert(make_pair(MC_PTP_WRITE,*(uint256*)(entity.GetTxID()))); + } + else + { + if(it->second != *(uint256*)(entity.GetTxID())) + { + strFailReason="Invalid publish script, multiple streams"; + return false; + } + } + } + } + } + else + { + if(mc_gState->m_Features->OpDropDetailsScripts() == 0)// May be Follow-on details from v10007 + { + strFailReason="Invalid publish script, not stream"; + return false; + } + } + } + else + { + strFailReason="Invalid publish script, stream not found"; + return false; + } + } + else + { + strFailReason="Invalid publish script"; + return false; + } + } + } + } + } + + if(required) + { + *required |= MC_PTP_SEND; + + if(lpScript->IsOpReturnScript() == 0) + { + type_ored=0; + mc_EntityDetails entity; + uint32_t admin_type; + entity.Zero(); + for (int e = 0; e < lpScript->GetNumElements(); e++) // Parsing permissions + { + unsigned char short_txid[MC_AST_SHORT_TXID_SIZE]; + lpScript->SetElement(e); + if(lpScript->GetEntity(short_txid) == 0) // Entity element + { + if(mc_gState->m_Assets->FindEntityByShortTxID(&entity,short_txid) == 0) + { + strFailReason="Entity not found"; + return false; + } + } + + if(lpScript->GetPermission(&type,&from,&to,×tamp) == 0)// Permission script found, admin permission needed + { + if(mc_gState->m_Permissions->IsActivateEnough(type)) + { + admin_type=MC_PTP_ACTIVATE; + } + else + { + admin_type=MC_PTP_ADMIN; + } + *required |= admin_type; + if( type & (MC_PTP_ADMIN | MC_PTP_MINE) ) + { + if(mc_gState->m_Features->CachedInputScript()) + { + if(mc_gState->m_NetworkParams->GetInt64Param("supportminerprecheck")) + { + *required |= MC_PTP_CACHED_SCRIPT_REQUIRED; + } + } + } + + if(hash == 0) + { + if(entity.GetEntityType()) + { + if(mapSpecialEntity) + { + std::map::const_iterator it = mapSpecialEntity->find(admin_type); + if (it == mapSpecialEntity->end()) + { + mapSpecialEntity->insert(make_pair(admin_type,*(uint256*)(entity.GetTxID()))); + } + else + { + if(it->second != *(uint256*)(entity.GetTxID())) + { + strFailReason="Invalid permission script, multiple entities"; + return false; + } + } + } + } + } + type_ored |= type; + entity.Zero(); + } + } + + if(expected_required & MC_PTP_RECEIVE) // Checking for dust + { + if( (type_ored == 0) && (lpScript->IsOpReturnScript() == 0) ) + { + if (txout.IsDust(::minRelayTxFee)) + { + strFailReason="Transaction output value too small"; + return false; + } + } + } + } + + if( ( (expected_allowed & MC_PTP_ISSUE) && (*allowed & MC_PTP_ISSUE) ) || + ( (expected_required & MC_PTP_ISSUE) && (*required & MC_PTP_ISSUE) && (hash == 0) ) ) + { + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + type=MC_PTP_ISSUE | MC_PTP_SEND; + quantity=1; + mc_PutLE(buf+4,&type,4); + mc_SetABRefType(buf,MC_AST_ASSET_REF_TYPE_SPECIAL); + mc_SetABQuantity(buf,quantity); + if(amounts->Seek(buf) < 0) + { + amounts->Add(buf); + } + } + + if( ( (expected_allowed & MC_PTP_CREATE) && (*allowed & MC_PTP_CREATE) ) || + ( (expected_required & MC_PTP_CREATE) && (*required & MC_PTP_CREATE) && (hash == 0) ) ) + { + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + type=MC_PTP_CREATE | MC_PTP_SEND; + quantity=1; + mc_PutLE(buf+4,&type,4); + mc_SetABRefType(buf,MC_AST_ASSET_REF_TYPE_SPECIAL); + mc_SetABQuantity(buf,quantity); + if(amounts->Seek(buf) < 0) + { + amounts->Add(buf); + } + } + + if( ( (expected_allowed & MC_PTP_ADMIN) && (*allowed & MC_PTP_ADMIN) ) || + ( (expected_required & MC_PTP_ADMIN) && (*required & MC_PTP_ADMIN) && (hash == 0) ) ) + { + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + type=MC_PTP_ADMIN | MC_PTP_SEND; + quantity=1; + mc_PutLE(buf+4,&type,4); + mc_SetABRefType(buf,MC_AST_ASSET_REF_TYPE_SPECIAL); + mc_SetABQuantity(buf,quantity); + if(amounts->Seek(buf) < 0) + { + amounts->Add(buf); + } + } + + if( ( (expected_allowed & MC_PTP_ACTIVATE) && (*allowed & MC_PTP_ACTIVATE) ) || + ( (expected_required & MC_PTP_ACTIVATE) && (*required & MC_PTP_ACTIVATE) && (hash == 0) ) ) + { + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + type=MC_PTP_ACTIVATE | MC_PTP_SEND; + quantity=1; + mc_PutLE(buf+4,&type,4); + mc_SetABRefType(buf,MC_AST_ASSET_REF_TYPE_SPECIAL); + mc_SetABQuantity(buf,quantity); + if(amounts->Seek(buf) < 0) + { + amounts->Add(buf); + } + } + +/* // Not required, publish addresses are passed explicitly + if( ( (expected_allowed & MC_PTP_WRITE) && (*allowed & MC_PTP_WRITE) ) || + ( (expected_required & MC_PTP_WRITE) && (*required & MC_PTP_WRITE) && (hash == 0) ) ) + { + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + type=MC_PTP_WRITE | MC_PTP_SEND; + quantity=1; + mc_PutLE(buf+4,&type,4); + mc_PutLE(buf+MC_AST_ASSET_QUANTITY_OFFSET,&quantity,MC_AST_ASSET_QUANTITY_SIZE); + if(amounts->Seek(buf) < 0) + { + amounts->Add(buf,buf+MC_AST_ASSET_QUANTITY_OFFSET); + } + } +*/ + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + type=MC_PTP_SEND; + mc_PutLE(buf+4,&type,4); + mc_SetABRefType(buf,MC_AST_ASSET_REF_TYPE_SPECIAL); + quantity=txout.nValue; + row=amounts->Seek(buf); + if(row >= 0) + { + quantity+=mc_GetABQuantity(amounts->GetRow(row)); + mc_SetABQuantity(amounts->GetRow(row),quantity); + } + else + { + mc_SetABQuantity(buf,quantity); + amounts->Add(buf); + } + } + + } + else // Protocol != multichain + { + if(allowed) + { + *allowed=MC_PTP_SEND; + } + if(required) + { + *required |= MC_PTP_SEND; + if(expected_required & MC_PTP_RECEIVE) // Checking for dust + { + if (txout.IsDust(::minRelayTxFee)) + { + strFailReason="Transaction output value too small"; + return false; + } + } + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + type=MC_PTP_SEND; + mc_PutLE(buf+4,&type,4); + mc_SetABRefType(buf,MC_AST_ASSET_REF_TYPE_SPECIAL); + quantity=txout.nValue; + row=amounts->Seek(buf); + if(row >= 0) + { + quantity+=mc_GetABQuantity(amounts->GetRow(row)); + mc_SetABQuantity(amounts->GetRow(row),quantity); + } + else + { + mc_SetABQuantity(buf,quantity); + amounts->Add(buf); + } + } + } + + return true; +} + +bool ParseMultichainTxOutToBuffer(uint256 hash, // IN, tx hash, if !=0 genesis asset reference is retrieved from asset DB + const CTxOut& txout, // IN, tx to be parsed + mc_Buffer *amounts, // OUT, output amount buffer + mc_Script *lpScript, // TMP, temporary script object + int *allowed, // IN/OUT/NULL returns permissions of output address ANDed with input value + int *required, // IN/OUT/NULL returns permission required by this output, adds special rows to output buffer according to input value + string& strFailReason) // OUT error +{ + return ParseMultichainTxOutToBuffer(hash,txout,amounts,lpScript,allowed,required,NULL,strFailReason); +} + +bool CreateAssetBalanceList(const CTxOut& txout,mc_Buffer *amounts,mc_Script *lpScript,int *required) +{ + string strFailReason; + + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE]; + int err; + int64_t quantity,total; + bool issue_found=false; + uint32_t type,from,to,timestamp; + + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE); + + if(required) + { + *required=0; + } + + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + const CScript& script1 = txout.scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + lpScript->Clear(); + lpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + total=0; + for (int e = 0; e < lpScript->GetNumElements(); e++) + { + lpScript->SetElement(e); + err=lpScript->GetAssetGenesis(&quantity); + if(err == 0) + { + issue_found=true; + if(quantity+total<0) + { + return false; + } + + total+=quantity; + } + else + { + if(err != MC_ERR_WRONG_SCRIPT) + { + return false; + } + } + + if(required) + { + if(lpScript->GetPermission(&type,&from,&to,×tamp) == 0) + { + if(mc_gState->m_Permissions->IsActivateEnough(type)) + { + *required |= MC_PTP_ACTIVATE; + } + else + { + *required |= MC_PTP_ADMIN; + } + } + } + } + + if(issue_found) // Checking that genesis was confirmed at least once + { + if(required) + { + *required |= MC_PTP_ISSUE; + } + memset(buf,0,MC_AST_ASSET_FULLREF_SIZE); + mc_SetABRefType(buf,MC_AST_ASSET_REF_TYPE_GENESIS); + mc_SetABQuantity(buf,total); + amounts->Add(buf); + } + + for (int e = 0; e < lpScript->GetNumElements(); e++) // Parsing asset quantities + { + lpScript->SetElement(e); + err=lpScript->GetAssetQuantities(amounts,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER); // Buffer is updated, new rows are added if needed + + if((err != MC_ERR_NOERROR) && (err != MC_ERR_WRONG_SCRIPT)) + { + return false; + } + + err=lpScript->GetAssetQuantities(amounts,MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON); + + if(err == 0) + { + if(required) + { + *required |= MC_PTP_ISSUE; + } + } + if((err != MC_ERR_NOERROR) && (err != MC_ERR_WRONG_SCRIPT)) + { + return false; + } + + } + } + + return true; + + +} + +bool CreateAssetBalanceList(const CTxOut& txout,mc_Buffer *amounts,mc_Script *lpScript) +{ + return CreateAssetBalanceList(txout,amounts,lpScript,NULL); +} + +bool FindFollowOnsInScript(const CScript& script1,mc_Buffer *amounts,mc_Script *lpScript) +{ + int err; + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + CScript::const_iterator pc1 = script1.begin(); + + lpScript->Clear(); + lpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + } + + for (int e = 0; e < lpScript->GetNumElements(); e++) // Parsing asset quantities + { + lpScript->SetElement(e); + err=lpScript->GetAssetQuantities(amounts,MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON); + if((err != MC_ERR_NOERROR) && (err != MC_ERR_WRONG_SCRIPT)) + { + return false; + } + } + return true; +} + +void LogAssetTxOut(string message,uint256 hash,int index,unsigned char* assetrefbin,int64_t quantity) +{ + string txid=hash.GetHex(); + + string assetref=""; + if(assetrefbin) + { + assetref += itostr((int)mc_GetLE(assetrefbin,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(assetrefbin+4,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(assetrefbin+8,2)); + } + else + { + assetref += "0-0-2"; + } + LogPrint("mcatxo", "mcatxo: %s: %s-%d %s %ld\n",message.c_str(),txid.c_str(),index,assetref.c_str(),quantity); +} + +bool AddressCanReceive(CTxDestination address) +{ + if(mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) + { + return true; + } + CKeyID *lpKeyID=boost::get (&address); + CScriptID *lpScriptID=boost::get (&address); + if((lpKeyID == NULL) && (lpScriptID == NULL)) + { + LogPrintf("mchn: Invalid address"); + return false; + } + unsigned char* ptr=NULL; + CBitcoinAddress addressPrint; + if(lpKeyID != NULL) + { + addressPrint=CBitcoinAddress(*lpKeyID); + ptr=(unsigned char*)(lpKeyID); + } + else + { + addressPrint=CBitcoinAddress(*lpScriptID); + ptr=(unsigned char*)(lpScriptID); + } + + if(mc_gState->m_Permissions->CanReceive(NULL,ptr) == 0) + { + LogPrintf("mchn: Destination address doesn't have receive permission: %s\n", + addressPrint.ToString().c_str()); + return false; + } + + return true; +} + diff --git a/src/utils/utilparse.h b/src/utils/utilparse.h new file mode 100644 index 00000000..dd82b5db --- /dev/null +++ b/src/utils/utilparse.h @@ -0,0 +1,29 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAINUTILS_H +#define MULTICHAINUTILS_H + +#include "structs/base58.h" +#include "multichain/multichain.h" +#include "primitives/transaction.h" +#include "keys/key.h" +#include "core/main.h" + +bool ExtractDestinationScriptValid(const CScript& scriptPubKey, CTxDestination& addressRet); +const unsigned char* GetAddressIDPtr(const CTxDestination& address); +bool ParseMultichainTxOutToBuffer(uint256 hash,const CTxOut& txout,mc_Buffer *amounts,mc_Script *lpScript,int *allowed,int *required,std::map* mapSpecialEntity,std::string& strFailReason); +bool ParseMultichainTxOutToBuffer(uint256 hash,const CTxOut& txout,mc_Buffer *amounts,mc_Script *lpScript,int *allowed,int *required,std::string& strFailReason); +bool CreateAssetBalanceList(const CTxOut& txout,mc_Buffer *amounts,mc_Script *lpScript,int *required); +bool CreateAssetBalanceList(const CTxOut& txout,mc_Buffer *amounts,mc_Script *lpScript); +void LogAssetTxOut(std::string message,uint256 hash,int index,unsigned char* assetrefbin,int64_t quantity); +bool AddressCanReceive(CTxDestination address); +bool FindFollowOnsInScript(const CScript& script1,mc_Buffer *amounts,mc_Script *lpScript); + + + + + + +#endif /* MULTICHAINUTILS_H */ + diff --git a/src/utils/utilstrencodings.cpp b/src/utils/utilstrencodings.cpp new file mode 100644 index 00000000..205b1d75 --- /dev/null +++ b/src/utils/utilstrencodings.cpp @@ -0,0 +1,538 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "utils/utilstrencodings.h" + +#include "utils/tinyformat.h" + +#include +#include +#include +#include + +using namespace std; + +string SanitizeString(const string& str) +{ + /** + * safeChars chosen to allow simple messages/URLs/email addresses, but avoid anything + * even possibly remotely dangerous like & or > + */ + static string safeChars("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890 .,;_/:?@()"); + string strResult; + for (std::string::size_type i = 0; i < str.size(); i++) + { + if (safeChars.find(str[i]) != std::string::npos) + strResult.push_back(str[i]); + } + return strResult; +} + +const signed char p_util_hexdigit[256] = +{ -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + 0,1,2,3,4,5,6,7,8,9,-1,-1,-1,-1,-1,-1, + -1,0xa,0xb,0xc,0xd,0xe,0xf,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,0xa,0xb,0xc,0xd,0xe,0xf,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, }; + +signed char HexDigit(char c) +{ + return p_util_hexdigit[(unsigned char)c]; +} + +bool IsHex(const string& str) +{ + for(std::string::const_iterator it(str.begin()); it != str.end(); ++it) + { + if (HexDigit(*it) < 0) + return false; + } + return (str.size() > 0) && (str.size()%2 == 0); +} + +vector ParseHex(const char* psz) +{ + // convert hex dump to vector + vector vch; + while (true) + { + while (isspace(*psz)) + psz++; + signed char c = HexDigit(*psz++); + if (c == (signed char)-1) + break; + unsigned char n = (c << 4); + c = HexDigit(*psz++); + if (c == (signed char)-1) + break; + n |= c; + vch.push_back(n); + } + return vch; +} +/* MCHN START */ +vector ParseHex(const char* psz,bool &fIsHex) +{ + // convert hex dump to vector + vector vch; + fIsHex=true; + if(strlen(psz) % 2) + { + fIsHex=false; + return vch; + } + while (*psz) + { + if(isspace(*psz)) + { + fIsHex=false; + break; + } + signed char c = HexDigit(*psz++); + if (c == (signed char)-1) + { + fIsHex=false; + break; + } + unsigned char n = (c << 4); + c = HexDigit(*psz++); + if (c == (signed char)-1) + { + fIsHex=false; + break; + } + n |= c; + vch.push_back(n); + } + return vch; +} + +/* MCHN END */ + +vector ParseHex(const string& str) +{ + return ParseHex(str.c_str()); +} + +string EncodeBase64(const unsigned char* pch, size_t len) +{ + static const char *pbase64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + + string strRet=""; + strRet.reserve((len+2)/3*4); + + int mode=0, left=0; + const unsigned char *pchEnd = pch+len; + + while (pch> 2]; + left = (enc & 3) << 4; + mode = 1; + break; + + case 1: // we have two bits + strRet += pbase64[left | (enc >> 4)]; + left = (enc & 15) << 2; + mode = 2; + break; + + case 2: // we have four bits + strRet += pbase64[left | (enc >> 6)]; + strRet += pbase64[enc & 63]; + mode = 0; + break; + } + } + + if (mode) + { + strRet += pbase64[left]; + strRet += '='; + if (mode == 1) + strRet += '='; + } + + return strRet; +} + +string EncodeBase64(const string& str) +{ + return EncodeBase64((const unsigned char*)str.c_str(), str.size()); +} + +vector DecodeBase64(const char* p, bool* pfInvalid) +{ + static const int decode64_table[256] = + { + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 62, -1, -1, -1, 63, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, -1, -1, + -1, -1, -1, -1, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, + 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, -1, -1, -1, -1, -1, -1, 26, 27, 28, + 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, + 49, 50, 51, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1 + }; + + if (pfInvalid) + *pfInvalid = false; + + vector vchRet; + vchRet.reserve(strlen(p)*3/4); + + int mode = 0; + int left = 0; + + while (1) + { + int dec = decode64_table[(unsigned char)*p]; + if (dec == -1) break; + p++; + switch (mode) + { + case 0: // we have no bits and get 6 + left = dec; + mode = 1; + break; + + case 1: // we have 6 bits and keep 4 + vchRet.push_back((left<<2) | (dec>>4)); + left = dec & 15; + mode = 2; + break; + + case 2: // we have 4 bits and get 6, we keep 2 + vchRet.push_back((left<<4) | (dec>>2)); + left = dec & 3; + mode = 3; + break; + + case 3: // we have 2 bits and get 6 + vchRet.push_back((left<<6) | dec); + mode = 0; + break; + } + } + + if (pfInvalid) + switch (mode) + { + case 0: // 4n base64 characters processed: ok + break; + + case 1: // 4n+1 base64 character processed: impossible + *pfInvalid = true; + break; + + case 2: // 4n+2 base64 characters processed: require '==' + if (left || p[0] != '=' || p[1] != '=' || decode64_table[(unsigned char)p[2]] != -1) + *pfInvalid = true; + break; + + case 3: // 4n+3 base64 characters processed: require '=' + if (left || p[0] != '=' || decode64_table[(unsigned char)p[1]] != -1) + *pfInvalid = true; + break; + } + + return vchRet; +} + +string DecodeBase64(const string& str) +{ + vector vchRet = DecodeBase64(str.c_str()); + return (vchRet.size() == 0) ? string() : string((const char*)&vchRet[0], vchRet.size()); +} + +string EncodeBase32(const unsigned char* pch, size_t len) +{ + static const char *pbase32 = "abcdefghijklmnopqrstuvwxyz234567"; + + string strRet=""; + strRet.reserve((len+4)/5*8); + + int mode=0, left=0; + const unsigned char *pchEnd = pch+len; + + while (pch> 3]; + left = (enc & 7) << 2; + mode = 1; + break; + + case 1: // we have three bits + strRet += pbase32[left | (enc >> 6)]; + strRet += pbase32[(enc >> 1) & 31]; + left = (enc & 1) << 4; + mode = 2; + break; + + case 2: // we have one bit + strRet += pbase32[left | (enc >> 4)]; + left = (enc & 15) << 1; + mode = 3; + break; + + case 3: // we have four bits + strRet += pbase32[left | (enc >> 7)]; + strRet += pbase32[(enc >> 2) & 31]; + left = (enc & 3) << 3; + mode = 4; + break; + + case 4: // we have two bits + strRet += pbase32[left | (enc >> 5)]; + strRet += pbase32[enc & 31]; + mode = 0; + } + } + + static const int nPadding[5] = {0, 6, 4, 3, 1}; + if (mode) + { + strRet += pbase32[left]; + for (int n=0; n DecodeBase32(const char* p, bool* pfInvalid) +{ + static const int decode32_table[256] = + { + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 26, 27, 28, 29, 30, 31, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, + 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, -1, -1, -1, -1, -1, -1, 0, 1, 2, + 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, + 23, 24, 25, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1 + }; + + if (pfInvalid) + *pfInvalid = false; + + vector vchRet; + vchRet.reserve((strlen(p))*5/8); + + int mode = 0; + int left = 0; + + while (1) + { + int dec = decode32_table[(unsigned char)*p]; + if (dec == -1) break; + p++; + switch (mode) + { + case 0: // we have no bits and get 5 + left = dec; + mode = 1; + break; + + case 1: // we have 5 bits and keep 2 + vchRet.push_back((left<<3) | (dec>>2)); + left = dec & 3; + mode = 2; + break; + + case 2: // we have 2 bits and keep 7 + left = left << 5 | dec; + mode = 3; + break; + + case 3: // we have 7 bits and keep 4 + vchRet.push_back((left<<1) | (dec>>4)); + left = dec & 15; + mode = 4; + break; + + case 4: // we have 4 bits, and keep 1 + vchRet.push_back((left<<4) | (dec>>1)); + left = dec & 1; + mode = 5; + break; + + case 5: // we have 1 bit, and keep 6 + left = left << 5 | dec; + mode = 6; + break; + + case 6: // we have 6 bits, and keep 3 + vchRet.push_back((left<<2) | (dec>>3)); + left = dec & 7; + mode = 7; + break; + + case 7: // we have 3 bits, and keep 0 + vchRet.push_back((left<<5) | dec); + mode = 0; + break; + } + } + + if (pfInvalid) + switch (mode) + { + case 0: // 8n base32 characters processed: ok + break; + + case 1: // 8n+1 base32 characters processed: impossible + case 3: // +3 + case 6: // +6 + *pfInvalid = true; + break; + + case 2: // 8n+2 base32 characters processed: require '======' + if (left || p[0] != '=' || p[1] != '=' || p[2] != '=' || p[3] != '=' || p[4] != '=' || p[5] != '=' || decode32_table[(unsigned char)p[6]] != -1) + *pfInvalid = true; + break; + + case 4: // 8n+4 base32 characters processed: require '====' + if (left || p[0] != '=' || p[1] != '=' || p[2] != '=' || p[3] != '=' || decode32_table[(unsigned char)p[4]] != -1) + *pfInvalid = true; + break; + + case 5: // 8n+5 base32 characters processed: require '===' + if (left || p[0] != '=' || p[1] != '=' || p[2] != '=' || decode32_table[(unsigned char)p[3]] != -1) + *pfInvalid = true; + break; + + case 7: // 8n+7 base32 characters processed: require '=' + if (left || p[0] != '=' || decode32_table[(unsigned char)p[1]] != -1) + *pfInvalid = true; + break; + } + + return vchRet; +} + +string DecodeBase32(const string& str) +{ + vector vchRet = DecodeBase32(str.c_str()); + return (vchRet.size() == 0) ? string() : string((const char*)&vchRet[0], vchRet.size()); +} + +bool ParseInt32(const std::string& str, int32_t *out) +{ + char *endp = NULL; + errno = 0; // strtol will not set errno if valid + long int n = strtol(str.c_str(), &endp, 10); + if(out) *out = (int)n; + // Note that strtol returns a *long int*, so even if strtol doesn't report a over/underflow + // we still have to check that the returned value is within the range of an *int32_t*. On 64-bit + // platforms the size of these types may be different. + return endp && *endp == 0 && !errno && + n >= std::numeric_limits::min() && + n <= std::numeric_limits::max(); +} + +std::string FormatParagraph(const std::string in, size_t width, size_t indent) +{ + std::stringstream out; + size_t col = 0; + size_t ptr = 0; + while(ptr < in.size()) + { + // Find beginning of next word + ptr = in.find_first_not_of(' ', ptr); + if (ptr == std::string::npos) + break; + // Find end of next word + size_t endword = in.find_first_of(' ', ptr); + if (endword == std::string::npos) + endword = in.size(); + // Add newline and indentation if this wraps over the allowed width + if (col > 0) + { + if ((col + endword - ptr) > width) + { + out << '\n'; + for(size_t i=0; i +#include +#include + +#define BEGIN(a) ((char*)&(a)) +#define END(a) ((char*)&((&(a))[1])) +#define UBEGIN(a) ((unsigned char*)&(a)) +#define UEND(a) ((unsigned char*)&((&(a))[1])) +#define ARRAYLEN(array) (sizeof(array)/sizeof((array)[0])) + +/** This is needed because the foreach macro can't get over the comma in pair */ +#define PAIRTYPE(t1, t2) std::pair + +std::string SanitizeString(const std::string& str); +std::vector ParseHex(const char* psz); +/* MCHN START */ +std::vector ParseHex(const char* psz,bool &fIsHex); +/* MCHN END */ +std::vector ParseHex(const std::string& str); +signed char HexDigit(char c); +bool IsHex(const std::string& str); +std::vector DecodeBase64(const char* p, bool* pfInvalid = NULL); +std::string DecodeBase64(const std::string& str); +std::string EncodeBase64(const unsigned char* pch, size_t len); +std::string EncodeBase64(const std::string& str); +std::vector DecodeBase32(const char* p, bool* pfInvalid = NULL); +std::string DecodeBase32(const std::string& str); +std::string EncodeBase32(const unsigned char* pch, size_t len); +std::string EncodeBase32(const std::string& str); + +std::string i64tostr(int64_t n); +std::string itostr(int n); +int64_t atoi64(const char* psz); +int64_t atoi64(const std::string& str); +int atoi(const std::string& str); + +/** + * Convert string to signed 32-bit integer with strict parse error feedback. + * @returns true if the entire string could be parsed as valid integer, + * false if not the entire string could be parsed or when overflow or underflow occurred. + */ +bool ParseInt32(const std::string& str, int32_t *out); + +template +std::string HexStr(const T itbegin, const T itend, bool fSpaces=false) +{ + std::string rv; + static const char hexmap[16] = { '0', '1', '2', '3', '4', '5', '6', '7', + '8', '9', 'a', 'b', 'c', 'd', 'e', 'f' }; + rv.reserve((itend-itbegin)*3); + for(T it = itbegin; it < itend; ++it) + { + unsigned char val = (unsigned char)(*it); + if(fSpaces && it != itbegin) + rv.push_back(' '); + rv.push_back(hexmap[val>>4]); + rv.push_back(hexmap[val&15]); + } + + return rv; +} + +template +inline std::string HexStr(const T& vch, bool fSpaces=false) +{ + return HexStr(vch.begin(), vch.end(), fSpaces); +} + +/** + * Format a paragraph of text to a fixed width, adding spaces for + * indentation to any added line. + */ +std::string FormatParagraph(const std::string in, size_t width=79, size_t indent=0); + +/** + * Timing-attack-resistant comparison. + * Takes time proportional to length + * of first argument. + */ +template +bool TimingResistantEqual(const T& a, const T& b) +{ + if (b.size() == 0) return a.size() == 0; + size_t accumulator = a.size() ^ b.size(); + for (size_t i = 0; i < a.size(); i++) + accumulator |= a[i] ^ b[i%b.size()]; + return accumulator == 0; +} + +#endif // BITCOIN_UTILSTRENCODINGS_H diff --git a/src/utils/utiltime.cpp b/src/utils/utiltime.cpp new file mode 100644 index 00000000..87e36112 --- /dev/null +++ b/src/utils/utiltime.cpp @@ -0,0 +1,70 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#if defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" +#endif + +#include "utiltime.h" + +#include +#include + +using namespace std; + +static int64_t nMockTime = 0; //! For unit testing + +int64_t GetTime() +{ + if (nMockTime) return nMockTime; + + return time(NULL); +} + +void SetMockTime(int64_t nMockTimeIn) +{ + nMockTime = nMockTimeIn; +} + +int64_t GetTimeMillis() +{ + return (boost::posix_time::ptime(boost::posix_time::microsec_clock::universal_time()) - + boost::posix_time::ptime(boost::gregorian::date(1970,1,1))).total_milliseconds(); +} + +int64_t GetTimeMicros() +{ + return (boost::posix_time::ptime(boost::posix_time::microsec_clock::universal_time()) - + boost::posix_time::ptime(boost::gregorian::date(1970,1,1))).total_microseconds(); +} + +void MilliSleep(int64_t n) +{ + +/** + * Boost's sleep_for was uninterruptable when backed by nanosleep from 1.50 + * until fixed in 1.52. Use the deprecated sleep method for the broken case. + * See: https://svn.boost.org/trac/boost/ticket/7238 + */ +#if defined(HAVE_WORKING_BOOST_SLEEP_FOR) + boost::this_thread::sleep_for(boost::chrono::milliseconds(n)); +#elif defined(HAVE_WORKING_BOOST_SLEEP) + boost::this_thread::sleep(boost::posix_time::milliseconds(n)); +#else +//should never get here +#error missing boost sleep implementation +#endif +} + +std::string DateTimeStrFormat(const char* pszFormat, int64_t nTime) +{ + // std::locale takes ownership of the pointer + std::locale loc(std::locale::classic(), new boost::posix_time::time_facet(pszFormat)); + std::stringstream ss; + ss.imbue(loc); + ss << boost::posix_time::from_time_t(nTime); + return ss.str(); +} diff --git a/src/utils/utiltime.h b/src/utils/utiltime.h new file mode 100644 index 00000000..9959d56c --- /dev/null +++ b/src/utils/utiltime.h @@ -0,0 +1,21 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_UTILTIME_H +#define BITCOIN_UTILTIME_H + +#include +#include + +int64_t GetTime(); +int64_t GetTimeMillis(); +int64_t GetTimeMicros(); +void SetMockTime(int64_t nMockTimeIn); +void MilliSleep(int64_t n); + +std::string DateTimeStrFormat(const char* pszFormat, int64_t nTime); + +#endif // BITCOIN_UTILTIME_H diff --git a/src/utils/utilwrapper.cpp b/src/utils/utilwrapper.cpp new file mode 100644 index 00000000..7658d47d --- /dev/null +++ b/src/utils/utilwrapper.cpp @@ -0,0 +1,873 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" +#include "crypto/sha256.h" +#include "structs/base58.h" + +#ifndef WIN32 + +#include + +#else + +#ifdef _MSC_VER +#pragma warning(disable:4786) +#pragma warning(disable:4804) +#pragma warning(disable:4805) +#pragma warning(disable:4717) +#endif + +#ifdef _WIN32_WINNT +#undef _WIN32_WINNT +#endif +#define _WIN32_WINNT 0x0501 + +#ifdef _WIN32_IE +#undef _WIN32_IE +#endif +#define _WIN32_IE 0x0501 + +#define WIN32_LEAN_AND_MEAN 1 +#ifndef NOMINMAX +#define NOMINMAX +#endif + +#include /* for _commit */ +#include + +#endif + + +#include "chainparams/chainparams.h" +#include "utils/util.h" +#include "utils/utilstrencodings.h" +#include "structs/hash.h" +#include "core/main.h" +#include "net/net.h" + +#define MC_DCT_SEED_NODE_MAX_SIZE 32 + +//#include // for to_lower() +//#include +//#include // for startswith() and endswith() +#include +#include +//#include +#include +#include +//#include +//#include +//#include +/* +#include +#include +#include +#include +*/ +using namespace std; + +const boost::filesystem::path mc_GetDataDir(const char *network_name,int create); + +void mc_Params::Parse(int argc, const char* const argv[]) +{ + int i,length; + const char* exe_name; + ParseParameters(argc,argv); + + m_NumArguments=0; + length=MC_DCT_SEED_NODE_MAX_SIZE+1; + for (i = 1; i < argc; i++) + { + if(argv[i][0] != '-') + { + m_NumArguments++; + length+=strlen(argv[i])+1; + } + } + + if(m_NumArguments) + { + m_Arguments=(char**)mc_New((m_NumArguments+1)*sizeof(char*)); + m_Arguments[0]=(char*)mc_New(length); + } + + m_NumArguments=0; + length=0; + for (i = 1; i < argc; i++) + { + if(argv[i][0] != '-') + { + m_Arguments[m_NumArguments]=m_Arguments[0]+length; + strcpy(m_Arguments[m_NumArguments],argv[i]); + m_NumArguments++; + length+=strlen(argv[i])+1; + } + } + + + + if(m_NumArguments) + { + exe_name=argv[0]; + for(i=0;i<(int)strlen(argv[0]);i++) + { + if((argv[0][i]=='/') || (argv[0][i]=='\\')) + { + exe_name=argv[0]+i+1; + } + } + + if((strcmp(exe_name,"multichain-util") == 0) || (strcmp(exe_name,"multichain-util.exe") == 0)) + { + m_FirstArgumentType=MC_FAT_COMMAND; + } + else + { + if((strcmp(exe_name,"multichain-cli") == 0) || + (strcmp(exe_name,"multichain-cli.exe") == 0) || + (strcmp(exe_name,"multichaind") == 0) || + (strcmp(exe_name,"multichaind.exe") == 0)) + { + m_FirstArgumentType=MC_FAT_NETWORK; + for(i=0;i<(int)strlen(m_Arguments[0]);i++) + { + if(m_FirstArgumentType == MC_FAT_NETWORK) + { + if(m_Arguments[0][i] == '@') + { + m_FirstArgumentType=MC_FAT_NETWORKSEED; + m_Arguments[0][i]=0x00; + } + } + } + if((m_FirstArgumentType == MC_FAT_NETWORK) && ((strcmp(exe_name,"multichaind") == 0) || (strcmp(exe_name,"multichaind.exe") == 0))) + { + m_Arguments[m_NumArguments]=m_Arguments[0]+length; + m_Arguments[m_NumArguments][0]=0x00; + length++; + + mc_MapStringString *mapConfig; + int err; + const char *seed_node; + + mapConfig=new mc_MapStringString; + + err=mc_ReadGeneralConfigFile(mapConfig,mc_gState->m_Params->NetworkName(),"seed",".dat"); + + if(err == MC_ERR_NOERROR) + { + seed_node=mapConfig->Get("seed"); + + if(seed_node) + { + if(strlen(seed_node) <= MC_DCT_SEED_NODE_MAX_SIZE) + { + strcpy(m_Arguments[m_NumArguments],seed_node); + length+=strlen(seed_node); + } + } + } + + delete mapConfig; + m_NumArguments++; + } + } + } + } + +} + +const char *mc_Params::NetworkName() +{ + if((m_FirstArgumentType == MC_FAT_NETWORK) || (m_FirstArgumentType == MC_FAT_NETWORKSEED)) + { + return m_Arguments[0]; + } + return NULL; +} + +const char *mc_Params::SeedNode() +{ + const char *seed_node; + if(m_FirstArgumentType == MC_FAT_NETWORKSEED) + { + return m_Arguments[0]+strlen(m_Arguments[0])+1; + } + if(m_FirstArgumentType == MC_FAT_NETWORK) + { + seed_node=m_Arguments[m_NumArguments-1]; + if(*seed_node) + { + return seed_node; + } + } + return NULL; +} + +const char* mc_State::GetSeedNode() +{ + const char *seed_node; + seed_node=mc_gState->m_Params->SeedNode(); + if(seed_node == NULL) + { +/* + int seed_node_size; + seed_node=(char*)mc_gState->m_NetworkParams->GetParam("seednode",&seed_node_size); + if(seed_node_size <= 1) + { + seed_node=NULL; + } + */ + } + + return seed_node; +} + + +const char *mc_Params::Command() +{ + if(m_FirstArgumentType == MC_FAT_COMMAND) + { + return m_Arguments[0]; + } + return NULL; +} + +const char *mc_Params::DataDir() +{ + return DataDir(1,1); +} + +const char *mc_Params::DataDir(int network_specific,int create) +{ + const char *name=NULL; + if(network_specific) + { + name=NetworkName(); + } + + boost::filesystem::path path=mc_GetDataDir(name,create); + + if(network_specific) + { + strcpy(m_DataDirNetSpecific,path.string().c_str()); + return m_DataDirNetSpecific; + } + + strcpy(m_DataDir,path.string().c_str()); + return m_DataDir; +} + +const char* mc_Params::GetOption(const char* strArg, const char* strDefault) +{ + return GetArg(string(strArg),string(strDefault)).c_str(); +} + +int64_t mc_Params::GetOption(const char* strArg, int64_t nDefault) +{ + return GetArg(string(strArg),nDefault); +} + +int64_t mc_Params::HasOption(const char* strArg) +{ + return mapArgs.count(string(strArg)); +} + + +boost::filesystem::path mc_GetDefaultDataDir() +{ + // Windows < Vista: C:\Documents and Settings\Username\Application Data\Bitcoin + // Windows >= Vista: C:\Users\Username\AppData\Roaming\Bitcoin + // Mac: ~/Library/Application Support/Bitcoin + // Unix: ~/.bitcoin +#ifdef WIN32 + // Windows + return GetSpecialFolderPath(CSIDL_APPDATA) / "MultiChain"; +#else + boost::filesystem::path pathRet; + char* pszHome = getenv("HOME"); + if (pszHome == NULL || strlen(pszHome) == 0) + pathRet = boost::filesystem::path("/"); + else + pathRet = boost::filesystem::path(pszHome); +#ifdef MAC_OSX + // Mac + pathRet /= "Library/Application Support"; + TryCreateDirectory(pathRet); + return pathRet / "Bitcoin"; +#else + // Unix + return pathRet / ".multichain"; +#endif +#endif +} + +int mc_GetDataDirArg(char *buf) +{ + if (mapArgs.count("-datadir")) + { + strcpy(buf,mapArgs["-datadir"].c_str()); + return 1; + } + return 0; +} + +void mc_UnsetDataDirArg() +{ + if (mapArgs.count("-datadir")) + { + mapArgs.erase("-datadir"); + } +} + +void mc_SetDataDirArg(char *buf) +{ + mapArgs["-datadir"] = string(buf); +} + + + +const boost::filesystem::path mc_GetDataDir(const char *network_name,int create) +{ + boost::filesystem::path path; + if (mapArgs.count("-datadir")) { + path = boost::filesystem::system_complete(mapArgs["-datadir"]); + if (!boost::filesystem::is_directory(path)) + { + return path; + } + } + else + { + path = mc_GetDefaultDataDir(); + } + if(network_name) + { + path /= std::string(network_name); + } + if(create) + { + boost::filesystem::create_directories(path); + } + return path; +} + +void mc_RemoveDataDir(const char *network_name) +{ + boost::filesystem::path path; + if (mapArgs.count("-datadir")) { + path = boost::filesystem::system_complete(mapArgs["-datadir"]); + } + else + { + path = mc_GetDefaultDataDir(); + } + if(network_name) + { + path /= std::string(network_name); + } + + boost::filesystem::remove_all(path); +} + +void mc_RemoveDir(const char *network_name,const char *dir_name) +{ + boost::filesystem::path path; + if (mapArgs.count("-datadir")) { + path = boost::filesystem::system_complete(mapArgs["-datadir"]); + } + else + { + path = mc_GetDefaultDataDir(); + } + if(network_name) + { + path /= std::string(network_name); + } + if(dir_name) + { + path /= std::string(dir_name); + } + + boost::filesystem::remove_all(path); +} + +string mc_GetFullFileName(const char *network_name,const char *filename, const char *extension,int options) +{ + int create; + std::string fullName = filename; + std::string backupName; + fullName += extension; + boost::filesystem::path pathFile; + + create=0; + if(options & MC_FOM_CREATE_DIR) + { + create=1; + } + switch(options & MC_FOM_RELATIVE_MASK) + { + case MC_FOM_NONE: + pathFile=fullName; + break; + case MC_FOM_RELATIVE_TO_DATADIR: + pathFile = mc_GetDataDir(network_name,create) / fullName; + break; + } + + return pathFile.string(); +} + +int mc_GetFullFileName(const char *network_name,const char *filename, const char *extension,int options,char *buf) +{ + strcpy(buf,mc_GetFullFileName(network_name,filename,extension,options).c_str()); + return MC_ERR_NOERROR; +} + +int mc_BackupFile(const char *network_name,const char *filename, const char *extension,int options) +{ + std::string fullName = mc_GetFullFileName(network_name,filename,extension,options); + std::string backupName; + + backupName=fullName + ".bak"; + + if(rename(fullName.c_str(),backupName.c_str())) + { + return MC_ERR_FILE_WRITE_ERROR; + } + + return MC_ERR_NOERROR; +} + +int mc_RecoverFile(const char *network_name,const char *filename, const char *extension,int options) +{ + std::string fullName = mc_GetFullFileName(network_name,filename,extension,options); + std::string backupName; + + backupName=fullName + ".bak"; + + if(rename(backupName.c_str(),fullName.c_str())) + { + return MC_ERR_FILE_WRITE_ERROR; + } + + return MC_ERR_NOERROR; +} + +FILE *mc_OpenFile(const char *network_name,const char *filename, const char *extension,const char *mode, int options) +{ + return fopen(mc_GetFullFileName(network_name,filename,extension,options).c_str(), mode); +} + +int mc_RemoveFile(const char *network_name,const char *filename, const char *extension,int options) +{ + return unlink(mc_GetFullFileName(network_name,filename,extension,options).c_str()); +} + + +void mc_CloseFile(FILE *fHan) +{ + if(fHan) + { + fclose(fHan); + } +} + +size_t mc_ReadFileToBuffer(FILE *fHan,char **lpptr) +{ + size_t size; + *lpptr=NULL; + + if(fHan==NULL) + { + return -MC_ERR_INTERNAL_ERROR; + } + + fseek(fHan, 0L, SEEK_END); + size = ftell(fHan); + fseek(fHan, 0L, SEEK_SET); + + if(size<=0) + { + return size; + } + + *lpptr=(char*)mc_New(size); + if(*lpptr) + { + if(fread(*lpptr,size,1,fHan) != size) + { + mc_Delete(*lpptr); + *lpptr=NULL; + return -MC_ERR_FILE_READ_ERROR; + } + } + else + { + return -MC_ERR_ALLOCATION; + } + + return size; +} + +boost::filesystem::path mc_GetConfigFile(const char *network_name,const char *file_name,const char *extension) +{ + string fileName="multichain"; + if(file_name) + { + fileName = file_name; + } + if(extension) + { + fileName += extension; + } + + boost::filesystem::path pathConfigFile(GetArg("-conf", fileName)); + if (!pathConfigFile.is_complete()) + pathConfigFile = mc_GetDataDir(network_name,0) / pathConfigFile; + return pathConfigFile; +} + +static void mc_InterpretNegativeSetting(string name, map& mapSettingsRet) +{ + // interpret -nofoo as -foo=0 (and -nofoo=0 as -foo=1) as long as -foo not set + if (name.find("-no") == 0) + { + std::string positive("-"); + positive.append(name.begin()+3, name.end()); + if (mapSettingsRet.count(positive) == 0) + { + bool value = !GetBoolArg(name, false); + mapSettingsRet[positive] = (value ? "1" : "0"); + } + } +} + +int mc_ReadParamArgs(mc_MapStringString *mapConfig, + int argc, char* argv[], + const char *prefix) +{ + int p; + char *ptr; + map* mapSettingsRet=(std::map*)mapConfig->mapObject; + for(int argi=1;argi* mapSettingsRet, + map >* mapMultiSettingsRet, + const char *prefix) +{ + try + { + boost::filesystem::ifstream streamConfig(fileConfig); + if (!streamConfig.good()) + return MC_ERR_NOERROR; // No bitcoin.conf file is OK + + set setOptions; + setOptions.insert("*"); + + for (boost::program_options::detail::config_file_iterator it(streamConfig, setOptions), end; it != end; ++it) + { + // Don't overwrite existing settings so command line settings override bitcoin.conf + + string strKey = it->string_key; + strKey.erase(std::remove(strKey.begin(), strKey.end(), '-'), strKey.end()); + strKey.erase(std::remove(strKey.begin(), strKey.end(), '_'), strKey.end()); + + strKey = string(prefix) + strKey; + if ((*mapSettingsRet).count(strKey) == 0) + { + (*mapSettingsRet)[strKey] = it->value[0]; + // interpret nofoo=1 as foo=0 (and nofoo=0 as foo=1) as long as foo not set) + mc_InterpretNegativeSetting(strKey, *mapSettingsRet); + } + if(mapMultiSettingsRet) + { + (*mapMultiSettingsRet)[strKey].push_back(it->value[0]); + } + } + } catch(std::exception &e) { + fprintf(stderr,"ERROR: reading configuration file: %s\n", e.what()); + return MC_ERR_FILE_READ_ERROR; + } + + return MC_ERR_NOERROR; +} + + +int mc_Params::ReadConfig(const char *network_name) +{ + mc_ReadConfigFile(mc_GetConfigFile(network_name,"multichain",".conf"),&mapArgs, &mapMultiArgs,"-"); + return mc_ReadConfigFile(mc_GetConfigFile(NULL,NULL,".conf"),&mapArgs, &mapMultiArgs,"-"); +} + +int mc_ReadGeneralConfigFile(mc_MapStringString *mapConfig,const char *network_name,const char *file_name,const char *extension) +{ + return mc_ReadConfigFile(mc_GetConfigFile(network_name,file_name,extension),(std::map*)mapConfig->mapObject, NULL,""); +} + + +int mc_MultichainParams::SetGlobals() +{ + m_IsProtocolMultiChain=1; + void *ptr=GetParam("chainprotocol",NULL); + if(ptr) + { + if(strcmp((char*)ptr,"multichain")) + { + m_IsProtocolMultiChain=0; + } + } + m_ProtocolVersion=ProtocolVersion(); + + MIN_RELAY_TX_FEE=(unsigned int)GetInt64Param("minimumrelayfee"); + MAX_OP_RETURN_RELAY=(unsigned int)GetInt64Param("maxstdopreturnsize"); + MAX_OP_RETURN_RELAY=GetArg("-datacarriersize", MAX_OP_RETURN_RELAY); + MAX_BLOCK_SIZE=(unsigned int)GetInt64Param("maximumblocksize"); + DEFAULT_BLOCK_MAX_SIZE=MAX_BLOCK_SIZE; + MAX_STANDARD_TX_SIZE=(unsigned int)GetInt64Param("maxstdtxsize"); + MAX_SCRIPT_ELEMENT_SIZE=(unsigned int)GetInt64Param("maxstdelementsize"); + COINBASE_MATURITY=(int)GetInt64Param("rewardspendabledelay"); + COIN=GetInt64Param("nativecurrencymultiple"); + CENT=COIN/100; + MAX_MONEY=GetInt64Param("maximumperoutput"); + if((mc_gState->m_NetworkParams->GetInt64Param("initialblockreward") == 0) && (mc_gState->m_NetworkParams->GetInt64Param("firstblockreward") <= 0)) + { + COIN=0; + CENT=0; + MAX_MONEY=0; + } + + if(mc_gState->m_Features->ShortTxIDAsAssetRef() == 0) + { + m_AssetRefSize=MC_AST_ASSET_REF_SIZE; + } + return MC_ERR_NOERROR; +} + + +void mc_SHA256::Init() +{ + m_HashObject=new CSHA256; + ((CSHA256*)m_HashObject)->Reset(); +} + +void mc_SHA256::Destroy() +{ + if(m_HashObject) + { + delete (CSHA256*)m_HashObject; + } +} + +void mc_SHA256::Reset() +{ + if(m_HashObject) + { + ((CSHA256*)m_HashObject)->Reset(); + } +} + +void mc_SHA256::Write(const void *lpData,int size) +{ + if(m_HashObject) + { + ((CSHA256*)m_HashObject)->Write((const unsigned char*)lpData,size); + } +} + +void mc_SHA256::GetHash(unsigned char *hash) +{ + if(m_HashObject) + { + ((CSHA256*)m_HashObject)->Finalize(hash); + } +} + +int mc_MultichainParams::Import(const char *name,const char *source_address) +{ + + return MC_ERR_NOERROR; +} + +std::string MultichainServerAddress() +{ + string result=string(mc_gState->m_NetworkParams->Name()); + unsigned char *ptr; + result+="@"; + if(mc_gState->m_IPv4Address) + { + ptr=(unsigned char *)(&(mc_gState->m_IPv4Address)); + result+=strprintf("%u.%u.%u.%u",ptr[3],ptr[2],ptr[1],ptr[0]); + } + else + { + result+=""; + } + + return result; +} + +int mc_SetIPv4ServerAddress(const char* host) +{ + char host_copy[16]; + char *ptr; + uint32_t result; + int count,v,i; + result=0; + mc_gState->m_IPv4Address=0; + + if((strlen(host)<7) || (strlen(host)>15)) + { + return 0; + } + memcpy(host_copy,host,strlen(host)+1); + count=0; + for(i=0;i<(int)strlen(host);i++) + { + if(host_copy[i] == '.') + { + host_copy[i]=0x00; + count++; + } + } + if(count != 3) + { + return 0; + } + ptr=host_copy; + for(i=0;i<4;i++) + { + if((strlen(ptr)<1) || (strlen(ptr)>3)) + { + return 0; + } + v=atoi(ptr); + if((v<0) || (v>255)) + { + return 0; + } + result=(result << 8) + v; + ptr+=strlen(ptr)+1; + } + + mc_gState->m_IPv4Address=result; + return result; +} + +int mc_FindIPv4ServerAddress(uint32_t *all_ips,int max_ips) +{ + int i, l, c; + uint32_t ip; + unsigned char *ptr; + int result; + + mc_gState->m_IPv4Address=0; + l=0; + + result=0; + c=0; + +#ifndef WIN32 + + int sock; + struct ifreq ifreqs[20]; + struct ifconf ic; + ic.ifc_len = sizeof ifreqs; + ic.ifc_req = ifreqs; + + sock = socket(AF_INET, SOCK_DGRAM, 0); + if( (sock >= 0) && (ioctl(sock, SIOCGIFCONF, &ic) >= 0) ) + { + for (i = 0; i < (int)(ic.ifc_len/sizeof(struct ifreq)); i++) + { + uint32_t a; + a=((struct sockaddr_in*)&ifreqs[i].ifr_addr)->sin_addr.s_addr; + ptr=(unsigned char*)&a; +#else + + struct hostent *phe = gethostbyname(""); + if (phe) + { + for (int i = 0; phe->h_addr_list[i] != 0; ++i) + { + struct in_addr addr; + memcpy(&addr, phe->h_addr_list[i], sizeof(struct in_addr)); + ptr=(unsigned char*)&addr; + +#endif + + if ((ptr[0] != 127) && (ptr[0] != 0)) + { + ip=((uint32_t)ptr[0]<<24)+((uint32_t)ptr[1]<<16)+((uint32_t)ptr[2]<<8)+(uint32_t)ptr[3]; + if( (ptr[0] == 10) || ((ptr[0] == 192) && (ptr[1] == 168)) || ((ptr[0] == 172) && ((ptr[1] >= 16) && (ptr[1] <= 31))) ) + { + if( (l == 0) && (result == 0) ) + { + mc_gState->m_IPv4Address=ip; + l=1; + result=1; + } + } + else + { + mc_gState->m_IPv4Address=ip; + l=0; + result=2; + } + if(c + +/** + * Name of client reported in the 'version' message. Report the same name + * for both bitcoind and bitcoin-core, to make it harder for attackers to + * target servers or GUI users specifically. + */ +const std::string CLIENT_NAME("Satoshi"); + +/** + * Client version number + */ +#define CLIENT_VERSION_SUFFIX "" + + +/** + * The following part of the code determines the CLIENT_BUILD variable. + * Several mechanisms are used for this: + * * first, if HAVE_BUILD_INFO is defined, include build.h, a file that is + * generated by the build environment, possibly containing the output + * of git-describe in a macro called BUILD_DESC + * * secondly, if this is an exported version of the code, GIT_ARCHIVE will + * be defined (automatically using the export-subst git attribute), and + * GIT_COMMIT will contain the commit id. + * * then, three options exist for determining CLIENT_BUILD: + * * if BUILD_DESC is defined, use that literally (output of git-describe) + * * if not, but GIT_COMMIT is defined, use v[maj].[min].[rev].[build]-g[commit] + * * otherwise, use v[maj].[min].[rev].[build]-unk + * finally CLIENT_VERSION_SUFFIX is added + */ + +//! First, include build.h if requested +#ifdef HAVE_BUILD_INFO +//#include "build.h" // MCHN +#endif + +//! git will put "#define GIT_ARCHIVE 1" on the next line inside archives. $Format:%n#define GIT_ARCHIVE 1$ +#ifdef GIT_ARCHIVE +#define GIT_COMMIT_ID "$Format:%h$" +#define GIT_COMMIT_DATE "$Format:%cD$" +#endif + +#define BUILD_DESC_WITH_SUFFIX(maj, min, rev, build, suffix) \ + "v" DO_STRINGIZE(maj) "." DO_STRINGIZE(min) "." DO_STRINGIZE(rev) "." DO_STRINGIZE(build) "-" DO_STRINGIZE(suffix) + +#define BUILD_DESC_FROM_COMMIT(maj, min, rev, build, commit) \ + "v" DO_STRINGIZE(maj) "." DO_STRINGIZE(min) "." DO_STRINGIZE(rev) "." DO_STRINGIZE(build) "-g" commit + +#define BUILD_DESC_FROM_UNKNOWN(maj, min, rev, build) \ + "v" DO_STRINGIZE(maj) "." DO_STRINGIZE(min) "." DO_STRINGIZE(rev) "." DO_STRINGIZE(build) "-unk" + +#ifndef BUILD_DESC +#ifdef BUILD_SUFFIX +#define BUILD_DESC BUILD_DESC_WITH_SUFFIX(CLIENT_VERSION_MAJOR, CLIENT_VERSION_MINOR, CLIENT_VERSION_REVISION, CLIENT_VERSION_BUILD, BUILD_SUFFIX) +#elif defined(GIT_COMMIT_ID) +#define BUILD_DESC BUILD_DESC_FROM_COMMIT(CLIENT_VERSION_MAJOR, CLIENT_VERSION_MINOR, CLIENT_VERSION_REVISION, CLIENT_VERSION_BUILD, GIT_COMMIT_ID) +#else +#define BUILD_DESC BUILD_DESC_FROM_UNKNOWN(CLIENT_VERSION_MAJOR, CLIENT_VERSION_MINOR, CLIENT_VERSION_REVISION, CLIENT_VERSION_BUILD) +#endif +#endif + +#ifndef BUILD_DATE +#ifdef GIT_COMMIT_DATE +#define BUILD_DATE GIT_COMMIT_DATE +#else +#define BUILD_DATE __DATE__ ", " __TIME__ +#endif +#endif + +const std::string CLIENT_BUILD(BUILD_DESC CLIENT_VERSION_SUFFIX); +const std::string CLIENT_DATE(BUILD_DATE); + +static std::string FormatVersion(int nVersion) +{ + if (nVersion % 100 == 0) + return strprintf("%d.%d.%d", nVersion / 1000000, (nVersion / 10000) % 100, (nVersion / 100) % 100); + else + return strprintf("%d.%d.%d.%d", nVersion / 1000000, (nVersion / 10000) % 100, (nVersion / 100) % 100, nVersion % 100); +} + +std::string FormatFullVersion() +{ + return CLIENT_BUILD; +} + +/** + * Format the subversion field according to BIP 14 spec (https://github.com/bitcoin/bips/blob/master/bip-0014.mediawiki) + */ +std::string FormatSubVersion(const std::string& name, int nClientVersion, const std::vector& comments) +{ + std::ostringstream ss; + ss << "/"; + ss << name << ":" << FormatVersion(nClientVersion); + if (!comments.empty()) + { + std::vector::const_iterator it(comments.begin()); + ss << "(" << *it; + for(++it; it != comments.end(); ++it) + ss << "; " << *it; + ss << ")"; + } + ss << "/"; + return ss.str(); +} diff --git a/src/version/clientversion.h b/src/version/clientversion.h new file mode 100644 index 00000000..51235172 --- /dev/null +++ b/src/version/clientversion.h @@ -0,0 +1,72 @@ +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_CLIENTVERSION_H +#define BITCOIN_CLIENTVERSION_H + +#if defined(HAVE_CONFIG_H) +#include "config/bitcoin-config.h" +#else + +/** + * client versioning and copyright year + */ + +//! These need to be macros, as clientversion.cpp's and bitcoin*-res.rc's voodoo requires it +#define CLIENT_VERSION_MAJOR 0 +#define CLIENT_VERSION_MINOR 10 +#define CLIENT_VERSION_REVISION 0 +#define CLIENT_VERSION_BUILD 0 + +//! Set to true for release, false for prerelease or test build +#define CLIENT_VERSION_IS_RELEASE true + +/** + * Copyright year (2009-this) + * Todo: update this when changing our copyright comments in the source + */ +#define COPYRIGHT_YEAR 2015 + +#endif //HAVE_CONFIG_H + +/** + * Converts the parameter X to a string after macro replacement on X has been performed. + * Don't merge these into one macro! + */ +#define STRINGIZE(X) DO_STRINGIZE(X) +#define DO_STRINGIZE(X) #X + +//! Copyright string used in Windows .rc files +/* MCHN START */ +#define COPYRIGHT_STR "2014-" STRINGIZE(COPYRIGHT_YEAR) " Coin Sciences Ltd" +/* MCHN END */ +/** + * bitcoind-res.rc includes this file, but it cannot cope with real c++ code. + * WINDRES_PREPROC is defined to indicate that its pre-processor is running. + * Anything other than a define should be guarded below. + */ + +#if !defined(WINDRES_PREPROC) + +#include +#include + +static const int CLIENT_VERSION = + 1000000 * CLIENT_VERSION_MAJOR + + 10000 * CLIENT_VERSION_MINOR + + 100 * CLIENT_VERSION_REVISION + + 1 * CLIENT_VERSION_BUILD; + +extern const std::string CLIENT_NAME; +extern const std::string CLIENT_BUILD; +extern const std::string CLIENT_DATE; + + +std::string FormatFullVersion(); +std::string FormatSubVersion(const std::string& name, int nClientVersion, const std::vector& comments); + +#endif // WINDRES_PREPROC + +#endif // BITCOIN_CLIENTVERSION_H diff --git a/src/version/version.cpp b/src/version/version.cpp new file mode 100644 index 00000000..4f63ebfa --- /dev/null +++ b/src/version/version.cpp @@ -0,0 +1,22 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" + + +#include "version/version.h" + +const char* mc_State::GetVersion() +{ + return MULTICHAIN_BUILD_DESC; +} + +const char* mc_State::GetFullVersion() +{ + return MULTICHAIN_FULL_VERSION; +} + +int mc_State::GetProtocolVersion() +{ + return MULTICHAIN_PROTOCOL_VERSION; +} diff --git a/src/version/version.h b/src/version/version.h new file mode 100644 index 00000000..1e4d1772 --- /dev/null +++ b/src/version/version.h @@ -0,0 +1,50 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAINVERSION_H +#define MULTICHAINVERSION_H + +#define MULTICHAIN_VERSION_MAJOR 1 +#define MULTICHAIN_VERSION_MINOR 0 +#define MULTICHAIN_VERSION_REVISION 0 +#define MULTICHAIN_VERSION_BUILD 27 + +#define MULTICHAIN_PROTOCOL_VERSION 10007 + + +#ifndef STRINGIZE +#define STRINGIZE(X) DO_STRINGIZE(X) +#endif + +#ifndef DO_STRINGIZE +#define DO_STRINGIZE(X) #X +#endif + +#define MULTICHAIN_BUILD_DESC_WITH_SUFFIX(maj, min, rev, build, suffix) \ + DO_STRINGIZE(maj) "." DO_STRINGIZE(min) "." DO_STRINGIZE(rev) "." DO_STRINGIZE(build) "-" DO_STRINGIZE(suffix) + +#define MULTICHAIN_BUILD_DESC_FROM_UNKNOWN(maj, min, rev, build) \ + DO_STRINGIZE(maj) "." DO_STRINGIZE(min) "." DO_STRINGIZE(rev) "." DO_STRINGIZE(build) + + +#define MULTICHAIN_BUILD_DESC "1.0 alpha 27" + +#ifndef MULTICHAIN_BUILD_DESC +#ifdef BUILD_SUFFIX +#define MULTICHAIN_BUILD_DESC MULTICHAIN_BUILD_DESC_WITH_SUFFIX(MULTICHAIN_VERSION_MAJOR, MULTICHAIN_VERSION_MINOR, MULTICHAIN_VERSION_REVISION, MULTICHAIN_VERSION_BUILD, BUILD_SUFFIX) +#else +#define MULTICHAIN_BUILD_DESC MULTICHAIN_BUILD_DESC_FROM_UNKNOWN(MULTICHAIN_VERSION_MAJOR, MULTICHAIN_VERSION_MINOR, MULTICHAIN_VERSION_REVISION, MULTICHAIN_VERSION_BUILD) +#endif +#endif + +#define MULTICHAIN_FULL_DESC(build, protocol) \ + "build " build " protocol " DO_STRINGIZE(protocol) + + +#ifndef MULTICHAIN_FULL_VERSION +#define MULTICHAIN_FULL_VERSION MULTICHAIN_FULL_DESC(MULTICHAIN_BUILD_DESC, MULTICHAIN_PROTOCOL_VERSION) +#endif + + +#endif /* MULTICHAINVERSION_H */ + diff --git a/src/wallet/coincontrol.h b/src/wallet/coincontrol.h new file mode 100644 index 00000000..dcaff799 --- /dev/null +++ b/src/wallet/coincontrol.h @@ -0,0 +1,63 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_COINCONTROL_H +#define BITCOIN_COINCONTROL_H + +#include "primitives/transaction.h" + +/** Coin Control Features. */ +class CCoinControl +{ +public: + CTxDestination destChange; + + CCoinControl() + { + SetNull(); + } + + void SetNull() + { + destChange = CNoDestination(); + setSelected.clear(); + } + + bool HasSelected() const + { + return (setSelected.size() > 0); + } + + bool IsSelected(const uint256& hash, unsigned int n) const + { + COutPoint outpt(hash, n); + return (setSelected.count(outpt) > 0); + } + + void Select(const COutPoint& output) + { + setSelected.insert(output); + } + + void UnSelect(const COutPoint& output) + { + setSelected.erase(output); + } + + void UnSelectAll() + { + setSelected.clear(); + } + + void ListSelected(std::vector& vOutpoints) + { + vOutpoints.assign(setSelected.begin(), setSelected.end()); + } + +private: + std::set setSelected; +}; + +#endif // BITCOIN_COINCONTROL_H diff --git a/src/wallet/crypter.cpp b/src/wallet/crypter.cpp new file mode 100644 index 00000000..83804f68 --- /dev/null +++ b/src/wallet/crypter.cpp @@ -0,0 +1,298 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "crypter.h" + +#include "script/script.h" +#include "script/standard.h" +#include "utils/util.h" + +#include +#include +#include +#include +#include + +bool CCrypter::SetKeyFromPassphrase(const SecureString& strKeyData, const std::vector& chSalt, const unsigned int nRounds, const unsigned int nDerivationMethod) +{ + if (nRounds < 1 || chSalt.size() != WALLET_CRYPTO_SALT_SIZE) + return false; + + int i = 0; + if (nDerivationMethod == 0) + i = EVP_BytesToKey(EVP_aes_256_cbc(), EVP_sha512(), &chSalt[0], + (unsigned char *)&strKeyData[0], strKeyData.size(), nRounds, chKey, chIV); + + if (i != (int)WALLET_CRYPTO_KEY_SIZE) + { + OPENSSL_cleanse(chKey, sizeof(chKey)); + OPENSSL_cleanse(chIV, sizeof(chIV)); + return false; + } + + fKeySet = true; + return true; +} + +bool CCrypter::SetKey(const CKeyingMaterial& chNewKey, const std::vector& chNewIV) +{ + if (chNewKey.size() != WALLET_CRYPTO_KEY_SIZE || chNewIV.size() != WALLET_CRYPTO_KEY_SIZE) + return false; + + memcpy(&chKey[0], &chNewKey[0], sizeof chKey); + memcpy(&chIV[0], &chNewIV[0], sizeof chIV); + + fKeySet = true; + return true; +} + +bool CCrypter::Encrypt(const CKeyingMaterial& vchPlaintext, std::vector &vchCiphertext) +{ + if (!fKeySet) + return false; + + // max ciphertext len for a n bytes of plaintext is + // n + AES_BLOCK_SIZE - 1 bytes + int nLen = vchPlaintext.size(); + int nCLen = nLen + AES_BLOCK_SIZE, nFLen = 0; + vchCiphertext = std::vector (nCLen); + + EVP_CIPHER_CTX ctx; + + bool fOk = true; + + EVP_CIPHER_CTX_init(&ctx); + if (fOk) fOk = EVP_EncryptInit_ex(&ctx, EVP_aes_256_cbc(), NULL, chKey, chIV) != 0; + if (fOk) fOk = EVP_EncryptUpdate(&ctx, &vchCiphertext[0], &nCLen, &vchPlaintext[0], nLen) != 0; + if (fOk) fOk = EVP_EncryptFinal_ex(&ctx, (&vchCiphertext[0]) + nCLen, &nFLen) != 0; + EVP_CIPHER_CTX_cleanup(&ctx); + + if (!fOk) return false; + + vchCiphertext.resize(nCLen + nFLen); + return true; +} + +bool CCrypter::Decrypt(const std::vector& vchCiphertext, CKeyingMaterial& vchPlaintext) +{ + if (!fKeySet) + return false; + + // plaintext will always be equal to or lesser than length of ciphertext + int nLen = vchCiphertext.size(); + int nPLen = nLen, nFLen = 0; + + vchPlaintext = CKeyingMaterial(nPLen); + + EVP_CIPHER_CTX ctx; + + bool fOk = true; + + EVP_CIPHER_CTX_init(&ctx); + if (fOk) fOk = EVP_DecryptInit_ex(&ctx, EVP_aes_256_cbc(), NULL, chKey, chIV) != 0; + if (fOk) fOk = EVP_DecryptUpdate(&ctx, &vchPlaintext[0], &nPLen, &vchCiphertext[0], nLen) != 0; + if (fOk) fOk = EVP_DecryptFinal_ex(&ctx, (&vchPlaintext[0]) + nPLen, &nFLen) != 0; + EVP_CIPHER_CTX_cleanup(&ctx); + + if (!fOk) return false; + + vchPlaintext.resize(nPLen + nFLen); + return true; +} + + +bool EncryptSecret(const CKeyingMaterial& vMasterKey, const CKeyingMaterial &vchPlaintext, const uint256& nIV, std::vector &vchCiphertext) +{ + CCrypter cKeyCrypter; + std::vector chIV(WALLET_CRYPTO_KEY_SIZE); + memcpy(&chIV[0], &nIV, WALLET_CRYPTO_KEY_SIZE); + if(!cKeyCrypter.SetKey(vMasterKey, chIV)) + return false; + return cKeyCrypter.Encrypt(*((const CKeyingMaterial*)&vchPlaintext), vchCiphertext); +} + +bool DecryptSecret(const CKeyingMaterial& vMasterKey, const std::vector& vchCiphertext, const uint256& nIV, CKeyingMaterial& vchPlaintext) +{ + CCrypter cKeyCrypter; + std::vector chIV(WALLET_CRYPTO_KEY_SIZE); + memcpy(&chIV[0], &nIV, WALLET_CRYPTO_KEY_SIZE); + if(!cKeyCrypter.SetKey(vMasterKey, chIV)) + return false; + return cKeyCrypter.Decrypt(vchCiphertext, *((CKeyingMaterial*)&vchPlaintext)); +} + +bool CCryptoKeyStore::SetCrypted() +{ + LOCK(cs_KeyStore); + if (fUseCrypto) + return true; + if (!mapKeys.empty()) + return false; + fUseCrypto = true; + return true; +} + +bool CCryptoKeyStore::Lock() +{ + if (!SetCrypted()) + return false; + + { + LOCK(cs_KeyStore); + vMasterKey.clear(); + } + + NotifyStatusChanged(this); + return true; +} + +bool CCryptoKeyStore::Unlock(const CKeyingMaterial& vMasterKeyIn) +{ + { + LOCK(cs_KeyStore); + if (!SetCrypted()) + return false; + + bool keyPass = false; + bool keyFail = false; + CryptedKeyMap::const_iterator mi = mapCryptedKeys.begin(); + for (; mi != mapCryptedKeys.end(); ++mi) + { + const CPubKey &vchPubKey = (*mi).second.first; + const std::vector &vchCryptedSecret = (*mi).second.second; + CKeyingMaterial vchSecret; + if(!DecryptSecret(vMasterKeyIn, vchCryptedSecret, vchPubKey.GetHash(), vchSecret)) + { + keyFail = true; + break; + } + if (vchSecret.size() != 32) + { + keyFail = true; + break; + } + CKey key; + key.Set(vchSecret.begin(), vchSecret.end(), vchPubKey.IsCompressed()); + if (key.GetPubKey() != vchPubKey) + { + keyFail = true; + break; + } + keyPass = true; + if (fDecryptionThoroughlyChecked) + break; + } + if (keyPass && keyFail) + { + LogPrintf("The wallet is probably corrupted: Some keys decrypt but not all."); + assert(false); + } + if (keyFail || !keyPass) + return false; + vMasterKey = vMasterKeyIn; + fDecryptionThoroughlyChecked = true; + } + NotifyStatusChanged(this); + return true; +} + +bool CCryptoKeyStore::AddKeyPubKey(const CKey& key, const CPubKey &pubkey) +{ + { + LOCK(cs_KeyStore); + if (!IsCrypted()) + return CBasicKeyStore::AddKeyPubKey(key, pubkey); + + if (IsLocked()) + return false; + + std::vector vchCryptedSecret; + CKeyingMaterial vchSecret(key.begin(), key.end()); + if (!EncryptSecret(vMasterKey, vchSecret, pubkey.GetHash(), vchCryptedSecret)) + return false; + + if (!AddCryptedKey(pubkey, vchCryptedSecret)) + return false; + } + return true; +} + + +bool CCryptoKeyStore::AddCryptedKey(const CPubKey &vchPubKey, const std::vector &vchCryptedSecret) +{ + { + LOCK(cs_KeyStore); + if (!SetCrypted()) + return false; + + mapCryptedKeys[vchPubKey.GetID()] = make_pair(vchPubKey, vchCryptedSecret); + } + return true; +} + +bool CCryptoKeyStore::GetKey(const CKeyID &address, CKey& keyOut) const +{ + { + LOCK(cs_KeyStore); + if (!IsCrypted()) + return CBasicKeyStore::GetKey(address, keyOut); + + CryptedKeyMap::const_iterator mi = mapCryptedKeys.find(address); + if (mi != mapCryptedKeys.end()) + { + const CPubKey &vchPubKey = (*mi).second.first; + const std::vector &vchCryptedSecret = (*mi).second.second; + CKeyingMaterial vchSecret; + if (!DecryptSecret(vMasterKey, vchCryptedSecret, vchPubKey.GetHash(), vchSecret)) + return false; + if (vchSecret.size() != 32) + return false; + keyOut.Set(vchSecret.begin(), vchSecret.end(), vchPubKey.IsCompressed()); + return true; + } + } + return false; +} + +bool CCryptoKeyStore::GetPubKey(const CKeyID &address, CPubKey& vchPubKeyOut) const +{ + { + LOCK(cs_KeyStore); + if (!IsCrypted()) + return CKeyStore::GetPubKey(address, vchPubKeyOut); + + CryptedKeyMap::const_iterator mi = mapCryptedKeys.find(address); + if (mi != mapCryptedKeys.end()) + { + vchPubKeyOut = (*mi).second.first; + return true; + } + } + return false; +} + +bool CCryptoKeyStore::EncryptKeys(CKeyingMaterial& vMasterKeyIn) +{ + { + LOCK(cs_KeyStore); + if (!mapCryptedKeys.empty() || IsCrypted()) + return false; + + fUseCrypto = true; + BOOST_FOREACH(KeyMap::value_type& mKey, mapKeys) + { + const CKey &key = mKey.second; + CPubKey vchPubKey = key.GetPubKey(); + CKeyingMaterial vchSecret(key.begin(), key.end()); + std::vector vchCryptedSecret; + if (!EncryptSecret(vMasterKeyIn, vchSecret, vchPubKey.GetHash(), vchCryptedSecret)) + return false; + if (!AddCryptedKey(vchPubKey, vchCryptedSecret)) + return false; + } + mapKeys.clear(); + } + return true; +} diff --git a/src/wallet/crypter.h b/src/wallet/crypter.h new file mode 100644 index 00000000..53a2c59b --- /dev/null +++ b/src/wallet/crypter.h @@ -0,0 +1,200 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_CRYPTER_H +#define BITCOIN_CRYPTER_H + +#include "utils/allocators.h" +#include "wallet/keystore.h" +#include "utils/serialize.h" + +class uint256; + +const unsigned int WALLET_CRYPTO_KEY_SIZE = 32; +const unsigned int WALLET_CRYPTO_SALT_SIZE = 8; + +/** + * Private key encryption is done based on a CMasterKey, + * which holds a salt and random encryption key. + * + * CMasterKeys are encrypted using AES-256-CBC using a key + * derived using derivation method nDerivationMethod + * (0 == EVP_sha512()) and derivation iterations nDeriveIterations. + * vchOtherDerivationParameters is provided for alternative algorithms + * which may require more parameters (such as scrypt). + * + * Wallet Private Keys are then encrypted using AES-256-CBC + * with the double-sha256 of the public key as the IV, and the + * master key's key as the encryption key (see keystore.[ch]). + */ + +/** Master key for wallet encryption */ +class CMasterKey +{ +public: + std::vector vchCryptedKey; + std::vector vchSalt; + //! 0 = EVP_sha512() + //! 1 = scrypt() + unsigned int nDerivationMethod; + unsigned int nDeriveIterations; + //! Use this for more parameters to key derivation, + //! such as the various parameters to scrypt + std::vector vchOtherDerivationParameters; + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(vchCryptedKey); + READWRITE(vchSalt); + READWRITE(nDerivationMethod); + READWRITE(nDeriveIterations); + READWRITE(vchOtherDerivationParameters); + } + + CMasterKey() + { + // 25000 rounds is just under 0.1 seconds on a 1.86 GHz Pentium M + // ie slightly lower than the lowest hardware we need bother supporting + nDeriveIterations = 25000; + nDerivationMethod = 0; + vchOtherDerivationParameters = std::vector(0); + } +}; + +typedef std::vector > CKeyingMaterial; + +/** Encryption/decryption context with key information */ +class CCrypter +{ +private: + unsigned char chKey[WALLET_CRYPTO_KEY_SIZE]; + unsigned char chIV[WALLET_CRYPTO_KEY_SIZE]; + bool fKeySet; + +public: + bool SetKeyFromPassphrase(const SecureString &strKeyData, const std::vector& chSalt, const unsigned int nRounds, const unsigned int nDerivationMethod); + bool Encrypt(const CKeyingMaterial& vchPlaintext, std::vector &vchCiphertext); + bool Decrypt(const std::vector& vchCiphertext, CKeyingMaterial& vchPlaintext); + bool SetKey(const CKeyingMaterial& chNewKey, const std::vector& chNewIV); + + void CleanKey() + { + OPENSSL_cleanse(chKey, sizeof(chKey)); + OPENSSL_cleanse(chIV, sizeof(chIV)); + fKeySet = false; + } + + CCrypter() + { + fKeySet = false; + + // Try to keep the key data out of swap (and be a bit over-careful to keep the IV that we don't even use out of swap) + // Note that this does nothing about suspend-to-disk (which will put all our key data on disk) + // Note as well that at no point in this program is any attempt made to prevent stealing of keys by reading the memory of the running process. + LockedPageManager::Instance().LockRange(&chKey[0], sizeof chKey); + LockedPageManager::Instance().LockRange(&chIV[0], sizeof chIV); + } + + ~CCrypter() + { + CleanKey(); + + LockedPageManager::Instance().UnlockRange(&chKey[0], sizeof chKey); + LockedPageManager::Instance().UnlockRange(&chIV[0], sizeof chIV); + } +}; + +bool EncryptSecret(const CKeyingMaterial& vMasterKey, const CKeyingMaterial &vchPlaintext, const uint256& nIV, std::vector &vchCiphertext); +bool DecryptSecret(const CKeyingMaterial& vMasterKey, const std::vector& vchCiphertext, const uint256& nIV, CKeyingMaterial& vchPlaintext); + +/** Keystore which keeps the private keys encrypted. + * It derives from the basic key store, which is used if no encryption is active. + */ +class CCryptoKeyStore : public CBasicKeyStore +{ +private: + CryptedKeyMap mapCryptedKeys; + + CKeyingMaterial vMasterKey; + + //! if fUseCrypto is true, mapKeys must be empty + //! if fUseCrypto is false, vMasterKey must be empty + bool fUseCrypto; + + //! keeps track of whether Unlock has run a thorough check before + bool fDecryptionThoroughlyChecked; + +protected: + bool SetCrypted(); + + //! will encrypt previously unencrypted keys + bool EncryptKeys(CKeyingMaterial& vMasterKeyIn); + + bool Unlock(const CKeyingMaterial& vMasterKeyIn); + +public: + CCryptoKeyStore() : fUseCrypto(false), fDecryptionThoroughlyChecked(false) + { + } + + bool IsCrypted() const + { + return fUseCrypto; + } + + bool IsLocked() const + { + if (!IsCrypted()) + return false; + bool result; + { + LOCK(cs_KeyStore); + result = vMasterKey.empty(); + } + return result; + } + + bool Lock(); + + virtual bool AddCryptedKey(const CPubKey &vchPubKey, const std::vector &vchCryptedSecret); + bool AddKeyPubKey(const CKey& key, const CPubKey &pubkey); + bool HaveKey(const CKeyID &address) const + { + { + LOCK(cs_KeyStore); + if (!IsCrypted()) + return CBasicKeyStore::HaveKey(address); + return mapCryptedKeys.count(address) > 0; + } + return false; + } + bool GetKey(const CKeyID &address, CKey& keyOut) const; + bool GetPubKey(const CKeyID &address, CPubKey& vchPubKeyOut) const; + void GetKeys(std::set &setAddress) const + { + if (!IsCrypted()) + { + CBasicKeyStore::GetKeys(setAddress); + return; + } + setAddress.clear(); + CryptedKeyMap::const_iterator mi = mapCryptedKeys.begin(); + while (mi != mapCryptedKeys.end()) + { + setAddress.insert((*mi).first); + mi++; + } + } + + /** + * Wallet status (encrypted, locked) changed. + * Note: Called without locks held. + */ + boost::signals2::signal NotifyStatusChanged; +}; + +#endif // BITCOIN_CRYPTER_H diff --git a/src/wallet/db.cpp b/src/wallet/db.cpp new file mode 100644 index 00000000..7421f9d3 --- /dev/null +++ b/src/wallet/db.cpp @@ -0,0 +1,465 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "wallet/db.h" + +#include "storage/addrman.h" +#include "structs/hash.h" +#include "protocol/netprotocol.h" +#include "utils/util.h" +#include "utils/utilstrencodings.h" + +#include + +#ifndef WIN32 +#include +#endif + +#include +#include +#include + +#include + +using namespace std; +using namespace boost; + + +unsigned int nWalletDBUpdated; + + +// +// CDB +// + +CDBEnv bitdb; + +void CDBEnv::EnvShutdown() +{ + if (!fDbEnvInit) + return; +/* MCHN START */ + FILE* fHan; + + dbenv.get_errfile(&fHan); + if(fHan) + { + fclose(fHan); + dbenv.set_errfile(NULL); + } + +/* MCHN END */ + fDbEnvInit = false; + int ret = dbenv.close(0); + if (ret != 0) + LogPrintf("CDBEnv::EnvShutdown : Error %d shutting down database environment: %s\n", ret, DbEnv::strerror(ret)); + if (!fMockDb) + DbEnv(0).remove(path.string().c_str(), 0); +} + +CDBEnv::CDBEnv() : dbenv(DB_CXX_NO_EXCEPTIONS) +{ + fDbEnvInit = false; + fMockDb = false; +} + +CDBEnv::~CDBEnv() +{ + EnvShutdown(); +} + +void CDBEnv::Close() +{ + EnvShutdown(); +} + +bool CDBEnv::Open(const boost::filesystem::path& pathIn) +{ + if (fDbEnvInit) + return true; + + boost::this_thread::interruption_point(); + + path = pathIn; + filesystem::path pathLogDir = path / "database"; + TryCreateDirectory(pathLogDir); + filesystem::path pathErrorFile = path / "db.log"; + LogPrintf("CDBEnv::Open : LogDir=%s ErrorFile=%s\n", pathLogDir.string(), pathErrorFile.string()); + + unsigned int nEnvFlags = 0; + if (GetBoolArg("-privdb", true)) + nEnvFlags |= DB_PRIVATE; + + dbenv.set_lg_dir(pathLogDir.string().c_str()); + dbenv.set_cachesize(0, 0x100000, 1); // 1 MiB should be enough for just the wallet + dbenv.set_lg_bsize(0x10000); + dbenv.set_lg_max(1048576); + dbenv.set_lk_max_locks(40000); + dbenv.set_lk_max_objects(40000); + dbenv.set_errfile(fopen(pathErrorFile.string().c_str(), "a")); /// debug + dbenv.set_flags(DB_AUTO_COMMIT, 1); + dbenv.set_flags(DB_TXN_WRITE_NOSYNC, 1); + dbenv.log_set_config(DB_LOG_AUTO_REMOVE, 1); + int ret = dbenv.open(path.string().c_str(), + DB_CREATE | + DB_INIT_LOCK | + DB_INIT_LOG | + DB_INIT_MPOOL | + DB_INIT_TXN | + DB_THREAD | + DB_RECOVER | + nEnvFlags, + S_IRUSR | S_IWUSR); + if (ret != 0) + return error("CDBEnv::Open : Error %d opening database environment: %s\n", ret, DbEnv::strerror(ret)); + + fDbEnvInit = true; + fMockDb = false; + return true; +} + +void CDBEnv::MakeMock() +{ + if (fDbEnvInit) + throw runtime_error("CDBEnv::MakeMock : Already initialized"); + + boost::this_thread::interruption_point(); + + LogPrint("db", "CDBEnv::MakeMock\n"); + + dbenv.set_cachesize(1, 0, 1); + dbenv.set_lg_bsize(10485760 * 4); + dbenv.set_lg_max(10485760); + dbenv.set_lk_max_locks(10000); + dbenv.set_lk_max_objects(10000); + dbenv.set_flags(DB_AUTO_COMMIT, 1); + dbenv.log_set_config(DB_LOG_IN_MEMORY, 1); + int ret = dbenv.open(NULL, + DB_CREATE | + DB_INIT_LOCK | + DB_INIT_LOG | + DB_INIT_MPOOL | + DB_INIT_TXN | + DB_THREAD | + DB_PRIVATE, + S_IRUSR | S_IWUSR); + if (ret > 0) + throw runtime_error(strprintf("CDBEnv::MakeMock : Error %d opening database environment.", ret)); + + fDbEnvInit = true; + fMockDb = true; +} + +CDBEnv::VerifyResult CDBEnv::Verify(std::string strFile, bool (*recoverFunc)(CDBEnv& dbenv, std::string strFile)) +{ + LOCK(cs_db); + assert(mapFileUseCount.count(strFile) == 0); + + Db db(&dbenv, 0); + int result = db.verify(strFile.c_str(), NULL, NULL, 0); + if (result == 0) + return VERIFY_OK; + else if (recoverFunc == NULL) + return RECOVER_FAIL; + + // Try to recover: + bool fRecovered = (*recoverFunc)(*this, strFile); + return (fRecovered ? RECOVER_OK : RECOVER_FAIL); +} + +bool CDBEnv::Salvage(std::string strFile, bool fAggressive, std::vector& vResult) +{ + LOCK(cs_db); + assert(mapFileUseCount.count(strFile) == 0); + + u_int32_t flags = DB_SALVAGE; + if (fAggressive) + flags |= DB_AGGRESSIVE; + + stringstream strDump; + + Db db(&dbenv, 0); + int result = db.verify(strFile.c_str(), NULL, &strDump, flags); + if (result == DB_VERIFY_BAD) { + LogPrintf("CDBEnv::Salvage : Database salvage found errors, all data may not be recoverable.\n"); + if (!fAggressive) { + LogPrintf("CDBEnv::Salvage : Rerun with aggressive mode to ignore errors and continue.\n"); + return false; + } + } + if (result != 0 && result != DB_VERIFY_BAD) { + LogPrintf("CDBEnv::Salvage : Database salvage failed with result %d.\n", result); + return false; + } + + // Format of bdb dump is ascii lines: + // header lines... + // HEADER=END + // hexadecimal key + // hexadecimal value + // ... repeated + // DATA=END + + string strLine; + while (!strDump.eof() && strLine != "HEADER=END") + getline(strDump, strLine); // Skip past header + + std::string keyHex, valueHex; + while (!strDump.eof() && keyHex != "DATA=END") { + getline(strDump, keyHex); + if (keyHex != "DATA_END") { + getline(strDump, valueHex); + vResult.push_back(make_pair(ParseHex(keyHex), ParseHex(valueHex))); + } + } + + return (result == 0); +} + + +void CDBEnv::CheckpointLSN(const std::string& strFile) +{ + dbenv.txn_checkpoint(0, 0, 0); + if (fMockDb) + return; + dbenv.lsn_reset(strFile.c_str(), 0); +} + + +CDB::CDB(const std::string& strFilename, const char* pszMode) : pdb(NULL), activeTxn(NULL) +{ + int ret; + fReadOnly = (!strchr(pszMode, '+') && !strchr(pszMode, 'w')); + if (strFilename.empty()) + return; + + bool fCreate = strchr(pszMode, 'c') != NULL; + unsigned int nFlags = DB_THREAD; + if (fCreate) + nFlags |= DB_CREATE; + + { + LOCK(bitdb.cs_db); + if (!bitdb.Open(GetDataDir())) + throw runtime_error("CDB : Failed to open database environment."); + + strFile = strFilename; + ++bitdb.mapFileUseCount[strFile]; + pdb = bitdb.mapDb[strFile]; + if (pdb == NULL) { + pdb = new Db(&bitdb.dbenv, 0); + + bool fMockDb = bitdb.IsMock(); + if (fMockDb) { + DbMpoolFile* mpf = pdb->get_mpf(); + ret = mpf->set_flags(DB_MPOOL_NOFILE, 1); + if (ret != 0) + throw runtime_error(strprintf("CDB : Failed to configure for no temp file backing for database %s", strFile)); + } + + ret = pdb->open(NULL, // Txn pointer + fMockDb ? NULL : strFile.c_str(), // Filename + fMockDb ? strFile.c_str() : "main", // Logical db name + DB_BTREE, // Database type + nFlags, // Flags + 0); + + if (ret != 0) { + delete pdb; + pdb = NULL; + --bitdb.mapFileUseCount[strFile]; + strFile = ""; + throw runtime_error(strprintf("CDB : Error %d, can't open database %s", ret, strFile)); + } + + if (fCreate && !Exists(string("version"))) { + bool fTmp = fReadOnly; + fReadOnly = false; + WriteVersion(CLIENT_VERSION); + fReadOnly = fTmp; + } + + bitdb.mapDb[strFile] = pdb; + } + } +} + +void CDB::Flush() +{ + if (activeTxn) + return; + + // Flush database activity from memory pool to disk log + unsigned int nMinutes = 0; + if (fReadOnly) + nMinutes = 1; + + bitdb.dbenv.txn_checkpoint(nMinutes ? GetArg("-dblogsize", 100) * 1024 : 0, nMinutes, 0); +} + +void CDB::Close() +{ + if (!pdb) + return; + if (activeTxn) + activeTxn->abort(); + activeTxn = NULL; + pdb = NULL; + + Flush(); + + { + LOCK(bitdb.cs_db); + --bitdb.mapFileUseCount[strFile]; + } +} + +void CDBEnv::CloseDb(const string& strFile) +{ + { + LOCK(cs_db); + if (mapDb[strFile] != NULL) { + // Close the database handle + Db* pdb = mapDb[strFile]; + pdb->close(0); + delete pdb; + mapDb[strFile] = NULL; + } + } +} + +bool CDBEnv::RemoveDb(const string& strFile) +{ + this->CloseDb(strFile); + + LOCK(cs_db); + int rc = dbenv.dbremove(NULL, strFile.c_str(), NULL, DB_AUTO_COMMIT); + return (rc == 0); +} + +bool CDB::Rewrite(const string& strFile, const char* pszSkip) +{ + while (true) { + { + LOCK(bitdb.cs_db); + if (!bitdb.mapFileUseCount.count(strFile) || bitdb.mapFileUseCount[strFile] == 0) { + // Flush log data to the dat file + bitdb.CloseDb(strFile); + bitdb.CheckpointLSN(strFile); + bitdb.mapFileUseCount.erase(strFile); + + bool fSuccess = true; + LogPrintf("CDB::Rewrite : Rewriting %s...\n", strFile); + string strFileRes = strFile + ".rewrite"; + { // surround usage of db with extra {} + CDB db(strFile.c_str(), "r"); + Db* pdbCopy = new Db(&bitdb.dbenv, 0); + + int ret = pdbCopy->open(NULL, // Txn pointer + strFileRes.c_str(), // Filename + "main", // Logical db name + DB_BTREE, // Database type + DB_CREATE, // Flags + 0); + if (ret > 0) { + LogPrintf("CDB::Rewrite : Can't create database file %s\n", strFileRes); + fSuccess = false; + } + + Dbc* pcursor = db.GetCursor(); + if (pcursor) + while (fSuccess) { + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + CDataStream ssValue(SER_DISK, CLIENT_VERSION); + int ret = db.ReadAtCursor(pcursor, ssKey, ssValue, DB_NEXT); + if (ret == DB_NOTFOUND) { + pcursor->close(); + break; + } else if (ret != 0) { + pcursor->close(); + fSuccess = false; + break; + } + if (pszSkip && + strncmp(&ssKey[0], pszSkip, std::min(ssKey.size(), strlen(pszSkip))) == 0) + continue; + if (strncmp(&ssKey[0], "\x07version", 8) == 0) { + // Update version: + ssValue.clear(); + ssValue << CLIENT_VERSION; + } + Dbt datKey(&ssKey[0], ssKey.size()); + Dbt datValue(&ssValue[0], ssValue.size()); + int ret2 = pdbCopy->put(NULL, &datKey, &datValue, DB_NOOVERWRITE); + if (ret2 > 0) + fSuccess = false; + } + if (fSuccess) { + db.Close(); + bitdb.CloseDb(strFile); + if (pdbCopy->close(0)) + fSuccess = false; + delete pdbCopy; + } + } + if (fSuccess) { + Db dbA(&bitdb.dbenv, 0); + if (dbA.remove(strFile.c_str(), NULL, 0)) + fSuccess = false; + Db dbB(&bitdb.dbenv, 0); + if (dbB.rename(strFileRes.c_str(), NULL, strFile.c_str(), 0)) + fSuccess = false; + } + if (!fSuccess) + LogPrintf("CDB::Rewrite : Failed to rewrite database file %s\n", strFileRes); + return fSuccess; + } + } + MilliSleep(100); + } + return false; +} + + +void CDBEnv::Flush(bool fShutdown) +{ + int64_t nStart = GetTimeMillis(); + // Flush log data to the actual data file on all files that are not in use + LogPrint("db", "CDBEnv::Flush : Flush(%s)%s\n", fShutdown ? "true" : "false", fDbEnvInit ? "" : " database not started"); + if (!fDbEnvInit) + return; + { + LOCK(cs_db); + map::iterator mi = mapFileUseCount.begin(); + while (mi != mapFileUseCount.end()) { + string strFile = (*mi).first; + int nRefCount = (*mi).second; + LogPrint("db", "CDBEnv::Flush : Flushing %s (refcount = %d)...\n", strFile, nRefCount); + if (nRefCount == 0) { + // Move log data to the dat file + CloseDb(strFile); + LogPrint("db", "CDBEnv::Flush : %s checkpoint\n", strFile); + dbenv.txn_checkpoint(0, 0, 0); + LogPrint("db", "CDBEnv::Flush : %s detach\n", strFile); + if (!fMockDb) + dbenv.lsn_reset(strFile.c_str(), 0); + LogPrint("db", "CDBEnv::Flush : %s closed\n", strFile); + mapFileUseCount.erase(mi++); + } else + mi++; + } + LogPrint("db", "CDBEnv::Flush : Flush(%s)%s took %15dms\n", fShutdown ? "true" : "false", fDbEnvInit ? "" : " database not started", GetTimeMillis() - nStart); + if (fShutdown) { + char** listp; + if (mapFileUseCount.empty()) { + dbenv.log_archive(&listp, DB_ARCH_REMOVE); + Close(); + if (!fMockDb) + boost::filesystem::remove_all(path / "database"); + } + } + } +} diff --git a/src/wallet/db.h b/src/wallet/db.h new file mode 100644 index 00000000..0188b29a --- /dev/null +++ b/src/wallet/db.h @@ -0,0 +1,313 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_DB_H +#define BITCOIN_DB_H + +#include "version/clientversion.h" +#include "utils/serialize.h" +#include "utils/streams.h" +#include "utils/sync.h" +#include "version/bcversion.h" + +#include +#include +#include + +#include + +#include + +class CDiskBlockIndex; +class COutPoint; + +struct CBlockLocator; + +extern unsigned int nWalletDBUpdated; + +void ThreadFlushWalletDB(const std::string& strWalletFile); + + +class CDBEnv +{ +private: + bool fDbEnvInit; + bool fMockDb; + boost::filesystem::path path; + + void EnvShutdown(); + +public: + mutable CCriticalSection cs_db; + DbEnv dbenv; + std::map mapFileUseCount; + std::map mapDb; + + CDBEnv(); + ~CDBEnv(); + void MakeMock(); + bool IsMock() { return fMockDb; } + + /** + * Verify that database file strFile is OK. If it is not, + * call the callback to try to recover. + * This must be called BEFORE strFile is opened. + * Returns true if strFile is OK. + */ + enum VerifyResult { VERIFY_OK, + RECOVER_OK, + RECOVER_FAIL }; + VerifyResult Verify(std::string strFile, bool (*recoverFunc)(CDBEnv& dbenv, std::string strFile)); + /** + * Salvage data from a file that Verify says is bad. + * fAggressive sets the DB_AGGRESSIVE flag (see berkeley DB->verify() method documentation). + * Appends binary key/value pairs to vResult, returns true if successful. + * NOTE: reads the entire database into memory, so cannot be used + * for huge databases. + */ + typedef std::pair, std::vector > KeyValPair; + bool Salvage(std::string strFile, bool fAggressive, std::vector& vResult); + + bool Open(const boost::filesystem::path& path); + void Close(); + void Flush(bool fShutdown); + void CheckpointLSN(const std::string& strFile); + + void CloseDb(const std::string& strFile); + bool RemoveDb(const std::string& strFile); + + DbTxn* TxnBegin(int flags = DB_TXN_WRITE_NOSYNC) + { + DbTxn* ptxn = NULL; + int ret = dbenv.txn_begin(NULL, &ptxn, flags); + if (!ptxn || ret != 0) + return NULL; + return ptxn; + } +}; + +extern CDBEnv bitdb; + + +/** RAII class that provides access to a Berkeley database */ +class CDB +{ +protected: + Db* pdb; + std::string strFile; + DbTxn* activeTxn; + bool fReadOnly; + + explicit CDB(const std::string& strFilename, const char* pszMode = "r+"); + ~CDB() { Close(); } + +public: + void Flush(); + void Close(); + +private: + CDB(const CDB&); + void operator=(const CDB&); + +protected: + template + bool Read(const K& key, T& value) + { + if (!pdb) + return false; + + // Key + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + ssKey.reserve(1000); + ssKey << key; + Dbt datKey(&ssKey[0], ssKey.size()); + + // Read + Dbt datValue; + datValue.set_flags(DB_DBT_MALLOC); + int ret = pdb->get(activeTxn, &datKey, &datValue, 0); + memset(datKey.get_data(), 0, datKey.get_size()); + if (datValue.get_data() == NULL) + return false; + + // Unserialize value + try { + CDataStream ssValue((char*)datValue.get_data(), (char*)datValue.get_data() + datValue.get_size(), SER_DISK, CLIENT_VERSION); + ssValue >> value; + } catch (const std::exception&) { + return false; + } + + // Clear and free memory + memset(datValue.get_data(), 0, datValue.get_size()); + free(datValue.get_data()); + return (ret == 0); + } + + template + bool Write(const K& key, const T& value, bool fOverwrite = true) + { + if (!pdb) + return false; + if (fReadOnly) + assert(!"Write called on database in read-only mode"); + + // Key + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + ssKey.reserve(1000); + ssKey << key; + Dbt datKey(&ssKey[0], ssKey.size()); + + // Value + CDataStream ssValue(SER_DISK, CLIENT_VERSION); + ssValue.reserve(10000); + ssValue << value; + Dbt datValue(&ssValue[0], ssValue.size()); + + // Write + int ret = pdb->put(activeTxn, &datKey, &datValue, (fOverwrite ? 0 : DB_NOOVERWRITE)); + + // Clear memory in case it was a private key + memset(datKey.get_data(), 0, datKey.get_size()); + memset(datValue.get_data(), 0, datValue.get_size()); + return (ret == 0); + } + + template + bool Erase(const K& key) + { + if (!pdb) + return false; + if (fReadOnly) + assert(!"Erase called on database in read-only mode"); + + // Key + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + ssKey.reserve(1000); + ssKey << key; + Dbt datKey(&ssKey[0], ssKey.size()); + + // Erase + int ret = pdb->del(activeTxn, &datKey, 0); + + // Clear memory + memset(datKey.get_data(), 0, datKey.get_size()); + return (ret == 0 || ret == DB_NOTFOUND); + } + + template + bool Exists(const K& key) + { + if (!pdb) + return false; + + // Key + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + ssKey.reserve(1000); + ssKey << key; + Dbt datKey(&ssKey[0], ssKey.size()); + + // Exists + int ret = pdb->exists(activeTxn, &datKey, 0); + + // Clear memory + memset(datKey.get_data(), 0, datKey.get_size()); + return (ret == 0); + } + + Dbc* GetCursor() + { + if (!pdb) + return NULL; + Dbc* pcursor = NULL; + int ret = pdb->cursor(NULL, &pcursor, 0); + if (ret != 0) + return NULL; + return pcursor; + } + + int ReadAtCursor(Dbc* pcursor, CDataStream& ssKey, CDataStream& ssValue, unsigned int fFlags = DB_NEXT) + { + // Read at cursor + Dbt datKey; + if (fFlags == DB_SET || fFlags == DB_SET_RANGE || fFlags == DB_GET_BOTH || fFlags == DB_GET_BOTH_RANGE) { + datKey.set_data(&ssKey[0]); + datKey.set_size(ssKey.size()); + } + Dbt datValue; + if (fFlags == DB_GET_BOTH || fFlags == DB_GET_BOTH_RANGE) { + datValue.set_data(&ssValue[0]); + datValue.set_size(ssValue.size()); + } + datKey.set_flags(DB_DBT_MALLOC); + datValue.set_flags(DB_DBT_MALLOC); + int ret = pcursor->get(&datKey, &datValue, fFlags); + if (ret != 0) + return ret; + else if (datKey.get_data() == NULL || datValue.get_data() == NULL) + return 99999; + + // Convert to streams + ssKey.SetType(SER_DISK); + ssKey.clear(); + ssKey.write((char*)datKey.get_data(), datKey.get_size()); + ssValue.SetType(SER_DISK); + ssValue.clear(); + ssValue.write((char*)datValue.get_data(), datValue.get_size()); + + // Clear and free memory + memset(datKey.get_data(), 0, datKey.get_size()); + memset(datValue.get_data(), 0, datValue.get_size()); + free(datKey.get_data()); + free(datValue.get_data()); + return 0; + } + +public: + bool TxnBegin() + { + if (!pdb || activeTxn) + return false; + DbTxn* ptxn = bitdb.TxnBegin(); + if (!ptxn) + return false; + activeTxn = ptxn; + return true; + } + + bool TxnCommit() + { + if (!pdb || !activeTxn) + return false; + int ret = activeTxn->commit(0); + activeTxn = NULL; + return (ret == 0); + } + + bool TxnAbort() + { + if (!pdb || !activeTxn) + return false; + int ret = activeTxn->abort(); + activeTxn = NULL; + return (ret == 0); + } + + bool ReadVersion(int& nVersion) + { + nVersion = 0; + return Read(std::string("version"), nVersion); + } + + bool WriteVersion(int nVersion) + { + return Write(std::string("version"), nVersion); + } + + bool static Rewrite(const std::string& strFile, const char* pszSkip = NULL); +}; + +#endif // BITCOIN_DB_H diff --git a/src/wallet/keystore.cpp b/src/wallet/keystore.cpp new file mode 100644 index 00000000..3b448e75 --- /dev/null +++ b/src/wallet/keystore.cpp @@ -0,0 +1,93 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "wallet/keystore.h" + +#include "wallet/crypter.h" +#include "keys/key.h" +#include "script/script.h" +#include "script/standard.h" +#include "utils/util.h" +#include "chainparams/chainparams.h" + +#include + +bool CKeyStore::GetPubKey(const CKeyID &address, CPubKey &vchPubKeyOut) const +{ + CKey key; + if (!GetKey(address, key)) + return false; + vchPubKeyOut = key.GetPubKey(); + return true; +} + +bool CKeyStore::AddKey(const CKey &key) { + return AddKeyPubKey(key, key.GetPubKey()); +} + +bool CBasicKeyStore::AddKeyPubKey(const CKey& key, const CPubKey &pubkey) +{ + LOCK(cs_KeyStore); + mapKeys[pubkey.GetID()] = key; + return true; +} + +bool CBasicKeyStore::AddCScript(const CScript& redeemScript) +{ + if(Params().RequireStandard()) + { + if (redeemScript.size() > MAX_SCRIPT_ELEMENT_SIZE) + return error("CBasicKeyStore::AddCScript() : redeemScripts > %i bytes are invalid", MAX_SCRIPT_ELEMENT_SIZE); + } + + LOCK(cs_KeyStore); + mapScripts[CScriptID(redeemScript)] = redeemScript; + return true; +} + +bool CBasicKeyStore::HaveCScript(const CScriptID& hash) const +{ + LOCK(cs_KeyStore); + return mapScripts.count(hash) > 0; +} + +bool CBasicKeyStore::GetCScript(const CScriptID &hash, CScript& redeemScriptOut) const +{ + LOCK(cs_KeyStore); + ScriptMap::const_iterator mi = mapScripts.find(hash); + if (mi != mapScripts.end()) + { + redeemScriptOut = (*mi).second; + return true; + } + return false; +} + +bool CBasicKeyStore::AddWatchOnly(const CScript &dest) +{ + LOCK(cs_KeyStore); + setWatchOnly.insert(dest); + return true; +} + +bool CBasicKeyStore::RemoveWatchOnly(const CScript &dest) +{ + LOCK(cs_KeyStore); + setWatchOnly.erase(dest); + return true; +} + +bool CBasicKeyStore::HaveWatchOnly(const CScript &dest) const +{ + LOCK(cs_KeyStore); + return setWatchOnly.count(dest) > 0; +} + +bool CBasicKeyStore::HaveWatchOnly() const +{ + LOCK(cs_KeyStore); + return (!setWatchOnly.empty()); +} diff --git a/src/wallet/keystore.h b/src/wallet/keystore.h new file mode 100644 index 00000000..baa5ac94 --- /dev/null +++ b/src/wallet/keystore.h @@ -0,0 +1,113 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_KEYSTORE_H +#define BITCOIN_KEYSTORE_H + +#include "keys/key.h" +#include "keys/pubkey.h" +#include "utils/sync.h" + +#include +#include + +class CScript; +class CScriptID; + +/** A virtual base class for key stores */ +class CKeyStore +{ +protected: + mutable CCriticalSection cs_KeyStore; + +public: + virtual ~CKeyStore() {} + + //! Add a key to the store. + virtual bool AddKeyPubKey(const CKey &key, const CPubKey &pubkey) =0; + virtual bool AddKey(const CKey &key); + + //! Check whether a key corresponding to a given address is present in the store. + virtual bool HaveKey(const CKeyID &address) const =0; + virtual bool GetKey(const CKeyID &address, CKey& keyOut) const =0; + virtual void GetKeys(std::set &setAddress) const =0; + virtual bool GetPubKey(const CKeyID &address, CPubKey& vchPubKeyOut) const; + + //! Support for BIP 0013 : see https://github.com/bitcoin/bips/blob/master/bip-0013.mediawiki + virtual bool AddCScript(const CScript& redeemScript) =0; + virtual bool HaveCScript(const CScriptID &hash) const =0; + virtual bool GetCScript(const CScriptID &hash, CScript& redeemScriptOut) const =0; + + //! Support for Watch-only addresses + virtual bool AddWatchOnly(const CScript &dest) =0; + virtual bool RemoveWatchOnly(const CScript &dest) =0; + virtual bool HaveWatchOnly(const CScript &dest) const =0; + virtual bool HaveWatchOnly() const =0; +}; + +typedef std::map KeyMap; +typedef std::map ScriptMap; +typedef std::set WatchOnlySet; + +/** Basic key store, that keeps keys in an address->secret map */ +class CBasicKeyStore : public CKeyStore +{ +protected: + KeyMap mapKeys; + ScriptMap mapScripts; + WatchOnlySet setWatchOnly; + +public: + bool AddKeyPubKey(const CKey& key, const CPubKey &pubkey); + bool HaveKey(const CKeyID &address) const + { + bool result; + { + LOCK(cs_KeyStore); + result = (mapKeys.count(address) > 0); + } + return result; + } + void GetKeys(std::set &setAddress) const + { + setAddress.clear(); + { + LOCK(cs_KeyStore); + KeyMap::const_iterator mi = mapKeys.begin(); + while (mi != mapKeys.end()) + { + setAddress.insert((*mi).first); + mi++; + } + } + } + bool GetKey(const CKeyID &address, CKey &keyOut) const + { + { + LOCK(cs_KeyStore); + KeyMap::const_iterator mi = mapKeys.find(address); + if (mi != mapKeys.end()) + { + keyOut = mi->second; + return true; + } + } + return false; + } + virtual bool AddCScript(const CScript& redeemScript); + virtual bool HaveCScript(const CScriptID &hash) const; + virtual bool GetCScript(const CScriptID &hash, CScript& redeemScriptOut) const; + + virtual bool AddWatchOnly(const CScript &dest); + virtual bool RemoveWatchOnly(const CScript &dest); + virtual bool HaveWatchOnly(const CScript &dest) const; + virtual bool HaveWatchOnly() const; +}; + +typedef std::vector > CKeyingMaterial; +typedef std::map > > CryptedKeyMap; + +#endif // BITCOIN_KEYSTORE_H diff --git a/src/wallet/wallet.cpp b/src/wallet/wallet.cpp new file mode 100644 index 00000000..57125b9c --- /dev/null +++ b/src/wallet/wallet.cpp @@ -0,0 +1,4075 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "wallet/wallet.h" +/* MCHN START */ +#include "wallet/wallettxs.h" +extern mc_WalletTxs* pwalletTxsMain; +/* MCHN END */ + +#include "structs/base58.h" +#include "chain/checkpoints.h" +#include "wallet/coincontrol.h" +#include "net/net.h" +#include "script/script.h" +#include "script/sign.h" +#include "utils/timedata.h" +#include "utils/util.h" +#include "utils/utilmoneystr.h" + +#include + +#include +#include + +using namespace std; + +/** + * Settings + */ +CFeeRate payTxFee(DEFAULT_TRANSACTION_FEE); +CAmount maxTxFee = DEFAULT_TRANSACTION_MAXFEE; +unsigned int nTxConfirmTarget = 1; +bool bSpendZeroConfChange = true; +bool fSendFreeTransactions = false; +bool fPayAtLeastCustomFee = true; + +/** + * Fees smaller than this (in satoshi) are considered zero fee (for transaction creation) + * Override with -mintxfee + */ + +/* MCHN START */ +//CFeeRate CWallet::minTxFee = CFeeRate(1000); +CFeeRate CWallet::minTxFee = CFeeRate(MIN_RELAY_TX_FEE); +bool OutputCanSend(COutput out); + +struct CompareValueOnlyIntDesc +{ + bool operator()(const pair >& t1, + const pair >& t2) const + { + return t1.first > t2.first; + } +}; + +int64_t mc_GetABCoinQuantity(void *ptr,int coin_id); +void mc_SetABCoinQuantity(void *ptr,int coin_id,int64_t quantity); + +/* MCHN END */ + + + + +/** @defgroup mapWallet + * + * @{ + */ + +struct CompareValueOnly +{ + bool operator()(const pair >& t1, + const pair >& t2) const + { + return t1.first < t2.first; + } +}; +struct CompareValueOnlyHash +{ + bool operator()(const pair >& t1, + const pair >& t2) const + { + return t1.first < t2.first; + } +}; + + +const CWalletTx* CWallet::GetWalletTx(const uint256& hash) const +{ + LOCK(cs_wallet); +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported as returns pointer + + std::map::const_iterator it = mapWallet.find(hash); + if (it == mapWallet.end()) + return NULL; + return &(it->second); +} + +CPubKey CWallet::GenerateNewKey() +{ + AssertLockHeld(cs_wallet); // mapKeyMetadata + bool fCompressed = CanSupportFeature(FEATURE_COMPRPUBKEY); // default to compressed public keys if we want 0.6.0 wallets + +// RandAddSeedPerfmon(); // MCHN removed in 146c0a7 when merging sec256k1 + CKey secret; + secret.MakeNewKey(fCompressed); + + // Compressed public keys were introduced in version 0.6.0 + if (fCompressed) + SetMinVersion(FEATURE_COMPRPUBKEY); + + CPubKey pubkey = secret.GetPubKey(); + assert(secret.VerifyPubKey(pubkey)); + + // Create new metadata + int64_t nCreationTime = GetTime(); + mapKeyMetadata[pubkey.GetID()] = CKeyMetadata(nCreationTime); + if (!nTimeFirstKey || nCreationTime < nTimeFirstKey) + nTimeFirstKey = nCreationTime; + + if (!AddKeyPubKey(secret, pubkey)) + throw std::runtime_error("CWallet::GenerateNewKey() : AddKey failed"); + return pubkey; +} + +bool CWallet::AddKeyPubKey(const CKey& secret, const CPubKey &pubkey) +{ + AssertLockHeld(cs_wallet); // mapKeyMetadata + if (!CCryptoKeyStore::AddKeyPubKey(secret, pubkey)) + return false; + + // check if we need to remove from watch-only + CScript script; + script = GetScriptForDestination(pubkey.GetID()); + if (HaveWatchOnly(script)) + RemoveWatchOnly(script); + + if (!fFileBacked) + return true; + if (!IsCrypted()) { + return CWalletDB(strWalletFile).WriteKey(pubkey, + secret.GetPrivKey(), + mapKeyMetadata[pubkey.GetID()]); + } + return true; +} + +bool CWallet::AddCryptedKey(const CPubKey &vchPubKey, + const vector &vchCryptedSecret) +{ + if (!CCryptoKeyStore::AddCryptedKey(vchPubKey, vchCryptedSecret)) + return false; + if (!fFileBacked) + return true; + { + LOCK(cs_wallet); + if (pwalletdbEncryption) + return pwalletdbEncryption->WriteCryptedKey(vchPubKey, + vchCryptedSecret, + mapKeyMetadata[vchPubKey.GetID()]); + else + return CWalletDB(strWalletFile).WriteCryptedKey(vchPubKey, + vchCryptedSecret, + mapKeyMetadata[vchPubKey.GetID()]); + } + return false; +} + +bool CWallet::LoadKeyMetadata(const CPubKey &pubkey, const CKeyMetadata &meta) +{ + AssertLockHeld(cs_wallet); // mapKeyMetadata + if (meta.nCreateTime && (!nTimeFirstKey || meta.nCreateTime < nTimeFirstKey)) + nTimeFirstKey = meta.nCreateTime; + + mapKeyMetadata[pubkey.GetID()] = meta; + return true; +} + +bool CWallet::LoadCryptedKey(const CPubKey &vchPubKey, const std::vector &vchCryptedSecret) +{ + return CCryptoKeyStore::AddCryptedKey(vchPubKey, vchCryptedSecret); +} + +bool CWallet::AddCScript(const CScript& redeemScript) +{ + if (!CCryptoKeyStore::AddCScript(redeemScript)) + return false; + if (!fFileBacked) + return true; + return CWalletDB(strWalletFile).WriteCScript(Hash160(redeemScript), redeemScript); +} + +bool CWallet::LoadCScript(const CScript& redeemScript) +{ + /* A sanity check was added in pull #3843 to avoid adding redeemScripts + * that never can be redeemed. However, old wallets may still contain + * these. Do not add them to the wallet and warn. */ + if (redeemScript.size() > MAX_SCRIPT_ELEMENT_SIZE) + { + std::string strAddr = CBitcoinAddress(CScriptID(redeemScript)).ToString(); + LogPrintf("%s: Warning: This wallet contains a redeemScript of size %i which exceeds maximum size %i thus can never be redeemed. Do not use address %s.\n", + __func__, redeemScript.size(), MAX_SCRIPT_ELEMENT_SIZE, strAddr); + return true; + } + + return CCryptoKeyStore::AddCScript(redeemScript); +} + +bool CWallet::AddWatchOnly(const CScript &dest) +{ + if (!CCryptoKeyStore::AddWatchOnly(dest)) + return false; + nTimeFirstKey = 1; // No birthday information for watch-only keys. + NotifyWatchonlyChanged(true); + if (!fFileBacked) + return true; + return CWalletDB(strWalletFile).WriteWatchOnly(dest); +} + +bool CWallet::RemoveWatchOnly(const CScript &dest) +{ + AssertLockHeld(cs_wallet); + if (!CCryptoKeyStore::RemoveWatchOnly(dest)) + return false; + if (!HaveWatchOnly()) + NotifyWatchonlyChanged(false); + if (fFileBacked) + if (!CWalletDB(strWalletFile).EraseWatchOnly(dest)) + return false; + + return true; +} + +bool CWallet::LoadWatchOnly(const CScript &dest) +{ + return CCryptoKeyStore::AddWatchOnly(dest); +} + +bool CWallet::Unlock(const SecureString& strWalletPassphrase) +{ + CCrypter crypter; + CKeyingMaterial vMasterKey; + + { + LOCK(cs_wallet); + BOOST_FOREACH(const MasterKeyMap::value_type& pMasterKey, mapMasterKeys) + { + if(!crypter.SetKeyFromPassphrase(strWalletPassphrase, pMasterKey.second.vchSalt, pMasterKey.second.nDeriveIterations, pMasterKey.second.nDerivationMethod)) + return false; + if (!crypter.Decrypt(pMasterKey.second.vchCryptedKey, vMasterKey)) + continue; // try another master key + if (CCryptoKeyStore::Unlock(vMasterKey)) + return true; + } + } + return false; +} + +bool CWallet::ChangeWalletPassphrase(const SecureString& strOldWalletPassphrase, const SecureString& strNewWalletPassphrase) +{ + bool fWasLocked = IsLocked(); + + { + LOCK(cs_wallet); + Lock(); + + CCrypter crypter; + CKeyingMaterial vMasterKey; + BOOST_FOREACH(MasterKeyMap::value_type& pMasterKey, mapMasterKeys) + { + if(!crypter.SetKeyFromPassphrase(strOldWalletPassphrase, pMasterKey.second.vchSalt, pMasterKey.second.nDeriveIterations, pMasterKey.second.nDerivationMethod)) + return false; + if (!crypter.Decrypt(pMasterKey.second.vchCryptedKey, vMasterKey)) + return false; + if (CCryptoKeyStore::Unlock(vMasterKey)) + { + int64_t nStartTime = GetTimeMillis(); + crypter.SetKeyFromPassphrase(strNewWalletPassphrase, pMasterKey.second.vchSalt, pMasterKey.second.nDeriveIterations, pMasterKey.second.nDerivationMethod); + pMasterKey.second.nDeriveIterations = pMasterKey.second.nDeriveIterations * (100 / ((double)(GetTimeMillis() - nStartTime))); + + nStartTime = GetTimeMillis(); + crypter.SetKeyFromPassphrase(strNewWalletPassphrase, pMasterKey.second.vchSalt, pMasterKey.second.nDeriveIterations, pMasterKey.second.nDerivationMethod); + pMasterKey.second.nDeriveIterations = (pMasterKey.second.nDeriveIterations + pMasterKey.second.nDeriveIterations * 100 / ((double)(GetTimeMillis() - nStartTime))) / 2; + + if (pMasterKey.second.nDeriveIterations < 25000) + pMasterKey.second.nDeriveIterations = 25000; + + LogPrintf("Wallet passphrase changed to an nDeriveIterations of %i\n", pMasterKey.second.nDeriveIterations); + + if (!crypter.SetKeyFromPassphrase(strNewWalletPassphrase, pMasterKey.second.vchSalt, pMasterKey.second.nDeriveIterations, pMasterKey.second.nDerivationMethod)) + return false; + if (!crypter.Encrypt(vMasterKey, pMasterKey.second.vchCryptedKey)) + return false; + CWalletDB(strWalletFile).WriteMasterKey(pMasterKey.first, pMasterKey.second); + if (fWasLocked) + Lock(); + return true; + } + } + } + + return false; +} + +void CWallet::SetBestChain(const CBlockLocator& loc) +{ + CWalletDB walletdb(strWalletFile); + walletdb.WriteBestBlock(loc); +} + +bool CWallet::SetMinVersion(enum WalletFeature nVersion, CWalletDB* pwalletdbIn, bool fExplicit) +{ + LOCK(cs_wallet); // nWalletVersion + if (nWalletVersion >= nVersion) + return true; + + // when doing an explicit upgrade, if we pass the max version permitted, upgrade all the way + if (fExplicit && nVersion > nWalletMaxVersion) + nVersion = FEATURE_LATEST; + + nWalletVersion = nVersion; + + if (nVersion > nWalletMaxVersion) + nWalletMaxVersion = nVersion; + + if (fFileBacked) + { + CWalletDB* pwalletdb = pwalletdbIn ? pwalletdbIn : new CWalletDB(strWalletFile); + if (nWalletVersion > 40000) + pwalletdb->WriteMinVersion(nWalletVersion); + if (!pwalletdbIn) + delete pwalletdb; + } + + return true; +} + +bool CWallet::SetMaxVersion(int nVersion) +{ + LOCK(cs_wallet); // nWalletVersion, nWalletMaxVersion + // cannot downgrade below current version + if (nWalletVersion > nVersion) + return false; + + nWalletMaxVersion = nVersion; + + return true; +} + +set CWallet::GetConflicts(const uint256& txid) const +{ + set result; +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported as Spent array is no longer maintained + + AssertLockHeld(cs_wallet); + + std::map::const_iterator it = mapWallet.find(txid); + if (it == mapWallet.end()) + return result; + const CWalletTx& wtx = it->second; + + std::pair range; + + BOOST_FOREACH(const CTxIn& txin, wtx.vin) + { + if (mapTxSpends.count(txin.prevout) <= 1) + continue; // No conflict if zero or one spends + range = mapTxSpends.equal_range(txin.prevout); + for (TxSpends::const_iterator it = range.first; it != range.second; ++it) + result.insert(it->second); + } + return result; +} + +void CWallet::SyncMetaData(pair range) +{ + // We want all the wallet transactions in range to have the same metadata as + // the oldest (smallest nOrderPos). + // So: find smallest nOrderPos: +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported as Spent array is no longer maintained + + int nMinOrderPos = std::numeric_limits::max(); + const CWalletTx* copyFrom = NULL; + for (TxSpends::iterator it = range.first; it != range.second; ++it) + { + const uint256& hash = it->second; + int n = mapWallet[hash].nOrderPos; + if (n < nMinOrderPos) + { + nMinOrderPos = n; + copyFrom = &mapWallet[hash]; + } + } + // Now copy data from copyFrom to rest: + for (TxSpends::iterator it = range.first; it != range.second; ++it) + { + const uint256& hash = it->second; + CWalletTx* copyTo = &mapWallet[hash]; + if (copyFrom == copyTo) continue; + copyTo->mapValue = copyFrom->mapValue; + copyTo->vOrderForm = copyFrom->vOrderForm; + // fTimeReceivedIsTxTime not copied on purpose + // nTimeReceived not copied on purpose + copyTo->nTimeSmart = copyFrom->nTimeSmart; + copyTo->fFromMe = copyFrom->fFromMe; + copyTo->strFromAccount = copyFrom->strFromAccount; + // nOrderPos not copied on purpose + // cached members not copied on purpose + } +} + +/** + * Outpoint is spent if any non-conflicted transaction + * spends it: + */ +bool CWallet::IsSpent(const uint256& hash, unsigned int n) const +{ +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported as Spends array not maintained + const COutPoint outpoint(hash, n); + pair range; + range = mapTxSpends.equal_range(outpoint); + + for (TxSpends::const_iterator it = range.first; it != range.second; ++it) + { + const uint256& wtxid = it->second; +/* MCHN START */ + if(setPurged.count(wtxid)) + return true; +/* MCHN END */ + std::map::const_iterator mit = mapWallet.find(wtxid); + if (mit != mapWallet.end() && mit->second.GetDepthInMainChain() >= 0) + return true; // Spent + } + return false; +} + +void CWallet::AddToSpends(const COutPoint& outpoint, const uint256& wtxid) +{ + mapTxSpends.insert(make_pair(outpoint, wtxid)); + + pair range; + range = mapTxSpends.equal_range(outpoint); + SyncMetaData(range); +} + + +void CWallet::AddToSpends(const uint256& wtxid) +{ +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported as Spends array not maintained + assert(mapWallet.count(wtxid)); + CWalletTx& thisTx = mapWallet[wtxid]; + if (thisTx.IsCoinBase()) // Coinbases don't spend anything! + return; + + BOOST_FOREACH(const CTxIn& txin, thisTx.vin) + AddToSpends(txin.prevout, wtxid); +} + +bool CWallet::EncryptWallet(const SecureString& strWalletPassphrase) +{ + if (IsCrypted()) + return false; + + CKeyingMaterial vMasterKey; + RandAddSeedPerfmon(); + + vMasterKey.resize(WALLET_CRYPTO_KEY_SIZE); + GetRandBytes(&vMasterKey[0], WALLET_CRYPTO_KEY_SIZE); + + CMasterKey kMasterKey; + RandAddSeedPerfmon(); + + kMasterKey.vchSalt.resize(WALLET_CRYPTO_SALT_SIZE); + GetRandBytes(&kMasterKey.vchSalt[0], WALLET_CRYPTO_SALT_SIZE); + + CCrypter crypter; + int64_t nStartTime = GetTimeMillis(); + crypter.SetKeyFromPassphrase(strWalletPassphrase, kMasterKey.vchSalt, 25000, kMasterKey.nDerivationMethod); + kMasterKey.nDeriveIterations = 2500000 / ((double)(GetTimeMillis() - nStartTime)); + + nStartTime = GetTimeMillis(); + crypter.SetKeyFromPassphrase(strWalletPassphrase, kMasterKey.vchSalt, kMasterKey.nDeriveIterations, kMasterKey.nDerivationMethod); + kMasterKey.nDeriveIterations = (kMasterKey.nDeriveIterations + kMasterKey.nDeriveIterations * 100 / ((double)(GetTimeMillis() - nStartTime))) / 2; + + if (kMasterKey.nDeriveIterations < 25000) + kMasterKey.nDeriveIterations = 25000; + + LogPrintf("Encrypting Wallet with an nDeriveIterations of %i\n", kMasterKey.nDeriveIterations); + + if (!crypter.SetKeyFromPassphrase(strWalletPassphrase, kMasterKey.vchSalt, kMasterKey.nDeriveIterations, kMasterKey.nDerivationMethod)) + return false; + if (!crypter.Encrypt(vMasterKey, kMasterKey.vchCryptedKey)) + return false; + + { + LOCK(cs_wallet); + mapMasterKeys[++nMasterKeyMaxID] = kMasterKey; + if (fFileBacked) + { + assert(!pwalletdbEncryption); + pwalletdbEncryption = new CWalletDB(strWalletFile); + if (!pwalletdbEncryption->TxnBegin()) { + delete pwalletdbEncryption; + pwalletdbEncryption = NULL; + return false; + } + pwalletdbEncryption->WriteMasterKey(nMasterKeyMaxID, kMasterKey); + } + + if (!EncryptKeys(vMasterKey)) + { + if (fFileBacked) { + pwalletdbEncryption->TxnAbort(); + delete pwalletdbEncryption; + } + // We now probably have half of our keys encrypted in memory, and half not... + // die and let the user reload their unencrypted wallet. + assert(false); + } + + // Encryption was introduced in version 0.4.0 + SetMinVersion(FEATURE_WALLETCRYPT, pwalletdbEncryption, true); + + if (fFileBacked) + { + if (!pwalletdbEncryption->TxnCommit()) { + delete pwalletdbEncryption; + // We now have keys encrypted in memory, but not on disk... + // die to avoid confusion and let the user reload their unencrypted wallet. + assert(false); + } + + delete pwalletdbEncryption; + pwalletdbEncryption = NULL; + } + + Lock(); + Unlock(strWalletPassphrase); + NewKeyPool(); + Lock(); + + // Need to completely rewrite the wallet file; if we don't, bdb might keep + // bits of the unencrypted private key in slack space in the database file. + CDB::Rewrite(strWalletFile); + + } + NotifyStatusChanged(this); + + return true; +} + +int64_t CWallet::IncOrderPosNext(CWalletDB *pwalletdb) +{ + AssertLockHeld(cs_wallet); // nOrderPosNext + int64_t nRet = nOrderPosNext++; + if (pwalletdb) { + pwalletdb->WriteOrderPosNext(nOrderPosNext); + } else { + CWalletDB(strWalletFile).WriteOrderPosNext(nOrderPosNext); + } + return nRet; +} + +CWallet::TxItems CWallet::OrderedTxItems(std::list& acentries, std::string strAccount) +{ + AssertLockHeld(cs_wallet); // mapWallet + CWalletDB walletdb(strWalletFile); + + // First: get all CWalletTx and CAccountingEntry into a sorted-by-order multimap. + TxItems txOrdered; + +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, not used in this case + + // Note: maintaining indices in the database of (account,time) --> txid and (account, time) --> acentry + // would make this much faster for applications that do this a lot. + for (map::iterator it = mapWallet.begin(); it != mapWallet.end(); ++it) + { + CWalletTx* wtx = &((*it).second); + txOrdered.insert(make_pair(wtx->nOrderPos, TxPair(wtx, (CAccountingEntry*)0))); + } + acentries.clear(); + walletdb.ListAccountCreditDebit(strAccount, acentries); + BOOST_FOREACH(CAccountingEntry& entry, acentries) + { + txOrdered.insert(make_pair(entry.nOrderPos, TxPair((CWalletTx*)0, &entry))); + } + + return txOrdered; +} + +void CWallet::MarkDirty() +{ +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, not used in this case + { + LOCK(cs_wallet); + BOOST_FOREACH(PAIRTYPE(const uint256, CWalletTx)& item, mapWallet) + item.second.MarkDirty(); + } +} + +bool CWallet::AddToWallet(const CWalletTx& wtxIn, bool fFromLoadWallet) +{ + uint256 hash = wtxIn.GetHash(); +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, not used in this case + + if (fFromLoadWallet) + { + mapWallet[hash] = wtxIn; + mapWallet[hash].BindWallet(this); + AddToSpends(hash); + } + else + { + LOCK(cs_wallet); + // Inserts only if not already there, returns tx inserted or tx found + pair::iterator, bool> ret = mapWallet.insert(make_pair(hash, wtxIn)); + CWalletTx& wtx = (*ret.first).second; + wtx.BindWallet(this); + bool fInsertedNew = ret.second; + if (fInsertedNew) + { + wtx.nTimeReceived = GetAdjustedTime(); +/* MCHN START */ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxDefRow txdef; + if(pwalletTxsMain->FindWalletTx(wtx.GetHash(),&txdef) == MC_ERR_NOERROR) + { + wtx.nTimeReceived=txdef.m_TimeReceived; + } + } +/* MCHN END */ + wtx.nOrderPos = IncOrderPosNext(); + + wtx.nTimeSmart = wtx.nTimeReceived; + if (wtxIn.hashBlock != 0) + { + if (mapBlockIndex.count(wtxIn.hashBlock)) + { + int64_t latestNow = wtx.nTimeReceived; + int64_t latestEntry = 0; +/* MCHN START */ +/* + { + // Tolerate times up to the last timestamp in the wallet not more than 5 minutes into the future + int64_t latestTolerated = latestNow + 300; + std::list acentries; + TxItems txOrdered = OrderedTxItems(acentries); + for (TxItems::reverse_iterator it = txOrdered.rbegin(); it != txOrdered.rend(); ++it) + { + CWalletTx *const pwtx = (*it).second.first; + if (pwtx == &wtx) + continue; + CAccountingEntry *const pacentry = (*it).second.second; + int64_t nSmartTime; + if (pwtx) + { + nSmartTime = pwtx->nTimeSmart; + if (!nSmartTime) + nSmartTime = pwtx->nTimeReceived; + } + else + nSmartTime = pacentry->nTime; + if (nSmartTime <= latestTolerated) + { + latestEntry = nSmartTime; + if (nSmartTime > latestNow) + latestNow = nSmartTime; + break; + } + } + } + */ +/* MCHN END */ + int64_t blocktime = mapBlockIndex[wtxIn.hashBlock]->GetBlockTime(); + wtx.nTimeSmart = std::max(latestEntry, std::min(blocktime, latestNow)); + } + else + LogPrintf("AddToWallet() : found %s in block %s not in index\n", + wtxIn.GetHash().ToString(), + wtxIn.hashBlock.ToString()); + } + AddToSpends(hash); + } + + bool fUpdated = false; + if (!fInsertedNew) + { + // Merge + if (wtxIn.hashBlock != 0 && wtxIn.hashBlock != wtx.hashBlock) + { + wtx.hashBlock = wtxIn.hashBlock; + fUpdated = true; + } + if (wtxIn.nIndex != -1 && (wtxIn.vMerkleBranch != wtx.vMerkleBranch || wtxIn.nIndex != wtx.nIndex)) + { + wtx.vMerkleBranch = wtxIn.vMerkleBranch; + wtx.nIndex = wtxIn.nIndex; + fUpdated = true; + } + if (wtxIn.fFromMe && wtxIn.fFromMe != wtx.fFromMe) + { + wtx.fFromMe = wtxIn.fFromMe; + fUpdated = true; + } +/* MCHN START */ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxDefRow txdef; + if(pwalletTxsMain->FindWalletTx(wtx.GetHash(),&txdef) == MC_ERR_NOERROR) + { + if(wtx.nTimeReceived != txdef.m_TimeReceived) + { + wtx.nTimeReceived=txdef.m_TimeReceived; + wtx.nTimeSmart=txdef.m_TimeReceived; + fUpdated = true; + } + } + } +/* MCHN END */ + } + +/* MCHN START */ +// LogPrintf("AddToWallet %s %s%s; Time \n", wtxIn.GetHash().ToString(), (fInsertedNew ? "new" : ""), (fUpdated ? "update" : "")); +/* MCHN END */ + + // Write to disk + if (fInsertedNew || fUpdated) + if (!wtx.WriteToDisk()) + return false; + + + // Break debit/credit balance caches: + wtx.MarkDirty(); + + // Notify UI of new or updated transaction + NotifyTransactionChanged(this, hash, fInsertedNew ? CT_NEW : CT_UPDATED); + + // notify an external script when a wallet transaction comes in or is updated + std::string strCmd = GetArg("-walletnotify", ""); + + if ( !strCmd.empty()) + { + boost::replace_all(strCmd, "%s", wtxIn.GetHash().GetHex()); + boost::thread t(runCommand, strCmd); // thread runs free + } + + UpdateUnspentList(wtx,true); + } + + + return true; +} + +/** + * Add a transaction to the wallet, or update it. + * pblock is optional, but should be provided if the transaction is known to be in a block. + * If fUpdate is true, existing transactions will be updated. + */ +bool CWallet::AddToWalletIfInvolvingMe(const CTransaction& tx, const CBlock* pblock, bool fUpdate) +{ + { + AssertLockHeld(cs_wallet); +/* MCHN START */ + if(pblock) + { + if(pblock->GetHash() == Params().HashGenesisBlock()) + { + return false; + } + } +/* MCHN END */ + bool fExisted = mapWallet.count(tx.GetHash()) != 0; + if (fExisted && !fUpdate) return false; + if (fExisted || IsMine(tx) || IsFromMe(tx)) + { + CWalletTx wtx(this,tx); + // Get merkle branch if transaction was found in a block + if (pblock) + wtx.SetMerkleBranch(*pblock); + return AddToWallet(wtx); + } + } + return false; +} + +void CWallet::SyncTransaction(const CTransaction& tx, const CBlock* pblock) +{ + LOCK2(cs_main, cs_wallet); + + /* MCHN START */ + bool fEmpty=false; + if((tx.vin.size() == 0) && ((tx.vout.size() == 0))) + { + LogPrint("mchn","mchn: Wallet optimization after block\n"); + fEmpty=true; + } + + if(!fEmpty) + { + if(((mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) == 0) || (mc_gState->m_WalletMode & MC_WMD_MAP_TXS)) + { +/* MCHN END */ + if (!AddToWalletIfInvolvingMe(tx, pblock, true)) + return; // Not one of ours + + // If a transaction changes 'conflicted' state, that changes the balance + // available of the outputs it spends. So force those to be + // recomputed, also: + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + if (mapWallet.count(txin.prevout.hash)) + mapWallet[txin.prevout.hash].MarkDirty(); + } + +/* MCHN START */ + } + } + if(fEmpty)// || (!fExisted && (pblock == NULL))) + { + OptimizeUnspentList(); + } + +/* MCHN END */ + +} + +void CWallet::EraseFromWallet(const uint256 &hash) +{ +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, not used in this case + if (!fFileBacked) + return; + { + LOCK(cs_wallet); + if (mapWallet.erase(hash)) + { + CWalletDB(strWalletFile).EraseTx(hash); + } + } + return; +} + + +isminetype CWallet::IsMine(const CTxIn &txin) const +{ + { + LOCK(cs_wallet); + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + int err; + const CWalletTx& prev=pwalletTxsMain->GetInternalWalletTx(txin.prevout.hash,NULL,&err); + if(err == MC_ERR_NOERROR) + { + if (txin.prevout.n < prev.vout.size()) + return IsMine(prev.vout[txin.prevout.n]); + } + } + else + { + map::const_iterator mi = mapWallet.find(txin.prevout.hash); + if (mi != mapWallet.end()) + { + const CWalletTx& prev = (*mi).second; + if (txin.prevout.n < prev.vout.size()) + return IsMine(prev.vout[txin.prevout.n]); + } + } + } + return ISMINE_NO; +} + +CAmount CWallet::GetDebit(const CTxIn &txin, const isminefilter& filter) const +{ + { + LOCK(cs_wallet); + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + int err; + const CWalletTx& prev=pwalletTxsMain->GetInternalWalletTx(txin.prevout.hash,NULL,&err); + if(err == MC_ERR_NOERROR) + { + if (txin.prevout.n < prev.vout.size()) + if (IsMine(prev.vout[txin.prevout.n]) & filter) + return prev.vout[txin.prevout.n].nValue; + } + } + else + { + map::const_iterator mi = mapWallet.find(txin.prevout.hash); + if (mi != mapWallet.end()) + { + const CWalletTx& prev = (*mi).second; + if (txin.prevout.n < prev.vout.size()) + if (IsMine(prev.vout[txin.prevout.n]) & filter) + return prev.vout[txin.prevout.n].nValue; + } + } + } + return 0; +} + +bool CWallet::IsFromMe(const CTxIn &txin, const isminefilter& filter) const +{ + { + LOCK(cs_wallet); + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + int err; + if(txin.prevout.hash == 0) + { + return false; + } + + const CWalletTx& prev=pwalletTxsMain->GetInternalWalletTx(txin.prevout.hash,NULL,&err); + + if(err == MC_ERR_NOERROR) + { + if (txin.prevout.n < prev.vout.size()) + if (IsMine(prev.vout[txin.prevout.n]) & filter) + return true; + } + } + else + { + map::const_iterator mi = mapWallet.find(txin.prevout.hash); + if (mi != mapWallet.end()) + { + const CWalletTx& prev = (*mi).second; + if (txin.prevout.n < prev.vout.size()) + if (IsMine(prev.vout[txin.prevout.n]) & filter) + return true; + } + } + } + return false; +} + +bool CWalletTx::IsTrusted(int nDepth) const +{ + // Quick answer in most cases + if (!IsFinalTx(*this)) + return false; + if (nDepth >= 1) + return true; + if (nDepth < 0) + return false; + if (!bSpendZeroConfChange || !IsFromMe(ISMINE_ALL)) // using wtx's cached debit + return false; + + // Trusted if all inputs are from us and are in the mempool: + BOOST_FOREACH(const CTxIn& txin, vin) + { + // Transactions not sent by us: not trusted +/* MCHN START */ + if(pwallet->setPurged.count(txin.prevout.hash) == 0) + { +/* MCHN END */ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + int err; + const CWalletTx& prev=pwalletTxsMain->GetInternalWalletTx(txin.prevout.hash,NULL,&err); + if(err) + { + return false; + } + const CTxOut& parentOut = prev.vout[txin.prevout.n]; + if (pwallet->IsMine(parentOut) != ISMINE_SPENDABLE) + { + return false; + } + } + else + { + const CWalletTx* parent = pwallet->GetWalletTx(txin.prevout.hash); + if (parent == NULL) + return false; + const CTxOut& parentOut = parent->vout[txin.prevout.n]; + if (pwallet->IsMine(parentOut) != ISMINE_SPENDABLE) + return false; + } +/* MCHN START */ + } +/* MCHN END */ + } + return true; +} + +bool CWallet::IsChange(const CTxOut& txout) const +{ + // TODO: fix handling of 'change' outputs. The assumption is that any + // payment to a script that is ours, but is not in the address book + // is change. That assumption is likely to break when we implement multisignature + // wallets that return change back into a multi-signature-protected address; + // a better way of identifying which outputs are 'the send' and which are + // 'the change' will need to be implemented (maybe extend CWalletTx to remember + // which output, if any, was change). + if (::IsMine(*this, txout.scriptPubKey)) + { + CTxDestination address; + if (!ExtractDestination(txout.scriptPubKey, address)) + return true; + + LOCK(cs_wallet); + if (!mapAddressBook.count(address)) + return true; + } + return false; +} + +int64_t CWalletTx::GetTxTime() const +{ + int64_t n = nTimeSmart; + return n ? n : nTimeReceived; +} + +int CWalletTx::GetRequestCount() const +{ + // Returns -1 if it wasn't being tracked + int nRequests = -1; + { + LOCK(pwallet->cs_wallet); + if (IsCoinBase()) + { + // Generated block + if (hashBlock != 0) + { + map::const_iterator mi = pwallet->mapRequestCount.find(hashBlock); + if (mi != pwallet->mapRequestCount.end()) + nRequests = (*mi).second; + } + } + else + { + // Did anyone request this transaction? + map::const_iterator mi = pwallet->mapRequestCount.find(GetHash()); + if (mi != pwallet->mapRequestCount.end()) + { + nRequests = (*mi).second; + + // How about the block it's in? + if (nRequests == 0 && hashBlock != 0) + { + map::const_iterator mi = pwallet->mapRequestCount.find(hashBlock); + if (mi != pwallet->mapRequestCount.end()) + nRequests = (*mi).second; + else + nRequests = 1; // If it's in someone else's block it must have got out + } + } + } + } + return nRequests; +} + +void CWalletTx::GetAmounts(list& listReceived, + list& listSent, CAmount& nFee, string& strSentAccount, const isminefilter& filter) const +{ + nFee = 0; + listReceived.clear(); + listSent.clear(); + strSentAccount = strFromAccount; + + // Compute fee: + CAmount nDebit = GetDebit(filter); + if (nDebit > 0) // debit>0 means we signed/sent this transaction + { + CAmount nValueOut = GetValueOut(); + nFee = nDebit - nValueOut; + } + +/* MCHN START */ + bool isFromMe=false; + if(nDebit > 0) + { + isFromMe=true; + } + else + { + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(IsFromMe(filter)) + { + isFromMe=true; + } + } + } + + bool isExchange=false; + if(isFromMe) + { + BOOST_FOREACH(const CTxIn& txin, vin) + { + if(!pwallet->IsFromMe(txin, filter)) + { + isExchange=true; + } + } + } + + bool isSelf=false; + + if(isFromMe && !isExchange) + { + isSelf=true; + for (unsigned int i = 0; i < vout.size(); ++i) + { + const CTxOut& txout = vout[i]; + if(!txout.scriptPubKey.IsUnspendable()) + { + isminetype fIsMine = pwallet->IsMine(txout); + if ((fIsMine & filter) == 0) + { + isSelf=false; + } + } + } + } + +/* MCHN END */ + // Sent/received. + for (unsigned int i = 0; i < vout.size(); ++i) + { + const CTxOut& txout = vout[i]; + isminetype fIsMine = pwallet->IsMine(txout); + // Only need to handle txouts if AT LEAST one of these is true: + // 1) they debit from us (sent) + // 2) the output is to us (received) +/* MCHN START */ +// if (nDebit > 0) + if(txout.scriptPubKey.IsUnspendable()) + continue; + + if(isFromMe) + { + // Don't report 'change' txouts + bool isChange=false; + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(!isSelf) + { + if(fIsMine & filter) + { + isChange = true; + } + } + } + else + { + isChange=pwallet->IsChange(txout); + } + if (isChange) + continue; +/* MCHN END */ + } + else if (!(fIsMine & filter)) + continue; + + // In either case, we need to get the destination address + CTxDestination address; + if (!ExtractDestination(txout.scriptPubKey, address)) + { + LogPrintf("CWalletTx::GetAmounts: Unknown transaction type found, txid %s\n", + this->GetHash().ToString()); + address = CNoDestination(); + } + + COutputEntry output = {address, txout.nValue, (int)i}; + + // If we are debited by the transaction, add the output as a "sent" entry +/* MCHN START */ +// if (nDebit > 0) + if(isFromMe) +/* MCHN END */ + listSent.push_back(output); + + // If we are receiving the output, add it as a "received" entry + if (fIsMine & filter) + listReceived.push_back(output); + } + +} + +void CWalletTx::GetAccountAmounts(const string& strAccount, CAmount& nReceived, + CAmount& nSent, CAmount& nFee, const isminefilter& filter) const +{ + nReceived = nSent = nFee = 0; + + CAmount allFee; + string strSentAccount; + list listReceived; + list listSent; + GetAmounts(listReceived, listSent, allFee, strSentAccount, filter); + + if (strAccount == strSentAccount) + { + BOOST_FOREACH(const COutputEntry& s, listSent) + nSent += s.amount; + nFee = allFee; + } + { + LOCK(pwallet->cs_wallet); + BOOST_FOREACH(const COutputEntry& r, listReceived) + { + if (pwallet->mapAddressBook.count(r.destination)) + { + map::const_iterator mi = pwallet->mapAddressBook.find(r.destination); + if (mi != pwallet->mapAddressBook.end() && (*mi).second.name == strAccount) + nReceived += r.amount; + } + else if (strAccount.empty()) + { + nReceived += r.amount; + } + } + } +} + + +bool CWalletTx::WriteToDisk() +{ + return CWalletDB(pwallet->strWalletFile).WriteTx(GetHash(), *this); +} + +/* MCHN START */ + +mc_TxImport *StartImport(CWallet *lpWallet,bool fOnlyUnsynced, int block, int *err) +{ + vector vAddressesToImport; + vector vStreamsToImport; + mc_TxEntity entity; + mc_TxImport *imp; + mc_Buffer *m_ChainEntities; + mc_TxEntityStat *lpent; + mc_EntityDetails stream_entity; + unsigned char *ptr; + int block_to_start_from,i,b; + + imp=NULL; + + *err=MC_ERR_NOERROR; + if( (mc_gState->m_WalletMode & MC_WMD_TXS) == 0) + { + return NULL; + } + + block_to_start_from=chainActive.Height()-1; + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, lpWallet->mapAddressBook) + { + const CBitcoinAddress& address = item.first; + mc_TxEntityStat entStat; + CTxDestination addressRet=address.Get(); + const CKeyID *lpKeyID=boost::get (&addressRet); + const CScriptID *lpScriptID=boost::get (&addressRet); + + entity.Zero(); + if(lpKeyID) + { + memcpy(entity.m_EntityID,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + } + if(lpScriptID) + { + memcpy(entity.m_EntityID,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + } + + if(entity.m_EntityType) + { + entStat.Zero(); + memcpy(&entStat,&entity,sizeof(mc_TxEntity)); + if(pwalletTxsMain->FindEntity(&entStat)) + { + if(!fOnlyUnsynced || ((entStat.m_Flags & MC_EFL_NOT_IN_SYNC) != 0) ) + { + vAddressesToImport.push_back(address); + } + } + else + { + vAddressesToImport.push_back(address); + } + } + } + } + + pwalletTxsMain->Lock(); + m_ChainEntities=pwalletTxsMain->GetEntityList(); + if(m_ChainEntities) + { + for(i=0;iGetCount();i++) + { + lpent=(mc_TxEntityStat*)m_ChainEntities->GetRow(i); + switch(lpent->m_Entity.m_EntityType & MC_TET_TYPE_MASK) + { + case MC_TET_STREAM: + case MC_TET_STREAM_KEY: + case MC_TET_STREAM_PUBLISHER: + case MC_TET_ASSET: + if(lpent->m_Entity.m_EntityType & MC_TET_CHAINPOS) + { + if(lpent->m_Flags & MC_EFL_NOT_IN_SYNC) + { + vStreamsToImport.push_back(lpent->m_Entity); + } + } + break; + } + } + } + pwalletTxsMain->UnLock(); + if( (vAddressesToImport.size() == 0) && (vStreamsToImport.size() == 0) ) + { + return NULL; + } + + mc_Buffer *lpEntities; + lpEntities=new mc_Buffer(); + lpEntities->Initialize(sizeof(mc_TxEntity),sizeof(mc_TxEntity),MC_BUF_MODE_DEFAULT); + if(vAddressesToImport.size()) + { + entity.Zero(); + entity.m_EntityType=MC_TET_WALLET_ALL | MC_TET_CHAINPOS; + lpEntities->Add(&entity,NULL); + entity.m_EntityType=MC_TET_WALLET_ALL | MC_TET_TIMERECEIVED; + lpEntities->Add(&entity,NULL); + entity.m_EntityType=MC_TET_WALLET_SPENDABLE | MC_TET_CHAINPOS; + lpEntities->Add(&entity,NULL); + entity.m_EntityType=MC_TET_WALLET_SPENDABLE | MC_TET_TIMERECEIVED; + lpEntities->Add(&entity,NULL); + + for(unsigned int i=0;i (&addressRet); + const CScriptID *lpScriptID=boost::get (&addressRet); + + entity.Zero(); + if(lpKeyID) + { + memcpy(entity.m_EntityID,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + lpEntities->Add(&entity,NULL); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_TIMERECEIVED; + lpEntities->Add(&entity,NULL); + } + if(lpScriptID) + { + memcpy(entity.m_EntityID,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + lpEntities->Add(&entity,NULL); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_TIMERECEIVED; + lpEntities->Add(&entity,NULL); + } + } + } + + if(vStreamsToImport.size()) + { + for(unsigned int i=0;im_Features->ShortTxIDAsAssetRef() == 0) ) + { + if(mc_gState->m_Assets->FindEntityByRef(&stream_entity,vStreamsToImport[i].m_EntityID)) + { + take_it=true; + } + } + else + { + if(mc_gState->m_Assets->FindEntityByShortTxID(&stream_entity,vStreamsToImport[i].m_EntityID)) + { + take_it=true; + } + } +// if(mc_gState->m_Assets->FindEntityByShortTxID(&stream_entity,vStreamsToImport[i].m_EntityID)) + if(take_it) + { + entity.Zero(); + memcpy(entity.m_EntityID,vStreamsToImport[i].m_EntityID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=vStreamsToImport[i].m_EntityType; + lpEntities->Add(&entity,NULL); + entity.m_EntityType=(vStreamsToImport[i].m_EntityType - MC_TET_CHAINPOS) | MC_TET_TIMERECEIVED; + lpEntities->Add(&entity,NULL); + ptr=(unsigned char *)stream_entity.GetRef(); + if(stream_entity.IsUnconfirmedGenesis() == 0) + { + b=(int)mc_GetLE(ptr,4)-1; + if(b < block_to_start_from) + { + block_to_start_from=b; + } + } + } + } + } + + + if(vAddressesToImport.size()) + { + block_to_start_from=block; + } + + if(lpEntities->GetCount()) + { + imp=pwalletTxsMain->StartImport(lpEntities,block_to_start_from,err); + } + + + delete lpEntities; + + return imp; +} + +/* MCHN END*/ + +/** + * Scan the block chain (starting in pindexStart) for transactions + * from or to us. If fUpdate is true, found transactions that already + * exist in the wallet will be updated. + */ +int CWallet::ScanForWalletTransactions(CBlockIndex* pindexStart, bool fUpdate,bool fOnlyUnsynced) +{ + int ret = 0; + int64_t nNow = GetTime(); + + CBlockIndex* pindex = pindexStart; + { + LOCK2(cs_main, cs_wallet); + +/* MCHN START */ + mc_TxImport *imp; + int err; + imp=StartImport(this,fOnlyUnsynced,pindex->nHeight-1,&err); +/* MCHN END */ + +/* MCHN START */ + // no need to read and scan block, if block was created before + // our wallet birthday (as adjusted for block time variability) + +/* Bad idea, imported address may be older than our wallet + while (pindex && nTimeFirstKey && (pindex->GetBlockTime() < (nTimeFirstKey - 7200))) + pindex = chainActive.Next(pindex); + */ +/* MCHN END */ + + if(imp) + { + while(pindex && (pindex->nHeight <= imp->m_Block)) + { + pindex = chainActive.Next(pindex); + } + } + + ShowProgress(_("Rescanning..."), 0); // show rescan progress in GUI as dialog or on splashscreen, if -rescan on startup + LogPrint("wallet","Rescanning for wallet transactions\n"); + double dProgressStart = Checkpoints::GuessVerificationProgress(pindex, false); + double dProgressTip = Checkpoints::GuessVerificationProgress(chainActive.Tip(), false); + if(mc_gState->m_WalletMode & MC_WMD_TXS) + { + if(imp == NULL) + { + LogPrint("wallet","No new entities, rescanning skipped\n"); + pindex=NULL; + } + } + + while (pindex) + { + if (pindex->nHeight % 100 == 0 && dProgressTip - dProgressStart > 0.0) + ShowProgress(_("Rescanning..."), std::max(1, std::min(99, (int)((Checkpoints::GuessVerificationProgress(pindex, false) - dProgressStart) / (dProgressTip - dProgressStart) * 100)))); + + CBlock block; + ReadBlockFromDisk(block, pindex); + +/* MCHN START */ + CDiskTxPos pos(pindex->GetBlockPos(), GetSizeOfCompactSize(block.vtx.size())); + int block_tx_index=0; + + BOOST_FOREACH(CTransaction& tx, block.vtx) + { + if(imp) + { + if(err == MC_ERR_NOERROR) + { + if(pindex->nHeight) // Skip 0-block coinbase + { + err=pwalletTxsMain->AddTx(imp,tx,pindex->nHeight,&pos,block_tx_index); + } + } + } + if(((mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) == 0) || (mc_gState->m_WalletMode & MC_WMD_MAP_TXS)) + { + if (AddToWalletIfInvolvingMe(tx, &block, fUpdate)) + ret++; + } + pos.nTxOffset += ::GetSerializeSize(tx, SER_DISK, CLIENT_VERSION); + block_tx_index++; + } + if(imp) + { + if(err == MC_ERR_NOERROR) + { + err=pwalletTxsMain->Commit(imp); + + } + if(err == MC_ERR_NOERROR) + { + pwalletTxsMain->CleanUpAfterBlock(imp,pindex->nHeight,pindex->nHeight-1); + } + } +/* MCHN END */ + if(!fOnlyUnsynced) + { + if((pindex->nHeight % 1000) == 0) + { + printf("%d of %d blocks rescanned\n",pindex->nHeight,chainActive.Height()); + } + } + pindex = chainActive.Next(pindex); + if (GetTime() >= nNow + 60) { + nNow = GetTime(); + LogPrintf("Still rescanning. At block %d. Progress=%f\n", pindex->nHeight, Checkpoints::GuessVerificationProgress(pindex)); + } + } + ShowProgress(_("Rescanning..."), 100); // hide progress dialog in GUI + if(imp) + { + if(err == MC_ERR_NOERROR) + { + LogPrint("wallet","wtxs: Replaying import mempool, %d items\n",mempool.hashList->m_Count); + for(int pos=0;posm_Count;pos++) + { + uint256 hash=*(uint256*)mempool.hashList->GetRow(pos); + if(mempool.exists(hash)) + { + const CTransaction& tx = mempool.mapTx[hash].GetTx(); + LogPrint("wallet","wtxs: Mempool tx: %s\n",hash.ToString().c_str()); + pwalletTxsMain->AddTx(imp,tx,-1,NULL,-1); + } + } + + err=pwalletTxsMain->CompleteImport(imp); + } + else + { + LogPrintf("Rescan failed with error %d\n",err); + err=pwalletTxsMain->DropImport(imp); + } + } + + if(err) + { + LogPrintf("Rescan failed with error %d\n",err); + ret=-1; + } + else + { + LogPrint("wallet","Rescan completed successfully\n"); + } + } + + return ret; +} + +void CWallet::ReacceptWalletTransactions() +{ + LOCK2(cs_main, cs_wallet); + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + pwalletTxsMain->Lock(); + LogPrint("wallet","ReacceptWalletTransactions: %ld txs in unconfirmed pool \n", pwalletTxsMain->m_UnconfirmedSends.size()); + BOOST_FOREACH(PAIRTYPE(const uint256, CWalletTx)& item, pwalletTxsMain->m_UnconfirmedSends) + { + const uint256& wtxid = item.first; + CWalletTx& wtx = item.second; + int nDepth = wtx.GetDepthInMainChain(); + LogPrint("wallet","Unconfirmed wtx: %s, depth: %d\n", wtxid.ToString(),nDepth); + + if (!wtx.IsCoinBase() && nDepth < 0) + { + LOCK(mempool.cs); + + LogPrint("wallet","Reaccepting wtx %s\n", wtxid.ToString()); + if(!wtx.AcceptToMemoryPool(false)) + { + LogPrintf("Tx %s was not accepted to mempool, setting INVALID flag\n", wtxid.ToString()); + pwalletTxsMain->SaveTxFlag((unsigned char*)&wtxid,MC_TFL_INVALID,1); + } + else + { + LogPrint("wallet","Unconfirmed wtx %s already in mempool, ignoring\n", wtxid.ToString()); + } + } + } + pwalletTxsMain->UnLock(); + } + else + { + BOOST_FOREACH(PAIRTYPE(const uint256, CWalletTx)& item, mapWallet) + { + const uint256& wtxid = item.first; + CWalletTx& wtx = item.second; + assert(wtx.GetHash() == wtxid); + + int nDepth = wtx.GetDepthInMainChain(); + + if (!wtx.IsCoinBase() && nDepth < 0) + { + // Try to add to memory pool + LOCK(mempool.cs); + wtx.AcceptToMemoryPool(false); + } + } + } +} + +void CWalletTx::RelayWalletTransaction() +{ + if (!IsCoinBase()) + { + if (GetDepthInMainChain() == 0) { + LogPrint("wallet","Relaying wtx %s\n", GetHash().ToString()); + RelayTransaction((CTransaction)*this); + } + } +} + +set CWalletTx::GetConflicts() const +{ + set result; + if (pwallet != NULL) + { + uint256 myHash = GetHash(); + result = pwallet->GetConflicts(myHash); + result.erase(myHash); + } + return result; +} + +void CWallet::ResendWalletTransactions(bool fForce) +{ + // Do this infrequently and randomly to avoid giving away + // that these are our transactions. + if (!fForce && (GetTime() < nNextResend)) + return; + bool fFirst = (nNextResend == 0); +/* MCHN START */ +// nNextResend = GetTime() + GetRand(30 * 60); + nNextResend = GetTime() + GetRand(3 * mc_gState->m_NetworkParams->GetInt64Param("targetblocktime")); +/* MCHN END */ + if (!fForce && fFirst) + return; + + // Only do it if there's been a new block since last time + if (!fForce && (nTimeBestReceived < nLastResend)) + return; + nLastResend = GetTime(); + + // Rebroadcast any of our txes that aren't in a block yet + LogPrintf("ResendWalletTransactions()\n"); + { + LOCK(cs_wallet); + // Sort them in chronological order + multimap mapSorted; + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + pwalletTxsMain->Lock(); + BOOST_FOREACH(PAIRTYPE(const uint256, CWalletTx)& item, pwalletTxsMain->m_UnconfirmedSends) + { + CWalletTx& wtx = item.second; + // Don't rebroadcast until it's had plenty of time that + // it should have gotten in already by now. + if (nTimeBestReceived - (int64_t)wtx.nTimeReceived > mc_gState->m_NetworkParams->GetInt64Param("targetblocktime")) + mapSorted.insert(make_pair(wtx.nTimeReceived, &wtx)); + + } + pwalletTxsMain->UnLock(); + } + else + { + BOOST_FOREACH(PAIRTYPE(const uint256, CWalletTx)& item, mapWallet) + { + CWalletTx& wtx = item.second; + // Don't rebroadcast until it's had plenty of time that + // it should have gotten in already by now. + /* MCHNS START */ + // if (nTimeBestReceived - (int64_t)wtx.nTimeReceived > 5 * 60) + if (nTimeBestReceived - (int64_t)wtx.nTimeReceived > mc_gState->m_NetworkParams->GetInt64Param("targetblocktime")) + /* MCHN END */ + mapSorted.insert(make_pair(wtx.nTimeReceived, &wtx)); + } + } + BOOST_FOREACH(PAIRTYPE(const unsigned int, CWalletTx*)& item, mapSorted) + { + CWalletTx& wtx = *item.second; + LogPrint("wallet","Wallet tx %s resent\n",wtx.GetHash().ToString().c_str()); + wtx.RelayWalletTransaction(); + } + } +} + +/** @} */ // end of mapWallet + + + + +/** @defgroup Actions + * + * @{ + */ + + +CAmount CWallet::GetBalance() const +{ + CAmount nTotal = 0; +/* MCHN START */ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + vector vecOutputs; + AvailableCoins(vecOutputs, false, NULL, false,true); + BOOST_FOREACH(const COutput& out, vecOutputs) + { + if(out.coin.IsTrusted()) + { + nTotal+=out.coin.m_TXOut.nValue; + } + } + return nTotal; + } +/* MCHN END*/ + + { + LOCK2(cs_main, cs_wallet); + +/* MCHN START */ +// for (map::const_iterator it = mapWallet.begin(); it != mapWallet.end(); ++it) +// { + + for (map::const_iterator itUnspent = mapUnspent.begin(); itUnspent != mapUnspent.end(); ++itUnspent) + { + const uint256& wtxid = itUnspent->first; + std::map::const_iterator it = mapWallet.find(wtxid); + + if (it == mapWallet.end()) + continue; + +/* MCHN END */ + + const CWalletTx* pcoin = &(*it).second; + if (pcoin->IsTrusted()) + nTotal += pcoin->GetAvailableCredit(); + } + } + + return nTotal; +} + +CAmount CWallet::GetUnconfirmedBalance() const +{ + CAmount nTotal = 0; +/* MCHN START */ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + vector vecOutputs; + AvailableCoins(vecOutputs, false, NULL, false,true); + BOOST_FOREACH(const COutput& out, vecOutputs) + { + if (!out.coin.IsFinal() || (!out.coin.IsTrusted() && out.nDepth == 0)) + { + nTotal+=out.coin.m_TXOut.nValue; + } + } + return nTotal; + } +/* MCHN END*/ + { + LOCK2(cs_main, cs_wallet); + for (map::const_iterator it = mapWallet.begin(); it != mapWallet.end(); ++it) + { + const CWalletTx* pcoin = &(*it).second; + if (!IsFinalTx(*pcoin) || (!pcoin->IsTrusted() && pcoin->GetDepthInMainChain() == 0)) + nTotal += pcoin->GetAvailableCredit(); + } + } + return nTotal; +} + +CAmount CWallet::GetImmatureBalance() const +{ +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, used only in QT + CAmount nTotal = 0; + { + LOCK2(cs_main, cs_wallet); + for (map::const_iterator it = mapWallet.begin(); it != mapWallet.end(); ++it) + { + const CWalletTx* pcoin = &(*it).second; + nTotal += pcoin->GetImmatureCredit(); + } + } + return nTotal; +} + +CAmount CWallet::GetWatchOnlyBalance() const +{ +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, used only in QT + CAmount nTotal = 0; + { + LOCK2(cs_main, cs_wallet); + for (map::const_iterator it = mapWallet.begin(); it != mapWallet.end(); ++it) + { + const CWalletTx* pcoin = &(*it).second; + if (pcoin->IsTrusted()) + nTotal += pcoin->GetAvailableWatchOnlyCredit(); + } + } + + return nTotal; +} + +CAmount CWallet::GetUnconfirmedWatchOnlyBalance() const +{ +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, used only in QT + CAmount nTotal = 0; + { + LOCK2(cs_main, cs_wallet); + for (map::const_iterator it = mapWallet.begin(); it != mapWallet.end(); ++it) + { + const CWalletTx* pcoin = &(*it).second; + if (!IsFinalTx(*pcoin) || (!pcoin->IsTrusted() && pcoin->GetDepthInMainChain() == 0)) + nTotal += pcoin->GetAvailableWatchOnlyCredit(); + } + } + return nTotal; +} + +CAmount CWallet::GetImmatureWatchOnlyBalance() const +{ +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, used only in QT + CAmount nTotal = 0; + { + LOCK2(cs_main, cs_wallet); + for (map::const_iterator it = mapWallet.begin(); it != mapWallet.end(); ++it) + { + const CWalletTx* pcoin = &(*it).second; + nTotal += pcoin->GetImmatureWatchOnlyCredit(); + } + } + return nTotal; +} + + +/** + * populate vCoins with vector of available COutputs. + */ +void CWallet::AvailableCoins(vector& vCoins, bool fOnlyConfirmed, const CCoinControl *coinControl, bool fOnlyUnlocked, bool fOnlyCoinsNoTxs, uint160 addr, uint32_t flags) const +{ + vCoins.clear(); + +/* MCHN START */ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) +// if(false) + { + LOCK2(cs_main, cs_wallet); +// std::map m_UTXOs[MC_TDB_MAX_IMPORTS]; + pwalletTxsMain->Lock(); + if(fOnlyCoinsNoTxs) + { + for (map::const_iterator it = pwalletTxsMain->m_UTXOs[0].begin(); it != pwalletTxsMain->m_UTXOs[0].end(); ++it) + { + const mc_Coin& coin = it->second; + if((addr == 0) || (addr == coin.m_EntityID)) + { + isminetype mine; + bool is_p2sh=false;; + if(coin.m_EntityType) + { + if(coin.m_Flags & MC_TFL_IS_SPENDABLE) + { + mine=ISMINE_SPENDABLE; + } + else + { + mine=ISMINE_WATCH_ONLY; + } + if( (coin.m_EntityType & MC_TET_TYPE_MASK) == MC_TET_SCRIPT_ADDRESS ) + { + is_p2sh=true; + } + + } + else + { + mine=IsMine(coin.m_TXOut); + is_p2sh=coin.m_TXOut.scriptPubKey.IsPayToScriptHash(); + } + if(flags & MC_CSF_ALLOW_NOT_SPENDABLE) + { + mine=ISMINE_SPENDABLE; + } + else + { + if(is_p2sh) + { + if((flags & MC_CSF_ALLOW_SPENDABLE_P2SH) == 0) + { + mine=ISMINE_NO; + } + else + { + if(flags & MC_CSF_ALLOW_NOT_SPENDABLE_P2SH) + { + mine=ISMINE_SPENDABLE; + } + } + } + } + uint256 txid=coin.m_OutPoint.hash; + uint32_t vout=coin.m_OutPoint.n; + int nDepth=coin.GetDepthInMainChain(); +// LogPrintf("DEBUG: %s %d %d %d\n",coin.m_OutPoint.ToString().c_str(),coin.GetDepthInMainChain(),coin.IsFinal(),coin.BlocksToMaturity()); +// LogPrintf("DEBUG: %s\n",coin.ToString().c_str()); + if ( (coin.IsFinal()) && + (coin.BlocksToMaturity() <= 0) && + (mine != ISMINE_NO) && + (!fOnlyUnlocked || !IsLockedCoin(txid, vout)) && + (!coinControl || !coinControl->HasSelected() || coinControl->IsSelected(txid, vout))) + { + vCoins.push_back(COutput(NULL, vout, nDepth, (mine & ISMINE_SPENDABLE) != ISMINE_NO, coin)); + } + } + } + } + else + { + vector vCoinsToTake; + pwalletTxsMain->vAvailableCoins.clear(); + + for (map::const_iterator it = pwalletTxsMain->m_UTXOs[0].begin(); it != pwalletTxsMain->m_UTXOs[0].end(); ++it) + { + const mc_Coin& coin = it->second; + isminetype mine = IsMine(coin.m_TXOut); + bool is_p2sh=coin.m_TXOut.scriptPubKey.IsPayToScriptHash(); + if(flags & MC_CSF_ALLOW_NOT_SPENDABLE) + { + mine=ISMINE_SPENDABLE; + } + else + { + if(is_p2sh) + { + if((flags & MC_CSF_ALLOW_SPENDABLE_P2SH) == 0) + { + mine=ISMINE_NO; + } + else + { + if(flags & MC_CSF_ALLOW_NOT_SPENDABLE_P2SH) + { + mine=ISMINE_SPENDABLE; + } + } + } + } + + uint256 txid=coin.m_OutPoint.hash; + uint32_t vout=coin.m_OutPoint.n; + if ( (coin.IsFinal()) && + (coin.BlocksToMaturity() <= 0) && + (mine != ISMINE_NO) && + (!fOnlyUnlocked || !IsLockedCoin(txid, vout)) && + (!coinControl || !coinControl->HasSelected() || coinControl->IsSelected(txid, vout))) + { + const CWalletTx& wtx=pwalletTxsMain->GetInternalWalletTx(txid,NULL,NULL); + std::map::const_iterator itold = pwalletTxsMain->vAvailableCoins.find(txid); + if (itold == pwalletTxsMain->vAvailableCoins.end()) + { + pwalletTxsMain->vAvailableCoins.insert(make_pair(txid, wtx)); + } + vCoinsToTake.push_back(coin); + } + } + for (unsigned int i=0;i::const_iterator itold = pwalletTxsMain->vAvailableCoins.find(txid); + const CWalletTx* pwtx=&(*itold).second; + vCoins.push_back(COutput(pwtx, vout, nDepth, (mine & ISMINE_SPENDABLE) != ISMINE_NO)); + } + } + pwalletTxsMain->UnLock(); + } + else + { +/* MCHN END */ + LOCK2(cs_main, cs_wallet); +/* MCHN START */ + LogPrint("mchn","mchn: Wallet coins: Total: %d, Unspent: %d\n",mapWallet.size(),mapUnspent.size()); +// for (map::const_iterator it = mapWallet.begin(); it != mapWallet.end(); ++it) +// { +// const uint256& wtxid = it->first; + + for (map::const_iterator itUnspent = mapUnspent.begin(); itUnspent != mapUnspent.end(); ++itUnspent) + { + const uint256& wtxid = itUnspent->first; + std::map::const_iterator it = mapWallet.find(wtxid); + + if (it == mapWallet.end()) + continue; +/* MCHN END */ + const CWalletTx* pcoin = &(*it).second; + + if (!IsFinalTx(*pcoin)) + { +// printf("A\n"); + continue; + } + + if (fOnlyConfirmed && !pcoin->IsTrusted()) + { +// printf("B\n"); + continue; + } + + if (pcoin->IsCoinBase() && pcoin->GetBlocksToMaturity() > 0) + { +// printf("C\n"); + continue; + } + + int nDepth = pcoin->GetDepthInMainChain(); + if (nDepth < 0) + { +// printf("D %s\n",pcoin->GetHash().GetHex().c_str()); + continue; + } + +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + for (unsigned int i = 0; i < pcoin->vout.size(); i++) { + isminetype mine = IsMine(pcoin->vout[i]); + if (!(IsSpent(wtxid, i)) && mine != ISMINE_NO && + (!fOnlyUnlocked || !IsLockedCoin((*it).first, i)) && pcoin->vout[i].nValue >= 0 && + (!coinControl || !coinControl->HasSelected() || coinControl->IsSelected((*it).first, i))) + vCoins.push_back(COutput(pcoin, i, nDepth, (mine & ISMINE_SPENDABLE) != ISMINE_NO)); + } + } + else + { +/* MCHN END */ + + for (unsigned int i = 0; i < pcoin->vout.size(); i++) { + isminetype mine = IsMine(pcoin->vout[i]); + if (!(IsSpent(wtxid, i)) && mine != ISMINE_NO && + (!fOnlyUnlocked || !IsLockedCoin((*it).first, i)) && pcoin->vout[i].nValue > 0 && + (!coinControl || !coinControl->HasSelected() || coinControl->IsSelected((*it).first, i))) + vCoins.push_back(COutput(pcoin, i, nDepth, (mine & ISMINE_SPENDABLE) != ISMINE_NO)); + } +/* MCHN START */ + } +/* MCHN END */ + } + } +} + +static void ApproximateBestSubset(vector > >vValue, const CAmount& nTotalLower, const CAmount& nTargetValue, + vector& vfBest, CAmount& nBest, int iterations = 1000) +{ + vector vfIncluded; + + vfBest.assign(vValue.size(), true); + nBest = nTotalLower; + + seed_insecure_rand(); + + for (int nRep = 0; nRep < iterations && nBest != nTargetValue; nRep++) + { + vfIncluded.assign(vValue.size(), false); + CAmount nTotal = 0; + bool fReachedTarget = false; + for (int nPass = 0; nPass < 2 && !fReachedTarget; nPass++) + { + for (unsigned int i = 0; i < vValue.size(); i++) + { + //The solver here uses a randomized algorithm, + //the randomness serves no real security purpose but is just + //needed to prevent degenerate behavior and it is important + //that the rng is fast. We do not use a constant random sequence, + //because there may be some privacy improvement by making + //the selection random. + if (nPass == 0 ? insecure_rand()&1 : !vfIncluded[i]) + { + nTotal += vValue[i].first; + vfIncluded[i] = true; + if (nTotal >= nTargetValue) + { + fReachedTarget = true; + if (nTotal < nBest) + { + nBest = nTotal; + vfBest = vfIncluded; + } + nTotal -= vValue[i].first; + vfIncluded[i] = false; + } + } + } + } + } +} + +static void ApproximateBestSubset(vector > >vValue, const CAmount& nTotalLower, const CAmount& nTargetValue, + vector& vfBest, CAmount& nBest, int iterations = 1000) +{ + vector vfIncluded; + + vfBest.assign(vValue.size(), true); + nBest = nTotalLower; + + seed_insecure_rand(); + + for (int nRep = 0; nRep < iterations && nBest != nTargetValue; nRep++) + { + vfIncluded.assign(vValue.size(), false); + CAmount nTotal = 0; + bool fReachedTarget = false; + for (int nPass = 0; nPass < 2 && !fReachedTarget; nPass++) + { + for (unsigned int i = 0; i < vValue.size(); i++) + { + //The solver here uses a randomized algorithm, + //the randomness serves no real security purpose but is just + //needed to prevent degenerate behavior and it is important + //that the rng is fast. We do not use a constant random sequence, + //because there may be some privacy improvement by making + //the selection random. + if (nPass == 0 ? insecure_rand()&1 : !vfIncluded[i]) + { + nTotal += vValue[i].first; + vfIncluded[i] = true; + if (nTotal >= nTargetValue) + { + fReachedTarget = true; + if (nTotal < nBest) + { + nBest = nTotal; + vfBest = vfIncluded; + } + nTotal -= vValue[i].first; + vfIncluded[i] = false; + } + } + } + } + } +} + + +bool CWallet::SelectCoinsMinConf(const CAmount& nTargetValue, int nConfMine, int nConfTheirs, vector vCoins, + set >& setCoinsRet, CAmount& nValueRet) const +{ + setCoinsRet.clear(); + nValueRet = 0; + + // List of values less than target + pair > coinLowestLarger; + coinLowestLarger.first = std::numeric_limits::max(); + coinLowestLarger.second.first = NULL; + vector > > vValue; + CAmount nTotalLower = 0; + + random_shuffle(vCoins.begin(), vCoins.end(), GetRandInt); + + BOOST_FOREACH(const COutput &output, vCoins) + { + if (!output.fSpendable) + continue; + +/* MCHN START */ + if(!OutputCanSend(output)) + { + continue; + } +/* MCHN END */ + + const CWalletTx *pcoin = output.tx; + + if (output.nDepth < (pcoin->IsFromMe(ISMINE_ALL) ? nConfMine : nConfTheirs)) + continue; + + int i = output.i; + CAmount n = pcoin->vout[i].nValue; + + pair > coin = make_pair(n,make_pair(pcoin, i)); + + if (n == nTargetValue) + { + setCoinsRet.insert(coin.second); + nValueRet += coin.first; + return true; + } + else if (n < nTargetValue + CENT) + { + vValue.push_back(coin); + nTotalLower += n; + } + else if (n < coinLowestLarger.first) + { + coinLowestLarger = coin; + } + } + + if (nTotalLower == nTargetValue) + { + for (unsigned int i = 0; i < vValue.size(); ++i) + { + setCoinsRet.insert(vValue[i].second); + nValueRet += vValue[i].first; + } + return true; + } + + if (nTotalLower < nTargetValue) + { + if (coinLowestLarger.second.first == NULL) + return false; + setCoinsRet.insert(coinLowestLarger.second); + nValueRet += coinLowestLarger.first; + return true; + } + + // Solve subset sum by stochastic approximation + sort(vValue.rbegin(), vValue.rend(), CompareValueOnly()); + vector vfBest; + CAmount nBest; + + ApproximateBestSubset(vValue, nTotalLower, nTargetValue, vfBest, nBest, 1000); + if (nBest != nTargetValue && nTotalLower >= nTargetValue + CENT) + ApproximateBestSubset(vValue, nTotalLower, nTargetValue + CENT, vfBest, nBest, 1000); + + // If we have a bigger coin and (either the stochastic approximation didn't find a good solution, + // or the next bigger coin is closer), return the bigger coin + if (coinLowestLarger.second.first && + ((nBest != nTargetValue && nBest < nTargetValue + CENT) || coinLowestLarger.first <= nBest)) + { + setCoinsRet.insert(coinLowestLarger.second); + nValueRet += coinLowestLarger.first; + } + else { + for (unsigned int i = 0; i < vValue.size(); i++) + if (vfBest[i]) + { + setCoinsRet.insert(vValue[i].second); + nValueRet += vValue[i].first; + } + + LogPrint("selectcoins", "SelectCoins() best subset: "); + for (unsigned int i = 0; i < vValue.size(); i++) + if (vfBest[i]) + LogPrint("selectcoins", "%s ", FormatMoney(vValue[i].first)); + LogPrint("selectcoins", "total %s\n", FormatMoney(nBest)); + } + + return true; +} + +bool CWallet::SelectCoins(const CAmount& nTargetValue, set >& setCoinsRet, CAmount& nValueRet, const CCoinControl* coinControl) const +{ + vector vCoins; + AvailableCoins(vCoins, true, coinControl); + + // coin control -> return all selected outputs (we want all selected to go into the transaction for sure) + if (coinControl && coinControl->HasSelected()) + { + BOOST_FOREACH(const COutput& out, vCoins) + { + if(!out.fSpendable) + continue; + nValueRet += out.tx->vout[out.i].nValue; + setCoinsRet.insert(make_pair(out.tx, out.i)); + } + return (nValueRet >= nTargetValue); + } + + return (SelectCoinsMinConf(nTargetValue, 1, 6, vCoins, setCoinsRet, nValueRet) || + SelectCoinsMinConf(nTargetValue, 1, 1, vCoins, setCoinsRet, nValueRet) || + (bSpendZeroConfChange && SelectCoinsMinConf(nTargetValue, 0, 1, vCoins, setCoinsRet, nValueRet))); +} + +/** + * This is not used anymore + */ + + +bool CWallet::CreateTransaction(const vector >& vecSend, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl* coinControl) +{ + CAmount nValue = 0; + BOOST_FOREACH (const PAIRTYPE(CScript, CAmount)& s, vecSend) + { + if (nValue < 0) + { + strFailReason = _("Transaction amounts must be positive"); + return false; + } + nValue += s.second; + } + if (vecSend.empty() || nValue < 0) + { + strFailReason = _("Transaction amounts must be positive"); + return false; + } + + wtxNew.fTimeReceivedIsTxTime = true; + wtxNew.BindWallet(this); + CMutableTransaction txNew; + + { + LOCK2(cs_main, cs_wallet); + { + nFeeRet = 0; + while (true) + { + txNew.vin.clear(); + txNew.vout.clear(); + wtxNew.fFromMe = true; + + CAmount nTotalValue = nValue + nFeeRet; + double dPriority = 0; + // vouts to the payees + BOOST_FOREACH (const PAIRTYPE(CScript, CAmount)& s, vecSend) + { + CTxOut txout(s.second, s.first); +/* MCHN START */ + + uint32_t type,from,to,timestamp,type_ored,no_dust_check; + + mc_gState->m_TmpScript->Clear(); + + const CScript& script1 = txout.scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + + type_ored=0; + no_dust_check=mc_gState->m_TmpScript->IsOpReturnScript(); + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + mc_gState->m_TmpScript->SetElement(e); + if(mc_gState->m_TmpScript->GetPermission(&type,&from,&to,×tamp) == 0) + { + if(from >= to) + { + no_dust_check=1; + } + type_ored |= type; + } + } + + + if((type_ored == 0))// || (type_ored & MC_PTP_RECEIVE)) + { + if(no_dust_check == 0) + { + ::minRelayTxFee = CFeeRate(MIN_RELAY_TX_FEE); + +/* MCHN END */ + + if (txout.IsDust(::minRelayTxFee)) + { + strFailReason = _("Transaction amount too small"); + return false; + } +/* MCHN START */ + } + } +/* MCHN END */ + txNew.vout.push_back(txout); + } + + // Choose coins to use + set > setCoins; + CAmount nValueIn = 0; + if (!SelectCoins(nTotalValue, setCoins, nValueIn, coinControl)) + { + strFailReason = _("Insufficient funds"); + return false; + } + BOOST_FOREACH(PAIRTYPE(const CWalletTx*, unsigned int) pcoin, setCoins) + { + CAmount nCredit = pcoin.first->vout[pcoin.second].nValue; + //The coin age after the next block (depth+1) is used instead of the current, + //reflecting an assumption the user would accept a bit more delay for + //a chance at a free transaction. + //But mempool inputs might still be in the mempool, so their age stays 0 + int age = pcoin.first->GetDepthInMainChain(); + if (age != 0) + age += 1; + dPriority += (double)nCredit * age; + } + + CAmount nChange = nValueIn - nValue - nFeeRet; + + if (nChange > 0) + { + // Fill a vout to ourself + // TODO: pass in scriptChange instead of reservekey so + // change transaction isn't always pay-to-bitcoin-address + CScript scriptChange; + + // coin control: send change to custom address + if (coinControl && !boost::get(&coinControl->destChange)) + scriptChange = GetScriptForDestination(coinControl->destChange); + + // no coin control: send change to newly generated address + else + { + // Note: We use a new key here to keep it from being obvious which side is the change. + // The drawback is that by not reusing a previous key, the change may be lost if a + // backup is restored, if the backup doesn't have the new private key for the change. + // If we reused the old key, it would be possible to add code to look for and + // rediscover unknown transactions that were written with keys of ours to recover + // post-backup change. + + // Reserve a new key pair from key pool + CPubKey vchPubKey; + bool ret; + ret = reservekey.GetReservedKey(vchPubKey); + assert(ret); // should never fail, as we just unlocked +/* MCHN START */ +// Using default key - cannot use other keys from pool as they don't have permission +// vchPubKey=vchDefaultKey; + + if(!GetKeyFromAddressBook(vchPubKey,MC_PTP_RECEIVE)) + { + LogPrintf("mchn: Internal error: Cannot find address for change having receive permission\n"); + strFailReason = _("Change address not found"); + return false; + } + +/* MCHN END */ + scriptChange = GetScriptForDestination(vchPubKey.GetID()); + } + + CTxOut newTxOut(nChange, scriptChange); + + // Never create dust outputs; if we would, just + // add the dust to the fee. + if (newTxOut.IsDust(::minRelayTxFee)) + { + nFeeRet += nChange; + reservekey.ReturnKey(); + } + else + { + // Insert change txn at random position: + vector::iterator position = txNew.vout.begin()+GetRandInt(txNew.vout.size()+1); + txNew.vout.insert(position, newTxOut); + } + } + else + reservekey.ReturnKey(); + + // Fill vin + BOOST_FOREACH(const PAIRTYPE(const CWalletTx*,unsigned int)& coin, setCoins) + txNew.vin.push_back(CTxIn(coin.first->GetHash(),coin.second)); + + // Sign + int nIn = 0; + BOOST_FOREACH(const PAIRTYPE(const CWalletTx*,unsigned int)& coin, setCoins) + if (!SignSignature(*this, *coin.first, txNew, nIn++)) + { + strFailReason = _("Signing transaction failed"); + return false; + } + + // Embed the constructed transaction data in wtxNew. + *static_cast(&wtxNew) = CTransaction(txNew); + + // Limit size + unsigned int nBytes = ::GetSerializeSize(*(CTransaction*)&wtxNew, SER_NETWORK, PROTOCOL_VERSION); + if (nBytes >= MAX_STANDARD_TX_SIZE) + { + strFailReason = _("Transaction too large"); + return false; + } + dPriority = wtxNew.ComputePriority(dPriority, nBytes); + + // Can we complete this as a free transaction? + if (fSendFreeTransactions && nBytes <= MAX_FREE_TRANSACTION_CREATE_SIZE) + { + // Not enough fee: enough priority? + double dPriorityNeeded = mempool.estimatePriority(nTxConfirmTarget); + // Not enough mempool history to estimate: use hard-coded AllowFree. + if (dPriorityNeeded <= 0 && AllowFree(dPriority)) + break; + + // Small enough, and priority high enough, to send for free + if (dPriorityNeeded > 0 && dPriority >= dPriorityNeeded) + break; + } + + CAmount nFeeNeeded = GetMinimumFee(nBytes, nTxConfirmTarget, mempool); + + // If we made it here and we aren't even able to meet the relay fee on the next pass, give up + // because we must be at the maximum allowed fee. + if (nFeeNeeded < ::minRelayTxFee.GetFee(nBytes)) + { + strFailReason = _("Transaction too large for fee policy"); + return false; + } + + if (nFeeRet >= nFeeNeeded) + break; // Done, enough fee included. + + // Include more fee and try again. + nFeeRet = nFeeNeeded; + continue; + } + } + } + return true; +} + +bool CWallet::CreateTransaction(CScript scriptPubKey, const CAmount& nValue, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl* coinControl) +{ + vector< pair > vecSend; + vecSend.push_back(make_pair(scriptPubKey, nValue)); + return CreateTransaction(vecSend, wtxNew, reservekey, nFeeRet, strFailReason, coinControl); +} + +/* MCHN START */ + +bool CWallet::CreateTransaction(CScript scriptPubKey, const CAmount& nValue, CScript scriptOpReturn, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl* coinControl, + const set* addresses,int min_conf,int min_inputs,int max_inputs,const vector* lpCoinsToUse) +{ + vector< pair > vecSend; + CAmount nAmount=nValue; + if(nAmount < 0) + { + minRelayTxFee = CFeeRate(MIN_RELAY_TX_FEE); + CTxOut txout(nAmount, scriptPubKey); + nAmount=txout.GetDustThreshold(minRelayTxFee); + } + vecSend.push_back(make_pair(scriptPubKey, nAmount)); + if(scriptOpReturn.size()) + { + vecSend.push_back(make_pair(scriptOpReturn, 0)); + } + return CreateMultiChainTransaction(vecSend, wtxNew, reservekey, nFeeRet, strFailReason, coinControl, addresses, min_conf, min_inputs, max_inputs, lpCoinsToUse); +} + +bool CWallet::CreateTransaction(std::vector scriptPubKeys, const CAmount& nValue, CScript scriptOpReturn, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl* coinControl, + const set* addresses,int min_conf,int min_inputs,int max_inputs,const vector* lpCoinsToUse) +{ + vector< pair > vecSend; + BOOST_FOREACH (const CScript& scriptPubKey, scriptPubKeys) + { + CAmount nAmount=nValue; + if(nAmount < 0) + { + minRelayTxFee = CFeeRate(MIN_RELAY_TX_FEE); + CTxOut txout(nAmount, scriptPubKey); + nAmount=txout.GetDustThreshold(minRelayTxFee); + } + vecSend.push_back(make_pair(scriptPubKey, nAmount)); + } + + if(scriptOpReturn.size()) + { + vecSend.push_back(make_pair(scriptOpReturn, 0)); + } + return CreateMultiChainTransaction(vecSend, wtxNew, reservekey, nFeeRet, strFailReason, coinControl, addresses, min_conf, min_inputs, max_inputs,lpCoinsToUse); +} + +int CWallet::SelectMultiChainCombineCoinsMinConf(int nConfMine, int nConfTheirs, vector vCoins, mc_Buffer *in_map, mc_Buffer *in_amounts, + int in_selected_row,int in_asset_row, + set >& setCoinsRet,int max_inputs) const +{ + setCoinsRet.clear(); + vector > > vValue; + bool take_it; + int coin_id=-1; + unsigned char buf_map[40]; + + BOOST_FOREACH(const COutput &output, vCoins) + { + take_it=true; + + if(take_it) + { + if (!output.fSpendable) + { + take_it=false; + } + } + + const CWalletTx *pcoin = output.tx; + + if(take_it) + { + if (output.nDepth < (pcoin->IsFromMe(ISMINE_ALL) ? nConfMine : nConfTheirs)) + { + take_it=false; + } + } + + // Retrieving coin index in the matrix + if(take_it) + { + uint256 hash=output.tx->GetHash(); + int out_i; + out_i=output.i; + + memcpy(buf_map,&hash,32); + mc_PutLE(buf_map+32,&out_i,4); + int row=in_map->Seek(buf_map); + if(row >= 0) + { + coin_id=mc_GetLE(in_map->GetRow(row)+36,4); + } + else + { + take_it=false; + } + } + + // Checking that the coin is not selected yet + if(take_it) + { + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_selected_row),coin_id)) + { + take_it=false; + } + } + + + int i = output.i; + + if(take_it) + { + // No changes below this line + pair > coin = make_pair(output.nDepth,make_pair(pcoin, i)); + + vValue.push_back(coin); + } + } + + + if((int)vValue.size() <= max_inputs) + { + for (unsigned int i = 0; i < vValue.size(); i++) + { + setCoinsRet.insert(vValue[i].second); + } + return vValue.size(); + } + + sort(vValue.rbegin(), vValue.rend(), CompareValueOnlyIntDesc()); + + for (int i = 0; i < max_inputs; i++) + { + setCoinsRet.insert(vValue[i].second); + } + + return vValue.size(); +} + + +bool CWallet::SelectMultiChainCoinsMinConf(const CAmount& nTargetValue, int nConfMine, int nConfTheirs, vector vCoins, mc_Buffer *in_map, mc_Buffer *in_amounts, + int in_selected_row,int in_asset_row,int in_preferred_row, + set >& setCoinsRet, CAmount& nValueRet) const +{ +// printf("SelectMultiChainCoinsMinConf\n"); + setCoinsRet.clear(); + nValueRet = 0; + + // List of values less than target + pair > coinLowestLarger; + coinLowestLarger.first = std::numeric_limits::max(); + coinLowestLarger.second.first = 0; + vector > > vValue; + CAmount nTotalLower = 0; + bool take_it; + CAmount local_cent=CENT; + if(in_asset_row != 3) + { + local_cent=0; + } + int coin_id=-1; + unsigned char buf_map[40]; + int64_t quantity; + + random_shuffle(vCoins.begin(), vCoins.end(), GetRandInt); + + BOOST_FOREACH(const COutput &output, vCoins) + { + take_it=true; + quantity=0; + +// printf("Coin: %s-%d\n",output.tx->GetHash().ToString().c_str(),output.i); + if(take_it) + { + if (!output.fSpendable) + { +// printf("Not spendable\n"); + take_it=false; + } + } + + const CWalletTx *pcoin = output.tx; + + if(take_it) + { + if(pcoin) + { + if (output.nDepth < (pcoin->IsFromMe(ISMINE_ALL) ? nConfMine : nConfTheirs)) + { + // printf("Not enough confitmations\n"); + take_it=false; + } + } + else + { + if (!output.coin.IsTrusted()) + { + // printf("Not spendable\n"); + take_it=false; + } + if (output.nDepth < (((output.coin.m_Flags & MC_TFL_FROM_ME) > 0) ? nConfMine : nConfTheirs)) + { + // printf("Not enough confitmations\n"); + take_it=false; + } + } + } + + // Retrieving coin index in the matrix + uint256 hash; + if(pcoin) + { + hash=pcoin->GetHash(); + } + else + { + hash=output.coin.m_OutPoint.hash; + } + + if(take_it) + { + int out_i; + out_i=output.i; + + memcpy(buf_map,&hash,32); + mc_PutLE(buf_map+32,&out_i,4); + int row=in_map->Seek(buf_map); + if(row >= 0) + { + coin_id=mc_GetLE(in_map->GetRow(row)+36,4); +// printf("Coin ID: %d\n",coin_id); + } + else + { +// printf("Not found in map\n"); + take_it=false; + } + } + + // Checking that the coin is not selected yet + if(take_it) + { + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_selected_row),coin_id)) + { +// printf("Already selected\n"); + take_it=false; + } + } + + if(take_it && (in_preferred_row > 0)) // Checking that this coin have positive value in "preferred row" - + // relevant asset or "pure native currency" flag + { + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_preferred_row),coin_id) == 0) + { + take_it=false; + } + } + // Retrieving quantity from matrix + if(take_it) + { + quantity=mc_GetABCoinQuantity(in_amounts->GetRow(in_asset_row),coin_id); + if(quantity <= 0) + { +// printf("Bad quantity: %ld \n",quantity); + take_it=false; + } + if(quantity == 0) + { + if(nTargetValue == 0) + { + pair > coin = make_pair(quantity,make_pair(hash, output.i)); + + setCoinsRet.insert(coin.second); + nValueRet += coin.first; + return true; + } + } + } + + + int i = output.i; + CAmount n = quantity; + + if(take_it) + { + + pair > coin = make_pair(n,make_pair(hash, i)); + + if (n == nTargetValue) + { + setCoinsRet.insert(coin.second); + nValueRet += coin.first; + return true; + } + else if (n < nTargetValue + local_cent) + { + vValue.push_back(coin); + nTotalLower += n; + } + else if (n < coinLowestLarger.first) + { + coinLowestLarger = coin; + } + } + } + + if (nTotalLower == nTargetValue) + { + for (unsigned int i = 0; i < vValue.size(); ++i) + { + setCoinsRet.insert(vValue[i].second); + nValueRet += vValue[i].first; + } + if(setCoinsRet.size() == 0) + { + return false; + } + return true; + } + + if (nTotalLower < nTargetValue) + { + if (coinLowestLarger.second.first == 0) + return false; + setCoinsRet.insert(coinLowestLarger.second); + nValueRet += coinLowestLarger.first; + return true; + } + + + // Solve subset sum by stochastic approximation + sort(vValue.rbegin(), vValue.rend(), CompareValueOnlyHash()); + vector vfBest; + CAmount nBest; + + ApproximateBestSubset(vValue, nTotalLower, nTargetValue, vfBest, nBest, 1000); + if (nBest != nTargetValue && nTotalLower >= nTargetValue + local_cent) + ApproximateBestSubset(vValue, nTotalLower, nTargetValue + local_cent, vfBest, nBest, 1000); + + // If we have a bigger coin and (either the stochastic approximation didn't find a good solution, + // or the next bigger coin is closer), return the bigger coin + if ((coinLowestLarger.second.first != 0) && + ((nBest != nTargetValue && nBest < nTargetValue + local_cent) || coinLowestLarger.first <= nBest)) + { + setCoinsRet.insert(coinLowestLarger.second); + nValueRet += coinLowestLarger.first; + } + else { + for (unsigned int i = 0; i < vValue.size(); i++) + if (vfBest[i]) + { + setCoinsRet.insert(vValue[i].second); + nValueRet += vValue[i].first; + } + + LogPrint("selectcoins", "SelectCoins() best subset: "); + for (unsigned int i = 0; i < vValue.size(); i++) + if (vfBest[i]) + LogPrint("selectcoins", "%s ", FormatMoney(vValue[i].first)); + LogPrint("selectcoins", "total %s\n", FormatMoney(nBest)); + } + + if(nTargetValue == 0) + { + if(setCoinsRet.size() == 0) + { + return false; + } + } + + return true; +} + +bool CWallet::SelectMultiChainCoins(const CAmount& nTargetValue, vector &vCoins, mc_Buffer *in_map, mc_Buffer *in_amounts, + int in_selected_row,int in_asset_row,int in_preferred_row, + set >& setCoinsRet, CAmount& nValueRet, const CCoinControl* coinControl) const +{ + int coin_id=0; + int buf_map[40]; + + // coin control -> return all selected outputs (we want all selected to go into the transaction for sure) + if (coinControl && coinControl->HasSelected()) + { + BOOST_FOREACH(const COutput& out, vCoins) + { + if(out.fSpendable) + { + uint256 hash=out.tx->GetHash(); + int out_i; + out_i=out.i; + + memcpy(buf_map,&hash,32); // Retrieving coin index in the matrix + mc_PutLE(buf_map+32,&out_i,4); + int row=in_map->Seek(buf_map); + + if(row >= 0) + { + coin_id=mc_GetLE(in_map->GetRow(row)+36,4); + // Coin is not selected yet +// if(mc_GetLE(in_amounts->GetRow(in_selected_row)+coin_id*MC_AST_ASSET_REF_SIZE,MC_AST_ASSET_QUANTITY_SIZE) == 0)// BUG??? + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_selected_row),coin_id) == 0) + { + // Value is take from the matrix + nValueRet += mc_GetABCoinQuantity(in_amounts->GetRow(in_asset_row),coin_id); + setCoinsRet.insert(make_pair(hash, out.i)); + } + } + } + coin_id++; + } + return (nValueRet >= nTargetValue); + } + + int preferred_row=in_preferred_row; + for(int attempt=0;attempt<2;attempt++) + { + if((SelectMultiChainCoinsMinConf(nTargetValue, 1, 6, vCoins, in_map, in_amounts, in_selected_row, in_asset_row, preferred_row, setCoinsRet, nValueRet) || + SelectMultiChainCoinsMinConf(nTargetValue, 1, 1, vCoins, in_map, in_amounts, in_selected_row, in_asset_row, preferred_row, setCoinsRet, nValueRet) || + (bSpendZeroConfChange && SelectMultiChainCoinsMinConf(nTargetValue, 0, 1, vCoins, in_map, in_amounts, in_selected_row, in_asset_row, preferred_row, setCoinsRet, nValueRet)))) + { + return true; + } + if(in_preferred_row == 0) + { + return false; + } + preferred_row=0; + } + + return false; +} + + + + +/* MCHN END */ + +/** + * Call after CreateTransaction unless you want to abort + */ + +bool CWallet::CommitTransaction(CWalletTx& wtxNew, CReserveKey& reservekey, string& reject_reason) +{ + double this_time,last_time; + this_time=mc_TimeNowAsDouble(); + last_time=this_time; + + { + LOCK2(cs_main, cs_wallet); + LogPrintf("CommitTransaction:\n%s", wtxNew.ToString()); + { +/* MCHN START */ + if(((mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) == 0) || (mc_gState->m_WalletMode & MC_WMD_MAP_TXS)) + { +/* MCHN END */ + // This is only to keep the database open to defeat the auto-flush for the + // duration of this scope. This is the only place where this optimization + // maybe makes sense; please don't do it anywhere else. + CWalletDB* pwalletdb = fFileBacked ? new CWalletDB(strWalletFile,"r") : NULL; + + // Take key pair from key pool so it won't be used again + reservekey.KeepKey(); + + // Add tx to wallet, because if it has change it's also ours, + // otherwise just for transaction history. + AddToWallet(wtxNew); + this_time=mc_TimeNowAsDouble(); +// if(csperf_debug_print)printf("AddToWallet : %8.6f\n",this_time-last_time); + last_time=this_time; + + // Notify that old coins are spent + set setCoins; + BOOST_FOREACH(const CTxIn& txin, wtxNew.vin) + { + CWalletTx &coin = mapWallet[txin.prevout.hash]; + coin.BindWallet(this); + NotifyTransactionChanged(this, coin.GetHash(), CT_UPDATED); + } + + if (fFileBacked) + delete pwalletdb; +/* MCHN START */ + } +/* MCHN END */ + } + + // Track how many getdata requests our transaction gets + mapRequestCount[wtxNew.GetHash()] = 0; + + this_time=mc_TimeNowAsDouble(); +// if(csperf_debug_print)printf("NotifyChanged : %8.6f\n",this_time-last_time); + last_time=this_time; + // Broadcast + + CPubKey pubkey; + uint32_t fCanMine= GetKeyFromAddressBook(pubkey,MC_PTP_MINE) ? MC_PTP_MINE : 0; + + if (!wtxNew.AcceptToMemoryPoolReturnReason(false,true,reject_reason)) // MCHN + { + // This must not fail. The transaction has already been signed and recorded. + LogPrintf("CommitTransaction() : Error: Transaction not valid: %s\n",reject_reason.c_str()); // MCHN + return false; + } + else + { + pwalletTxsMain->AddTx(NULL,wtxNew,-1,NULL,-1); + SyncWithWallets(wtxNew, NULL); + } + + if(fCanMine) + { + if(!GetKeyFromAddressBook(pubkey,MC_PTP_MINE)) + { + CValidationState state; + + LogPrint("mchn","mchn: Wallet lost mine permission on tx: %s (height %d) - commit, reactivating best chain\n", + wtxNew.GetHash().ToString().c_str(), chainActive.Tip()->nHeight); + if (!::ActivateBestChain(state, NULL)) + reject_reason = "ActivateBestChain failed"; + } + } + + this_time=mc_TimeNowAsDouble(); +// if(csperf_debug_print)printf("Accept : %8.6f\n",this_time-last_time); + LogPrint("mcperf","mcperf: Commit: AcceptToMemoryPool: Time: %8.6f \n", this_time-last_time); + last_time=this_time; + wtxNew.RelayWalletTransaction(); + this_time=mc_TimeNowAsDouble(); +// if(csperf_debug_print)printf("Relay : %8.6f\n",this_time-last_time); + LogPrint("mcperf","mcperf: Commit: RelayWalletTransaction : Time: %8.6f \n", this_time-last_time); + last_time=this_time; + } + return true; +} + +CAmount CWallet::GetMinimumFee(unsigned int nTxBytes, unsigned int nConfirmTarget, const CTxMemPool& pool) +{ + // payTxFee is user-set "I want to pay this much" + CAmount nFeeNeeded = payTxFee.GetFee(nTxBytes); + // user selected total at least (default=true) + if (fPayAtLeastCustomFee && nFeeNeeded > 0 && nFeeNeeded < payTxFee.GetFeePerK()) + nFeeNeeded = payTxFee.GetFeePerK(); + // User didn't set: use -txconfirmtarget to estimate... + if (nFeeNeeded == 0) + nFeeNeeded = pool.estimateFee(nConfirmTarget).GetFee(nTxBytes); + // ... unless we don't have enough mempool data, in which case fall + // back to a hard-coded fee + if (nFeeNeeded == 0) + nFeeNeeded = minTxFee.GetFee(nTxBytes); + // prevent user from paying a non-sense fee (like 1 satoshi): 0 < fee < minRelayFee + if (nFeeNeeded < ::minRelayTxFee.GetFee(nTxBytes)) + nFeeNeeded = ::minRelayTxFee.GetFee(nTxBytes); + // But always obey the maximum + if (nFeeNeeded > maxTxFee) + nFeeNeeded = maxTxFee; + return nFeeNeeded; +} + + + + +DBErrors CWallet::LoadWallet(bool& fFirstRunRet) +{ + if (!fFileBacked) + return DB_LOAD_OK; + fFirstRunRet = false; + DBErrors nLoadWalletRet = CWalletDB(strWalletFile,"cr+").LoadWallet(this); + if (nLoadWalletRet == DB_NEED_REWRITE) + { + if (CDB::Rewrite(strWalletFile, "\x04pool")) + { + LOCK(cs_wallet); + setKeyPool.clear(); + // Note: can't top-up keypool here, because wallet is locked. + // User will be prompted to unlock wallet the next operation + // the requires a new key. + } + } + + if (nLoadWalletRet != DB_LOAD_OK) + return nLoadWalletRet; + fFirstRunRet = !vchDefaultKey.IsValid(); + + uiInterface.LoadWallet(this); + + return DB_LOAD_OK; +} + + +DBErrors CWallet::ZapWalletTx(std::vector& vWtx) +{ + if (!fFileBacked) + return DB_LOAD_OK; + DBErrors nZapWalletTxRet = CWalletDB(strWalletFile,"cr+").ZapWalletTx(this, vWtx); + if (nZapWalletTxRet == DB_NEED_REWRITE) + { + if (CDB::Rewrite(strWalletFile, "\x04pool")) + { + LOCK(cs_wallet); + setKeyPool.clear(); + // Note: can't top-up keypool here, because wallet is locked. + // User will be prompted to unlock wallet the next operation + // that requires a new key. + } + } + + if (nZapWalletTxRet != DB_LOAD_OK) + return nZapWalletTxRet; + + return DB_LOAD_OK; +} + + +bool CWallet::SetAddressBook(const CTxDestination& address, const string& strName, const string& strPurpose) +{ + bool fUpdated = false; + { + LOCK(cs_wallet); // mapAddressBook + std::map::iterator mi = mapAddressBook.find(address); + fUpdated = mi != mapAddressBook.end(); + mapAddressBook[address].name = strName; + if (!strPurpose.empty()) /* update purpose only if requested */ + mapAddressBook[address].purpose = strPurpose; + } + NotifyAddressBookChanged(this, address, strName, ::IsMine(*this, address) != ISMINE_NO, + strPurpose, (fUpdated ? CT_UPDATED : CT_NEW) ); + if (!fFileBacked) + return false; + if (!strPurpose.empty() && !CWalletDB(strWalletFile).WritePurpose(CBitcoinAddress(address).ToString(), strPurpose)) + return false; + LogPrint("mchn","Stored address %s in address book, account: %s, purpose: %s.\n",CBitcoinAddress(address).ToString().c_str(),strName.c_str(),strPurpose.c_str()); + return CWalletDB(strWalletFile).WriteName(CBitcoinAddress(address).ToString(), strName); +} + +bool CWallet::DelAddressBook(const CTxDestination& address) +{ + { + LOCK(cs_wallet); // mapAddressBook + + if(fFileBacked) + { + // Delete destdata tuples associated with address + std::string strAddress = CBitcoinAddress(address).ToString(); + BOOST_FOREACH(const PAIRTYPE(string, string) &item, mapAddressBook[address].destdata) + { + CWalletDB(strWalletFile).EraseDestData(strAddress, item.first); + } + } + mapAddressBook.erase(address); + } + + NotifyAddressBookChanged(this, address, "", ::IsMine(*this, address) != ISMINE_NO, "", CT_DELETED); + + if (!fFileBacked) + return false; + CWalletDB(strWalletFile).ErasePurpose(CBitcoinAddress(address).ToString()); + return CWalletDB(strWalletFile).EraseName(CBitcoinAddress(address).ToString()); +} + +bool CWallet::SetDefaultKey(const CPubKey &vchPubKey) +{ + if (fFileBacked) + { + if (!CWalletDB(strWalletFile).WriteDefaultKey(vchPubKey)) + return false; + } + vchDefaultKey = vchPubKey; + return true; +} + +/** + * Mark old keypool keys as used, + * and generate all new keys + */ +bool CWallet::NewKeyPool() +{ + { + LOCK(cs_wallet); + CWalletDB walletdb(strWalletFile); + BOOST_FOREACH(int64_t nIndex, setKeyPool) + walletdb.ErasePool(nIndex); + setKeyPool.clear(); + + if (IsLocked()) + return false; + + int64_t nKeys = max(GetArg("-keypool", 100), (int64_t)0); + for (int i = 0; i < nKeys; i++) + { + int64_t nIndex = i+1; + walletdb.WritePool(nIndex, CKeyPool(GenerateNewKey())); + setKeyPool.insert(nIndex); + } + LogPrintf("CWallet::NewKeyPool wrote %d new keys\n", nKeys); + } + return true; +} + +bool CWallet::TopUpKeyPool(unsigned int kpSize) +{ + { + LOCK(cs_wallet); + + if (IsLocked()) + return false; + + CWalletDB walletdb(strWalletFile); + + // Top up key pool + unsigned int nTargetSize; + if (kpSize > 0) + nTargetSize = kpSize; + else +/* MCHN START */ + { + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + nTargetSize = max(GetArg("-keypool", 1), (int64_t) 0); + } + else + { + nTargetSize = max(GetArg("-keypool", 100), (int64_t) 0); + } + } +/* MCHN END */ + + while (setKeyPool.size() < (nTargetSize + 1)) + { + int64_t nEnd = 1; + if (!setKeyPool.empty()) + nEnd = *(--setKeyPool.end()) + 1; + if (!walletdb.WritePool(nEnd, CKeyPool(GenerateNewKey()))) + throw runtime_error("TopUpKeyPool() : writing generated key failed"); + setKeyPool.insert(nEnd); + LogPrintf("keypool added key %d, size=%u\n", nEnd, setKeyPool.size()); + } + } + return true; +} + +void CWallet::ReserveKeyFromKeyPool(int64_t& nIndex, CKeyPool& keypool) +{ + nIndex = -1; + keypool.vchPubKey = CPubKey(); + { + LOCK(cs_wallet); + + if (!IsLocked()) + TopUpKeyPool(); + + // Get the oldest key + if(setKeyPool.empty()) + return; + + CWalletDB walletdb(strWalletFile); + + nIndex = *(setKeyPool.begin()); + setKeyPool.erase(setKeyPool.begin()); + if (!walletdb.ReadPool(nIndex, keypool)) + throw runtime_error("ReserveKeyFromKeyPool() : read failed"); + if (!HaveKey(keypool.vchPubKey.GetID())) + throw runtime_error("ReserveKeyFromKeyPool() : unknown key in key pool"); + assert(keypool.vchPubKey.IsValid()); +// LogPrintf("keypool reserve %d\n", nIndex); + } +} + +void CWallet::KeepKey(int64_t nIndex) +{ + // Remove from key pool + if (fFileBacked) + { + CWalletDB walletdb(strWalletFile); + walletdb.ErasePool(nIndex); + } +// LogPrintf("keypool keep %d\n", nIndex); +} + +void CWallet::ReturnKey(int64_t nIndex) +{ + // Return to key pool + { + LOCK(cs_wallet); + setKeyPool.insert(nIndex); + } +// LogPrintf("keypool return %d\n", nIndex); +} + +bool CWallet::GetKeyFromPool(CPubKey& result) +{ + int64_t nIndex = 0; + CKeyPool keypool; + { + LOCK(cs_wallet); + ReserveKeyFromKeyPool(nIndex, keypool); + if (nIndex == -1) + { + if (IsLocked()) return false; + result = GenerateNewKey(); + return true; + } + KeepKey(nIndex); + result = keypool.vchPubKey; + } + return true; +} + +/* MCHN START */ +bool CWallet::GetKeyFromAddressBook(CPubKey& result,uint32_t type,const set* addresses,map* mapSpecialEntity) +{ + if((mc_gState->m_NetworkParams->IsProtocolMultichain() == 0) || + (type == 0)) + { + result=vchDefaultKey; + return true; + } + + CKeyID keyID; + uint32_t perm; + + if((addresses == NULL) && (mapSpecialEntity == NULL)) + { + keyID=vchDefaultKey.GetID(); + perm=mc_gState->m_Permissions->GetAllPermissions(NULL,(unsigned char*)(&keyID),type); + if(perm == type) + { + result=vchDefaultKey; + return true; + } + } + + BOOST_FOREACH(const PAIRTYPE(CBitcoinAddress, CAddressBookData)& item, mapAddressBook) + { + const CBitcoinAddress& address = item.first; + + if((addresses == NULL) || (addresses->count(address.Get()))) + { + if(address.GetKeyID(keyID)) + { + perm=mc_gState->m_Permissions->GetAllPermissions(NULL,(unsigned char*)(&keyID),type); + if(perm == type) + { + bool take_it=true; + if(mapSpecialEntity) + { + if(type & MC_PTP_ISSUE) + { + unsigned char *lpEntity=NULL; + + std::map::const_iterator it = mapSpecialEntity->find(MC_PTP_ISSUE); + if (it != mapSpecialEntity->end()) + { + lpEntity=(unsigned char*)(&(it->second)); + } + take_it=false; + if(mc_gState->m_Permissions->CanIssue(lpEntity,(unsigned char*)(&keyID))) + { + take_it=true; + } + } + if(type & MC_PTP_WRITE) + { + unsigned char *lpEntity=NULL; + + std::map::const_iterator it = mapSpecialEntity->find(MC_PTP_WRITE); + if (it != mapSpecialEntity->end()) + { + lpEntity=(unsigned char*)(&(it->second)); + } + take_it=false; + if(mc_gState->m_Permissions->CanWrite(lpEntity,(unsigned char*)(&keyID))) + { + take_it=true; + } + } + } + + CKey key; + if(take_it) + { + if(GetKey(keyID, key)) + { + result=key.GetPubKey(); + return true; + } + } + } + } + } + } + return false; +} +/* MCHN END */ + +int64_t CWallet::GetOldestKeyPoolTime() +{ + int64_t nIndex = 0; + CKeyPool keypool; + ReserveKeyFromKeyPool(nIndex, keypool); + if (nIndex == -1) + return GetTime(); + ReturnKey(nIndex); + return keypool.nTime; +} + +std::map CWallet::GetAddressBalances() +{ + map balances; + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + LOCK(cs_wallet); + vector vecOutputs; + AvailableCoins(vecOutputs, false, NULL, false,true); + BOOST_FOREACH(const COutput& out, vecOutputs) + { + if(out.coin.IsTrusted() && + out.coin.IsFinal() && + (out.coin.BlocksToMaturity() <=0) ) + { + if(out.coin.m_EntityType) + { + CTxOut txout; + out.GetHashAndTxOut(txout); + + CTxDestination address; + if (ExtractDestination(txout.scriptPubKey, address)) + { + if (!balances.count(address)) + balances[address] = 0; + balances[address] += txout.nValue; + } + } + } + } + } + else + { + LOCK(cs_wallet); + BOOST_FOREACH(PAIRTYPE(uint256, CWalletTx) walletEntry, mapWallet) + { + CWalletTx *pcoin = &walletEntry.second; + + if (!IsFinalTx(*pcoin) || !pcoin->IsTrusted()) + continue; + + if (pcoin->IsCoinBase() && pcoin->GetBlocksToMaturity() > 0) + continue; + + int nDepth = pcoin->GetDepthInMainChain(); + if (nDepth < (pcoin->IsFromMe(ISMINE_ALL) ? 0 : 1)) + continue; + + for (unsigned int i = 0; i < pcoin->vout.size(); i++) + { + CTxDestination addr; + if (!IsMine(pcoin->vout[i])) + continue; + if(!ExtractDestination(pcoin->vout[i].scriptPubKey, addr)) + continue; + + CAmount n = IsSpent(walletEntry.first, i) ? 0 : pcoin->vout[i].nValue; + + if (!balances.count(addr)) + balances[addr] = 0; + balances[addr] += n; + } + } + } + + return balances; +} + +set< set > CWallet::GetAddressGroupings() +{ + AssertLockHeld(cs_wallet); // mapWallet + set< set > groupings; + set grouping; + set< set > ret; + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + LOCK(cs_wallet); + set addresses_with_outputs; + vector vecOutputs; + AvailableCoins(vecOutputs, false, NULL, false,true); + BOOST_FOREACH(const COutput& out, vecOutputs) + { + if(out.coin.IsTrusted() && + out.coin.IsFinal() && + (out.coin.BlocksToMaturity() <=0) ) + { + if(out.coin.m_EntityType) + { + CTxOut txout; + out.GetHashAndTxOut(txout); + + CTxDestination address; + if (ExtractDestination(txout.scriptPubKey, address)) + { + if (!addresses_with_outputs.count(address)) + addresses_with_outputs.insert(address); + } + } + } + } + BOOST_FOREACH(const CTxDestination& address, addresses_with_outputs) + { + grouping.clear(); + grouping.insert(address); + groupings.insert(grouping); + } + return groupings; + } + else + { + BOOST_FOREACH(PAIRTYPE(uint256, CWalletTx) walletEntry, mapWallet) + { + CWalletTx *pcoin = &walletEntry.second; + + if (pcoin->vin.size() > 0) + { + bool any_mine = false; + // group all input addresses with each other + BOOST_FOREACH(CTxIn txin, pcoin->vin) + { + CTxDestination address; + if(!IsMine(txin)) /* If this input isn't mine, ignore it */ + continue; + if(!ExtractDestination(mapWallet[txin.prevout.hash].vout[txin.prevout.n].scriptPubKey, address)) + continue; + grouping.insert(address); + any_mine = true; + } + + // group change with input addresses + if (any_mine) + { + BOOST_FOREACH(CTxOut txout, pcoin->vout) + if (IsChange(txout)) + { + CTxDestination txoutAddr; + if(!ExtractDestination(txout.scriptPubKey, txoutAddr)) + continue; + grouping.insert(txoutAddr); + } + } + if (grouping.size() > 0) + { + groupings.insert(grouping); + grouping.clear(); + } + } + + // group lone addrs by themselves + for (unsigned int i = 0; i < pcoin->vout.size(); i++) + if (IsMine(pcoin->vout[i])) + { + CTxDestination address; + if(!ExtractDestination(pcoin->vout[i].scriptPubKey, address)) + continue; + grouping.insert(address); + groupings.insert(grouping); + grouping.clear(); + } + } + + set< set* > uniqueGroupings; // a set of pointers to groups of addresses + map< CTxDestination, set* > setmap; // map addresses to the unique group containing it + BOOST_FOREACH(set grouping, groupings) + { + // make a set of all the groups hit by this new group + set< set* > hits; + map< CTxDestination, set* >::iterator it; + BOOST_FOREACH(CTxDestination address, grouping) + if ((it = setmap.find(address)) != setmap.end()) + hits.insert((*it).second); + + // merge all hit groups into a new single group and delete old groups + set* merged = new set(grouping); + BOOST_FOREACH(set* hit, hits) + { + merged->insert(hit->begin(), hit->end()); + uniqueGroupings.erase(hit); + delete hit; + } + uniqueGroupings.insert(merged); + + // update setmap + BOOST_FOREACH(CTxDestination element, *merged) + setmap[element] = merged; + } + + BOOST_FOREACH(set* uniqueGrouping, uniqueGroupings) + { + ret.insert(*uniqueGrouping); + delete uniqueGrouping; + } + } + + return ret; +} + +set CWallet::GetAccountAddresses(string strAccount) const +{ + LOCK(cs_wallet); + set result; + BOOST_FOREACH(const PAIRTYPE(CTxDestination, CAddressBookData)& item, mapAddressBook) + { + const CTxDestination& address = item.first; + const string& strName = item.second.name; + if (strName == strAccount) + result.insert(address); + } + return result; +} + +bool CReserveKey::GetReservedKey(CPubKey& pubkey) +{ + if (nIndex == -1) + { + CKeyPool keypool; + pwallet->ReserveKeyFromKeyPool(nIndex, keypool); + if (nIndex != -1) + vchPubKey = keypool.vchPubKey; + else { + return false; + } + } + assert(vchPubKey.IsValid()); + pubkey = vchPubKey; + return true; +} + +void CReserveKey::KeepKey() +{ + if (nIndex != -1) + pwallet->KeepKey(nIndex); + nIndex = -1; + vchPubKey = CPubKey(); +} + +void CReserveKey::ReturnKey() +{ + if (nIndex != -1) + pwallet->ReturnKey(nIndex); + nIndex = -1; + vchPubKey = CPubKey(); +} + +void CWallet::GetAllReserveKeys(set& setAddress) const +{ + setAddress.clear(); + + CWalletDB walletdb(strWalletFile); + + LOCK2(cs_main, cs_wallet); + BOOST_FOREACH(const int64_t& id, setKeyPool) + { + CKeyPool keypool; + if (!walletdb.ReadPool(id, keypool)) + throw runtime_error("GetAllReserveKeyHashes() : read failed"); + assert(keypool.vchPubKey.IsValid()); + CKeyID keyID = keypool.vchPubKey.GetID(); + if (!HaveKey(keyID)) + throw runtime_error("GetAllReserveKeyHashes() : unknown key in key pool"); + setAddress.insert(keyID); + } +} + +void CWallet::UpdatedTransaction(const uint256 &hashTx) +{ +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, used only in QT + { + LOCK(cs_wallet); + // Only notify UI if this transaction is in this wallet + map::const_iterator mi = mapWallet.find(hashTx); + if (mi != mapWallet.end()) + NotifyTransactionChanged(this, hashTx, CT_UPDATED); + } +} + +void CWallet::LockCoin(COutPoint& output) +{ + AssertLockHeld(cs_wallet); // setLockedCoins + setLockedCoins.insert(output); +} + +void CWallet::UnlockCoin(COutPoint& output) +{ + AssertLockHeld(cs_wallet); // setLockedCoins + setLockedCoins.erase(output); +} + +void CWallet::UnlockAllCoins() +{ + AssertLockHeld(cs_wallet); // setLockedCoins + setLockedCoins.clear(); +} + +bool CWallet::IsLockedCoin(uint256 hash, unsigned int n) const +{ + AssertLockHeld(cs_wallet); // setLockedCoins + COutPoint outpt(hash, n); + + return (setLockedCoins.count(outpt) > 0); +} + +void CWallet::ListLockedCoins(std::vector& vOutpts) +{ + AssertLockHeld(cs_wallet); // setLockedCoins + for (std::set::iterator it = setLockedCoins.begin(); + it != setLockedCoins.end(); it++) { + COutPoint outpt = (*it); + vOutpts.push_back(outpt); + } +} + +/** @} */ // end of Actions + +class CAffectedKeysVisitor : public boost::static_visitor { +private: + const CKeyStore &keystore; + std::vector &vKeys; + +public: + CAffectedKeysVisitor(const CKeyStore &keystoreIn, std::vector &vKeysIn) : keystore(keystoreIn), vKeys(vKeysIn) {} + + void Process(const CScript &script) { + txnouttype type; + std::vector vDest; + int nRequired; + if (ExtractDestinations(script, type, vDest, nRequired)) { + BOOST_FOREACH(const CTxDestination &dest, vDest) + boost::apply_visitor(*this, dest); + } + } + + void operator()(const CKeyID &keyId) { + if (keystore.HaveKey(keyId)) + vKeys.push_back(keyId); + } + + void operator()(const CScriptID &scriptId) { + CScript script; + if (keystore.GetCScript(scriptId, script)) + Process(script); + } + + void operator()(const CNoDestination &none) {} +}; + +void CWallet::GetKeyBirthTimes(std::map &mapKeyBirth) const { + AssertLockHeld(cs_wallet); // mapKeyMetadata + mapKeyBirth.clear(); + + // get birth times for keys with metadata + + for (std::map::const_iterator it = mapKeyMetadata.begin(); it != mapKeyMetadata.end(); it++) + if (it->second.nCreateTime) + mapKeyBirth[it->first] = it->second.nCreateTime; + + // map in which we'll infer heights of other keys + CBlockIndex *pindexMax = chainActive[std::max(0, chainActive.Height() - 144)]; // the tip can be reorganised; use a 144-block safety margin + std::map mapKeyFirstBlock; + std::set setKeys; + GetKeys(setKeys); + BOOST_FOREACH(const CKeyID &keyid, setKeys) { + if (mapKeyBirth.count(keyid) == 0) + mapKeyFirstBlock[keyid] = pindexMax; + } + setKeys.clear(); + + // if there are no such keys, we're done + if (mapKeyFirstBlock.empty()) + return; + + // find first block that affects those keys, if there are any left +/* MCHN START */ + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_Buffer *entity_rows; + entity_rows=new mc_Buffer; + entity_rows->Initialize(MC_TDB_ENTITY_KEY_SIZE,MC_TDB_ROW_SIZE,MC_BUF_MODE_DEFAULT); + + mc_TxEntity entity; + mc_TxEntityRow *lpEntTx; + int first_block; + int tx_count; + + entity.Zero(); + for (std::map::const_iterator it = mapKeyFirstBlock.begin(); it != mapKeyFirstBlock.end(); it++) + { + memcpy(entity.m_EntityID,&(it->first),MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + + first_block=chainActive.Height(); + tx_count=pwalletTxsMain->GetListSize(&entity,NULL); + if(tx_count) + { + pwalletTxsMain->GetList(&entity,1,1,entity_rows); + lpEntTx=(mc_TxEntityRow*)entity_rows->GetRow(0); + first_block=lpEntTx->m_Block; + if( (first_block < 0) || (first_block > chainActive.Height()) ) + { + first_block=chainActive.Height(); + } + } + CBlockIndex* pindex=chainActive[first_block]; + mapKeyBirth[it->first] = pindex->GetBlockTime() - 12 * Params().TargetSpacing(); // block times can be 2h off + } + + delete entity_rows; + } + else + { +/* MCHN END */ + std::vector vAffected; + for (std::map::const_iterator it = mapWallet.begin(); it != mapWallet.end(); it++) { + // iterate over all wallet transactions... + const CWalletTx &wtx = (*it).second; + BlockMap::const_iterator blit = mapBlockIndex.find(wtx.hashBlock); + if (blit != mapBlockIndex.end() && chainActive.Contains(blit->second)) { + // ... which are already in a block + int nHeight = blit->second->nHeight; + BOOST_FOREACH(const CTxOut &txout, wtx.vout) { + // iterate over all their outputs + CAffectedKeysVisitor(*this, vAffected).Process(txout.scriptPubKey); + BOOST_FOREACH(const CKeyID &keyid, vAffected) { + // ... and all their affected keys + std::map::iterator rit = mapKeyFirstBlock.find(keyid); + if (rit != mapKeyFirstBlock.end() && nHeight < rit->second->nHeight) + rit->second = blit->second; + } + vAffected.clear(); + } + } + } + // Extract block timestamps for those keys + for (std::map::const_iterator it = mapKeyFirstBlock.begin(); it != mapKeyFirstBlock.end(); it++) + mapKeyBirth[it->first] = it->second->GetBlockTime() - 12 * Params().TargetSpacing(); // MCHN block times can be 2h off + } + +} + +bool CWallet::AddDestData(const CTxDestination &dest, const std::string &key, const std::string &value) +{ + if (boost::get(&dest)) + return false; + + mapAddressBook[dest].destdata.insert(std::make_pair(key, value)); + if (!fFileBacked) + return true; + return CWalletDB(strWalletFile).WriteDestData(CBitcoinAddress(dest).ToString(), key, value); +} + +bool CWallet::EraseDestData(const CTxDestination &dest, const std::string &key) +{ + if (!mapAddressBook[dest].destdata.erase(key)) + return false; + if (!fFileBacked) + return true; + return CWalletDB(strWalletFile).EraseDestData(CBitcoinAddress(dest).ToString(), key); +} + +bool CWallet::LoadDestData(const CTxDestination &dest, const std::string &key, const std::string &value) +{ + mapAddressBook[dest].destdata.insert(std::make_pair(key, value)); + return true; +} + +bool CWallet::GetDestData(const CTxDestination &dest, const std::string &key, std::string *value) const +{ + std::map::const_iterator i = mapAddressBook.find(dest); + if(i != mapAddressBook.end()) + { + CAddressBookData::StringMap::const_iterator j = i->second.destdata.find(key); + if(j != i->second.destdata.end()) + { + if(value) + *value = j->second; + return true; + } + } + return false; +} + +CKeyPool::CKeyPool() +{ + nTime = GetTime(); +} + +CKeyPool::CKeyPool(const CPubKey& vchPubKeyIn) +{ + nTime = GetTime(); + vchPubKey = vchPubKeyIn; +} + +CWalletKey::CWalletKey(int64_t nExpires) +{ + nTimeCreated = (nExpires ? GetTime() : 0); + nTimeExpires = nExpires; +} + +int CMerkleTx::SetMerkleBranch(const CBlock& block) +{ + AssertLockHeld(cs_main); + CBlock blockTmp; + + // Update the tx's hashBlock + hashBlock = block.GetHash(); + + // Locate the transaction + for (nIndex = 0; nIndex < (int)block.vtx.size(); nIndex++) + if (block.vtx[nIndex] == *(CTransaction*)this) + break; + if (nIndex == (int)block.vtx.size()) + { + vMerkleBranch.clear(); + nIndex = -1; + LogPrintf("ERROR: SetMerkleBranch() : couldn't find tx in block\n"); + return 0; + } + // Fill in merkle branch + vMerkleBranch = block.GetMerkleBranch(nIndex); + + // Is the tx in a block that's in the main chain + BlockMap::iterator mi = mapBlockIndex.find(hashBlock); + if (mi == mapBlockIndex.end()) + return 0; + const CBlockIndex* pindex = (*mi).second; + if (!pindex || !chainActive.Contains(pindex)) + return 0; + + return chainActive.Height() - pindex->nHeight + 1; +} + +int CMerkleTx::GetDepthInMainChainINTERNAL(const CBlockIndex* &pindexRet) const +{ + if((mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) == 0) + { + if (hashBlock == 0 || nIndex == -1) + return 0; + } + AssertLockHeld(cs_main); + + CBlockIndex* pindex = NULL; + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) + { + mc_TxDefRow txdef; + pwalletTxsMain->FindWalletTx(GetHash(),&txdef); + if((txdef.m_Flags & MC_TFL_INVALID) == 0) + { + if(txdef.m_Block >= 0) + { + if(txdef.m_Block <= chainActive.Height()) + { + pindex=chainActive[txdef.m_Block]; + } + } + } + if (!pindex || !chainActive.Contains(pindex)) + return 0; + } + else + { + // Find the block it claims to be in + BlockMap::iterator mi = mapBlockIndex.find(hashBlock); + if (mi == mapBlockIndex.end()) + return 0; +// CBlockIndex* pindex = (*mi).second; + pindex = (*mi).second; + if (!pindex || !chainActive.Contains(pindex)) + return 0; + + // Make sure the merkle branch connects to this block + if (!fMerkleVerified) + { + if (CBlock::CheckMerkleBranch(GetHash(), vMerkleBranch, nIndex) != pindex->hashMerkleRoot) + return 0; + fMerkleVerified = true; + } + } + pindexRet = pindex; + return chainActive.Height() - pindex->nHeight + 1; +} + +int CMerkleTx::GetDepthInMainChain(const CBlockIndex* &pindexRet) const +{ + AssertLockHeld(cs_main); + int nResult = GetDepthInMainChainINTERNAL(pindexRet); + if (nResult == 0 && !mempool.exists(GetHash())) + return -1; // Not in chain, not in mempool + + return nResult; +} + +int CMerkleTx::GetBlocksToMaturity() const +{ + if (!IsCoinBase()) + return 0; + return max(0, (COINBASE_MATURITY+1) - GetDepthInMainChain()); +} + + +bool CMerkleTx::AcceptToMemoryPool(bool fLimitFree, bool fRejectInsaneFee) +{ + CValidationState state; + return ::AcceptToMemoryPool(mempool, state, *this, fLimitFree, NULL, fRejectInsaneFee); +} + +/* MCHN START */ +bool CMerkleTx::AcceptToMemoryPoolReturnReason(bool fLimitFree, bool fRejectInsaneFee,string& reject_reason) +{ + CValidationState state; + bool result=::AcceptToMemoryPool(mempool, state, *this, fLimitFree, NULL, fRejectInsaneFee,false); + + if(!result) + { + if(state.IsInvalid()) + reject_reason = strprintf("%i: %s", state.GetRejectCode(), state.GetRejectReason()); + else + reject_reason = state.GetRejectReason(); + } + + return result; +} +/* MCHN END */ + diff --git a/src/wallet/wallet.h b/src/wallet/wallet.h new file mode 100644 index 00000000..250e2dc6 --- /dev/null +++ b/src/wallet/wallet.h @@ -0,0 +1,1347 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_WALLET_H +#define BITCOIN_WALLET_H + +#include "structs/amount.h" +#include "primitives/block.h" +#include "primitives/transaction.h" +#include "wallet/crypter.h" +#include "keys/key.h" +#include "wallet/keystore.h" +#include "core/main.h" +#include "ui/ui_interface.h" +#include "wallet_ismine.h" +#include "walletdb.h" + +/* MCHN START */ +#include "wallet/wallettxdb.h" +/* MCHN END */ + +#include +#include +#include +#include +#include +#include +#include +#include + + + +/** + * Settings + */ +extern CFeeRate payTxFee; +extern CAmount maxTxFee; +extern unsigned int nTxConfirmTarget; +extern bool bSpendZeroConfChange; +extern bool fSendFreeTransactions; +extern bool fPayAtLeastCustomFee; + +//! -paytxfee default +static const CAmount DEFAULT_TRANSACTION_FEE = 0; +//! -paytxfee will warn if called with a higher fee than this amount (in satoshis) per KB +static const CAmount nHighTransactionFeeWarning = 0.01 * COIN; +//! -maxtxfee default +static const CAmount DEFAULT_TRANSACTION_MAXFEE = 0.1 * COIN; +//! -maxtxfee will warn if called with a higher fee than this amount (in satoshis) +static const CAmount nHighTransactionMaxFeeWarning = 100 * nHighTransactionFeeWarning; +//! Largest (in bytes) free transaction we're willing to create +static const unsigned int MAX_FREE_TRANSACTION_CREATE_SIZE = 1000; + +class CAccountingEntry; +class CCoinControl; +class COutput; +class CReserveKey; +class CScript; +class CWalletTx; +/* MCHN START */ +struct mc_WalletTxs; +#define MC_TFL_INVALID 0x00000001 +#define MC_TFL_IS_COINBASE 0x00000002 +#define MC_TFL_ALL_INPUTS_ARE_FINAL 0x00000004 +#define MC_TFL_FROM_ME 0x00010000 +#define MC_TFL_ALL_INPUTS_FROM_ME 0x00020000 +#define MC_TFL_IS_CHANGE 0x00040000 +#define MC_TFL_IS_SPENDABLE 0x00080000 +#define MC_TFL_IMPOSSIBLE 0x80000000 + +#define MC_CSF_ALLOW_SPENDABLE_P2SH 0x00000001 +#define MC_CSF_ALLOW_NOT_SPENDABLE_P2SH 0x00000002 +#define MC_CSF_ALLOW_NOT_SPENDABLE 0x00000004 +#define MC_CSF_SIGN 0x00000008 + + +class mc_Coin +{ +public: + COutPoint m_OutPoint; + CTxOut m_TXOut; + uint160 m_EntityID; + uint32_t m_EntityType; + int m_Block; + uint32_t m_Flags; + uint32_t m_LockTime; + + mc_Coin() + { + Zero(); + } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(m_OutPoint); + READWRITE(m_TXOut); + READWRITE(m_EntityID); + READWRITE(m_EntityType); + READWRITE(m_Block); + READWRITE(m_Flags); + } + + void Zero(); + bool IsFinal() const; + int BlocksToMaturity() const; + bool IsTrusted() const; + int GetDepthInMainChain() const; + + std::string ToString() const; +}; + + +/* MCHN END */ +/** (client) version numbers for particular wallet features */ +enum WalletFeature +{ + FEATURE_BASE = 10500, // the earliest version new wallets supports (only useful for getinfo's clientversion output) + + FEATURE_WALLETCRYPT = 40000, // wallet encryption + FEATURE_COMPRPUBKEY = 60000, // compressed public keys + + FEATURE_LATEST = 60000 +}; + +/* MCHN START */ +class CExchangeStatus +{ +public: + uint256 nOfferHash; + int32_t nLockStatus; + uint32_t nTimestamp; + + CExchangeStatus(uint256 offerHashIn,int32_t lockStatus,int32_t timestampIn) + { + nOfferHash = offerHashIn; nLockStatus = lockStatus; nTimestamp=timestampIn; + } + + std::string ToString() const; +}; + +class CAssetGroup +{ + +public: + + int nThisGroup; + int nPrevGroup; + int nNextGroup; + int nSize; + + CAssetGroup() + { + nThisGroup=0; + nPrevGroup=0; + nNextGroup=0; + nSize=0; + } + + ~CAssetGroup() + { + } + +}; + + +class CAssetGroupTree +{ +private: + int nAssetsPerGroup; + int nMaxAssetsPerGroup; + int nOptimalGroupCount; + int nMode; + void Clear(); + void Destroy(); + + mc_Buffer *lpAssets; + mc_Buffer *lpAssetGroups; + int *lpTmpGroupBuffer; + + CAssetGroup *FindAndShiftBestGroup(int assets); + +public: + + + CAssetGroupTree() + { + Clear(); + } + + ~CAssetGroupTree() + { + Destroy(); + } + + void Dump(); + int Resize(int newAssets); + int Initialize(int assetsPerGroup,int maxAssetsPerGroup,int optimalGroupCount,int mode); + int GroupCount(); + int GetGroup(mc_Buffer *assets,int addIfNeeded); + int GetGroup(unsigned char *assetRef,int addIfNeeded); +}; + +/* MCHN END */ + + +/** A key pool entry */ +class CKeyPool +{ +public: + int64_t nTime; + CPubKey vchPubKey; + + CKeyPool(); + CKeyPool(const CPubKey& vchPubKeyIn); + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + if (!(nType & SER_GETHASH)) + READWRITE(nVersion); + READWRITE(nTime); + READWRITE(vchPubKey); + } +}; + +/** Address book data */ +class CAddressBookData +{ +public: + std::string name; + std::string purpose; + + CAddressBookData() + { + purpose = "unknown"; + } + + typedef std::map StringMap; + StringMap destdata; +}; + +/** + * A CWallet is an extension of a keystore, which also maintains a set of transactions and balances, + * and provides the ability to create new transactions. + */ +class CWallet : public CCryptoKeyStore, public CValidationInterface +{ +private: + bool SelectCoins(const CAmount& nTargetValue, std::set >& setCoinsRet, CAmount& nValueRet, const CCoinControl *coinControl = NULL) const; + + CWalletDB *pwalletdbEncryption; + + //! the current wallet version: clients below this version are not able to load the wallet + int nWalletVersion; + + //! the maximum wallet format version: memory-only variable that specifies to what version this wallet may be upgraded + int nWalletMaxVersion; + + int64_t nNextResend; + int64_t nLastResend; + + /** + * Used to keep track of spent outpoints, and + * detect and report conflicts (double-spends or + * mutated transactions where the mutant gets mined). + */ + typedef std::multimap TxSpends; + TxSpends mapTxSpends; + void AddToSpends(const COutPoint& outpoint, const uint256& wtxid); + void AddToSpends(const uint256& wtxid); + + void SyncMetaData(std::pair); + +public: + /* + * Main wallet lock. + * This lock protects all the fields added by CWallet + * except for: + * fFileBacked (immutable after instantiation) + * strWalletFile (immutable after instantiation) + */ + mutable CCriticalSection cs_wallet; + +/* MCHN START */ + mutable CCriticalSection cs_wallet_send; +/* MCHN END */ + + bool fFileBacked; + std::string strWalletFile; + + std::set setKeyPool; + std::map mapKeyMetadata; + + typedef std::map MasterKeyMap; + MasterKeyMap mapMasterKeys; + unsigned int nMasterKeyMaxID; + + CWallet() + { + SetNull(); + } + + CWallet(std::string strWalletFileIn) + { + SetNull(); + + strWalletFile = strWalletFileIn; + fFileBacked = true; + } + + ~CWallet() + { + delete pwalletdbEncryption; + pwalletdbEncryption = NULL; +/* MCHN START */ + if(lpAssetGroups) + { + delete lpAssetGroups; + lpAssetGroups=NULL; + } +/* MCHN END */ + } + + void SetNull() + { + nWalletVersion = FEATURE_BASE; + nWalletMaxVersion = FEATURE_BASE; + fFileBacked = false; + nMasterKeyMaxID = 0; + pwalletdbEncryption = NULL; + nOrderPosNext = 0; + nNextResend = 0; + nLastResend = 0; + nTimeFirstKey = 0; +/* MCHN START */ + nNextUnspentOptimization=0; + lpAssetGroups=NULL; +/* MCHN END */ + } + + std::map mapWallet; + +/* MCHN START */ + + std::map mapUnspent; + std::set setPurged; + uint32_t nNextUnspentOptimization; + + CAssetGroupTree *lpAssetGroups; + mc_WalletTxs *lpWalletTxs; + void DestroyWalletTxs(); + +// std::map mapExchanges; + +/* MCHN END */ + + int64_t nOrderPosNext; + std::map mapRequestCount; + + std::map mapAddressBook; + + CPubKey vchDefaultKey; + + std::set setLockedCoins; + + int64_t nTimeFirstKey; + + const CWalletTx* GetWalletTx(const uint256& hash) const; + + //! check whether we are allowed to upgrade (or already support) to the named feature + bool CanSupportFeature(enum WalletFeature wf) { AssertLockHeld(cs_wallet); return nWalletMaxVersion >= wf; } + +/* MCHN START */ +// void AvailableCoins(std::vector& vCoins, bool fOnlyConfirmed=true, const CCoinControl *coinControl = NULL) const; + void AvailableCoins(std::vector& vCoins, bool fOnlyConfirmed=true, const CCoinControl *coinControl = NULL, bool fOnlyUnlocked=true, + bool fOnlyCoinsNoTxs=false, uint160 addr=0, uint32_t flags=MC_CSF_ALLOW_SPENDABLE_P2SH) const; +/* MCHN END */ + bool SelectCoinsMinConf(const CAmount& nTargetValue, int nConfMine, int nConfTheirs, std::vector vCoins, std::set >& setCoinsRet, CAmount& nValueRet) const; + + bool IsSpent(const uint256& hash, unsigned int n) const; + + bool IsLockedCoin(uint256 hash, unsigned int n) const; + void LockCoin(COutPoint& output); + void UnlockCoin(COutPoint& output); + void UnlockAllCoins(); + void ListLockedCoins(std::vector& vOutpts); + + /** + * keystore implementation + * Generate a new key + */ + CPubKey GenerateNewKey(); + //! Adds a key to the store, and saves it to disk. + bool AddKeyPubKey(const CKey& key, const CPubKey &pubkey); + //! Adds a key to the store, without saving it to disk (used by LoadWallet) + bool LoadKey(const CKey& key, const CPubKey &pubkey) { return CCryptoKeyStore::AddKeyPubKey(key, pubkey); } + //! Load metadata (used by LoadWallet) + bool LoadKeyMetadata(const CPubKey &pubkey, const CKeyMetadata &metadata); + + bool LoadMinVersion(int nVersion) { AssertLockHeld(cs_wallet); nWalletVersion = nVersion; nWalletMaxVersion = std::max(nWalletMaxVersion, nVersion); return true; } + + //! Adds an encrypted key to the store, and saves it to disk. + bool AddCryptedKey(const CPubKey &vchPubKey, const std::vector &vchCryptedSecret); + //! Adds an encrypted key to the store, without saving it to disk (used by LoadWallet) + bool LoadCryptedKey(const CPubKey &vchPubKey, const std::vector &vchCryptedSecret); + bool AddCScript(const CScript& redeemScript); + bool LoadCScript(const CScript& redeemScript); + + //! Adds a destination data tuple to the store, and saves it to disk + bool AddDestData(const CTxDestination &dest, const std::string &key, const std::string &value); + //! Erases a destination data tuple in the store and on disk + bool EraseDestData(const CTxDestination &dest, const std::string &key); + //! Adds a destination data tuple to the store, without saving it to disk + bool LoadDestData(const CTxDestination &dest, const std::string &key, const std::string &value); + //! Look up a destination data tuple in the store, return true if found false otherwise + bool GetDestData(const CTxDestination &dest, const std::string &key, std::string *value) const; + + //! Adds a watch-only address to the store, and saves it to disk. + bool AddWatchOnly(const CScript &dest); + bool RemoveWatchOnly(const CScript &dest); + //! Adds a watch-only address to the store, without saving it to disk (used by LoadWallet) + bool LoadWatchOnly(const CScript &dest); + + bool Unlock(const SecureString& strWalletPassphrase); + bool ChangeWalletPassphrase(const SecureString& strOldWalletPassphrase, const SecureString& strNewWalletPassphrase); + bool EncryptWallet(const SecureString& strWalletPassphrase); + + void GetKeyBirthTimes(std::map &mapKeyBirth) const; + + /** + * Increment the next transaction order id + * @return next transaction order id + */ + int64_t IncOrderPosNext(CWalletDB *pwalletdb = NULL); + + typedef std::pair TxPair; + typedef std::multimap TxItems; + + /** + * Get the wallet's activity log + * @return multimap of ordered transactions and accounting entries + * @warning Returned pointers are *only* valid within the scope of passed acentries + */ + TxItems OrderedTxItems(std::list& acentries, std::string strAccount = ""); + + void MarkDirty(); + bool AddToWallet(const CWalletTx& wtxIn, bool fFromLoadWallet=false); + void SyncTransaction(const CTransaction& tx, const CBlock* pblock); + bool AddToWalletIfInvolvingMe(const CTransaction& tx, const CBlock* pblock, bool fUpdate); + void EraseFromWallet(const uint256 &hash); + int ScanForWalletTransactions(CBlockIndex* pindexStart, bool fUpdate = false, bool fOnlyUnsynced = false); + void ReacceptWalletTransactions(); + void ResendWalletTransactions(bool fForce = false); + CAmount GetBalance() const; + CAmount GetUnconfirmedBalance() const; + CAmount GetImmatureBalance() const; + CAmount GetWatchOnlyBalance() const; + CAmount GetUnconfirmedWatchOnlyBalance() const; + CAmount GetImmatureWatchOnlyBalance() const; + bool CreateTransaction(const std::vector >& vecSend, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl *coinControl = NULL); + bool CreateTransaction(CScript scriptPubKey, const CAmount& nValue, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl *coinControl = NULL); +/* MCHN START */ + int SelectMultiChainCombineCoinsMinConf(int nConfMine, int nConfTheirs, std::vector vCoins, mc_Buffer *in_map, mc_Buffer *in_amounts, + int in_selected_row,int in_asset_row, + std::set >& setCoinsRet,int max_inputs) const; + bool SelectMultiChainCoinsMinConf(const CAmount& nTargetValue, int nConfMine, int nConfTheirs, std::vector vCoins, mc_Buffer *in_map, mc_Buffer *in_amounts, + int in_selected_row,int in_asset_row,int in_preferred_row, + std::set >& setCoinsRet, CAmount& nValueRet) const; + bool SelectMultiChainCoins(const CAmount& nTargetValue, std::vector &vCoins, mc_Buffer *in_map, mc_Buffer *in_amounts, + int in_selected_row,int in_asset_row,int in_preferred_row, + std::set >& setCoinsRet, CAmount& nValueRet, const CCoinControl* coinControl) const; + bool CreateMultiChainTransaction(const std::vector >& vecSend, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl *coinControl = NULL, + const std::set* addresses = NULL,int min_conf = 1,int min_inputs = -1,int max_inputs = -1, const std::vector *lpCoinsToUse = NULL); + bool CreateTransaction(CScript scriptPubKey, const CAmount& nValue, CScript scriptOpReturn, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl *coinControl = NULL, + const std::set* addresses = NULL,int min_conf = 1,int min_inputs = -1,int max_inputs = -1, const std::vector *lpCoinsToUse = NULL); + bool CreateTransaction(std::vector scriptPubKeys, const CAmount& nValue, CScript scriptOpReturn, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl* coinControl = NULL, + const std::set* addresses = NULL,int min_conf = 1,int min_inputs = -1,int max_inputs = -1, const std::vector *lpCoinsToUse = NULL); + bool CreateAndCommitOptimizeTransaction(CWalletTx& wtxNew,std::string& strFailReason, + const std::set* addresses = NULL,int min_conf = 1,int min_inputs = -1,int max_inputs = -1); + bool OptimizeUnspentList(); + bool UpdateUnspentList(const CWalletTx& wtx, bool update_inputs); + bool InitializeUnspentList(); + void PurgeSpentCoins(int min_depth,int max_coins); +/* MCHN END */ + bool CommitTransaction(CWalletTx& wtxNew, CReserveKey& reservekey, std::string& reject_reason); + + static CFeeRate minTxFee; + static CAmount GetMinimumFee(unsigned int nTxBytes, unsigned int nConfirmTarget, const CTxMemPool& pool); + + bool NewKeyPool(); + bool TopUpKeyPool(unsigned int kpSize = 0); + void ReserveKeyFromKeyPool(int64_t& nIndex, CKeyPool& keypool); + void KeepKey(int64_t nIndex); + void ReturnKey(int64_t nIndex); + bool GetKeyFromPool(CPubKey &key); + int64_t GetOldestKeyPoolTime(); + void GetAllReserveKeys(std::set& setAddress) const; + +/* MCHN START */ + bool GetKeyFromAddressBook(CPubKey& result,uint32_t type,const std::set* addresses = NULL,std::map* mapSpecialEntity = NULL); +/* MCHN END */ + + + std::set< std::set > GetAddressGroupings(); + std::map GetAddressBalances(); + + std::set GetAccountAddresses(std::string strAccount) const; + + isminetype IsMine(const CTxIn& txin) const; + CAmount GetDebit(const CTxIn& txin, const isminefilter& filter) const; + bool IsFromMe(const CTxIn& txin, const isminefilter& filter) const; + isminetype IsMine(const CTxOut& txout) const + { + return ::IsMine(*this, txout.scriptPubKey); + } + CAmount GetCredit(const CTxOut& txout, const isminefilter& filter) const + { + if (!MoneyRange(txout.nValue)) + throw std::runtime_error("CWallet::GetCredit() : value out of range"); + return ((IsMine(txout) & filter) ? txout.nValue : 0); + } + bool IsChange(const CTxOut& txout) const; + CAmount GetChange(const CTxOut& txout) const + { + if (!MoneyRange(txout.nValue)) + throw std::runtime_error("CWallet::GetChange() : value out of range"); + return (IsChange(txout) ? txout.nValue : 0); + } + bool IsMine(const CTransaction& tx) const + { + BOOST_FOREACH(const CTxOut& txout, tx.vout) + if (IsMine(txout)) + return true; + return false; + } + /** should probably be renamed to IsRelevantToMe */ + bool IsFromMe(const CTransaction& tx) const + { +// return (GetDebit(tx, ISMINE_ALL) > 0); +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + if(IsFromMe(txin, ISMINE_ALL)) + { + return true; + } + } + } + else + { + return (GetDebit(tx, ISMINE_ALL) > 0); + } + return false; +/* MCHN END */ + } + CAmount GetDebit(const CTransaction& tx, const isminefilter& filter) const + { + CAmount nDebit = 0; + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + nDebit += GetDebit(txin, filter); + if (!MoneyRange(nDebit)) + throw std::runtime_error("CWallet::GetDebit() : value out of range"); + } + return nDebit; + } + CAmount GetCredit(const CTransaction& tx, const isminefilter& filter) const + { + CAmount nCredit = 0; + BOOST_FOREACH(const CTxOut& txout, tx.vout) + { + nCredit += GetCredit(txout, filter); + if (!MoneyRange(nCredit)) + throw std::runtime_error("CWallet::GetCredit() : value out of range"); + } + return nCredit; + } + CAmount GetChange(const CTransaction& tx) const + { + CAmount nChange = 0; + BOOST_FOREACH(const CTxOut& txout, tx.vout) + { + nChange += GetChange(txout); + if (!MoneyRange(nChange)) + throw std::runtime_error("CWallet::GetChange() : value out of range"); + } + return nChange; + } + void SetBestChain(const CBlockLocator& loc); + + DBErrors LoadWallet(bool& fFirstRunRet); + DBErrors ZapWalletTx(std::vector& vWtx); + + bool SetAddressBook(const CTxDestination& address, const std::string& strName, const std::string& purpose); + + bool DelAddressBook(const CTxDestination& address); + + void UpdatedTransaction(const uint256 &hashTx); + + void Inventory(const uint256 &hash) + { + { + LOCK(cs_wallet); + std::map::iterator mi = mapRequestCount.find(hash); + if (mi != mapRequestCount.end()) + (*mi).second++; + } + } + + unsigned int GetKeyPoolSize() + { + AssertLockHeld(cs_wallet); // setKeyPool + return setKeyPool.size(); + } + + bool SetDefaultKey(const CPubKey &vchPubKey); + + //! signify that a particular wallet feature is now used. this may change nWalletVersion and nWalletMaxVersion if those are lower + bool SetMinVersion(enum WalletFeature, CWalletDB* pwalletdbIn = NULL, bool fExplicit = false); + + //! change which version we're allowed to upgrade to (note that this does not immediately imply upgrading to that format) + bool SetMaxVersion(int nVersion); + + //! get the current wallet format (the oldest client version guaranteed to understand this wallet) + int GetVersion() { LOCK(cs_wallet); return nWalletVersion; } + + //! Get wallet transactions that conflict with given transaction (spend same outputs) + std::set GetConflicts(const uint256& txid) const; + + /** + * Address book entry changed. + * @note called with lock cs_wallet held. + */ + boost::signals2::signal NotifyAddressBookChanged; + + /** + * Wallet transaction added, removed or updated. + * @note called with lock cs_wallet held. + */ + boost::signals2::signal NotifyTransactionChanged; + + /** Show progress e.g. for rescan */ + boost::signals2::signal ShowProgress; + + /** Watch-only address added */ + boost::signals2::signal NotifyWatchonlyChanged; +}; + +/** A key allocated from the key pool. */ +class CReserveKey +{ +protected: + CWallet* pwallet; + int64_t nIndex; + CPubKey vchPubKey; +public: + CReserveKey(CWallet* pwalletIn) + { + nIndex = -1; + pwallet = pwalletIn; + } + + ~CReserveKey() + { + ReturnKey(); + } + + void ReturnKey(); + bool GetReservedKey(CPubKey &pubkey); + void KeepKey(); +}; + + +typedef std::map mapValue_t; + + +static void ReadOrderPos(int64_t& nOrderPos, mapValue_t& mapValue) +{ + if (!mapValue.count("n")) + { + nOrderPos = -1; // TODO: calculate elsewhere + return; + } + nOrderPos = atoi64(mapValue["n"].c_str()); +} + + +static void WriteOrderPos(const int64_t& nOrderPos, mapValue_t& mapValue) +{ + if (nOrderPos == -1) + return; + mapValue["n"] = i64tostr(nOrderPos); +} + +struct COutputEntry +{ + CTxDestination destination; + CAmount amount; + int vout; +}; + +/** A transaction with a merkle branch linking it to the block chain. */ +class CMerkleTx : public CTransaction +{ +private: + int GetDepthInMainChainINTERNAL(const CBlockIndex* &pindexRet) const; + +public: + uint256 hashBlock; + std::vector vMerkleBranch; + int nIndex; + + // memory only + mutable bool fMerkleVerified; + + + CMerkleTx() + { + Init(); + } + + CMerkleTx(const CTransaction& txIn) : CTransaction(txIn) + { + Init(); + } + + void Init() + { + hashBlock = 0; + nIndex = -1; + fMerkleVerified = false; + } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(*(CTransaction*)this); + nVersion = this->nVersion; + READWRITE(hashBlock); + READWRITE(vMerkleBranch); + READWRITE(nIndex); + } + + int SetMerkleBranch(const CBlock& block); + + + /** + * Return depth of transaction in blockchain: + * -1 : not in blockchain, and not in memory pool (conflicted transaction) + * 0 : in memory pool, waiting to be included in a block + * >=1 : this many blocks deep in the main chain + */ + int GetDepthInMainChain(const CBlockIndex* &pindexRet) const; + int GetDepthInMainChain() const { const CBlockIndex *pindexRet; return GetDepthInMainChain(pindexRet); } + bool IsInMainChain() const { const CBlockIndex *pindexRet; return GetDepthInMainChainINTERNAL(pindexRet) > 0; } + int GetBlocksToMaturity() const; + bool AcceptToMemoryPool(bool fLimitFree=true, bool fRejectInsaneFee=true); + bool AcceptToMemoryPoolReturnReason(bool fLimitFree, bool fRejectInsaneFee,std::string& reject_reason); + +}; + +/** + * A transaction with a bunch of additional info that only the owner cares about. + * It includes any unrecorded transactions needed to link it back to the block chain. + */ +class CWalletTx : public CMerkleTx +{ +private: + const CWallet* pwallet; + +public: + mapValue_t mapValue; + std::vector > vOrderForm; + unsigned int fTimeReceivedIsTxTime; + unsigned int nTimeReceived; //! time received by this node + unsigned int nTimeSmart; + char fFromMe; + std::string strFromAccount; + int64_t nOrderPos; //! position in ordered transaction list +/* MCHN START */ + mc_TxDefRow txDef; +/* MCHN END */ + + // memory only + mutable bool fDebitCached; + mutable bool fCreditCached; + mutable bool fImmatureCreditCached; + mutable bool fAvailableCreditCached; + mutable bool fWatchDebitCached; + mutable bool fWatchCreditCached; + mutable bool fImmatureWatchCreditCached; + mutable bool fAvailableWatchCreditCached; + mutable bool fChangeCached; + mutable CAmount nDebitCached; + mutable CAmount nCreditCached; + mutable CAmount nImmatureCreditCached; + mutable CAmount nAvailableCreditCached; + mutable CAmount nWatchDebitCached; + mutable CAmount nWatchCreditCached; + mutable CAmount nImmatureWatchCreditCached; + mutable CAmount nAvailableWatchCreditCached; + mutable CAmount nChangeCached; + + CWalletTx() + { + Init(NULL); + } + + CWalletTx(const CWallet* pwalletIn) + { + Init(pwalletIn); + } + + CWalletTx(const CWallet* pwalletIn, const CMerkleTx& txIn) : CMerkleTx(txIn) + { + Init(pwalletIn); + } + + CWalletTx(const CWallet* pwalletIn, const CTransaction& txIn) : CMerkleTx(txIn) + { + Init(pwalletIn); + } + + void Init(const CWallet* pwalletIn) + { + pwallet = pwalletIn; + mapValue.clear(); + vOrderForm.clear(); + fTimeReceivedIsTxTime = false; + nTimeReceived = 0; + nTimeSmart = 0; + fFromMe = false; + strFromAccount.clear(); + fDebitCached = false; + fCreditCached = false; + fImmatureCreditCached = false; + fAvailableCreditCached = false; + fWatchDebitCached = false; + fWatchCreditCached = false; + fImmatureWatchCreditCached = false; + fAvailableWatchCreditCached = false; + fChangeCached = false; + nDebitCached = 0; + nCreditCached = 0; + nImmatureCreditCached = 0; + nAvailableCreditCached = 0; + nWatchDebitCached = 0; + nWatchCreditCached = 0; + nAvailableWatchCreditCached = 0; + nImmatureWatchCreditCached = 0; + nChangeCached = 0; + nOrderPos = -1; + } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + if (ser_action.ForRead()) + Init(NULL); + char fSpent = false; + + if (!ser_action.ForRead()) + { + mapValue["fromaccount"] = strFromAccount; + + WriteOrderPos(nOrderPos, mapValue); + + if (nTimeSmart) + mapValue["timesmart"] = strprintf("%u", nTimeSmart); + } + + READWRITE(*(CMerkleTx*)this); + std::vector vUnused; //! Used to be vtxPrev + READWRITE(vUnused); + READWRITE(mapValue); + READWRITE(vOrderForm); + READWRITE(fTimeReceivedIsTxTime); + READWRITE(nTimeReceived); + READWRITE(fFromMe); + READWRITE(fSpent); + + if (ser_action.ForRead()) + { + strFromAccount = mapValue["fromaccount"]; + + ReadOrderPos(nOrderPos, mapValue); + + nTimeSmart = mapValue.count("timesmart") ? (unsigned int)atoi64(mapValue["timesmart"]) : 0; + } + + mapValue.erase("fromaccount"); + mapValue.erase("version"); + mapValue.erase("spent"); + mapValue.erase("n"); + mapValue.erase("timesmart"); + } + + //! make sure balances are recalculated + void MarkDirty() + { + fCreditCached = false; + fAvailableCreditCached = false; + fWatchDebitCached = false; + fWatchCreditCached = false; + fAvailableWatchCreditCached = false; + fImmatureWatchCreditCached = false; + fDebitCached = false; + fChangeCached = false; + } + + void BindWallet(CWallet *pwalletIn) + { + pwallet = pwalletIn; + MarkDirty(); + } + + //! filter decides which addresses will count towards the debit + CAmount GetDebit(const isminefilter& filter) const + { + if (vin.empty()) + return 0; + + CAmount debit = 0; + if(filter & ISMINE_SPENDABLE) + { + if (fDebitCached) + debit += nDebitCached; + else + { + nDebitCached = pwallet->GetDebit(*this, ISMINE_SPENDABLE); + fDebitCached = true; + debit += nDebitCached; + } + } + if(filter & ISMINE_WATCH_ONLY) + { + if(fWatchDebitCached) + debit += nWatchDebitCached; + else + { + nWatchDebitCached = pwallet->GetDebit(*this, ISMINE_WATCH_ONLY); + fWatchDebitCached = true; + debit += nWatchDebitCached; + } + } + return debit; + } + + CAmount GetCredit(const isminefilter& filter) const + { + // Must wait until coinbase is safely deep enough in the chain before valuing it + if (IsCoinBase() && GetBlocksToMaturity() > 0) + return 0; + + int64_t credit = 0; + if (filter & ISMINE_SPENDABLE) + { + // GetBalance can assume transactions in mapWallet won't change + if (fCreditCached) + credit += nCreditCached; + else + { + nCreditCached = pwallet->GetCredit(*this, ISMINE_SPENDABLE); + fCreditCached = true; + credit += nCreditCached; + } + } + if (filter & ISMINE_WATCH_ONLY) + { + if (fWatchCreditCached) + credit += nWatchCreditCached; + else + { + nWatchCreditCached = pwallet->GetCredit(*this, ISMINE_WATCH_ONLY); + fWatchCreditCached = true; + credit += nWatchCreditCached; + } + } + return credit; + } + + CAmount GetImmatureCredit(bool fUseCache=true) const + { + if (IsCoinBase() && GetBlocksToMaturity() > 0 && IsInMainChain()) + { + if (fUseCache && fImmatureCreditCached) + return nImmatureCreditCached; + nImmatureCreditCached = pwallet->GetCredit(*this, ISMINE_SPENDABLE); + fImmatureCreditCached = true; + return nImmatureCreditCached; + } + + return 0; + } + + CAmount GetAvailableCredit(bool fUseCache=true) const + { + if (pwallet == 0) + return 0; + + // Must wait until coinbase is safely deep enough in the chain before valuing it + if (IsCoinBase() && GetBlocksToMaturity() > 0) + return 0; + + if (fUseCache && fAvailableCreditCached) + return nAvailableCreditCached; + + CAmount nCredit = 0; + uint256 hashTx = GetHash(); + for (unsigned int i = 0; i < vout.size(); i++) + { + if (!pwallet->IsSpent(hashTx, i)) + { + const CTxOut &txout = vout[i]; + nCredit += pwallet->GetCredit(txout, ISMINE_SPENDABLE); + if (!MoneyRange(nCredit)) + throw std::runtime_error("CWalletTx::GetAvailableCredit() : value out of range"); + } + } + + nAvailableCreditCached = nCredit; + fAvailableCreditCached = true; + return nCredit; + } + + CAmount GetImmatureWatchOnlyCredit(const bool& fUseCache=true) const + { + if (IsCoinBase() && GetBlocksToMaturity() > 0 && IsInMainChain()) + { + if (fUseCache && fImmatureWatchCreditCached) + return nImmatureWatchCreditCached; + nImmatureWatchCreditCached = pwallet->GetCredit(*this, ISMINE_WATCH_ONLY); + fImmatureWatchCreditCached = true; + return nImmatureWatchCreditCached; + } + + return 0; + } + + CAmount GetAvailableWatchOnlyCredit(const bool& fUseCache=true) const + { + if (pwallet == 0) + return 0; + + // Must wait until coinbase is safely deep enough in the chain before valuing it + if (IsCoinBase() && GetBlocksToMaturity() > 0) + return 0; + + if (fUseCache && fAvailableWatchCreditCached) + return nAvailableWatchCreditCached; + + CAmount nCredit = 0; + for (unsigned int i = 0; i < vout.size(); i++) + { + if (!pwallet->IsSpent(GetHash(), i)) + { + const CTxOut &txout = vout[i]; + nCredit += pwallet->GetCredit(txout, ISMINE_WATCH_ONLY); + if (!MoneyRange(nCredit)) + throw std::runtime_error("CWalletTx::GetAvailableCredit() : value out of range"); + } + } + + nAvailableWatchCreditCached = nCredit; + fAvailableWatchCreditCached = true; + return nCredit; + } + + CAmount GetChange() const + { + if (fChangeCached) + return nChangeCached; + nChangeCached = pwallet->GetChange(*this); + fChangeCached = true; + return nChangeCached; + } + + void GetAmounts(std::list& listReceived, + std::list& listSent, CAmount& nFee, std::string& strSentAccount, const isminefilter& filter) const; + + void GetAccountAmounts(const std::string& strAccount, CAmount& nReceived, + CAmount& nSent, CAmount& nFee, const isminefilter& filter) const; + + bool IsFromMe(const isminefilter& filter) const + { +/* MCHN START */ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + BOOST_FOREACH(const CTxIn& txin, vin) + { + if(pwallet->IsFromMe(txin, filter)) + { + return true; + } + } + } + else + { + return (GetDebit(filter) > 0); + } + + return false; +// return (GetDebit(filter) >= 0); // MCHN > in original code +/* MCHN END */ + } + + bool IsTrusted(int nDepth) const; + + bool IsTrusted() const + { + // Quick answer in most cases + if (!IsFinalTx(*this)) + return false; + int nDepth = GetDepthInMainChain(); + if (nDepth >= 1) + return true; + if (nDepth < 0) + return false; + if (!bSpendZeroConfChange || !IsFromMe(ISMINE_ALL)) // using wtx's cached debit + return false; + + // Trusted if all inputs are from us and are in the mempool: + BOOST_FOREACH(const CTxIn& txin, vin) + { + // Transactions not sent by us: not trusted +/* MCHN START */ + if(pwallet->setPurged.count(txin.prevout.hash) == 0) + { +/* MCHN END */ + const CWalletTx* parent = pwallet->GetWalletTx(txin.prevout.hash); + if (parent == NULL) + return false; + const CTxOut& parentOut = parent->vout[txin.prevout.n]; + if (pwallet->IsMine(parentOut) != ISMINE_SPENDABLE) + return false; +/* MCHN START */ + } +/* MCHN END */ + } + return true; + } + + bool WriteToDisk(); + + int64_t GetTxTime() const; + int GetRequestCount() const; + + void RelayWalletTransaction(); + + std::set GetConflicts() const; +}; + + + + +class COutput +{ +public: + const CWalletTx *tx; + int i; + int nDepth; + bool fSpendable; + mc_Coin coin; + + COutput(const CWalletTx *txIn, int iIn, int nDepthIn, bool fSpendableIn) + { + tx = txIn; i = iIn; nDepth = nDepthIn; fSpendable = fSpendableIn; + } + + COutput(const CWalletTx *txIn, int iIn, int nDepthIn, bool fSpendableIn, const mc_Coin& coinIn) + { + tx = txIn; i = iIn; nDepth = nDepthIn; fSpendable = fSpendableIn; coin = coinIn; + } + + std::string ToString() const; + + uint256 GetHashAndTxOut(CTxOut& txout) const; + bool IsTrusted() const; +}; + + + + + +/** Private key that includes an expiration date in case it never gets used. */ +class CWalletKey +{ +public: + CPrivKey vchPrivKey; + int64_t nTimeCreated; + int64_t nTimeExpires; + std::string strComment; + //! todo: add something to note what created it (user, getnewaddress, change) + //! maybe should have a map property map + + CWalletKey(int64_t nExpires=0); + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + if (!(nType & SER_GETHASH)) + READWRITE(nVersion); + READWRITE(vchPrivKey); + READWRITE(nTimeCreated); + READWRITE(nTimeExpires); + READWRITE(LIMITED_STRING(strComment, 65536)); + } +}; + + + + + + +/** + * Account information. + * Stored in wallet with key "acc"+string account name. + */ +class CAccount +{ +public: + CPubKey vchPubKey; + + CAccount() + { + SetNull(); + } + + void SetNull() + { + vchPubKey = CPubKey(); + } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + if (!(nType & SER_GETHASH)) + READWRITE(nVersion); + READWRITE(vchPubKey); + } +}; + + + +/** + * Internal transfers. + * Database key is acentry. + */ +class CAccountingEntry +{ +public: + std::string strAccount; + CAmount nCreditDebit; + int64_t nTime; + std::string strOtherAccount; + std::string strComment; + mapValue_t mapValue; + int64_t nOrderPos; //! position in ordered transaction list + uint64_t nEntryNo; + + CAccountingEntry() + { + SetNull(); + } + + void SetNull() + { + nCreditDebit = 0; + nTime = 0; + strAccount.clear(); + strOtherAccount.clear(); + strComment.clear(); + nOrderPos = -1; + nEntryNo = 0; + } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + if (!(nType & SER_GETHASH)) + READWRITE(nVersion); + //! Note: strAccount is serialized as part of the key, not here. + READWRITE(nCreditDebit); + READWRITE(nTime); + READWRITE(LIMITED_STRING(strOtherAccount, 65536)); + + if (!ser_action.ForRead()) + { + WriteOrderPos(nOrderPos, mapValue); + + if (!(mapValue.empty() && _ssExtra.empty())) + { + CDataStream ss(nType, nVersion); + ss.insert(ss.begin(), '\0'); + ss << mapValue; + ss.insert(ss.end(), _ssExtra.begin(), _ssExtra.end()); + strComment.append(ss.str()); + } + } + + READWRITE(LIMITED_STRING(strComment, 65536)); + + size_t nSepPos = strComment.find("\0", 0, 1); + if (ser_action.ForRead()) + { + mapValue.clear(); + if (std::string::npos != nSepPos) + { + CDataStream ss(std::vector(strComment.begin() + nSepPos + 1, strComment.end()), nType, nVersion); + ss >> mapValue; + _ssExtra = std::vector(ss.begin(), ss.end()); + } + ReadOrderPos(nOrderPos, mapValue); + } + if (std::string::npos != nSepPos) + strComment.erase(nSepPos); + + mapValue.erase("n"); + } + +private: + std::vector _ssExtra; +}; + +#endif // BITCOIN_WALLET_H diff --git a/src/wallet/wallet_ismine.cpp b/src/wallet/wallet_ismine.cpp new file mode 100644 index 00000000..9978c4fe --- /dev/null +++ b/src/wallet/wallet_ismine.cpp @@ -0,0 +1,99 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "wallet_ismine.h" + +#include "keys/key.h" +#include "keystore.h" +#include "script/script.h" +#include "script/standard.h" + +#include + +using namespace std; + +typedef vector valtype; + +unsigned int HaveKeys(const vector& pubkeys, const CKeyStore& keystore) +{ + unsigned int nResult = 0; + BOOST_FOREACH(const valtype& pubkey, pubkeys) + { + CKeyID keyID = CPubKey(pubkey).GetID(); + if (keystore.HaveKey(keyID)) + ++nResult; + } + return nResult; +} + +isminetype IsMine(const CKeyStore &keystore, const CTxDestination& dest) +{ + CScript script = GetScriptForDestination(dest); + return IsMine(keystore, script); +} + +isminetype IsMine(const CKeyStore &keystore, const CScript& scriptPubKey) +{ + vector vSolutions; + txnouttype whichType; + if (!Solver(scriptPubKey, whichType, vSolutions)) { +/* MCHN START */ +// if (keystore.HaveWatchOnly(scriptPubKey)) + if (keystore.HaveWatchOnly(scriptPubKey.RemoveOpDrops())) +/* MCHN END */ + return ISMINE_WATCH_ONLY; + return ISMINE_NO; + } + + + CKeyID keyID; + switch (whichType) + { + case TX_NONSTANDARD: + case TX_NULL_DATA: + break; + case TX_PUBKEY: + keyID = CPubKey(vSolutions[0]).GetID(); + if (keystore.HaveKey(keyID)) + return ISMINE_SPENDABLE; + break; + case TX_PUBKEYHASH: + keyID = CKeyID(uint160(vSolutions[0])); + if (keystore.HaveKey(keyID)) + return ISMINE_SPENDABLE; + break; + case TX_SCRIPTHASH: + { + CScriptID scriptID = CScriptID(uint160(vSolutions[0])); + CScript subscript; + if (keystore.GetCScript(scriptID, subscript)) { + isminetype ret = IsMine(keystore, subscript); + if (ret == ISMINE_SPENDABLE) + return ret; + } + break; + } + case TX_MULTISIG: + { + // Only consider transactions "mine" if we own ALL the + // keys involved. multi-signature transactions that are + // partially owned (somebody else has a key that can spend + // them) enable spend-out-from-under-you attacks, especially + // in shared-wallet situations. + vector keys(vSolutions.begin()+1, vSolutions.begin()+vSolutions.size()-1); + if (HaveKeys(keys, keystore) == keys.size()) + return ISMINE_SPENDABLE; + break; + } + } + +/* MCHN START */ +// if (keystore.HaveWatchOnly(scriptPubKey)) + if (keystore.HaveWatchOnly(scriptPubKey.RemoveOpDrops())) +/* MCHN END */ + return ISMINE_WATCH_ONLY; + return ISMINE_NO; +} diff --git a/src/wallet/wallet_ismine.h b/src/wallet/wallet_ismine.h new file mode 100644 index 00000000..8fa49a6c --- /dev/null +++ b/src/wallet/wallet_ismine.h @@ -0,0 +1,30 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_WALLET_ISMINE_H +#define BITCOIN_WALLET_ISMINE_H + +#include "keys/key.h" +#include "script/standard.h" + +class CKeyStore; +class CScript; + +/** IsMine() return codes */ +enum isminetype +{ + ISMINE_NO = 0, + ISMINE_WATCH_ONLY = 1, + ISMINE_SPENDABLE = 2, + ISMINE_ALL = ISMINE_WATCH_ONLY | ISMINE_SPENDABLE +}; +/** used for bitflags of isminetype */ +typedef uint8_t isminefilter; + +isminetype IsMine(const CKeyStore& keystore, const CScript& scriptPubKey); +isminetype IsMine(const CKeyStore& keystore, const CTxDestination& dest); + +#endif // BITCOIN_WALLET_ISMINE_H diff --git a/src/wallet/walletcoins.cpp b/src/wallet/walletcoins.cpp new file mode 100644 index 00000000..25452b9e --- /dev/null +++ b/src/wallet/walletcoins.cpp @@ -0,0 +1,2764 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "wallet/wallet.h" +#include "wallet/wallettxs.h" +#include "utils/utilparse.h" +#include "coincontrol.h" +#include "script/sign.h" +#include "utils/utilmoneystr.h" + +extern mc_WalletTxs* pwalletTxsMain; + +using namespace std; + +bool debug_print=false; +bool csperf_debug_print=false; + +void CAssetGroupTree::Clear() +{ + nAssetsPerGroup=0; + nMaxAssetsPerGroup=0; + nOptimalGroupCount=0; + nMode=0; + lpAssets=NULL; + lpAssetGroups=NULL; + lpTmpGroupBuffer=NULL; +} + +void CAssetGroupTree::Destroy() +{ + if(lpAssets) + { + delete lpAssets; + } + if(lpAssetGroups) + { + delete lpAssetGroups; + } + if(lpTmpGroupBuffer) + { + mc_Delete(lpTmpGroupBuffer); + } + Clear(); +} + +void CAssetGroupTree::Dump() +{ + CAssetGroup *thisGroup; + int *aptr; + unsigned char *assetrefbin; + int i,j; + + if(debug_print)printf("Asset Grouping. Group Size: %d. Group Count: %d\n",nAssetsPerGroup,lpAssetGroups->GetCount()-1); + LogPrint("mchn","mchn: Asset Grouping. Group Size: %d. Group Count: %d\n",nAssetsPerGroup,lpAssetGroups->GetCount()-1); + for(i=1;iGetCount();i++) + { + thisGroup=(CAssetGroup*)lpAssetGroups->GetRow(i); + if(debug_print) + { + printf("Group: %4d. Asset Count: %d\n",i,thisGroup->nSize); + aptr=(int*)(lpAssetGroups->GetRow(i)+sizeof(CAssetGroup)); + for(j=0;jnSize;j++) + { + assetrefbin=(unsigned char*)lpAssets->GetRow(aptr[j]); + + string assetref=""; + assetref += itostr((int)mc_GetLE(assetrefbin,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(assetrefbin+4,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(assetrefbin+8,2)); + + printf(" Asset: %d: %s\n",j+1,assetref.c_str()); + } + } + } + +} + +/* + * Resizes asset group set to cointain given number of new assets + */ + +int CAssetGroupTree::Resize(int newAssets) +{ + int err; + int n,i; + int last_full; + CAssetGroup *thisGroup; + + n=nAssetsPerGroup; + + while(nOptimalGroupCount*n < lpAssets->GetCount()+newAssets) + { + n*=2; + } + + if(n == nAssetsPerGroup) // Current value is OK + { + return MC_ERR_NOERROR; + } + if(n > nMaxAssetsPerGroup) // We cannot increase assets-per-group, we have no choice but increase number of groups + { + return MC_ERR_NOERROR; + } + + if(debug_print)printf("Asset grouping resize %d -> %d\n",nAssetsPerGroup,n); + LogPrint("mchn","Asset grouping resize %d -> %d\n",nAssetsPerGroup,n); + + mc_Buffer *new_asset_groups_buffer; + + new_asset_groups_buffer=new mc_Buffer; + + err=new_asset_groups_buffer->Initialize(sizeof(CAssetGroup),sizeof(CAssetGroup)+n*sizeof(int),MC_BUF_MODE_DEFAULT); + if(err) + { + return err; + } + // Old assets stay stay in the same group, no change in lpAssets is needed + // But group size is increased, group are partially filled + + new_asset_groups_buffer->Realloc(lpAssetGroups->GetCount()); + new_asset_groups_buffer->SetCount(lpAssetGroups->GetCount()); + for(i=0;iGetCount();i++) + { + memset(new_asset_groups_buffer->GetRow(i),0,sizeof(CAssetGroup)+n*sizeof(int)); + memcpy(new_asset_groups_buffer->GetRow(i),lpAssetGroups->GetRow(i),sizeof(CAssetGroup)+nAssetsPerGroup*sizeof(int)); + } + + last_full=0; // Restore list of previously full groups + for(i=0;iGetCount();i++) + { + thisGroup=(CAssetGroup*)lpAssetGroups->GetRow(i); + if(thisGroup->nSize == nAssetsPerGroup) + { + thisGroup->nNextGroup=last_full; + last_full=thisGroup->nThisGroup; + } + } + + *((int*)(new_asset_groups_buffer->GetRow(0)+sizeof(CAssetGroup)) + nAssetsPerGroup)=last_full; + + nAssetsPerGroup=n; + + delete lpAssetGroups; + lpAssetGroups=new_asset_groups_buffer; + + return MC_ERR_NOERROR; +} + +int CAssetGroupTree::Initialize(int assetsPerGroup,int maxAssetsPerGroup,int optimalGroupCount,int mode) +{ + int err; + + Destroy(); + + if(assetsPerGroup == 0) + { + return MC_ERR_NOERROR; + } + + nAssetsPerGroup=assetsPerGroup; + nMaxAssetsPerGroup=maxAssetsPerGroup; + nOptimalGroupCount=optimalGroupCount; + nMode=mode; + + lpAssets=new mc_Buffer; + err=lpAssets->Initialize(MC_AST_ASSET_QUANTITY_OFFSET,MC_AST_ASSET_QUANTITY_OFFSET+sizeof(int),MC_BUF_MODE_MAP); + + if(err) + { + return err; + } + + // key - asset group header + // value - list of asset ids in the group + + lpAssetGroups=new mc_Buffer; + err=lpAssetGroups->Initialize(sizeof(CAssetGroup),sizeof(CAssetGroup)+assetsPerGroup*sizeof(int),MC_BUF_MODE_DEFAULT); + + if(err) + { + return err; + } + + // Group id is 1-based. + // 0-row is used for storing id of the group with given number of assets 0-(assetsPerGroup-1) + // All groups with identical number of assets are linked in the list using nNextGroup field + + err=lpAssetGroups->Realloc(1); + if(err) + { + return err; + } + + lpAssetGroups->SetCount(1); + memset(lpAssetGroups->GetRow(0),0,sizeof(CAssetGroup)+assetsPerGroup*sizeof(int)); + + lpTmpGroupBuffer=(int*)mc_New(maxAssetsPerGroup*sizeof(int)); + if(lpTmpGroupBuffer == NULL) + { + return MC_ERR_ALLOCATION; + } + + return MC_ERR_NOERROR; +} + +/* + * Finds asset group big enough to contain given number of new assets. + */ + +CAssetGroup *CAssetGroupTree::FindAndShiftBestGroup(int assets) +{ + if(assets > nAssetsPerGroup) // Too many assets - cannot find single group + { + return NULL; + } + + int max_underfilled_size,i; + int *iptr; + + iptr=(int*)(lpAssetGroups->GetRow(0)+sizeof(CAssetGroup)); + + max_underfilled_size=-1; // Find group with maximal size, having free space for required asset number + i=nAssetsPerGroup-assets; + while( (i >= 0) && (max_underfilled_size == -1)) + { + if(iptr[i]) + { + max_underfilled_size=i; + } + i--; + } + + CAssetGroup *thisGroup; + + if(max_underfilled_size < 0) // Not found - add new group + { + CAssetGroup assetGroup; + assetGroup.nThisGroup=lpAssetGroups->GetCount(); + assetGroup.nNextGroup=0; + assetGroup.nSize=0; + memset(lpTmpGroupBuffer,0,nAssetsPerGroup*sizeof(int)); + iptr[0]=assetGroup.nThisGroup; // There is no group with 0 assets, this is the first + if(lpAssetGroups->Add(&assetGroup,lpTmpGroupBuffer)) + { + return NULL; + } + max_underfilled_size=0; + } + + thisGroup=(CAssetGroup*)lpAssetGroups->GetRow(iptr[max_underfilled_size]); + + iptr[max_underfilled_size]=thisGroup->nNextGroup; // Pop up found group from the list + + if(max_underfilled_size+assetsnNextGroup=iptr[max_underfilled_size+assets]; // Pushing it to new location (previous value is linked to the new group) + iptr[max_underfilled_size+assets]=thisGroup->nThisGroup; + } + else + { + thisGroup->nNextGroup=0; // The group is full + } + + + return thisGroup; +} + +int CAssetGroupTree::GroupCount() +{ + return lpAssetGroups->GetCount(); +} + +/* + * Returns group id of list of assets if all belong to the same group or error + * Adds new assets to the groups if addIfNeeded==1 + */ + +int CAssetGroupTree::GetGroup(mc_Buffer* assets, int addIfNeeded) +{ + if(nAssetsPerGroup == 0) // No grouping + { + return -1; + } + + int group_id,last_asset_count,new_asset_count; + int i,g; + int *iptr; + int *aptr; + unsigned char *assetRef; + CAssetGroup *thisGroup; + + group_id=-2; // No assets in this buffer + last_asset_count=lpAssets->GetCount(); + for(i=0;iGetCount();i++) + { + assetRef=assets->GetRow(i); +// if(mc_GetLE(assetRef+MC_AST_ASSET_REF_TYPE_OFFSET,MC_AST_ASSET_REF_TYPE_SIZE) != MC_AST_ASSET_REF_TYPE_SPECIAL)// Special rows are ignored - there is no real assets with asset-ref block=0 + if( (mc_GetABRefType(assetRef) != MC_AST_ASSET_REF_TYPE_SPECIAL) && + (mc_GetABRefType(assetRef) != MC_AST_ASSET_REF_TYPE_GENESIS) ) +// if(mc_GetLE(assetRef,4) > 0) + { + g=GetGroup(assetRef,0); + if(g>0) + { + if(group_id == -2) + { + if(g != group_id) + { + group_id = -3; // Assets belongs to different groups + } + } + else + { + group_id=g; + } + } + else + { + if(lpAssets->Add(assetRef,&g)) + { + lpAssets->SetCount(last_asset_count); + return -1; // Internal error + } + } + } + } + + if(last_asset_count == lpAssets->GetCount()) + { + return group_id; // No new assets + } + + // We have new assets + new_asset_count=lpAssets->GetCount(); + + if(group_id == -2) + { + group_id=0; // Don't belong to any of the groups + } + + if(!addIfNeeded) + { + lpAssets->SetCount(last_asset_count); + return 0; + } + + thisGroup=NULL; + + Resize(new_asset_count - last_asset_count); + + if(group_id > 0) // There are old assets, so we know what should be group id + { + thisGroup=(CAssetGroup*)lpAssetGroups->GetRow(group_id); + if(thisGroup->nSize + (new_asset_count - last_asset_count) <= nAssetsPerGroup)// There is large enough space in the group + { + iptr=(int*)(lpAssetGroups->GetRow(0)+sizeof(CAssetGroup)); + iptr[thisGroup->nSize]=thisGroup->nNextGroup; // Pop found group from the list + if(thisGroup->nSize+(new_asset_count - last_asset_count)nNextGroup=iptr[thisGroup->nSize+(new_asset_count - last_asset_count)];// Connection other groups with this size + iptr[thisGroup->nSize+(new_asset_count - last_asset_count)]=thisGroup->nThisGroup;// Push found group into new list + } + else + { + thisGroup->nNextGroup=0; + } + } + else + { + thisGroup=NULL; // Not enough space - we have to put new assets in different groups one by one + } + } + else + { + thisGroup=FindAndShiftBestGroup(new_asset_count - last_asset_count); // Try to put assets in the same group + } + + + if(thisGroup) // We have group with enough free space + // Group is already in the right place in the tree, but size is not updated yet + { + for(i=last_asset_count;iGetRow(i)+MC_AST_ASSET_QUANTITY_OFFSET)=thisGroup->nThisGroup; + } + aptr=(int*)(lpAssetGroups->GetRow(thisGroup->nThisGroup)+sizeof(CAssetGroup)); + for(i=last_asset_count;inSize]=i; + thisGroup->nSize++; + *(int*)(lpAssets->GetRow(i)+MC_AST_ASSET_QUANTITY_OFFSET)=thisGroup->nThisGroup; + } + if(group_id < 0) // Though we were able to put new assets in new group, if old assets belong to different group + // we should return -3 + { + return group_id; + } + return thisGroup->nThisGroup; + } + + lpAssets->SetCount(last_asset_count); // Rewinding assets list + for(i=last_asset_count;iGetRow(i),1); // lpAsset buffer is untouched in rewind + if(g<0) + { + return g; // Return in case of error + } + } + + return 0; +} + +/* + * Returns group id of the asset + * Adds to the groups if addIfNeeded==1 + */ + +int CAssetGroupTree::GetGroup(unsigned char *assetRef,int addIfNeeded) +{ + if(nAssetsPerGroup == 0) + { + return -1; + } + + int row,group_id,asset_id; + int *aptr; + + row=lpAssets->Seek(assetRef); + + if(row >= 0) // Found + { + return *(int*)(lpAssets->GetRow(row)+MC_AST_ASSET_QUANTITY_OFFSET); + } + + if(!addIfNeeded) + { + return 0; + } + + Resize(1); // New asset + + CAssetGroup *thisGroup; + + asset_id=lpAssets->GetCount(); + group_id=0; + if(lpAssets->Add(assetRef,&group_id)) + { + return -1; + } + + thisGroup=FindAndShiftBestGroup(1); // Find a group with at least one free space + if(thisGroup == NULL) + { + return -1; + } + + *(int*)(lpAssets->GetRow(asset_id)+MC_AST_ASSET_QUANTITY_OFFSET)=thisGroup->nThisGroup; + aptr=(int*)(lpAssetGroups->GetRow(thisGroup->nThisGroup)+sizeof(CAssetGroup)); + aptr[thisGroup->nSize]=asset_id; + thisGroup->nSize++; + + return group_id; +} + +int64_t mc_GetABCoinQuantity(void *ptr,int coin_id) +{ + return (int64_t)mc_GetLE((unsigned char*)ptr+MC_AST_ASSET_QUANTITY_OFFSET+coin_id*MC_AST_ASSET_QUANTITY_SIZE,MC_AST_ASSET_QUANTITY_SIZE); +} + +void mc_SetABCoinQuantity(void *ptr,int coin_id,int64_t quantity) +{ + mc_PutLE((unsigned char*)ptr+MC_AST_ASSET_QUANTITY_OFFSET+coin_id*MC_AST_ASSET_QUANTITY_SIZE,&quantity,MC_AST_ASSET_QUANTITY_SIZE); +} + + + +void DebugPrintAssetTxOut(uint256 hash,int index,unsigned char* assetrefbin,int64_t quantity) +{ + string txid=hash.GetHex(); + + + if(mc_gState->m_Features->ShortTxIDAsAssetRef()) + { + if(debug_print) + { + printf("TxOut: %s-%d ",txid.c_str(),index); + if(mc_GetABRefType(assetrefbin) == MC_AST_ASSET_REF_TYPE_SPECIAL) + { + printf("Special: %08x%08x",(uint32_t)mc_GetLE(assetrefbin,4),(uint32_t)mc_GetLE(assetrefbin+4,4)); + } + else + { + for(int i=MC_AST_SHORT_TXID_OFFSET+MC_AST_SHORT_TXID_SIZE-1;i>=MC_AST_SHORT_TXID_OFFSET;i--) + { + printf("%02x",assetrefbin[i]); + } + } + printf(" %ld\n",quantity); + } + } + else + { + string assetref=""; + assetref += itostr((int)mc_GetLE(assetrefbin,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(assetrefbin+4,4)); + assetref += "-"; + assetref += itostr((int)mc_GetLE(assetrefbin+8,2)); + + if(debug_print)printf("TxOut: %s-%d %s %ld\n",txid.c_str(),index,assetref.c_str(),quantity); + } +} + + + +struct CompareCOutputByDepthAndScriptSizeDesc +{ + bool operator()(const COutput& t1, + const COutput& t2) const + { + if(t1.tx) + { + if(t1.tx->vout[t1.i].scriptPubKey.size() == t2.tx->vout[t2.i].scriptPubKey.size()) + { + return t1.nDepth > t2.nDepth; + } + return t1.tx->vout[t1.i].scriptPubKey.size() > t2.tx->vout[t2.i].scriptPubKey.size(); + } + if(t1.coin.m_TXOut.scriptPubKey.size() == t2.coin.m_TXOut.scriptPubKey.size()) + { + return t1.nDepth > t2.nDepth; + } + return t1.coin.m_TXOut.scriptPubKey.size() > t2.coin.m_TXOut.scriptPubKey.size(); +// return t1.nDepth > t2.nDepth; + } +}; + + +std::string COutput::ToString() const +{ + return strprintf("COutput(%s, %d, %d) [%s]", tx->GetHash().ToString(), i, nDepth, FormatMoney(tx->vout[i].nValue)); +} + + +std::string CExchangeStatus::ToString() const +{ + return strprintf("CExchangeStatus(lock: %d, time: %d, hash: %s", nLockStatus, nTimestamp, nOfferHash.GetHex().c_str()); +} + + + +/* + * Return unspent UTXOs belonging to the specified address, or all addresses of addresses==NULL + */ + +void AvalableCoinsForAddress(CWallet *lpWallet,vector& vCoins, const CCoinControl* coinControl,const set* addresses,uint32_t flags) +{ + double start_time=mc_TimeNowAsDouble(); + double last_time,this_time; + last_time=start_time; + uint160 addr=0; + + if(addresses) + { + if(addresses->size() == 1) + { + set::const_iterator it = addresses->begin(); + CTxDestination addressRet=*it; + const CKeyID *lpKeyID=boost::get (&addressRet); + const CScriptID *lpScriptID=boost::get (&addressRet); + if(lpKeyID) + { + addr=*(uint160*)lpKeyID; + } + if(lpScriptID) + { + addr=*(uint160*)lpScriptID; + } + } + } + + lpWallet->AvailableCoins(vCoins, true, coinControl,true,true,addr,flags); + this_time=mc_TimeNowAsDouble(); + LogPrint("mcperf","mcperf: AvailableCoins: Time: %8.6f \n", this_time-last_time); + last_time=this_time; + + if(addresses == NULL) + { + return; + } + + if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // We already processed from-address + { + if(addresses->size() == 1) + { + return; + } + } + + vector vFilteredCoins; + + BOOST_FOREACH(const COutput& out, vCoins) + { + CTxOut txout; + out.GetHashAndTxOut(txout); + const CScript& script1 = txout.scriptPubKey; + CTxDestination addressRet; + if(ExtractDestinationScriptValid(script1, addressRet)) + { + const CKeyID *KeyID=boost::get (&addressRet); + if(KeyID) + { + if(addresses) + { + if(addresses->count(addressRet)) + { + vFilteredCoins.push_back(out); + } + } + else + { + vFilteredCoins.push_back(out); + } + } + } + } + + vCoins=vFilteredCoins; + this_time=mc_TimeNowAsDouble(); + LogPrint("mcperf","mcperf: Address Filtering: Time: %8.6f \n", this_time-last_time); + last_time=this_time; +} + +/* + * Updates input asset matrix + */ + +bool InsertCoinIntoMatrix(int coin_id, // IN coin id in the vCoins array + uint256 hash,int out_i, // IN txid/vout of the UTXO + mc_Buffer *tmp_amounts, // IN Asset values of the UTXO + mc_Buffer *out_amounts, // IN Only assets found in this buffer will be added, if NULL - all assets + mc_Buffer *in_amounts, // OUT Buffer to fill, rows -assets, columns - coins + mc_Buffer *in_map, // OUT txid/vout->coin id map (used in coin selection) + unsigned char *in_row,int in_size, // TMP temporary buffer row and its size + int *in_special_row, // IN Coordinates of special rows + int64_t pure_native) // IN Pure native currency flag +{ + unsigned char buf_map[32+4+4]; + int row,err; + int64_t quantity; + + + memcpy(buf_map,&hash,32); // Updating txid/vout->coin id map + mc_PutLE(buf_map+32,&out_i,4); + mc_PutLE(buf_map+36,&coin_id,4); + in_map->Add(buf_map,buf_map+36); + + for(int i=0;iGetCount();i++) // Inserting asset amounts into the matrix + { + quantity=mc_GetABQuantity(tmp_amounts->GetRow(i)); + if( (out_amounts == NULL) || (out_amounts->Seek(tmp_amounts->GetRow(i)) >= 0) )// Only assets found in out_amounts if specified + { + row=in_amounts->Seek(tmp_amounts->GetRow(i)); + if(row < 0) // New asset + { + memset(in_row,0,in_size); + memcpy(in_row,tmp_amounts->GetRow(i),MC_AST_ASSET_QUANTITY_OFFSET); + mc_SetABCoinQuantity(in_row,coin_id,quantity); + err=in_amounts->Add(in_row); + if(err) + { + return false; + } + } + else // Old asset. We didn't insert this coin yet, so current value is 0 + { + mc_SetABCoinQuantity(in_amounts->GetRow(row),coin_id,quantity); + } + } + } + + quantity=0; // Setting "selected" flag to 0 + mc_SetABCoinQuantity(in_amounts->GetRow(in_special_row[0]),coin_id,quantity); + quantity=1; // Setting "parsed" flag to 1 + mc_SetABCoinQuantity(in_amounts->GetRow(in_special_row[1]),coin_id,quantity); + + mc_SetABCoinQuantity(in_amounts->GetRow(in_special_row[4]),coin_id,pure_native); + + return true; +} + +/* + * Find relevant coins for asset transfer + */ + +bool FindRelevantCoins(CWallet *lpWallet, // IN Wallet (to access grouping object) + vector& vCoins, // IN unspent coins + mc_Buffer *out_amounts, // IN assets amounts to be found + int expected_required, // IN expected permissions + bool *no_send_coins, // OUT flag, there are relevant coins without send permission + mc_Buffer *in_amounts, // OUT Buffer to fill, rows - assets, columns - coins + mc_Buffer *in_map, // OUT txid/vout->coin id map (used in coin selection) + mc_Buffer *tmp_amounts, // TMP temporary asset-quantity buffer + unsigned char *in_row,int in_size, // TMP temporary buffer row and its size + mc_Script *lpScript, // TMP temporary multichain script object + int *in_special_row, // IN Coordinates of special rows + map* mapSpecialEntity, + std::string& strFailReason) // OUT error message +{ + int coin_id=0; + int group_id; + int64_t pure_native; + + if(debug_print)printf("debg: Inputs - normal\n"); + BOOST_FOREACH(const COutput& out, vCoins) + { + string strError; + int allowed=expected_required; + int required=expected_required; + int out_i; + bool is_relevant; + + CTxOut txout; + uint256 hash=out.GetHashAndTxOut(txout); + + out_i=out.i; + tmp_amounts->Clear(); + if(ParseMultichainTxOutToBuffer(hash,txout,tmp_amounts,lpScript,&allowed,&required,mapSpecialEntity,strError)) + { + // All coins are taken, possible future optimization +/* + is_relevant=false; + i=0; + while(!is_relevant && iGetCount()) + { +// if( (mc_GetLE(assetRef+MC_AST_ASSET_REF_TYPE_OFFSET,MC_AST_ASSET_REF_TYPE_SIZE) != MC_AST_ASSET_REF_TYPE_SPECIAL) + || (mc_GetLE(tmp_amounts->GetRow(i)+4,4) != MC_PTP_SEND) ) + { + if(out_amounts->Seek(tmp_amounts->GetRow(i)) >= 0) + { + is_relevant=true; + } + } + i++; + } + */ + group_id=lpWallet->lpAssetGroups->GetGroup(tmp_amounts,1); // Assign group to all assets, we don't care at this stage about coin group + if(group_id == -1) // Only about error + { + strFailReason=_("Internal error: Cannot put assets into groups"); + return false; + } + + pure_native=0; + if(group_id == -2) + { + pure_native=1; + } + + for(int i=0;iGetCount();i++) + { + DebugPrintAssetTxOut(hash,out_i,tmp_amounts->GetRow(i),mc_GetABQuantity(tmp_amounts->GetRow(i))); + } + + is_relevant=true; + if(required & MC_PTP_ISSUE) // This txout contains unconfirmed genesis, cannot be spent + { + is_relevant=false; + } + if(is_relevant) + { + if(allowed & MC_PTP_SEND) + { + if(!InsertCoinIntoMatrix(coin_id,hash,out_i,tmp_amounts,out_amounts,in_amounts,in_map,in_row,in_size,in_special_row,pure_native)) + { + strFailReason=_("Internal error: Cannot update input amount matrix"); + return false; + } + } + else + { + *no_send_coins=true; + } + } + } + tmp_amounts->Clear(); + coin_id++; + } + + return true; +} + +struct CompareByFirst +{ + bool operator()(const pair& t1, + const pair& t2) const + { + return (t1.first < t2.first); + } +}; + +struct CompareBySecond +{ + bool operator()(const pair& t1, + const pair& t2) const + { + return (t1.second < t2.second); + } +}; + + +/* + * Selecting coins fro auto-combine + */ + +bool FindCoinsToCombine(CWallet *lpWallet, // IN Wallet (to access grouping object) + vector& vCoins, // IN unspent coins + int min_conf,int min_inputs,int max_inputs, // IN Auto-combine parameters + mc_Buffer *in_amounts, // OUT Buffer to fill, rows - assets, columns - coins + mc_Buffer *in_map, // OUT txid/vout->coin id map (used in coin selection) + mc_Buffer *tmp_amounts, // TMP temporary asset-quantity buffer + unsigned char *in_row,int in_size, // TMP temporary buffer row and its size + mc_Script *lpScript, // TMP temporary multichain script object + int *in_special_row, // IN Coordinates of special rows + std::string& strFailReason) // OUT error message +{ + int coin_id=0; + int group_id,group_count,i,pure_native_group; + int count=0; + int full_count,this_count,pure_native_count; + + vector > active_groups; // Groups found in UTXOs + + group_count=lpWallet->lpAssetGroups->GroupCount(); + pure_native_count=0; + active_groups.resize(group_count); + for(i=0;i= min_conf) ) + { + CTxOut txout; + uint256 hash=out.GetHashAndTxOut(txout); + out_i=out.i; + tmp_amounts->Clear(); + if(ParseMultichainTxOutToBuffer(hash,txout,tmp_amounts,lpScript,&allowed,&required,strError)) + { + if( (required & MC_PTP_ISSUE) == 0 ) // Ignore txouts containing unconfirmed geneses + { + group_id=lpWallet->lpAssetGroups->GetGroup(tmp_amounts,1); // Find group id, insert new assets if needed + if(debug_print)printf("%s-%d: group %d\n",hash.ToString().c_str(),out.i,group_id); + if(group_id == -1) + { + strFailReason=_("Internal error: Cannot update asset grouping information"); + return false; + } + else + { + if( (group_id == 0) || // We couldn't add new asset into one group + (group_id == -3) ) // Assets in this group belongs to different groups + // Take it + { + for(int i=0;iGetCount();i++) + { + DebugPrintAssetTxOut(hash,out_i,tmp_amounts->GetRow(i),mc_GetABQuantity(tmp_amounts->GetRow(i))); + } + if(!InsertCoinIntoMatrix(coin_id,hash,out_i,tmp_amounts,NULL,in_amounts,in_map,in_row,in_size,in_special_row,0)) + { + strFailReason=_("Internal error: Cannot update input amount matrix"); + return false; + } + count++; + } + else + { + if(group_id == -2) // Pure native currency + { + group_id=0; + pure_native_count++; + } + while(group_id >= group_count) + { + active_groups.push_back(make_pair(group_count,0)); + group_count++; + } + if(group_id) + { + active_groups[group_id]=make_pair(group_id,active_groups[group_id].second+1); + } + } + } + } + } + } + + coin_id++; + } + + pure_native_group=0; + if(pure_native_count) + { + pure_native_group=group_count;//+1; + active_groups.push_back(make_pair(group_count,pure_native_count)); + } + + if(count < max_inputs) // The number of out-of-group coins is not enough + { + sort(active_groups.begin(), active_groups.end(), CompareBySecond()); // Groups with few coins first + full_count=0; + for(i=0;i<(int)active_groups.size();i++) + { + this_count=active_groups[i].second; + if(this_count<=1) // There is only one coin - nothing to combine + { + active_groups[i].second=0; + } + else + { + if(full_count >= max_inputs-1) // We already have enough coins + { + this_count=0; + } + else + { + if(full_count+this_count>max_inputs) // We need only part of this group + { + this_count=max_inputs-full_count; + } + } + active_groups[i].second=this_count; + } + } + sort(active_groups.begin(), active_groups.end(), CompareByFirst()); // Sort them back + } + + + if(debug_print)printf("debg: Inputs - combine, multiple coins\n"); + + coin_id=0; + + BOOST_FOREACH(const COutput& out, vCoins) + { + string strError; + int allowed=MC_PTP_SEND; + int required=MC_PTP_SEND; + int out_i; + + if( (count= min_conf) ) + { + CTxOut txout; + uint256 hash=out.GetHashAndTxOut(txout); + out_i=out.i; + tmp_amounts->Clear(); + if(ParseMultichainTxOutToBuffer(hash,txout,tmp_amounts,lpScript,&allowed,&required,strError)) + { + if( (required & MC_PTP_ISSUE) == 0 ) // Ignore txouts containing unconfirmed geneses + { + group_id=lpWallet->lpAssetGroups->GetGroup(tmp_amounts,0); // No adding this time, this will remove bad and newly created coins added in previous loop + if( (group_id > 0) || + (group_id == -2) ) // Pure native currency + { + if(group_id == -2) + { + group_id=pure_native_group; + } + if(active_groups[group_id].second > 0) + { + for(int i=0;iGetCount();i++) + { + DebugPrintAssetTxOut(hash,out_i,tmp_amounts->GetRow(i),mc_GetABQuantity(tmp_amounts->GetRow(i))); + } + if(!InsertCoinIntoMatrix(coin_id,hash,out_i,tmp_amounts,NULL,in_amounts,in_map,in_row,in_size,in_special_row,0)) + { + strFailReason=_("Internal error"); + return false; + } + count++; + active_groups[group_id].second-=1; + } + } + } + } + } + + coin_id++; + } + + if(countGetRow(in_special_row[0]),in_amounts->GetRow(in_special_row[1]),in_size); + + + return true; +} + +/* + * Fills change amount buffer "selected"-"output" + */ + +bool CalculateChangeAmounts(CWallet *lpWallet, // IN Wallet (to access grouping object) + vector& vCoins, // IN unspent coins + CAmount TotalOutAmount, // IN native currency amount used for change + mc_Buffer *out_amounts, // IN assets amounts to be found + int expected_required, // IN expected permissions + mc_Buffer *in_amounts, // IN selected asset amounts + mc_Buffer *change_amounts, // OUT change amounts, asset-quantity buffer with additional field - group + mc_Buffer *tmp_amounts, // TMP temporary asset-quantity buffer + mc_Script *lpScript, // TMP temporary multichain script object + int *in_special_row, // IN Coordinates of special rows + map* mapSpecialEntity, // IN Special permission entry, like issue txid of follow-on issuance + set* usedAddresses, // OUT List of addresses used in selection. + std::string& strFailReason) // OUT error message +{ + int coin_id=0; + unsigned char buf[MC_AST_ASSET_FULLREF_BUF_SIZE+sizeof(int)]; + int64_t quantity; + string strError; + int allowed=expected_required; + int required=expected_required; + int out_i,row,group_id; + int err; + + if(debug_print)printf("debg: Selected:\n"); + + usedAddresses->clear(); + change_amounts->Clear(); + BOOST_FOREACH(const COutput& out, vCoins) // Inputs + { + // Coin is taken + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_special_row[1]),coin_id)) + { + // Coin is selected + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_special_row[0]),coin_id)) + { + CTxOut txout; + uint256 hash=out.GetHashAndTxOut(txout); + out_i=out.i; + tmp_amounts->Clear(); + if(ParseMultichainTxOutToBuffer(hash,txout,tmp_amounts,lpScript,&allowed,&required,mapSpecialEntity,strError)) + { + for(int i=0;iGetCount();i++) + { + DebugPrintAssetTxOut(hash,out.i,tmp_amounts->GetRow(i),mc_GetABQuantity(tmp_amounts->GetRow(i))); + LogAssetTxOut("Input : ",hash,out_i,tmp_amounts->GetRow(i),mc_GetABQuantity(tmp_amounts->GetRow(i))); + } + for(int i=0;iGetCount();i++) // Add all assets onto change buffer, not only those from out_amounts + { + quantity=mc_GetABQuantity(tmp_amounts->GetRow(i)); + if( (mc_GetABRefType(tmp_amounts->GetRow(i)) != MC_AST_ASSET_REF_TYPE_SPECIAL) && + (mc_GetABRefType(tmp_amounts->GetRow(i)) != MC_AST_ASSET_REF_TYPE_GENESIS) ) + { // All assets should be already populated into groups as coin is parsed + group_id=lpWallet->lpAssetGroups->GetGroup(tmp_amounts->GetRow(i),0); + } + else + { + group_id=0; // Native or special asset - no group + } + row=change_amounts->Seek(tmp_amounts->GetRow(i)); + if(row < 0) // New asset + { + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE+sizeof(int)); + memcpy(buf,tmp_amounts->GetRow(i),MC_AST_ASSET_QUANTITY_OFFSET); + mc_SetABQuantity(buf,quantity); + mc_PutLE(buf+MC_AST_ASSET_FULLREF_BUF_SIZE,&group_id,sizeof(int)); + err=change_amounts->Add(buf); + if(err) + { + strFailReason=_("Internal error: Cannot update change asset matrix"); + return false; + } + } + else // Old asset + { + quantity+=mc_GetABQuantity(change_amounts->GetRow(row)); + mc_SetABQuantity(change_amounts->GetRow(row),quantity); + } + } + + const CScript& script1 = txout.scriptPubKey; + CTxDestination addressRet; + if(ExtractDestinationScriptValid(script1, addressRet)) + { + if(usedAddresses->count(addressRet) == 0) + { + usedAddresses->insert(addressRet); + } + } + } + else + { + strFailReason=strError; + return false; + } + } + } + coin_id++; + } + + for(int i=0;iGetCount();i++) // Outputs + { + if(mc_GetABRefType(out_amounts->GetRow(i)) != MC_AST_ASSET_REF_TYPE_GENESIS) + { + quantity=-mc_GetABQuantity(out_amounts->GetRow(i)); + if(quantity) + { + row=change_amounts->Seek(out_amounts->GetRow(i)); + if(row < 0) // New asset, we don't care about group, it is already bad - negative amount + { + memset(buf,0,MC_AST_ASSET_FULLREF_BUF_SIZE+sizeof(int)); + memcpy(buf,out_amounts->GetRow(i),MC_AST_ASSET_QUANTITY_OFFSET); + mc_SetABQuantity(buf,quantity); + err=change_amounts->Add(buf); + if(err) + { + return false; + } + } + else // Old asset + { + quantity+=mc_GetABQuantity(change_amounts->GetRow(row)); + mc_SetABQuantity(change_amounts->GetRow(row),quantity); + } + } + } + } + + if(debug_print)printf("debg: Change:\n"); + for(int i=0;iGetCount();i++) + { + DebugPrintAssetTxOut(0,0,change_amounts->GetRow(i),mc_GetABQuantity(change_amounts->GetRow(i))); + } + + return true; + +} + +/* + * Finds address for change + */ + + +bool FindChangeAddress(CWallet *lpWallet,CTxDestination& address,const set* addresses, CReserveKey& reservekey, const CCoinControl* coinControl,int expected_required,map* mapSpecialEntity) +{ + int required; + bool change_address_found=true; + if (coinControl && !boost::get(&coinControl->destChange)) + { + address = coinControl->destChange; + } + else + { + required=expected_required; + if(required & MC_PTP_SEND) + { + required-=MC_PTP_SEND; + } + required |= MC_PTP_RECEIVE; + + if(addresses) + { + if(addresses->size() == 1) + { + address=*addresses->begin(); + const unsigned char *aptr; + aptr=GetAddressIDPtr(address); + if(mc_gState->m_Permissions->CanReceive(NULL,aptr)) + { + return true; + } + else + { + address=CNoDestination(); + return false; + } + } + } + + CPubKey vchPubKey; + if(!lpWallet->GetKeyFromAddressBook(vchPubKey,MC_PTP_SEND | required,addresses,mapSpecialEntity)) + { + if(!lpWallet->GetKeyFromAddressBook(vchPubKey,MC_PTP_SEND | MC_PTP_RECEIVE,addresses)) + { + if(!lpWallet->GetKeyFromAddressBook(vchPubKey,required,addresses)) + { + if(!lpWallet->GetKeyFromAddressBook(vchPubKey,MC_PTP_RECEIVE,addresses)) + { + change_address_found=false; + } + } + } + } + if(change_address_found) + { + address=vchPubKey.GetID(); + } + else + { + address=CNoDestination(); + } + } + return change_address_found; +} + +/* + * Select coins we have to use + */ + +bool SelectCoinsToUse(const vector* lpCoinsToUse, // IN List of coins to be used in this transaction + mc_Buffer *in_map, // IN txid/vout->coin id map (used in coin selection) + mc_Buffer *in_amounts, // IN/OUT Input amounts buffer, output - selected row + int *in_special_row, // IN Coordinates of special rows + std::string& strFailReason) // OUT error message +{ + int64_t quantity; + int coin_id; + unsigned char buf_map[32+4+4]; + + if(lpCoinsToUse == NULL) + { + return true; + } + + BOOST_FOREACH(const COutPoint& coin, *lpCoinsToUse) + { + // Setting "selected" flag + uint256 hash=coin.hash; + int out_i; + out_i=coin.n; + + memcpy(buf_map,&hash,32); + mc_PutLE(buf_map+32,&out_i,4); + int row=in_map->Seek(buf_map); + + if(row<0) + { + strFailReason="Could not find pre-selected output"; + return false; + } + + coin_id=mc_GetLE(in_map->GetRow(row)+36,4); + quantity=1; + mc_SetABCoinQuantity(in_amounts->GetRow(in_special_row[0]),coin_id,quantity); + } + + return true; +} +/* + * Select coins for specific asset + */ + + +bool SelectAssetCoins(CWallet *lpWallet, // IN Wallet (to access grouping object) + CAmount nTotalOutValue, // IN Total output value to find + vector& vCoins, // IN unspent coins + mc_Buffer *in_map, // IN txid/vout->coin id map (used in coin selection) + mc_Buffer *in_amounts, // IN/OUT Input amounts buffer, output - selected row + int *in_special_row, // IN Coordinates of special rows + int in_asset_row, // IN Row asset we are looking coins for + const CCoinControl* coinControl) +{ + int64_t quantity; + CAmount nTotalInValue; + CAmount nTotalSelectedValue; + CAmount nTargetValue; + int coin_id; + unsigned char buf_map[32+4+4]; + bool found_selected=false; + + nTotalInValue=0; + nTotalSelectedValue=0; + if(in_asset_row>=0) + { + for(coin_id=0;coin_id<(int)vCoins.size();coin_id++) + { + // Coin is parsed + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_special_row[1]),coin_id)) + { + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_special_row[0]),coin_id)) + { + found_selected=true; + } + quantity=mc_GetABCoinQuantity(in_amounts->GetRow(in_asset_row),coin_id); + if(quantity) + { + nTotalInValue+=quantity; + // Coin is selected + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_special_row[0]),coin_id)) + { + nTotalSelectedValue+=quantity; + } + } + } + } + } + + nTargetValue=nTotalOutValue-nTotalSelectedValue; + if(debug_print)printf("debg: Row: %d. In: %ld; Out: %ld; Selected: %ld; Target: %ld \n",in_asset_row,nTotalInValue,nTotalOutValue,nTotalSelectedValue,nTargetValue); + + if(nTotalOutValue>nTotalInValue) // Insufficient funds + { + return false; + } + + + if(( nTargetValue > 0) || (!found_selected) ) + { + set > setAssetCoins; + CAmount nAssetValueIn = 0; + + int in_prefered_row=in_asset_row; + for(int i=0;i<7;i++) + { + if(in_asset_row == in_special_row[i]) + { + in_prefered_row=4; // Special asset - prefer pure native currency coins + } + } + // Selecting coins covering remaining asset amount + if(!lpWallet->SelectMultiChainCoins(nTargetValue,vCoins,in_map,in_amounts,in_special_row[0],in_asset_row,in_prefered_row,setAssetCoins, nAssetValueIn, coinControl)) + { + return false; + } + + BOOST_FOREACH(PAIRTYPE(uint256, unsigned int) pcoin, setAssetCoins) + { + // Setting "selected" flag + uint256 hash=pcoin.first; + int out_i; + out_i=pcoin.second; + + memcpy(buf_map,&hash,32); + mc_PutLE(buf_map+32,&out_i,4); + int row=in_map->Seek(buf_map); + coin_id=mc_GetLE(in_map->GetRow(row)+36,4); + quantity=1; + + mc_SetABCoinQuantity(in_amounts->GetRow(in_special_row[0]),coin_id,quantity); + } + } + + return true; +} + +/* + * Build signed transaction with given outputs, inputs and change amounts and fee + * Returns 0 on success + * Returns negative value on error + * Returns missing native currency value if positive + */ + + +CAmount BuildAssetTransaction(CWallet *lpWallet, // IN Wallet object + CWalletTx& wtxNew, // OUT Resulting transaction + CTxDestination& change_address, // IN Address for change + CAmount& nFeeRet, // IN/OUT Required fee + const vector >& vecSend, // IN Output amount/script array + vector& vCoins, // IN Unspent UTXOs + mc_Buffer *in_amounts, // IN Input amount matrix + mc_Buffer *change_amounts, // IN Change amount matrix + int required, // IN Required special permissions + CAmount min_output, // IN minimum native currency value (to be set in change outputs)) + mc_Buffer *tmp_amounts, // TMP temporary asset-quantity buffer + mc_Script *lpScript, // TMP temporary multichain script object + int *in_special_row, // IN Coordinates of special rows + set* usedAddresses, // In List of addresses used in selection. + uint32_t flags, // In Coin selection flags + std::string& strFailReason) // OUT error message +{ + set active_groups; + + int group_id; + int change_count; + int extra_change_count; + CAmount missing_amount; + int64_t quantity; + + CAmount nTotalInValue=0; + CAmount nTotalChangeValue=0; + CAmount default_change_output; + CAmount change_amount; + + for(int i=0;iGetCount();i++) // Finding relevant asset groups and calculating native currency total + { + quantity=mc_GetABQuantity(change_amounts->GetRow(i)); + if(quantity<0) // We didn't find inputs properly + { + strFailReason=_("Internal error: negative change amount"); + return -1; + } + if(quantity>0) + { + group_id=mc_GetLE(change_amounts->GetRow(i)+MC_AST_ASSET_FULLREF_BUF_SIZE,sizeof(int)); + if(mc_GetABRefType(change_amounts->GetRow(i)) != MC_AST_ASSET_REF_TYPE_SPECIAL) + //if(mc_GetLE(change_amounts->GetRow(i),4) != 0) + { + if(debug_print)printf("debg: Found asset belonging to group %d\n",group_id); + } + if(group_id) // We don't care about native and special assets + { + std::set::const_iterator it = active_groups.find(group_id); + if (it == active_groups.end()) + { + active_groups.insert(group_id); + } + } + if( (mc_GetABRefType(change_amounts->GetRow(i)) == MC_AST_ASSET_REF_TYPE_SPECIAL) && + (mc_GetLE(change_amounts->GetRow(i)+4,4) == MC_PTP_SEND) ) + { + nTotalInValue=quantity; + } + } + } + + change_count=0; + if(nTotalInValue > nFeeRet) // Pure native currency change is required + { + change_count=1; + } + change_count+=active_groups.size(); + + const unsigned char* change_aptr=GetAddressIDPtr(change_address); + + if(change_aptr == NULL) + { + if(change_count>0) // We have to send change, but we couldn't find change address with receive permission + { + strFailReason = _("Change address not found"); + return -2; + } + } + else + { + if(change_count == 0) + { + change_count=1; + } + } + + extra_change_count=0; + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + BOOST_FOREACH(const CTxDestination& address, *usedAddresses) // Sending empty change to all addresses used in inputs, except change_address + { + const unsigned char* aptr=GetAddressIDPtr(address); + if(aptr) + { + if((change_aptr == NULL) || memcmp(change_aptr,aptr,20)) + { + if(mc_gState->m_Permissions->CanReceive(NULL,aptr)) + { + extra_change_count++; + } + else + { + if((mc_gState->m_Features->AnyoneCanReceiveEmpty() != 0) && (min_output == 0)) + { + extra_change_count++; + } + } + } + } + } + } + + if(min_output >= 0) + { + default_change_output=min_output; + } + else + { + default_change_output=182; // 34 + 148 (see CTxOut.IsDust for explanation) + } + + missing_amount=nFeeRet+(change_count+extra_change_count)*default_change_output-nTotalInValue; + if(missing_amount > 0) // Inputs don't carry enough native currency, go out and select additional coins + { + LogPrint("mcatxo","mcatxo: Missing amount: %ld. Fee: %ld, Change: %ld, Inputs: %ld\n",missing_amount,nFeeRet,change_count*min_output,nTotalInValue); + if(debug_print)printf("Missing amount: %ld. Fee: %ld, Change: %ld, Inputs: %ld\n",missing_amount,nFeeRet,change_count*min_output,nTotalInValue); + return missing_amount; + } + + missing_amount = 0; + + while(missing_amount == 0) + { + CMutableTransaction txNew; + txNew.vin.clear(); + txNew.vout.clear(); + nTotalChangeValue=0; + + BOOST_FOREACH (const PAIRTYPE(CScript, CAmount)& s, vecSend) // Original outputs + { + CTxOut txout(s.second, s.first); + + txNew.vout.push_back(txout); + } + + int assets_per_opdrop=(mc_gState->m_NetworkParams->GetInt64Param("maxstdelementsize")-4)/(mc_gState->m_NetworkParams->m_AssetRefSize+MC_AST_ASSET_QUANTITY_SIZE); + + if(mc_gState->m_Features->VerifySizeOfOpDropElements()) + { + assets_per_opdrop=(mc_gState->m_NetworkParams->GetInt64Param("maxstdelementsize")-4)/(mc_gState->m_NetworkParams->m_AssetRefSize+MC_AST_ASSET_QUANTITY_SIZE); + } + + size_t elem_size; + const unsigned char *elem; + + BOOST_FOREACH (const int& g, active_groups) // Asset change outputs + { + tmp_amounts->Clear(); + CScript scriptChange=GetScriptForDestination(change_address); + for(int i=0;iGetCount();i++) + { + group_id=mc_GetLE(change_amounts->GetRow(i)+MC_AST_ASSET_FULLREF_BUF_SIZE,sizeof(int)); + if(group_id == g) + { + quantity=mc_GetABQuantity(change_amounts->GetRow(i)); + if(quantity>0) + { + tmp_amounts->Add(change_amounts->GetRow(i)); + } + } + // flush asset buffer into script] + if( (tmp_amounts->GetCount()>=assets_per_opdrop) || (i == change_amounts->GetCount()-1) ) + { + if(tmp_amounts->GetCount()) + { + if(debug_print)printf("debg: Change output, group %d:\n",g); + for(int i=0;iGetCount();i++) + { + DebugPrintAssetTxOut(0,0,tmp_amounts->GetRow(i),mc_GetABQuantity(tmp_amounts->GetRow(i))); + LogAssetTxOut("Change: ",0,0,tmp_amounts->GetRow(i),mc_GetABQuantity(tmp_amounts->GetRow(i))); + } + lpScript->Clear(); + lpScript->SetAssetQuantities(tmp_amounts,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER); + for(int element=0;element < lpScript->GetNumElements();element++) + { + elem = lpScript->GetData(element,&elem_size); + if(elem) + { + scriptChange << vector(elem, elem + elem_size) << OP_DROP; + } + else + { + strFailReason = _("Internal error: cannot create asset transfer script"); + return -1; + } + } + tmp_amounts->Clear(); + } + } + } + change_amount=min_output; + if(change_amount<0) + { + CTxOut txout_for_size(0, scriptChange); + change_amount=3*(::minRelayTxFee.GetFee(txout_for_size.GetSerializeSize(SER_DISK,0)+148u)); + } + CTxOut txout(change_amount, scriptChange); + nTotalChangeValue += change_amount; + txNew.vout.push_back(txout); + } + + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + BOOST_FOREACH(const CTxDestination& address, *usedAddresses) // Sending empty change to all addresses used in inputs, except change_address + { + const unsigned char* aptr=GetAddressIDPtr(address); + if(aptr) + { + if((change_aptr == NULL) || memcmp(change_aptr,aptr,20)) + { + if( (mc_gState->m_Permissions->CanReceive(NULL,aptr) != 0) || + ((mc_gState->m_Features->AnyoneCanReceiveEmpty() != 0) && (default_change_output == 0)) ) + { + CScript scriptChange=GetScriptForDestination(address); + CAmount nAmount=default_change_output; + CTxOut txout(nAmount, scriptChange); + nTotalChangeValue += nAmount; + if(debug_print)printf("debg: Extra change: %ld\n",nAmount); + txNew.vout.push_back(txout); + } + } + } + } + } + + if(change_count > (int)active_groups.size()) // Native currency change + { + CScript scriptChange=GetScriptForDestination(change_address); + CAmount nAmount=nTotalInValue-nTotalChangeValue-nFeeRet; + CTxOut txout(nAmount, scriptChange); + if(debug_print)printf("debg: Native change output: %ld\n",nAmount); + LogAssetTxOut("Change: ",0,0,NULL,nAmount); + txNew.vout.push_back(txout); + } + + double dPriority = 0; + bool fScriptCached=false; + + int coin_id=0; + BOOST_FOREACH(const COutput& out, vCoins) // Adding inputs + { + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_special_row[1]),coin_id)) + { + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_special_row[0]),coin_id)) + { + CAmount nCredit; + int age; + if(out.tx) + { + age = out.tx->GetDepthInMainChain(); + nCredit = out.tx->vout[out.i].nValue; + } + else + { + age = out.coin.GetDepthInMainChain(); + nCredit=out.coin.m_TXOut.nValue; + } + if (age != 0) + age += 1; + dPriority += (double)nCredit * age; + CTxOut txout; + uint256 hash=out.GetHashAndTxOut(txout); + txNew.vin.push_back(CTxIn(hash,out.i)); + if(required & MC_PTP_CACHED_SCRIPT_REQUIRED) + { + if(!fScriptCached) + { + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_special_row[6]),coin_id)) + { + int cs_offset; + CScript script3; + size_t elem_size; + const unsigned char *elem; + + if(out.tx) + { + script3 = out.tx->vout[out.i].scriptPubKey; + } + else + { + script3=out.coin.m_TXOut.scriptPubKey; + } + CScript::const_iterator pc3 = script3.begin(); + + lpScript->Clear(); + lpScript->SetCachedScript(0,&cs_offset,-1,NULL,-1); + lpScript->SetCachedScript(cs_offset,&cs_offset,(int)txNew.vin.size()-1,(unsigned char*)&pc3[0],script3.size()); + + CScript scriptCachedScript=CScript(); + elem = lpScript->GetData(0,&elem_size); + scriptCachedScript << vector(elem, elem + elem_size) << OP_DROP << OP_RETURN; + + CTxOut txout(0, scriptCachedScript); + if(debug_print)printf("debg: Cached Script for input %d\n",(int)txNew.vin.size()-1); + LogAssetTxOut("Cached Script: ",0,0,NULL,0); + txNew.vout.push_back(txout); + fScriptCached=true; + } + } + } + } + } + coin_id++; + } + + if(txNew.vin.size() == 0) + { + strFailReason = _("Internal error: no inputs"); + return -2; + } + + int nIn=0; + unsigned int nSignatureBytes=0; + coin_id=0; + BOOST_FOREACH(const COutput& out, vCoins) // Signing + { + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_special_row[1]),coin_id)) + { + if(mc_GetABCoinQuantity(in_amounts->GetRow(in_special_row[0]),coin_id)) + { + CTxOut txout; + out.GetHashAndTxOut(txout); + + if(flags & MC_CSF_SIGN) + { + if (!SignSignature(*lpWallet, txout.scriptPubKey, txNew, nIn++)) + { + CTransaction printableTx=CTransaction(txNew); + LogPrint("mchn","Cannot sign transaction input %d: (%s,%d), scriptPubKey %s \n", + nIn-1,txNew.vin[nIn-1].prevout.hash.ToString().c_str(),txNew.vin[nIn-1].prevout.n,txout.scriptPubKey.ToString().c_str()); + + strFailReason = _("Signing transaction failed"); + return -2; + } + } + else + { + if(txout.scriptPubKey.IsPayToScriptHash()) // We don't know what is the script generally speaking, using 2-of-3 + { + nSignatureBytes+=256; // 73*2+33*3+... + } + else + { + nSignatureBytes+=112; + } + } + } + } + coin_id++; + } + + wtxNew.fTimeReceivedIsTxTime = true; + wtxNew.fFromMe=true; + wtxNew.BindWallet(lpWallet); + *static_cast(&wtxNew) = CTransaction(txNew); + + // Limit size + unsigned int nBytes = nSignatureBytes + ::GetSerializeSize(*(CTransaction*)&wtxNew, SER_NETWORK, PROTOCOL_VERSION); + if (nBytes >= MAX_STANDARD_TX_SIZE) + { + strFailReason = _("Transaction too large"); + return -2; + } + dPriority = wtxNew.ComputePriority(dPriority, nBytes); + + // Can we complete this as a free transaction? + if (fSendFreeTransactions && nBytes <= MAX_FREE_TRANSACTION_CREATE_SIZE) + { + // Not enough fee: enough priority? + double dPriorityNeeded = mempool.estimatePriority(nTxConfirmTarget); + // Not enough mempool history to estimate: use hard-coded AllowFree. + if (dPriorityNeeded <= 0 && AllowFree(dPriority)) + break; + + // Small enough, and priority high enough, to send for free + if (dPriorityNeeded > 0 && dPriority >= dPriorityNeeded) + break; + } + + CAmount nFeeNeeded = lpWallet->GetMinimumFee(nBytes, nTxConfirmTarget, mempool); + + // If we made it here and we aren't even able to meet the relay fee on the next pass, give up + // because we must be at the maximum allowed fee. + if (nFeeNeeded < ::minRelayTxFee.GetFee(nBytes)) + { + strFailReason = _("Transaction too large for fee policy"); + return -2; + } + + if (nFeeRet >= nFeeNeeded) // Done + { + if(debug_print)printf("Transaction created. Fee: %ld, Change: %ld, Inputs: %ld\n",nFeeRet,nTotalChangeValue,nTotalInValue); + return 0; + } + + if(nFeeNeeded > nFeeRet) + { + nFeeRet = nFeeNeeded; + } + + missing_amount=nFeeRet+nTotalChangeValue-nTotalInValue; + if(missing_amount > 0) // Inputs don't carry enough native currency, go out and select additional coins + { + if(debug_print)printf("Missing amount: %ld. Fee: %ld, Change: %ld, Inputs: %ld\n",missing_amount,nFeeRet,nTotalChangeValue,nTotalInValue); + return missing_amount; + } + + if(debug_print)printf("Increased fee: Fee: %ld, Change: %ld, Inputs: %ld\n",nFeeRet,nTotalChangeValue,nTotalInValue); + + missing_amount=0; // We have enough native currency, we can try again by increasing the fee + } + + if(debug_print)printf("Transaction created (free). Fee: %ld, Change: %ld, Inputs: %ld\n",nFeeRet,nTotalChangeValue,nTotalInValue); + + return 0; +} + +bool CreateAssetGroupingTransaction(CWallet *lpWallet, const vector >& vecSend, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl* coinControl, + const set* addresses,int min_conf,int min_inputs,int max_inputs,const vector* lpCoinsToUse,uint32_t flags) +{ + double start_time=mc_TimeNowAsDouble(); + double last_time,this_time; + last_time=start_time; + + this_time=mc_TimeNowAsDouble(); + if(csperf_debug_print)if(vecSend.size())printf("Start \n"); + last_time=this_time; + + CAmount nValue = 0; + BOOST_FOREACH (const PAIRTYPE(CScript, CAmount)& s, vecSend) + { + if (nValue < 0) // Multichain allows protocol zero-value outputs + { + strFailReason = _("Transaction amounts must be non-negative"); + return false; + } + nValue += s.second; + } +// if (vecSend.empty() || nValue < 0) + if (nValue < 0) + { + strFailReason = _("Transaction amounts must be non-negative"); + return false; + } + + nFeeRet=0; + + CWallet::minTxFee = CFeeRate(MIN_RELAY_TX_FEE); + minRelayTxFee = CFeeRate(MIN_RELAY_TX_FEE); + + mc_Buffer *out_amounts; + + out_amounts=new mc_Buffer; + mc_InitABufferMap(out_amounts); + + mc_Buffer *change_amounts; + + change_amounts=new mc_Buffer; + change_amounts->Initialize(MC_AST_ASSET_QUANTITY_OFFSET,MC_AST_ASSET_FULLREF_BUF_SIZE+sizeof(int),MC_BUF_MODE_MAP); + + mc_Buffer *tmp_amounts; + + tmp_amounts=new mc_Buffer; + mc_InitABufferMap(tmp_amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; + + mc_Buffer *in_amounts; + in_amounts=NULL; + + mc_Buffer *in_map; + in_map=NULL; + + unsigned char *in_row; + int required; + uint256 hash; + int32_t type; + bool no_send_coins; + vector vCoins; + CTxDestination change_address; + CAmount min_output; + CAmount missing_amount; + CAmount nTotalOutValue; + CMutableTransaction txNew; + map mapSpecialEntity; + set usedAddresses; + bool skip_error_message=false; + + in_row=NULL; + out_amounts->Clear(); + + hash=0; + + required=0; + + if(vecSend.size()) + { + required=0; + + BOOST_FOREACH (const PAIRTYPE(CScript, CAmount)& s, vecSend) // Filling buffer with output amounts + { + CTxOut txout(s.second, s.first); + int this_required=MC_PTP_ALL; + if(!ParseMultichainTxOutToBuffer(hash,txout,out_amounts,lpScript,NULL,&this_required,&mapSpecialEntity,strFailReason)) + { + goto exitlbl; + } + required |= this_required; + } + } + else + { + required=MC_PTP_SEND; + CTxOut txout(0, CScript()); + if(!ParseMultichainTxOutToBuffer(hash,txout,out_amounts,lpScript,NULL,&required,strFailReason)) + { + goto exitlbl; + } + } + + if(debug_print)printf("debg: Outputs: (required %d)\n",required); + LogPrint("mcatxo","mcatxo: ====== New transaction, required %d\n",required); + for(int i=0;iGetCount();i++) + { + DebugPrintAssetTxOut(0,0,out_amounts->GetRow(i),mc_GetABQuantity(out_amounts->GetRow(i))); + LogAssetTxOut("Output: ",0,0,out_amounts->GetRow(i),mc_GetABQuantity(out_amounts->GetRow(i))); + } + + this_time=mc_TimeNowAsDouble(); + if(csperf_debug_print)if(vecSend.size())printf("Output : %8.6f\n",this_time-last_time); + last_time=this_time; + + // Getting available coin list + AvalableCoinsForAddress(lpWallet,vCoins,coinControl,addresses,flags); + + this_time=mc_TimeNowAsDouble(); + if(csperf_debug_print)if(vecSend.size())printf("Available : %8.6f\n",this_time-last_time); + LogPrint("mcperf","Coin Selection, available coins time: %8.6f\n",this_time-last_time); + last_time=this_time; + + int in_special_row[10]; + int in_size; + // Input coin matrix + // Rows - assets, columns - coins + in_size=MC_AST_ASSET_QUANTITY_OFFSET+vCoins.size()*MC_AST_ASSET_QUANTITY_SIZE; + in_amounts=new mc_Buffer; + in_amounts->Initialize(MC_AST_ASSET_QUANTITY_OFFSET,in_size,MC_BUF_MODE_MAP); + + this_time=mc_TimeNowAsDouble(); + if(csperf_debug_print)if(vecSend.size())printf("Alloc : %8.6f (%d)\n",this_time-last_time,in_size); + last_time=this_time; + // Map coin -> index in in_amounts + // We need this as in SelectMultiChainCoinsMinConf coins will be shuffled + // Key: 32-byte txid, 4-byte output id. Value - index in in_amounts + in_map=new mc_Buffer; + in_map->Initialize(36,40,MC_BUF_MODE_MAP); + + in_row=(unsigned char*)mc_New(in_size); + + + for(int i=0;i<10;i++) + { + in_special_row[i]=-1; + } + // Special row, if coin is already selected (or not used at all) the value is 1 + in_special_row[0]=in_amounts->GetCount(); + memset(in_row,-1,in_size); + memset(in_row,0,MC_AST_ASSET_QUANTITY_OFFSET); + type=0; + mc_PutLE(in_row+4,&type,4); + mc_SetABRefType(in_row,MC_AST_ASSET_REF_TYPE_SPECIAL); + in_amounts->Add(in_row); + + // Special row, if coin is taken for selection the value is 1 + in_special_row[1]=in_amounts->GetCount(); + memset(in_row,0,in_size); + type=0; + mc_PutLE(in_row+4,&type,4); + mc_SetABRefType(in_row,MC_AST_ASSET_REF_TYPE_SPECIAL); + in_amounts->Add(in_row); + + // Special row, saved selection + in_special_row[2]=in_amounts->GetCount(); + memset(in_row,0,in_size); + type=0; + mc_PutLE(in_row+4,&type,4); + mc_SetABRefType(in_row,MC_AST_ASSET_REF_TYPE_SPECIAL); + in_amounts->Add(in_row); + + // Native currency row, coin value if input has send permission, 0 otherwise + in_special_row[3]=in_amounts->GetCount(); + memset(in_row,0,in_size); + type=MC_PTP_SEND; + mc_PutLE(in_row+4,&type,4); + mc_SetABRefType(in_row,MC_AST_ASSET_REF_TYPE_SPECIAL); + in_amounts->Add(in_row); + + // Pure native currency flag, 1 if n o other assets were found in the coin + in_special_row[4]=in_amounts->GetCount(); + memset(in_row,0,in_size); + type=MC_PTP_SEND; + mc_SetABRefType(in_row,MC_AST_ASSET_REF_TYPE_SPECIAL); + mc_PutLE(in_row+4,&type,4); + in_amounts->Add(in_row); + + // Issue row, coin value if input has issue permission, 0 otherwise + if(required & MC_PTP_ISSUE) + { + in_special_row[5]=in_amounts->GetCount(); + memset(in_row,0,in_size); + type=MC_PTP_ISSUE | MC_PTP_SEND; + mc_PutLE(in_row+4,&type,4); + mc_SetABRefType(in_row,MC_AST_ASSET_REF_TYPE_SPECIAL); + in_amounts->Add(in_row); + } + + // Admin row, coin value if input has admin permission, 0 otherwise + if(required & MC_PTP_ADMIN) + { + in_special_row[6]=in_amounts->GetCount(); + memset(in_row,0,in_size); + type=MC_PTP_ADMIN | MC_PTP_SEND; + mc_PutLE(in_row+4,&type,4); + mc_SetABRefType(in_row,MC_AST_ASSET_REF_TYPE_SPECIAL); + in_amounts->Add(in_row); + } + + // Activate row, coin value if input has activate permission, 0 otherwise + if(required & MC_PTP_ACTIVATE) + { + in_special_row[7]=in_amounts->GetCount(); + memset(in_row,0,in_size); + type=MC_PTP_ACTIVATE | MC_PTP_SEND; + mc_PutLE(in_row+4,&type,4); + mc_SetABRefType(in_row,MC_AST_ASSET_REF_TYPE_SPECIAL); + in_amounts->Add(in_row); + } + + // Write row, coin value if input has write permission, 0 otherwise + if(required & MC_PTP_WRITE) + { + in_special_row[8]=in_amounts->GetCount(); + memset(in_row,0,in_size); + type=MC_PTP_WRITE | MC_PTP_SEND; + mc_PutLE(in_row+4,&type,4); + mc_SetABRefType(in_row,MC_AST_ASSET_REF_TYPE_SPECIAL); + in_amounts->Add(in_row); + } + + // Open row, coin value if input has open permission, 0 otherwise + if(required & MC_PTP_CREATE) + { + in_special_row[9]=in_amounts->GetCount(); + memset(in_row,0,in_size); + type=MC_PTP_CREATE | MC_PTP_SEND; + mc_PutLE(in_row+4,&type,4); + mc_SetABRefType(in_row,MC_AST_ASSET_REF_TYPE_SPECIAL); + in_amounts->Add(in_row); + } + + this_time=mc_TimeNowAsDouble(); + if(csperf_debug_print)if(vecSend.size())printf("Initialize : %8.6f\n",this_time-last_time); + last_time=this_time; + + { + LOCK2(cs_main, lpWallet->cs_wallet); + { + this_time=mc_TimeNowAsDouble(); + if(csperf_debug_print)if(vecSend.size())printf("Lock : %8.6f\n",this_time-last_time); + last_time=this_time; + + no_send_coins=false; + if(vecSend.size()) // Normal transaction + { + // Find coins for relevant assets + if(!FindRelevantCoins(lpWallet,vCoins,out_amounts,required,&no_send_coins,in_amounts,in_map,tmp_amounts,in_row,in_size,lpScript,in_special_row,&mapSpecialEntity,strFailReason)) + { + goto exitlbl; + } + + + if(!SelectCoinsToUse(lpCoinsToUse,in_map,in_amounts,in_special_row,strFailReason)) + { + goto exitlbl; + } + + this_time=mc_TimeNowAsDouble(); + if(csperf_debug_print)if(vecSend.size())printf("Inputs : %8.6f\n",this_time-last_time); + LogPrint("mcperf","mcperf: CS: Input Parsing: Time: %8.6f \n", this_time-last_time); + last_time=this_time; + + for(int asset=0;assetGetCount();asset++) + { + if(mc_GetABRefType(out_amounts->GetRow(asset)) != MC_AST_ASSET_REF_TYPE_GENESIS) + { + int in_asset_row; + nTotalOutValue=mc_GetABQuantity(out_amounts->GetRow(asset)); + in_asset_row=in_amounts->Seek(out_amounts->GetRow(asset)); + + if((nTotalOutValue > 0) || // Can send 0 amount of specific asset if there are no outputs carrying this asset + ((mc_GetABRefType(out_amounts->GetRow(asset)) == MC_AST_ASSET_REF_TYPE_SPECIAL) && + (mc_GetLE(out_amounts->GetRow(asset)+4,4) == MC_PTP_SEND)))// But not for native currency, otherwise we can get "no inputs" + { + if(!SelectAssetCoins(lpWallet,nTotalOutValue,vCoins,in_map,in_amounts,in_special_row,in_asset_row,coinControl)) + { + strFailReason = _("Insufficient funds"); + if(mc_GetABRefType(out_amounts->GetRow(asset)) == MC_AST_ASSET_REF_TYPE_SPECIAL) + { + switch(mc_GetLE(out_amounts->GetRow(asset)+4,4)) + { + case MC_PTP_ISSUE | MC_PTP_SEND: + strFailReason = _("No unspent output with issue permission"); + break; + case MC_PTP_CREATE | MC_PTP_SEND: + strFailReason = _("No unspent output with create permission"); + break; + case MC_PTP_ACTIVATE | MC_PTP_SEND: + strFailReason = _("No unspent output with activate or admin permission"); + break; + case MC_PTP_ADMIN | MC_PTP_SEND: + strFailReason = _("No unspent output with admin permission"); + break; + case MC_PTP_WRITE | MC_PTP_SEND: + strFailReason = _("No unspent output with write permission"); + break; + case MC_PTP_SEND: + if(nTotalOutValue > 0) + { + if(no_send_coins) + { + strFailReason = _("Insufficient funds, but there are coins belonging to addresses without send or receive permission."); + } + } + else + { + if(required & MC_PTP_WRITE) // publish always comes with addresses set, SEND fails before write + { + strFailReason = _("No unspent output with write permission"); + } + else + { + if(addresses->size() == 1) + { + strFailReason = _("No unspent output from this address"); + } + else + { + strFailReason = _("No unspent outputs found in this wallet"); + } + } + } + break; + } + } + else + { + if(no_send_coins) + { + strFailReason = _("Insufficient funds, but there are coins belonging to addresses without send or receive permission."); + } + } + + goto exitlbl; + } + } + } + } + + this_time=mc_TimeNowAsDouble(); + if(csperf_debug_print)if(vecSend.size())printf("Select : %8.6f\n",this_time-last_time); + LogPrint("mcperf","mcperf: CS: Coin Selection: Time: %8.6f \n", this_time-last_time); + last_time=this_time; + } + else + { + if(!FindCoinsToCombine(lpWallet,vCoins,min_conf,min_inputs,max_inputs,in_amounts,in_map,tmp_amounts,in_row,in_size,lpScript,in_special_row,strFailReason)) + { + // We didn't find enough coins to combine or error occurred + skip_error_message=true; + goto exitlbl; + } + } + + // Calculate change + if(!CalculateChangeAmounts(lpWallet,vCoins,nValue,out_amounts,required,in_amounts,change_amounts,tmp_amounts,lpScript,in_special_row,&mapSpecialEntity,&usedAddresses,strFailReason)) + { + goto exitlbl; + } + + + // Find change address + //mAy be we don't need the change so the error is ignored + FindChangeAddress(lpWallet,change_address,&usedAddresses,reservekey,coinControl,required,&mapSpecialEntity); + nFeeRet = 0; + + min_output=-1; // Calculate minimal output for the change + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + min_output=mc_gState->m_NetworkParams->GetInt64Param("minimumperoutput"); + } + // Storing selection before pure-native + memcpy(in_amounts->GetRow(in_special_row[2]),in_amounts->GetRow(in_special_row[0]),in_size); + nTotalOutValue=nValue; + + this_time=mc_TimeNowAsDouble(); + if(csperf_debug_print)if(vecSend.size())printf("Address : %8.6f\n",this_time-last_time); + last_time=this_time; + + missing_amount=BuildAssetTransaction(lpWallet,wtxNew,change_address,nFeeRet,vecSend,vCoins,in_amounts,change_amounts,required,min_output,tmp_amounts,lpScript,in_special_row,&usedAddresses,flags,strFailReason); + if(missing_amount<0) // Error + { + goto exitlbl; + } + + this_time=mc_TimeNowAsDouble(); + if(csperf_debug_print)if(vecSend.size())printf("Build : %8.6f\n",this_time-last_time); + LogPrint("mcperf","mcperf: Transaction Building: Time: %8.6f \n", this_time-last_time); + last_time=this_time; + + + while(missing_amount > 0) // We still need more native currency + { + if(vecSend.size() == 0) // We cannot select new coins as only combined outputs are parsed + { + strFailReason = _("Combine transaction requires extra native currency amount"); + goto exitlbl; + } + // Restoring pure-asset selection + memcpy(in_amounts->GetRow(in_special_row[0]),in_amounts->GetRow(in_special_row[2]),in_size); + nTotalOutValue+=missing_amount; + // Select coins to cover native currency output and fee + if(!SelectAssetCoins(lpWallet,nTotalOutValue,vCoins,in_map,in_amounts,in_special_row,in_special_row[3],coinControl)) + { + strFailReason = _("Insufficient funds"); + if(no_send_coins) + { + strFailReason = _("Insufficient funds, but there are coins belonging to addresses without send permission."); + } + goto exitlbl; + } + // Calculate new change matrix + if(!CalculateChangeAmounts(lpWallet,vCoins,nTotalOutValue,out_amounts,required,in_amounts,change_amounts,tmp_amounts,lpScript,in_special_row,&mapSpecialEntity,&usedAddresses,strFailReason)) + { + goto exitlbl; + } + + // Try to build transaction again + missing_amount=BuildAssetTransaction(lpWallet,wtxNew,change_address,nFeeRet,vecSend,vCoins,in_amounts,change_amounts,required,min_output,tmp_amounts,lpScript,in_special_row,&usedAddresses,flags,strFailReason); + if(missing_amount<0) // Error + { + goto exitlbl; + } + } + } + } + +exitlbl: + + if(in_map) + { + delete in_map; + } + + if(in_row) + { + mc_Delete(in_row); + } + + if(in_amounts) + { + delete in_amounts; + } + + delete lpScript; + delete tmp_amounts; + delete change_amounts; + delete out_amounts; + + + lpWallet->lpAssetGroups->Dump(); + + if(strFailReason.size()) + { + LogPrint("mcatxo","mcatxo: ====== Error: %s\n",strFailReason.c_str()); + if(!skip_error_message) + { + LogPrintf("mchn: Coin selection: %s\n",strFailReason.c_str()); + } + return false; + } + + this_time=mc_TimeNowAsDouble(); + if(csperf_debug_print)if(vecSend.size())printf("CS Exit : %8.6f\n",this_time-last_time); + last_time=this_time; + + LogPrint("mcatxo","mcatxo: ====== Created: %s\n",wtxNew.GetHash().ToString().c_str()); + return true; +} + +bool CWallet::CreateMultiChainTransaction(const vector >& vecSend, + CWalletTx& wtxNew, CReserveKey& reservekey, CAmount& nFeeRet, std::string& strFailReason, const CCoinControl* coinControl, + const set* addresses,int min_conf,int min_inputs,int max_inputs,const vector* lpCoinsToUse) +{ + if( (lpAssetGroups != NULL) && (lpAssetGroups != 0) ) + { + return CreateAssetGroupingTransaction(this, vecSend,wtxNew,reservekey,nFeeRet,strFailReason,coinControl,addresses,min_conf,min_inputs,max_inputs,lpCoinsToUse,MC_CSF_ALLOW_SPENDABLE_P2SH | MC_CSF_SIGN); + } + return true; +} + +bool CWallet::UpdateUnspentList(const CWalletTx& wtx, bool update_inputs) +{ + const CWalletTx* pcoin = &wtx; + const uint256& wtxid = pcoin->GetHash(); + + int unspent_count=0; +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, not used in this case + + int nDepth = pcoin->GetDepthInMainChain(); + if (nDepth >= 0) + { + for (unsigned int i = 0; i < pcoin->vout.size(); i++) { + isminetype mine = IsMine(pcoin->vout[i]); + if (!(IsSpent(wtxid, i)) && mine != ISMINE_NO) + { + unspent_count++; + } + } + } + + std::map::const_iterator mit = mapUnspent.find(wtxid); + + if(unspent_count) + { + if (mit == mapUnspent.end()) + { + mapUnspent.insert(make_pair(wtxid, unspent_count)); + } + } + else + { + if (mit != mapUnspent.end()) + { + mapUnspent.erase(wtxid); + } + } + + if(update_inputs) + { + BOOST_FOREACH(const CTxIn& txin, wtx.vin) + { + std::map::const_iterator it = mapWallet.find(txin.prevout.hash); + + if (it != mapWallet.end()) + { + UpdateUnspentList(it->second,false); + } + } + } + + return true; +} + +bool CWallet::InitializeUnspentList() +{ + LOCK2(cs_main, cs_wallet); + mapUnspent.clear(); + + int asset_count=0; +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, not used in this case + + for (map::const_iterator it = mapWallet.begin(); it != mapWallet.end(); ++it) + { + const uint256& wtxid = it->first; + const CWalletTx* pcoin = &(*it).second; + + int unspent_count=0; + + int nDepth = pcoin->GetDepthInMainChain(); + if (nDepth >= 0) + { + for (unsigned int i = 0; i < pcoin->vout.size(); i++) { + isminetype mine = IsMine(pcoin->vout[i]); + if (!(IsSpent(wtxid, i)) && (mine & ISMINE_SPENDABLE) != ISMINE_NO) + { + unspent_count++; + } + } + } + + if(unspent_count) + { + mapUnspent.insert(make_pair(wtxid, unspent_count)); + } + } + + if(lpAssetGroups) + { + delete lpAssetGroups; + lpAssetGroups=NULL; + } + + lpAssetGroups=new CAssetGroupTree; + + int assets_per_opdrop=(mc_gState->m_NetworkParams->GetInt64Param("maxstdelementsize")-4)/(mc_gState->m_NetworkParams->m_AssetRefSize+MC_AST_ASSET_QUANTITY_SIZE); + + if(mc_gState->m_Features->VerifySizeOfOpDropElements()) + { + assets_per_opdrop=(mc_gState->m_NetworkParams->GetInt64Param("maxstdelementsize")-4)/(mc_gState->m_NetworkParams->m_AssetRefSize+MC_AST_ASSET_QUANTITY_SIZE); + } + + int max_assets_per_group=assets_per_opdrop*mc_gState->m_NetworkParams->GetInt64Param("maxstdopdropscount"); + + lpAssetGroups->Initialize(1,max_assets_per_group,32,1); + + vector vCoins; + + mc_Buffer *tmp_amounts; + + tmp_amounts=new mc_Buffer; + mc_InitABufferMap(tmp_amounts); + + mc_Script *lpScript; + lpScript=new mc_Script; + if( (tmp_amounts != NULL) && (lpScript != NULL) ) // Count assets in unspent coins + { + AvailableCoins(vCoins, true, NULL,true,true); + sort(vCoins.begin(),vCoins.end(),CompareCOutputByDepthAndScriptSizeDesc()); + tmp_amounts->Clear(); + BOOST_FOREACH(const COutput& out, vCoins) + { + string strError; + CTxOut txout; + uint256 hash=out.GetHashAndTxOut(txout); + + ParseMultichainTxOutToBuffer(hash,txout,tmp_amounts,lpScript,NULL,NULL,strError); + } + asset_count=tmp_amounts->GetCount(); + if(asset_count) // Resize asset grouping to prevent crazy autocombine on + // already autocombined with higher assets-per-group setting + { + lpAssetGroups->Resize(asset_count); + } + BOOST_FOREACH(const COutput& out, vCoins) + { + string strError; + + CTxOut txout; + uint256 hash=out.GetHashAndTxOut(txout); + tmp_amounts->Clear(); + ParseMultichainTxOutToBuffer(hash,txout,tmp_amounts,lpScript,NULL,NULL,strError); + lpAssetGroups->GetGroup(tmp_amounts,1); + } + LogPrint("mchn","mchn: Found %d assets in %d groups\n",asset_count,lpAssetGroups->GroupCount()-1); + lpAssetGroups->Dump(); + } + + if(tmp_amounts) + { + delete tmp_amounts; + } + + if(lpScript) + { + delete lpScript; + } + + + LogPrint("mchn","mchn: Unspent list initialized: Total: %d, Unspent: %d\n",mapWallet.size(),mapUnspent.size()); + + return true; +} + +void CWallet::PurgeSpentCoins(int min_depth,int max_coins) +{ + int unspent_count=0; + int skipped=0; + int count=0; + int total; + +// if(mc_gState->m_WalletMode & MC_WMD_ADDRESS_TXS) // Not supported, not used + + double this_time,last_time; + this_time=mc_TimeNowAsDouble(); + last_time=this_time; + + set should_keep; + for (map::const_iterator itUnspent = mapUnspent.begin(); itUnspent != mapUnspent.end(); ++itUnspent) + { + const uint256& wtxid = itUnspent->first; + if(should_keep.count(wtxid) == 0) + { + should_keep.insert(wtxid); + } +/* + std::map::const_iterator it = mapWallet.find(wtxid); + + if (it != mapWallet.end()) + { + const CWalletTx* pcoin = &(*it).second; + BOOST_FOREACH(const CTxIn& txin, pcoin->vin) + { + if(mapWallet.count(txin.prevout.hash)) + { + if(should_keep.count(txin.prevout.hash) == 0) + { + should_keep.insert(txin.prevout.hash); + } + } + } + } +*/ + } + + vector to_erase; + to_erase.resize(max_coins); + + for (map::const_iterator it = mapWallet.begin(); it != mapWallet.end(); ++it) + { + const uint256& wtxid = it->first; +// std::map::const_iterator itUnspent = mapUnspent.find(wtxid); + +// if (itUnspent == mapUnspent.end()) + if(should_keep.count(wtxid) == 0) + { + const CWalletTx* pcoin = &(*it).second; + + int nDepth = pcoin->GetDepthInMainChain(); + if (nDepth > min_depth) + { + to_erase[count]=wtxid; + count++; + } + else + { + skipped++; + } + + if(count>=max_coins) + { + goto exitlbl; + } + } + else + { + unspent_count++; + skipped++; + } + + + } + + + +exitlbl: + + total=mapWallet.size(); + for(int i=0;ivout[i]; + return tx->GetHash(); + } + txout=coin.m_TXOut; + return coin.m_OutPoint.hash; +} + +bool COutput::IsTrusted() const +{ + if(tx) + { + return tx->IsTrusted(nDepth); + } + return coin.IsTrusted(); +} + +bool OutputCanSend(COutput out) +{ + if(mc_gState->m_NetworkParams->IsProtocolMultichain()) + { + if(mc_gState->m_NetworkParams->GetInt64Param("anyonecansend") == 0) + { + CTxOut txout; + out.GetHashAndTxOut(txout); + const CScript& script1 = txout.scriptPubKey; + CTxDestination addressRet; + if(ExtractDestinationScriptValid(script1, addressRet)) + { + CKeyID *lpKeyID=boost::get (&addressRet); + if(lpKeyID != NULL) + { + if(!mc_gState->m_Permissions->CanSend(NULL,(unsigned char*)(lpKeyID))) + { + return false; + } + } + else + { + return false; + } + + } + else + { + return false; + } + } + } + + return true; +} + + +bool CWallet::CreateAndCommitOptimizeTransaction(CWalletTx& wtx,std::string& strFailReason,const std::set* addresses,int min_conf,int min_inputs,int max_inputs) +{ + CReserveKey reservekey(this); + + LOCK(cs_wallet_send); + + std::vector scriptPubKeys; + CScript scriptOpReturn=CScript(); + CAmount nValue=0; + CAmount nFeeRequired; + + if (!CreateTransaction(scriptPubKeys, nValue, scriptOpReturn, wtx, reservekey, nFeeRequired, strFailReason, NULL, addresses, min_conf, min_inputs, max_inputs)) + { + if (nValue + nFeeRequired > GetBalance()) + strFailReason = strprintf("This transaction requires a transaction fee of at least %s because of its amount, complexity, or use of recently received funds!", FormatMoney(nFeeRequired)); + } + else + { + LogPrint("mchn","Committing wallet optimization tx. Inputs: %ld, Outputs: %ld\n",wtx.vin.size(),wtx.vout.size()); + + if (CommitTransaction(wtx, reservekey, strFailReason)) + { + LogPrint("mchn","Committing wallet optimization tx completed\n"); + return true; + } + } + + return false; +} + +/* + * Should be called under cs_main and cs_wallet + */ + + +bool CWallet::OptimizeUnspentList() +{ + if(mc_TimeNowAsUInt() vCoins; + AvailableCoins(vCoins, true, NULL,true,true); + + map addressesToOptimize; + vector txOutCounts; + int pos; + +// PurgeSpentCoins(8,1000); + BOOST_FOREACH(const COutput& out, vCoins) + { + const CScript& script1 = out.coin.m_TXOut.scriptPubKey; + if(out.fSpendable) + { + CTxDestination addressRet; + if(ExtractDestinationScriptValid(script1, addressRet)) + { + CKeyID *lpKeyID=boost::get (&addressRet); + + if(lpKeyID) + { + if (out.nDepth >= (((out.coin.m_Flags & MC_TFL_FROM_ME)) > 0 ? min_conf : ((min_conf == 0) ? 1 : min_conf))) + { + map ::iterator mi=addressesToOptimize.find(addressRet); + if (mi != addressesToOptimize.end()) + { + pos=mi->second; + txOutCounts[pos]+=1; + } + else + { + pos=txOutCounts.size(); + addressesToOptimize.insert(pair(addressRet, pos)); + txOutCounts.push_back(1); + } + } + } + } + } + } + + bool result=false; + + vector vAddresses; + for (map::iterator it = addressesToOptimize.begin(); + it != addressesToOptimize.end(); + ++it) + { + const unsigned char *aptr; + + aptr=GetAddressIDPtr(it->first); + if(aptr) + { + if(mc_gState->m_Permissions->CanSend(NULL,aptr)) + { + if(mc_gState->m_Permissions->CanReceive(NULL,aptr)) + { + vAddresses.push_back(it->first); + } + } + } + } + + random_shuffle(vAddresses.begin(), vAddresses.end(), GetRandInt); + + BOOST_FOREACH (const CTxDestination& dest, vAddresses) + { + if(!result) + { + map ::iterator it=addressesToOptimize.find(dest); + if (it != addressesToOptimize.end()) + { + if(txOutCounts[it->second] >= min_outputs) + { + set thisAddresses; + set* lpAddresses; + string strError; + + thisAddresses.clear(); + thisAddresses.insert(it->first); + lpAddresses=&thisAddresses; + + const CKeyID *lpKeyID=boost::get (&(it->first)); + CBitcoinAddress bitcoin_address=CBitcoinAddress(*lpKeyID); + if(lpKeyID) + { + CWalletTx wtx; + result=CreateAndCommitOptimizeTransaction(wtx,strError,lpAddresses,min_conf,min_inputs,max_inputs); + if(result) + { + LogPrintf("Combine transaction for address %s (%d inputs,%d outputs): %s; Time: %8.3fs\n", + bitcoin_address.ToString().c_str(), wtx.vin.size(),wtx.vout.size(),wtx.GetHash().GetHex().c_str(),mc_TimeNowAsDouble()-start_time); + } + } + } + } + } + } + + nNextUnspentOptimization=mc_TimeNowAsUInt()+next_delay; + return result; +} diff --git a/src/wallet/walletdb.cpp b/src/wallet/walletdb.cpp new file mode 100644 index 00000000..8fa8e3bb --- /dev/null +++ b/src/wallet/walletdb.cpp @@ -0,0 +1,988 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "walletdb.h" + +#include "structs/base58.h" +#include "protocol/netprotocol.h" +#include "utils/serialize.h" +#include "utils/sync.h" +#include "utils/util.h" +#include "utils/utiltime.h" +#include "wallet/wallet.h" + +#include +#include +#include +#include + +using namespace boost; +using namespace std; + +static uint64_t nAccountingEntryNumber = 0; + +// +// CWalletDB +// + +bool CWalletDB::WriteName(const string& strAddress, const string& strName) +{ + nWalletDBUpdated++; + return Write(make_pair(string("name"), strAddress), strName); +} + +bool CWalletDB::EraseName(const string& strAddress) +{ + // This should only be used for sending addresses, never for receiving addresses, + // receiving addresses must always have an address book entry if they're not change return. + nWalletDBUpdated++; + return Erase(make_pair(string("name"), strAddress)); +} + +bool CWalletDB::WritePurpose(const string& strAddress, const string& strPurpose) +{ + nWalletDBUpdated++; + return Write(make_pair(string("purpose"), strAddress), strPurpose); +} + +bool CWalletDB::ErasePurpose(const string& strPurpose) +{ + nWalletDBUpdated++; + return Erase(make_pair(string("purpose"), strPurpose)); +} + +bool CWalletDB::WriteTx(uint256 hash, const CWalletTx& wtx) +{ + nWalletDBUpdated++; + return Write(std::make_pair(std::string("tx"), hash), wtx); +} + +bool CWalletDB::EraseTx(uint256 hash) +{ + nWalletDBUpdated++; + return Erase(std::make_pair(std::string("tx"), hash)); +} + +bool CWalletDB::WriteKey(const CPubKey& vchPubKey, const CPrivKey& vchPrivKey, const CKeyMetadata& keyMeta) +{ + nWalletDBUpdated++; + + if (!Write(std::make_pair(std::string("keymeta"), vchPubKey), + keyMeta, false)) + return false; + + // hash pubkey/privkey to accelerate wallet load + std::vector vchKey; + vchKey.reserve(vchPubKey.size() + vchPrivKey.size()); + vchKey.insert(vchKey.end(), vchPubKey.begin(), vchPubKey.end()); + vchKey.insert(vchKey.end(), vchPrivKey.begin(), vchPrivKey.end()); + + return Write(std::make_pair(std::string("key"), vchPubKey), std::make_pair(vchPrivKey, Hash(vchKey.begin(), vchKey.end())), false); +} + +bool CWalletDB::WriteCryptedKey(const CPubKey& vchPubKey, + const std::vector& vchCryptedSecret, + const CKeyMetadata &keyMeta) +{ + const bool fEraseUnencryptedKey = true; + nWalletDBUpdated++; + + if (!Write(std::make_pair(std::string("keymeta"), vchPubKey), + keyMeta)) + return false; + + if (!Write(std::make_pair(std::string("ckey"), vchPubKey), vchCryptedSecret, false)) + return false; + if (fEraseUnencryptedKey) + { + Erase(std::make_pair(std::string("key"), vchPubKey)); + Erase(std::make_pair(std::string("wkey"), vchPubKey)); + } + return true; +} + +bool CWalletDB::WriteMasterKey(unsigned int nID, const CMasterKey& kMasterKey) +{ + nWalletDBUpdated++; + return Write(std::make_pair(std::string("mkey"), nID), kMasterKey, true); +} + +bool CWalletDB::WriteCScript(const uint160& hash, const CScript& redeemScript) +{ + nWalletDBUpdated++; + return Write(std::make_pair(std::string("cscript"), hash), redeemScript, false); +} + +bool CWalletDB::WriteWatchOnly(const CScript &dest) +{ + nWalletDBUpdated++; + return Write(std::make_pair(std::string("watchs"), dest), '1'); +} + +bool CWalletDB::EraseWatchOnly(const CScript &dest) +{ + nWalletDBUpdated++; + return Erase(std::make_pair(std::string("watchs"), dest)); +} + +bool CWalletDB::WriteBestBlock(const CBlockLocator& locator) +{ + nWalletDBUpdated++; + return Write(std::string("bestblock"), locator); +} + +bool CWalletDB::ReadBestBlock(CBlockLocator& locator) +{ + return Read(std::string("bestblock"), locator); +} + +bool CWalletDB::WriteOrderPosNext(int64_t nOrderPosNext) +{ + nWalletDBUpdated++; + return Write(std::string("orderposnext"), nOrderPosNext); +} + +bool CWalletDB::WriteDefaultKey(const CPubKey& vchPubKey) +{ + nWalletDBUpdated++; + return Write(std::string("defaultkey"), vchPubKey); +} + +bool CWalletDB::ReadPool(int64_t nPool, CKeyPool& keypool) +{ + return Read(std::make_pair(std::string("pool"), nPool), keypool); +} + +bool CWalletDB::WritePool(int64_t nPool, const CKeyPool& keypool) +{ + nWalletDBUpdated++; + return Write(std::make_pair(std::string("pool"), nPool), keypool); +} + +bool CWalletDB::ErasePool(int64_t nPool) +{ + nWalletDBUpdated++; + return Erase(std::make_pair(std::string("pool"), nPool)); +} + +bool CWalletDB::WriteMinVersion(int nVersion) +{ + return Write(std::string("minversion"), nVersion); +} + +bool CWalletDB::ReadAccount(const string& strAccount, CAccount& account) +{ + account.SetNull(); + return Read(make_pair(string("acc"), strAccount), account); +} + +bool CWalletDB::WriteAccount(const string& strAccount, const CAccount& account) +{ + return Write(make_pair(string("acc"), strAccount), account); +} + +bool CWalletDB::WriteAccountingEntry(const uint64_t nAccEntryNum, const CAccountingEntry& acentry) +{ + return Write(std::make_pair(std::string("acentry"), std::make_pair(acentry.strAccount, nAccEntryNum)), acentry); +} + +bool CWalletDB::WriteAccountingEntry(const CAccountingEntry& acentry) +{ + return WriteAccountingEntry(++nAccountingEntryNumber, acentry); +} + +CAmount CWalletDB::GetAccountCreditDebit(const string& strAccount) +{ + list entries; + ListAccountCreditDebit(strAccount, entries); + + CAmount nCreditDebit = 0; + BOOST_FOREACH (const CAccountingEntry& entry, entries) + nCreditDebit += entry.nCreditDebit; + + return nCreditDebit; +} + +void CWalletDB::ListAccountCreditDebit(const string& strAccount, list& entries) +{ + bool fAllAccounts = (strAccount == "*"); + + Dbc* pcursor = GetCursor(); + if (!pcursor) + throw runtime_error("CWalletDB::ListAccountCreditDebit() : cannot create DB cursor"); + unsigned int fFlags = DB_SET_RANGE; + while (true) + { + // Read next record + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + if (fFlags == DB_SET_RANGE) + ssKey << std::make_pair(std::string("acentry"), std::make_pair((fAllAccounts ? string("") : strAccount), uint64_t(0))); + CDataStream ssValue(SER_DISK, CLIENT_VERSION); + int ret = ReadAtCursor(pcursor, ssKey, ssValue, fFlags); + fFlags = DB_NEXT; + if (ret == DB_NOTFOUND) + break; + else if (ret != 0) + { + pcursor->close(); + throw runtime_error("CWalletDB::ListAccountCreditDebit() : error scanning DB"); + } + + // Unserialize + string strType; + ssKey >> strType; + if (strType != "acentry") + break; + CAccountingEntry acentry; + ssKey >> acentry.strAccount; + if (!fAllAccounts && acentry.strAccount != strAccount) + break; + + ssValue >> acentry; + ssKey >> acentry.nEntryNo; + entries.push_back(acentry); + } + + pcursor->close(); +} + +DBErrors CWalletDB::ReorderTransactions(CWallet* pwallet) +{ + LOCK(pwallet->cs_wallet); + // Old wallets didn't have any defined order for transactions + // Probably a bad idea to change the output of this + + // First: get all CWalletTx and CAccountingEntry into a sorted-by-time multimap. + typedef pair TxPair; + typedef multimap TxItems; + TxItems txByTime; + + for (map::iterator it = pwallet->mapWallet.begin(); it != pwallet->mapWallet.end(); ++it) + { + CWalletTx* wtx = &((*it).second); + txByTime.insert(make_pair(wtx->nTimeReceived, TxPair(wtx, (CAccountingEntry*)0))); + } + list acentries; + ListAccountCreditDebit("", acentries); + BOOST_FOREACH(CAccountingEntry& entry, acentries) + { + txByTime.insert(make_pair(entry.nTime, TxPair((CWalletTx*)0, &entry))); + } + + int64_t& nOrderPosNext = pwallet->nOrderPosNext; + nOrderPosNext = 0; + std::vector nOrderPosOffsets; + for (TxItems::iterator it = txByTime.begin(); it != txByTime.end(); ++it) + { + CWalletTx *const pwtx = (*it).second.first; + CAccountingEntry *const pacentry = (*it).second.second; + int64_t& nOrderPos = (pwtx != 0) ? pwtx->nOrderPos : pacentry->nOrderPos; + + if (nOrderPos == -1) + { + nOrderPos = nOrderPosNext++; + nOrderPosOffsets.push_back(nOrderPos); + + if (pwtx) + { + if (!WriteTx(pwtx->GetHash(), *pwtx)) + return DB_LOAD_FAIL; + } + else + if (!WriteAccountingEntry(pacentry->nEntryNo, *pacentry)) + return DB_LOAD_FAIL; + } + else + { + int64_t nOrderPosOff = 0; + BOOST_FOREACH(const int64_t& nOffsetStart, nOrderPosOffsets) + { + if (nOrderPos >= nOffsetStart) + ++nOrderPosOff; + } + nOrderPos += nOrderPosOff; + nOrderPosNext = std::max(nOrderPosNext, nOrderPos + 1); + + if (!nOrderPosOff) + continue; + + // Since we're changing the order, write it back + if (pwtx) + { + if (!WriteTx(pwtx->GetHash(), *pwtx)) + return DB_LOAD_FAIL; + } + else + if (!WriteAccountingEntry(pacentry->nEntryNo, *pacentry)) + return DB_LOAD_FAIL; + } + } + WriteOrderPosNext(nOrderPosNext); + + return DB_LOAD_OK; +} + +class CWalletScanState { +public: + unsigned int nKeys; + unsigned int nCKeys; + unsigned int nKeyMeta; + bool fIsEncrypted; + bool fAnyUnordered; + int nFileVersion; + vector vWalletUpgrade; + + CWalletScanState() { + nKeys = nCKeys = nKeyMeta = 0; + fIsEncrypted = false; + fAnyUnordered = false; + nFileVersion = 0; + } +}; + +bool +ReadKeyValue(CWallet* pwallet, CDataStream& ssKey, CDataStream& ssValue, + CWalletScanState &wss, string& strType, string& strErr) +{ + try { + // Unserialize + // Taking advantage of the fact that pair serialization + // is just the two items serialized one after the other + ssKey >> strType; + if (strType == "name") + { + string strAddress; + ssKey >> strAddress; + ssValue >> pwallet->mapAddressBook[CBitcoinAddress(strAddress).Get()].name; + } + else if (strType == "purpose") + { + string strAddress; + ssKey >> strAddress; + ssValue >> pwallet->mapAddressBook[CBitcoinAddress(strAddress).Get()].purpose; + } + else if (strType == "tx") + { + uint256 hash; + ssKey >> hash; + CWalletTx wtx; + ssValue >> wtx; + CValidationState state; + if (!(CheckTransaction(wtx, state) && (wtx.GetHash() == hash) && state.IsValid())) + return false; + + // Undo serialize changes in 31600 + if (31404 <= wtx.fTimeReceivedIsTxTime && wtx.fTimeReceivedIsTxTime <= 31703) + { + if (!ssValue.empty()) + { + char fTmp; + char fUnused; + ssValue >> fTmp >> fUnused >> wtx.strFromAccount; + strErr = strprintf("LoadWallet() upgrading tx ver=%d %d '%s' %s", + wtx.fTimeReceivedIsTxTime, fTmp, wtx.strFromAccount, hash.ToString()); + wtx.fTimeReceivedIsTxTime = fTmp; + } + else + { + strErr = strprintf("LoadWallet() repairing tx ver=%d %s", wtx.fTimeReceivedIsTxTime, hash.ToString()); + wtx.fTimeReceivedIsTxTime = 0; + } + wss.vWalletUpgrade.push_back(hash); + } + + if (wtx.nOrderPos == -1) + wss.fAnyUnordered = true; + + pwallet->AddToWallet(wtx, true); + } + else if (strType == "acentry") + { + string strAccount; + ssKey >> strAccount; + uint64_t nNumber; + ssKey >> nNumber; + if (nNumber > nAccountingEntryNumber) + nAccountingEntryNumber = nNumber; + + if (!wss.fAnyUnordered) + { + CAccountingEntry acentry; + ssValue >> acentry; + if (acentry.nOrderPos == -1) + wss.fAnyUnordered = true; + } + } + else if (strType == "watchs") + { + CScript script; + ssKey >> script; + char fYes; + ssValue >> fYes; + if (fYes == '1') + pwallet->LoadWatchOnly(script); + + // Watch-only addresses have no birthday information for now, + // so set the wallet birthday to the beginning of time. + pwallet->nTimeFirstKey = 1; + } + else if (strType == "key" || strType == "wkey") + { + CPubKey vchPubKey; + ssKey >> vchPubKey; + if (!vchPubKey.IsValid()) + { + strErr = "Error reading wallet database: CPubKey corrupt"; + return false; + } + CKey key; + CPrivKey pkey; + uint256 hash = 0; + + if (strType == "key") + { + wss.nKeys++; + ssValue >> pkey; + } else { + CWalletKey wkey; + ssValue >> wkey; + pkey = wkey.vchPrivKey; + } + + // Old wallets store keys as "key" [pubkey] => [privkey] + // ... which was slow for wallets with lots of keys, because the public key is re-derived from the private key + // using EC operations as a checksum. + // Newer wallets store keys as "key"[pubkey] => [privkey][hash(pubkey,privkey)], which is much faster while + // remaining backwards-compatible. + try + { + ssValue >> hash; + } + catch(...){} + + bool fSkipCheck = false; + + if (hash != 0) + { + // hash pubkey/privkey to accelerate wallet load + std::vector vchKey; + vchKey.reserve(vchPubKey.size() + pkey.size()); + vchKey.insert(vchKey.end(), vchPubKey.begin(), vchPubKey.end()); + vchKey.insert(vchKey.end(), pkey.begin(), pkey.end()); + + if (Hash(vchKey.begin(), vchKey.end()) != hash) + { + strErr = "Error reading wallet database: CPubKey/CPrivKey corrupt"; + return false; + } + + fSkipCheck = true; + } + + if (!key.Load(pkey, vchPubKey, fSkipCheck)) + { + strErr = "Error reading wallet database: CPrivKey corrupt"; + return false; + } + if (!pwallet->LoadKey(key, vchPubKey)) + { + strErr = "Error reading wallet database: LoadKey failed"; + return false; + } + } + else if (strType == "mkey") + { + unsigned int nID; + ssKey >> nID; + CMasterKey kMasterKey; + ssValue >> kMasterKey; + if(pwallet->mapMasterKeys.count(nID) != 0) + { + strErr = strprintf("Error reading wallet database: duplicate CMasterKey id %u", nID); + return false; + } + pwallet->mapMasterKeys[nID] = kMasterKey; + if (pwallet->nMasterKeyMaxID < nID) + pwallet->nMasterKeyMaxID = nID; + } + else if (strType == "ckey") + { + vector vchPubKey; + ssKey >> vchPubKey; + vector vchPrivKey; + ssValue >> vchPrivKey; + wss.nCKeys++; + + if (!pwallet->LoadCryptedKey(vchPubKey, vchPrivKey)) + { + strErr = "Error reading wallet database: LoadCryptedKey failed"; + return false; + } + wss.fIsEncrypted = true; + } + else if (strType == "keymeta") + { + CPubKey vchPubKey; + ssKey >> vchPubKey; + CKeyMetadata keyMeta; + ssValue >> keyMeta; + wss.nKeyMeta++; + + pwallet->LoadKeyMetadata(vchPubKey, keyMeta); + + // find earliest key creation time, as wallet birthday + if (!pwallet->nTimeFirstKey || + (keyMeta.nCreateTime < pwallet->nTimeFirstKey)) + pwallet->nTimeFirstKey = keyMeta.nCreateTime; + } + else if (strType == "defaultkey") + { + ssValue >> pwallet->vchDefaultKey; + } + else if (strType == "pool") + { + int64_t nIndex; + ssKey >> nIndex; + CKeyPool keypool; + ssValue >> keypool; + pwallet->setKeyPool.insert(nIndex); + + // If no metadata exists yet, create a default with the pool key's + // creation time. Note that this may be overwritten by actually + // stored metadata for that key later, which is fine. + CKeyID keyid = keypool.vchPubKey.GetID(); + if (pwallet->mapKeyMetadata.count(keyid) == 0) + pwallet->mapKeyMetadata[keyid] = CKeyMetadata(keypool.nTime); + } + else if (strType == "version") + { + ssValue >> wss.nFileVersion; + if (wss.nFileVersion == 10300) + wss.nFileVersion = 300; + } + else if (strType == "cscript") + { + uint160 hash; + ssKey >> hash; + CScript script; + ssValue >> script; + if (!pwallet->LoadCScript(script)) + { + strErr = "Error reading wallet database: LoadCScript failed"; + return false; + } + } + else if (strType == "orderposnext") + { + ssValue >> pwallet->nOrderPosNext; + } + else if (strType == "destdata") + { + std::string strAddress, strKey, strValue; + ssKey >> strAddress; + ssKey >> strKey; + ssValue >> strValue; + if (!pwallet->LoadDestData(CBitcoinAddress(strAddress).Get(), strKey, strValue)) + { + strErr = "Error reading wallet database: LoadDestData failed"; + return false; + } + } + } catch (...) + { + return false; + } + return true; +} + +static bool IsKeyType(string strType) +{ + return (strType== "key" || strType == "wkey" || + strType == "mkey" || strType == "ckey"); +} + +DBErrors CWalletDB::LoadWallet(CWallet* pwallet) +{ + pwallet->vchDefaultKey = CPubKey(); + CWalletScanState wss; + bool fNoncriticalErrors = false; + DBErrors result = DB_LOAD_OK; + + try { + LOCK(pwallet->cs_wallet); + int nMinVersion = 0; + if (Read((string)"minversion", nMinVersion)) + { + if (nMinVersion > CLIENT_VERSION) + return DB_TOO_NEW; + pwallet->LoadMinVersion(nMinVersion); + } + + // Get cursor + Dbc* pcursor = GetCursor(); + if (!pcursor) + { + LogPrintf("Error getting wallet database cursor\n"); + return DB_CORRUPT; + } + + while (true) + { + // Read next record + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + CDataStream ssValue(SER_DISK, CLIENT_VERSION); + int ret = ReadAtCursor(pcursor, ssKey, ssValue); + if (ret == DB_NOTFOUND) + break; + else if (ret != 0) + { + LogPrintf("Error reading next record from wallet database\n"); + return DB_CORRUPT; + } + + // Try to be tolerant of single corrupt records: + string strType, strErr; + if (!ReadKeyValue(pwallet, ssKey, ssValue, wss, strType, strErr)) + { + // losing keys is considered a catastrophic error, anything else + // we assume the user can live with: + if (IsKeyType(strType)) + result = DB_CORRUPT; + else + { + // Leave other errors alone, if we try to fix them we might make things worse. + fNoncriticalErrors = true; // ... but do warn the user there is something wrong. + if (strType == "tx") + // Rescan if there is a bad transaction record: + SoftSetBoolArg("-rescan", true); + } + } + if (!strErr.empty()) + LogPrintf("%s\n", strErr); + } + pcursor->close(); + } + catch (boost::thread_interrupted) { + throw; + } + catch (...) { + result = DB_CORRUPT; + } + + if (fNoncriticalErrors && result == DB_LOAD_OK) + result = DB_NONCRITICAL_ERROR; + + // Any wallet corruption at all: skip any rewriting or + // upgrading, we don't want to make it worse. + if (result != DB_LOAD_OK) + return result; + + LogPrintf("nFileVersion = %d\n", wss.nFileVersion); + + LogPrintf("Keys: %u plaintext, %u encrypted, %u w/ metadata, %u total\n", + wss.nKeys, wss.nCKeys, wss.nKeyMeta, wss.nKeys + wss.nCKeys); + + // nTimeFirstKey is only reliable if all keys have metadata + if ((wss.nKeys + wss.nCKeys) != wss.nKeyMeta) + pwallet->nTimeFirstKey = 1; // 0 would be considered 'no value' + + BOOST_FOREACH(uint256 hash, wss.vWalletUpgrade) + WriteTx(hash, pwallet->mapWallet[hash]); + + // Rewrite encrypted wallets of versions 0.4.0 and 0.5.0rc: + if (wss.fIsEncrypted && (wss.nFileVersion == 40000 || wss.nFileVersion == 50000)) + return DB_NEED_REWRITE; + + if (wss.nFileVersion < CLIENT_VERSION) // Update + WriteVersion(CLIENT_VERSION); + + if (wss.fAnyUnordered) + result = ReorderTransactions(pwallet); + + return result; +} + +DBErrors CWalletDB::FindWalletTx(CWallet* pwallet, vector& vTxHash, vector& vWtx) +{ + pwallet->vchDefaultKey = CPubKey(); + bool fNoncriticalErrors = false; + DBErrors result = DB_LOAD_OK; + + try { + LOCK(pwallet->cs_wallet); + int nMinVersion = 0; + if (Read((string)"minversion", nMinVersion)) + { + if (nMinVersion > CLIENT_VERSION) + return DB_TOO_NEW; + pwallet->LoadMinVersion(nMinVersion); + } + + // Get cursor + Dbc* pcursor = GetCursor(); + if (!pcursor) + { + LogPrintf("Error getting wallet database cursor\n"); + return DB_CORRUPT; + } + + while (true) + { + // Read next record + CDataStream ssKey(SER_DISK, CLIENT_VERSION); + CDataStream ssValue(SER_DISK, CLIENT_VERSION); + int ret = ReadAtCursor(pcursor, ssKey, ssValue); + if (ret == DB_NOTFOUND) + break; + else if (ret != 0) + { + LogPrintf("Error reading next record from wallet database\n"); + return DB_CORRUPT; + } + + string strType; + ssKey >> strType; + if (strType == "tx") { + uint256 hash; + ssKey >> hash; + + CWalletTx wtx; + ssValue >> wtx; + + vTxHash.push_back(hash); + vWtx.push_back(wtx); + } + } + pcursor->close(); + } + catch (boost::thread_interrupted) { + throw; + } + catch (...) { + result = DB_CORRUPT; + } + + if (fNoncriticalErrors && result == DB_LOAD_OK) + result = DB_NONCRITICAL_ERROR; + + return result; +} + +DBErrors CWalletDB::ZapWalletTx(CWallet* pwallet, vector& vWtx) +{ + // build list of wallet TXs + vector vTxHash; + DBErrors err = FindWalletTx(pwallet, vTxHash, vWtx); + if (err != DB_LOAD_OK) + return err; + + // erase each wallet TX + BOOST_FOREACH (uint256& hash, vTxHash) { + if (!EraseTx(hash)) + return DB_CORRUPT; + } + + return DB_LOAD_OK; +} + +void ThreadFlushWalletDB(const string& strFile) +{ + // Make this thread recognisable as the wallet flushing thread + RenameThread("bitcoin-wallet"); + + static bool fOneThread; + if (fOneThread) + return; + fOneThread = true; + if (!GetBoolArg("-flushwallet", true)) + return; + + unsigned int nLastSeen = nWalletDBUpdated; + unsigned int nLastFlushed = nWalletDBUpdated; + int64_t nLastWalletUpdate = GetTime(); + while (true) + { + MilliSleep(500); + + if (nLastSeen != nWalletDBUpdated) + { + nLastSeen = nWalletDBUpdated; + nLastWalletUpdate = GetTime(); + } + + if (nLastFlushed != nWalletDBUpdated && GetTime() - nLastWalletUpdate >= 2) + { + TRY_LOCK(bitdb.cs_db,lockDb); + if (lockDb) + { + // Don't do this if any databases are in use + int nRefCount = 0; + map::iterator mi = bitdb.mapFileUseCount.begin(); + while (mi != bitdb.mapFileUseCount.end()) + { + nRefCount += (*mi).second; + mi++; + } + + if (nRefCount == 0) + { + boost::this_thread::interruption_point(); + map::iterator mi = bitdb.mapFileUseCount.find(strFile); + if (mi != bitdb.mapFileUseCount.end()) + { + LogPrint("db", "Flushing wallet.dat\n"); + nLastFlushed = nWalletDBUpdated; + int64_t nStart = GetTimeMillis(); + + // Flush wallet.dat so it's self contained + bitdb.CloseDb(strFile); + bitdb.CheckpointLSN(strFile); + + bitdb.mapFileUseCount.erase(mi++); + LogPrint("db", "Flushed wallet.dat %dms\n", GetTimeMillis() - nStart); + } + } + } + } + } +} + +bool BackupWallet(const CWallet& wallet, const string& strDest) +{ + if (!wallet.fFileBacked) + return false; + while (true) + { + { + LOCK(bitdb.cs_db); + if (!bitdb.mapFileUseCount.count(wallet.strWalletFile) || bitdb.mapFileUseCount[wallet.strWalletFile] == 0) + { + // Flush log data to the dat file + bitdb.CloseDb(wallet.strWalletFile); + bitdb.CheckpointLSN(wallet.strWalletFile); + bitdb.mapFileUseCount.erase(wallet.strWalletFile); + + // Copy wallet.dat + filesystem::path pathSrc = GetDataDir() / wallet.strWalletFile; + filesystem::path pathDest(strDest); + if (filesystem::is_directory(pathDest)) + pathDest /= wallet.strWalletFile; + + try { +#if BOOST_VERSION >= 104000 + filesystem::copy_file(pathSrc, pathDest, filesystem::copy_option::overwrite_if_exists); +#else + filesystem::copy_file(pathSrc, pathDest); +#endif + LogPrintf("copied wallet.dat to %s\n", pathDest.string()); + return true; + } catch(const filesystem::filesystem_error &e) { + LogPrintf("error copying wallet.dat to %s - %s\n", pathDest.string(), e.what()); + return false; + } + } + } + MilliSleep(100); + } + return false; +} + +// +// Try to (very carefully!) recover wallet.dat if there is a problem. +// +bool CWalletDB::Recover(CDBEnv& dbenv, std::string filename, bool fOnlyKeys) +{ + // Recovery procedure: + // move wallet.dat to wallet.timestamp.bak + // Call Salvage with fAggressive=true to + // get as much data as possible. + // Rewrite salvaged data to wallet.dat + // Set -rescan so any missing transactions will be + // found. + int64_t now = GetTime(); + std::string newFilename = strprintf("wallet.%d.bak", now); + + int result = dbenv.dbenv.dbrename(NULL, filename.c_str(), NULL, + newFilename.c_str(), DB_AUTO_COMMIT); + if (result == 0) + LogPrintf("Renamed %s to %s\n", filename, newFilename); + else + { + LogPrintf("Failed to rename %s to %s\n", filename, newFilename); + return false; + } + + std::vector salvagedData; + bool allOK = dbenv.Salvage(newFilename, true, salvagedData); + if (salvagedData.empty()) + { + LogPrintf("Salvage(aggressive) found no records in %s.\n", newFilename); + return false; + } + LogPrintf("Salvage(aggressive) found %u records\n", salvagedData.size()); + + bool fSuccess = allOK; + boost::scoped_ptr pdbCopy(new Db(&dbenv.dbenv, 0)); + int ret = pdbCopy->open(NULL, // Txn pointer + filename.c_str(), // Filename + "main", // Logical db name + DB_BTREE, // Database type + DB_CREATE, // Flags + 0); + if (ret > 0) + { + LogPrintf("Cannot create database file %s\n", filename); + return false; + } + CWallet dummyWallet; + CWalletScanState wss; + + DbTxn* ptxn = dbenv.TxnBegin(); + BOOST_FOREACH(CDBEnv::KeyValPair& row, salvagedData) + { + if (fOnlyKeys) + { + CDataStream ssKey(row.first, SER_DISK, CLIENT_VERSION); + CDataStream ssValue(row.second, SER_DISK, CLIENT_VERSION); + string strType, strErr; + bool fReadOK = ReadKeyValue(&dummyWallet, ssKey, ssValue, + wss, strType, strErr); + if (!IsKeyType(strType)) + continue; + if (!fReadOK) + { + LogPrintf("WARNING: CWalletDB::Recover skipping %s: %s\n", strType, strErr); + continue; + } + } + Dbt datKey(&row.first[0], row.first.size()); + Dbt datValue(&row.second[0], row.second.size()); + int ret2 = pdbCopy->put(ptxn, &datKey, &datValue, DB_NOOVERWRITE); + if (ret2 > 0) + fSuccess = false; + } + ptxn->commit(0); + pdbCopy->close(0); + + return fSuccess; +} + +bool CWalletDB::Recover(CDBEnv& dbenv, std::string filename) +{ + return CWalletDB::Recover(dbenv, filename, false); +} + +bool CWalletDB::WriteDestData(const std::string &address, const std::string &key, const std::string &value) +{ + nWalletDBUpdated++; + return Write(std::make_pair(std::string("destdata"), std::make_pair(address, key)), value); +} + +bool CWalletDB::EraseDestData(const std::string &address, const std::string &key) +{ + nWalletDBUpdated++; + return Erase(std::make_pair(std::string("destdata"), std::make_pair(address, key))); +} diff --git a/src/wallet/walletdb.h b/src/wallet/walletdb.h new file mode 100644 index 00000000..59bfa52d --- /dev/null +++ b/src/wallet/walletdb.h @@ -0,0 +1,143 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef BITCOIN_WALLETDB_H +#define BITCOIN_WALLETDB_H + +#include "structs/amount.h" +#include "wallet/db.h" +#include "keys/key.h" +#include "wallet/keystore.h" + +#include +#include +#include +#include +#include + +class CAccount; +class CAccountingEntry; +struct CBlockLocator; +class CKeyPool; +class CMasterKey; +class CScript; +class CWallet; +class CWalletTx; +class uint160; +class uint256; + +/** Error statuses for the wallet database */ +enum DBErrors +{ + DB_LOAD_OK, + DB_CORRUPT, + DB_NONCRITICAL_ERROR, + DB_TOO_NEW, + DB_LOAD_FAIL, + DB_NEED_REWRITE +}; + +class CKeyMetadata +{ +public: + static const int CURRENT_VERSION=1; + int nVersion; + int64_t nCreateTime; // 0 means unknown + + CKeyMetadata() + { + SetNull(); + } + CKeyMetadata(int64_t nCreateTime_) + { + nVersion = CKeyMetadata::CURRENT_VERSION; + nCreateTime = nCreateTime_; + } + + ADD_SERIALIZE_METHODS; + + template + inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) { + READWRITE(this->nVersion); + nVersion = this->nVersion; + READWRITE(nCreateTime); + } + + void SetNull() + { + nVersion = CKeyMetadata::CURRENT_VERSION; + nCreateTime = 0; + } +}; + +/** Access to the wallet database (wallet.dat) */ +class CWalletDB : public CDB +{ +public: + CWalletDB(const std::string& strFilename, const char* pszMode = "r+") : CDB(strFilename, pszMode) + { + } + + bool WriteName(const std::string& strAddress, const std::string& strName); + bool EraseName(const std::string& strAddress); + + bool WritePurpose(const std::string& strAddress, const std::string& purpose); + bool ErasePurpose(const std::string& strAddress); + + bool WriteTx(uint256 hash, const CWalletTx& wtx); + bool EraseTx(uint256 hash); + + bool WriteKey(const CPubKey& vchPubKey, const CPrivKey& vchPrivKey, const CKeyMetadata &keyMeta); + bool WriteCryptedKey(const CPubKey& vchPubKey, const std::vector& vchCryptedSecret, const CKeyMetadata &keyMeta); + bool WriteMasterKey(unsigned int nID, const CMasterKey& kMasterKey); + + bool WriteCScript(const uint160& hash, const CScript& redeemScript); + + bool WriteWatchOnly(const CScript &script); + bool EraseWatchOnly(const CScript &script); + + bool WriteBestBlock(const CBlockLocator& locator); + bool ReadBestBlock(CBlockLocator& locator); + + bool WriteOrderPosNext(int64_t nOrderPosNext); + + bool WriteDefaultKey(const CPubKey& vchPubKey); + + bool ReadPool(int64_t nPool, CKeyPool& keypool); + bool WritePool(int64_t nPool, const CKeyPool& keypool); + bool ErasePool(int64_t nPool); + + bool WriteMinVersion(int nVersion); + + bool ReadAccount(const std::string& strAccount, CAccount& account); + bool WriteAccount(const std::string& strAccount, const CAccount& account); + + /// Write destination data key,value tuple to database + bool WriteDestData(const std::string &address, const std::string &key, const std::string &value); + /// Erase destination data tuple from wallet database + bool EraseDestData(const std::string &address, const std::string &key); + + bool WriteAccountingEntry(const CAccountingEntry& acentry); + CAmount GetAccountCreditDebit(const std::string& strAccount); + void ListAccountCreditDebit(const std::string& strAccount, std::list& acentries); + + DBErrors ReorderTransactions(CWallet* pwallet); + DBErrors LoadWallet(CWallet* pwallet); + DBErrors FindWalletTx(CWallet* pwallet, std::vector& vTxHash, std::vector& vWtx); + DBErrors ZapWalletTx(CWallet* pwallet, std::vector& vWtx); + static bool Recover(CDBEnv& dbenv, std::string filename, bool fOnlyKeys); + static bool Recover(CDBEnv& dbenv, std::string filename); + +private: + CWalletDB(const CWalletDB&); + void operator=(const CWalletDB&); + + bool WriteAccountingEntry(const uint64_t nAccEntryNum, const CAccountingEntry& acentry); +}; + +bool BackupWallet(const CWallet& wallet, const std::string& strDest); + +#endif // BITCOIN_WALLETDB_H diff --git a/src/wallet/wallettxdb.cpp b/src/wallet/wallettxdb.cpp new file mode 100644 index 00000000..c81e3dea --- /dev/null +++ b/src/wallet/wallettxdb.cpp @@ -0,0 +1,3039 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "multichain/multichain.h" +#include "wallet/wallettxdb.h" + +#define MC_TDB_MAX_TXS_FILE_SIZE 0x8000000 // Maximal data file size, 128MB + +void sprintf_hex(char *hex,const unsigned char *bin,int size) +{ + int i; + for(i=0;iInitialize(sizeof(mc_TxEntity),sizeof(mc_TxEntity),MC_BUF_MODE_MAP); + return m_Entities->Initialize(sizeof(mc_TxEntity),sizeof(mc_TxEntityStat),MC_BUF_MODE_MAP); +} + +int mc_TxImport::AddEntity(mc_TxEntity *entity) +{ + mc_TxEntityStat stat; + if(m_Entities == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + if(m_Entities->Seek((unsigned char*)entity) >= 0) + { + return MC_ERR_FOUND; + } + + stat.Zero(); + memcpy(&stat.m_Entity,entity,sizeof(mc_TxEntity)); + stat.m_Generation=m_ImportID; + stat.m_PosInImport=1; + if(m_Entities->GetCount()) + { + stat.m_PosInImport=((mc_TxEntityStat*)m_Entities->GetRow(m_Entities->GetCount()-1))->m_PosInImport+1; + } + return m_Entities->Add(&stat,(char*)&stat+sizeof(mc_TxEntity)); +} + +int mc_TxImport::AddEntity(mc_TxEntityStat *entity) +{ + if(m_Entities == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + if(m_Entities->Seek((unsigned char*)entity) >= 0) + { + return MC_ERR_FOUND; + } + return m_Entities->Add(entity,(char*)entity+sizeof(mc_TxEntity)); +} + +int mc_TxImport::FindEntity(mc_TxEntityStat *entity) +{ + return m_Entities->Seek((unsigned char*)entity); +} + +int mc_TxImport::FindEntity(mc_TxEntity *entity) +{ + return m_Entities->Seek((unsigned char*)entity); +} + +mc_TxEntityStat *mc_TxImport::GetEntity(int row) +{ + if((row < 0) || (row >= m_Entities->GetCount())) + { + return NULL; + } + return (mc_TxEntityStat *)m_Entities->GetRow(row); +} + + +void mc_TxImport::Destroy() +{ + if(m_Entities) + { + delete m_Entities; + } + if(m_TmpEntities) + { + delete m_TmpEntities; + } + + Zero(); +} + + +void mc_TxDefRow::Zero() +{ + memset(this,0,sizeof(mc_TxDefRow)); +} + +void mc_TxEntityDB::Zero() +{ + m_FileName[0]=0; + m_DB=NULL; + m_KeyOffset=0; + m_KeySize=32; + m_ValueOffset=32; + m_ValueSize=48; + m_TotalSize=m_KeySize+m_ValueSize; +} + +void mc_TxEntityDB::SetName(const char* name) +{ + mc_GetFullFileName(name,"wallet/txs",".db",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR,m_FileName); +} + +int mc_TxEntityDB::Open() +{ + + m_DB=new mc_Database; + + m_DB->SetOption("KeySize",0,m_KeySize); + m_DB->SetOption("ValueSize",0,m_ValueSize); + + return m_DB->Open(m_FileName,MC_OPT_DB_DATABASE_CREATE_IF_MISSING | MC_OPT_DB_DATABASE_TRANSACTIONAL | MC_OPT_DB_DATABASE_LEVELDB); +} + +int mc_TxEntityDB::Close() +{ + if(m_DB) + { + m_DB->Close(); + delete m_DB; + m_DB=NULL; + } + return 0; +} + +void mc_TxDB::Zero() +{ + int i; + m_Database=NULL; + m_DBStat.Zero(); + + for(i=0;iSetName(name); + + err=m_Database->Open(); + + if(err) + { + LogString("Initialize: Cannot open database"); + return err; + } + + m_DBStat.Zero(); + + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&m_DBStat+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(err) + { + LogString("Initialize: Cannot read from database"); + return err; + } + + + + + if(ptr) + { + memcpy((char*)&m_DBStat+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + + edbImport.Zero(); + edbImport.m_ImportID=0; + edbImport.m_Pos=0; + edbImport.SwapPosBytes(); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,MC_OPT_DB_DATABASE_SEEK_ON_READ,&err); + edbImport.SwapPosBytes(); + if(err) + { + return err; + } + if(ptr == NULL) + { + return MC_ERR_NOERROR; + } + + import_count=-1; + last_import=-1; + memcpy((char*)&edbImport+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + + while(ptr) + { + if(edbImport.m_ImportID != last_import) + { + if(import_count+1m_DB->MoveNext(&err); + if(err) + { + LogString("Error on MoveNext"); + return MC_ERR_CORRUPTED; + } + if(ptr) + { + memcpy((char*)&edbImport+m_Database->m_KeyOffset,ptr,m_Database->m_TotalSize); + edbImport.SwapPosBytes(); + } + if(edbImport.m_RowType != MC_TET_IMPORT) + { + ptr=NULL; + } + } + if(last_import<0) + { + LogString("Initialize: Entity lists not found"); + return MC_ERR_CORRUPTED; + } + if(m_Imports[0].m_Block != m_DBStat.m_Block) + { + LogString("Initialize: Block count mismatch"); + return MC_ERR_CORRUPTED; + } + } + else + { + edbRow.Zero(); + err=m_Database->m_DB->Write((char*)&edbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbRow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + return err; + } + + m_DBStat.Zero(); + if(m_Mode == MC_WMD_AUTO) + { + m_Mode=MC_WMD_TXS | MC_WMD_ADDRESS_TXS; + } + m_DBStat.m_InitMode=m_Mode & MC_WMD_MODE_MASK; + err=m_Database->m_DB->Write((char*)&m_DBStat+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&m_DBStat+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + return err; + } + + m_Imports[0].Init(0,-1); // Chain import + + edbImport.Zero(); + edbImport.m_ImportID=0; + edbImport.m_Block=m_Imports[0].m_Block; + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Write((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbImport+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + return err; + } + + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + return err; + } + } + + m_MemPools[0]=new mc_Buffer; // Key - entity with m_Pos set to 0 + txid + err=m_MemPools[0]->Initialize(MC_TDB_ENTITY_KEY_SIZE+MC_TDB_TXID_SIZE,m_Database->m_TotalSize,MC_BUF_MODE_MAP); + + m_RawMemPools[0]=new mc_Buffer; + err=m_RawMemPools[0]->Initialize(MC_TDB_TXID_SIZE,m_Database->m_TotalSize,MC_BUF_MODE_MAP); + + m_RawUpdatePool=new mc_Buffer; + err=m_RawUpdatePool->Initialize(MC_TDB_TXID_SIZE,m_Database->m_TotalSize,MC_BUF_MODE_MAP); + + Dump("Initialize"); + sprintf(msg, "Initialized. Chain height: %d, Txs: %d",m_DBStat.m_Block,m_DBStat.m_Count); + LogString(msg); + + for(i=1;im_Entities) + { + sprintf(msg, "Initialization, Dropping import %d",(m_Imports+i)->m_ImportID); + LogString(msg); + DropImport(m_Imports+i); + } + } + + + return err; +} + +int mc_TxDB::Destroy() +{ + int i; + + if(m_Database) + { + delete m_Database; + } + + for(i=0;im_DB->Read((char*)&edbRow+m_Database->m_KeyOffset,m_Database->m_KeySize,&dbvalue_len,MC_OPT_DB_DATABASE_SEEK_ON_READ,&err); + if(err) + { + return; + } + + if(ptr) + { + memcpy((char*)&edbRow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + while(ptr) + { + mc_MemoryDumpCharSizeToFile(fHan,(char*)&edbRow+m_Database->m_KeyOffset,0,m_Database->m_TotalSize,m_Database->m_TotalSize); + ptr=(unsigned char*)m_Database->m_DB->MoveNext(&err); + if(ptr) + { + memcpy((char*)&edbRow+m_Database->m_KeyOffset,ptr,m_Database->m_TotalSize); + } + } + } + + for(i=0;iGetCount()) + { + fprintf(fHan,"RawMemPool %d\n",m_Imports[i].m_ImportID); + mc_MemoryDumpCharSizeToFile(fHan,m_RawMemPools[i]->GetRow(0),0,m_RawMemPools[i]->GetCount()*m_Database->m_TotalSize,m_Database->m_TotalSize); + } + } + if(m_MemPools[i]) + { + if(m_MemPools[i]->GetCount()) + { + fprintf(fHan,"MemPool %d\n",m_Imports[i].m_ImportID); + mc_MemoryDumpCharSizeToFile(fHan,m_MemPools[i]->GetRow(0),0,m_MemPools[i]->GetCount()*m_Database->m_TotalSize,m_Database->m_TotalSize); + } + } + } + if(m_RawUpdatePool) + { + if(m_RawUpdatePool->GetCount()) + { + fprintf(fHan,"RawUpdatePool\n"); + mc_MemoryDumpCharSizeToFile(fHan,m_RawUpdatePool->GetRow(0),0,m_RawUpdatePool->GetCount()*m_Database->m_TotalSize,m_Database->m_TotalSize); + } + } + + fprintf(fHan,"\n<<<<<< \tChain height: %6d\t%s\n\n",m_DBStat.m_Block,message); + fclose(fHan); +} + +int mc_TxDB::AddToFile(const unsigned char *tx, + uint32_t txsize, + uint32_t fileid, + uint32_t offset) +{ + char FileName[MC_DCT_DB_MAX_PATH]; + int FileHan,err; + sprintf(FileName,"%s%05u.dat",m_LobFileNamePrefix,fileid); + + err=MC_ERR_NOERROR; + + FileHan=open(FileName,_O_BINARY | O_RDWR | O_CREAT, S_IRUSR | S_IWUSR); + + if(FileHan<=0) + { + return MC_ERR_INTERNAL_ERROR; + } + + if(lseek64(FileHan,offset,SEEK_SET) != offset) + { + err=MC_ERR_INTERNAL_ERROR; + goto exitlbl; + } + + if(write(FileHan,tx,txsize) != txsize) + { + err=MC_ERR_INTERNAL_ERROR; + return MC_ERR_INTERNAL_ERROR; + } + +exitlbl: + + close(FileHan); + return err; +} + +/* + * Adds entity to chain import + */ + +int mc_TxDB::AddEntity(mc_TxEntity *entity,uint32_t flags) +{ + return AddEntity(m_Imports,entity,flags); +} + +int mc_TxDB::AddEntity(mc_TxImport *import,mc_TxEntity *entity,uint32_t flags) +{ + int err; + char msg[256]; + char enthex[65]; + mc_TxEntityStat stat; + + + mc_TxImportRow edbImport; + + err=MC_ERR_NOERROR; + + if(import->FindEntity(entity) < 0) + { + stat.Zero(); + memcpy(&(stat.m_Entity), entity,sizeof(mc_TxEntity)); + stat.m_Generation=0; + stat.m_LastPos=0; + stat.m_LastClearedPos=0; + stat.m_PosInImport=1; + if(import->m_Entities->GetCount()) + { + stat.m_PosInImport=((mc_TxEntityStat*)import->m_Entities->GetRow(import->m_Entities->GetCount()-1))->m_PosInImport+1; + } + stat.m_LastImportedBlock=m_DBStat.m_Block; // Block the entity was added on, relevant if out-of-sync + stat.m_TimeAdded=mc_TimeNowAsUInt(); + stat.m_Flags=flags; // Mainly for out-of-sync flag + + edbImport.Zero(); + memcpy(&(edbImport.m_Entity), entity,sizeof(mc_TxEntity)); + edbImport.m_Generation=0; + edbImport.m_LastPos=0; + edbImport.m_Pos=stat.m_PosInImport; + edbImport.m_LastImportedBlock=stat.m_LastImportedBlock; + edbImport.m_TimeAdded=stat.m_TimeAdded; + edbImport.m_Flags=stat.m_Flags; + + edbImport.m_Block=m_DBStat.m_Block; + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Write((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbImport+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + goto exitlbl; + } + + import->AddEntity(&stat); + } + else + { + err=MC_ERR_FOUND; + } + + + +exitlbl: + + sprintf_hex(enthex,entity->m_EntityID,MC_TDB_ENTITY_ID_SIZE); + + if(err) + { + if(err == MC_ERR_FOUND) + { + sprintf(msg,"Entity (%08X, %s) already in the wallet",entity->m_EntityType,enthex); + LogString(msg); + } + else + { + sprintf(msg,"Could not add entity (%08X, %s), error: %d",entity->m_EntityType,enthex,err); + LogString(msg); + } + } + else + { + sprintf(msg,"Entity (%08X, %s) added successfully",entity->m_EntityType,enthex); + LogString(msg); + } + + return err; +} + +int mc_TxDB::FindEntity(mc_TxImport *import, + mc_TxEntity *entity) +{ + mc_TxImport *imp; + imp=m_Imports; + if(import) // Find import + { + imp=import; + } + + return imp->FindEntity(entity); +} + +int mc_TxDB::FindEntity(mc_TxImport *import,mc_TxEntityStat *entity) +{ + mc_TxImport *imp; + int row; + imp=m_Imports; + if(import) // Find import + { + imp=import; + } + + row=imp->FindEntity(entity); + if(row >= 0) + { + memcpy(entity,imp->GetEntity(row),sizeof(mc_TxEntityStat)); + } + return row; +} + +int mc_TxDB::AddSubKeyDef( + mc_TxImport *import, + const unsigned char *hash, + const unsigned char *subkey, + uint32_t subkeysize, + uint32_t flags) +{ + mc_TxDefRow txdef; + mc_Buffer *rawmempool; + int err,size; + uint32_t LastFileID,LastFileSize; + char msg[256]; + + rawmempool=m_RawMemPools[0]; + if(import) // Find import + { + rawmempool=m_RawMemPools[import-m_Imports]; + } + + err=GetTx(&txdef,hash); + + if(err == MC_ERR_NOT_FOUND) // SubKey is not found, neither on disk, nor in the mempool + { + err=MC_ERR_NOERROR; + + LastFileID=m_DBStat.m_LastFileID; + LastFileSize=m_DBStat.m_LastFileSize; + + size=mc_AllocSize(subkeysize,m_Database->m_TotalSize,1); // Size SubKey takes in the file (padded) + if(subkeysize) + { + if(LastFileSize+size>MC_TDB_MAX_TXS_FILE_SIZE) // New file is needed + { + LastFileID+=1; + LastFileSize=0; + } + + err=AddToFile(subkey,subkeysize,LastFileID,LastFileSize); + if(err) + { + sprintf(msg,"Couldn't store key in file, error: %d",err); + LogString(msg); + return err; + } + } + + txdef.Zero(); + memcpy(txdef.m_TxId,hash, MC_TDB_TXID_SIZE); + txdef.m_Size=subkeysize; + txdef.m_FullSize=subkeysize; + txdef.m_InternalFileID=LastFileID; + txdef.m_InternalFileOffset=LastFileSize; + txdef.m_Block=-1; + txdef.m_BlockFileID=-1; + txdef.m_TimeReceived=mc_TimeNowAsUInt(); + txdef.m_Flags=flags | ( (subkey != NULL) ? MC_SFL_NONE : MC_SFL_NODATA ); + LastFileSize+=size; + m_DBStat.m_LastFileID=LastFileID; // Even if commit is unsuccessful we'll lose some place. On restart we return to previous values + m_DBStat.m_LastFileSize=LastFileSize; + rawmempool->Add(&txdef,(unsigned char*)&txdef+MC_TDB_TXID_SIZE); + } + + return err; +} + +int mc_TxDB::DecrementSubKey( + mc_TxImport *import, + mc_TxEntity *parent_entity, + mc_TxEntity *entity) +{ + mc_TxImport *imp; + mc_Buffer *mempool; + mc_TxEntityStat *stat; + mc_TxEntityRow erow; + + int err; + int mprow,last_pos; + char msg[256]; + int value_len; + unsigned char *ptr; + + err=MC_ERR_NOERROR; + + imp=m_Imports; + mempool=m_MemPools[0]; + if(import) // Find import + { + imp=import; + mempool=m_MemPools[import-m_Imports]; + } + + stat=imp->GetEntity(imp->FindEntity(parent_entity)); + if(stat == NULL) + { + LogString("Could not find parent entity"); + err=MC_ERR_INTERNAL_ERROR; + goto exitlbl; + } + + erow.Zero(); // Decrement row in mempool + memcpy(&erow.m_Entity,entity,sizeof(mc_TxEntity)); + erow.m_Pos=1; + erow.m_Generation=stat->m_Generation; + + + last_pos=0; + mprow=mempool->Seek(&erow); + if(mprow >= 0) + { + last_pos=((mc_TxEntityRow*)(mempool->GetRow(mprow)))->m_LastSubKeyPos; + ((mc_TxEntityRow*)(mempool->GetRow(mprow)))->m_LastSubKeyPos=last_pos-1; + } + else + { + erow.Zero(); + memcpy(&erow.m_Entity,entity,sizeof(mc_TxEntity)); + erow.m_Generation=stat->m_Generation; + erow.m_Pos=1; + erow.SwapPosBytes(); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + erow.SwapPosBytes(); + if(err) + { + sprintf(msg,"Error while reading subkey entry from database: %d",err); + LogString(msg); + err=MC_ERR_INTERNAL_ERROR; + goto exitlbl; + } + if(ptr) + { + memcpy((char*)&erow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + last_pos=erow.m_LastSubKeyPos; + } + erow.Zero(); + memcpy(&erow.m_Entity,entity,sizeof(mc_TxEntity)); + erow.m_Generation=stat->m_Generation; + erow.m_Pos=1; + erow.m_Flags=last_pos; + erow.m_LastSubKeyPos=last_pos-1; + mempool->Add(&erow,(unsigned char*)&erow+MC_TDB_ENTITY_KEY_SIZE+MC_TDB_TXID_SIZE); + } +exitlbl: + + return err; +} + +int mc_TxDB::IncrementSubKey( + mc_TxImport *import, + mc_TxEntity *parent_entity, + mc_TxEntity *entity, + const unsigned char *subkey_hash, + const unsigned char *tx_hash, + int block, + uint32_t flags, + int newtx) +{ + mc_TxImport *imp; + mc_Buffer *mempool; + mc_TxEntityStat *stat; + mc_TxEntityRow erow; + int err; + int mprow,last_pos; + char msg[256]; + int value_len; + unsigned char *ptr; + + err=MC_ERR_NOERROR; + + imp=m_Imports; + mempool=m_MemPools[0]; + if(import) // Find import + { + imp=import; + mempool=m_MemPools[import-m_Imports]; + } + + stat=imp->GetEntity(imp->FindEntity(parent_entity)); + if(stat == NULL) + { + LogString("Could not find parent entity"); + err=MC_ERR_INTERNAL_ERROR; + goto exitlbl; + } + + if((newtx != 0) || // Row is new - add always + ( (imp->m_ImportID > 0) && + ( (stat->m_Flags & MC_EFL_NOT_IN_SYNC) != 0 ) ) || // All tx entities rows are new, except those in sync - like old addresses/wallet by timereceived + ((parent_entity->m_EntityType & MC_TET_ORDERMASK) != MC_TET_TIMERECEIVED))// Ordered by chain position - add always + { + erow.Zero(); // Anchor row in mempool, will not be stored in database + memcpy(&erow.m_Entity,entity,sizeof(mc_TxEntity)); + erow.m_Generation=stat->m_Generation; + + + last_pos=0; + mprow=mempool->Seek(&erow); + if(mprow >= 0) + { + mprow+=1; // Anchor doesn't carry info except key, but the next row is the first for this entity + last_pos=((mc_TxEntityRow*)(mempool->GetRow(mprow)))->m_LastSubKeyPos; + ((mc_TxEntityRow*)(mempool->GetRow(mprow)))->m_LastSubKeyPos=last_pos+1; + } + else + { + erow.Zero(); + memcpy(&erow.m_Entity,entity,sizeof(mc_TxEntity)); + erow.m_Generation=stat->m_Generation; + erow.m_Pos=1; + erow.SwapPosBytes(); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + erow.SwapPosBytes(); + if(err) + { + sprintf(msg,"Error while reading subkey entry from database: %d",err); + LogString(msg); + err=MC_ERR_INTERNAL_ERROR; + goto exitlbl; + } + if(ptr) + { + memcpy((char*)&erow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + last_pos=erow.m_LastSubKeyPos; + } + erow.Zero(); + memcpy(&erow.m_Entity,entity,sizeof(mc_TxEntity)); + erow.m_Generation=stat->m_Generation; + erow.m_LastSubKeyPos=last_pos+1; + mempool->Add(&erow,(unsigned char*)&erow+MC_TDB_ENTITY_KEY_SIZE+MC_TDB_TXID_SIZE); + } + + erow.Zero(); + memcpy(&erow.m_Entity,entity,sizeof(mc_TxEntity)); + erow.m_Generation=stat->m_Generation; + memcpy(erow.m_TxId,tx_hash,MC_TDB_TXID_SIZE); + erow.m_LastSubKeyPos=last_pos+1; + erow.m_TempPos=last_pos+1; + erow.m_Block=block; + erow.m_Flags=flags; + mempool->Add(&erow,(unsigned char*)&erow+MC_TDB_ENTITY_KEY_SIZE+MC_TDB_TXID_SIZE); + + if(last_pos == 0) + { + erow.Zero(); + memcpy(&erow.m_Entity,&stat->m_Entity,sizeof(mc_TxEntity)); + erow.m_Generation=stat->m_Generation; + memcpy(erow.m_TxId,subkey_hash,MC_TDB_ENTITY_ID_SIZE); + erow.m_Block=block; + erow.m_Flags=flags; + stat->m_LastPos+=1; + erow.m_TempPos=stat->m_LastPos; // Will be copied to m_LastPos on commit. m_LastPos=0 to allow Seek() above + mempool->Add(&erow,(unsigned char*)&erow+MC_TDB_ENTITY_KEY_SIZE+MC_TDB_TXID_SIZE); + } + } + + +exitlbl: + + return err; +} + +/* + * Adds tx to specific import + */ + +int mc_TxDB::AddTx(mc_TxImport *import, + const unsigned char *hash, + const unsigned char *tx, + uint32_t txsize, + uint32_t txfullsize, + int block, + int block_file, + uint32_t block_offset, + uint32_t block_tx_offset, + uint32_t block_tx_index, + uint32_t flags, + uint32_t timestamp, + mc_Buffer *entities) +{ + int err; + char txhex[65]; + char txtype[16]; + char msg[256]; + + int newtx,duplicate,isrelevant,ondisk,i,mprow,size; + uint32_t LastFileID,LastFileSize; + mc_TxImport *imp; + mc_Buffer *mempool; + mc_Buffer *rawmempool; + mc_TxEntityStat *stat; + mc_TxEntityRow erow; + mc_TxDefRow txdef; + mc_TxDefRow *lptxdef; + + err=MC_ERR_NOERROR; + + sprintf_hex(txhex,hash,MC_TDB_TXID_SIZE); + + imp=m_Imports; + mempool=m_MemPools[0]; + rawmempool=m_RawMemPools[0]; + if(import) // Find import + { + imp=import; + mempool=m_MemPools[import-m_Imports]; + rawmempool=m_RawMemPools[import-m_Imports]; + } + + isrelevant=0; + if(imp->m_ImportID) // Takes tx only if it has relevant entity for this import + { + for(i=0;iGetCount();i++) + { + if(isrelevant == 0) + { + if(imp->FindEntity((mc_TxEntity*)entities->GetRow(i)) >= 0) + { + isrelevant=1; + } + } + } + } + else + { + isrelevant=1; // Chain import updates its entity list + } + + if(isrelevant == 0) + { + sprintf(msg,"Tx %s ignored for import %d",txhex,imp->m_ImportID); + LogString(msg); + goto exitlbl; + } + + newtx=0; + ondisk=0; + err=GetTx(&txdef,hash); + if(err == MC_ERR_NOT_FOUND) // Tx is not found, neither on disk, nor in the mempool + { + err=MC_ERR_NOERROR; + newtx=1; + + size=mc_AllocSize(txsize,m_Database->m_TotalSize,1); // Size tx takes in the file (padded) + LastFileID=m_DBStat.m_LastFileID; + LastFileSize=m_DBStat.m_LastFileSize; + + if(LastFileSize+size>MC_TDB_MAX_TXS_FILE_SIZE) // New file is needed + { + LastFileID+=1; + LastFileSize=0; + } + + err=AddToFile(tx,txsize,LastFileID,LastFileSize); + if(err) + { + sprintf(msg,"Couldn't store tx %s in file, error: %d",txhex,err); + LogString(msg); + goto exitlbl; + } + + txdef.Zero(); + memcpy(txdef.m_TxId,hash, MC_TDB_TXID_SIZE); + txdef.m_Size=txsize; + txdef.m_FullSize=txfullsize; + txdef.m_InternalFileID=LastFileID; + txdef.m_InternalFileOffset=LastFileSize; + txdef.m_Block=block; + if(block >= 0) // Set block coordinates only if processed in the block + { + txdef.m_BlockFileID=block_file; + txdef.m_BlockOffset=block_offset; + txdef.m_BlockTxOffset=block_tx_offset; + txdef.m_BlockIndex=block_tx_index; + } + else + { + txdef.m_BlockFileID=-1; + } + txdef.m_TimeReceived=timestamp; + txdef.m_Flags=flags; + LastFileSize+=size; + m_DBStat.m_LastFileID=LastFileID; // Even if commit is unsuccessful we'll lose some place. On restart we return to previous values + m_DBStat.m_LastFileSize=LastFileSize; + rawmempool->Add(&txdef,(unsigned char*)&txdef+MC_TDB_TXID_SIZE); + } + else + { + if(err) + { + sprintf(msg,"Internal error while looking for tx %s in raw database, error: %d",txhex,err); + LogString(msg); + goto exitlbl; + } + ondisk=1; + mprow=rawmempool->Seek((unsigned char*)hash); + if(mprow >= 0) // Found in mempool + { + ondisk=0; + lptxdef=(mc_TxDefRow *)rawmempool->GetRow(mprow); + lptxdef->m_Block=block; // If on disk, block is not updated. In case in case of rollback, on-disk value will be higher + // than chain height -considered as mempool. + lptxdef->m_Flags=flags; + if(block >= 0) // Update block pos only if processed in the block + // If on disk, no matter what is going on here, old reference is good + { + lptxdef->m_BlockFileID=block_file; + lptxdef->m_BlockOffset=block_offset; + lptxdef->m_BlockTxOffset=block_tx_offset; + lptxdef->m_BlockIndex=block_tx_index; + } + else + { + lptxdef->m_BlockFileID=-1; + } + } + } + + duplicate=1; + for(i=0;iGetCount();i++) // Processing entities + { + isrelevant=0; + if(imp->FindEntity((mc_TxEntity*)entities->GetRow(i)) >= 0) + { + isrelevant=1; + } + else + { +// if(imp->m_ImportID == 0) + if(false) // Disabled. Fully synced addresses should be added by higher level + // Using AddEntity call + { + imp->AddEntity((mc_TxEntity*)entities->GetRow(i)); // New entities added to chain import + isrelevant=1; + } + } + + if(isrelevant) + { + stat=imp->GetEntity(imp->FindEntity((mc_TxEntity*)entities->GetRow(i))); + if(stat == NULL) + { + sprintf(msg,"Could not add tx %s, entity not found",txhex); + LogString(msg); + err=MC_ERR_INTERNAL_ERROR; + goto exitlbl; + } + + erow.Zero(); + memcpy(&erow.m_Entity,&stat->m_Entity,sizeof(mc_TxEntity)); + erow.m_Generation=stat->m_Generation; + memcpy(erow.m_TxId,hash,MC_TDB_TXID_SIZE); + mprow=mempool->Seek(&erow); + if(mprow >= 0) // Update block and flags if found in mempool + { + ((mc_TxEntityRow*)(mempool->GetRow(mprow)))->m_Block=block; + ((mc_TxEntityRow*)(mempool->GetRow(mprow)))->m_Flags=flags; + } + else + { + if((newtx != 0) || // Row is new - add always + ( (imp->m_ImportID > 0) && + ( (stat->m_Flags & MC_EFL_NOT_IN_SYNC) != 0 ) ) || // All tx entities rows are new, except those in sync - like old addresses/wallet by timereceived + ((erow.m_Entity.m_EntityType & MC_TET_ORDERMASK) != MC_TET_TIMERECEIVED)) // Ordered by chain position - add always + { + erow.m_Block=block; + erow.m_Flags=flags; + stat->m_LastPos+=1; + erow.m_TempPos=stat->m_LastPos; // Will be copied to m_LastPos on commit. m_LastPos=0 to allow Seek() above + mempool->Add(&erow,(unsigned char*)&erow+MC_TDB_ENTITY_KEY_SIZE+MC_TDB_TXID_SIZE); + duplicate=0; + } + } + } + } + + if(imp->m_ImportID == 0) // Update block and flags for txs already on disk + { + if(newtx == 0) + { + if(ondisk) + { + txdef.m_Block=block; + if(block >= 0) // Don't erase old block coordinates + { + txdef.m_BlockFileID=block_file; + txdef.m_BlockOffset=block_offset; + txdef.m_BlockTxOffset=block_tx_offset; + txdef.m_BlockIndex=block_tx_index; + } + txdef.m_Flags=flags; + mprow=m_RawUpdatePool->Seek((unsigned char*)hash); + if(mprow >= 0) + { + memcpy((unsigned char*)m_RawUpdatePool->GetRow(mprow)+MC_TDB_TXID_SIZE,(unsigned char*)&txdef+MC_TDB_TXID_SIZE,m_Database->m_ValueSize); + } + else + { + m_RawUpdatePool->Add((unsigned char *)&txdef,(unsigned char*)&txdef+MC_TDB_TXID_SIZE); + } + } + } + } + + if(newtx) + { + sprintf(txtype,"New"); + } + else + { + if(duplicate) + { + sprintf(txtype,"Duplicate"); + } + else + { + sprintf(txtype,"Update"); + } + } + + sprintf(msg,"AddTx %s, block %d, flags %08X, import %d, %s",txhex,block,flags,imp->m_ImportID,txtype); + LogString(msg); +exitlbl: + return err; +} + +/* + * Saves flag (normally "invalid") for specific transaction, + * If invalid transaction becomes valid again this flag will be overwritten by AddTx + */ + + +int mc_TxDB::SaveTxFlag(const unsigned char *hash,uint32_t flag,int set_flag) +{ + int err,value_len,mprow; + unsigned char *ptr; + char txhex[65]; + char msg[256]; + mc_TxDefRow *txdef; + mc_TxDefRow StoredTxDef; + err=MC_ERR_NOERROR; + + sprintf_hex(txhex,hash,MC_TDB_TXID_SIZE); + + if(set_flag) + { + sprintf(msg,"Setting flag %08X to tx %s",flag,txhex); + } + else + { + sprintf(msg,"Unsetting flag %08X to tx %s",flag,txhex); + } + LogString(msg); + + mprow=m_RawMemPools[0]->Seek((unsigned char*)hash); + if(mprow >= 0) + { + txdef=(mc_TxDefRow *)m_RawMemPools[0]->GetRow(mprow); + if(set_flag) + { + txdef->m_Flags |= flag; + } + else + { + txdef->m_Flags &= ~flag; + } + return MC_ERR_NOERROR; + } + + mprow=m_RawUpdatePool->Seek((unsigned char*)hash); + if(mprow >= 0) + { + txdef=(mc_TxDefRow *)m_RawUpdatePool->GetRow(mprow); + if(set_flag) + { + txdef->m_Flags |= flag; + } + else + { + txdef->m_Flags &= ~flag; + } + } + + StoredTxDef.Zero(); + + memcpy(StoredTxDef.m_TxId,hash, MC_TDB_TXID_SIZE); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&StoredTxDef+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(err) + { + return err; + } + + if(ptr) + { + memcpy((char*)&StoredTxDef+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + if(set_flag) + { + StoredTxDef.m_Flags |= flag; + } + else + { + StoredTxDef.m_Flags &= ~flag; + } + } + else + { + return MC_ERR_NOT_FOUND; + } + + err=m_Database->m_DB->Write((char*)&StoredTxDef+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&StoredTxDef+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + return err; + } + + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + return err; + } + + return MC_ERR_NOERROR; +} + + +int mc_TxDB::GetTx(mc_TxDefRow *txdef, + const unsigned char *hash) +{ + int err,value_len,mprow; + unsigned char *ptr; + + txdef->Zero(); + + err=MC_ERR_NOERROR; + + mprow=m_RawMemPools[0]->Seek((unsigned char*)hash); + if(mprow >= 0) + { + memcpy(txdef,(mc_TxDefRow *)m_RawMemPools[0]->GetRow(mprow),sizeof(mc_TxDefRow)); + if(txdef->m_Block > m_DBStat.m_Block) + { + txdef->m_Block=-1; + } + return MC_ERR_NOERROR; + } + + mprow=m_RawUpdatePool->Seek((unsigned char*)hash); + if(mprow >= 0) + { + memcpy(txdef,(mc_TxDefRow *)m_RawUpdatePool->GetRow(mprow),sizeof(mc_TxDefRow)); + if(txdef->m_Block > m_DBStat.m_Block) + { + txdef->m_Block=-1; + } + return MC_ERR_NOERROR; + } + + memcpy(txdef->m_TxId,hash, MC_TDB_TXID_SIZE); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)txdef+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(err) + { + return err; + } + + if(ptr) + { + memcpy((char*)txdef+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + if(txdef->m_Block > m_DBStat.m_Block) // On-disk records are not updated on rollback + { + txdef->m_Block=-1; + } + } + else + { + err=MC_ERR_NOT_FOUND; + } + + + return err; + +} + +int mc_TxDB::BeforeCommit(mc_TxImport *import) +{ + int err,i,j; + mc_TxImport *imp; + mc_Buffer *mempool; + mc_TxEntityStat *stat; + mc_TxEntityRow *lperow; + + err=MC_ERR_NOERROR; + + Dump("Before BeforeCommit"); + + imp=m_Imports; + mempool=m_MemPools[0]; + if(import) + { + imp=import; + mempool=m_MemPools[import-m_Imports]; + } + + if(imp->m_ImportID) + { + return err; + } + + for(i=0;im_Entities->GetCount();i++) // All not MC_TET_TIMERECEIVED are removed from mempool + { + stat=(mc_TxEntityStat*)imp->m_Entities->GetRow(i); + if((stat->m_Entity.m_EntityType & MC_TET_ORDERMASK) != MC_TET_TIMERECEIVED) + { + stat->m_LastPos=stat->m_LastClearedPos; + } + } + + j=0; + for(i=0;iGetCount();i++) + { + lperow=(mc_TxEntityRow*)mempool->GetRow(i); + if((lperow->m_Entity.m_EntityType & MC_TET_ORDERMASK) == MC_TET_TIMERECEIVED) + { + memcpy(mempool->GetRow(j),mempool->GetRow(i),sizeof(mc_TxEntityRow)); + j++; + } + } + mempool->SetCount(j); + Dump("After BeforeCommit"); + + return err; +} + +int mc_TxDB::Commit(mc_TxImport *import) +{ + int err; + char msg[256]; + + int i,block; + mc_TxImport *imp; + mc_Buffer *mempool; + mc_Buffer *rawmempool; + mc_TxEntityStat *stat; + mc_TxEntityRow *lperow; + mc_TxDefRow *lptxdef; + mc_TxEntityRow erow; + mc_TxImportRow edbImport; + int anchor; + int value_len; + unsigned char *ptr; + + + Dump("Before Commit"); + err=MC_ERR_NOERROR; + + + imp=m_Imports; + mempool=m_MemPools[0]; + rawmempool=m_RawMemPools[0]; + if(import) + { + imp=import; + mempool=m_MemPools[import-m_Imports]; + rawmempool=m_RawMemPools[import-m_Imports]; + } + block=imp->m_Block+1; + + sprintf(msg,"Committing block %d, New Txs: %d, New Rows: %d",block,rawmempool->GetCount(),mempool->GetCount()); + LogString(msg); + + for(i=0;iGetCount();i++) // New raw transactions + { + lptxdef=(mc_TxDefRow *)rawmempool->GetRow(i); + m_DBStat.m_Count+=1; + m_DBStat.m_FullSize+=mc_AllocSize(lptxdef->m_Size,m_Database->m_TotalSize,1); + err=m_Database->m_DB->Write((char*)lptxdef+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)lptxdef+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + goto exitlbl; + } + } + + if(imp->m_ImportID == 0) // Raw transaction updates + { + for(i=0;iGetCount();i++) + { + lptxdef=(mc_TxDefRow *)m_RawUpdatePool->GetRow(i); + m_DBStat.m_FullSize+=mc_AllocSize(lptxdef->m_Size,m_Database->m_TotalSize,1); + err=m_Database->m_DB->Write((char*)lptxdef+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)lptxdef+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + goto exitlbl; + } + } + } + + anchor=0; + for(i=0;iGetCount();i++) // mempool items + { + lperow=(mc_TxEntityRow *)mempool->GetRow(i); + if((lperow->m_Entity.m_EntityType & MC_TET_SUBKEY) && (lperow->m_TempPos == 0)) + { + anchor=1; + } + else + { + if(anchor) + { + anchor=0; + if(lperow->m_TempPos>1) + { + erow.Zero(); + memcpy(&erow.m_Entity,&(lperow->m_Entity),sizeof(mc_TxEntity)); + erow.m_Generation=lperow->m_Generation; + erow.m_Pos=1; + erow.SwapPosBytes(); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + erow.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + if(ptr == NULL) + { + err=MC_ERR_CORRUPTED; + goto exitlbl; + } + memcpy((char*)&erow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + erow.m_LastSubKeyPos=lperow->m_LastSubKeyPos; + erow.SwapPosBytes(); + err=m_Database->m_DB->Write((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&erow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + erow.SwapPosBytes(); + } + } + lperow->m_Pos=lperow->m_TempPos; // m_Pos in mempool was 0 - to support search by TxID in mempool + lperow->m_TempPos=0; + lperow->SwapPosBytes(); + err=m_Database->m_DB->Write((char*)lperow+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)lperow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + lperow->SwapPosBytes(); + if(err) + { + goto exitlbl; + } + } + } + + edbImport.Zero(); // Import block update + edbImport.m_Block=block; + edbImport.m_ImportID=imp->m_ImportID; + imp->m_Block=block; + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Write((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbImport+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + + for(i=0;im_Entities->GetCount();i++) // Import entity update + { + stat=(mc_TxEntityStat*)imp->m_Entities->GetRow(i); + if(stat->m_LastPos != stat->m_LastClearedPos) // Entity has new items + { + edbImport.Zero(); + memcpy(&(edbImport.m_Entity), &(stat->m_Entity),sizeof(mc_TxEntity)); + stat->m_LastClearedPos=stat->m_LastPos; + edbImport.m_ImportID=imp->m_ImportID; + edbImport.m_Generation=stat->m_Generation; + edbImport.m_LastPos=stat->m_LastPos; + edbImport.m_Pos=stat->m_PosInImport; + edbImport.m_LastImportedBlock=stat->m_LastImportedBlock; + edbImport.m_TimeAdded=stat->m_TimeAdded; + edbImport.m_Flags=stat->m_Flags; + edbImport.m_Block=block; + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Write((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbImport+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + } + } + + if(imp->m_ImportID == 0) + { + m_DBStat.m_Block=block; // DB stats + err=m_Database->m_DB->Write((char*)&m_DBStat+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&m_DBStat+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + goto exitlbl; + } + } + + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + goto exitlbl; + } + + +exitlbl: + + if(err) + { + sprintf(msg,"Could not commit block %d, error: %d",block,err); + LogString(msg); + RollBack(import,block-1); + } + else + { + mempool->Clear(); + rawmempool->Clear(); + m_RawUpdatePool->Clear(); + sprintf(msg,"Block %d committed successfully",block); + LogString(msg); + } + Dump("After Commit"); + + return err; +} + +int mc_TxDB::RollBack(mc_TxImport *import,int block) +{ + int value_len; + unsigned char *ptr; + int err; + char msg[256]; + + int i,j; + mc_TxImport *imp; + mc_Buffer *mempool; + mc_TxEntityStat *stat; + mc_TxEntityRow *lperow; + mc_TxImportRow edbImport; + mc_TxEntityRow erow; + uint32_t pos; + + err=MC_ERR_NOERROR; + + Dump("Before RollBack"); + imp=m_Imports; + mempool=m_MemPools[0]; + if(import) + { + imp=import; + mempool=m_MemPools[import-m_Imports]; + } + + for(i=0;iGetCount();i++) // removing decremented subkey rows + { + lperow=(mc_TxEntityRow *)mempool->GetRow(i); + if((lperow->m_Entity.m_EntityType & MC_TET_SUBKEY) && (lperow->m_TempPos == 0) && (lperow->m_Pos == 1)) + { + + for(j=lperow->m_LastSubKeyPos;j<(int)lperow->m_Flags;j++) // old count is carried by flags field of anchor row + { + erow.Zero(); + memcpy(&erow.m_Entity,&(lperow->m_Entity),sizeof(mc_TxEntity)); + erow.m_Generation=lperow->m_Generation; + erow.m_Pos=j+1; + erow.SwapPosBytes(); + err=m_Database->m_DB->Delete((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + erow.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + } + if(lperow->m_LastSubKeyPos) // Updating first subkey row + { + erow.Zero(); + memcpy(&erow.m_Entity,&(lperow->m_Entity),sizeof(mc_TxEntity)); + erow.m_Generation=lperow->m_Generation; + erow.m_Pos=1; + erow.SwapPosBytes(); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + erow.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + if(ptr == NULL) + { + err=MC_ERR_CORRUPTED; + goto exitlbl; + } + memcpy((char*)&erow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + erow.m_LastSubKeyPos=lperow->m_LastSubKeyPos; + erow.SwapPosBytes(); + err=m_Database->m_DB->Write((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&erow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + erow.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + } + } + } + + + j=0; // Clearing mempool transactions except MC_TET_TIMERECEIVED + for(i=0;iGetCount();i++) + { + lperow=(mc_TxEntityRow*)mempool->GetRow(i); + if((lperow->m_Entity.m_EntityType & MC_TET_ORDERMASK) == MC_TET_TIMERECEIVED) + { + memcpy(mempool->GetRow(j),mempool->GetRow(i),sizeof(mc_TxEntityRow)); + j++; + } + } + mempool->SetCount(j); + if(imp->m_ImportID == 0) + { + m_RawUpdatePool->Clear(); + } + + for(i=0;im_Entities->GetCount();i++) // Removing ordered-by-blockchain-position rows + { + stat=(mc_TxEntityStat*)imp->m_Entities->GetRow(i); + if((stat->m_Entity.m_EntityType & MC_TET_ORDERMASK) != MC_TET_TIMERECEIVED) + { + stat->m_LastPos=stat->m_LastClearedPos; + pos=stat->m_LastPos; + erow.Zero(); + memcpy(&erow.m_Entity,&(stat->m_Entity),sizeof(mc_TxEntity)); + erow.m_Generation=stat->m_Generation; + while(pos) + { + erow.m_Pos=pos; + erow.SwapPosBytes(); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + erow.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + + if(ptr) + { + memcpy((char*)&erow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + if(erow.m_Block > block) + { + erow.SwapPosBytes(); + err=m_Database->m_DB->Delete((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + erow.SwapPosBytes(); + pos--; + stat->m_LastPos=pos; + } + else + { + pos=0; + } + } + else + { + err=MC_ERR_NOT_FOUND; + goto exitlbl; + } + } + + edbImport.Zero(); // Updating import entity stats + memcpy(&(edbImport.m_Entity), &(stat->m_Entity),sizeof(mc_TxEntity)); + edbImport.m_ImportID=imp->m_ImportID; + stat->m_LastClearedPos=stat->m_LastPos; + edbImport.m_Generation=stat->m_Generation; + edbImport.m_LastPos=stat->m_LastPos; + edbImport.m_Pos=stat->m_PosInImport; + edbImport.m_LastImportedBlock=stat->m_LastImportedBlock; + edbImport.m_TimeAdded=stat->m_TimeAdded; + edbImport.m_Flags=stat->m_Flags; + edbImport.m_Block=block; + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Write((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbImport+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + } + } + + edbImport.Zero(); // Import block update + edbImport.m_Block=block; + edbImport.m_ImportID=imp->m_ImportID; + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Write((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbImport+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + + + if(imp->m_ImportID == 0) + { + m_DBStat.m_Block=block; + err=m_Database->m_DB->Write((char*)&m_DBStat+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&m_DBStat+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + goto exitlbl; + } + } + + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + goto exitlbl; + } + + imp->m_Block=block; + +exitlbl: + + if(err) + { + sprintf(msg,"Could not roll back to block %d, error: %d",block,err); + LogString(msg); + } + else + { + sprintf(msg,"Rolled back to block %d successfully",block); + LogString(msg); + } + + Dump("After RollBack"); + return err; +} + +int mc_TxDB::GetRow( + mc_TxEntityRow *erow) +{ + int err, value_len,i; + unsigned char *ptr; + mc_TxEntityRow *lpEnt; + + err=MC_ERR_NOERROR; + + erow->SwapPosBytes(); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + erow->SwapPosBytes(); + if(err) + { + return err; + } + if(ptr) + { + memcpy((char*)erow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + return MC_ERR_NOERROR; + } + else + { + for(i=0;iGetCount();i++) + { + lpEnt=(mc_TxEntityRow *)m_MemPools[0]->GetRow(i); + if( (lpEnt->m_TempPos == erow->m_Pos) && + (memcmp(&(lpEnt->m_Entity),&(erow->m_Entity),sizeof(mc_TxEntity)) == 0)) + { + memcpy(erow,lpEnt,MC_TDB_ROW_SIZE); + erow->m_Pos=lpEnt->m_TempPos; + return MC_ERR_NOERROR; + } + } + } + + return MC_ERR_NOT_FOUND; + +} + +int mc_TxDB::GetList(mc_TxEntity *entity, + int from, + int count, + mc_Buffer *txs) +{ + return GetList(m_Imports,entity,from,count,txs); +} + +int mc_TxDB::GetList( + mc_TxImport *import, + mc_TxEntity *entity, + int from, + int count, + mc_Buffer *txs) +{ + int first,last,i; + mc_TxEntityRow erow; + mc_TxEntityRow *lpEnt; + mc_Buffer *mempool; + int value_len; + unsigned char *ptr; + int err,mprow,found; + char msg[256]; + + txs->Clear(); + + int row; + mc_TxEntityStat *stat; + + row=import->FindEntity(entity); + if(row < 0) + { + return MC_ERR_NOERROR; + } + + mempool=m_MemPools[import-m_Imports]; + stat=(mc_TxEntityStat*)import->m_Entities->GetRow(row); + + first=from; + last=GetListSize(entity,NULL); + if(first <= 0) + { + first=last-count+1+first; + } + if(first+count-1 < 1) + { + return MC_ERR_NOERROR; + } + if(first<=0) + { + first=1; + } + if(first+count-1 <= last) + { + last=first+count-1; + } + + + erow.Zero(); + memcpy(&erow.m_Entity,&(stat->m_Entity),sizeof(mc_TxEntity)); + erow.m_Generation=stat->m_Generation; + mprow=-1; + for(i=first;i<=last;i++) + { + erow.m_Pos=i; + if(erow.m_Pos <= stat->m_LastClearedPos) // Database rows + { + erow.SwapPosBytes(); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + erow.SwapPosBytes(); + if(err) + { + return err; + } + + if(ptr) + { + txs->Add((char*)&erow,ptr); + } + else + { + sprintf(msg,"GetList: couldn't find item %d in database, entity type %08X",i,erow.m_Entity.m_EntityType); + LogString(msg); + return MC_ERR_NOT_FOUND; + } + } + else // mempool rows + { + mprow++; + found=0; + while((found == 0) && (mprow < mempool->GetCount())) + { + lpEnt=(mc_TxEntityRow *)mempool->GetRow(mprow); + if( (lpEnt->m_TempPos == erow.m_Pos) && + (memcmp(&(lpEnt->m_Entity),entity,sizeof(mc_TxEntity)) == 0)) + { + memcpy(&erow,lpEnt,MC_TDB_ROW_SIZE); + erow.m_Pos=i; + txs->Add((char*)&erow,(char*)&erow+MC_TDB_ENTITY_KEY_SIZE); + found=1; + } + else + { + mprow++; + } + } + if(mprow >= mempool->GetCount()) + { + sprintf(msg,"GetList: couldn't find item %d in mempool, entity type %08X",i,erow.m_Entity.m_EntityType); + LogString(msg); + return MC_ERR_NOT_FOUND; + } + } + } + + return MC_ERR_NOERROR; +} + +int mc_TxDB::GetList(mc_TxEntity *entity, + int generation, + int from, + int count, + mc_Buffer *txs) +{ + return GetList(m_Imports,entity,generation,from,count,txs); +} + +int mc_TxDB::GetList( + mc_TxImport *import, + mc_TxEntity *entity, + int generation, + int from, + int count, + mc_Buffer *txs) +{ + int first,last,i,confirmed; + mc_TxEntityRow erow; + mc_TxEntityRow *lpEnt; + mc_Buffer *mempool; + int value_len; + unsigned char *ptr; + int err,mprow,found; + char msg[256]; + + txs->Clear(); + + mempool=m_MemPools[import-m_Imports]; + + first=from; + last=GetListSize(entity,generation,&confirmed); + + if(last <= 0) + { + return MC_ERR_NOERROR; + } + + if(first <= 0) + { + first=last-count+1+first; + } + if(first+count-1 < 1) + { + return MC_ERR_NOERROR; + } + if(first<=0) + { + first=1; + } + if(first+count-1 <= last) + { + last=first+count-1; + } + + + erow.Zero(); + memcpy(&erow.m_Entity,entity,sizeof(mc_TxEntity)); + erow.m_Generation=generation; + mprow=-1; + for(i=first;i<=last;i++) + { + erow.m_Pos=i; + if((int)erow.m_Pos <= confirmed) // Database rows + { + erow.SwapPosBytes(); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + erow.SwapPosBytes(); + if(err) + { + return err; + } + + if(ptr) + { + txs->Add((char*)&erow,ptr); + } + else + { + sprintf(msg,"GetList: couldn't find item %d in database, entity type %08X",i,erow.m_Entity.m_EntityType); + LogString(msg); + return MC_ERR_NOT_FOUND; + } + } + else // mempool rows + { + mprow++; + found=0; + while((found == 0) && (mprow < mempool->GetCount())) + { + lpEnt=(mc_TxEntityRow *)mempool->GetRow(mprow); + if( (lpEnt->m_TempPos == erow.m_Pos) && + (memcmp(&(lpEnt->m_Entity),entity,sizeof(mc_TxEntity)) == 0)) + { + memcpy(&erow,lpEnt,MC_TDB_ROW_SIZE); + erow.m_Pos=i; + txs->Add((char*)&erow,(char*)&erow+MC_TDB_ENTITY_KEY_SIZE); + found=1; + } + else + { + mprow++; + } + } + if(mprow >= mempool->GetCount()) + { + sprintf(msg,"GetList: couldn't find item %d in mempool, entity type %08X",i,erow.m_Entity.m_EntityType); + LogString(msg); + return MC_ERR_NOT_FOUND; + } + } + } + + return MC_ERR_NOERROR; +} + +int mc_TxDB::GetListSize(mc_TxEntity *entity,int generation,int *confirmed) +{ + int last_pos,mprow,err,value_len,last_conf; + mc_TxEntityRow erow; + unsigned char *ptr; + + last_conf=0; + if(entity->m_EntityType & MC_TET_SUBKEY) + { + erow.Zero(); + memcpy(&erow.m_Entity,entity,sizeof(mc_TxEntity)); + erow.m_Generation=generation; + + last_pos=0; + mprow=m_MemPools[0]->Seek(&erow); + if(mprow >= 0) + { + mprow+=1; // Anchor doesn't carry info except key, but the next row is the first for this entity + last_pos=((mc_TxEntityRow*)(m_MemPools[0]->GetRow(mprow)))->m_LastSubKeyPos; + last_conf=((mc_TxEntityRow*)(m_MemPools[0]->GetRow(mprow)))->m_TempPos-1; + } + else + { + erow.Zero(); + memcpy(&erow.m_Entity,entity,sizeof(mc_TxEntity)); + erow.m_Generation=generation; + erow.m_Pos=1; + erow.SwapPosBytes(); + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + erow.SwapPosBytes(); + if(err) + { + return 0; + } + if(ptr) + { + memcpy((char*)&erow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + last_pos=erow.m_LastSubKeyPos; + last_conf=last_pos; + } + } + if(confirmed) + { + *confirmed=last_conf; + } + } + else + { + return GetListSize(entity,confirmed); + } + return last_pos; +} + +int mc_TxDB::GetListSize(mc_TxEntity *entity,int *confirmed) +{ + int row; + mc_TxEntityStat *stat; + + row=m_Imports[0].FindEntity(entity); + if(row < 0) + { + return 0; + } + + stat=(mc_TxEntityStat*)m_Imports[0].m_Entities->GetRow(row); + + if(confirmed) + { + *confirmed=stat->m_LastClearedPos; + } + + return (int)(stat->m_LastPos); +} + +mc_TxImport *mc_TxDB::StartImport(mc_Buffer *lpEntities,int block,int *err) +{ + char msg[256]; + char enthex[65]; + int i,j,slot,generation,row,count; + uint32_t pos; + int value_len; + mc_TxImportRow edbImport; + mc_TxEntityStat *lpChainEntStat; + mc_TxEntityStat *lpEntStat; + mc_TxEntityRow erow; + mc_TxEntityRow *lperow; + unsigned char* ptr; + + + Dump("Before StartImport"); + generation=m_DBStat.m_LastGeneration+1; + slot=0; + for(i=1;im_Entities == NULL) + { + slot=i; + } + } + } + + if(slot == 0) + { + sprintf(msg,"Couldn't start new import - %d imports already in progress",MC_TDB_MAX_IMPORTS-1); + LogString(msg); + *err=MC_ERR_NOT_ALLOWED; + goto exitlbl; + } + + sprintf(msg,"Starting new import %d with %d entities from block %d",generation, lpEntities->GetCount(),block); + LogString(msg); + + (m_Imports+slot)->Init(generation,block); + edbImport.Zero(); + edbImport.m_Generation=(m_Imports+slot)->m_ImportID; + edbImport.m_ImportID=edbImport.m_Generation; + edbImport.SwapPosBytes(); + *err=m_Database->m_DB->Write((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbImport+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(*err) + { + goto exitlbl; + } + + for(j=0;jGetCount();j++) + { + (m_Imports+slot)->AddEntity((mc_TxEntity*)lpEntities->GetRow(j)); + count=0; + lpEntStat=(m_Imports+slot)->GetEntity(j); + lpEntStat->m_Flags |= MC_EFL_NOT_IN_SYNC; + row=m_Imports->FindEntity(lpEntStat); + if(row >= 0) // Old entity + { + lpChainEntStat=m_Imports->GetEntity(row); + if( (lpChainEntStat->m_Flags & MC_EFL_NOT_IN_SYNC) == 0 ) // in-sync rows ordered by timereceived should be copied + { + if( (lpChainEntStat->m_Entity.m_EntityType & MC_TET_ORDERMASK) == MC_TET_TIMERECEIVED) + { + lpEntStat->m_Flags -= MC_EFL_NOT_IN_SYNC; + erow.Zero(); + memcpy(&erow.m_Entity,&(lpEntStat->m_Entity),sizeof(mc_TxEntity)); + for(pos=1;pos<=lpChainEntStat->m_LastClearedPos;pos++) // Copy old chain wallet list ordered by timereceived + { + erow.m_Pos=pos; + erow.SwapPosBytes(); + erow.m_Generation=lpChainEntStat->m_Generation; + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,err); + erow.m_Generation=(m_Imports+slot)->m_ImportID; // Change generation when writing to DB + *err=m_Database->m_DB->Write((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)ptr,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + erow.SwapPosBytes(); + count++; + if(*err) + { + goto exitlbl; + } + if(((pos+1) % 1000) == 0) + { + m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + } + } + if(lpChainEntStat->m_LastClearedPos < lpChainEntStat->m_LastPos) + { + for(i=0;iGetCount();i++) // mempool transactions + { + lperow=(mc_TxEntityRow *)m_MemPools[0]->GetRow(i); + if(memcmp(&(lperow->m_Entity),&(lpEntStat->m_Entity),sizeof(mc_TxEntity)) == 0) + { + memcpy(&erow,lperow,sizeof(mc_TxEntityRow)); + erow.m_TempPos=0; + erow.m_Pos=lperow->m_TempPos; + erow.m_Generation=(m_Imports+slot)->m_ImportID; + erow.SwapPosBytes(); + *err=m_Database->m_DB->Write((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&erow+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + erow.SwapPosBytes(); + count++; + } + } + } + lpEntStat->m_LastPos=lpChainEntStat->m_LastPos; + lpEntStat->m_LastClearedPos=lpChainEntStat->m_LastPos; + } + } + } + + memcpy(&edbImport.m_Entity,lpEntities->GetRow(j),sizeof(mc_TxEntity)); + sprintf_hex(enthex,edbImport.m_Entity.m_EntityID,MC_TDB_ENTITY_ID_SIZE); + sprintf(msg,"Import: %d, Entity: (%08X, %s), transferred txs: %d",generation,edbImport.m_Entity.m_EntityType,enthex,count); + LogString(msg); + edbImport.m_Pos+=1; + edbImport.m_LastPos=lpEntStat->m_LastPos; + edbImport.m_Flags=lpEntStat->m_Flags; + edbImport.SwapPosBytes(); + *err=m_Database->m_DB->Write((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbImport+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(*err) + { + goto exitlbl; + } + } + + m_DBStat.m_LastGeneration=generation; + + *err=m_Database->m_DB->Write((char*)&m_DBStat+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&m_DBStat+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(*err) + { + goto exitlbl; + } + + *err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(*err) + { + goto exitlbl; + } + +exitlbl: + + if(*err) + { + sprintf(msg,"Couldn't start new import, error: %d",*err); + LogString(msg); + m_DBStat.m_LastGeneration=generation-1; + if(slot) + { + (m_Imports+slot)->m_Entities->Clear(); + delete (m_Imports+slot)->m_Entities; + delete (m_Imports+slot)->m_TmpEntities; + (m_Imports+slot)->m_Block=-1; + (m_Imports+slot)->m_ImportID=0; + } + return NULL; + } + + if(m_MemPools[slot]) + { + m_MemPools[slot]->Clear(); + } + else + { + m_MemPools[slot]=new mc_Buffer; + m_MemPools[slot]->Initialize(MC_TDB_ENTITY_KEY_SIZE+MC_TDB_TXID_SIZE,m_Database->m_TotalSize,MC_BUF_MODE_MAP); + } + + if(m_RawMemPools[slot]) + { + m_RawMemPools[slot]->Clear(); + } + else + { + m_RawMemPools[slot]=new mc_Buffer; + m_RawMemPools[slot]->Initialize(MC_TDB_TXID_SIZE,m_Database->m_TotalSize,MC_BUF_MODE_MAP); + } + + + sprintf(msg,"Import %d successfully started",generation); + LogString(msg); + Dump("After StartImport"); + + return m_Imports+slot; +} + + +mc_TxImport *mc_TxDB::FindImport(int import_id) +{ + int i; + if(import_id == 0) + { + return m_Imports; + } + for(i=0;im_Entities) + { + if((m_Imports+i)->m_ImportID == import_id) + { + return (m_Imports+i); + } + } + } + return NULL; +} + +int mc_TxDB::ImportGetBlock(mc_TxImport *import) +{ + return import->m_Block; +} + +int mc_TxDB::Unsubscribe(mc_Buffer* lpEntities) +{ + char msg[256]; + char enthex[65]; + int err; + int deleted_items,row,unsubscribed_entities,subkey_list_size,value_len; + int i,j; + uint32_t pos; + mc_TxImportRow edbImport; + mc_TxEntityStat *lpent; + mc_TxEntityRow erow; + mc_TxEntityRow subkey_erow; + unsigned char stream_subkey_hash160[20]; + unsigned char *ptr; + + err=MC_ERR_NOERROR; + + if(lpEntities->GetCount() == 0) + { + return MC_ERR_NOERROR; + } + + deleted_items=0; + unsubscribed_entities=0; + for(j=0;jGetCount();j++) // Removing previous generation rows + { + row=m_Imports->FindEntity((mc_TxEntity*)lpEntities->GetRow(j)); + if(row >= 0) + { + unsubscribed_entities+=1; + lpent=(mc_TxEntityStat*)m_Imports->m_Entities->GetRow(row); + lpent->m_Flags |= MC_EFL_UNSUBSCRIBED; + for(pos=1;pos<=lpent->m_LastClearedPos;pos++) + { + erow.Zero(); + memcpy(&erow.m_Entity,&(lpent->m_Entity),sizeof(mc_TxEntity)); + erow.m_Generation=lpent->m_Generation; + erow.m_Pos=pos; + erow.SwapPosBytes(); + + switch(lpent->m_Entity.m_EntityType & MC_TET_TYPE_MASK) + { + case MC_TET_STREAM_KEY: + case MC_TET_STREAM_PUBLISHER: + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(err == MC_ERR_NOERROR) + { + if(ptr) + { + memcpy((char*)&erow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + } + mc_GetCompoundHash160(&stream_subkey_hash160,&(erow.m_Entity.m_EntityID),erow.m_TxId); + subkey_erow.Zero(); + memcpy(subkey_erow.m_Entity.m_EntityID,&stream_subkey_hash160,MC_TDB_ENTITY_ID_SIZE); + subkey_erow.m_Entity.m_EntityType=lpent->m_Entity.m_EntityType | MC_TET_SUBKEY; + subkey_erow.m_Generation=erow.m_Generation; + GetListSize(&(subkey_erow.m_Entity),subkey_erow.m_Generation,&subkey_list_size); + for(i=0;im_DB->Delete((char*)&subkey_erow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + subkey_erow.SwapPosBytes(); + deleted_items++; + if(deleted_items >= 1000) + { + m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + deleted_items=0; + } + } + } + break; + } + + m_Database->m_DB->Delete((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + erow.SwapPosBytes(); + deleted_items++; + + if(deleted_items >= 1000) + { + m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + deleted_items=0; + } + } + + sprintf_hex(enthex,lpent->m_Entity.m_EntityID,MC_TDB_ENTITY_ID_SIZE); + sprintf(msg,"Entity (%08X, %s) unsubscribed",lpent->m_Entity.m_EntityType,enthex); + LogString(msg); + } + } + + i=0; + for(j=0;jm_Entities->GetCount();j++) + { + lpent=(mc_TxEntityStat*)m_Imports->m_Entities->GetRow(j); + if((lpent->m_Flags & MC_EFL_UNSUBSCRIBED) == 0) + { + if(i != j) + { + lpent->m_PosInImport=i+1; + + memcpy(m_Imports->m_Entities->GetRow(i),lpent,m_Imports->m_Entities->m_RowSize); + edbImport.Zero(); + memcpy(&(edbImport.m_Entity), &(lpent->m_Entity),sizeof(mc_TxEntity)); + edbImport.m_ImportID=0; + edbImport.m_Generation=lpent->m_Generation; + edbImport.m_LastPos=lpent->m_LastPos; + edbImport.m_Pos=lpent->m_PosInImport; + edbImport.m_LastImportedBlock=lpent->m_LastImportedBlock; + edbImport.m_TimeAdded=lpent->m_TimeAdded; + edbImport.m_Flags=lpent->m_Flags; + edbImport.m_Block=m_Imports->m_Block; + + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Write((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbImport+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + deleted_items++; + if(err) + { + goto exitlbl; + } + } + i++; + } + } + + for(j=i;jm_Entities->GetCount();j++) + { + edbImport.Zero(); + edbImport.m_ImportID=0; + edbImport.m_Pos=j+1; + + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Delete((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + deleted_items++; + if(err) + { + goto exitlbl; + } + } + + m_Imports->m_Entities->SetCount(i); + + if(deleted_items) + { + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + deleted_items=0; + } + +exitlbl: + + return err; +} + +int mc_TxDB::CompleteImport(mc_TxImport *import) +{ + char msg[256]; + char enthex[65]; + char txhex[65]; + int err; + int i,j,row,gen,chain_entities,take_it,value_len; + int subkey_list_size,deleted_items,mprow; + uint32_t pos,clearedpos; + mc_Buffer *mempool; + mc_Buffer *rawmempool; + + mc_TxImportRow edbImport; + mc_TxEntityStat *lpent; + mc_TxEntityStat *lpdel; + mc_TxEntityRow erow; + mc_TxEntityRow subkey_erow; + mc_TxEntityRow *lperow; + mc_TxDefRow *lptxdef; + unsigned char stream_subkey_hash160[20]; + unsigned char *ptr; + + + Dump("Before CompleteImport"); + + err=MC_ERR_NOERROR; + + chain_entities=m_Imports->m_Entities->GetCount(); + +/* + if(m_MemPools[0]->GetCount()) + { + sprintf(msg,"CompleteImport: Import %d cannot be completed because chain mempool is not empty",import->m_ImportID); + LogString(msg); + err=MC_ERR_NOT_ALLOWED; + goto exitlbl; + } + */ + mempool=m_MemPools[import-m_Imports]; + rawmempool=m_RawMemPools[import-m_Imports]; + if((import->m_Block != m_Imports->m_Block) && (m_Imports->m_Block != -1)) + { + sprintf(msg,"CompleteImport: Import %d is out-of-sync with chain, import block: %d, chain block: %d",import->m_ImportID,import->m_Block,m_Imports->m_Block); + LogString(msg); + err=MC_ERR_NOT_ALLOWED; + goto exitlbl; + } + + + for(j=0;jm_Entities->GetCount();j++) + { + lpent=(mc_TxEntityStat*)import->m_Entities->GetRow(j); + lpdel=NULL; + row=m_Imports->FindEntity(lpent); + if(row >= 0) // Storing old generation info in import entity list + { + lpdel=m_Imports->GetEntity(row); + pos=lpdel->m_LastPos; + clearedpos=lpdel->m_LastClearedPos; + lpdel->m_LastPos=lpent->m_LastPos; + if(lpent->m_Flags & MC_EFL_NOT_IN_SYNC) + { + lpdel->m_LastPos+=pos-clearedpos; // New position including mempool + } + lpdel->m_LastClearedPos=lpent->m_LastClearedPos; // New position excluding mempool + lpent->m_LastClearedPos=clearedpos; // Old position, excluding mempool +// lpent->m_LastPos=lpent->m_LastPos; // New position excluding mempool + gen=lpdel->m_Generation; + lpdel->m_Generation=lpent->m_Generation; // Swapping generation + lpent->m_Generation=gen; + } + else + { + row=m_Imports->m_Entities->GetCount(); + m_Imports->AddEntity(lpent); + lpdel=m_Imports->GetEntity(row); + } + + lpdel->m_LastImportedBlock=import->m_Block; + edbImport.m_Flags=lpdel->m_Flags; + if(lpdel->m_Flags & MC_EFL_NOT_IN_SYNC) + { + lpdel->m_Flags-=MC_EFL_NOT_IN_SYNC; + } + + + edbImport.Zero(); // Storing new entity + edbImport.m_ImportID=0; + memcpy(&edbImport.m_Entity,&(lpent->m_Entity),sizeof(mc_TxEntity)); + edbImport.m_Generation=lpdel->m_Generation; + edbImport.m_Block=import->m_Block; + edbImport.m_Pos=lpdel->m_PosInImport; + edbImport.m_LastPos=lpdel->m_LastPos; + edbImport.m_LastImportedBlock=import->m_Block; + edbImport.m_TimeAdded=lpdel->m_TimeAdded; + edbImport.m_Flags=lpdel->m_Flags; + sprintf_hex(enthex,edbImport.m_Entity.m_EntityID,MC_TDB_ENTITY_ID_SIZE); + sprintf(msg,"Import: %d, Transferring entity (%08X, %s) to the chain, txs: %d",import->m_ImportID,edbImport.m_Entity.m_EntityType,enthex,lpdel->m_LastPos); + LogString(msg); + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Write((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbImport+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + + edbImport.Zero(); // Deleting old entity + edbImport.m_ImportID=import->m_ImportID; + edbImport.m_Pos=lpent->m_PosInImport; + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Delete((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + } + + edbImport.Zero(); // Import block update + edbImport.m_Block=import->m_Block; + edbImport.m_ImportID=0; + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Write((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&edbImport+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + + m_DBStat.m_Block=import->m_Block; // DB stats + err=m_Database->m_DB->Write((char*)&m_DBStat+m_Database->m_KeyOffset,m_Database->m_KeySize,(char*)&m_DBStat+m_Database->m_ValueOffset,m_Database->m_ValueSize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + goto exitlbl; + } + + + edbImport.Zero(); // Deleting old import + edbImport.m_ImportID=import->m_ImportID; + edbImport.m_Pos=0; + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Delete((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + + + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + goto exitlbl; + } + + m_Imports->m_Block=import->m_Block; + + // Cleanup below this point, cannot fail + j=0; + for(i=0;iGetCount();i++) + { + take_it=1; + lperow=(mc_TxEntityRow*)m_MemPools[0]->GetRow(i); + row=import->FindEntity(&(lperow->m_Entity)); + if(row >= 0) // Shifting mempool transactions, in-sync transactions are removed + { + lpent=import->GetEntity(row); + lperow->m_TempPos+=lpent->m_LastPos-lpent->m_LastClearedPos; + lperow->m_Generation=import->m_ImportID; + if( (lpent->m_Flags & MC_EFL_NOT_IN_SYNC) == 0 ) + { + mprow=rawmempool->Seek(lperow->m_TxId); + if(mprow >= 0) + { + lptxdef=(mc_TxDefRow*)rawmempool->GetRow(mprow); + lptxdef->m_Flags |= 0x80000000; // MC_TFL_IMPOSSIBLE + } + take_it=0; + } + } + if(take_it) + { + memcpy(m_MemPools[0]->GetRow(j),m_MemPools[0]->GetRow(i),sizeof(mc_TxEntityRow)); + j++; + } + } + m_MemPools[0]->SetCount(j); + + for(i=0;iGetCount();i++) // Copying mempool transactions + { + lperow=(mc_TxEntityRow *)mempool->GetRow(i); + if( ((lperow->m_Entity.m_EntityType & MC_TET_TYPE_MASK) == MC_TET_PUBKEY_ADDRESS) || + ((lperow->m_Entity.m_EntityType & MC_TET_TYPE_MASK) == MC_TET_SCRIPT_ADDRESS)) + { + mprow=rawmempool->Seek(lperow->m_TxId); // Will be added later when block is processed + if(mprow >= 0) + { + lptxdef=(mc_TxDefRow*)rawmempool->GetRow(mprow); + lptxdef->m_Flags |= 0x80000000; // MC_TFL_IMPOSSIBLE + } + } + sprintf_hex(txhex,lperow->m_TxId,MC_TDB_TXID_SIZE); + sprintf_hex(enthex,lperow->m_Entity.m_EntityID,MC_TDB_ENTITY_ID_SIZE); + sprintf(msg,"Import: %d, Transferring mempool: tx: %s, entity: (%08X, %s)",import->m_ImportID,txhex,lperow->m_Entity.m_EntityType,enthex); + LogString(msg); + m_MemPools[0]->Add(lperow,(unsigned char*)lperow+MC_TDB_ENTITY_KEY_SIZE+MC_TDB_TXID_SIZE); + } + + for(i=0;iGetCount();i++) // Copying rawmempool transactions + { + lptxdef=(mc_TxDefRow*)rawmempool->GetRow(i); + if((lptxdef->m_Flags & 0x80000000) == 0) + { + mprow=m_RawMemPools[0]->Seek(lptxdef->m_TxId); + if(mprow < 0) + { + sprintf_hex(txhex,lptxdef->m_TxId,MC_TDB_TXID_SIZE); + sprintf(msg,"Import: %d, Transferring rawmempool: tx: %s",import->m_ImportID,txhex); + LogString(msg); + m_RawMemPools[0]->Add(lptxdef,(unsigned char*)lptxdef+MC_TDB_TXID_SIZE); + } + } + } + + sprintf(msg,"CompleteImport: Successfully updated chain after import %d, removing old records",import->m_ImportID); + LogString(msg); + + deleted_items=0; + for(j=0;jm_Entities->GetCount();j++) // Removing previous generation rows + { + lpent=(mc_TxEntityStat*)import->m_Entities->GetRow(j); + if(lpent->m_Generation != import->m_ImportID) // generation was changed when storing old chain entity + { + for(pos=1;pos<=lpent->m_LastClearedPos;pos++) + { + erow.Zero(); + memcpy(&erow.m_Entity,&(lpent->m_Entity),sizeof(mc_TxEntity)); + erow.m_Generation=lpent->m_Generation; + erow.m_Pos=pos; + erow.SwapPosBytes(); + + switch(lpent->m_Entity.m_EntityType & MC_TET_TYPE_MASK) + { + case MC_TET_STREAM_KEY: + case MC_TET_STREAM_PUBLISHER: + ptr=(unsigned char*)m_Database->m_DB->Read((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,&value_len,0,&err); + if(err == MC_ERR_NOERROR) + { + if(ptr) + { + memcpy((char*)&erow+m_Database->m_ValueOffset,ptr,m_Database->m_ValueSize); + } + mc_GetCompoundHash160(&stream_subkey_hash160,&(erow.m_Entity.m_EntityID),erow.m_TxId); + subkey_erow.Zero(); + memcpy(subkey_erow.m_Entity.m_EntityID,&stream_subkey_hash160,MC_TDB_ENTITY_ID_SIZE); + subkey_erow.m_Entity.m_EntityType=lpent->m_Entity.m_EntityType | MC_TET_SUBKEY; + subkey_erow.m_Generation=erow.m_Generation; + GetListSize(&(subkey_erow.m_Entity),subkey_erow.m_Generation,&subkey_list_size); + for(i=0;im_DB->Delete((char*)&subkey_erow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + subkey_erow.SwapPosBytes(); + deleted_items++; + if(deleted_items >= 1000) + { + m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + deleted_items=0; + } + } + } + break; + } + + m_Database->m_DB->Delete((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + erow.SwapPosBytes(); + deleted_items++; + + if(deleted_items >= 1000) + { + m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + deleted_items=0; + } + } + if(deleted_items) + { + m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + deleted_items=0; + } + } + } + + +exitlbl: + + if(err) + { + for(j=0;jm_Entities->GetRow(j); // Restoring chain entities in case of failure (swapping back) + if(lpdel->m_Generation == import->m_ImportID) + { + row=import->FindEntity(lpdel); + if(row >= 0) + { + lpent=import->GetEntity(row); + pos=lpdel->m_LastPos; + lpdel->m_LastPos=lpent->m_LastPos; + lpdel->m_LastClearedPos=lpent->m_LastClearedPos; + lpent->m_LastPos=pos; + lpent->m_LastClearedPos=pos; + gen=lpdel->m_Generation; + lpdel->m_Generation=lpent->m_Generation; + lpent->m_Generation=gen; + lpent->m_Flags |= MC_EFL_NOT_IN_SYNC; + } + } + } + m_Imports->m_Entities->SetCount(chain_entities); + sprintf(msg,"Could not complete import %d, error: %d",import->m_ImportID,err); + LogString(msg); + + } + else + { + sprintf(msg,"CompleteImport: Import %d completed",import->m_ImportID); + LogString(msg); + m_MemPools[import-m_Imports]->Clear(); + m_RawMemPools[import-m_Imports]->Clear(); + delete import->m_Entities; + delete import->m_TmpEntities; + import->m_Block=-1; + import->m_ImportID=0; + import->m_Entities=NULL; + import->m_TmpEntities=NULL; + } + + Dump("After CompleteImport"); + + return err; +} + +int mc_TxDB::DropImport(mc_TxImport *import) +{ + char msg[256]; + int err; + int j,commit_required; + uint32_t pos; + mc_TxImportRow edbImport; + mc_TxEntityStat *lpent; + mc_TxEntityRow erow; + + Dump("Before DropImport"); + + err=MC_ERR_NOERROR; + + for(j=0;jm_Entities->GetCount();j++) + { + lpent=(mc_TxEntityStat*)import->m_Entities->GetRow(j); + edbImport.Zero(); + edbImport.m_ImportID=import->m_ImportID; + edbImport.m_Pos=lpent->m_PosInImport; + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Delete((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + } + + edbImport.Zero(); + edbImport.m_ImportID=import->m_ImportID; + edbImport.m_Pos=0; + edbImport.SwapPosBytes(); + err=m_Database->m_DB->Delete((char*)&edbImport+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + edbImport.SwapPosBytes(); + if(err) + { + goto exitlbl; + } + + err=m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + if(err) + { + goto exitlbl; + } + + sprintf(msg,"DropImport: Successfully dropped Import %d, removing old records",import->m_ImportID); + LogString(msg); + + commit_required=0; + for(j=0;jm_Entities->GetCount();j++) + { + lpent=(mc_TxEntityStat*)import->m_Entities->GetRow(j); + for(pos=1;pos<=lpent->m_LastPos;pos++) + { + erow.Zero(); + memcpy(&erow.m_Entity,&(lpent->m_Entity),sizeof(mc_TxEntity)); + erow.m_Generation=lpent->m_Generation; + erow.m_Pos=pos; + erow.SwapPosBytes(); + m_Database->m_DB->Delete((char*)&erow+m_Database->m_KeyOffset,m_Database->m_KeySize,MC_OPT_DB_DATABASE_TRANSACTIONAL); + erow.SwapPosBytes(); + commit_required=1; + if(((pos+1) % 1000) == 0) + { + m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + commit_required=0; + } + } + if(commit_required) + { + m_Database->m_DB->Commit(MC_OPT_DB_DATABASE_TRANSACTIONAL); + commit_required=0; + } + } + + +exitlbl: + + if(err) + { + sprintf(msg,"Could not drop import %d, error: %d",import->m_ImportID,err); + LogString(msg); + } + else + { + sprintf(msg,"DropImport: Import %d dropped",import->m_ImportID); + LogString(msg); + delete import->m_Entities; + delete import->m_TmpEntities; + import->m_Block=-1; + import->m_ImportID=0; + import->m_Entities=NULL; + import->m_TmpEntities=NULL; + } + + Dump("After DropImport"); + return err; +} diff --git a/src/wallet/wallettxdb.h b/src/wallet/wallettxdb.h new file mode 100644 index 00000000..d3178ade --- /dev/null +++ b/src/wallet/wallettxdb.h @@ -0,0 +1,382 @@ +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAIN_WALLETTXDB_H +#define MULTICHAIN_WALLETTXDB_H + +#include "utils/declare.h" +#include "utils/dbwrapper.h" + +#define MC_TDB_TXID_SIZE 32 +#define MC_TDB_ENTITY_KEY_SIZE 32 +#define MC_TDB_ENTITY_ID_SIZE 20 +#define MC_TDB_ENTITY_TYPE_SIZE 4 +#define MC_TDB_GENERATION_SIZE 4 +#define MC_TDB_POS_SIZE 4 +#define MC_TDB_ROW_SIZE 80 + +#define MC_TDB_MAX_IMPORTS 16 + +#define MC_TDB_WALLET_VERSION 2 + + +#define MC_TET_NONE 0x00000000 +#define MC_TET_WALLET_ALL 0x00000001 +#define MC_TET_WALLET_SPENDABLE 0x00000002 +#define MC_TET_PUBKEY_ADDRESS 0x00000003 +#define MC_TET_SCRIPT_ADDRESS 0x00000004 +#define MC_TET_STREAM 0x00000005 +#define MC_TET_STREAM_KEY 0x00000006 +#define MC_TET_STREAM_PUBLISHER 0x00000007 +#define MC_TET_ASSET 0x00000008 +#define MC_TET_SUBKEY_STREAM_KEY 0x00000046 +#define MC_TET_SUBKEY_STREAM_PUBLISHER 0x00000047 +#define MC_TET_SUBKEY 0x00000040 +#define MC_TET_TYPE_MASK 0x000000FF +#define MC_TET_CHAINPOS 0x00000100 +#define MC_TET_TIMERECEIVED 0x00000200 +#define MC_TET_ORDERMASK 0x0000FF00 +#define MC_TET_DB_STAT 0x01000000 +#define MC_TET_IMPORT 0x02000000 +#define MC_TET_SPECIALMASK 0xFF000000 + +#define MC_EFL_NOT_IN_SYNC 0x01000000 +#define MC_EFL_UNSUBSCRIBED 0x10000000 + +#define MC_SFL_NONE 0x00000000 +#define MC_SFL_IS_SCRIPTHASH 0x00000001 +#define MC_SFL_IS_ADDRESS 0x00000002 +#define MC_SFL_NODATA 0x01000000 +#define MC_SFL_SUBKEY 0x02000000 + + +/** Entity - wallet, address, stream, etc. **/ + +typedef struct mc_TxEntity +{ + unsigned char m_EntityID[MC_TDB_ENTITY_ID_SIZE]; // 20-byte entity id = pubkeyhash, scripthash, stream-reference + uint32_t m_EntityType; // Entity type, MC_TET_ constants + void Zero(); + void Init(unsigned char *entity_id,uint32_t entity_type); +} mc_TxEntity; + +/** Entity row pos->txid **/ + +typedef struct mc_TxEntityRow +{ + mc_TxEntity m_Entity; // Entity + int m_Generation; // Generation of entity data + uint32_t m_Pos; // position of the row in generation, 1-based + unsigned char m_TxId[MC_TDB_TXID_SIZE]; // txID + int m_Block; // block confirmed in + uint32_t m_Flags; // Flags passed from higher level + uint32_t m_LastSubKeyPos; // Position of the last subkey row (only for the first) + uint32_t m_TempPos; // Not stored in DB. Equal to m_Pos, added for ability to search in mempool + void Zero(); + void SwapPosBytes(); +} mc_TxEntityRow; + +/** Entity stats - last position, etc. In-memory structure **/ + +typedef struct mc_TxEntityStat +{ + mc_TxEntity m_Entity; // Entity + uint32_t m_PosInImport; // Position of the entity in the import + uint32_t m_KeyReserved1; + uint64_t m_ValueReserved1; + uint64_t m_ValueReserved2; + uint64_t m_ValueReserved3; + uint32_t m_TimeAdded; + uint32_t m_Flags; + int m_Generation; // Generation of entity within import + int m_LastImportedBlock; // Last imported block before merging with chain. O for the chain import. + uint32_t m_LastPos; // Current last position + uint32_t m_LastClearedPos; // Last position without mempool + void Zero(); +} mc_TxEntityStat; + +/** Import row - image of mc_TxEntityStat on disk **/ + +typedef struct mc_TxImportRow +{ + unsigned char m_Zero[MC_TDB_ENTITY_ID_SIZE]; + uint32_t m_RowType; // MC_TET_IMPORT + int m_ImportID; // Import ID. + uint32_t m_Pos; // Entity Position within Import, 1-based + mc_TxEntity m_Entity; // Entity + int m_Block; // Last block + int m_LastImportedBlock; // Last imported block before merging with chain. O for the chain import. + uint32_t m_LastPos; // Current last position + int m_Generation; // Generation of entity within import + uint32_t m_TimeAdded; + uint32_t m_Flags; + void Zero(); + void SwapPosBytes(); +} mc_TxImportRow; + +/** Global database stats row **/ + +typedef struct mc_TxEntityDBStat +{ + unsigned char m_Zero[MC_TDB_ENTITY_ID_SIZE]; + uint32_t m_RowType; // MC_TET_DB_STAT + uint64_t m_KeyReserved1; + uint64_t m_ValueReserved1; + uint64_t m_ValueReserved2; + uint32_t m_InitMode; + uint32_t m_WalletVersion; + int m_Block; // Last block of the chain import + int m_LastGeneration; // Last ImportID used + uint32_t m_Count; // Total tx count + uint32_t m_FullSize; // Total tx size + uint32_t m_LastFileID; // Last data file ID + uint32_t m_LastFileSize; // Last data file size + void Zero(); +} mc_TxEntityDBStat; + +/** Import structure, in-memory **/ + +typedef struct mc_TxImport +{ + int m_ImportID; // Import ID + int m_Block; // last block index + mc_Buffer *m_Entities; // List of import entities (mc_TxEntityStat) + mc_Buffer *m_TmpEntities; // Temporary list of entities (mc_TxEntity) + mc_TxImport() + { + Zero(); + } + + ~mc_TxImport() + { + Destroy(); + } + void Zero(); + int Init(int generation,int block); + int AddEntity(mc_TxEntity *entity); + int AddEntity(mc_TxEntityStat *entity); + int FindEntity(mc_TxEntity *entity); + int FindEntity(mc_TxEntityStat *entity); + mc_TxEntityStat *GetEntity(int row); + void Destroy(); +} mc_TxImport; + + +/** Tx definition row **/ + +typedef struct mc_TxDefRow +{ + unsigned char m_TxId[MC_TDB_TXID_SIZE]; // TxID + uint32_t m_Size; // Tx Size + uint32_t m_FullSize; // Original Tx size, before chopping long OP_RETURNs + uint32_t m_InternalFileID; // Data file ID + uint32_t m_InternalFileOffset; // Offset in the data file + int m_Block; // Block confirmed in + int m_BlockFileID; // Block file ID + uint32_t m_BlockOffset; // Position of the block in the block file + uint32_t m_BlockTxOffset; // Position of the tx within block (relative to header)) + uint32_t m_TimeReceived; // Time received by this wallet + uint32_t m_Flags; // Flags passed from higher level + uint32_t m_BlockIndex; // Tx index in block + uint32_t m_ValueReserved1; + void Zero(); +} mc_TxDefRow; + +/** Internal database **/ + +typedef struct mc_TxEntityDB +{ + char m_FileName[MC_DCT_DB_MAX_PATH]; // Full file name + mc_Database *m_DB; // Database object + uint32_t m_KeyOffset; // Offset of the key in mc_PermissionDBRow structure, 32 for protocol<=10003, 0 otherwise + uint32_t m_KeySize; // Size of the database key, 24 for protocol<=10003, 56 otherwise + uint32_t m_ValueOffset; // Offset of the value in mc_PermissionDBRow structure, 56 + uint32_t m_ValueSize; // Size of the database value, 24 + uint32_t m_TotalSize; // Totals size of the database row + mc_TxEntityDB() + { + Zero(); + } + + ~mc_TxEntityDB() + { + Close(); + } + void Zero(); + int Open(); + int Close(); + void SetName(const char *name); +} mc_TxEntityDB; + +/** Transactions DB **/ + +typedef struct mc_TxDB +{ + mc_TxEntityDB *m_Database; // Database + mc_Buffer *m_MemPools[MC_TDB_MAX_IMPORTS]; // mc_TxEntityRow mempool + mc_Buffer *m_RawMemPools[MC_TDB_MAX_IMPORTS]; // mc_TxDefRow mempool + mc_Buffer *m_RawUpdatePool; // Updated txs mempool + mc_TxImport m_Imports[MC_TDB_MAX_IMPORTS]; // Imports, 0 - chain + mc_TxEntityDBStat m_DBStat; // Database stats + + char m_Name[MC_PRM_NETWORK_NAME_MAX_SIZE+1]; // Chain name + char m_LobFileNamePrefix[MC_DCT_DB_MAX_PATH]; // Full data file name + char m_LogFileName[MC_DCT_DB_MAX_PATH]; // Full log file name + + uint32_t m_Mode; + void *m_Semaphore; // mc_TxDB object semaphore + uint64_t m_LockedBy; // ID of the thread locking it + + mc_TxDB() + { + Zero(); + } + + ~mc_TxDB() + { + Destroy(); + } + + int Initialize( // Initalization + const char *name, // Chain name + uint32_t mode); // Unused + + int AddEntity(mc_TxEntity *entity,uint32_t flags); // Adds entity to chain import + int AddEntity(mc_TxImport *import,mc_TxEntity *entity,uint32_t flags); // Adds entity to import + + int FindEntity(mc_TxImport *import,mc_TxEntity *entity); // Finds entity in import + int FindEntity(mc_TxImport *import,mc_TxEntityStat *entity); // Finds entity in import + + int FindSubKey( + mc_TxImport *import, // Import object, if NULL - chain + mc_TxEntity *parent_entity, // Parent entity - stream + mc_TxEntity *entity); // Subkey entity + + int AddSubKeyDef( + mc_TxImport *import, // Import object, if NULL - chain + const unsigned char *hash, // SubKey hash + const unsigned char *subkey, // Subkey + uint32_t subkeysize, // Subkey size + uint32_t flags // Subkey flags + ); + + int IncrementSubKey( + mc_TxImport *import, // Import object, if NULL - chain + mc_TxEntity *parent_entity, // Parent entity - stream + mc_TxEntity *entity, // Subkey entity + const unsigned char *subkey_hash, // Subkey hash + const unsigned char *tx_hash, // Tx hash (before chopping) + int block, // Block we are processing now, -1 for mempool + uint32_t flags, // Flags passed by the higher level + int newtx // New tx flag + ); + + int DecrementSubKey( + mc_TxImport *import, // Import object, if NULL - chain + mc_TxEntity *parent_entity, // Parent entity - stream + mc_TxEntity *entity); // Subkey entity + + int AddTx( // Adds transaction to mempool + mc_TxImport *import, // Import object, if NULL - chain + const unsigned char *hash, // Tx hash (before chopping) + const unsigned char *tx, // Tx (chopped)) + uint32_t txsize, // Tx size + uint32_t txfullsize, // Original Tx size, before chopping long OP_RETURNs + int block, // Block we are processing now, -1 for mempool + int block_file, // Block file ID + uint32_t block_offset, // Block offset in the block file + uint32_t block_tx_offset, // Tx offset in the block, relative to the header + uint32_t block_tx_index, // Tx index in block + uint32_t flags, // Flags passed by the higher level + uint32_t timestamp, // timestamp to be stored as timereceived + mc_Buffer *entities); // List of relevant entities for this tx + + int GetTx( // Returns tx definition if found, error if not found + mc_TxDefRow *txdef, // Output. Tx def + const unsigned char *hash); // Input. Tx hash + + int GetList( + mc_TxImport *import, // Import object, if NULL - chain + mc_TxEntity *entity, // Returns Txs in range for specific entity + int from, // If positive - from this tx, if not positive - this number from the end + int count, // Number of txs to return + mc_Buffer *txs); // Output list. mc_TxEntityRow + + int GetList(mc_TxEntity *entity, // Returns Txs in range for specific entity + int from, // If positive - from this tx, if not positive - this number from the end + int count, // Number of txs to return + mc_Buffer *txs); // Output list. mc_TxEntityRow + + int GetList( + mc_TxImport *import, // Import object, if NULL - chain + mc_TxEntity *entity, // Returns Txs in range for specific entity + int generation, // Entity generation + int from, // If positive - from this tx, if not positive - this number from the end + int count, // Number of txs to return + mc_Buffer *txs); // Output list. mc_TxEntityRow + + int GetList(mc_TxEntity *entity, // Returns Txs in range for specific entity + int generation, // Entity generation + int from, // If positive - from this tx, if not positive - this number from the end + int count, // Number of txs to return + mc_Buffer *txs); // Output list. mc_TxEntityRow + + int GetRow( + mc_TxEntityRow *erow); + + int GetListSize( // Return total number of tx in the list for specific entity + mc_TxEntity *entity, // Entity to return info for + int *confirmed); // Out: number of confirmed items + + int GetListSize( // Return total number of tx in the list for specific entity + mc_TxEntity *entity, // Entity to return info for + int generation, // Entity generation + int *confirmed); // Out: number of confirmed items + + int BeforeCommit(mc_TxImport *import); // Should be called before re-adding tx while processing block + int Commit(mc_TxImport *import); // Commit when block was processed + int RollBack(mc_TxImport *import,int block); // Rollback to specific block + + int Unsubscribe(mc_Buffer *lpEntities); // List of the entities to unsubscribe from + + mc_TxImport *StartImport( // Starts new import + mc_Buffer *lpEntities, // List of entities to import + int block, // Star from this block + int *err); // Output. Error + + mc_TxImport *FindImport( // Find import with specific ID + int import_id); + + int ImportGetBlock( // Returns last processed block in the import + mc_TxImport *import); + + int CompleteImport(mc_TxImport *import); // Completes import - merges with chain + + int DropImport(mc_TxImport *import); // Drops uncompleted import + + int SaveTxFlag( // Changes tx flag setting (if tx is found)) + const unsigned char *hash, // Tx ID + uint32_t flag, // Flag to set/unset + int set_flag); // 1 if set, 0 if unset + +// Internal functions + void Zero(); + int Destroy(); + void Dump(const char *message); + + int Lock(int write_mode, int allow_secondary); + void UnLock(); + + int AddToFile(const unsigned char *tx, + uint32_t txsize, + uint32_t fileid, + uint32_t offset); + + void LogString(const char *message); + + +} mc_TxDB; + + +#endif /* MULTICHAIN_WALLETTXDB_H */ + diff --git a/src/wallet/wallettxs.cpp b/src/wallet/wallettxs.cpp new file mode 100644 index 00000000..72a23159 --- /dev/null +++ b/src/wallet/wallettxs.cpp @@ -0,0 +1,2751 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#include "wallet/wallettxs.h" + +int64_t GetAdjustedTime(); + +using namespace std; + +void mc_Coin::Zero() +{ + m_EntityID=0; + m_EntityType=MC_TET_NONE; + m_Block=-1; + m_Flags=0; + m_LockTime=0; +} + +bool mc_Coin::IsFinal() const +{ + int nBlockHeight,nBlockTime; + AssertLockHeld(cs_main); + // Time based nLockTime implemented in 0.1.6 + if (m_LockTime == 0) + return true; + nBlockHeight = chainActive.Height(); + nBlockTime = GetAdjustedTime(); + if ((int64_t)m_LockTime < ((int64_t)m_LockTime < LOCKTIME_THRESHOLD ? (int64_t)nBlockHeight : nBlockTime)) + return true; + + return (m_Flags & MC_TFL_ALL_INPUTS_ARE_FINAL) > 0; + +} + +int mc_Coin::BlocksToMaturity() const +{ + if ((m_Flags & MC_TFL_IS_COINBASE) == 0) + { + return 0; + } + return max(0, (COINBASE_MATURITY+1) - GetDepthInMainChain()); +} + +bool mc_Coin::IsTrusted() const +{ + if (!IsFinal()) + { + return false; + } + int nDepth=GetDepthInMainChain(); + + if (nDepth >= 1) + { + return true; + } + if (nDepth < 0) + { + return false; + } + + return (m_Flags & MC_TFL_ALL_INPUTS_FROM_ME) > 0; +} + +int mc_Coin::GetDepthInMainChain() const +{ + int nDepth=0; + if(m_Block >= 0) + { + nDepth= chainActive.Height()-m_Block+1; + } + return nDepth; +} + +string mc_Coin::ToString() const +{ + CBitcoinAddress addr; + if( (m_EntityType & MC_TET_TYPE_MASK) == MC_TET_PUBKEY_ADDRESS ) + { + CKeyID lpKeyID=CKeyID(m_EntityID); + addr=CBitcoinAddress(lpKeyID); + } + if( (m_EntityType & MC_TET_TYPE_MASK) == MC_TET_SCRIPT_ADDRESS ) + { + CScriptID lpScriptID=CScriptID(m_EntityID); + addr=CBitcoinAddress(lpScriptID); + } + + return strprintf("Coin: %s %s (%08X,%d) %s",m_OutPoint.ToString().c_str(),m_TXOut.ToString().c_str(),m_Flags,m_Block,addr.ToString().c_str()); +} + + + +void mc_WalletTxs::Zero() +{ + int i; + m_Database=NULL; + m_lpWallet=NULL; + for(i=0;iInitialize(name,mode); + + if(err == MC_ERR_NOERROR) + { + m_Mode=m_Database->m_DBStat.m_InitMode; + for(i=0;im_Imports[i].m_Entities) + { + err=LoadUTXOMap(m_Database->m_Imports[i].m_ImportID,m_Database->m_Imports[i].m_Block); + } + } + + } + + m_Mode |= (mode & MC_WMD_AUTOSUBSCRIBE_STREAMS); + m_Mode |= (mode & MC_WMD_AUTOSUBSCRIBE_ASSETS); + + if(err == MC_ERR_NOERROR) + { + m_UnconfirmedSends= GetUnconfirmedSends(m_Database->m_DBStat.m_Block); + } + return err; +} + +int mc_WalletTxs::Destroy() +{ + if(m_Database) + { + delete m_Database; + } + + Zero(); + return MC_ERR_NOERROR; + +} + +bool mc_WalletTxs::FindEntity(mc_TxEntityStat *entity) +{ + int row; + if((m_Mode & MC_WMD_TXS) == 0) + { + return false; + } + if(m_Database == NULL) + { + return false; + } + row=m_Database->FindEntity(NULL,entity); + return (row >= 0) ? true : false; +} + + +int mc_WalletTxs::AddEntity(mc_TxEntity *entity,uint32_t flags) +{ + int err; + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOERROR; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + m_Database->Lock(1,0); + err=m_Database->AddEntity(entity,flags); + m_Database->UnLock(); + return err; +} + +mc_Buffer* mc_WalletTxs::GetEntityList() +{ + if((m_Mode & MC_WMD_TXS) == 0) + { + return NULL; + } + if(m_Database == NULL) + { + return NULL; + } + return m_Database->m_Imports->m_Entities; +} + +void mc_WalletTxs::Lock() +{ + if(m_Database) + { + m_Database->Lock(0,0); + } +} + +void mc_WalletTxs::UnLock() +{ + if(m_Database) + { + m_Database->UnLock(); + } +} + +int mc_WalletTxs::GetBlock() +{ + if(m_Database) + { + return m_Database->m_DBStat.m_Block; + } + return -1; +} + +/* + BeforeCommit - called before adding block txs + + Raw Txs: Do Nothing + Entity lists by blockchain position: Clear mempool + Entity lists by timestamp: Do nothing + UTXOs: Restore UTXO from disk (clear mempool) + Unconfirmed txs: Do Nothing + */ + +int mc_WalletTxs::BeforeCommit(mc_TxImport *import) +{ + int err,count; + mc_TxImport *imp; + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOERROR; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + imp=m_Database->m_Imports; + if(import) + { + imp=import; + } + m_Database->Lock(1,0); + + count=0; + if(imp->m_ImportID == 0) // No need for non-chain imports as there is no mempool there + { + count=m_Database->m_MemPools[0]->GetCount(); + } + err=m_Database->BeforeCommit(imp); + + if(err == MC_ERR_NOERROR) // Clearing all UTXO changes since last block + { + if(count) + { + err=LoadUTXOMap(imp->m_ImportID,m_Database->m_DBStat.m_Block); + } + } + + if(err) + { + LogPrintf("wtxs: BeforeCommit: Error: %d\n",err); + m_Database->Dump("Error in BeforeCommit"); + } + LogPrint("wallet","wtxs: BeforeCommit: Import: %d, Block: %d\n",imp->m_ImportID,imp->m_Block); + + m_Database->UnLock(); + return err; +} + +/* + Commit - called after adding block txs + + Raw Txs: Store raw tx mempool on disk + Entity lists by blockchain position: Store mempool on disk + Entity lists by timestamp: Store mempool on disk + UTXOs: Store UTXO on disk + Unconfirmed txs: After committing entities: Copy list from (block-1) to (block) without confirmed transactions + */ + +int mc_WalletTxs::Commit(mc_TxImport *import) +{ + int err; + mc_TxImport *imp; + + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOERROR; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + err = MC_ERR_NOERROR; + imp=m_Database->m_Imports; + if(import) + { + imp=import; + } + m_Database->Lock(1,0); + + if(err == MC_ERR_NOERROR) + { + err=SaveUTXOMap(imp->m_ImportID,imp->m_Block+1); + } + + if(err == MC_ERR_NOERROR) + { + err=m_Database->Commit(imp); + } + + if(err == MC_ERR_NOERROR) + { + if(imp->m_ImportID == 0) + { + std::map mapUnconfirmed= GetUnconfirmedSends(m_Database->m_DBStat.m_Block-1);// Confirmed txs are filtered out + m_UnconfirmedSends.clear(); + LogPrint("wallet","wtxs: Unconfirmed wallet transactions: %d\n",mapUnconfirmed.size()); + for (map::const_iterator it = mapUnconfirmed.begin(); it != mapUnconfirmed.end(); ++it) + { + AddToUnconfirmedSends(m_Database->m_DBStat.m_Block,it->second); + } + } + } + + if(err) + { + LogPrintf("wtxs: Commit: Error: %d\n",err); + m_Database->Dump("Error in Commit"); + } + LogPrint("wallet","wtxs: Commit: Import: %d, Block: %d\n",imp->m_ImportID,imp->m_Block); + m_Database->UnLock(); + return err; +} + +/* + CleanUpAfterBlock - called after full processing of the block (either committed or rolled back) + + Raw Txs: Do Nothing + Entity lists by blockchain position: Do Nothing + Entity lists by timestamp: Do Nothing + UTXOs: Remove previous block file + Unconfirmed txs: Remove previous block file + */ + +int mc_WalletTxs::CleanUpAfterBlock(mc_TxImport *import,int block,int prev_block) +{ + int err; + mc_TxImport *imp; + + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOERROR; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + err = MC_ERR_NOERROR; + imp=m_Database->m_Imports; + if(import) + { + imp=import; + } + + m_Database->Lock(1,0); + RemoveUTXOMap(imp->m_ImportID,prev_block); + if(imp->m_ImportID == 0) + { + RemoveUnconfirmedSends(prev_block); + } + LogPrint("wallet","wtxs: CleanUpAfterBlock: Import: %d, Block: %d\n",imp->m_ImportID,imp->m_Block); + m_Database->UnLock(); + + return err; +} + +/* + * Returns input entity if it is identical for all inputs + */ + +void mc_WalletTxs::GetSingleInputEntity(const CWalletTx& tx,mc_TxEntity *input_entity) +{ + mc_TxEntity entity; + mc_TxDefRow prevtxdef; + int err; + + input_entity->Zero(); + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + CWalletTx prevwtx=GetInternalWalletTx(txin.prevout.hash,&prevtxdef,&err); + if(err) + { + return; + } + + if(txin.prevout.n >= prevwtx.vout.size()) + { + return; + } + + const CScript& script1 = prevwtx.vout[txin.prevout.n].scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + CTxDestination addressRet; + + if(!ExtractDestination(script1, addressRet)) + { + return; + } + + entity.Zero(); + const CKeyID *lpKeyID=boost::get (&addressRet); + const CScriptID *lpScriptID=boost::get (&addressRet); + if(lpKeyID) + { + memcpy(entity.m_EntityID,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + } + if(lpScriptID) + { + memcpy(entity.m_EntityID,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + } + if(input_entity->m_EntityType) + { + if(memcmp(input_entity,&entity,sizeof(mc_TxEntity))) + { + return; + } + } + else + { + memcpy(input_entity,&entity,sizeof(mc_TxEntity)); + } + } +} + +int mc_WalletTxs::RollBackSubKeys(mc_TxImport *import,int block,mc_TxEntityStat *parent_entity,mc_Buffer *lpSubKeyEntRowBuffer) +{ + int err,i,j,r,from,count; + mc_TxImport *imp; + mc_TxEntityRow *entrow; + mc_TxEntity entity; + mc_TxEntity subkey_entity; + bool fInBlocks; + unsigned char item_key[MC_ENT_MAX_ITEM_KEY_SIZE]; + int item_key_size; + uint160 subkey_hash160; + uint160 stream_subkey_hash160; + set publishers_set; + + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOERROR; + } + + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + err = MC_ERR_NOERROR; + imp=m_Database->m_Imports; + if(import) + { + imp=import; + } + + if(block >= imp->m_Block) + { + if(block == imp->m_Block) + { + return MC_ERR_NOERROR; + } + return MC_ERR_INTERNAL_ERROR; + } + + fInBlocks=true; + from=0; + count=10; + while(fInBlocks) // While in relevant block range + { + err=m_Database->GetList(&(parent_entity->m_Entity),from,count,lpSubKeyEntRowBuffer); + if( (err == MC_ERR_NOERROR) && (lpSubKeyEntRowBuffer->GetCount() > 0) ) + { + for(r=lpSubKeyEntRowBuffer->GetCount()-1;r >= 0;r--) // Processing wallet rows in reverse order + { + entrow=(mc_TxEntityRow *)(lpSubKeyEntRowBuffer->GetRow(r)); + if(fInBlocks && // Still in loop, no error + (entrow->m_Block > block)) // Block to delete, mempool rows will be deleted in db rollback + { + uint256 hash; + memcpy(&hash,entrow->m_TxId,MC_TDB_TXID_SIZE); + CWalletTx wtx=GetInternalWalletTx(hash,NULL,&err); // Tx to delete + if(err) + { + LogPrintf("wtxs: RollBackSubKeys: Couldn't find tx %s, error: %d\n",hash.ToString().c_str(),err); + fInBlocks=false; + } + else + { + for(i=0;i<(int)wtx.vout.size();i++) + { + const CTxOut txout=wtx.vout[i]; + const CScript& script1 = txout.scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + if( (mc_gState->m_TmpScript->IsOpReturnScript() != 0 ) && (mc_gState->m_TmpScript->GetNumElements() == 3) ) + { + unsigned char short_txid[MC_AST_SHORT_TXID_SIZE]; + mc_gState->m_TmpScript->SetElement(0); + + if( (mc_gState->m_TmpScript->GetEntity(short_txid) == 0) && + (memcmp(short_txid,parent_entity->m_Entity.m_EntityID,MC_AST_SHORT_TXID_SIZE) == 0) ) + { + mc_gState->m_TmpScript->SetElement(1); + if(mc_gState->m_TmpScript->GetItemKey(item_key,&item_key_size)) // Item key + { + err=MC_ERR_INTERNAL_ERROR; + goto exitlbl; + } + + subkey_hash160=Hash160(item_key,item_key+item_key_size); + mc_GetCompoundHash160(&stream_subkey_hash160,parent_entity->m_Entity.m_EntityID,&subkey_hash160); + + entity.Zero(); + memcpy(entity.m_EntityID,short_txid,MC_AST_SHORT_TXID_SIZE); + entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_CHAINPOS; + subkey_entity.Zero(); + memcpy(subkey_entity.m_EntityID,&stream_subkey_hash160,MC_TDB_ENTITY_ID_SIZE); + subkey_entity.m_EntityType=MC_TET_SUBKEY_STREAM_KEY | MC_TET_CHAINPOS; + err= m_Database->DecrementSubKey(imp,&entity,&subkey_entity); + if(err) + { + goto exitlbl; + } + + publishers_set.clear(); + for (j = 0; j < (int)wtx.vin.size(); ++j) + { + int op_addr_offset,op_addr_size,is_redeem_script,sighash_type; + const unsigned char *ptr; + + const CScript& script2 = wtx.vin[j].scriptSig; + CScript::const_iterator pc2 = script2.begin(); + + ptr=mc_ExtractAddressFromInputScript((unsigned char*)(&pc2[0]),(int)(script2.end()-pc2), + &op_addr_offset,&op_addr_size,&is_redeem_script,&sighash_type,0); + if(ptr) + { + if( (sighash_type == SIGHASH_ALL) || ( (sighash_type == SIGHASH_SINGLE) && (j == i) ) ) + { + subkey_hash160=Hash160(ptr+op_addr_offset,ptr+op_addr_offset+op_addr_size); + if(publishers_set.count(subkey_hash160) == 0) + { + publishers_set.insert(subkey_hash160); + mc_GetCompoundHash160(&stream_subkey_hash160,parent_entity->m_Entity.m_EntityID,&subkey_hash160); + + entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_CHAINPOS; + subkey_entity.Zero(); + memcpy(subkey_entity.m_EntityID,&stream_subkey_hash160,MC_TDB_ENTITY_ID_SIZE); + subkey_entity.m_EntityType=MC_TET_SUBKEY_STREAM_PUBLISHER | MC_TET_CHAINPOS; + err= m_Database->DecrementSubKey(imp,&entity,&subkey_entity); + if(err) + { + goto exitlbl; + } + } + } + } + } + } + } + } + } + } + else + { + if(entrow->m_Block >= 0) + { + fInBlocks=false; + } + } + } + } + else + { + if(err) + { + LogPrintf("wtxs: RollBackSubKeys: Error on getting rollback tx list: %d\n",err); + } + fInBlocks=false; + } + from-=count; + } + +exitlbl: + return err; +} + +/* + RollBack - called before restoring block transactions in mempool + + Raw Txs: Do Nothing + Entity lists by blockchain position: Clear mempool. remove all transactions from the rolled back blocks + Entity lists by timestamp: Do Nothing + UTXOs: Erase all outputs, restore all inputs for removed transactions. Use full-wallet list by blockchain position + Unconfirmed txs: Retrieve tx list from file (block) before rolling back, add all conflicted txs to (block-1). Other txs will be added by AddTx + */ + +int mc_WalletTxs::RollBack(mc_TxImport *import,int block) +{ + int err,i,r,from,count,import_pos,last_err; + mc_TxImport *imp; + mc_TxDefRow txdef; + mc_Buffer *lpEntRowBuffer; + mc_TxEntityRow *entrow; + mc_TxEntity entity; + mc_TxEntity wallet_entity; + mc_TxEntityStat *stat; + mc_Buffer *lpSubKeyEntRowBuffer; + bool fInBlocks; + std::vector txouts; + + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOERROR; + } + + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + err = MC_ERR_NOERROR; + imp=m_Database->m_Imports; + if(import) + { + imp=import; + } + import_pos=imp-m_Database->m_Imports; + + if(block >= imp->m_Block) + { + if(block == imp->m_Block) + { + return MC_ERR_NOERROR; + } + return MC_ERR_INTERNAL_ERROR; + } + + + m_Database->Lock(1,0); + + lpSubKeyEntRowBuffer=NULL; + for(i=0;im_Entities->GetCount();i++) // Removing ordered-by-blockchain-position rows + { + stat=(mc_TxEntityStat*)imp->m_Entities->GetRow(i); + if( stat->m_Entity.m_EntityType == (MC_TET_STREAM | MC_TET_CHAINPOS) ) + { + if(err == MC_ERR_NOERROR) + { + if(lpSubKeyEntRowBuffer == NULL) + { + lpSubKeyEntRowBuffer=new mc_Buffer; + lpSubKeyEntRowBuffer->Initialize(MC_TDB_ENTITY_KEY_SIZE,sizeof(mc_TxEntityRow),MC_BUF_MODE_DEFAULT); + } + err=RollBackSubKeys(imp,block,stat,lpSubKeyEntRowBuffer); + } + } + } + if(err) + { + LogPrintf("wtxs: RollBack: Error when rolling back soubkeys: %d\n",err); + } + if(lpSubKeyEntRowBuffer) + { + delete lpSubKeyEntRowBuffer; + } + + if(err == MC_ERR_NOERROR) + { + if(imp->m_ImportID == 0) // Copying conflicted transactions, they may become valid and should be rechecked + { + std::map mapUnconfirmed= GetUnconfirmedSends(m_Database->m_DBStat.m_Block); + for (map::const_iterator it = mapUnconfirmed.begin(); it != mapUnconfirmed.end(); ++it) + { + m_Database->GetTx(&txdef,(unsigned char*)&(it->first)); + if(txdef.m_Flags & MC_TFL_INVALID) + { + AddToUnconfirmedSends(block,it->second); + } + } + } + } + if(err == MC_ERR_NOERROR) + { + wallet_entity.Zero(); + wallet_entity.m_EntityType=MC_TET_WALLET_ALL | MC_TET_CHAINPOS; + if(imp->FindEntity(&wallet_entity) >= 0) // Only "address" imports have this entity and should update UTXOs + { + lpEntRowBuffer=new mc_Buffer; + lpEntRowBuffer->Initialize(MC_TDB_ENTITY_KEY_SIZE,sizeof(mc_TxEntityRow),MC_BUF_MODE_DEFAULT); + fInBlocks=true; + from=0; + count=10; + while(fInBlocks) // While in relevant block range + { + err=m_Database->GetList(&wallet_entity,from,count,lpEntRowBuffer); + if( (err == MC_ERR_NOERROR) && (lpEntRowBuffer->GetCount() > 0) ) + { + for(r=lpEntRowBuffer->GetCount()-1;r >= 0;r--) // Processing wallet rows in reverse order + { + entrow=(mc_TxEntityRow *)(lpEntRowBuffer->GetRow(r)); + if(fInBlocks && // Still in loop, no error + ((entrow->m_Block > block) || // Block to delete + (entrow->m_Block < 0))) // In mempool + { + uint256 hash; + memcpy(&hash,entrow->m_TxId,MC_TDB_TXID_SIZE); + CWalletTx wtx=GetInternalWalletTx(hash,NULL,&err); // Tx to delete + if(err) + { + LogPrintf("wtxs: RollBack: Couldn't find tx %s, error: %d\n",hash.ToString().c_str(),err); + fInBlocks=false; + } + else + { + BOOST_FOREACH(const CTxIn& txin, wtx.vin) // Inputs-to-restore list + { + COutPoint outp(txin.prevout.hash,txin.prevout.n); + mc_TxDefRow prevtxdef; + const CWalletTx& prevwtx=GetInternalWalletTx(txin.prevout.hash,&prevtxdef,&last_err); + // Previous transaction + + if(last_err == MC_ERR_NOERROR) // Tx found, probably it is relevant input + { + if(txin.prevout.n < prevwtx.vout.size()) + { + mc_Coin txout; + txout.m_OutPoint=outp; + txout.m_TXOut=prevwtx.vout[txin.prevout.n]; + txout.m_Flags=prevtxdef.m_Flags; + txout.m_Block=prevtxdef.m_Block; + txout.m_EntityID=0; + txout.m_EntityType=MC_TET_NONE; + txout.m_LockTime=prevwtx.nLockTime; + + const CScript& script1 = txout.m_TXOut.scriptPubKey; + txnouttype typeRet; + int nRequiredRet; + std::vector addressRets; + + ExtractDestinations(script1,typeRet,addressRets,nRequiredRet); + + BOOST_FOREACH(const CTxDestination& dest, addressRets) + { + entity.Zero(); + const CKeyID *lpKeyID=boost::get (&dest); + const CScriptID *lpScriptID=boost::get (&dest); + if(lpKeyID) + { + memcpy(entity.m_EntityID,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + } + if(lpScriptID) + { + memcpy(entity.m_EntityID,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + } + if(entity.m_EntityType) + { + if(imp->FindEntity(&entity) >= 0) // It is relevant input + { + if(addressRets.size() == 1) + { + memcpy(&(txout.m_EntityID),entity.m_EntityID,MC_TDB_ENTITY_ID_SIZE); + txout.m_EntityType=entity.m_EntityType; + isminefilter mine = m_lpWallet ? IsMine(*m_lpWallet, dest) : ISMINE_NO; + if(mine & ISMINE_SPENDABLE) //Spendable flag is used to avoid IsMine calculation in coin selection + { + txout.m_Flags |= MC_TFL_IS_SPENDABLE; + } + } + mc_TxEntity input_entity; + GetSingleInputEntity(prevwtx,&input_entity); // Check if the entity coinsides with single input entity of prev tx - change + if(memcmp(&input_entity,&entity,sizeof(mc_TxEntity)) == 0) + { + txout.m_Flags |= MC_TFL_IS_CHANGE; + } + txouts.push_back(txout); + } + } + } + } + else // Index on prev tx out of range + { + fInBlocks=false; + if(err == MC_ERR_NOERROR) // Something is wrong, set error if not set + { + err=MC_ERR_CORRUPTED; + } + } + } + else // prev tx not found - it cannot be our input + { + if(last_err != MC_ERR_NOT_FOUND) // If not our input - ignore it + { + fInBlocks=false; + if(err == MC_ERR_NOERROR) + { + err=last_err; + } + } + } + + } + + for(i=0;i<(int)wtx.vout.size();i++) // outputs-to-remove list + { + mc_Coin txout; + txout.m_OutPoint=COutPoint(wtx.GetHash(),i); + txout.m_TXOut=wtx.vout[i]; + txout.m_Flags=MC_TFL_IMPOSSIBLE; // We will delete this txout, setting this flag to distinguish from restored inputs + txout.m_Block=entrow->m_Block; + txout.m_EntityID=0; + txout.m_EntityType=MC_TET_NONE; + txout.m_LockTime=wtx.nLockTime; + + const CScript& script1 = txout.m_TXOut.scriptPubKey; + txnouttype typeRet; + int nRequiredRet; + std::vector addressRets; + + ExtractDestinations(script1,typeRet,addressRets,nRequiredRet); + + BOOST_FOREACH(const CTxDestination& dest, addressRets) + { + entity.Zero(); + const CKeyID *lpKeyID=boost::get (&dest); + const CScriptID *lpScriptID=boost::get (&dest); + if(lpKeyID) + { + memcpy(entity.m_EntityID,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + } + if(lpScriptID) + { + memcpy(entity.m_EntityID,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + } + if(entity.m_EntityType) + { + if(imp->FindEntity(&entity) >= 0) // It is relevant output + { + if(addressRets.size() == 1) + { + memcpy(&(txout.m_EntityID),entity.m_EntityID,MC_TDB_ENTITY_ID_SIZE); + txout.m_EntityType=entity.m_EntityType; + } + txouts.push_back(txout); // Added to the same array to preserve removing/restoring order + // otherwise earlier transactions may restore just-deleted outputs + } + } + } + } + LogPrint("wallet","wtxs: Removing tx %s, block %d, flags: %08X, import %d\n",hash.ToString().c_str(),entrow->m_Block,entrow->m_Flags,imp->m_ImportID); + } + } + else + { + if(entrow->m_Block >= 0) + { + fInBlocks=false; + } + } + } + } + else + { + if(err) + { + LogPrintf("wtxs: RollBack: Error on getting rollback tx list: %d\n",err); + } + fInBlocks=false; + } + from-=count; + } + + delete lpEntRowBuffer; + + if(err == MC_ERR_NOERROR) + { + for(i=0;i<(int)txouts.size();i++) + { + if(txouts[i].m_Flags == MC_TFL_IMPOSSIBLE) // Outputs to delete + { + m_UTXOs[import_pos].erase(txouts[i].m_OutPoint); + } + else // Inputs to restore + { + std::map::const_iterator itold = m_UTXOs[import_pos].find(txouts[i].m_OutPoint); + if (itold == m_UTXOs[import_pos].end()) + { + m_UTXOs[import_pos].insert(make_pair(txouts[i].m_OutPoint, txouts[i])); + } + } + } + } + } + } + if(err == MC_ERR_NOERROR) + { + err=SaveUTXOMap(imp->m_ImportID,block); + } + if(err == MC_ERR_NOERROR) + { + err=m_Database->RollBack(import,block); // Database rollback + } + if(err) + { + LogPrintf("wtxs: RollBack: Error: %d\n",err); + m_Database->Dump("Error in RollBack"); + } + LogPrint("wallet","wtxs: RollBack: Import: %d, Block: %d\n",imp->m_ImportID,imp->m_Block); + m_Database->UnLock(); + return err; +} + +int mc_WalletTxs::GetRow( + mc_TxEntityRow *erow) +{ + int err; + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + m_Database->Lock(0,0); + err=m_Database->GetRow(erow); + m_Database->UnLock(); + return err; +} + +int mc_WalletTxs::GetList(mc_TxEntity *entity,int from,int count,mc_Buffer *txs) +{ + int err; + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + m_Database->Lock(0,0); + err=m_Database->GetList(entity,from,count,txs); + m_Database->UnLock(); + return err; +} + +int mc_WalletTxs::GetList(mc_TxEntity *entity,int generation,int from,int count,mc_Buffer *txs) +{ + int err; + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + m_Database->Lock(0,0); + err=m_Database->GetList(entity,generation,from,count,txs); + m_Database->UnLock(); + return err; +} + +int mc_WalletTxs::GetListSize(mc_TxEntity *entity,int *confirmed) +{ + int res; + if((m_Mode & MC_WMD_TXS) == 0) + { + return 0; + } + if(m_Database == NULL) + { + return 0; + } + m_Database->Lock(0,0); + res=m_Database->GetListSize(entity,confirmed); + m_Database->UnLock(); + return res; +} + +int mc_WalletTxs::GetListSize(mc_TxEntity *entity,int generation,int *confirmed) +{ + int res; + if((m_Mode & MC_WMD_TXS) == 0) + { + return 0; + } + if(m_Database == NULL) + { + return 0; + } + m_Database->Lock(0,0); + res=m_Database->GetListSize(entity,generation,confirmed); + m_Database->UnLock(); + return res; +} + +int mc_WalletTxs::Unsubscribe(mc_Buffer* lpEntities) +{ + int err; + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + m_Database->Lock(1,0); + err=m_Database->Unsubscribe(lpEntities); + LogPrint("wallet","wtxs: Unsubscribed from %d entities\n",lpEntities->GetCount()); + m_Database->UnLock(); + return err; +} + +mc_TxImport *mc_WalletTxs::StartImport(mc_Buffer *lpEntities,int block,int *err) +{ + mc_TxImport *imp; + if((m_Mode & MC_WMD_TXS) == 0) + { + *err=MC_ERR_NOT_SUPPORTED; + return NULL; + } + if(m_Database == NULL) + { + *err=MC_ERR_INTERNAL_ERROR; + return NULL; + } + m_Database->Lock(1,0); + imp=m_Database->StartImport(lpEntities,block,err); + if(*err == MC_ERR_NOERROR) // BAD If block!=-1 old UXOs should be copied? + { + m_UTXOs[imp-m_Database->m_Imports].clear(); + } + LogPrint("wallet","wtxs: StartImport: Import: %d, Block: %d\n",imp->m_ImportID,imp->m_Block); + m_Database->UnLock(); + return imp; +} + +mc_TxImport *mc_WalletTxs::FindImport(int import_id) +{ + mc_TxImport *imp; + if((m_Mode & MC_WMD_TXS) == 0) + { + return NULL; + } + if(m_Database == NULL) + { + return NULL; + } + m_Database->Lock(0,0); + imp=m_Database->FindImport(import_id); + m_Database->UnLock(); + return imp; +} + +int mc_WalletTxs::ImportGetBlock(mc_TxImport *import) +{ + int block; + if((m_Mode & MC_WMD_TXS) == 0) + { + return -2; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + m_Database->Lock(0,0); + block=m_Database->ImportGetBlock(import); + m_Database->UnLock(); + return block; + +} + +int mc_WalletTxs::CompleteImport(mc_TxImport *import) +{ + int err,import_pos,gen,count; + + std::map mapCurrent; + + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + m_Database->Lock(1,0); + + gen=import->m_ImportID; + + err=m_Database->CompleteImport(import); + + + if(m_Database->m_DBStat.m_Block != m_Database->m_Imports->m_Block) + { + LogPrintf("wtxs: Internal error, block count mismatch: %d-%d\n",m_Database->m_DBStat.m_Block, m_Database->m_Imports->m_Block); + err=MC_ERR_INTERNAL_ERROR; + } + + import_pos=import-m_Database->m_Imports; + if(m_UTXOs[import_pos].size()) + { + count=m_Database->m_MemPools[0]->GetCount(); + + if(err == MC_ERR_NOERROR) + { + if(count > 0) // If mempool is not empty the following Save should not add mempool UTXOs + { + mapCurrent=m_UTXOs[0]; + err=LoadUTXOMap(0,m_Database->m_DBStat.m_Block); // Loading confirmed UTXO map + } + } + + if(err == MC_ERR_NOERROR) // Adding confirmed unspent coins + { + for (map::const_iterator it = m_UTXOs[import_pos].begin(); it != m_UTXOs[import_pos].end(); ++it) + { + std::map::iterator itold = m_UTXOs[0].find(it->first); + if (itold == m_UTXOs[0].end()) + { + if(it->second.m_Block >= 0) + { + m_UTXOs[0].insert(make_pair(it->first, it->second)); + } + } + else + { + itold->second.m_Flags=it->second.m_Flags; + } + } + } + + if(err == MC_ERR_NOERROR) // Saving confirmed UTXO map + { + err=SaveUTXOMap(0,m_Database->m_DBStat.m_Block); + } + + if(err == MC_ERR_NOERROR) // Restoring current UTXO map + { + if(count) + { + m_UTXOs[0]=mapCurrent; + } + } + + if(err == MC_ERR_NOERROR) + { + import_pos=import-m_Database->m_Imports; // Adding unspent coins to the "chain import" + for (map::const_iterator it = m_UTXOs[import_pos].begin(); it != m_UTXOs[import_pos].end(); ++it) + { + std::map::iterator itold = m_UTXOs[0].find(it->first); + if (itold == m_UTXOs[0].end()) + { + m_UTXOs[0].insert(make_pair(it->first, it->second)); + } + else + { + itold->second.m_Flags=it->second.m_Flags; + // in case of merge flags should be merged. If flag is MC_TFL_FROM_ME and not MC_TFL_ALL_INPUTS_FROM_ME in both cases + // it may become MC_TFL_ALL_INPUTS_FROM_ME after merge. + // This flag is important as txout may become unspendable after rollback + // if flag is updated - should update transaction + // Current import implementation sets final flag as it scans all transactions from the beginning + } + } + } + } + + if(err == MC_ERR_NOERROR) + { + RemoveUTXOMap(gen,m_Database->m_DBStat.m_Block); + } + if(err) + { + LogPrintf("wtxs: CompleteImport: Error: %d\n",err); + } + LogPrint("wallet","wtxs: CompleteImport: Import: %d, Block: %d\n",gen,m_Database->m_DBStat.m_Block); + m_Database->UnLock(); + return err; +} + +int mc_WalletTxs::DropImport(mc_TxImport *import) +{ + int err,gen; + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + m_Database->Lock(1,0); + + gen=import->m_ImportID; + err=m_Database->DropImport(import); + if(err == MC_ERR_NOERROR) + { + RemoveUTXOMap(import->m_ImportID,import->m_Block); + } + LogPrint("wallet","wtxs: DropImport: Import: %d, Block: %d\n",gen,m_Database->m_DBStat.m_Block); + m_Database->UnLock(); + return err; +} + +int mc_WalletTxs::AddToUnconfirmedSends(int block, const CWalletTx& tx) +{ + FILE* fHan; + char ShortName[65]; + char FileName[MC_DCT_DB_MAX_PATH]; + + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + + sprintf(ShortName,"wallet/uncsend_%d",block); + + mc_GetFullFileName(m_Database->m_Name,ShortName,".dat",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR,FileName); + + fHan=fopen(FileName,"ab+"); + + if(fHan == NULL) + { + return MC_ERR_FILE_WRITE_ERROR; + } + + CAutoFile fileout(fHan, SER_DISK, CLIENT_VERSION); + + fileout << tx; + + m_UnconfirmedSends.insert(make_pair(tx.GetHash(), tx)); + + return MC_ERR_NOERROR; +} + +int mc_WalletTxs::SaveTxFlag(const unsigned char *hash,uint32_t flag,int set_flag) +{ + int err,lockres; + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + + lockres=m_Database->Lock(1,1); + + err=m_Database->SaveTxFlag(hash,flag,set_flag); + + if(lockres == 0) + { + m_Database->UnLock(); + } + + return err; +} + +std::map mc_WalletTxs::GetUnconfirmedSends(int block) +{ + map outMap; + char ShortName[65]; + char FileName[MC_DCT_DB_MAX_PATH]; + bool fHaveTx; + uint256 hash; + int err; + mc_TxDefRow txdef; + + if((m_Mode & MC_WMD_TXS) == 0) + { + return outMap; + } + if(m_Database == NULL) + { + return outMap; + } + sprintf(ShortName,"wallet/uncsend_%d",block); + + mc_GetFullFileName(m_Database->m_Name,ShortName,".dat",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR,FileName); + + CAutoFile filein(fopen(FileName,"rb+"), SER_DISK, CLIENT_VERSION); + + fHaveTx=true; + while(fHaveTx) + { + CWalletTx wtx; + try + { + filein >> wtx; + hash=wtx.GetHash(); + err=m_Database->GetTx(&txdef,(unsigned char*)&hash); + if((err != MC_ERR_NOERROR) || (txdef.m_Block < 0) || ((txdef.m_Flags & MC_TFL_INVALID) != 0)) + // Invalid txs are returned for rechecking + { + std::map::const_iterator it = outMap.find(hash); + if (it == outMap.end()) + { + outMap.insert(make_pair(hash, wtx)); + } + } + } + catch (std::exception &e) + { + fHaveTx=false; + } + } + + return outMap; +} + +int mc_WalletTxs::RemoveUnconfirmedSends(int block) +{ + char ShortName[65]; + char FileName[MC_DCT_DB_MAX_PATH]; + + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + + sprintf(ShortName,"wallet/uncsend_%d",block); + + mc_GetFullFileName(m_Database->m_Name,ShortName,".dat",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR, FileName); + + remove(FileName); + + return MC_ERR_NOERROR; +} + +int mc_WalletTxs::RemoveUTXOMap(int import_id,int block) +{ + char ShortName[65]; + char FileName[MC_DCT_DB_MAX_PATH]; + + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + + if(block < 0) + { + return MC_ERR_NOERROR; + } + + sprintf(ShortName,"wallet/utxo%d_%d",import_id,block); + + mc_GetFullFileName(m_Database->m_Name,ShortName,".dat",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR, FileName); + + remove(FileName); + + return MC_ERR_NOERROR; +} + +int mc_WalletTxs::SaveUTXOMap(int import_id,int block) +{ + char ShortName[65]; + char FileName[MC_DCT_DB_MAX_PATH]; + FILE *fHan; + int import_pos; + + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + + if(block < 0) + { + return MC_ERR_NOERROR; + } +// printf("Save UTXO file %d\n",block); + + import_pos=m_Database->FindImport(import_id)-m_Database->m_Imports; + sprintf(ShortName,"wallet/utxo%d_%d",import_id,block); + + mc_GetFullFileName(m_Database->m_Name,ShortName,".dat",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR, FileName); + + remove(FileName); + + fHan=fopen(FileName,"wb+"); + + if(fHan == NULL) + { + return MC_ERR_FILE_WRITE_ERROR; + } + + CAutoFile fileout(fHan, SER_DISK, CLIENT_VERSION); + + for (map::const_iterator it = m_UTXOs[import_pos].begin(); it != m_UTXOs[import_pos].end(); ++it) + { + fileout << it->second; + } + + + return MC_ERR_NOERROR; +} + +int mc_WalletTxs::LoadUTXOMap(int import_id,int block) +{ + char ShortName[65]; + char FileName[MC_DCT_DB_MAX_PATH]; + int import_pos; + bool fHaveUtxo; + map mapOut; + + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOT_SUPPORTED; + } + if(m_Database == NULL) + { + return MC_ERR_INTERNAL_ERROR; + } + + import_pos=m_Database->FindImport(import_id)-m_Database->m_Imports; + + if(block < 0) + { + m_UTXOs[import_pos].clear(); + return MC_ERR_NOERROR; + } + + sprintf(ShortName,"wallet/utxo%d_%d",import_id,block); + + mc_GetFullFileName(m_Database->m_Name,ShortName,".dat",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR, FileName); + + CAutoFile filein(fopen(FileName,"rb+"), SER_DISK, CLIENT_VERSION); + + fHaveUtxo=true; + while(fHaveUtxo) + { + mc_Coin utxo; + try + { + filein >> utxo; + mapOut.insert(make_pair(utxo.m_OutPoint, utxo)); + } + catch (std::exception &e) + { + fHaveUtxo=false; + } + } + + m_UTXOs[import_pos]=mapOut; + + LogPrint("wallet","wtxs: Loaded %u unspent outputs for import %d\n",m_UTXOs[import_pos].size(),import_pos); + + return MC_ERR_NOERROR; +} + +/* + * Preparing tx with shortened OP_RETURN metadata + */ + +CWalletTx mc_WalletTxChoppedCopy(CWallet *lpWallet,const CWalletTx& tx) +{ + int i; + CWalletTx wtx(lpWallet); + + wtx.mapValue = tx.mapValue; + wtx.vOrderForm = tx.vOrderForm; + wtx.nTimeReceived = tx.nTimeReceived; + wtx.nTimeSmart = tx.nTimeSmart; + wtx.fFromMe = tx.fFromMe; + wtx.strFromAccount = tx.strFromAccount; + wtx.nOrderPos = tx.nOrderPos; + + CMutableTransaction txNew(tx); + txNew.vin.clear(); + txNew.vout.clear(); + + + BOOST_FOREACH(const CTxIn& txin, tx.vin) + { + txNew.vin.push_back(txin); + } + for(i=0;i<(int)tx.vout.size();i++) + { + const CTxOut txout=tx.vout[i]; + const CScript& script1 = txout.scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + std::vector addressRets; + CScript scriptOpReturn=CScript(); + bool fChopped=false; + size_t full_size=0; + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + if(mc_gState->m_TmpScript->IsOpReturnScript()) + { + size_t elem_size; + const unsigned char *elem; + int num_elems=mc_gState->m_TmpScript->GetNumElements(); + + for(int elem_id=0;elem_idm_TmpScript->GetData(elem_id,&elem_size); + scriptOpReturn << vector(elem, elem + elem_size) << OP_DROP; + } + + elem = mc_gState->m_TmpScript->GetData(num_elems-1,&elem_size); + + if(elem_size > MC_TDB_MAX_OP_RETURN_SIZE) + { + full_size=elem_size; + elem_size = MC_TDB_MAX_OP_RETURN_SIZE; + scriptOpReturn << OP_RETURN << vector(elem, elem + elem_size); + fChopped=true; + } + } + if(fChopped) + { + CTxOut txoutChopped(txout.nValue, scriptOpReturn); + txNew.vout.push_back(txoutChopped); + string str=strprintf("fullsize%05d",i); + wtx.mapValue[str] = full_size; + } + else + { + txNew.vout.push_back(txout); + } + } + + *static_cast(&wtx) = CTransaction(txNew); + + return wtx; +} + + +int mc_WalletTxs::AddTx(mc_TxImport *import,const CTransaction& tx,int block,CDiskTxPos* block_pos,uint32_t block_tx_index) +{ + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOERROR; + } + CWalletTx wtx(m_lpWallet,tx); + + return AddTx(import,wtx,block,block_pos,block_tx_index); +} + +int mc_WalletTxs::AddTx(mc_TxImport *import,const CWalletTx& tx,int block,CDiskTxPos* block_pos,uint32_t block_tx_index) +{ + int err,i,j,entcount,lockres,entpos; + mc_TxImport *imp; + mc_TxEntity entity; + mc_TxEntity subkey_entity; + mc_TxEntityStat* lpentstat; + mc_TxEntity input_entity; + mc_TxEntity *lpent; + mc_TxDefRow txdef; + const CWalletTx *fullTx; + const CWalletTx *storedTx; + CWalletTx stx; + unsigned char item_key[MC_ENT_MAX_ITEM_KEY_SIZE]; + int item_key_size; + uint256 subkey_hash256; + uint160 subkey_hash160; + uint160 stream_subkey_hash160; + set publishers_set; + + int import_pos; + bool fFound; + bool fIsFromMe; + bool fIsToMe; + bool fIsSpendable; + bool fAllInputsFromMe; + bool fAllInputsAreFinal; + bool fSingleInputEntity; + bool fRelevantEntity; + bool fOutputIsSpendable; + bool fNewStream; + bool fNewAsset; + std::vector txoutsIn; + std::vector txoutsOut; + uint256 hash; + unsigned char *ptrOut; + + uint32_t txsize; + uint32_t txfullsize; + int block_file; + uint32_t block_offset; + uint32_t block_tx_offset; + uint32_t flags; + uint32_t timestamp; + CDataStream ss(SER_DISK, CLIENT_VERSION); + + if((m_Mode & MC_WMD_TXS) == 0) + { + return MC_ERR_NOERROR; + } + + err=MC_ERR_NOERROR; + lockres=m_Database->Lock(1,1); + imp=import; + if(imp == NULL) + { + imp=m_Database->FindImport(0); + } + + import_pos=imp-m_Database->m_Imports; + hash=tx.GetHash(); + + fullTx=&tx; + storedTx=&tx; + + fFound=false; + txdef.Zero(); + if(m_Database->GetTx(&txdef,(unsigned char*)&hash) == MC_ERR_NOERROR) + { + fFound=true; + } + + if(!fFound) + { + for(i=0;i<(int)tx.vout.size();i++) // Checking that tx has long OP_RETURN + { + const CTxOut txout=tx.vout[i]; + const CScript& script1 = txout.scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + std::vector addressRets; + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + if(mc_gState->m_TmpScript->IsOpReturnScript()) + { + size_t elem_size; +// const unsigned char *elem; + int num_elems=mc_gState->m_TmpScript->GetNumElements(); + + mc_gState->m_TmpScript->GetData(num_elems-1,&elem_size); + if(elem_size > MC_TDB_MAX_OP_RETURN_SIZE) + { + storedTx=NULL; + } + } + } + } + + if(storedTx==NULL) + { + stx=mc_WalletTxChoppedCopy(m_lpWallet,tx); + storedTx=&stx; + if(fullTx->GetSerializeSize(SER_DISK, CLIENT_VERSION) <= storedTx->GetSerializeSize(SER_DISK, CLIENT_VERSION)) + { + storedTx=fullTx; + } + } + + + imp->m_TmpEntities->Clear(); + + fIsFromMe=false; + fIsToMe=false; + fIsSpendable=false; + fAllInputsFromMe=true; + fAllInputsAreFinal=true; + fSingleInputEntity=true; + fNewStream=false; + fNewAsset=false; + input_entity.Zero(); + BOOST_FOREACH(const CTxIn& txin, tx.vin) //Checking inputs + { + if(!tx.IsCoinBase()) + { + COutPoint outp(txin.prevout.hash,txin.prevout.n); + std::map::const_iterator it = m_UTXOs[import_pos].find(outp); + if (it != m_UTXOs[import_pos].end()) // We have this input in our unspent coins list, otherwise - it is not our input + { + mc_Coin *lpTxOut; + lpTxOut=(mc_Coin*)(&(it->second)); + mc_Coin utxo; + utxo.m_OutPoint=outp; + utxo.m_TXOut=lpTxOut->m_TXOut; + utxo.m_Block=block; + utxo.m_Flags=lpTxOut->m_Flags; + utxo.m_EntityID=0; + utxo.m_EntityType=MC_TET_NONE; + utxo.m_LockTime=lpTxOut->m_LockTime; + + if (!txin.IsFinal()) + { + fAllInputsAreFinal=false; + } + + const CScript& script1 = lpTxOut->m_TXOut.scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + txnouttype typeRet; + int nRequiredRet; + std::vector addressRets; + + if(!ExtractDestinations(script1,typeRet,addressRets,nRequiredRet)) + { + err=MC_ERR_CORRUPTED; + goto exitlbl; + } + + BOOST_FOREACH(const CTxDestination& dest, addressRets) + { + entity.Zero(); + const CKeyID *lpKeyID=boost::get (&dest); + const CScriptID *lpScriptID=boost::get (&dest); + if(lpKeyID) + { + memcpy(entity.m_EntityID,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + } + if(lpScriptID) + { + memcpy(entity.m_EntityID,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + } + isminefilter mine = IsMine(*m_lpWallet, dest); + if((mine & ISMINE_SPENDABLE) == ISMINE_NO) + { + fAllInputsFromMe=false; + } + if(entity.m_EntityType) + { + if(mine & ISMINE_SPENDABLE) + { + fIsSpendable=true; + } + if(mine & ISMINE_ALL) + { + fIsFromMe=true; + } + if(addressRets.size() == 1) + { + if(fSingleInputEntity) + { + if(input_entity.m_EntityType) + { + if(memcmp(&input_entity,&entity,sizeof(mc_TxEntity))) + { + fSingleInputEntity=false; + } + } + else + { + memcpy(&input_entity,&entity,sizeof(mc_TxEntity)); + } + } + } + else + { + fSingleInputEntity=false; + } + fRelevantEntity=false; + if(imp->FindEntity(&entity) >= 0) + { + fRelevantEntity=true; + if( (imp->m_ImportID > 0) && (block < 0) ) // When replaying mempool when completing import + { + entpos=m_Database->m_Imports->FindEntity(&entity); + if(entpos >= 0) // Ignore entities already processed in the chain import + { + lpentstat=m_Database->m_Imports->GetEntity(entpos); + if( (lpentstat->m_Flags & MC_EFL_NOT_IN_SYNC) == 0 ) + { + fRelevantEntity=false; + } + } + } + } + if(fRelevantEntity) + { + if(addressRets.size() == 1) + { + memcpy(&(utxo.m_EntityID),entity.m_EntityID,MC_TDB_ENTITY_ID_SIZE); + utxo.m_EntityType=entity.m_EntityType; + } + txoutsIn.push_back(utxo); + + if(mine & ISMINE_ALL) + { + if(imp->m_TmpEntities->Seek(&entity) < 0) + { + imp->m_TmpEntities->Add(&entity,NULL); + } + } + } + } + } + } + else + { + fAllInputsFromMe=false; + fSingleInputEntity=false; + } + } + else + { + fAllInputsFromMe=false; + fSingleInputEntity=false; + } + } + + mc_gState->m_TmpAssetsOut->Clear(); + + for(i=0;i<(int)tx.vout.size();i++) // Checking outputs + { + const CTxOut txout=tx.vout[i]; + const CScript& script1 = txout.scriptPubKey; + CScript::const_iterator pc1 = script1.begin(); + + txnouttype typeRet; + int nRequiredRet; + std::vector addressRets; + int64_t quantity; + + mc_gState->m_TmpScript->Clear(); + mc_gState->m_TmpScript->SetScript((unsigned char*)(&pc1[0]),(size_t)(script1.end()-pc1),MC_SCR_TYPE_SCRIPTPUBKEY); + if(mc_gState->m_TmpScript->IsOpReturnScript() == 0) + { + mc_Coin utxo; + utxo.m_OutPoint=COutPoint(tx.GetHash(),i); + utxo.m_TXOut=txout; + utxo.m_Block=block; + utxo.m_Flags=0; + utxo.m_EntityID=0; + utxo.m_EntityType=MC_TET_NONE; + utxo.m_LockTime=tx.nLockTime; + if(!ExtractDestinations(script1,typeRet,addressRets,nRequiredRet)) + { + err=MC_ERR_CORRUPTED; + goto exitlbl; + } + + for (int e = 0; e < mc_gState->m_TmpScript->GetNumElements(); e++) + { + mc_gState->m_TmpScript->SetElement(e); + mc_gState->m_TmpScript->GetAssetQuantities(mc_gState->m_TmpAssetsOut,MC_SCR_ASSET_SCRIPT_TYPE_TRANSFER | MC_SCR_ASSET_SCRIPT_TYPE_FOLLOWON); + if(mc_gState->m_TmpScript->GetAssetGenesis(&quantity) == 0) + { + fNewAsset=true; + } + } + + BOOST_FOREACH(const CTxDestination& dest, addressRets) + { + entity.Zero(); + const CKeyID *lpKeyID=boost::get (&dest); + const CScriptID *lpScriptID=boost::get (&dest); + if(lpKeyID) + { + memcpy(entity.m_EntityID,lpKeyID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_PUBKEY_ADDRESS | MC_TET_CHAINPOS; + } + if(lpScriptID) + { + memcpy(entity.m_EntityID,lpScriptID,MC_TDB_ENTITY_ID_SIZE); + entity.m_EntityType=MC_TET_SCRIPT_ADDRESS | MC_TET_CHAINPOS; + } + if(entity.m_EntityType) + { + isminefilter mine = m_lpWallet ? IsMine(*m_lpWallet, dest) : ISMINE_NO; + + fOutputIsSpendable=false; + if(mine & ISMINE_SPENDABLE) + { + fIsSpendable=true; + fOutputIsSpendable=true; + } + if(mine & ISMINE_ALL) + { + fIsToMe=true; + } + fRelevantEntity=false; + if(imp->FindEntity(&entity) >= 0) + { + fRelevantEntity=true; + if( (imp->m_ImportID > 0) && (block < 0) ) // When replaying mempool when completing import + { + entpos=m_Database->m_Imports->FindEntity(&entity); + if(entpos >= 0) // Ignore entities already processed in the chain import + { + lpentstat=m_Database->m_Imports->GetEntity(entpos); + if( (lpentstat->m_Flags & MC_EFL_NOT_IN_SYNC) == 0 ) + { + fRelevantEntity=false; + } + } + } + } + if(fRelevantEntity) + { + if(addressRets.size() == 1) + { + memcpy(&(utxo.m_EntityID),entity.m_EntityID,MC_TDB_ENTITY_ID_SIZE); + utxo.m_EntityType=entity.m_EntityType; + if(fOutputIsSpendable) + { + utxo.m_Flags |= MC_TFL_IS_SPENDABLE; + } + } + txoutsOut.push_back(utxo); + + if(mine & ISMINE_ALL) + { + if(imp->m_TmpEntities->Seek(&entity) < 0) + { + imp->m_TmpEntities->Add(&entity,NULL); + } + } + } + } + } + } + else + { + if(mc_gState->m_TmpScript->GetNumElements() == 3) // 2 OP_DROPs + OP_RETURN - item key + { + unsigned char short_txid[MC_AST_SHORT_TXID_SIZE]; + mc_gState->m_TmpScript->SetElement(0); + + if(mc_gState->m_TmpScript->GetEntity(short_txid) == 0) + { + entity.Zero(); + memcpy(entity.m_EntityID,short_txid,MC_AST_SHORT_TXID_SIZE); + entity.m_EntityType=MC_TET_STREAM | MC_TET_CHAINPOS; + if(imp->FindEntity(&entity) >= 0) + { + if(imp->m_TmpEntities->Seek(&entity) < 0) + { + imp->m_TmpEntities->Add(&entity,NULL); + + mc_gState->m_TmpScript->SetElement(1); + if(mc_gState->m_TmpScript->GetItemKey(item_key,&item_key_size)) // Item key + { + err=MC_ERR_INTERNAL_ERROR; + goto exitlbl; + } + + subkey_hash160=Hash160(item_key,item_key+item_key_size); + subkey_hash256=0; + memcpy(&subkey_hash256,&subkey_hash160,sizeof(subkey_hash160)); + err=m_Database->AddSubKeyDef(imp,(unsigned char*)&subkey_hash256,item_key,item_key_size,MC_SFL_SUBKEY); + if(err) + { + goto exitlbl; + } + + mc_GetCompoundHash160(&stream_subkey_hash160,entity.m_EntityID,&subkey_hash160); + + entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_CHAINPOS; + subkey_entity.Zero(); + memcpy(subkey_entity.m_EntityID,&stream_subkey_hash160,MC_TDB_ENTITY_ID_SIZE); + subkey_entity.m_EntityType=MC_TET_SUBKEY_STREAM_KEY | MC_TET_CHAINPOS; + err= m_Database->IncrementSubKey(imp,&entity,&subkey_entity,(unsigned char*)&subkey_hash160,(unsigned char*)&hash,block,0,fFound ? 0 : 1); + if(err) + { + goto exitlbl; + } + + entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_TIMERECEIVED; + subkey_entity.m_EntityType=MC_TET_SUBKEY_STREAM_KEY | MC_TET_TIMERECEIVED; + err= m_Database->IncrementSubKey(imp,&entity,&subkey_entity,(unsigned char*)&subkey_hash160,(unsigned char*)&hash,block,0,fFound ? 0 : 1); + if(err) + { + goto exitlbl; + } + + publishers_set.clear(); + for (j = 0; j < (int)tx.vin.size(); ++j) + { + int op_addr_offset,op_addr_size,is_redeem_script,sighash_type; + uint32_t publisher_flags; + const unsigned char *ptr; + + const CScript& script2 = tx.vin[j].scriptSig; + CScript::const_iterator pc2 = script2.begin(); + + ptr=mc_ExtractAddressFromInputScript((unsigned char*)(&pc2[0]),(int)(script2.end()-pc2),&op_addr_offset,&op_addr_size,&is_redeem_script,&sighash_type,0); + if(ptr) + { + if( (sighash_type == SIGHASH_ALL) || ( (sighash_type == SIGHASH_SINGLE) && (j == i) ) ) + { + subkey_hash160=Hash160(ptr+op_addr_offset,ptr+op_addr_offset+op_addr_size); + if(publishers_set.count(subkey_hash160) == 0) + { + publishers_set.insert(subkey_hash160); + subkey_hash256=0; + memcpy(&subkey_hash256,&subkey_hash160,sizeof(subkey_hash160)); + publisher_flags=MC_SFL_SUBKEY | MC_SFL_IS_ADDRESS; + if(is_redeem_script) + { + publisher_flags |= MC_SFL_IS_SCRIPTHASH; + } + err=m_Database->AddSubKeyDef(imp,(unsigned char*)&subkey_hash256,NULL,0,publisher_flags); + if(err) + { + goto exitlbl; + } + + mc_GetCompoundHash160(&stream_subkey_hash160,entity.m_EntityID,&subkey_hash160); + + entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_CHAINPOS; + subkey_entity.Zero(); + memcpy(subkey_entity.m_EntityID,&stream_subkey_hash160,MC_TDB_ENTITY_ID_SIZE); + subkey_entity.m_EntityType=MC_TET_SUBKEY_STREAM_PUBLISHER | MC_TET_CHAINPOS; + err= m_Database->IncrementSubKey(imp,&entity,&subkey_entity,(unsigned char*)&subkey_hash160,(unsigned char*)&hash,block,0,fFound ? 0 : 1); + if(err) + { + goto exitlbl; + } + + entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_TIMERECEIVED; + subkey_entity.m_EntityType=MC_TET_SUBKEY_STREAM_PUBLISHER | MC_TET_TIMERECEIVED; + err= m_Database->IncrementSubKey(imp,&entity,&subkey_entity,(unsigned char*)&subkey_hash160,(unsigned char*)&hash,block,0,fFound ? 0 : 1); + if(err) + { + goto exitlbl; + } + } + } + } + } + } + } + } + } + if(m_Mode & MC_WMD_AUTOSUBSCRIBE_STREAMS) + { + if(mc_gState->m_TmpScript->GetNumElements() == 2) + { + uint32_t new_entity_type; + mc_gState->m_TmpScript->SetElement(0); + // Should be spkn + if(mc_gState->m_TmpScript->GetNewEntityType(&new_entity_type) == 0) // New entity element + { + if(new_entity_type == MC_ENT_TYPE_STREAM) + { + entity.Zero(); + memcpy(entity.m_EntityID,(unsigned char*)&hash+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + entity.m_EntityType=MC_TET_STREAM | MC_TET_CHAINPOS; + if(imp->FindEntity(&entity) < 0) + { + if(imp->m_ImportID == 0) + { + fNewStream=true; + } + else + { + int chain_row=m_Database->m_Imports->FindEntity(&entity); + if(chain_row < 0) + { + fNewStream=true; + } + else + { + if(m_Database->m_Imports->GetEntity(chain_row)->m_Flags & MC_EFL_NOT_IN_SYNC) + { + fNewStream=true; + } + } + } + } + } + } + } + } + } + } + + if(fNewAsset) + { + entity.Zero(); + if(mc_gState->m_Features->ShortTxIDAsAssetRef()) + { + entity.m_EntityType=MC_TET_ASSET | MC_TET_CHAINPOS; + memcpy(entity.m_EntityID,(unsigned char*)&hash+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + } + else + { + if(block >= 0) + { + entity.m_EntityType=MC_TET_ASSET | MC_TET_CHAINPOS; + int tx_offset=sizeof(CBlockHeader)+block_pos->nTxOffset; + mc_PutLE(entity.m_EntityID,&block,4); + mc_PutLE(entity.m_EntityID+4,&tx_offset,4); + for(i=0;iFindEntity(&entity) >= 0) + { + if(imp->m_TmpEntities->Seek(&entity) < 0) + { + imp->m_TmpEntities->Add(&entity,NULL); + } + } + else + { + if(m_Mode & MC_WMD_AUTOSUBSCRIBE_ASSETS) + { + m_Database->AddEntity(imp,&entity,0); + imp->m_TmpEntities->Add(&entity,NULL); + entity.m_EntityType=MC_TET_ASSET | MC_TET_TIMERECEIVED; + m_Database->AddEntity(imp,&entity,0); + } + } + } + } + + for(i=0;im_TmpAssetsOut->GetCount();i++) + { + ptrOut=mc_gState->m_TmpAssetsOut->GetRow(i); + entity.Zero(); + if(mc_gState->m_Features->ShortTxIDAsAssetRef()) + { + memcpy(entity.m_EntityID,ptrOut+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + } + else + { + memcpy(entity.m_EntityID,ptrOut,MC_AST_ASSET_REF_SIZE); + } + entity.m_EntityType=MC_TET_ASSET | MC_TET_CHAINPOS; + fRelevantEntity=false; + if(imp->FindEntity(&entity) >= 0) + { + fRelevantEntity=true; + if( (imp->m_ImportID > 0) && (block < 0) ) // When replaying mempool when completing import + { + entpos=m_Database->m_Imports->FindEntity(&entity); + if(entpos >= 0) // Ignore entities already processed in the chain import + { + lpentstat=m_Database->m_Imports->GetEntity(entpos); + if( (lpentstat->m_Flags & MC_EFL_NOT_IN_SYNC) == 0 ) + { + fRelevantEntity=false; + } + } + } + } + if(fRelevantEntity) + { + if(imp->m_TmpEntities->Seek(&entity) < 0) + { + imp->m_TmpEntities->Add(&entity,NULL); + } + } + } + + + entcount=imp->m_TmpEntities->GetCount(); + + if( (imp->m_ImportID > 0) && (block < 0) ) // When replaying mempool when completing import + { + if(entcount == 0) // Skip AddTx if nothing was found + { + goto exitlbl; + } + } +// if(entcount == 0) + if(!fIsFromMe && !fIsToMe && (entcount == 0)) + { +// printf("Nothing found\n"); + goto exitlbl; + } + + if(block == 0) // Ignoring genesis coinbase + { + goto exitlbl; + } + + if( (imp->m_ImportID == 0) || (block >= 0) ) // Ignore when replaying mempool when completing import + { + if(fIsFromMe || fIsToMe) + { + entity.Zero(); + entity.m_EntityType=MC_TET_WALLET_ALL | MC_TET_CHAINPOS; + imp->m_TmpEntities->Add(&entity,NULL); + if(fIsSpendable) + { + entity.m_EntityType=MC_TET_WALLET_SPENDABLE | MC_TET_CHAINPOS; + imp->m_TmpEntities->Add(&entity,NULL); + } + } + } + + entcount=imp->m_TmpEntities->GetCount(); + + for(i=0;im_TmpEntities->GetRow(i); + memcpy(&entity,lpent,sizeof(mc_TxEntity)); + entity.m_EntityType -= MC_TET_CHAINPOS; + entity.m_EntityType |= MC_TET_TIMERECEIVED; + imp->m_TmpEntities->Add(&entity,NULL); + } + + ss.reserve(10000); + ss << *storedTx; + + txsize=ss.size(); + if(fFound) + { + txfullsize=txdef.m_FullSize; + } + else + { + txfullsize=fullTx->GetSerializeSize(SER_DISK, CLIENT_VERSION); + } + flags=0; + if(tx.IsCoinBase()) + { + flags |= MC_TFL_IS_COINBASE; + } + if(fAllInputsAreFinal) + { + flags |= MC_TFL_ALL_INPUTS_ARE_FINAL; + } + if(fIsFromMe) + { + flags |= MC_TFL_FROM_ME; + } + if(fAllInputsFromMe) + { + flags |= MC_TFL_ALL_INPUTS_FROM_ME; + } + + timestamp=mc_TimeNowAsUInt(); + if(block >= 0) + { + block_file=block_pos->nFile; + block_offset=block_pos->nPos; + block_tx_offset=block_pos->nTxOffset; + } + else + { + block_file=-1; + block_offset=0; + block_tx_offset=0; + } + + LogPrint("wallet","wtxs: Found %d entities in tx %s, flags: %08X, import %d\n",imp->m_TmpEntities->GetCount(),tx.GetHash().ToString().c_str(),flags,imp->m_ImportID); + err=m_Database->AddTx(imp,(unsigned char*)&hash,(unsigned char*)&ss[0],txsize,txfullsize,block,block_file,block_offset,block_tx_offset,block_tx_index,flags,timestamp,imp->m_TmpEntities); + if(err == MC_ERR_NOERROR) // Adding tx to unconfirmed send + { + if((block < 0) && (imp->m_ImportID == 0) && (fIsFromMe || (txsize != txfullsize))) + { + std::map::const_iterator it = m_UnconfirmedSends.find(hash); + if (it == m_UnconfirmedSends.end()) + { + err=AddToUnconfirmedSends(m_Database->m_DBStat.m_Block,tx); + } + } + } + + if(err == MC_ERR_NOERROR) // Updating UTXO map + { + for(i=0;i<(int)txoutsIn.size();i++) + { + m_UTXOs[import_pos].erase(txoutsIn[i].m_OutPoint); + } + for(i=0;i<(int)txoutsOut.size();i++) + { + std::map::const_iterator itold = m_UTXOs[import_pos].find(txoutsOut[i].m_OutPoint); + if (itold == m_UTXOs[import_pos].end()) + { + txoutsOut[i].m_Flags |= flags; + if(fSingleInputEntity) + { + if(input_entity.m_EntityType) + { + if(input_entity.m_EntityType == txoutsOut[i].m_EntityType) + { + if(memcmp(input_entity.m_EntityID,&(txoutsOut[i].m_EntityID),MC_TDB_ENTITY_ID_SIZE) == 0) + { + txoutsOut[i].m_Flags |= MC_TFL_IS_CHANGE; + } + } + } + } + m_UTXOs[import_pos].insert(make_pair(txoutsOut[i].m_OutPoint, txoutsOut[i])); + } + } + } + +exitlbl: + + if(err == MC_ERR_NOERROR) // Adding new entities + { + if(fNewStream) + { + entity.Zero(); + memcpy(entity.m_EntityID,(unsigned char*)&hash+MC_AST_SHORT_TXID_OFFSET,MC_AST_SHORT_TXID_SIZE); + entity.m_EntityType=MC_TET_STREAM | MC_TET_CHAINPOS; + m_Database->AddEntity(imp,&entity,0); + entity.m_EntityType=MC_TET_STREAM | MC_TET_TIMERECEIVED; + m_Database->AddEntity(imp,&entity,0); + entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_CHAINPOS; + m_Database->AddEntity(imp,&entity,0); + entity.m_EntityType=MC_TET_STREAM_KEY | MC_TET_TIMERECEIVED; + m_Database->AddEntity(imp,&entity,0); + entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_CHAINPOS; + m_Database->AddEntity(imp,&entity,0); + entity.m_EntityType=MC_TET_STREAM_PUBLISHER | MC_TET_TIMERECEIVED; + m_Database->AddEntity(imp,&entity,0); + } + } + + if(err) + { + LogPrintf("wtxs: AddTx Error: %d\n",err); + } + if(lockres == 0) + { + m_Database->UnLock(); + } + return err; +} + +int mc_WalletTxs::FindWalletTx(uint256 hash,mc_TxDefRow *txdef) +{ + int err,lockres; + mc_TxDefRow StoredTxDef; + + lockres=1; + if((m_Mode & MC_WMD_TXS) == 0) + { + err=MC_ERR_NOT_SUPPORTED; + goto exitlbl; + } + err = MC_ERR_NOERROR; + + if(m_Database == NULL) + { + goto exitlbl; + } + + lockres=m_Database->Lock(0,1); + + err=m_Database->GetTx(&StoredTxDef,(unsigned char*)&hash); + if(err) + { + goto exitlbl; + } + + if(txdef) + { + memcpy(txdef,&StoredTxDef,sizeof(mc_TxDefRow)); + } + +exitlbl: + + if(lockres == 0) + { + m_Database->UnLock(); + } + return err; + +} + +CWalletTx mc_WalletTxs::GetWalletTx(uint256 hash,mc_TxDefRow *txdef,int *errOut) +{ + int err; + CWalletTx wtx; + mc_TxDefRow StoredTxDef; + FILE* fHan; + char ShortName[65]; + char FileName[MC_DCT_DB_MAX_PATH]; + + if((m_Mode & MC_WMD_TXS) == 0) + { + err=MC_ERR_NOT_SUPPORTED; + goto exitlbl; + } + err = MC_ERR_NOERROR; + + if(m_Database == NULL) + { + goto exitlbl; + } + + m_Database->Lock(0,0); + + err=m_Database->GetTx(&StoredTxDef,(unsigned char*)&hash); + if(err) + { + goto exitlbl; + } + + if((StoredTxDef.m_Block >= 0) || (StoredTxDef.m_Size == StoredTxDef.m_FullSize))// We have full tx in wallet or in the block + { + sprintf(ShortName,"wallet/txs%05u",StoredTxDef.m_InternalFileID); + + mc_GetFullFileName(m_Database->m_Name,ShortName,".dat",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR,FileName); + + fHan=fopen(FileName,"rb+"); + + if(fHan == NULL) + { + goto exitlbl; + } + + CAutoFile filein(fHan, SER_DISK, CLIENT_VERSION); + + fseek(filein.Get(), StoredTxDef.m_InternalFileOffset, SEEK_SET); + try + { + filein >> wtx; // Extract wallet tx anyway - for metadata + } + catch (std::exception &e) + { + err=MC_ERR_FILE_READ_ERROR; + goto exitlbl; + } + + if(StoredTxDef.m_Size != StoredTxDef.m_FullSize) // if tx is shortened, extract OP_RETURN metadata from block + { + CDiskTxPos postx(CDiskBlockPos(StoredTxDef.m_BlockFileID,StoredTxDef.m_BlockOffset),StoredTxDef.m_BlockTxOffset); + + CAutoFile file(OpenBlockFile(postx, true), SER_DISK, CLIENT_VERSION); + if (file.IsNull()) + { + err=MC_ERR_FILE_READ_ERROR; + goto exitlbl; + } + CBlockHeader header; + CTransaction tx; + try + { + file >> header; + fseek(file.Get(), postx.nTxOffset, SEEK_CUR); + file >> tx; + } + catch (std::exception &e) + { + err=MC_ERR_FILE_READ_ERROR; + goto exitlbl; + } + + *static_cast(&wtx) = CTransaction(tx); + } + } + else // We have full tx only in unconfirmed sends + { + std::map::const_iterator it = m_UnconfirmedSends.find(hash); + if (it != m_UnconfirmedSends.end()) + { + wtx=it->second; + } + else + { + err=MC_ERR_CORRUPTED; + } + } + + +exitlbl: + + if(err == MC_ERR_NOERROR) + { + if(wtx.GetHash() != hash) + { + err=MC_ERR_CORRUPTED; + } + } + + if(err == MC_ERR_NOERROR) + { + wtx.BindWallet(m_lpWallet); + wtx.nTimeReceived=StoredTxDef.m_TimeReceived; + wtx.nTimeSmart=StoredTxDef.m_TimeReceived; + memcpy(&wtx.txDef,&StoredTxDef,sizeof(mc_TxDefRow)); + if(txdef) + { + memcpy(txdef,&StoredTxDef,sizeof(mc_TxDefRow)); + } + } + + if(errOut) + { + *errOut=err; + } + + m_Database->UnLock(); + return wtx; +} + +/* + * Extract wallet tx as it is stored in the wallet, ignoring OP_RETURN metadata + */ + +CWalletTx mc_WalletTxs::GetInternalWalletTx(uint256 hash,mc_TxDefRow *txdef,int *errOut) +{ + int err; + CWalletTx wtx; + mc_TxDefRow StoredTxDef; + char ShortName[65]; + char FileName[MC_DCT_DB_MAX_PATH]; + + err = MC_ERR_NOERROR; + + if(hash == 0) + { + err=MC_ERR_NOT_FOUND; + goto exitlbl; + } + + err=m_Database->GetTx(&StoredTxDef,(unsigned char*)&hash); + + if(err) + { + goto exitlbl; + } + else + { + sprintf(ShortName,"wallet/txs%05u",StoredTxDef.m_InternalFileID); + + mc_GetFullFileName(m_Database->m_Name,ShortName,".dat",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR,FileName); + + CAutoFile filein(fopen(FileName,"rb+"), SER_DISK, CLIENT_VERSION); + + fseek(filein.Get(), StoredTxDef.m_InternalFileOffset, SEEK_SET); + try + { + filein >> wtx; + } + catch (std::exception &e) + { + err=MC_ERR_FILE_READ_ERROR; + goto exitlbl; + } + } + +exitlbl: + + if(err == MC_ERR_NOERROR) + { + if(StoredTxDef.m_Size == StoredTxDef.m_FullSize) + { + if(wtx.GetHash() != hash) // Check it only if full transaction was found + { + err=MC_ERR_CORRUPTED; + } + } + } + + if(err == MC_ERR_NOERROR) + { + wtx.BindWallet(m_lpWallet); + wtx.nTimeReceived=StoredTxDef.m_TimeReceived; + wtx.nTimeSmart=StoredTxDef.m_TimeReceived; + memcpy(&wtx.txDef,&StoredTxDef,sizeof(mc_TxDefRow)); + if(txdef) + { + memcpy(txdef,&StoredTxDef,sizeof(mc_TxDefRow)); + } + } + + if(errOut) + { + *errOut=err; + } + + return wtx; +} + +string mc_WalletTxs::GetSubKey(void *hash,mc_TxDefRow *txdef,int *errOut) +{ + int err; + string ret; + mc_TxDefRow StoredTxDef; + char ShortName[65]; + char FileName[MC_DCT_DB_MAX_PATH]; + int fHan=0; + uint160 subkey_hash; + + err = MC_ERR_NOERROR; + + ret=""; + err=m_Database->GetTx(&StoredTxDef,(unsigned char*)hash); + + if(err) + { + goto exitlbl; + } + else + { + if(StoredTxDef.m_Flags & MC_SFL_NODATA) + { + if(StoredTxDef.m_Flags & MC_SFL_IS_ADDRESS) + { + memcpy(&subkey_hash,StoredTxDef.m_TxId,MC_TDB_ENTITY_ID_SIZE); + if(StoredTxDef.m_Flags & MC_SFL_IS_SCRIPTHASH) + { + ret=CBitcoinAddress((CScriptID)subkey_hash).ToString(); + } + else + { + ret=CBitcoinAddress((CKeyID)subkey_hash).ToString(); + } + } + } + else + { + unsigned char buf[256]; + int total_bytes_read,bytes_to_read; + + sprintf(ShortName,"wallet/txs%05u",StoredTxDef.m_InternalFileID); + + mc_GetFullFileName(m_Database->m_Name,ShortName,".dat",MC_FOM_RELATIVE_TO_DATADIR | MC_FOM_CREATE_DIR,FileName); + + fHan=open(FileName,_O_BINARY | O_RDONLY, S_IRUSR | S_IWUSR); + + if(fHan <= 0) + { + err= MC_ERR_FILE_READ_ERROR; + goto exitlbl; + + } + + if(lseek64(fHan,StoredTxDef.m_InternalFileOffset,SEEK_SET) != StoredTxDef.m_InternalFileOffset) + { + err= MC_ERR_FILE_READ_ERROR; + goto exitlbl; + } + + total_bytes_read=0; + while(total_bytes_read < (int)StoredTxDef.m_Size) + { + bytes_to_read=StoredTxDef.m_Size-total_bytes_read; + if(bytes_to_read > 256) + { + bytes_to_read=256; + } + if(read(fHan,buf,bytes_to_read) != bytes_to_read) + { + err= MC_ERR_FILE_READ_ERROR; + goto exitlbl; + } + total_bytes_read+=bytes_to_read; + ret += string((char*)buf,bytes_to_read); + } + } + } + +exitlbl: + + if(fHan) + { + close(fHan); + } + + + if(err == MC_ERR_NOERROR) + { + if(txdef) + { + memcpy(txdef,&StoredTxDef,sizeof(mc_TxDefRow)); + } + } + + if(errOut) + { + *errOut=err; + } + + return ret; +} + + +int mc_WalletTxs::GetEntityListCount() +{ + if((m_Mode & MC_WMD_TXS) == 0) + { + return 0; + } + if(m_Database == NULL) + { + return 0; + } + + return m_Database->m_Imports->m_Entities->GetCount(); +} + +mc_TxEntityStat *mc_WalletTxs::GetEntity(int row) +{ + if((m_Mode & MC_WMD_TXS) == 0) + { + return NULL; + } + if(m_Database == NULL) + { + return NULL; + } + + return m_Database->m_Imports->GetEntity(row); +} diff --git a/src/wallet/wallettxs.h b/src/wallet/wallettxs.h new file mode 100644 index 00000000..75d2e021 --- /dev/null +++ b/src/wallet/wallettxs.h @@ -0,0 +1,149 @@ +// Copyright (c) 2014-2016 The Bitcoin Core developers +// Original code was distributed under the MIT software license. +// Copyright (c) 2014-2017 Coin Sciences Ltd +// MultiChain code distributed under the GPLv3 license, see COPYING file. + +#ifndef MULTICHAIN_WALLETTXS_H +#define MULTICHAIN_WALLETTXS_H + +#include "utils/util.h" +#include "structs/base58.h" +#include "wallet/wallet.h" +#include "multichain/multichain.h" +#include "wallet/wallettxdb.h" + +#define MC_TDB_MAX_OP_RETURN_SIZE 256 + +typedef struct mc_WalletTxs +{ + mc_TxDB *m_Database; + CWallet *m_lpWallet; + uint32_t m_Mode; + std::map m_UTXOs[MC_TDB_MAX_IMPORTS]; + std::map m_UnconfirmedSends; + std::map vAvailableCoins; + + mc_WalletTxs() + { + Zero(); + } + + ~mc_WalletTxs() + { + Destroy(); + } + + int Initialize( // Initialization + const char *name, // Chain name + uint32_t mode); // Unused + + void BindWallet(CWallet *lpWallet); + + int AddEntity(mc_TxEntity *entity,uint32_t flags); // Adds entity to chain import + mc_Buffer* GetEntityList(); // Retruns list of chain import entities + + int AddTx( // Adds tx to the wallet + mc_TxImport *import, // Import object, NULL if chain update + const CTransaction& tx, // Tx to add + int block, // block height, -1 for mempool + CDiskTxPos* block_pos, // Position in the block + uint32_t block_tx_index); // Tx index in block + + + int AddTx( // Adds tx to the wallet + mc_TxImport *import, // Import object, NULL if chain update + const CWalletTx& tx, // Tx to add + int block, // block height, -1 for mempool + CDiskTxPos* block_pos, // Position in the block + uint32_t block_tx_index); // Tx index in block + + int BeforeCommit(mc_TxImport *import); // Should be called before re-adding tx while processing block + int Commit(mc_TxImport *import); // Commit when block was processed + int RollBack(mc_TxImport *import,int block); // Rollback to specific block + int CleanUpAfterBlock(mc_TxImport *import,int block,int prev_block); // Should be called after full processing of the block, prev_block - block before UpdateTip + + int FindWalletTx(uint256 hash,mc_TxDefRow *txdef); // Returns only txdef + CWalletTx GetWalletTx(uint256 hash,mc_TxDefRow *txdef,int *errOut); // Returns wallet transaction. If not found returns empty transaction + + CWalletTx GetInternalWalletTx(uint256 hash,mc_TxDefRow *txdef,int *errOut); + + int GetList(mc_TxEntity *entity, // Returns Txs in range for specific entity + int from, // If positive - from this tx, if not positive - this number from the end + int count, // Number of txs to return + mc_Buffer *txs); // Output list. mc_TxEntityRow + + int GetList(mc_TxEntity *entity, // Returns Txs in range for specific entity + int generation, // Entity generation + int from, // If positive - from this tx, if not positive - this number from the end + int count, // Number of txs to return + mc_Buffer *txs); // Output list. mc_TxEntityRow + + int GetListSize( // Return total number of tx in the list for specific entity + mc_TxEntity *entity, // Entity to return info for + int *confirmed); // Out: number of confirmed items + + int GetListSize( // Return total number of tx in the list for specific entity + mc_TxEntity *entity, // Entity to return info for + int generation, // Entity generation + int *confirmed); // Out: number of confirmed items + + int GetRow( + mc_TxEntityRow *erow); + + int Unsubscribe(mc_Buffer *lpEntities); // List of the entities to unsubscribe from + + mc_TxImport *StartImport( // Starts new import + mc_Buffer *lpEntities, // List of entities to import + int block, // Star from this block + int *err); // Output. Error + + mc_TxImport *FindImport( // Find import with specific ID + int import_id); + + bool FindEntity(mc_TxEntityStat *entity); // Finds entity in chain import + + int ImportGetBlock( // Returns last processed block in the import + mc_TxImport *import); + + int CompleteImport(mc_TxImport *import); // Completes import - merges with chain + + int DropImport(mc_TxImport *import); // Drops uncompleted import + + std::string GetSubKey(void *hash,mc_TxDefRow *txdef,int *errOut); + + int GetEntityListCount(); + mc_TxEntityStat *GetEntity(int row); + + +// Internal functions + + void Zero(); + int Destroy(); + + int AddToUnconfirmedSends(int block,const CWalletTx& tx); + int SaveTxFlag( // Changes tx flag setting (if tx is found)) + const unsigned char *hash, // Tx ID + uint32_t flag, // Flag to set/unset + int set_flag); // 1 if set, 0 if unset + + int RollBackSubKeys(mc_TxImport *import,int block,mc_TxEntityStat *parent_entity,mc_Buffer *lpSubKeyEntRowBuffer); // Rollback subkeys to specific block + + std::map GetUnconfirmedSends(int block); // Internal. Retrieves list of unconfirmed txs sent by this wallet for specific block + void GetSingleInputEntity(const CWalletTx& tx,mc_TxEntity *input_entity); + int RemoveUnconfirmedSends(int block); + int SaveUTXOMap(int import_id,int block); + int LoadUTXOMap(int import_id,int block); + int RemoveUTXOMap(int import_id,int block); + int GetBlock(); + + void Lock(); + void UnLock(); + + + +} mc_WalletTxs; + + + +#endif /* MULTICHAIN_WALLETTXS_H */ +