adding pods method of package managing
This commit is contained in:
talksik
Generated
+203
@@ -0,0 +1,203 @@
|
||||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
https://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
https://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Generated
+114
@@ -0,0 +1,114 @@
|
||||
# Abseil - C++ Common Libraries
|
||||
|
||||
The repository contains the Abseil C++ library code. Abseil is an open-source
|
||||
collection of C++ code (compliant to C++11) designed to augment the C++
|
||||
standard library.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [About Abseil](#about)
|
||||
- [Quickstart](#quickstart)
|
||||
- [Building Abseil](#build)
|
||||
- [Codemap](#codemap)
|
||||
- [License](#license)
|
||||
- [Links](#links)
|
||||
|
||||
<a name="about"></a>
|
||||
## About Abseil
|
||||
|
||||
Abseil is an open-source collection of C++ library code designed to augment
|
||||
the C++ standard library. The Abseil library code is collected from Google's
|
||||
own C++ code base, has been extensively tested and used in production, and
|
||||
is the same code we depend on in our daily coding lives.
|
||||
|
||||
In some cases, Abseil provides pieces missing from the C++ standard; in
|
||||
others, Abseil provides alternatives to the standard for special needs
|
||||
we've found through usage in the Google code base. We denote those cases
|
||||
clearly within the library code we provide you.
|
||||
|
||||
Abseil is not meant to be a competitor to the standard library; we've
|
||||
just found that many of these utilities serve a purpose within our code
|
||||
base, and we now want to provide those resources to the C++ community as
|
||||
a whole.
|
||||
|
||||
<a name="quickstart"></a>
|
||||
## Quickstart
|
||||
|
||||
If you want to just get started, make sure you at least run through the
|
||||
[Abseil Quickstart](https://abseil.io/docs/cpp/quickstart). The Quickstart
|
||||
contains information about setting up your development environment, downloading
|
||||
the Abseil code, running tests, and getting a simple binary working.
|
||||
|
||||
<a name="build"></a>
|
||||
## Building Abseil
|
||||
|
||||
[Bazel](https://bazel.build) is the official build system for Abseil,
|
||||
which is supported on most major platforms (Linux, Windows, macOS, for example)
|
||||
and compilers. See the [quickstart](https://abseil.io/docs/cpp/quickstart) for
|
||||
more information on building Abseil using the Bazel build system.
|
||||
|
||||
<a name="cmake"></a>
|
||||
If you require CMake support, please check the
|
||||
[CMake build instructions](CMake/README.md).
|
||||
|
||||
## Codemap
|
||||
|
||||
Abseil contains the following C++ library components:
|
||||
|
||||
* [`base`](absl/base/) Abseil Fundamentals
|
||||
<br /> The `base` library contains initialization code and other code which
|
||||
all other Abseil code depends on. Code within `base` may not depend on any
|
||||
other code (other than the C++ standard library).
|
||||
* [`algorithm`](absl/algorithm/)
|
||||
<br /> The `algorithm` library contains additions to the C++ `<algorithm>`
|
||||
library and container-based versions of such algorithms.
|
||||
* [`container`](absl/container/)
|
||||
<br /> The `container` library contains additional STL-style containers,
|
||||
including Abseil's unordered "Swiss table" containers.
|
||||
* [`debugging`](absl/debugging/)
|
||||
<br /> The `debugging` library contains code useful for enabling leak
|
||||
checks, and stacktrace and symbolization utilities.
|
||||
* [`hash`](absl/hash/)
|
||||
<br /> The `hash` library contains the hashing framework and default hash
|
||||
functor implementations for hashable types in Abseil.
|
||||
* [`memory`](absl/memory/)
|
||||
<br /> The `memory` library contains C++11-compatible versions of
|
||||
`std::make_unique()` and related memory management facilities.
|
||||
* [`meta`](absl/meta/)
|
||||
<br /> The `meta` library contains C++11-compatible versions of type checks
|
||||
available within C++14 and C++17 versions of the C++ `<type_traits>` library.
|
||||
* [`numeric`](absl/numeric/)
|
||||
<br /> The `numeric` library contains C++11-compatible 128-bit integers.
|
||||
* [`strings`](absl/strings/)
|
||||
<br /> The `strings` library contains a variety of strings routines and
|
||||
utilities, including a C++11-compatible version of the C++17
|
||||
`std::string_view` type.
|
||||
* [`synchronization`](absl/synchronization/)
|
||||
<br /> The `synchronization` library contains concurrency primitives (Abseil's
|
||||
`absl::Mutex` class, an alternative to `std::mutex`) and a variety of
|
||||
synchronization abstractions.
|
||||
* [`time`](absl/time/)
|
||||
<br /> The `time` library contains abstractions for computing with absolute
|
||||
points in time, durations of time, and formatting and parsing time within
|
||||
time zones.
|
||||
* [`types`](absl/types/)
|
||||
<br /> The `types` library contains non-container utility types, like a
|
||||
C++11-compatible version of the C++17 `std::optional` type.
|
||||
* [`utility`](absl/utility/)
|
||||
<br /> The `utility` library contains utility and helper code.
|
||||
|
||||
## License
|
||||
|
||||
The Abseil C++ library is licensed under the terms of the Apache
|
||||
license. See [LICENSE](LICENSE) for more information.
|
||||
|
||||
## Links
|
||||
|
||||
For more information about Abseil:
|
||||
|
||||
* Consult our [Abseil Introduction](https://abseil.io/about/intro)
|
||||
* Read [Why Adopt Abseil](https://abseil.io/about/philosophy) to understand our
|
||||
design philosophy.
|
||||
* Peruse our
|
||||
[Abseil Compatibility Guarantees](https://abseil.io/about/compatibility) to
|
||||
understand both what we promise to you, and what we expect of you in return.
|
||||
Generated
+159
@@ -0,0 +1,159 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: algorithm.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This header file contains Google extensions to the standard <algorithm> C++
|
||||
// header.
|
||||
|
||||
#ifndef ABSL_ALGORITHM_ALGORITHM_H_
|
||||
#define ABSL_ALGORITHM_ALGORITHM_H_
|
||||
|
||||
#include <algorithm>
|
||||
#include <iterator>
|
||||
#include <type_traits>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
|
||||
namespace algorithm_internal {
|
||||
|
||||
// Performs comparisons with operator==, similar to C++14's `std::equal_to<>`.
|
||||
struct EqualTo {
|
||||
template <typename T, typename U>
|
||||
bool operator()(const T& a, const U& b) const {
|
||||
return a == b;
|
||||
}
|
||||
};
|
||||
|
||||
template <typename InputIter1, typename InputIter2, typename Pred>
|
||||
bool EqualImpl(InputIter1 first1, InputIter1 last1, InputIter2 first2,
|
||||
InputIter2 last2, Pred pred, std::input_iterator_tag,
|
||||
std::input_iterator_tag) {
|
||||
while (true) {
|
||||
if (first1 == last1) return first2 == last2;
|
||||
if (first2 == last2) return false;
|
||||
if (!pred(*first1, *first2)) return false;
|
||||
++first1;
|
||||
++first2;
|
||||
}
|
||||
}
|
||||
|
||||
template <typename InputIter1, typename InputIter2, typename Pred>
|
||||
bool EqualImpl(InputIter1 first1, InputIter1 last1, InputIter2 first2,
|
||||
InputIter2 last2, Pred&& pred, std::random_access_iterator_tag,
|
||||
std::random_access_iterator_tag) {
|
||||
return (last1 - first1 == last2 - first2) &&
|
||||
std::equal(first1, last1, first2, std::forward<Pred>(pred));
|
||||
}
|
||||
|
||||
// When we are using our own internal predicate that just applies operator==, we
|
||||
// forward to the non-predicate form of std::equal. This enables an optimization
|
||||
// in libstdc++ that can result in std::memcmp being used for integer types.
|
||||
template <typename InputIter1, typename InputIter2>
|
||||
bool EqualImpl(InputIter1 first1, InputIter1 last1, InputIter2 first2,
|
||||
InputIter2 last2, algorithm_internal::EqualTo /* unused */,
|
||||
std::random_access_iterator_tag,
|
||||
std::random_access_iterator_tag) {
|
||||
return (last1 - first1 == last2 - first2) &&
|
||||
std::equal(first1, last1, first2);
|
||||
}
|
||||
|
||||
template <typename It>
|
||||
It RotateImpl(It first, It middle, It last, std::true_type) {
|
||||
return std::rotate(first, middle, last);
|
||||
}
|
||||
|
||||
template <typename It>
|
||||
It RotateImpl(It first, It middle, It last, std::false_type) {
|
||||
std::rotate(first, middle, last);
|
||||
return std::next(first, std::distance(middle, last));
|
||||
}
|
||||
|
||||
} // namespace algorithm_internal
|
||||
|
||||
// equal()
|
||||
//
|
||||
// Compares the equality of two ranges specified by pairs of iterators, using
|
||||
// the given predicate, returning true iff for each corresponding iterator i1
|
||||
// and i2 in the first and second range respectively, pred(*i1, *i2) == true
|
||||
//
|
||||
// This comparison takes at most min(`last1` - `first1`, `last2` - `first2`)
|
||||
// invocations of the predicate. Additionally, if InputIter1 and InputIter2 are
|
||||
// both random-access iterators, and `last1` - `first1` != `last2` - `first2`,
|
||||
// then the predicate is never invoked and the function returns false.
|
||||
//
|
||||
// This is a C++11-compatible implementation of C++14 `std::equal`. See
|
||||
// https://en.cppreference.com/w/cpp/algorithm/equal for more information.
|
||||
template <typename InputIter1, typename InputIter2, typename Pred>
|
||||
bool equal(InputIter1 first1, InputIter1 last1, InputIter2 first2,
|
||||
InputIter2 last2, Pred&& pred) {
|
||||
return algorithm_internal::EqualImpl(
|
||||
first1, last1, first2, last2, std::forward<Pred>(pred),
|
||||
typename std::iterator_traits<InputIter1>::iterator_category{},
|
||||
typename std::iterator_traits<InputIter2>::iterator_category{});
|
||||
}
|
||||
|
||||
// Overload of equal() that performs comparison of two ranges specified by pairs
|
||||
// of iterators using operator==.
|
||||
template <typename InputIter1, typename InputIter2>
|
||||
bool equal(InputIter1 first1, InputIter1 last1, InputIter2 first2,
|
||||
InputIter2 last2) {
|
||||
return absl::equal(first1, last1, first2, last2,
|
||||
algorithm_internal::EqualTo{});
|
||||
}
|
||||
|
||||
// linear_search()
|
||||
//
|
||||
// Performs a linear search for `value` using the iterator `first` up to
|
||||
// but not including `last`, returning true if [`first`, `last`) contains an
|
||||
// element equal to `value`.
|
||||
//
|
||||
// A linear search is of O(n) complexity which is guaranteed to make at most
|
||||
// n = (`last` - `first`) comparisons. A linear search over short containers
|
||||
// may be faster than a binary search, even when the container is sorted.
|
||||
template <typename InputIterator, typename EqualityComparable>
|
||||
bool linear_search(InputIterator first, InputIterator last,
|
||||
const EqualityComparable& value) {
|
||||
return std::find(first, last, value) != last;
|
||||
}
|
||||
|
||||
// rotate()
|
||||
//
|
||||
// Performs a left rotation on a range of elements (`first`, `last`) such that
|
||||
// `middle` is now the first element. `rotate()` returns an iterator pointing to
|
||||
// the first element before rotation. This function is exactly the same as
|
||||
// `std::rotate`, but fixes a bug in gcc
|
||||
// <= 4.9 where `std::rotate` returns `void` instead of an iterator.
|
||||
//
|
||||
// The complexity of this algorithm is the same as that of `std::rotate`, but if
|
||||
// `ForwardIterator` is not a random-access iterator, then `absl::rotate`
|
||||
// performs an additional pass over the range to construct the return value.
|
||||
template <typename ForwardIterator>
|
||||
ForwardIterator rotate(ForwardIterator first, ForwardIterator middle,
|
||||
ForwardIterator last) {
|
||||
return algorithm_internal::RotateImpl(
|
||||
first, middle, last,
|
||||
std::is_same<decltype(std::rotate(first, middle, last)),
|
||||
ForwardIterator>());
|
||||
}
|
||||
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_ALGORITHM_ALGORITHM_H_
|
||||
Generated
+1727
File diff suppressed because it is too large
Load Diff
Generated
+621
@@ -0,0 +1,621 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// This header file defines macros for declaring attributes for functions,
|
||||
// types, and variables.
|
||||
//
|
||||
// These macros are used within Abseil and allow the compiler to optimize, where
|
||||
// applicable, certain function calls.
|
||||
//
|
||||
// This file is used for both C and C++!
|
||||
//
|
||||
// Most macros here are exposing GCC or Clang features, and are stubbed out for
|
||||
// other compilers.
|
||||
//
|
||||
// GCC attributes documentation:
|
||||
// https://gcc.gnu.org/onlinedocs/gcc-4.7.0/gcc/Function-Attributes.html
|
||||
// https://gcc.gnu.org/onlinedocs/gcc-4.7.0/gcc/Variable-Attributes.html
|
||||
// https://gcc.gnu.org/onlinedocs/gcc-4.7.0/gcc/Type-Attributes.html
|
||||
//
|
||||
// Most attributes in this file are already supported by GCC 4.7. However, some
|
||||
// of them are not supported in older version of Clang. Thus, we check
|
||||
// `__has_attribute()` first. If the check fails, we check if we are on GCC and
|
||||
// assume the attribute exists on GCC (which is verified on GCC 4.7).
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// Sanitizer Attributes
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// Sanitizer-related attributes are not "defined" in this file (and indeed
|
||||
// are not defined as such in any file). To utilize the following
|
||||
// sanitizer-related attributes within your builds, define the following macros
|
||||
// within your build using a `-D` flag, along with the given value for
|
||||
// `-fsanitize`:
|
||||
//
|
||||
// * `ADDRESS_SANITIZER` + `-fsanitize=address` (Clang, GCC 4.8)
|
||||
// * `MEMORY_SANITIZER` + `-fsanitize=memory` (Clang-only)
|
||||
// * `THREAD_SANITIZER + `-fsanitize=thread` (Clang, GCC 4.8+)
|
||||
// * `UNDEFINED_BEHAVIOR_SANITIZER` + `-fsanitize=undefined` (Clang, GCC 4.9+)
|
||||
// * `CONTROL_FLOW_INTEGRITY` + -fsanitize=cfi (Clang-only)
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// // Enable branches in the Abseil code that are tagged for ASan:
|
||||
// $ bazel build --copt=-DADDRESS_SANITIZER --copt=-fsanitize=address
|
||||
// --linkopt=-fsanitize=address *target*
|
||||
//
|
||||
// Since these macro names are only supported by GCC and Clang, we only check
|
||||
// for `__GNUC__` (GCC or Clang) and the above macros.
|
||||
#ifndef ABSL_BASE_ATTRIBUTES_H_
|
||||
#define ABSL_BASE_ATTRIBUTES_H_
|
||||
|
||||
// ABSL_HAVE_ATTRIBUTE
|
||||
//
|
||||
// A function-like feature checking macro that is a wrapper around
|
||||
// `__has_attribute`, which is defined by GCC 5+ and Clang and evaluates to a
|
||||
// nonzero constant integer if the attribute is supported or 0 if not.
|
||||
//
|
||||
// It evaluates to zero if `__has_attribute` is not defined by the compiler.
|
||||
//
|
||||
// GCC: https://gcc.gnu.org/gcc-5/changes.html
|
||||
// Clang: https://clang.llvm.org/docs/LanguageExtensions.html
|
||||
#ifdef __has_attribute
|
||||
#define ABSL_HAVE_ATTRIBUTE(x) __has_attribute(x)
|
||||
#else
|
||||
#define ABSL_HAVE_ATTRIBUTE(x) 0
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_CPP_ATTRIBUTE
|
||||
//
|
||||
// A function-like feature checking macro that accepts C++11 style attributes.
|
||||
// It's a wrapper around `__has_cpp_attribute`, defined by ISO C++ SD-6
|
||||
// (https://en.cppreference.com/w/cpp/experimental/feature_test). If we don't
|
||||
// find `__has_cpp_attribute`, will evaluate to 0.
|
||||
#if defined(__cplusplus) && defined(__has_cpp_attribute)
|
||||
// NOTE: requiring __cplusplus above should not be necessary, but
|
||||
// works around https://bugs.llvm.org/show_bug.cgi?id=23435.
|
||||
#define ABSL_HAVE_CPP_ATTRIBUTE(x) __has_cpp_attribute(x)
|
||||
#else
|
||||
#define ABSL_HAVE_CPP_ATTRIBUTE(x) 0
|
||||
#endif
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// Function Attributes
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// GCC: https://gcc.gnu.org/onlinedocs/gcc/Function-Attributes.html
|
||||
// Clang: https://clang.llvm.org/docs/AttributeReference.html
|
||||
|
||||
// ABSL_PRINTF_ATTRIBUTE
|
||||
// ABSL_SCANF_ATTRIBUTE
|
||||
//
|
||||
// Tells the compiler to perform `printf` format string checking if the
|
||||
// compiler supports it; see the 'format' attribute in
|
||||
// <https://gcc.gnu.org/onlinedocs/gcc-4.7.0/gcc/Function-Attributes.html>.
|
||||
//
|
||||
// Note: As the GCC manual states, "[s]ince non-static C++ methods
|
||||
// have an implicit 'this' argument, the arguments of such methods
|
||||
// should be counted from two, not one."
|
||||
#if ABSL_HAVE_ATTRIBUTE(format) || (defined(__GNUC__) && !defined(__clang__))
|
||||
#define ABSL_PRINTF_ATTRIBUTE(string_index, first_to_check) \
|
||||
__attribute__((__format__(__printf__, string_index, first_to_check)))
|
||||
#define ABSL_SCANF_ATTRIBUTE(string_index, first_to_check) \
|
||||
__attribute__((__format__(__scanf__, string_index, first_to_check)))
|
||||
#else
|
||||
#define ABSL_PRINTF_ATTRIBUTE(string_index, first_to_check)
|
||||
#define ABSL_SCANF_ATTRIBUTE(string_index, first_to_check)
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_ALWAYS_INLINE
|
||||
// ABSL_ATTRIBUTE_NOINLINE
|
||||
//
|
||||
// Forces functions to either inline or not inline. Introduced in gcc 3.1.
|
||||
#if ABSL_HAVE_ATTRIBUTE(always_inline) || \
|
||||
(defined(__GNUC__) && !defined(__clang__))
|
||||
#define ABSL_ATTRIBUTE_ALWAYS_INLINE __attribute__((always_inline))
|
||||
#define ABSL_HAVE_ATTRIBUTE_ALWAYS_INLINE 1
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_ALWAYS_INLINE
|
||||
#endif
|
||||
|
||||
#if ABSL_HAVE_ATTRIBUTE(noinline) || (defined(__GNUC__) && !defined(__clang__))
|
||||
#define ABSL_ATTRIBUTE_NOINLINE __attribute__((noinline))
|
||||
#define ABSL_HAVE_ATTRIBUTE_NOINLINE 1
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_NOINLINE
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_NO_TAIL_CALL
|
||||
//
|
||||
// Prevents the compiler from optimizing away stack frames for functions which
|
||||
// end in a call to another function.
|
||||
#if ABSL_HAVE_ATTRIBUTE(disable_tail_calls)
|
||||
#define ABSL_HAVE_ATTRIBUTE_NO_TAIL_CALL 1
|
||||
#define ABSL_ATTRIBUTE_NO_TAIL_CALL __attribute__((disable_tail_calls))
|
||||
#elif defined(__GNUC__) && !defined(__clang__)
|
||||
#define ABSL_HAVE_ATTRIBUTE_NO_TAIL_CALL 1
|
||||
#define ABSL_ATTRIBUTE_NO_TAIL_CALL \
|
||||
__attribute__((optimize("no-optimize-sibling-calls")))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_NO_TAIL_CALL
|
||||
#define ABSL_HAVE_ATTRIBUTE_NO_TAIL_CALL 0
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_WEAK
|
||||
//
|
||||
// Tags a function as weak for the purposes of compilation and linking.
|
||||
// Weak attributes currently do not work properly in LLVM's Windows backend,
|
||||
// so disable them there. See https://bugs.llvm.org/show_bug.cgi?id=37598
|
||||
// for further information.
|
||||
// The MinGW compiler doesn't complain about the weak attribute until the link
|
||||
// step, presumably because Windows doesn't use ELF binaries.
|
||||
#if (ABSL_HAVE_ATTRIBUTE(weak) || \
|
||||
(defined(__GNUC__) && !defined(__clang__))) && \
|
||||
!(defined(__llvm__) && defined(_WIN32)) && !defined(__MINGW32__)
|
||||
#undef ABSL_ATTRIBUTE_WEAK
|
||||
#define ABSL_ATTRIBUTE_WEAK __attribute__((weak))
|
||||
#define ABSL_HAVE_ATTRIBUTE_WEAK 1
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_WEAK
|
||||
#define ABSL_HAVE_ATTRIBUTE_WEAK 0
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_NONNULL
|
||||
//
|
||||
// Tells the compiler either (a) that a particular function parameter
|
||||
// should be a non-null pointer, or (b) that all pointer arguments should
|
||||
// be non-null.
|
||||
//
|
||||
// Note: As the GCC manual states, "[s]ince non-static C++ methods
|
||||
// have an implicit 'this' argument, the arguments of such methods
|
||||
// should be counted from two, not one."
|
||||
//
|
||||
// Args are indexed starting at 1.
|
||||
//
|
||||
// For non-static class member functions, the implicit `this` argument
|
||||
// is arg 1, and the first explicit argument is arg 2. For static class member
|
||||
// functions, there is no implicit `this`, and the first explicit argument is
|
||||
// arg 1.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// /* arg_a cannot be null, but arg_b can */
|
||||
// void Function(void* arg_a, void* arg_b) ABSL_ATTRIBUTE_NONNULL(1);
|
||||
//
|
||||
// class C {
|
||||
// /* arg_a cannot be null, but arg_b can */
|
||||
// void Method(void* arg_a, void* arg_b) ABSL_ATTRIBUTE_NONNULL(2);
|
||||
//
|
||||
// /* arg_a cannot be null, but arg_b can */
|
||||
// static void StaticMethod(void* arg_a, void* arg_b)
|
||||
// ABSL_ATTRIBUTE_NONNULL(1);
|
||||
// };
|
||||
//
|
||||
// If no arguments are provided, then all pointer arguments should be non-null.
|
||||
//
|
||||
// /* No pointer arguments may be null. */
|
||||
// void Function(void* arg_a, void* arg_b, int arg_c) ABSL_ATTRIBUTE_NONNULL();
|
||||
//
|
||||
// NOTE: The GCC nonnull attribute actually accepts a list of arguments, but
|
||||
// ABSL_ATTRIBUTE_NONNULL does not.
|
||||
#if ABSL_HAVE_ATTRIBUTE(nonnull) || (defined(__GNUC__) && !defined(__clang__))
|
||||
#define ABSL_ATTRIBUTE_NONNULL(arg_index) __attribute__((nonnull(arg_index)))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_NONNULL(...)
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_NORETURN
|
||||
//
|
||||
// Tells the compiler that a given function never returns.
|
||||
#if ABSL_HAVE_ATTRIBUTE(noreturn) || (defined(__GNUC__) && !defined(__clang__))
|
||||
#define ABSL_ATTRIBUTE_NORETURN __attribute__((noreturn))
|
||||
#elif defined(_MSC_VER)
|
||||
#define ABSL_ATTRIBUTE_NORETURN __declspec(noreturn)
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_NORETURN
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_NO_SANITIZE_ADDRESS
|
||||
//
|
||||
// Tells the AddressSanitizer (or other memory testing tools) to ignore a given
|
||||
// function. Useful for cases when a function reads random locations on stack,
|
||||
// calls _exit from a cloned subprocess, deliberately accesses buffer
|
||||
// out of bounds or does other scary things with memory.
|
||||
// NOTE: GCC supports AddressSanitizer(asan) since 4.8.
|
||||
// https://gcc.gnu.org/gcc-4.8/changes.html
|
||||
#if defined(__GNUC__)
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_ADDRESS __attribute__((no_sanitize_address))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_ADDRESS
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_NO_SANITIZE_MEMORY
|
||||
//
|
||||
// Tells the MemorySanitizer to relax the handling of a given function. All
|
||||
// "Use of uninitialized value" warnings from such functions will be suppressed,
|
||||
// and all values loaded from memory will be considered fully initialized.
|
||||
// This attribute is similar to the ADDRESS_SANITIZER attribute above, but deals
|
||||
// with initialized-ness rather than addressability issues.
|
||||
// NOTE: MemorySanitizer(msan) is supported by Clang but not GCC.
|
||||
#if defined(__clang__)
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_MEMORY __attribute__((no_sanitize_memory))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_MEMORY
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_NO_SANITIZE_THREAD
|
||||
//
|
||||
// Tells the ThreadSanitizer to not instrument a given function.
|
||||
// NOTE: GCC supports ThreadSanitizer(tsan) since 4.8.
|
||||
// https://gcc.gnu.org/gcc-4.8/changes.html
|
||||
#if defined(__GNUC__)
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_THREAD __attribute__((no_sanitize_thread))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_THREAD
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_NO_SANITIZE_UNDEFINED
|
||||
//
|
||||
// Tells the UndefinedSanitizer to ignore a given function. Useful for cases
|
||||
// where certain behavior (eg. division by zero) is being used intentionally.
|
||||
// NOTE: GCC supports UndefinedBehaviorSanitizer(ubsan) since 4.9.
|
||||
// https://gcc.gnu.org/gcc-4.9/changes.html
|
||||
#if defined(__GNUC__) && \
|
||||
(defined(UNDEFINED_BEHAVIOR_SANITIZER) || defined(ADDRESS_SANITIZER))
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_UNDEFINED \
|
||||
__attribute__((no_sanitize("undefined")))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_UNDEFINED
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_NO_SANITIZE_CFI
|
||||
//
|
||||
// Tells the ControlFlowIntegrity sanitizer to not instrument a given function.
|
||||
// See https://clang.llvm.org/docs/ControlFlowIntegrity.html for details.
|
||||
#if defined(__GNUC__) && defined(CONTROL_FLOW_INTEGRITY)
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_CFI __attribute__((no_sanitize("cfi")))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_CFI
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_NO_SANITIZE_SAFESTACK
|
||||
//
|
||||
// Tells the SafeStack to not instrument a given function.
|
||||
// See https://clang.llvm.org/docs/SafeStack.html for details.
|
||||
#if defined(__GNUC__) && defined(SAFESTACK_SANITIZER)
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_SAFESTACK \
|
||||
__attribute__((no_sanitize("safe-stack")))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_NO_SANITIZE_SAFESTACK
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_RETURNS_NONNULL
|
||||
//
|
||||
// Tells the compiler that a particular function never returns a null pointer.
|
||||
#if ABSL_HAVE_ATTRIBUTE(returns_nonnull) || \
|
||||
(defined(__GNUC__) && \
|
||||
(__GNUC__ > 5 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 9)) && \
|
||||
!defined(__clang__))
|
||||
#define ABSL_ATTRIBUTE_RETURNS_NONNULL __attribute__((returns_nonnull))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_RETURNS_NONNULL
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_ATTRIBUTE_SECTION
|
||||
//
|
||||
// Indicates whether labeled sections are supported. Weak symbol support is
|
||||
// a prerequisite. Labeled sections are not supported on Darwin/iOS.
|
||||
#ifdef ABSL_HAVE_ATTRIBUTE_SECTION
|
||||
#error ABSL_HAVE_ATTRIBUTE_SECTION cannot be directly set
|
||||
#elif (ABSL_HAVE_ATTRIBUTE(section) || \
|
||||
(defined(__GNUC__) && !defined(__clang__))) && \
|
||||
!defined(__APPLE__) && ABSL_HAVE_ATTRIBUTE_WEAK
|
||||
#define ABSL_HAVE_ATTRIBUTE_SECTION 1
|
||||
|
||||
// ABSL_ATTRIBUTE_SECTION
|
||||
//
|
||||
// Tells the compiler/linker to put a given function into a section and define
|
||||
// `__start_ ## name` and `__stop_ ## name` symbols to bracket the section.
|
||||
// This functionality is supported by GNU linker. Any function annotated with
|
||||
// `ABSL_ATTRIBUTE_SECTION` must not be inlined, or it will be placed into
|
||||
// whatever section its caller is placed into.
|
||||
//
|
||||
#ifndef ABSL_ATTRIBUTE_SECTION
|
||||
#define ABSL_ATTRIBUTE_SECTION(name) \
|
||||
__attribute__((section(#name))) __attribute__((noinline))
|
||||
#endif
|
||||
|
||||
|
||||
// ABSL_ATTRIBUTE_SECTION_VARIABLE
|
||||
//
|
||||
// Tells the compiler/linker to put a given variable into a section and define
|
||||
// `__start_ ## name` and `__stop_ ## name` symbols to bracket the section.
|
||||
// This functionality is supported by GNU linker.
|
||||
#ifndef ABSL_ATTRIBUTE_SECTION_VARIABLE
|
||||
#define ABSL_ATTRIBUTE_SECTION_VARIABLE(name) __attribute__((section(#name)))
|
||||
#endif
|
||||
|
||||
// ABSL_DECLARE_ATTRIBUTE_SECTION_VARS
|
||||
//
|
||||
// A weak section declaration to be used as a global declaration
|
||||
// for ABSL_ATTRIBUTE_SECTION_START|STOP(name) to compile and link
|
||||
// even without functions with ABSL_ATTRIBUTE_SECTION(name).
|
||||
// ABSL_DEFINE_ATTRIBUTE_SECTION should be in the exactly one file; it's
|
||||
// a no-op on ELF but not on Mach-O.
|
||||
//
|
||||
#ifndef ABSL_DECLARE_ATTRIBUTE_SECTION_VARS
|
||||
#define ABSL_DECLARE_ATTRIBUTE_SECTION_VARS(name) \
|
||||
extern char __start_##name[] ABSL_ATTRIBUTE_WEAK; \
|
||||
extern char __stop_##name[] ABSL_ATTRIBUTE_WEAK
|
||||
#endif
|
||||
#ifndef ABSL_DEFINE_ATTRIBUTE_SECTION_VARS
|
||||
#define ABSL_INIT_ATTRIBUTE_SECTION_VARS(name)
|
||||
#define ABSL_DEFINE_ATTRIBUTE_SECTION_VARS(name)
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_SECTION_START
|
||||
//
|
||||
// Returns `void*` pointers to start/end of a section of code with
|
||||
// functions having ABSL_ATTRIBUTE_SECTION(name).
|
||||
// Returns 0 if no such functions exist.
|
||||
// One must ABSL_DECLARE_ATTRIBUTE_SECTION_VARS(name) for this to compile and
|
||||
// link.
|
||||
//
|
||||
#define ABSL_ATTRIBUTE_SECTION_START(name) \
|
||||
(reinterpret_cast<void *>(__start_##name))
|
||||
#define ABSL_ATTRIBUTE_SECTION_STOP(name) \
|
||||
(reinterpret_cast<void *>(__stop_##name))
|
||||
|
||||
#else // !ABSL_HAVE_ATTRIBUTE_SECTION
|
||||
|
||||
#define ABSL_HAVE_ATTRIBUTE_SECTION 0
|
||||
|
||||
// provide dummy definitions
|
||||
#define ABSL_ATTRIBUTE_SECTION(name)
|
||||
#define ABSL_ATTRIBUTE_SECTION_VARIABLE(name)
|
||||
#define ABSL_INIT_ATTRIBUTE_SECTION_VARS(name)
|
||||
#define ABSL_DEFINE_ATTRIBUTE_SECTION_VARS(name)
|
||||
#define ABSL_DECLARE_ATTRIBUTE_SECTION_VARS(name)
|
||||
#define ABSL_ATTRIBUTE_SECTION_START(name) (reinterpret_cast<void *>(0))
|
||||
#define ABSL_ATTRIBUTE_SECTION_STOP(name) (reinterpret_cast<void *>(0))
|
||||
|
||||
#endif // ABSL_ATTRIBUTE_SECTION
|
||||
|
||||
// ABSL_ATTRIBUTE_STACK_ALIGN_FOR_OLD_LIBC
|
||||
//
|
||||
// Support for aligning the stack on 32-bit x86.
|
||||
#if ABSL_HAVE_ATTRIBUTE(force_align_arg_pointer) || \
|
||||
(defined(__GNUC__) && !defined(__clang__))
|
||||
#if defined(__i386__)
|
||||
#define ABSL_ATTRIBUTE_STACK_ALIGN_FOR_OLD_LIBC \
|
||||
__attribute__((force_align_arg_pointer))
|
||||
#define ABSL_REQUIRE_STACK_ALIGN_TRAMPOLINE (0)
|
||||
#elif defined(__x86_64__)
|
||||
#define ABSL_REQUIRE_STACK_ALIGN_TRAMPOLINE (1)
|
||||
#define ABSL_ATTRIBUTE_STACK_ALIGN_FOR_OLD_LIBC
|
||||
#else // !__i386__ && !__x86_64
|
||||
#define ABSL_REQUIRE_STACK_ALIGN_TRAMPOLINE (0)
|
||||
#define ABSL_ATTRIBUTE_STACK_ALIGN_FOR_OLD_LIBC
|
||||
#endif // __i386__
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_STACK_ALIGN_FOR_OLD_LIBC
|
||||
#define ABSL_REQUIRE_STACK_ALIGN_TRAMPOLINE (0)
|
||||
#endif
|
||||
|
||||
// ABSL_MUST_USE_RESULT
|
||||
//
|
||||
// Tells the compiler to warn about unused results.
|
||||
//
|
||||
// When annotating a function, it must appear as the first part of the
|
||||
// declaration or definition. The compiler will warn if the return value from
|
||||
// such a function is unused:
|
||||
//
|
||||
// ABSL_MUST_USE_RESULT Sprocket* AllocateSprocket();
|
||||
// AllocateSprocket(); // Triggers a warning.
|
||||
//
|
||||
// When annotating a class, it is equivalent to annotating every function which
|
||||
// returns an instance.
|
||||
//
|
||||
// class ABSL_MUST_USE_RESULT Sprocket {};
|
||||
// Sprocket(); // Triggers a warning.
|
||||
//
|
||||
// Sprocket MakeSprocket();
|
||||
// MakeSprocket(); // Triggers a warning.
|
||||
//
|
||||
// Note that references and pointers are not instances:
|
||||
//
|
||||
// Sprocket* SprocketPointer();
|
||||
// SprocketPointer(); // Does *not* trigger a warning.
|
||||
//
|
||||
// ABSL_MUST_USE_RESULT allows using cast-to-void to suppress the unused result
|
||||
// warning. For that, warn_unused_result is used only for clang but not for gcc.
|
||||
// https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66425
|
||||
//
|
||||
// Note: past advice was to place the macro after the argument list.
|
||||
#if ABSL_HAVE_ATTRIBUTE(nodiscard)
|
||||
#define ABSL_MUST_USE_RESULT [[nodiscard]]
|
||||
#elif defined(__clang__) && ABSL_HAVE_ATTRIBUTE(warn_unused_result)
|
||||
#define ABSL_MUST_USE_RESULT __attribute__((warn_unused_result))
|
||||
#else
|
||||
#define ABSL_MUST_USE_RESULT
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_HOT, ABSL_ATTRIBUTE_COLD
|
||||
//
|
||||
// Tells GCC that a function is hot or cold. GCC can use this information to
|
||||
// improve static analysis, i.e. a conditional branch to a cold function
|
||||
// is likely to be not-taken.
|
||||
// This annotation is used for function declarations.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// int foo() ABSL_ATTRIBUTE_HOT;
|
||||
#if ABSL_HAVE_ATTRIBUTE(hot) || (defined(__GNUC__) && !defined(__clang__))
|
||||
#define ABSL_ATTRIBUTE_HOT __attribute__((hot))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_HOT
|
||||
#endif
|
||||
|
||||
#if ABSL_HAVE_ATTRIBUTE(cold) || (defined(__GNUC__) && !defined(__clang__))
|
||||
#define ABSL_ATTRIBUTE_COLD __attribute__((cold))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_COLD
|
||||
#endif
|
||||
|
||||
// ABSL_XRAY_ALWAYS_INSTRUMENT, ABSL_XRAY_NEVER_INSTRUMENT, ABSL_XRAY_LOG_ARGS
|
||||
//
|
||||
// We define the ABSL_XRAY_ALWAYS_INSTRUMENT and ABSL_XRAY_NEVER_INSTRUMENT
|
||||
// macro used as an attribute to mark functions that must always or never be
|
||||
// instrumented by XRay. Currently, this is only supported in Clang/LLVM.
|
||||
//
|
||||
// For reference on the LLVM XRay instrumentation, see
|
||||
// http://llvm.org/docs/XRay.html.
|
||||
//
|
||||
// A function with the XRAY_ALWAYS_INSTRUMENT macro attribute in its declaration
|
||||
// will always get the XRay instrumentation sleds. These sleds may introduce
|
||||
// some binary size and runtime overhead and must be used sparingly.
|
||||
//
|
||||
// These attributes only take effect when the following conditions are met:
|
||||
//
|
||||
// * The file/target is built in at least C++11 mode, with a Clang compiler
|
||||
// that supports XRay attributes.
|
||||
// * The file/target is built with the -fxray-instrument flag set for the
|
||||
// Clang/LLVM compiler.
|
||||
// * The function is defined in the translation unit (the compiler honors the
|
||||
// attribute in either the definition or the declaration, and must match).
|
||||
//
|
||||
// There are cases when, even when building with XRay instrumentation, users
|
||||
// might want to control specifically which functions are instrumented for a
|
||||
// particular build using special-case lists provided to the compiler. These
|
||||
// special case lists are provided to Clang via the
|
||||
// -fxray-always-instrument=... and -fxray-never-instrument=... flags. The
|
||||
// attributes in source take precedence over these special-case lists.
|
||||
//
|
||||
// To disable the XRay attributes at build-time, users may define
|
||||
// ABSL_NO_XRAY_ATTRIBUTES. Do NOT define ABSL_NO_XRAY_ATTRIBUTES on specific
|
||||
// packages/targets, as this may lead to conflicting definitions of functions at
|
||||
// link-time.
|
||||
//
|
||||
#if ABSL_HAVE_CPP_ATTRIBUTE(clang::xray_always_instrument) && \
|
||||
!defined(ABSL_NO_XRAY_ATTRIBUTES)
|
||||
#define ABSL_XRAY_ALWAYS_INSTRUMENT [[clang::xray_always_instrument]]
|
||||
#define ABSL_XRAY_NEVER_INSTRUMENT [[clang::xray_never_instrument]]
|
||||
#if ABSL_HAVE_CPP_ATTRIBUTE(clang::xray_log_args)
|
||||
#define ABSL_XRAY_LOG_ARGS(N) \
|
||||
[[clang::xray_always_instrument, clang::xray_log_args(N)]]
|
||||
#else
|
||||
#define ABSL_XRAY_LOG_ARGS(N) [[clang::xray_always_instrument]]
|
||||
#endif
|
||||
#else
|
||||
#define ABSL_XRAY_ALWAYS_INSTRUMENT
|
||||
#define ABSL_XRAY_NEVER_INSTRUMENT
|
||||
#define ABSL_XRAY_LOG_ARGS(N)
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_REINITIALIZES
|
||||
//
|
||||
// Indicates that a member function reinitializes the entire object to a known
|
||||
// state, independent of the previous state of the object.
|
||||
//
|
||||
// The clang-tidy check bugprone-use-after-move allows member functions marked
|
||||
// with this attribute to be called on objects that have been moved from;
|
||||
// without the attribute, this would result in a use-after-move warning.
|
||||
#if ABSL_HAVE_CPP_ATTRIBUTE(clang::reinitializes)
|
||||
#define ABSL_ATTRIBUTE_REINITIALIZES [[clang::reinitializes]]
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_REINITIALIZES
|
||||
#endif
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// Variable Attributes
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
// ABSL_ATTRIBUTE_UNUSED
|
||||
//
|
||||
// Prevents the compiler from complaining about variables that appear unused.
|
||||
#if ABSL_HAVE_ATTRIBUTE(unused) || (defined(__GNUC__) && !defined(__clang__))
|
||||
#undef ABSL_ATTRIBUTE_UNUSED
|
||||
#define ABSL_ATTRIBUTE_UNUSED __attribute__((__unused__))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_UNUSED
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_INITIAL_EXEC
|
||||
//
|
||||
// Tells the compiler to use "initial-exec" mode for a thread-local variable.
|
||||
// See http://people.redhat.com/drepper/tls.pdf for the gory details.
|
||||
#if ABSL_HAVE_ATTRIBUTE(tls_model) || (defined(__GNUC__) && !defined(__clang__))
|
||||
#define ABSL_ATTRIBUTE_INITIAL_EXEC __attribute__((tls_model("initial-exec")))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_INITIAL_EXEC
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_PACKED
|
||||
//
|
||||
// Instructs the compiler not to use natural alignment for a tagged data
|
||||
// structure, but instead to reduce its alignment to 1. This attribute can
|
||||
// either be applied to members of a structure or to a structure in its
|
||||
// entirety. Applying this attribute (judiciously) to a structure in its
|
||||
// entirety to optimize the memory footprint of very commonly-used structs is
|
||||
// fine. Do not apply this attribute to a structure in its entirety if the
|
||||
// purpose is to control the offsets of the members in the structure. Instead,
|
||||
// apply this attribute only to structure members that need it.
|
||||
//
|
||||
// When applying ABSL_ATTRIBUTE_PACKED only to specific structure members the
|
||||
// natural alignment of structure members not annotated is preserved. Aligned
|
||||
// member accesses are faster than non-aligned member accesses even if the
|
||||
// targeted microprocessor supports non-aligned accesses.
|
||||
#if ABSL_HAVE_ATTRIBUTE(packed) || (defined(__GNUC__) && !defined(__clang__))
|
||||
#define ABSL_ATTRIBUTE_PACKED __attribute__((__packed__))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_PACKED
|
||||
#endif
|
||||
|
||||
// ABSL_ATTRIBUTE_FUNC_ALIGN
|
||||
//
|
||||
// Tells the compiler to align the function start at least to certain
|
||||
// alignment boundary
|
||||
#if ABSL_HAVE_ATTRIBUTE(aligned) || (defined(__GNUC__) && !defined(__clang__))
|
||||
#define ABSL_ATTRIBUTE_FUNC_ALIGN(bytes) __attribute__((aligned(bytes)))
|
||||
#else
|
||||
#define ABSL_ATTRIBUTE_FUNC_ALIGN(bytes)
|
||||
#endif
|
||||
|
||||
// ABSL_CONST_INIT
|
||||
//
|
||||
// A variable declaration annotated with the `ABSL_CONST_INIT` attribute will
|
||||
// not compile (on supported platforms) unless the variable has a constant
|
||||
// initializer. This is useful for variables with static and thread storage
|
||||
// duration, because it guarantees that they will not suffer from the so-called
|
||||
// "static init order fiasco". Prefer to put this attribute on the most visible
|
||||
// declaration of the variable, if there's more than one, because code that
|
||||
// accesses the variable can then use the attribute for optimization.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// class MyClass {
|
||||
// public:
|
||||
// ABSL_CONST_INIT static MyType my_var;
|
||||
// };
|
||||
//
|
||||
// MyType MyClass::my_var = MakeMyType(...);
|
||||
//
|
||||
// Note that this attribute is redundant if the variable is declared constexpr.
|
||||
#if ABSL_HAVE_CPP_ATTRIBUTE(clang::require_constant_initialization)
|
||||
#define ABSL_CONST_INIT [[clang::require_constant_initialization]]
|
||||
#else
|
||||
#define ABSL_CONST_INIT
|
||||
#endif // ABSL_HAVE_CPP_ATTRIBUTE(clang::require_constant_initialization)
|
||||
|
||||
#endif // ABSL_BASE_ATTRIBUTES_H_
|
||||
Generated
+226
@@ -0,0 +1,226 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: call_once.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This header file provides an Abseil version of `std::call_once` for invoking
|
||||
// a given function at most once, across all threads. This Abseil version is
|
||||
// faster than the C++11 version and incorporates the C++17 argument-passing
|
||||
// fix, so that (for example) non-const references may be passed to the invoked
|
||||
// function.
|
||||
|
||||
#ifndef ABSL_BASE_CALL_ONCE_H_
|
||||
#define ABSL_BASE_CALL_ONCE_H_
|
||||
|
||||
#include <algorithm>
|
||||
#include <atomic>
|
||||
#include <cstdint>
|
||||
#include <type_traits>
|
||||
#include <utility>
|
||||
|
||||
#include "absl/base/internal/invoke.h"
|
||||
#include "absl/base/internal/low_level_scheduling.h"
|
||||
#include "absl/base/internal/raw_logging.h"
|
||||
#include "absl/base/internal/scheduling_mode.h"
|
||||
#include "absl/base/internal/spinlock_wait.h"
|
||||
#include "absl/base/macros.h"
|
||||
#include "absl/base/optimization.h"
|
||||
#include "absl/base/port.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
|
||||
class once_flag;
|
||||
|
||||
namespace base_internal {
|
||||
std::atomic<uint32_t>* ControlWord(absl::once_flag* flag);
|
||||
} // namespace base_internal
|
||||
|
||||
// call_once()
|
||||
//
|
||||
// For all invocations using a given `once_flag`, invokes a given `fn` exactly
|
||||
// once across all threads. The first call to `call_once()` with a particular
|
||||
// `once_flag` argument (that does not throw an exception) will run the
|
||||
// specified function with the provided `args`; other calls with the same
|
||||
// `once_flag` argument will not run the function, but will wait
|
||||
// for the provided function to finish running (if it is still running).
|
||||
//
|
||||
// This mechanism provides a safe, simple, and fast mechanism for one-time
|
||||
// initialization in a multi-threaded process.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// class MyInitClass {
|
||||
// public:
|
||||
// ...
|
||||
// mutable absl::once_flag once_;
|
||||
//
|
||||
// MyInitClass* init() const {
|
||||
// absl::call_once(once_, &MyInitClass::Init, this);
|
||||
// return ptr_;
|
||||
// }
|
||||
//
|
||||
template <typename Callable, typename... Args>
|
||||
void call_once(absl::once_flag& flag, Callable&& fn, Args&&... args);
|
||||
|
||||
// once_flag
|
||||
//
|
||||
// Objects of this type are used to distinguish calls to `call_once()` and
|
||||
// ensure the provided function is only invoked once across all threads. This
|
||||
// type is not copyable or movable. However, it has a `constexpr`
|
||||
// constructor, and is safe to use as a namespace-scoped global variable.
|
||||
class once_flag {
|
||||
public:
|
||||
constexpr once_flag() : control_(0) {}
|
||||
once_flag(const once_flag&) = delete;
|
||||
once_flag& operator=(const once_flag&) = delete;
|
||||
|
||||
private:
|
||||
friend std::atomic<uint32_t>* base_internal::ControlWord(once_flag* flag);
|
||||
std::atomic<uint32_t> control_;
|
||||
};
|
||||
|
||||
//------------------------------------------------------------------------------
|
||||
// End of public interfaces.
|
||||
// Implementation details follow.
|
||||
//------------------------------------------------------------------------------
|
||||
|
||||
namespace base_internal {
|
||||
|
||||
// Like call_once, but uses KERNEL_ONLY scheduling. Intended to be used to
|
||||
// initialize entities used by the scheduler implementation.
|
||||
template <typename Callable, typename... Args>
|
||||
void LowLevelCallOnce(absl::once_flag* flag, Callable&& fn, Args&&... args);
|
||||
|
||||
// Disables scheduling while on stack when scheduling mode is non-cooperative.
|
||||
// No effect for cooperative scheduling modes.
|
||||
class SchedulingHelper {
|
||||
public:
|
||||
explicit SchedulingHelper(base_internal::SchedulingMode mode) : mode_(mode) {
|
||||
if (mode_ == base_internal::SCHEDULE_KERNEL_ONLY) {
|
||||
guard_result_ = base_internal::SchedulingGuard::DisableRescheduling();
|
||||
}
|
||||
}
|
||||
|
||||
~SchedulingHelper() {
|
||||
if (mode_ == base_internal::SCHEDULE_KERNEL_ONLY) {
|
||||
base_internal::SchedulingGuard::EnableRescheduling(guard_result_);
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
base_internal::SchedulingMode mode_;
|
||||
bool guard_result_;
|
||||
};
|
||||
|
||||
// Bit patterns for call_once state machine values. Internal implementation
|
||||
// detail, not for use by clients.
|
||||
//
|
||||
// The bit patterns are arbitrarily chosen from unlikely values, to aid in
|
||||
// debugging. However, kOnceInit must be 0, so that a zero-initialized
|
||||
// once_flag will be valid for immediate use.
|
||||
enum {
|
||||
kOnceInit = 0,
|
||||
kOnceRunning = 0x65C2937B,
|
||||
kOnceWaiter = 0x05A308D2,
|
||||
// A very small constant is chosen for kOnceDone so that it fit in a single
|
||||
// compare with immediate instruction for most common ISAs. This is verified
|
||||
// for x86, POWER and ARM.
|
||||
kOnceDone = 221, // Random Number
|
||||
};
|
||||
|
||||
template <typename Callable, typename... Args>
|
||||
ABSL_ATTRIBUTE_NOINLINE
|
||||
void CallOnceImpl(std::atomic<uint32_t>* control,
|
||||
base_internal::SchedulingMode scheduling_mode, Callable&& fn,
|
||||
Args&&... args) {
|
||||
#ifndef NDEBUG
|
||||
{
|
||||
uint32_t old_control = control->load(std::memory_order_relaxed);
|
||||
if (old_control != kOnceInit &&
|
||||
old_control != kOnceRunning &&
|
||||
old_control != kOnceWaiter &&
|
||||
old_control != kOnceDone) {
|
||||
ABSL_RAW_LOG(FATAL, "Unexpected value for control word: 0x%lx",
|
||||
static_cast<unsigned long>(old_control)); // NOLINT
|
||||
}
|
||||
}
|
||||
#endif // NDEBUG
|
||||
static const base_internal::SpinLockWaitTransition trans[] = {
|
||||
{kOnceInit, kOnceRunning, true},
|
||||
{kOnceRunning, kOnceWaiter, false},
|
||||
{kOnceDone, kOnceDone, true}};
|
||||
|
||||
// Must do this before potentially modifying control word's state.
|
||||
base_internal::SchedulingHelper maybe_disable_scheduling(scheduling_mode);
|
||||
// Short circuit the simplest case to avoid procedure call overhead.
|
||||
// The base_internal::SpinLockWait() call returns either kOnceInit or
|
||||
// kOnceDone. If it returns kOnceDone, it must have loaded the control word
|
||||
// with std::memory_order_acquire and seen a value of kOnceDone.
|
||||
uint32_t old_control = kOnceInit;
|
||||
if (control->compare_exchange_strong(old_control, kOnceRunning,
|
||||
std::memory_order_relaxed) ||
|
||||
base_internal::SpinLockWait(control, ABSL_ARRAYSIZE(trans), trans,
|
||||
scheduling_mode) == kOnceInit) {
|
||||
base_internal::Invoke(std::forward<Callable>(fn),
|
||||
std::forward<Args>(args)...);
|
||||
// The call to SpinLockWake below is an optimization, because the waiter
|
||||
// in SpinLockWait is waiting with a short timeout. The atomic load/store
|
||||
// sequence is slightly faster than an atomic exchange:
|
||||
// old_control = control->exchange(base_internal::kOnceDone,
|
||||
// std::memory_order_release);
|
||||
// We opt for a slightly faster case when there are no waiters, in spite
|
||||
// of longer tail latency when there are waiters.
|
||||
old_control = control->load(std::memory_order_relaxed);
|
||||
control->store(base_internal::kOnceDone, std::memory_order_release);
|
||||
if (old_control == base_internal::kOnceWaiter) {
|
||||
base_internal::SpinLockWake(control, true);
|
||||
}
|
||||
} // else *control is already kOnceDone
|
||||
}
|
||||
|
||||
inline std::atomic<uint32_t>* ControlWord(once_flag* flag) {
|
||||
return &flag->control_;
|
||||
}
|
||||
|
||||
template <typename Callable, typename... Args>
|
||||
void LowLevelCallOnce(absl::once_flag* flag, Callable&& fn, Args&&... args) {
|
||||
std::atomic<uint32_t>* once = base_internal::ControlWord(flag);
|
||||
uint32_t s = once->load(std::memory_order_acquire);
|
||||
if (ABSL_PREDICT_FALSE(s != base_internal::kOnceDone)) {
|
||||
base_internal::CallOnceImpl(once, base_internal::SCHEDULE_KERNEL_ONLY,
|
||||
std::forward<Callable>(fn),
|
||||
std::forward<Args>(args)...);
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
|
||||
template <typename Callable, typename... Args>
|
||||
void call_once(absl::once_flag& flag, Callable&& fn, Args&&... args) {
|
||||
std::atomic<uint32_t>* once = base_internal::ControlWord(&flag);
|
||||
uint32_t s = once->load(std::memory_order_acquire);
|
||||
if (ABSL_PREDICT_FALSE(s != base_internal::kOnceDone)) {
|
||||
base_internal::CallOnceImpl(
|
||||
once, base_internal::SCHEDULE_COOPERATIVE_AND_KERNEL,
|
||||
std::forward<Callable>(fn), std::forward<Args>(args)...);
|
||||
}
|
||||
}
|
||||
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_CALL_ONCE_H_
|
||||
Generated
+184
@@ -0,0 +1,184 @@
|
||||
//
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: casts.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This header file defines casting templates to fit use cases not covered by
|
||||
// the standard casts provided in the C++ standard. As with all cast operations,
|
||||
// use these with caution and only if alternatives do not exist.
|
||||
|
||||
#ifndef ABSL_BASE_CASTS_H_
|
||||
#define ABSL_BASE_CASTS_H_
|
||||
|
||||
#include <cstring>
|
||||
#include <memory>
|
||||
#include <type_traits>
|
||||
#include <utility>
|
||||
|
||||
#include "absl/base/internal/identity.h"
|
||||
#include "absl/base/macros.h"
|
||||
#include "absl/meta/type_traits.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
|
||||
namespace internal_casts {
|
||||
|
||||
template <class Dest, class Source>
|
||||
struct is_bitcastable
|
||||
: std::integral_constant<
|
||||
bool,
|
||||
sizeof(Dest) == sizeof(Source) &&
|
||||
type_traits_internal::is_trivially_copyable<Source>::value &&
|
||||
type_traits_internal::is_trivially_copyable<Dest>::value &&
|
||||
std::is_default_constructible<Dest>::value> {};
|
||||
|
||||
} // namespace internal_casts
|
||||
|
||||
// implicit_cast()
|
||||
//
|
||||
// Performs an implicit conversion between types following the language
|
||||
// rules for implicit conversion; if an implicit conversion is otherwise
|
||||
// allowed by the language in the given context, this function performs such an
|
||||
// implicit conversion.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// // If the context allows implicit conversion:
|
||||
// From from;
|
||||
// To to = from;
|
||||
//
|
||||
// // Such code can be replaced by:
|
||||
// implicit_cast<To>(from);
|
||||
//
|
||||
// An `implicit_cast()` may also be used to annotate numeric type conversions
|
||||
// that, although safe, may produce compiler warnings (such as `long` to `int`).
|
||||
// Additionally, an `implicit_cast()` is also useful within return statements to
|
||||
// indicate a specific implicit conversion is being undertaken.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// return implicit_cast<double>(size_in_bytes) / capacity_;
|
||||
//
|
||||
// Annotating code with `implicit_cast()` allows you to explicitly select
|
||||
// particular overloads and template instantiations, while providing a safer
|
||||
// cast than `reinterpret_cast()` or `static_cast()`.
|
||||
//
|
||||
// Additionally, an `implicit_cast()` can be used to allow upcasting within a
|
||||
// type hierarchy where incorrect use of `static_cast()` could accidentally
|
||||
// allow downcasting.
|
||||
//
|
||||
// Finally, an `implicit_cast()` can be used to perform implicit conversions
|
||||
// from unrelated types that otherwise couldn't be implicitly cast directly;
|
||||
// C++ will normally only implicitly cast "one step" in such conversions.
|
||||
//
|
||||
// That is, if C is a type which can be implicitly converted to B, with B being
|
||||
// a type that can be implicitly converted to A, an `implicit_cast()` can be
|
||||
// used to convert C to B (which the compiler can then implicitly convert to A
|
||||
// using language rules).
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// // Assume an object C is convertible to B, which is implicitly convertible
|
||||
// // to A
|
||||
// A a = implicit_cast<B>(C);
|
||||
//
|
||||
// Such implicit cast chaining may be useful within template logic.
|
||||
template <typename To>
|
||||
constexpr To implicit_cast(typename absl::internal::identity_t<To> to) {
|
||||
return to;
|
||||
}
|
||||
|
||||
// bit_cast()
|
||||
//
|
||||
// Performs a bitwise cast on a type without changing the underlying bit
|
||||
// representation of that type's value. The two types must be of the same size
|
||||
// and both types must be trivially copyable. As with most casts, use with
|
||||
// caution. A `bit_cast()` might be needed when you need to temporarily treat a
|
||||
// type as some other type, such as in the following cases:
|
||||
//
|
||||
// * Serialization (casting temporarily to `char *` for those purposes is
|
||||
// always allowed by the C++ standard)
|
||||
// * Managing the individual bits of a type within mathematical operations
|
||||
// that are not normally accessible through that type
|
||||
// * Casting non-pointer types to pointer types (casting the other way is
|
||||
// allowed by `reinterpret_cast()` but round-trips cannot occur the other
|
||||
// way).
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// float f = 3.14159265358979;
|
||||
// int i = bit_cast<int32_t>(f);
|
||||
// // i = 0x40490fdb
|
||||
//
|
||||
// Casting non-pointer types to pointer types and then dereferencing them
|
||||
// traditionally produces undefined behavior.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// // WRONG
|
||||
// float f = 3.14159265358979; // WRONG
|
||||
// int i = * reinterpret_cast<int*>(&f); // WRONG
|
||||
//
|
||||
// The address-casting method produces undefined behavior according to the ISO
|
||||
// C++ specification section [basic.lval]. Roughly, this section says: if an
|
||||
// object in memory has one type, and a program accesses it with a different
|
||||
// type, the result is undefined behavior for most values of "different type".
|
||||
//
|
||||
// Such casting results in type punning: holding an object in memory of one type
|
||||
// and reading its bits back using a different type. A `bit_cast()` avoids this
|
||||
// issue by implementing its casts using `memcpy()`, which avoids introducing
|
||||
// this undefined behavior.
|
||||
//
|
||||
// NOTE: The requirements here are more strict than the bit_cast of standard
|
||||
// proposal p0476 due to the need for workarounds and lack of intrinsics.
|
||||
// Specifically, this implementation also requires `Dest` to be
|
||||
// default-constructible.
|
||||
template <
|
||||
typename Dest, typename Source,
|
||||
typename std::enable_if<internal_casts::is_bitcastable<Dest, Source>::value,
|
||||
int>::type = 0>
|
||||
inline Dest bit_cast(const Source& source) {
|
||||
Dest dest;
|
||||
memcpy(static_cast<void*>(std::addressof(dest)),
|
||||
static_cast<const void*>(std::addressof(source)), sizeof(dest));
|
||||
return dest;
|
||||
}
|
||||
|
||||
// NOTE: This overload is only picked if the requirements of bit_cast are not
|
||||
// met. It is therefore UB, but is provided temporarily as previous versions of
|
||||
// this function template were unchecked. Do not use this in new code.
|
||||
template <
|
||||
typename Dest, typename Source,
|
||||
typename std::enable_if<
|
||||
!internal_casts::is_bitcastable<Dest, Source>::value, int>::type = 0>
|
||||
ABSL_DEPRECATED(
|
||||
"absl::bit_cast type requirements were violated. Update the types being "
|
||||
"used such that they are the same size and are both TriviallyCopyable.")
|
||||
inline Dest bit_cast(const Source& source) {
|
||||
static_assert(sizeof(Dest) == sizeof(Source),
|
||||
"Source and destination types should have equal sizes.");
|
||||
|
||||
Dest dest;
|
||||
memcpy(&dest, &source, sizeof(dest));
|
||||
return dest;
|
||||
}
|
||||
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_CASTS_H_
|
||||
Generated
+671
@@ -0,0 +1,671 @@
|
||||
//
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: config.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This header file defines a set of macros for checking the presence of
|
||||
// important compiler and platform features. Such macros can be used to
|
||||
// produce portable code by parameterizing compilation based on the presence or
|
||||
// lack of a given feature.
|
||||
//
|
||||
// We define a "feature" as some interface we wish to program to: for example,
|
||||
// a library function or system call. A value of `1` indicates support for
|
||||
// that feature; any other value indicates the feature support is undefined.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// Suppose a programmer wants to write a program that uses the 'mmap()' system
|
||||
// call. The Abseil macro for that feature (`ABSL_HAVE_MMAP`) allows you to
|
||||
// selectively include the `mmap.h` header and bracket code using that feature
|
||||
// in the macro:
|
||||
//
|
||||
// #include "absl/base/config.h"
|
||||
//
|
||||
// #ifdef ABSL_HAVE_MMAP
|
||||
// #include "sys/mman.h"
|
||||
// #endif //ABSL_HAVE_MMAP
|
||||
//
|
||||
// ...
|
||||
// #ifdef ABSL_HAVE_MMAP
|
||||
// void *ptr = mmap(...);
|
||||
// ...
|
||||
// #endif // ABSL_HAVE_MMAP
|
||||
|
||||
#ifndef ABSL_BASE_CONFIG_H_
|
||||
#define ABSL_BASE_CONFIG_H_
|
||||
|
||||
// Included for the __GLIBC__ macro (or similar macros on other systems).
|
||||
#include <limits.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
// Included for __GLIBCXX__, _LIBCPP_VERSION
|
||||
#include <cstddef>
|
||||
#endif // __cplusplus
|
||||
|
||||
#if defined(__APPLE__)
|
||||
// Included for TARGET_OS_IPHONE, __IPHONE_OS_VERSION_MIN_REQUIRED,
|
||||
// __IPHONE_8_0.
|
||||
#include <Availability.h>
|
||||
#include <TargetConditionals.h>
|
||||
#endif
|
||||
|
||||
#include "absl/base/options.h"
|
||||
#include "absl/base/policy_checks.h"
|
||||
|
||||
// Helper macro to convert a CPP variable to a string literal.
|
||||
#define ABSL_INTERNAL_DO_TOKEN_STR(x) #x
|
||||
#define ABSL_INTERNAL_TOKEN_STR(x) ABSL_INTERNAL_DO_TOKEN_STR(x)
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// Abseil namespace annotations
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
// ABSL_NAMESPACE_BEGIN/ABSL_NAMESPACE_END
|
||||
//
|
||||
// An annotation placed at the beginning/end of each `namespace absl` scope.
|
||||
// This is used to inject an inline namespace.
|
||||
//
|
||||
// The proper way to write Abseil code in the `absl` namespace is:
|
||||
//
|
||||
// namespace absl {
|
||||
// ABSL_NAMESPACE_BEGIN
|
||||
//
|
||||
// void Foo(); // absl::Foo().
|
||||
//
|
||||
// ABSL_NAMESPACE_END
|
||||
// } // namespace absl
|
||||
//
|
||||
// Users of Abseil should not use these macros, because users of Abseil should
|
||||
// not write `namespace absl {` in their own code for any reason. (Abseil does
|
||||
// not support forward declarations of its own types, nor does it support
|
||||
// user-provided specialization of Abseil templates. Code that violates these
|
||||
// rules may be broken without warning.)
|
||||
#if !defined(ABSL_OPTION_USE_INLINE_NAMESPACE) || \
|
||||
!defined(ABSL_OPTION_INLINE_NAMESPACE_NAME)
|
||||
#error options.h is misconfigured.
|
||||
#endif
|
||||
|
||||
// Check that ABSL_OPTION_INLINE_NAMESPACE_NAME is neither "head" nor ""
|
||||
#if defined(__cplusplus) && ABSL_OPTION_USE_INLINE_NAMESPACE == 1
|
||||
|
||||
#define ABSL_INTERNAL_INLINE_NAMESPACE_STR \
|
||||
ABSL_INTERNAL_TOKEN_STR(ABSL_OPTION_INLINE_NAMESPACE_NAME)
|
||||
|
||||
static_assert(ABSL_INTERNAL_INLINE_NAMESPACE_STR[0] != '\0',
|
||||
"options.h misconfigured: ABSL_OPTION_INLINE_NAMESPACE_NAME must "
|
||||
"not be empty.");
|
||||
static_assert(ABSL_INTERNAL_INLINE_NAMESPACE_STR[0] != 'h' ||
|
||||
ABSL_INTERNAL_INLINE_NAMESPACE_STR[1] != 'e' ||
|
||||
ABSL_INTERNAL_INLINE_NAMESPACE_STR[2] != 'a' ||
|
||||
ABSL_INTERNAL_INLINE_NAMESPACE_STR[3] != 'd' ||
|
||||
ABSL_INTERNAL_INLINE_NAMESPACE_STR[4] != '\0',
|
||||
"options.h misconfigured: ABSL_OPTION_INLINE_NAMESPACE_NAME must "
|
||||
"be changed to a new, unique identifier name.");
|
||||
|
||||
#endif
|
||||
|
||||
#if ABSL_OPTION_USE_INLINE_NAMESPACE == 0
|
||||
#define ABSL_NAMESPACE_BEGIN
|
||||
#define ABSL_NAMESPACE_END
|
||||
#elif ABSL_OPTION_USE_INLINE_NAMESPACE == 1
|
||||
#define ABSL_NAMESPACE_BEGIN \
|
||||
inline namespace ABSL_OPTION_INLINE_NAMESPACE_NAME {
|
||||
#define ABSL_NAMESPACE_END }
|
||||
#else
|
||||
#error options.h is misconfigured.
|
||||
#endif
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// Compiler Feature Checks
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
// ABSL_HAVE_BUILTIN()
|
||||
//
|
||||
// Checks whether the compiler supports a Clang Feature Checking Macro, and if
|
||||
// so, checks whether it supports the provided builtin function "x" where x
|
||||
// is one of the functions noted in
|
||||
// https://clang.llvm.org/docs/LanguageExtensions.html
|
||||
//
|
||||
// Note: Use this macro to avoid an extra level of #ifdef __has_builtin check.
|
||||
// http://releases.llvm.org/3.3/tools/clang/docs/LanguageExtensions.html
|
||||
#ifdef __has_builtin
|
||||
#define ABSL_HAVE_BUILTIN(x) __has_builtin(x)
|
||||
#else
|
||||
#define ABSL_HAVE_BUILTIN(x) 0
|
||||
#endif
|
||||
|
||||
#if defined(__is_identifier)
|
||||
#define ABSL_INTERNAL_HAS_KEYWORD(x) !(__is_identifier(x))
|
||||
#else
|
||||
#define ABSL_INTERNAL_HAS_KEYWORD(x) 0
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_TLS is defined to 1 when __thread should be supported.
|
||||
// We assume __thread is supported on Linux when compiled with Clang or compiled
|
||||
// against libstdc++ with _GLIBCXX_HAVE_TLS defined.
|
||||
#ifdef ABSL_HAVE_TLS
|
||||
#error ABSL_HAVE_TLS cannot be directly set
|
||||
#elif defined(__linux__) && (defined(__clang__) || defined(_GLIBCXX_HAVE_TLS))
|
||||
#define ABSL_HAVE_TLS 1
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_STD_IS_TRIVIALLY_DESTRUCTIBLE
|
||||
//
|
||||
// Checks whether `std::is_trivially_destructible<T>` is supported.
|
||||
//
|
||||
// Notes: All supported compilers using libc++ support this feature, as does
|
||||
// gcc >= 4.8.1 using libstdc++, and Visual Studio.
|
||||
#ifdef ABSL_HAVE_STD_IS_TRIVIALLY_DESTRUCTIBLE
|
||||
#error ABSL_HAVE_STD_IS_TRIVIALLY_DESTRUCTIBLE cannot be directly set
|
||||
#elif defined(_LIBCPP_VERSION) || \
|
||||
(!defined(__clang__) && defined(__GNUC__) && defined(__GLIBCXX__) && \
|
||||
(__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8))) || \
|
||||
defined(_MSC_VER)
|
||||
#define ABSL_HAVE_STD_IS_TRIVIALLY_DESTRUCTIBLE 1
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_STD_IS_TRIVIALLY_CONSTRUCTIBLE
|
||||
//
|
||||
// Checks whether `std::is_trivially_default_constructible<T>` and
|
||||
// `std::is_trivially_copy_constructible<T>` are supported.
|
||||
|
||||
// ABSL_HAVE_STD_IS_TRIVIALLY_ASSIGNABLE
|
||||
//
|
||||
// Checks whether `std::is_trivially_copy_assignable<T>` is supported.
|
||||
|
||||
// Notes: Clang with libc++ supports these features, as does gcc >= 5.1 with
|
||||
// either libc++ or libstdc++, and Visual Studio (but not NVCC).
|
||||
#if defined(ABSL_HAVE_STD_IS_TRIVIALLY_CONSTRUCTIBLE)
|
||||
#error ABSL_HAVE_STD_IS_TRIVIALLY_CONSTRUCTIBLE cannot be directly set
|
||||
#elif defined(ABSL_HAVE_STD_IS_TRIVIALLY_ASSIGNABLE)
|
||||
#error ABSL_HAVE_STD_IS_TRIVIALLY_ASSIGNABLE cannot directly set
|
||||
#elif (defined(__clang__) && defined(_LIBCPP_VERSION)) || \
|
||||
(!defined(__clang__) && defined(__GNUC__) && \
|
||||
(__GNUC__ > 7 || (__GNUC__ == 7 && __GNUC_MINOR__ >= 4)) && \
|
||||
(defined(_LIBCPP_VERSION) || defined(__GLIBCXX__))) || \
|
||||
(defined(_MSC_VER) && !defined(__NVCC__))
|
||||
#define ABSL_HAVE_STD_IS_TRIVIALLY_CONSTRUCTIBLE 1
|
||||
#define ABSL_HAVE_STD_IS_TRIVIALLY_ASSIGNABLE 1
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_SOURCE_LOCATION_CURRENT
|
||||
//
|
||||
// Indicates whether `absl::SourceLocation::current()` will return useful
|
||||
// information in some contexts.
|
||||
#ifndef ABSL_HAVE_SOURCE_LOCATION_CURRENT
|
||||
#if ABSL_INTERNAL_HAS_KEYWORD(__builtin_LINE) && \
|
||||
ABSL_INTERNAL_HAS_KEYWORD(__builtin_FILE)
|
||||
#define ABSL_HAVE_SOURCE_LOCATION_CURRENT 1
|
||||
#endif
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_THREAD_LOCAL
|
||||
//
|
||||
// Checks whether C++11's `thread_local` storage duration specifier is
|
||||
// supported.
|
||||
#ifdef ABSL_HAVE_THREAD_LOCAL
|
||||
#error ABSL_HAVE_THREAD_LOCAL cannot be directly set
|
||||
#elif defined(__APPLE__)
|
||||
// Notes:
|
||||
// * Xcode's clang did not support `thread_local` until version 8, and
|
||||
// even then not for all iOS < 9.0.
|
||||
// * Xcode 9.3 started disallowing `thread_local` for 32-bit iOS simulator
|
||||
// targeting iOS 9.x.
|
||||
// * Xcode 10 moves the deployment target check for iOS < 9.0 to link time
|
||||
// making __has_feature unreliable there.
|
||||
//
|
||||
// Otherwise, `__has_feature` is only supported by Clang so it has be inside
|
||||
// `defined(__APPLE__)` check.
|
||||
#if __has_feature(cxx_thread_local) && \
|
||||
!(TARGET_OS_IPHONE && __IPHONE_OS_VERSION_MIN_REQUIRED < __IPHONE_9_0)
|
||||
#define ABSL_HAVE_THREAD_LOCAL 1
|
||||
#endif
|
||||
#else // !defined(__APPLE__)
|
||||
#define ABSL_HAVE_THREAD_LOCAL 1
|
||||
#endif
|
||||
|
||||
// There are platforms for which TLS should not be used even though the compiler
|
||||
// makes it seem like it's supported (Android NDK < r12b for example).
|
||||
// This is primarily because of linker problems and toolchain misconfiguration:
|
||||
// Abseil does not intend to support this indefinitely. Currently, the newest
|
||||
// toolchain that we intend to support that requires this behavior is the
|
||||
// r11 NDK - allowing for a 5 year support window on that means this option
|
||||
// is likely to be removed around June of 2021.
|
||||
// TLS isn't supported until NDK r12b per
|
||||
// https://developer.android.com/ndk/downloads/revision_history.html
|
||||
// Since NDK r16, `__NDK_MAJOR__` and `__NDK_MINOR__` are defined in
|
||||
// <android/ndk-version.h>. For NDK < r16, users should define these macros,
|
||||
// e.g. `-D__NDK_MAJOR__=11 -D__NKD_MINOR__=0` for NDK r11.
|
||||
#if defined(__ANDROID__) && defined(__clang__)
|
||||
#if __has_include(<android/ndk-version.h>)
|
||||
#include <android/ndk-version.h>
|
||||
#endif // __has_include(<android/ndk-version.h>)
|
||||
#if defined(__ANDROID__) && defined(__clang__) && defined(__NDK_MAJOR__) && \
|
||||
defined(__NDK_MINOR__) && \
|
||||
((__NDK_MAJOR__ < 12) || ((__NDK_MAJOR__ == 12) && (__NDK_MINOR__ < 1)))
|
||||
#undef ABSL_HAVE_TLS
|
||||
#undef ABSL_HAVE_THREAD_LOCAL
|
||||
#endif
|
||||
#endif // defined(__ANDROID__) && defined(__clang__)
|
||||
|
||||
// Emscripten doesn't yet support `thread_local` or `__thread`.
|
||||
// https://github.com/emscripten-core/emscripten/issues/3502
|
||||
#if defined(__EMSCRIPTEN__)
|
||||
#undef ABSL_HAVE_TLS
|
||||
#undef ABSL_HAVE_THREAD_LOCAL
|
||||
#endif // defined(__EMSCRIPTEN__)
|
||||
|
||||
// ABSL_HAVE_INTRINSIC_INT128
|
||||
//
|
||||
// Checks whether the __int128 compiler extension for a 128-bit integral type is
|
||||
// supported.
|
||||
//
|
||||
// Note: __SIZEOF_INT128__ is defined by Clang and GCC when __int128 is
|
||||
// supported, but we avoid using it in certain cases:
|
||||
// * On Clang:
|
||||
// * Building using Clang for Windows, where the Clang runtime library has
|
||||
// 128-bit support only on LP64 architectures, but Windows is LLP64.
|
||||
// * On Nvidia's nvcc:
|
||||
// * nvcc also defines __GNUC__ and __SIZEOF_INT128__, but not all versions
|
||||
// actually support __int128.
|
||||
#ifdef ABSL_HAVE_INTRINSIC_INT128
|
||||
#error ABSL_HAVE_INTRINSIC_INT128 cannot be directly set
|
||||
#elif defined(__SIZEOF_INT128__)
|
||||
#if (defined(__clang__) && !defined(_WIN32)) || \
|
||||
(defined(__CUDACC__) && __CUDACC_VER_MAJOR__ >= 9) || \
|
||||
(defined(__GNUC__) && !defined(__clang__) && !defined(__CUDACC__))
|
||||
#define ABSL_HAVE_INTRINSIC_INT128 1
|
||||
#elif defined(__CUDACC__)
|
||||
// __CUDACC_VER__ is a full version number before CUDA 9, and is defined to a
|
||||
// string explaining that it has been removed starting with CUDA 9. We use
|
||||
// nested #ifs because there is no short-circuiting in the preprocessor.
|
||||
// NOTE: `__CUDACC__` could be undefined while `__CUDACC_VER__` is defined.
|
||||
#if __CUDACC_VER__ >= 70000
|
||||
#define ABSL_HAVE_INTRINSIC_INT128 1
|
||||
#endif // __CUDACC_VER__ >= 70000
|
||||
#endif // defined(__CUDACC__)
|
||||
#endif // ABSL_HAVE_INTRINSIC_INT128
|
||||
|
||||
// ABSL_HAVE_EXCEPTIONS
|
||||
//
|
||||
// Checks whether the compiler both supports and enables exceptions. Many
|
||||
// compilers support a "no exceptions" mode that disables exceptions.
|
||||
//
|
||||
// Generally, when ABSL_HAVE_EXCEPTIONS is not defined:
|
||||
//
|
||||
// * Code using `throw` and `try` may not compile.
|
||||
// * The `noexcept` specifier will still compile and behave as normal.
|
||||
// * The `noexcept` operator may still return `false`.
|
||||
//
|
||||
// For further details, consult the compiler's documentation.
|
||||
#ifdef ABSL_HAVE_EXCEPTIONS
|
||||
#error ABSL_HAVE_EXCEPTIONS cannot be directly set.
|
||||
|
||||
#elif defined(__clang__)
|
||||
|
||||
#if __clang_major__ > 3 || (__clang_major__ == 3 && __clang_minor__ >= 6)
|
||||
// Clang >= 3.6
|
||||
#if __has_feature(cxx_exceptions)
|
||||
#define ABSL_HAVE_EXCEPTIONS 1
|
||||
#endif // __has_feature(cxx_exceptions)
|
||||
#else
|
||||
// Clang < 3.6
|
||||
// http://releases.llvm.org/3.6.0/tools/clang/docs/ReleaseNotes.html#the-exceptions-macro
|
||||
#if defined(__EXCEPTIONS) && __has_feature(cxx_exceptions)
|
||||
#define ABSL_HAVE_EXCEPTIONS 1
|
||||
#endif // defined(__EXCEPTIONS) && __has_feature(cxx_exceptions)
|
||||
#endif // __clang_major__ > 3 || (__clang_major__ == 3 && __clang_minor__ >= 6)
|
||||
|
||||
// Handle remaining special cases and default to exceptions being supported.
|
||||
#elif !(defined(__GNUC__) && (__GNUC__ < 5) && !defined(__EXCEPTIONS)) && \
|
||||
!(defined(__GNUC__) && (__GNUC__ >= 5) && !defined(__cpp_exceptions)) && \
|
||||
!(defined(_MSC_VER) && !defined(_CPPUNWIND))
|
||||
#define ABSL_HAVE_EXCEPTIONS 1
|
||||
#endif
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// Platform Feature Checks
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
// Currently supported operating systems and associated preprocessor
|
||||
// symbols:
|
||||
//
|
||||
// Linux and Linux-derived __linux__
|
||||
// Android __ANDROID__ (implies __linux__)
|
||||
// Linux (non-Android) __linux__ && !__ANDROID__
|
||||
// Darwin (macOS and iOS) __APPLE__
|
||||
// Akaros (http://akaros.org) __ros__
|
||||
// Windows _WIN32
|
||||
// NaCL __native_client__
|
||||
// AsmJS __asmjs__
|
||||
// WebAssembly __wasm__
|
||||
// Fuchsia __Fuchsia__
|
||||
//
|
||||
// Note that since Android defines both __ANDROID__ and __linux__, one
|
||||
// may probe for either Linux or Android by simply testing for __linux__.
|
||||
|
||||
// ABSL_HAVE_MMAP
|
||||
//
|
||||
// Checks whether the platform has an mmap(2) implementation as defined in
|
||||
// POSIX.1-2001.
|
||||
#ifdef ABSL_HAVE_MMAP
|
||||
#error ABSL_HAVE_MMAP cannot be directly set
|
||||
#elif defined(__linux__) || defined(__APPLE__) || defined(__FreeBSD__) || \
|
||||
defined(__ros__) || defined(__native_client__) || defined(__asmjs__) || \
|
||||
defined(__wasm__) || defined(__Fuchsia__) || defined(__sun) || \
|
||||
defined(__ASYLO__)
|
||||
#define ABSL_HAVE_MMAP 1
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_PTHREAD_GETSCHEDPARAM
|
||||
//
|
||||
// Checks whether the platform implements the pthread_(get|set)schedparam(3)
|
||||
// functions as defined in POSIX.1-2001.
|
||||
#ifdef ABSL_HAVE_PTHREAD_GETSCHEDPARAM
|
||||
#error ABSL_HAVE_PTHREAD_GETSCHEDPARAM cannot be directly set
|
||||
#elif defined(__linux__) || defined(__APPLE__) || defined(__FreeBSD__) || \
|
||||
defined(__ros__)
|
||||
#define ABSL_HAVE_PTHREAD_GETSCHEDPARAM 1
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_SCHED_YIELD
|
||||
//
|
||||
// Checks whether the platform implements sched_yield(2) as defined in
|
||||
// POSIX.1-2001.
|
||||
#ifdef ABSL_HAVE_SCHED_YIELD
|
||||
#error ABSL_HAVE_SCHED_YIELD cannot be directly set
|
||||
#elif defined(__linux__) || defined(__ros__) || defined(__native_client__)
|
||||
#define ABSL_HAVE_SCHED_YIELD 1
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_SEMAPHORE_H
|
||||
//
|
||||
// Checks whether the platform supports the <semaphore.h> header and sem_init(3)
|
||||
// family of functions as standardized in POSIX.1-2001.
|
||||
//
|
||||
// Note: While Apple provides <semaphore.h> for both iOS and macOS, it is
|
||||
// explicitly deprecated and will cause build failures if enabled for those
|
||||
// platforms. We side-step the issue by not defining it here for Apple
|
||||
// platforms.
|
||||
#ifdef ABSL_HAVE_SEMAPHORE_H
|
||||
#error ABSL_HAVE_SEMAPHORE_H cannot be directly set
|
||||
#elif defined(__linux__) || defined(__ros__)
|
||||
#define ABSL_HAVE_SEMAPHORE_H 1
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_ALARM
|
||||
//
|
||||
// Checks whether the platform supports the <signal.h> header and alarm(2)
|
||||
// function as standardized in POSIX.1-2001.
|
||||
#ifdef ABSL_HAVE_ALARM
|
||||
#error ABSL_HAVE_ALARM cannot be directly set
|
||||
#elif defined(__GOOGLE_GRTE_VERSION__)
|
||||
// feature tests for Google's GRTE
|
||||
#define ABSL_HAVE_ALARM 1
|
||||
#elif defined(__GLIBC__)
|
||||
// feature test for glibc
|
||||
#define ABSL_HAVE_ALARM 1
|
||||
#elif defined(_MSC_VER)
|
||||
// feature tests for Microsoft's library
|
||||
#elif defined(__MINGW32__)
|
||||
// mingw32 doesn't provide alarm(2):
|
||||
// https://osdn.net/projects/mingw/scm/git/mingw-org-wsl/blobs/5.2-trunk/mingwrt/include/unistd.h
|
||||
// mingw-w64 provides a no-op implementation:
|
||||
// https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-crt/misc/alarm.c
|
||||
#elif defined(__EMSCRIPTEN__)
|
||||
// emscripten doesn't support signals
|
||||
#elif defined(__Fuchsia__)
|
||||
// Signals don't exist on fuchsia.
|
||||
#elif defined(__native_client__)
|
||||
#else
|
||||
// other standard libraries
|
||||
#define ABSL_HAVE_ALARM 1
|
||||
#endif
|
||||
|
||||
// ABSL_IS_LITTLE_ENDIAN
|
||||
// ABSL_IS_BIG_ENDIAN
|
||||
//
|
||||
// Checks the endianness of the platform.
|
||||
//
|
||||
// Notes: uses the built in endian macros provided by GCC (since 4.6) and
|
||||
// Clang (since 3.2); see
|
||||
// https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html.
|
||||
// Otherwise, if _WIN32, assume little endian. Otherwise, bail with an error.
|
||||
#if defined(ABSL_IS_BIG_ENDIAN)
|
||||
#error "ABSL_IS_BIG_ENDIAN cannot be directly set."
|
||||
#endif
|
||||
#if defined(ABSL_IS_LITTLE_ENDIAN)
|
||||
#error "ABSL_IS_LITTLE_ENDIAN cannot be directly set."
|
||||
#endif
|
||||
|
||||
#if (defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) && \
|
||||
__BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__)
|
||||
#define ABSL_IS_LITTLE_ENDIAN 1
|
||||
#elif defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && \
|
||||
__BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
|
||||
#define ABSL_IS_BIG_ENDIAN 1
|
||||
#elif defined(_WIN32)
|
||||
#define ABSL_IS_LITTLE_ENDIAN 1
|
||||
#else
|
||||
#error "absl endian detection needs to be set up for your compiler"
|
||||
#endif
|
||||
|
||||
// macOS 10.13 and iOS 10.11 don't let you use <any>, <optional>, or <variant>
|
||||
// even though the headers exist and are publicly noted to work. See
|
||||
// https://github.com/abseil/abseil-cpp/issues/207 and
|
||||
// https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes
|
||||
// libc++ spells out the availability requirements in the file
|
||||
// llvm-project/libcxx/include/__config via the #define
|
||||
// _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS.
|
||||
#if defined(__APPLE__) && defined(_LIBCPP_VERSION) && \
|
||||
((defined(__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__) && \
|
||||
__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ < 101400) || \
|
||||
(defined(__ENVIRONMENT_IPHONE_OS_VERSION_MIN_REQUIRED__) && \
|
||||
__ENVIRONMENT_IPHONE_OS_VERSION_MIN_REQUIRED__ < 120000) || \
|
||||
(defined(__ENVIRONMENT_WATCH_OS_VERSION_MIN_REQUIRED__) && \
|
||||
__ENVIRONMENT_WATCH_OS_VERSION_MIN_REQUIRED__ < 120000) || \
|
||||
(defined(__ENVIRONMENT_TV_OS_VERSION_MIN_REQUIRED__) && \
|
||||
__ENVIRONMENT_TV_OS_VERSION_MIN_REQUIRED__ < 50000))
|
||||
#define ABSL_INTERNAL_APPLE_CXX17_TYPES_UNAVAILABLE 1
|
||||
#else
|
||||
#define ABSL_INTERNAL_APPLE_CXX17_TYPES_UNAVAILABLE 0
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_STD_ANY
|
||||
//
|
||||
// Checks whether C++17 std::any is available by checking whether <any> exists.
|
||||
#ifdef ABSL_HAVE_STD_ANY
|
||||
#error "ABSL_HAVE_STD_ANY cannot be directly set."
|
||||
#endif
|
||||
|
||||
#ifdef __has_include
|
||||
#if __has_include(<any>) && __cplusplus >= 201703L && \
|
||||
!ABSL_INTERNAL_APPLE_CXX17_TYPES_UNAVAILABLE
|
||||
#define ABSL_HAVE_STD_ANY 1
|
||||
#endif
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_STD_OPTIONAL
|
||||
//
|
||||
// Checks whether C++17 std::optional is available.
|
||||
#ifdef ABSL_HAVE_STD_OPTIONAL
|
||||
#error "ABSL_HAVE_STD_OPTIONAL cannot be directly set."
|
||||
#endif
|
||||
|
||||
#ifdef __has_include
|
||||
#if __has_include(<optional>) && __cplusplus >= 201703L && \
|
||||
!ABSL_INTERNAL_APPLE_CXX17_TYPES_UNAVAILABLE
|
||||
#define ABSL_HAVE_STD_OPTIONAL 1
|
||||
#endif
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_STD_VARIANT
|
||||
//
|
||||
// Checks whether C++17 std::variant is available.
|
||||
#ifdef ABSL_HAVE_STD_VARIANT
|
||||
#error "ABSL_HAVE_STD_VARIANT cannot be directly set."
|
||||
#endif
|
||||
|
||||
#ifdef __has_include
|
||||
#if __has_include(<variant>) && __cplusplus >= 201703L && \
|
||||
!ABSL_INTERNAL_APPLE_CXX17_TYPES_UNAVAILABLE
|
||||
#define ABSL_HAVE_STD_VARIANT 1
|
||||
#endif
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_STD_STRING_VIEW
|
||||
//
|
||||
// Checks whether C++17 std::string_view is available.
|
||||
#ifdef ABSL_HAVE_STD_STRING_VIEW
|
||||
#error "ABSL_HAVE_STD_STRING_VIEW cannot be directly set."
|
||||
#endif
|
||||
|
||||
#ifdef __has_include
|
||||
#if __has_include(<string_view>) && __cplusplus >= 201703L
|
||||
#define ABSL_HAVE_STD_STRING_VIEW 1
|
||||
#endif
|
||||
#endif
|
||||
|
||||
// For MSVC, `__has_include` is supported in VS 2017 15.3, which is later than
|
||||
// the support for <optional>, <any>, <string_view>, <variant>. So we use
|
||||
// _MSC_VER to check whether we have VS 2017 RTM (when <optional>, <any>,
|
||||
// <string_view>, <variant> is implemented) or higher. Also, `__cplusplus` is
|
||||
// not correctly set by MSVC, so we use `_MSVC_LANG` to check the language
|
||||
// version.
|
||||
// TODO(zhangxy): fix tests before enabling aliasing for `std::any`.
|
||||
#if defined(_MSC_VER) && _MSC_VER >= 1910 && \
|
||||
((defined(_MSVC_LANG) && _MSVC_LANG > 201402) || __cplusplus > 201402)
|
||||
// #define ABSL_HAVE_STD_ANY 1
|
||||
#define ABSL_HAVE_STD_OPTIONAL 1
|
||||
#define ABSL_HAVE_STD_VARIANT 1
|
||||
#define ABSL_HAVE_STD_STRING_VIEW 1
|
||||
#endif
|
||||
|
||||
// ABSL_USES_STD_ANY
|
||||
//
|
||||
// Indicates whether absl::any is an alias for std::any.
|
||||
#if !defined(ABSL_OPTION_USE_STD_ANY)
|
||||
#error options.h is misconfigured.
|
||||
#elif ABSL_OPTION_USE_STD_ANY == 0 || \
|
||||
(ABSL_OPTION_USE_STD_ANY == 2 && !defined(ABSL_HAVE_STD_ANY))
|
||||
#undef ABSL_USES_STD_ANY
|
||||
#elif ABSL_OPTION_USE_STD_ANY == 1 || \
|
||||
(ABSL_OPTION_USE_STD_ANY == 2 && defined(ABSL_HAVE_STD_ANY))
|
||||
#define ABSL_USES_STD_ANY 1
|
||||
#else
|
||||
#error options.h is misconfigured.
|
||||
#endif
|
||||
|
||||
// ABSL_USES_STD_OPTIONAL
|
||||
//
|
||||
// Indicates whether absl::optional is an alias for std::optional.
|
||||
#if !defined(ABSL_OPTION_USE_STD_OPTIONAL)
|
||||
#error options.h is misconfigured.
|
||||
#elif ABSL_OPTION_USE_STD_OPTIONAL == 0 || \
|
||||
(ABSL_OPTION_USE_STD_OPTIONAL == 2 && !defined(ABSL_HAVE_STD_OPTIONAL))
|
||||
#undef ABSL_USES_STD_OPTIONAL
|
||||
#elif ABSL_OPTION_USE_STD_OPTIONAL == 1 || \
|
||||
(ABSL_OPTION_USE_STD_OPTIONAL == 2 && defined(ABSL_HAVE_STD_OPTIONAL))
|
||||
#define ABSL_USES_STD_OPTIONAL 1
|
||||
#else
|
||||
#error options.h is misconfigured.
|
||||
#endif
|
||||
|
||||
// ABSL_USES_STD_VARIANT
|
||||
//
|
||||
// Indicates whether absl::variant is an alias for std::variant.
|
||||
#if !defined(ABSL_OPTION_USE_STD_VARIANT)
|
||||
#error options.h is misconfigured.
|
||||
#elif ABSL_OPTION_USE_STD_VARIANT == 0 || \
|
||||
(ABSL_OPTION_USE_STD_VARIANT == 2 && !defined(ABSL_HAVE_STD_VARIANT))
|
||||
#undef ABSL_USES_STD_VARIANT
|
||||
#elif ABSL_OPTION_USE_STD_VARIANT == 1 || \
|
||||
(ABSL_OPTION_USE_STD_VARIANT == 2 && defined(ABSL_HAVE_STD_VARIANT))
|
||||
#define ABSL_USES_STD_VARIANT 1
|
||||
#else
|
||||
#error options.h is misconfigured.
|
||||
#endif
|
||||
|
||||
// ABSL_USES_STD_STRING_VIEW
|
||||
//
|
||||
// Indicates whether absl::string_view is an alias for std::string_view.
|
||||
#if !defined(ABSL_OPTION_USE_STD_STRING_VIEW)
|
||||
#error options.h is misconfigured.
|
||||
#elif ABSL_OPTION_USE_STD_STRING_VIEW == 0 || \
|
||||
(ABSL_OPTION_USE_STD_STRING_VIEW == 2 && \
|
||||
!defined(ABSL_HAVE_STD_STRING_VIEW))
|
||||
#undef ABSL_USES_STD_STRING_VIEW
|
||||
#elif ABSL_OPTION_USE_STD_STRING_VIEW == 1 || \
|
||||
(ABSL_OPTION_USE_STD_STRING_VIEW == 2 && \
|
||||
defined(ABSL_HAVE_STD_STRING_VIEW))
|
||||
#define ABSL_USES_STD_STRING_VIEW 1
|
||||
#else
|
||||
#error options.h is misconfigured.
|
||||
#endif
|
||||
|
||||
// In debug mode, MSVC 2017's std::variant throws a EXCEPTION_ACCESS_VIOLATION
|
||||
// SEH exception from emplace for variant<SomeStruct> when constructing the
|
||||
// struct can throw. This defeats some of variant_test and
|
||||
// variant_exception_safety_test.
|
||||
#if defined(_MSC_VER) && _MSC_VER >= 1700 && defined(_DEBUG)
|
||||
#define ABSL_INTERNAL_MSVC_2017_DBG_MODE
|
||||
#endif
|
||||
|
||||
// ABSL_INTERNAL_MANGLED_NS
|
||||
// ABSL_INTERNAL_MANGLED_BACKREFERENCE
|
||||
//
|
||||
// Internal macros for building up mangled names in our internal fork of CCTZ.
|
||||
// This implementation detail is only needed and provided for the MSVC build.
|
||||
//
|
||||
// These macros both expand to string literals. ABSL_INTERNAL_MANGLED_NS is
|
||||
// the mangled spelling of the `absl` namespace, and
|
||||
// ABSL_INTERNAL_MANGLED_BACKREFERENCE is a back-reference integer representing
|
||||
// the proper count to skip past the CCTZ fork namespace names. (This number
|
||||
// is one larger when there is an inline namespace name to skip.)
|
||||
#if defined(_MSC_VER)
|
||||
#if ABSL_OPTION_USE_INLINE_NAMESPACE == 0
|
||||
#define ABSL_INTERNAL_MANGLED_NS "absl"
|
||||
#define ABSL_INTERNAL_MANGLED_BACKREFERENCE "5"
|
||||
#else
|
||||
#define ABSL_INTERNAL_MANGLED_NS \
|
||||
ABSL_INTERNAL_TOKEN_STR(ABSL_OPTION_INLINE_NAMESPACE_NAME) "@absl"
|
||||
#define ABSL_INTERNAL_MANGLED_BACKREFERENCE "6"
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#undef ABSL_INTERNAL_HAS_KEYWORD
|
||||
|
||||
// ABSL_DLL
|
||||
//
|
||||
// When building Abseil as a DLL, this macro expands to `__declspec(dllexport)`
|
||||
// so we can annotate symbols appropriately as being exported. When used in
|
||||
// headers consuming a DLL, this macro expands to `__declspec(dllimport)` so
|
||||
// that consumers know the symbol is defined inside the DLL. In all other cases,
|
||||
// the macro expands to nothing.
|
||||
#if defined(_MSC_VER)
|
||||
#if defined(ABSL_BUILD_DLL)
|
||||
#define ABSL_DLL __declspec(dllexport)
|
||||
#elif defined(ABSL_CONSUME_DLL)
|
||||
#define ABSL_DLL __declspec(dllimport)
|
||||
#else
|
||||
#define ABSL_DLL
|
||||
#endif
|
||||
#else
|
||||
#define ABSL_DLL
|
||||
#endif // defined(_MSC_VER)
|
||||
|
||||
#endif // ABSL_BASE_CONFIG_H_
|
||||
Generated
+76
@@ -0,0 +1,76 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// kConstInit
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// A constructor tag used to mark an object as safe for use as a global
|
||||
// variable, avoiding the usual lifetime issues that can affect globals.
|
||||
|
||||
#ifndef ABSL_BASE_CONST_INIT_H_
|
||||
#define ABSL_BASE_CONST_INIT_H_
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
// In general, objects with static storage duration (such as global variables)
|
||||
// can trigger tricky object lifetime situations. Attempting to access them
|
||||
// from the constructors or destructors of other global objects can result in
|
||||
// undefined behavior, unless their constructors and destructors are designed
|
||||
// with this issue in mind.
|
||||
//
|
||||
// The normal way to deal with this issue in C++11 is to use constant
|
||||
// initialization and trivial destructors.
|
||||
//
|
||||
// Constant initialization is guaranteed to occur before any other code
|
||||
// executes. Constructors that are declared 'constexpr' are eligible for
|
||||
// constant initialization. You can annotate a variable declaration with the
|
||||
// ABSL_CONST_INIT macro to express this intent. For compilers that support
|
||||
// it, this annotation will cause a compilation error for declarations that
|
||||
// aren't subject to constant initialization (perhaps because a runtime value
|
||||
// was passed as a constructor argument).
|
||||
//
|
||||
// On program shutdown, lifetime issues can be avoided on global objects by
|
||||
// ensuring that they contain trivial destructors. A class has a trivial
|
||||
// destructor unless it has a user-defined destructor, a virtual method or base
|
||||
// class, or a data member or base class with a non-trivial destructor of its
|
||||
// own. Objects with static storage duration and a trivial destructor are not
|
||||
// cleaned up on program shutdown, and are thus safe to access from other code
|
||||
// running during shutdown.
|
||||
//
|
||||
// For a few core Abseil classes, we make a best effort to allow for safe global
|
||||
// instances, even though these classes have non-trivial destructors. These
|
||||
// objects can be created with the absl::kConstInit tag. For example:
|
||||
// ABSL_CONST_INIT absl::Mutex global_mutex(absl::kConstInit);
|
||||
//
|
||||
// The line above declares a global variable of type absl::Mutex which can be
|
||||
// accessed at any point during startup or shutdown. global_mutex's destructor
|
||||
// will still run, but will not invalidate the object. Note that C++ specifies
|
||||
// that accessing an object after its destructor has run results in undefined
|
||||
// behavior, but this pattern works on the toolchains we support.
|
||||
//
|
||||
// The absl::kConstInit tag should only be used to define objects with static
|
||||
// or thread_local storage duration.
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
|
||||
enum ConstInitType {
|
||||
kConstInit,
|
||||
};
|
||||
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_CONST_INIT_H_
|
||||
+129
@@ -0,0 +1,129 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
#include "absl/base/dynamic_annotations.h"
|
||||
|
||||
#ifndef __has_feature
|
||||
#define __has_feature(x) 0
|
||||
#endif
|
||||
|
||||
/* Compiler-based ThreadSanitizer defines
|
||||
DYNAMIC_ANNOTATIONS_EXTERNAL_IMPL = 1
|
||||
and provides its own definitions of the functions. */
|
||||
|
||||
#ifndef DYNAMIC_ANNOTATIONS_EXTERNAL_IMPL
|
||||
# define DYNAMIC_ANNOTATIONS_EXTERNAL_IMPL 0
|
||||
#endif
|
||||
|
||||
/* Each function is empty and called (via a macro) only in debug mode.
|
||||
The arguments are captured by dynamic tools at runtime. */
|
||||
|
||||
#if DYNAMIC_ANNOTATIONS_EXTERNAL_IMPL == 0 && !defined(__native_client__)
|
||||
|
||||
#if __has_feature(memory_sanitizer)
|
||||
#include <sanitizer/msan_interface.h>
|
||||
#endif
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
void AnnotateRWLockCreate(const char *, int,
|
||||
const volatile void *){}
|
||||
void AnnotateRWLockDestroy(const char *, int,
|
||||
const volatile void *){}
|
||||
void AnnotateRWLockAcquired(const char *, int,
|
||||
const volatile void *, long){}
|
||||
void AnnotateRWLockReleased(const char *, int,
|
||||
const volatile void *, long){}
|
||||
void AnnotateBenignRace(const char *, int,
|
||||
const volatile void *,
|
||||
const char *){}
|
||||
void AnnotateBenignRaceSized(const char *, int,
|
||||
const volatile void *,
|
||||
size_t,
|
||||
const char *) {}
|
||||
void AnnotateThreadName(const char *, int,
|
||||
const char *){}
|
||||
void AnnotateIgnoreReadsBegin(const char *, int){}
|
||||
void AnnotateIgnoreReadsEnd(const char *, int){}
|
||||
void AnnotateIgnoreWritesBegin(const char *, int){}
|
||||
void AnnotateIgnoreWritesEnd(const char *, int){}
|
||||
void AnnotateEnableRaceDetection(const char *, int, int){}
|
||||
void AnnotateMemoryIsInitialized(const char *, int,
|
||||
const volatile void *mem, size_t size) {
|
||||
#if __has_feature(memory_sanitizer)
|
||||
__msan_unpoison(mem, size);
|
||||
#else
|
||||
(void)mem;
|
||||
(void)size;
|
||||
#endif
|
||||
}
|
||||
|
||||
void AnnotateMemoryIsUninitialized(const char *, int,
|
||||
const volatile void *mem, size_t size) {
|
||||
#if __has_feature(memory_sanitizer)
|
||||
__msan_allocated_memory(mem, size);
|
||||
#else
|
||||
(void)mem;
|
||||
(void)size;
|
||||
#endif
|
||||
}
|
||||
|
||||
static int GetRunningOnValgrind(void) {
|
||||
#ifdef RUNNING_ON_VALGRIND
|
||||
if (RUNNING_ON_VALGRIND) return 1;
|
||||
#endif
|
||||
char *running_on_valgrind_str = getenv("RUNNING_ON_VALGRIND");
|
||||
if (running_on_valgrind_str) {
|
||||
return strcmp(running_on_valgrind_str, "0") != 0;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* See the comments in dynamic_annotations.h */
|
||||
int RunningOnValgrind(void) {
|
||||
static volatile int running_on_valgrind = -1;
|
||||
int local_running_on_valgrind = running_on_valgrind;
|
||||
/* C doesn't have thread-safe initialization of statics, and we
|
||||
don't want to depend on pthread_once here, so hack it. */
|
||||
ANNOTATE_BENIGN_RACE(&running_on_valgrind, "safe hack");
|
||||
if (local_running_on_valgrind == -1)
|
||||
running_on_valgrind = local_running_on_valgrind = GetRunningOnValgrind();
|
||||
return local_running_on_valgrind;
|
||||
}
|
||||
|
||||
/* See the comments in dynamic_annotations.h */
|
||||
double ValgrindSlowdown(void) {
|
||||
/* Same initialization hack as in RunningOnValgrind(). */
|
||||
static volatile double slowdown = 0.0;
|
||||
double local_slowdown = slowdown;
|
||||
ANNOTATE_BENIGN_RACE(&slowdown, "safe hack");
|
||||
if (RunningOnValgrind() == 0) {
|
||||
return 1.0;
|
||||
}
|
||||
if (local_slowdown == 0.0) {
|
||||
char *env = getenv("VALGRIND_SLOWDOWN");
|
||||
slowdown = local_slowdown = env ? atof(env) : 50.0;
|
||||
}
|
||||
return local_slowdown;
|
||||
}
|
||||
|
||||
#ifdef __cplusplus
|
||||
} // extern "C"
|
||||
#endif
|
||||
#endif /* DYNAMIC_ANNOTATIONS_EXTERNAL_IMPL == 0 */
|
||||
+389
@@ -0,0 +1,389 @@
|
||||
/*
|
||||
* Copyright 2017 The Abseil Authors.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
/* This file defines dynamic annotations for use with dynamic analysis
|
||||
tool such as valgrind, PIN, etc.
|
||||
|
||||
Dynamic annotation is a source code annotation that affects
|
||||
the generated code (that is, the annotation is not a comment).
|
||||
Each such annotation is attached to a particular
|
||||
instruction and/or to a particular object (address) in the program.
|
||||
|
||||
The annotations that should be used by users are macros in all upper-case
|
||||
(e.g., ANNOTATE_THREAD_NAME).
|
||||
|
||||
Actual implementation of these macros may differ depending on the
|
||||
dynamic analysis tool being used.
|
||||
|
||||
This file supports the following configurations:
|
||||
- Dynamic Annotations enabled (with static thread-safety warnings disabled).
|
||||
In this case, macros expand to functions implemented by Thread Sanitizer,
|
||||
when building with TSan. When not provided an external implementation,
|
||||
dynamic_annotations.cc provides no-op implementations.
|
||||
|
||||
- Static Clang thread-safety warnings enabled.
|
||||
When building with a Clang compiler that supports thread-safety warnings,
|
||||
a subset of annotations can be statically-checked at compile-time. We
|
||||
expand these macros to static-inline functions that can be analyzed for
|
||||
thread-safety, but afterwards elided when building the final binary.
|
||||
|
||||
- All annotations are disabled.
|
||||
If neither Dynamic Annotations nor Clang thread-safety warnings are
|
||||
enabled, then all annotation-macros expand to empty. */
|
||||
|
||||
#ifndef ABSL_BASE_DYNAMIC_ANNOTATIONS_H_
|
||||
#define ABSL_BASE_DYNAMIC_ANNOTATIONS_H_
|
||||
|
||||
#ifndef DYNAMIC_ANNOTATIONS_ENABLED
|
||||
# define DYNAMIC_ANNOTATIONS_ENABLED 0
|
||||
#endif
|
||||
|
||||
#if DYNAMIC_ANNOTATIONS_ENABLED != 0
|
||||
|
||||
/* -------------------------------------------------------------
|
||||
Annotations that suppress errors. It is usually better to express the
|
||||
program's synchronization using the other annotations, but these can
|
||||
be used when all else fails. */
|
||||
|
||||
/* Report that we may have a benign race at "pointer", with size
|
||||
"sizeof(*(pointer))". "pointer" must be a non-void* pointer. Insert at the
|
||||
point where "pointer" has been allocated, preferably close to the point
|
||||
where the race happens. See also ANNOTATE_BENIGN_RACE_STATIC. */
|
||||
#define ANNOTATE_BENIGN_RACE(pointer, description) \
|
||||
AnnotateBenignRaceSized(__FILE__, __LINE__, pointer, \
|
||||
sizeof(*(pointer)), description)
|
||||
|
||||
/* Same as ANNOTATE_BENIGN_RACE(address, description), but applies to
|
||||
the memory range [address, address+size). */
|
||||
#define ANNOTATE_BENIGN_RACE_SIZED(address, size, description) \
|
||||
AnnotateBenignRaceSized(__FILE__, __LINE__, address, size, description)
|
||||
|
||||
/* Enable (enable!=0) or disable (enable==0) race detection for all threads.
|
||||
This annotation could be useful if you want to skip expensive race analysis
|
||||
during some period of program execution, e.g. during initialization. */
|
||||
#define ANNOTATE_ENABLE_RACE_DETECTION(enable) \
|
||||
AnnotateEnableRaceDetection(__FILE__, __LINE__, enable)
|
||||
|
||||
/* -------------------------------------------------------------
|
||||
Annotations useful for debugging. */
|
||||
|
||||
/* Report the current thread name to a race detector. */
|
||||
#define ANNOTATE_THREAD_NAME(name) \
|
||||
AnnotateThreadName(__FILE__, __LINE__, name)
|
||||
|
||||
/* -------------------------------------------------------------
|
||||
Annotations useful when implementing locks. They are not
|
||||
normally needed by modules that merely use locks.
|
||||
The "lock" argument is a pointer to the lock object. */
|
||||
|
||||
/* Report that a lock has been created at address "lock". */
|
||||
#define ANNOTATE_RWLOCK_CREATE(lock) \
|
||||
AnnotateRWLockCreate(__FILE__, __LINE__, lock)
|
||||
|
||||
/* Report that a linker initialized lock has been created at address "lock".
|
||||
*/
|
||||
#ifdef THREAD_SANITIZER
|
||||
#define ANNOTATE_RWLOCK_CREATE_STATIC(lock) \
|
||||
AnnotateRWLockCreateStatic(__FILE__, __LINE__, lock)
|
||||
#else
|
||||
#define ANNOTATE_RWLOCK_CREATE_STATIC(lock) ANNOTATE_RWLOCK_CREATE(lock)
|
||||
#endif
|
||||
|
||||
/* Report that the lock at address "lock" is about to be destroyed. */
|
||||
#define ANNOTATE_RWLOCK_DESTROY(lock) \
|
||||
AnnotateRWLockDestroy(__FILE__, __LINE__, lock)
|
||||
|
||||
/* Report that the lock at address "lock" has been acquired.
|
||||
is_w=1 for writer lock, is_w=0 for reader lock. */
|
||||
#define ANNOTATE_RWLOCK_ACQUIRED(lock, is_w) \
|
||||
AnnotateRWLockAcquired(__FILE__, __LINE__, lock, is_w)
|
||||
|
||||
/* Report that the lock at address "lock" is about to be released. */
|
||||
#define ANNOTATE_RWLOCK_RELEASED(lock, is_w) \
|
||||
AnnotateRWLockReleased(__FILE__, __LINE__, lock, is_w)
|
||||
|
||||
#else /* DYNAMIC_ANNOTATIONS_ENABLED == 0 */
|
||||
|
||||
#define ANNOTATE_RWLOCK_CREATE(lock) /* empty */
|
||||
#define ANNOTATE_RWLOCK_CREATE_STATIC(lock) /* empty */
|
||||
#define ANNOTATE_RWLOCK_DESTROY(lock) /* empty */
|
||||
#define ANNOTATE_RWLOCK_ACQUIRED(lock, is_w) /* empty */
|
||||
#define ANNOTATE_RWLOCK_RELEASED(lock, is_w) /* empty */
|
||||
#define ANNOTATE_BENIGN_RACE(address, description) /* empty */
|
||||
#define ANNOTATE_BENIGN_RACE_SIZED(address, size, description) /* empty */
|
||||
#define ANNOTATE_THREAD_NAME(name) /* empty */
|
||||
#define ANNOTATE_ENABLE_RACE_DETECTION(enable) /* empty */
|
||||
|
||||
#endif /* DYNAMIC_ANNOTATIONS_ENABLED */
|
||||
|
||||
/* These annotations are also made available to LLVM's Memory Sanitizer */
|
||||
#if DYNAMIC_ANNOTATIONS_ENABLED == 1 || defined(MEMORY_SANITIZER)
|
||||
#define ANNOTATE_MEMORY_IS_INITIALIZED(address, size) \
|
||||
AnnotateMemoryIsInitialized(__FILE__, __LINE__, address, size)
|
||||
|
||||
#define ANNOTATE_MEMORY_IS_UNINITIALIZED(address, size) \
|
||||
AnnotateMemoryIsUninitialized(__FILE__, __LINE__, address, size)
|
||||
#else
|
||||
#define ANNOTATE_MEMORY_IS_INITIALIZED(address, size) /* empty */
|
||||
#define ANNOTATE_MEMORY_IS_UNINITIALIZED(address, size) /* empty */
|
||||
#endif /* DYNAMIC_ANNOTATIONS_ENABLED || MEMORY_SANITIZER */
|
||||
|
||||
/* TODO(delesley) -- Replace __CLANG_SUPPORT_DYN_ANNOTATION__ with the
|
||||
appropriate feature ID. */
|
||||
#if defined(__clang__) && (!defined(SWIG)) \
|
||||
&& defined(__CLANG_SUPPORT_DYN_ANNOTATION__)
|
||||
|
||||
#if DYNAMIC_ANNOTATIONS_ENABLED == 0
|
||||
#define ANNOTALYSIS_ENABLED
|
||||
#endif
|
||||
|
||||
/* When running in opt-mode, GCC will issue a warning, if these attributes are
|
||||
compiled. Only include them when compiling using Clang. */
|
||||
#define ATTRIBUTE_IGNORE_READS_BEGIN \
|
||||
__attribute((exclusive_lock_function("*")))
|
||||
#define ATTRIBUTE_IGNORE_READS_END \
|
||||
__attribute((unlock_function("*")))
|
||||
#else
|
||||
#define ATTRIBUTE_IGNORE_READS_BEGIN /* empty */
|
||||
#define ATTRIBUTE_IGNORE_READS_END /* empty */
|
||||
#endif /* defined(__clang__) && ... */
|
||||
|
||||
#if (DYNAMIC_ANNOTATIONS_ENABLED != 0) || defined(ANNOTALYSIS_ENABLED)
|
||||
#define ANNOTATIONS_ENABLED
|
||||
#endif
|
||||
|
||||
#if (DYNAMIC_ANNOTATIONS_ENABLED != 0)
|
||||
|
||||
/* Request the analysis tool to ignore all reads in the current thread
|
||||
until ANNOTATE_IGNORE_READS_END is called.
|
||||
Useful to ignore intentional racey reads, while still checking
|
||||
other reads and all writes.
|
||||
See also ANNOTATE_UNPROTECTED_READ. */
|
||||
#define ANNOTATE_IGNORE_READS_BEGIN() \
|
||||
AnnotateIgnoreReadsBegin(__FILE__, __LINE__)
|
||||
|
||||
/* Stop ignoring reads. */
|
||||
#define ANNOTATE_IGNORE_READS_END() \
|
||||
AnnotateIgnoreReadsEnd(__FILE__, __LINE__)
|
||||
|
||||
/* Similar to ANNOTATE_IGNORE_READS_BEGIN, but ignore writes instead. */
|
||||
#define ANNOTATE_IGNORE_WRITES_BEGIN() \
|
||||
AnnotateIgnoreWritesBegin(__FILE__, __LINE__)
|
||||
|
||||
/* Stop ignoring writes. */
|
||||
#define ANNOTATE_IGNORE_WRITES_END() \
|
||||
AnnotateIgnoreWritesEnd(__FILE__, __LINE__)
|
||||
|
||||
/* Clang provides limited support for static thread-safety analysis
|
||||
through a feature called Annotalysis. We configure macro-definitions
|
||||
according to whether Annotalysis support is available. */
|
||||
#elif defined(ANNOTALYSIS_ENABLED)
|
||||
|
||||
#define ANNOTATE_IGNORE_READS_BEGIN() \
|
||||
StaticAnnotateIgnoreReadsBegin(__FILE__, __LINE__)
|
||||
|
||||
#define ANNOTATE_IGNORE_READS_END() \
|
||||
StaticAnnotateIgnoreReadsEnd(__FILE__, __LINE__)
|
||||
|
||||
#define ANNOTATE_IGNORE_WRITES_BEGIN() \
|
||||
StaticAnnotateIgnoreWritesBegin(__FILE__, __LINE__)
|
||||
|
||||
#define ANNOTATE_IGNORE_WRITES_END() \
|
||||
StaticAnnotateIgnoreWritesEnd(__FILE__, __LINE__)
|
||||
|
||||
#else
|
||||
#define ANNOTATE_IGNORE_READS_BEGIN() /* empty */
|
||||
#define ANNOTATE_IGNORE_READS_END() /* empty */
|
||||
#define ANNOTATE_IGNORE_WRITES_BEGIN() /* empty */
|
||||
#define ANNOTATE_IGNORE_WRITES_END() /* empty */
|
||||
#endif
|
||||
|
||||
/* Implement the ANNOTATE_IGNORE_READS_AND_WRITES_* annotations using the more
|
||||
primitive annotations defined above. */
|
||||
#if defined(ANNOTATIONS_ENABLED)
|
||||
|
||||
/* Start ignoring all memory accesses (both reads and writes). */
|
||||
#define ANNOTATE_IGNORE_READS_AND_WRITES_BEGIN() \
|
||||
do { \
|
||||
ANNOTATE_IGNORE_READS_BEGIN(); \
|
||||
ANNOTATE_IGNORE_WRITES_BEGIN(); \
|
||||
}while (0)
|
||||
|
||||
/* Stop ignoring both reads and writes. */
|
||||
#define ANNOTATE_IGNORE_READS_AND_WRITES_END() \
|
||||
do { \
|
||||
ANNOTATE_IGNORE_WRITES_END(); \
|
||||
ANNOTATE_IGNORE_READS_END(); \
|
||||
}while (0)
|
||||
|
||||
#else
|
||||
#define ANNOTATE_IGNORE_READS_AND_WRITES_BEGIN() /* empty */
|
||||
#define ANNOTATE_IGNORE_READS_AND_WRITES_END() /* empty */
|
||||
#endif
|
||||
|
||||
/* Use the macros above rather than using these functions directly. */
|
||||
#include <stddef.h>
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
void AnnotateRWLockCreate(const char *file, int line,
|
||||
const volatile void *lock);
|
||||
void AnnotateRWLockCreateStatic(const char *file, int line,
|
||||
const volatile void *lock);
|
||||
void AnnotateRWLockDestroy(const char *file, int line,
|
||||
const volatile void *lock);
|
||||
void AnnotateRWLockAcquired(const char *file, int line,
|
||||
const volatile void *lock, long is_w); /* NOLINT */
|
||||
void AnnotateRWLockReleased(const char *file, int line,
|
||||
const volatile void *lock, long is_w); /* NOLINT */
|
||||
void AnnotateBenignRace(const char *file, int line,
|
||||
const volatile void *address,
|
||||
const char *description);
|
||||
void AnnotateBenignRaceSized(const char *file, int line,
|
||||
const volatile void *address,
|
||||
size_t size,
|
||||
const char *description);
|
||||
void AnnotateThreadName(const char *file, int line,
|
||||
const char *name);
|
||||
void AnnotateEnableRaceDetection(const char *file, int line, int enable);
|
||||
void AnnotateMemoryIsInitialized(const char *file, int line,
|
||||
const volatile void *mem, size_t size);
|
||||
void AnnotateMemoryIsUninitialized(const char *file, int line,
|
||||
const volatile void *mem, size_t size);
|
||||
|
||||
/* Annotations expand to these functions, when Dynamic Annotations are enabled.
|
||||
These functions are either implemented as no-op calls, if no Sanitizer is
|
||||
attached, or provided with externally-linked implementations by a library
|
||||
like ThreadSanitizer. */
|
||||
void AnnotateIgnoreReadsBegin(const char *file, int line)
|
||||
ATTRIBUTE_IGNORE_READS_BEGIN;
|
||||
void AnnotateIgnoreReadsEnd(const char *file, int line)
|
||||
ATTRIBUTE_IGNORE_READS_END;
|
||||
void AnnotateIgnoreWritesBegin(const char *file, int line);
|
||||
void AnnotateIgnoreWritesEnd(const char *file, int line);
|
||||
|
||||
#if defined(ANNOTALYSIS_ENABLED)
|
||||
/* When Annotalysis is enabled without Dynamic Annotations, the use of
|
||||
static-inline functions allows the annotations to be read at compile-time,
|
||||
while still letting the compiler elide the functions from the final build.
|
||||
|
||||
TODO(delesley) -- The exclusive lock here ignores writes as well, but
|
||||
allows IGNORE_READS_AND_WRITES to work properly. */
|
||||
#pragma GCC diagnostic push
|
||||
#pragma GCC diagnostic ignored "-Wunused-function"
|
||||
static inline void StaticAnnotateIgnoreReadsBegin(const char *file, int line)
|
||||
ATTRIBUTE_IGNORE_READS_BEGIN { (void)file; (void)line; }
|
||||
static inline void StaticAnnotateIgnoreReadsEnd(const char *file, int line)
|
||||
ATTRIBUTE_IGNORE_READS_END { (void)file; (void)line; }
|
||||
static inline void StaticAnnotateIgnoreWritesBegin(
|
||||
const char *file, int line) { (void)file; (void)line; }
|
||||
static inline void StaticAnnotateIgnoreWritesEnd(
|
||||
const char *file, int line) { (void)file; (void)line; }
|
||||
#pragma GCC diagnostic pop
|
||||
#endif
|
||||
|
||||
/* Return non-zero value if running under valgrind.
|
||||
|
||||
If "valgrind.h" is included into dynamic_annotations.cc,
|
||||
the regular valgrind mechanism will be used.
|
||||
See http://valgrind.org/docs/manual/manual-core-adv.html about
|
||||
RUNNING_ON_VALGRIND and other valgrind "client requests".
|
||||
The file "valgrind.h" may be obtained by doing
|
||||
svn co svn://svn.valgrind.org/valgrind/trunk/include
|
||||
|
||||
If for some reason you can't use "valgrind.h" or want to fake valgrind,
|
||||
there are two ways to make this function return non-zero:
|
||||
- Use environment variable: export RUNNING_ON_VALGRIND=1
|
||||
- Make your tool intercept the function RunningOnValgrind() and
|
||||
change its return value.
|
||||
*/
|
||||
int RunningOnValgrind(void);
|
||||
|
||||
/* ValgrindSlowdown returns:
|
||||
* 1.0, if (RunningOnValgrind() == 0)
|
||||
* 50.0, if (RunningOnValgrind() != 0 && getenv("VALGRIND_SLOWDOWN") == NULL)
|
||||
* atof(getenv("VALGRIND_SLOWDOWN")) otherwise
|
||||
This function can be used to scale timeout values:
|
||||
EXAMPLE:
|
||||
for (;;) {
|
||||
DoExpensiveBackgroundTask();
|
||||
SleepForSeconds(5 * ValgrindSlowdown());
|
||||
}
|
||||
*/
|
||||
double ValgrindSlowdown(void);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
|
||||
/* ANNOTATE_UNPROTECTED_READ is the preferred way to annotate racey reads.
|
||||
|
||||
Instead of doing
|
||||
ANNOTATE_IGNORE_READS_BEGIN();
|
||||
... = x;
|
||||
ANNOTATE_IGNORE_READS_END();
|
||||
one can use
|
||||
... = ANNOTATE_UNPROTECTED_READ(x); */
|
||||
#if defined(__cplusplus) && defined(ANNOTATIONS_ENABLED)
|
||||
template <typename T>
|
||||
inline T ANNOTATE_UNPROTECTED_READ(const volatile T &x) { /* NOLINT */
|
||||
ANNOTATE_IGNORE_READS_BEGIN();
|
||||
T res = x;
|
||||
ANNOTATE_IGNORE_READS_END();
|
||||
return res;
|
||||
}
|
||||
#else
|
||||
#define ANNOTATE_UNPROTECTED_READ(x) (x)
|
||||
#endif
|
||||
|
||||
#if DYNAMIC_ANNOTATIONS_ENABLED != 0 && defined(__cplusplus)
|
||||
/* Apply ANNOTATE_BENIGN_RACE_SIZED to a static variable. */
|
||||
#define ANNOTATE_BENIGN_RACE_STATIC(static_var, description) \
|
||||
namespace { \
|
||||
class static_var ## _annotator { \
|
||||
public: \
|
||||
static_var ## _annotator() { \
|
||||
ANNOTATE_BENIGN_RACE_SIZED(&static_var, \
|
||||
sizeof(static_var), \
|
||||
# static_var ": " description); \
|
||||
} \
|
||||
}; \
|
||||
static static_var ## _annotator the ## static_var ## _annotator;\
|
||||
} // namespace
|
||||
#else /* DYNAMIC_ANNOTATIONS_ENABLED == 0 */
|
||||
#define ANNOTATE_BENIGN_RACE_STATIC(static_var, description) /* empty */
|
||||
#endif /* DYNAMIC_ANNOTATIONS_ENABLED */
|
||||
|
||||
#ifdef ADDRESS_SANITIZER
|
||||
/* Describe the current state of a contiguous container such as e.g.
|
||||
* std::vector or std::string. For more details see
|
||||
* sanitizer/common_interface_defs.h, which is provided by the compiler. */
|
||||
#include <sanitizer/common_interface_defs.h>
|
||||
#define ANNOTATE_CONTIGUOUS_CONTAINER(beg, end, old_mid, new_mid) \
|
||||
__sanitizer_annotate_contiguous_container(beg, end, old_mid, new_mid)
|
||||
#define ADDRESS_SANITIZER_REDZONE(name) \
|
||||
struct { char x[8] __attribute__ ((aligned (8))); } name
|
||||
#else
|
||||
#define ANNOTATE_CONTIGUOUS_CONTAINER(beg, end, old_mid, new_mid)
|
||||
#define ADDRESS_SANITIZER_REDZONE(name) static_assert(true, "")
|
||||
#endif // ADDRESS_SANITIZER
|
||||
|
||||
/* Undefine the macros intended only in this file. */
|
||||
#undef ANNOTALYSIS_ENABLED
|
||||
#undef ANNOTATIONS_ENABLED
|
||||
#undef ATTRIBUTE_IGNORE_READS_BEGIN
|
||||
#undef ATTRIBUTE_IGNORE_READS_END
|
||||
|
||||
#endif /* ABSL_BASE_DYNAMIC_ANNOTATIONS_H_ */
|
||||
+200
@@ -0,0 +1,200 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_ATOMIC_HOOK_H_
|
||||
#define ABSL_BASE_INTERNAL_ATOMIC_HOOK_H_
|
||||
|
||||
#include <atomic>
|
||||
#include <cassert>
|
||||
#include <cstdint>
|
||||
#include <utility>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/config.h"
|
||||
|
||||
#if defined(_MSC_VER) && !defined(__clang__)
|
||||
#define ABSL_HAVE_WORKING_CONSTEXPR_STATIC_INIT 0
|
||||
#else
|
||||
#define ABSL_HAVE_WORKING_CONSTEXPR_STATIC_INIT 1
|
||||
#endif
|
||||
|
||||
#if defined(_MSC_VER)
|
||||
#define ABSL_HAVE_WORKING_ATOMIC_POINTER 0
|
||||
#else
|
||||
#define ABSL_HAVE_WORKING_ATOMIC_POINTER 1
|
||||
#endif
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
template <typename T>
|
||||
class AtomicHook;
|
||||
|
||||
// To workaround AtomicHook not being constant-initializable on some platforms,
|
||||
// prefer to annotate instances with `ABSL_INTERNAL_ATOMIC_HOOK_ATTRIBUTES`
|
||||
// instead of `ABSL_CONST_INIT`.
|
||||
#if ABSL_HAVE_WORKING_CONSTEXPR_STATIC_INIT
|
||||
#define ABSL_INTERNAL_ATOMIC_HOOK_ATTRIBUTES ABSL_CONST_INIT
|
||||
#else
|
||||
#define ABSL_INTERNAL_ATOMIC_HOOK_ATTRIBUTES
|
||||
#endif
|
||||
|
||||
// `AtomicHook` is a helper class, templatized on a raw function pointer type,
|
||||
// for implementing Abseil customization hooks. It is a callable object that
|
||||
// dispatches to the registered hook. Objects of type `AtomicHook` must have
|
||||
// static or thread storage duration.
|
||||
//
|
||||
// A default constructed object performs a no-op (and returns a default
|
||||
// constructed object) if no hook has been registered.
|
||||
//
|
||||
// Hooks can be pre-registered via constant initialization, for example:
|
||||
//
|
||||
// ABSL_INTERNAL_ATOMIC_HOOK_ATTRIBUTES static AtomicHook<void(*)()>
|
||||
// my_hook(DefaultAction);
|
||||
//
|
||||
// and then changed at runtime via a call to `Store()`.
|
||||
//
|
||||
// Reads and writes guarantee memory_order_acquire/memory_order_release
|
||||
// semantics.
|
||||
template <typename ReturnType, typename... Args>
|
||||
class AtomicHook<ReturnType (*)(Args...)> {
|
||||
public:
|
||||
using FnPtr = ReturnType (*)(Args...);
|
||||
|
||||
// Constructs an object that by default performs a no-op (and
|
||||
// returns a default constructed object) when no hook as been registered.
|
||||
constexpr AtomicHook() : AtomicHook(DummyFunction) {}
|
||||
|
||||
// Constructs an object that by default dispatches to/returns the
|
||||
// pre-registered default_fn when no hook has been registered at runtime.
|
||||
#if ABSL_HAVE_WORKING_ATOMIC_POINTER && ABSL_HAVE_WORKING_CONSTEXPR_STATIC_INIT
|
||||
explicit constexpr AtomicHook(FnPtr default_fn)
|
||||
: hook_(default_fn), default_fn_(default_fn) {}
|
||||
#elif ABSL_HAVE_WORKING_CONSTEXPR_STATIC_INIT
|
||||
explicit constexpr AtomicHook(FnPtr default_fn)
|
||||
: hook_(kUninitialized), default_fn_(default_fn) {}
|
||||
#else
|
||||
// As of January 2020, on all known versions of MSVC this constructor runs in
|
||||
// the global constructor sequence. If `Store()` is called by a dynamic
|
||||
// initializer, we want to preserve the value, even if this constructor runs
|
||||
// after the call to `Store()`. If not, `hook_` will be
|
||||
// zero-initialized by the linker and we have no need to set it.
|
||||
// https://developercommunity.visualstudio.com/content/problem/336946/class-with-constexpr-constructor-not-using-static.html
|
||||
explicit constexpr AtomicHook(FnPtr default_fn)
|
||||
: /* hook_(deliberately omitted), */ default_fn_(default_fn) {
|
||||
static_assert(kUninitialized == 0, "here we rely on zero-initialization");
|
||||
}
|
||||
#endif
|
||||
|
||||
// Stores the provided function pointer as the value for this hook.
|
||||
//
|
||||
// This is intended to be called once. Multiple calls are legal only if the
|
||||
// same function pointer is provided for each call. The store is implemented
|
||||
// as a memory_order_release operation, and read accesses are implemented as
|
||||
// memory_order_acquire.
|
||||
void Store(FnPtr fn) {
|
||||
bool success = DoStore(fn);
|
||||
static_cast<void>(success);
|
||||
assert(success);
|
||||
}
|
||||
|
||||
// Invokes the registered callback. If no callback has yet been registered, a
|
||||
// default-constructed object of the appropriate type is returned instead.
|
||||
template <typename... CallArgs>
|
||||
ReturnType operator()(CallArgs&&... args) const {
|
||||
return DoLoad()(std::forward<CallArgs>(args)...);
|
||||
}
|
||||
|
||||
// Returns the registered callback, or nullptr if none has been registered.
|
||||
// Useful if client code needs to conditionalize behavior based on whether a
|
||||
// callback was registered.
|
||||
//
|
||||
// Note that atomic_hook.Load()() and atomic_hook() have different semantics:
|
||||
// operator()() will perform a no-op if no callback was registered, while
|
||||
// Load()() will dereference a null function pointer. Prefer operator()() to
|
||||
// Load()() unless you must conditionalize behavior on whether a hook was
|
||||
// registered.
|
||||
FnPtr Load() const {
|
||||
FnPtr ptr = DoLoad();
|
||||
return (ptr == DummyFunction) ? nullptr : ptr;
|
||||
}
|
||||
|
||||
private:
|
||||
static ReturnType DummyFunction(Args...) {
|
||||
return ReturnType();
|
||||
}
|
||||
|
||||
// Current versions of MSVC (as of September 2017) have a broken
|
||||
// implementation of std::atomic<T*>: Its constructor attempts to do the
|
||||
// equivalent of a reinterpret_cast in a constexpr context, which is not
|
||||
// allowed.
|
||||
//
|
||||
// This causes an issue when building with LLVM under Windows. To avoid this,
|
||||
// we use a less-efficient, intptr_t-based implementation on Windows.
|
||||
#if ABSL_HAVE_WORKING_ATOMIC_POINTER
|
||||
// Return the stored value, or DummyFunction if no value has been stored.
|
||||
FnPtr DoLoad() const { return hook_.load(std::memory_order_acquire); }
|
||||
|
||||
// Store the given value. Returns false if a different value was already
|
||||
// stored to this object.
|
||||
bool DoStore(FnPtr fn) {
|
||||
assert(fn);
|
||||
FnPtr expected = default_fn_;
|
||||
const bool store_succeeded = hook_.compare_exchange_strong(
|
||||
expected, fn, std::memory_order_acq_rel, std::memory_order_acquire);
|
||||
const bool same_value_already_stored = (expected == fn);
|
||||
return store_succeeded || same_value_already_stored;
|
||||
}
|
||||
|
||||
std::atomic<FnPtr> hook_;
|
||||
#else // !ABSL_HAVE_WORKING_ATOMIC_POINTER
|
||||
// Use a sentinel value unlikely to be the address of an actual function.
|
||||
static constexpr intptr_t kUninitialized = 0;
|
||||
|
||||
static_assert(sizeof(intptr_t) >= sizeof(FnPtr),
|
||||
"intptr_t can't contain a function pointer");
|
||||
|
||||
FnPtr DoLoad() const {
|
||||
const intptr_t value = hook_.load(std::memory_order_acquire);
|
||||
if (value == kUninitialized) {
|
||||
return default_fn_;
|
||||
}
|
||||
return reinterpret_cast<FnPtr>(value);
|
||||
}
|
||||
|
||||
bool DoStore(FnPtr fn) {
|
||||
assert(fn);
|
||||
const auto value = reinterpret_cast<intptr_t>(fn);
|
||||
intptr_t expected = kUninitialized;
|
||||
const bool store_succeeded = hook_.compare_exchange_strong(
|
||||
expected, value, std::memory_order_acq_rel, std::memory_order_acquire);
|
||||
const bool same_value_already_stored = (expected == value);
|
||||
return store_succeeded || same_value_already_stored;
|
||||
}
|
||||
|
||||
std::atomic<intptr_t> hook_;
|
||||
#endif
|
||||
|
||||
const FnPtr default_fn_;
|
||||
};
|
||||
|
||||
#undef ABSL_HAVE_WORKING_ATOMIC_POINTER
|
||||
#undef ABSL_HAVE_WORKING_CONSTEXPR_STATIC_INIT
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_ATOMIC_HOOK_H_
|
||||
Generated
+218
@@ -0,0 +1,218 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_BITS_H_
|
||||
#define ABSL_BASE_INTERNAL_BITS_H_
|
||||
|
||||
// This file contains bitwise ops which are implementation details of various
|
||||
// absl libraries.
|
||||
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
// Clang on Windows has __builtin_clzll; otherwise we need to use the
|
||||
// windows intrinsic functions.
|
||||
#if defined(_MSC_VER)
|
||||
#include <intrin.h>
|
||||
#if defined(_M_X64)
|
||||
#pragma intrinsic(_BitScanReverse64)
|
||||
#pragma intrinsic(_BitScanForward64)
|
||||
#endif
|
||||
#pragma intrinsic(_BitScanReverse)
|
||||
#pragma intrinsic(_BitScanForward)
|
||||
#endif
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
|
||||
#if defined(_MSC_VER)
|
||||
// We can achieve something similar to attribute((always_inline)) with MSVC by
|
||||
// using the __forceinline keyword, however this is not perfect. MSVC is
|
||||
// much less aggressive about inlining, and even with the __forceinline keyword.
|
||||
#define ABSL_BASE_INTERNAL_FORCEINLINE __forceinline
|
||||
#else
|
||||
// Use default attribute inline.
|
||||
#define ABSL_BASE_INTERNAL_FORCEINLINE inline ABSL_ATTRIBUTE_ALWAYS_INLINE
|
||||
#endif
|
||||
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
ABSL_BASE_INTERNAL_FORCEINLINE int CountLeadingZeros64Slow(uint64_t n) {
|
||||
int zeroes = 60;
|
||||
if (n >> 32) {
|
||||
zeroes -= 32;
|
||||
n >>= 32;
|
||||
}
|
||||
if (n >> 16) {
|
||||
zeroes -= 16;
|
||||
n >>= 16;
|
||||
}
|
||||
if (n >> 8) {
|
||||
zeroes -= 8;
|
||||
n >>= 8;
|
||||
}
|
||||
if (n >> 4) {
|
||||
zeroes -= 4;
|
||||
n >>= 4;
|
||||
}
|
||||
return "\4\3\2\2\1\1\1\1\0\0\0\0\0\0\0"[n] + zeroes;
|
||||
}
|
||||
|
||||
ABSL_BASE_INTERNAL_FORCEINLINE int CountLeadingZeros64(uint64_t n) {
|
||||
#if defined(_MSC_VER) && defined(_M_X64)
|
||||
// MSVC does not have __buitin_clzll. Use _BitScanReverse64.
|
||||
unsigned long result = 0; // NOLINT(runtime/int)
|
||||
if (_BitScanReverse64(&result, n)) {
|
||||
return 63 - result;
|
||||
}
|
||||
return 64;
|
||||
#elif defined(_MSC_VER)
|
||||
// MSVC does not have __buitin_clzll. Compose two calls to _BitScanReverse
|
||||
unsigned long result = 0; // NOLINT(runtime/int)
|
||||
if ((n >> 32) && _BitScanReverse(&result, n >> 32)) {
|
||||
return 31 - result;
|
||||
}
|
||||
if (_BitScanReverse(&result, n)) {
|
||||
return 63 - result;
|
||||
}
|
||||
return 64;
|
||||
#elif defined(__GNUC__)
|
||||
// Use __builtin_clzll, which uses the following instructions:
|
||||
// x86: bsr
|
||||
// ARM64: clz
|
||||
// PPC: cntlzd
|
||||
static_assert(sizeof(unsigned long long) == sizeof(n), // NOLINT(runtime/int)
|
||||
"__builtin_clzll does not take 64-bit arg");
|
||||
|
||||
// Handle 0 as a special case because __builtin_clzll(0) is undefined.
|
||||
if (n == 0) {
|
||||
return 64;
|
||||
}
|
||||
return __builtin_clzll(n);
|
||||
#else
|
||||
return CountLeadingZeros64Slow(n);
|
||||
#endif
|
||||
}
|
||||
|
||||
ABSL_BASE_INTERNAL_FORCEINLINE int CountLeadingZeros32Slow(uint64_t n) {
|
||||
int zeroes = 28;
|
||||
if (n >> 16) {
|
||||
zeroes -= 16;
|
||||
n >>= 16;
|
||||
}
|
||||
if (n >> 8) {
|
||||
zeroes -= 8;
|
||||
n >>= 8;
|
||||
}
|
||||
if (n >> 4) {
|
||||
zeroes -= 4;
|
||||
n >>= 4;
|
||||
}
|
||||
return "\4\3\2\2\1\1\1\1\0\0\0\0\0\0\0"[n] + zeroes;
|
||||
}
|
||||
|
||||
ABSL_BASE_INTERNAL_FORCEINLINE int CountLeadingZeros32(uint32_t n) {
|
||||
#if defined(_MSC_VER)
|
||||
unsigned long result = 0; // NOLINT(runtime/int)
|
||||
if (_BitScanReverse(&result, n)) {
|
||||
return 31 - result;
|
||||
}
|
||||
return 32;
|
||||
#elif defined(__GNUC__)
|
||||
// Use __builtin_clz, which uses the following instructions:
|
||||
// x86: bsr
|
||||
// ARM64: clz
|
||||
// PPC: cntlzd
|
||||
static_assert(sizeof(int) == sizeof(n),
|
||||
"__builtin_clz does not take 32-bit arg");
|
||||
|
||||
// Handle 0 as a special case because __builtin_clz(0) is undefined.
|
||||
if (n == 0) {
|
||||
return 32;
|
||||
}
|
||||
return __builtin_clz(n);
|
||||
#else
|
||||
return CountLeadingZeros32Slow(n);
|
||||
#endif
|
||||
}
|
||||
|
||||
ABSL_BASE_INTERNAL_FORCEINLINE int CountTrailingZerosNonZero64Slow(uint64_t n) {
|
||||
int c = 63;
|
||||
n &= ~n + 1;
|
||||
if (n & 0x00000000FFFFFFFF) c -= 32;
|
||||
if (n & 0x0000FFFF0000FFFF) c -= 16;
|
||||
if (n & 0x00FF00FF00FF00FF) c -= 8;
|
||||
if (n & 0x0F0F0F0F0F0F0F0F) c -= 4;
|
||||
if (n & 0x3333333333333333) c -= 2;
|
||||
if (n & 0x5555555555555555) c -= 1;
|
||||
return c;
|
||||
}
|
||||
|
||||
ABSL_BASE_INTERNAL_FORCEINLINE int CountTrailingZerosNonZero64(uint64_t n) {
|
||||
#if defined(_MSC_VER) && defined(_M_X64)
|
||||
unsigned long result = 0; // NOLINT(runtime/int)
|
||||
_BitScanForward64(&result, n);
|
||||
return result;
|
||||
#elif defined(_MSC_VER)
|
||||
unsigned long result = 0; // NOLINT(runtime/int)
|
||||
if (static_cast<uint32_t>(n) == 0) {
|
||||
_BitScanForward(&result, n >> 32);
|
||||
return result + 32;
|
||||
}
|
||||
_BitScanForward(&result, n);
|
||||
return result;
|
||||
#elif defined(__GNUC__)
|
||||
static_assert(sizeof(unsigned long long) == sizeof(n), // NOLINT(runtime/int)
|
||||
"__builtin_ctzll does not take 64-bit arg");
|
||||
return __builtin_ctzll(n);
|
||||
#else
|
||||
return CountTrailingZerosNonZero64Slow(n);
|
||||
#endif
|
||||
}
|
||||
|
||||
ABSL_BASE_INTERNAL_FORCEINLINE int CountTrailingZerosNonZero32Slow(uint32_t n) {
|
||||
int c = 31;
|
||||
n &= ~n + 1;
|
||||
if (n & 0x0000FFFF) c -= 16;
|
||||
if (n & 0x00FF00FF) c -= 8;
|
||||
if (n & 0x0F0F0F0F) c -= 4;
|
||||
if (n & 0x33333333) c -= 2;
|
||||
if (n & 0x55555555) c -= 1;
|
||||
return c;
|
||||
}
|
||||
|
||||
ABSL_BASE_INTERNAL_FORCEINLINE int CountTrailingZerosNonZero32(uint32_t n) {
|
||||
#if defined(_MSC_VER)
|
||||
unsigned long result = 0; // NOLINT(runtime/int)
|
||||
_BitScanForward(&result, n);
|
||||
return result;
|
||||
#elif defined(__GNUC__)
|
||||
static_assert(sizeof(int) == sizeof(n),
|
||||
"__builtin_ctz does not take 32-bit arg");
|
||||
return __builtin_ctz(n);
|
||||
#else
|
||||
return CountTrailingZerosNonZero32Slow(n);
|
||||
#endif
|
||||
}
|
||||
|
||||
#undef ABSL_BASE_INTERNAL_FORCEINLINE
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_BITS_H_
|
||||
+107
@@ -0,0 +1,107 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// The implementation of CycleClock::Frequency.
|
||||
//
|
||||
// NOTE: only i386 and x86_64 have been well tested.
|
||||
// PPC, sparc, alpha, and ia64 are based on
|
||||
// http://peter.kuscsik.com/wordpress/?p=14
|
||||
// with modifications by m3b. See also
|
||||
// https://setisvn.ssl.berkeley.edu/svn/lib/fftw-3.0.1/kernel/cycle.h
|
||||
|
||||
#include "absl/base/internal/cycleclock.h"
|
||||
|
||||
#include <atomic>
|
||||
#include <chrono> // NOLINT(build/c++11)
|
||||
|
||||
#include "absl/base/internal/unscaledcycleclock.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
#if ABSL_USE_UNSCALED_CYCLECLOCK
|
||||
|
||||
namespace {
|
||||
|
||||
#ifdef NDEBUG
|
||||
#ifdef ABSL_INTERNAL_UNSCALED_CYCLECLOCK_FREQUENCY_IS_CPU_FREQUENCY
|
||||
// Not debug mode and the UnscaledCycleClock frequency is the CPU
|
||||
// frequency. Scale the CycleClock to prevent overflow if someone
|
||||
// tries to represent the time as cycles since the Unix epoch.
|
||||
static constexpr int32_t kShift = 1;
|
||||
#else
|
||||
// Not debug mode and the UnscaledCycleClock isn't operating at the
|
||||
// raw CPU frequency. There is no need to do any scaling, so don't
|
||||
// needlessly sacrifice precision.
|
||||
static constexpr int32_t kShift = 0;
|
||||
#endif
|
||||
#else
|
||||
// In debug mode use a different shift to discourage depending on a
|
||||
// particular shift value.
|
||||
static constexpr int32_t kShift = 2;
|
||||
#endif
|
||||
|
||||
static constexpr double kFrequencyScale = 1.0 / (1 << kShift);
|
||||
static std::atomic<CycleClockSourceFunc> cycle_clock_source;
|
||||
|
||||
CycleClockSourceFunc LoadCycleClockSource() {
|
||||
// Optimize for the common case (no callback) by first doing a relaxed load;
|
||||
// this is significantly faster on non-x86 platforms.
|
||||
if (cycle_clock_source.load(std::memory_order_relaxed) == nullptr) {
|
||||
return nullptr;
|
||||
}
|
||||
// This corresponds to the store(std::memory_order_release) in
|
||||
// CycleClockSource::Register, and makes sure that any updates made prior to
|
||||
// registering the callback are visible to this thread before the callback is
|
||||
// invoked.
|
||||
return cycle_clock_source.load(std::memory_order_acquire);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
int64_t CycleClock::Now() {
|
||||
auto fn = LoadCycleClockSource();
|
||||
if (fn == nullptr) {
|
||||
return base_internal::UnscaledCycleClock::Now() >> kShift;
|
||||
}
|
||||
return fn() >> kShift;
|
||||
}
|
||||
|
||||
double CycleClock::Frequency() {
|
||||
return kFrequencyScale * base_internal::UnscaledCycleClock::Frequency();
|
||||
}
|
||||
|
||||
void CycleClockSource::Register(CycleClockSourceFunc source) {
|
||||
// Corresponds to the load(std::memory_order_acquire) in LoadCycleClockSource.
|
||||
cycle_clock_source.store(source, std::memory_order_release);
|
||||
}
|
||||
|
||||
#else
|
||||
|
||||
int64_t CycleClock::Now() {
|
||||
return std::chrono::duration_cast<std::chrono::nanoseconds>(
|
||||
std::chrono::steady_clock::now().time_since_epoch())
|
||||
.count();
|
||||
}
|
||||
|
||||
double CycleClock::Frequency() {
|
||||
return 1e9;
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
+94
@@ -0,0 +1,94 @@
|
||||
//
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: cycleclock.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This header file defines a `CycleClock`, which yields the value and frequency
|
||||
// of a cycle counter that increments at a rate that is approximately constant.
|
||||
//
|
||||
// NOTE:
|
||||
//
|
||||
// The cycle counter frequency is not necessarily related to the core clock
|
||||
// frequency and should not be treated as such. That is, `CycleClock` cycles are
|
||||
// not necessarily "CPU cycles" and code should not rely on that behavior, even
|
||||
// if experimentally observed.
|
||||
//
|
||||
// An arbitrary offset may have been added to the counter at power on.
|
||||
//
|
||||
// On some platforms, the rate and offset of the counter may differ
|
||||
// slightly when read from different CPUs of a multiprocessor. Usually,
|
||||
// we try to ensure that the operating system adjusts values periodically
|
||||
// so that values agree approximately. If you need stronger guarantees,
|
||||
// consider using alternate interfaces.
|
||||
//
|
||||
// The CPU is not required to maintain the ordering of a cycle counter read
|
||||
// with respect to surrounding instructions.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_CYCLECLOCK_H_
|
||||
#define ABSL_BASE_INTERNAL_CYCLECLOCK_H_
|
||||
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// CycleClock
|
||||
// -----------------------------------------------------------------------------
|
||||
class CycleClock {
|
||||
public:
|
||||
// CycleClock::Now()
|
||||
//
|
||||
// Returns the value of a cycle counter that counts at a rate that is
|
||||
// approximately constant.
|
||||
static int64_t Now();
|
||||
|
||||
// CycleClock::Frequency()
|
||||
//
|
||||
// Returns the amount by which `CycleClock::Now()` increases per second. Note
|
||||
// that this value may not necessarily match the core CPU clock frequency.
|
||||
static double Frequency();
|
||||
|
||||
private:
|
||||
CycleClock() = delete; // no instances
|
||||
CycleClock(const CycleClock&) = delete;
|
||||
CycleClock& operator=(const CycleClock&) = delete;
|
||||
};
|
||||
|
||||
using CycleClockSourceFunc = int64_t (*)();
|
||||
|
||||
class CycleClockSource {
|
||||
private:
|
||||
// CycleClockSource::Register()
|
||||
//
|
||||
// Register a function that provides an alternate source for the unscaled CPU
|
||||
// cycle count value. The source function must be async signal safe, must not
|
||||
// call CycleClock::Now(), and must have a frequency that matches that of the
|
||||
// unscaled clock used by CycleClock. A nullptr value resets CycleClock to use
|
||||
// the default source.
|
||||
static void Register(CycleClockSourceFunc source);
|
||||
};
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_CYCLECLOCK_H_
|
||||
+161
@@ -0,0 +1,161 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Functions for directly invoking mmap() via syscall, avoiding the case where
|
||||
// mmap() has been locally overridden.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_DIRECT_MMAP_H_
|
||||
#define ABSL_BASE_INTERNAL_DIRECT_MMAP_H_
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
#if ABSL_HAVE_MMAP
|
||||
|
||||
#include <sys/mman.h>
|
||||
|
||||
#ifdef __linux__
|
||||
|
||||
#include <sys/types.h>
|
||||
#ifdef __BIONIC__
|
||||
#include <sys/syscall.h>
|
||||
#else
|
||||
#include <syscall.h>
|
||||
#endif
|
||||
|
||||
#include <linux/unistd.h>
|
||||
#include <unistd.h>
|
||||
#include <cerrno>
|
||||
#include <cstdarg>
|
||||
#include <cstdint>
|
||||
|
||||
#ifdef __mips__
|
||||
// Include definitions of the ABI currently in use.
|
||||
#ifdef __BIONIC__
|
||||
// Android doesn't have sgidefs.h, but does have asm/sgidefs.h, which has the
|
||||
// definitions we need.
|
||||
#include <asm/sgidefs.h>
|
||||
#else
|
||||
#include <sgidefs.h>
|
||||
#endif // __BIONIC__
|
||||
#endif // __mips__
|
||||
|
||||
// SYS_mmap and SYS_munmap are not defined in Android.
|
||||
#ifdef __BIONIC__
|
||||
extern "C" void* __mmap2(void*, size_t, int, int, int, size_t);
|
||||
#if defined(__NR_mmap) && !defined(SYS_mmap)
|
||||
#define SYS_mmap __NR_mmap
|
||||
#endif
|
||||
#ifndef SYS_munmap
|
||||
#define SYS_munmap __NR_munmap
|
||||
#endif
|
||||
#endif // __BIONIC__
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// Platform specific logic extracted from
|
||||
// https://chromium.googlesource.com/linux-syscall-support/+/master/linux_syscall_support.h
|
||||
inline void* DirectMmap(void* start, size_t length, int prot, int flags, int fd,
|
||||
off64_t offset) noexcept {
|
||||
#if defined(__i386__) || defined(__ARM_ARCH_3__) || defined(__ARM_EABI__) || \
|
||||
(defined(__mips__) && _MIPS_SIM == _MIPS_SIM_ABI32) || \
|
||||
(defined(__PPC__) && !defined(__PPC64__)) || \
|
||||
(defined(__s390__) && !defined(__s390x__))
|
||||
// On these architectures, implement mmap with mmap2.
|
||||
static int pagesize = 0;
|
||||
if (pagesize == 0) {
|
||||
#if defined(__wasm__) || defined(__asmjs__)
|
||||
pagesize = getpagesize();
|
||||
#else
|
||||
pagesize = sysconf(_SC_PAGESIZE);
|
||||
#endif
|
||||
}
|
||||
if (offset < 0 || offset % pagesize != 0) {
|
||||
errno = EINVAL;
|
||||
return MAP_FAILED;
|
||||
}
|
||||
#ifdef __BIONIC__
|
||||
// SYS_mmap2 has problems on Android API level <= 16.
|
||||
// Workaround by invoking __mmap2() instead.
|
||||
return __mmap2(start, length, prot, flags, fd, offset / pagesize);
|
||||
#else
|
||||
return reinterpret_cast<void*>(
|
||||
syscall(SYS_mmap2, start, length, prot, flags, fd,
|
||||
static_cast<off_t>(offset / pagesize)));
|
||||
#endif
|
||||
#elif defined(__s390x__)
|
||||
// On s390x, mmap() arguments are passed in memory.
|
||||
unsigned long buf[6] = {reinterpret_cast<unsigned long>(start), // NOLINT
|
||||
static_cast<unsigned long>(length), // NOLINT
|
||||
static_cast<unsigned long>(prot), // NOLINT
|
||||
static_cast<unsigned long>(flags), // NOLINT
|
||||
static_cast<unsigned long>(fd), // NOLINT
|
||||
static_cast<unsigned long>(offset)}; // NOLINT
|
||||
return reinterpret_cast<void*>(syscall(SYS_mmap, buf));
|
||||
#elif defined(__x86_64__)
|
||||
// The x32 ABI has 32 bit longs, but the syscall interface is 64 bit.
|
||||
// We need to explicitly cast to an unsigned 64 bit type to avoid implicit
|
||||
// sign extension. We can't cast pointers directly because those are
|
||||
// 32 bits, and gcc will dump ugly warnings about casting from a pointer
|
||||
// to an integer of a different size. We also need to make sure __off64_t
|
||||
// isn't truncated to 32-bits under x32.
|
||||
#define MMAP_SYSCALL_ARG(x) ((uint64_t)(uintptr_t)(x))
|
||||
return reinterpret_cast<void*>(
|
||||
syscall(SYS_mmap, MMAP_SYSCALL_ARG(start), MMAP_SYSCALL_ARG(length),
|
||||
MMAP_SYSCALL_ARG(prot), MMAP_SYSCALL_ARG(flags),
|
||||
MMAP_SYSCALL_ARG(fd), static_cast<uint64_t>(offset)));
|
||||
#undef MMAP_SYSCALL_ARG
|
||||
#else // Remaining 64-bit aritectures.
|
||||
static_assert(sizeof(unsigned long) == 8, "Platform is not 64-bit");
|
||||
return reinterpret_cast<void*>(
|
||||
syscall(SYS_mmap, start, length, prot, flags, fd, offset));
|
||||
#endif
|
||||
}
|
||||
|
||||
inline int DirectMunmap(void* start, size_t length) {
|
||||
return static_cast<int>(syscall(SYS_munmap, start, length));
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#else // !__linux__
|
||||
|
||||
// For non-linux platforms where we have mmap, just dispatch directly to the
|
||||
// actual mmap()/munmap() methods.
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
inline void* DirectMmap(void* start, size_t length, int prot, int flags, int fd,
|
||||
off_t offset) {
|
||||
return mmap(start, length, prot, flags, fd, offset);
|
||||
}
|
||||
|
||||
inline int DirectMunmap(void* start, size_t length) {
|
||||
return munmap(start, length);
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // __linux__
|
||||
|
||||
#endif // ABSL_HAVE_MMAP
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_DIRECT_MMAP_H_
|
||||
Generated
+266
@@ -0,0 +1,266 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_ENDIAN_H_
|
||||
#define ABSL_BASE_INTERNAL_ENDIAN_H_
|
||||
|
||||
// The following guarantees declaration of the byte swap functions
|
||||
#ifdef _MSC_VER
|
||||
#include <stdlib.h> // NOLINT(build/include)
|
||||
#elif defined(__FreeBSD__)
|
||||
#include <sys/endian.h>
|
||||
#elif defined(__GLIBC__)
|
||||
#include <byteswap.h> // IWYU pragma: export
|
||||
#endif
|
||||
|
||||
#include <cstdint>
|
||||
#include "absl/base/config.h"
|
||||
#include "absl/base/internal/unaligned_access.h"
|
||||
#include "absl/base/port.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
|
||||
// Use compiler byte-swapping intrinsics if they are available. 32-bit
|
||||
// and 64-bit versions are available in Clang and GCC as of GCC 4.3.0.
|
||||
// The 16-bit version is available in Clang and GCC only as of GCC 4.8.0.
|
||||
// For simplicity, we enable them all only for GCC 4.8.0 or later.
|
||||
#if defined(__clang__) || \
|
||||
(defined(__GNUC__) && \
|
||||
((__GNUC__ == 4 && __GNUC_MINOR__ >= 8) || __GNUC__ >= 5))
|
||||
inline uint64_t gbswap_64(uint64_t host_int) {
|
||||
return __builtin_bswap64(host_int);
|
||||
}
|
||||
inline uint32_t gbswap_32(uint32_t host_int) {
|
||||
return __builtin_bswap32(host_int);
|
||||
}
|
||||
inline uint16_t gbswap_16(uint16_t host_int) {
|
||||
return __builtin_bswap16(host_int);
|
||||
}
|
||||
|
||||
#elif defined(_MSC_VER)
|
||||
inline uint64_t gbswap_64(uint64_t host_int) {
|
||||
return _byteswap_uint64(host_int);
|
||||
}
|
||||
inline uint32_t gbswap_32(uint32_t host_int) {
|
||||
return _byteswap_ulong(host_int);
|
||||
}
|
||||
inline uint16_t gbswap_16(uint16_t host_int) {
|
||||
return _byteswap_ushort(host_int);
|
||||
}
|
||||
|
||||
#else
|
||||
inline uint64_t gbswap_64(uint64_t host_int) {
|
||||
#if defined(__GNUC__) && defined(__x86_64__) && !defined(__APPLE__)
|
||||
// Adapted from /usr/include/byteswap.h. Not available on Mac.
|
||||
if (__builtin_constant_p(host_int)) {
|
||||
return __bswap_constant_64(host_int);
|
||||
} else {
|
||||
uint64_t result;
|
||||
__asm__("bswap %0" : "=r"(result) : "0"(host_int));
|
||||
return result;
|
||||
}
|
||||
#elif defined(__GLIBC__)
|
||||
return bswap_64(host_int);
|
||||
#else
|
||||
return (((host_int & uint64_t{0xFF}) << 56) |
|
||||
((host_int & uint64_t{0xFF00}) << 40) |
|
||||
((host_int & uint64_t{0xFF0000}) << 24) |
|
||||
((host_int & uint64_t{0xFF000000}) << 8) |
|
||||
((host_int & uint64_t{0xFF00000000}) >> 8) |
|
||||
((host_int & uint64_t{0xFF0000000000}) >> 24) |
|
||||
((host_int & uint64_t{0xFF000000000000}) >> 40) |
|
||||
((host_int & uint64_t{0xFF00000000000000}) >> 56));
|
||||
#endif // bswap_64
|
||||
}
|
||||
|
||||
inline uint32_t gbswap_32(uint32_t host_int) {
|
||||
#if defined(__GLIBC__)
|
||||
return bswap_32(host_int);
|
||||
#else
|
||||
return (((host_int & uint32_t{0xFF}) << 24) |
|
||||
((host_int & uint32_t{0xFF00}) << 8) |
|
||||
((host_int & uint32_t{0xFF0000}) >> 8) |
|
||||
((host_int & uint32_t{0xFF000000}) >> 24));
|
||||
#endif
|
||||
}
|
||||
|
||||
inline uint16_t gbswap_16(uint16_t host_int) {
|
||||
#if defined(__GLIBC__)
|
||||
return bswap_16(host_int);
|
||||
#else
|
||||
return (((host_int & uint16_t{0xFF}) << 8) |
|
||||
((host_int & uint16_t{0xFF00}) >> 8));
|
||||
#endif
|
||||
}
|
||||
|
||||
#endif // intrinsics available
|
||||
|
||||
#ifdef ABSL_IS_LITTLE_ENDIAN
|
||||
|
||||
// Definitions for ntohl etc. that don't require us to include
|
||||
// netinet/in.h. We wrap gbswap_32 and gbswap_16 in functions rather
|
||||
// than just #defining them because in debug mode, gcc doesn't
|
||||
// correctly handle the (rather involved) definitions of bswap_32.
|
||||
// gcc guarantees that inline functions are as fast as macros, so
|
||||
// this isn't a performance hit.
|
||||
inline uint16_t ghtons(uint16_t x) { return gbswap_16(x); }
|
||||
inline uint32_t ghtonl(uint32_t x) { return gbswap_32(x); }
|
||||
inline uint64_t ghtonll(uint64_t x) { return gbswap_64(x); }
|
||||
|
||||
#elif defined ABSL_IS_BIG_ENDIAN
|
||||
|
||||
// These definitions are simpler on big-endian machines
|
||||
// These are functions instead of macros to avoid self-assignment warnings
|
||||
// on calls such as "i = ghtnol(i);". This also provides type checking.
|
||||
inline uint16_t ghtons(uint16_t x) { return x; }
|
||||
inline uint32_t ghtonl(uint32_t x) { return x; }
|
||||
inline uint64_t ghtonll(uint64_t x) { return x; }
|
||||
|
||||
#else
|
||||
#error \
|
||||
"Unsupported byte order: Either ABSL_IS_BIG_ENDIAN or " \
|
||||
"ABSL_IS_LITTLE_ENDIAN must be defined"
|
||||
#endif // byte order
|
||||
|
||||
inline uint16_t gntohs(uint16_t x) { return ghtons(x); }
|
||||
inline uint32_t gntohl(uint32_t x) { return ghtonl(x); }
|
||||
inline uint64_t gntohll(uint64_t x) { return ghtonll(x); }
|
||||
|
||||
// Utilities to convert numbers between the current hosts's native byte
|
||||
// order and little-endian byte order
|
||||
//
|
||||
// Load/Store methods are alignment safe
|
||||
namespace little_endian {
|
||||
// Conversion functions.
|
||||
#ifdef ABSL_IS_LITTLE_ENDIAN
|
||||
|
||||
inline uint16_t FromHost16(uint16_t x) { return x; }
|
||||
inline uint16_t ToHost16(uint16_t x) { return x; }
|
||||
|
||||
inline uint32_t FromHost32(uint32_t x) { return x; }
|
||||
inline uint32_t ToHost32(uint32_t x) { return x; }
|
||||
|
||||
inline uint64_t FromHost64(uint64_t x) { return x; }
|
||||
inline uint64_t ToHost64(uint64_t x) { return x; }
|
||||
|
||||
inline constexpr bool IsLittleEndian() { return true; }
|
||||
|
||||
#elif defined ABSL_IS_BIG_ENDIAN
|
||||
|
||||
inline uint16_t FromHost16(uint16_t x) { return gbswap_16(x); }
|
||||
inline uint16_t ToHost16(uint16_t x) { return gbswap_16(x); }
|
||||
|
||||
inline uint32_t FromHost32(uint32_t x) { return gbswap_32(x); }
|
||||
inline uint32_t ToHost32(uint32_t x) { return gbswap_32(x); }
|
||||
|
||||
inline uint64_t FromHost64(uint64_t x) { return gbswap_64(x); }
|
||||
inline uint64_t ToHost64(uint64_t x) { return gbswap_64(x); }
|
||||
|
||||
inline constexpr bool IsLittleEndian() { return false; }
|
||||
|
||||
#endif /* ENDIAN */
|
||||
|
||||
// Functions to do unaligned loads and stores in little-endian order.
|
||||
inline uint16_t Load16(const void *p) {
|
||||
return ToHost16(ABSL_INTERNAL_UNALIGNED_LOAD16(p));
|
||||
}
|
||||
|
||||
inline void Store16(void *p, uint16_t v) {
|
||||
ABSL_INTERNAL_UNALIGNED_STORE16(p, FromHost16(v));
|
||||
}
|
||||
|
||||
inline uint32_t Load32(const void *p) {
|
||||
return ToHost32(ABSL_INTERNAL_UNALIGNED_LOAD32(p));
|
||||
}
|
||||
|
||||
inline void Store32(void *p, uint32_t v) {
|
||||
ABSL_INTERNAL_UNALIGNED_STORE32(p, FromHost32(v));
|
||||
}
|
||||
|
||||
inline uint64_t Load64(const void *p) {
|
||||
return ToHost64(ABSL_INTERNAL_UNALIGNED_LOAD64(p));
|
||||
}
|
||||
|
||||
inline void Store64(void *p, uint64_t v) {
|
||||
ABSL_INTERNAL_UNALIGNED_STORE64(p, FromHost64(v));
|
||||
}
|
||||
|
||||
} // namespace little_endian
|
||||
|
||||
// Utilities to convert numbers between the current hosts's native byte
|
||||
// order and big-endian byte order (same as network byte order)
|
||||
//
|
||||
// Load/Store methods are alignment safe
|
||||
namespace big_endian {
|
||||
#ifdef ABSL_IS_LITTLE_ENDIAN
|
||||
|
||||
inline uint16_t FromHost16(uint16_t x) { return gbswap_16(x); }
|
||||
inline uint16_t ToHost16(uint16_t x) { return gbswap_16(x); }
|
||||
|
||||
inline uint32_t FromHost32(uint32_t x) { return gbswap_32(x); }
|
||||
inline uint32_t ToHost32(uint32_t x) { return gbswap_32(x); }
|
||||
|
||||
inline uint64_t FromHost64(uint64_t x) { return gbswap_64(x); }
|
||||
inline uint64_t ToHost64(uint64_t x) { return gbswap_64(x); }
|
||||
|
||||
inline constexpr bool IsLittleEndian() { return true; }
|
||||
|
||||
#elif defined ABSL_IS_BIG_ENDIAN
|
||||
|
||||
inline uint16_t FromHost16(uint16_t x) { return x; }
|
||||
inline uint16_t ToHost16(uint16_t x) { return x; }
|
||||
|
||||
inline uint32_t FromHost32(uint32_t x) { return x; }
|
||||
inline uint32_t ToHost32(uint32_t x) { return x; }
|
||||
|
||||
inline uint64_t FromHost64(uint64_t x) { return x; }
|
||||
inline uint64_t ToHost64(uint64_t x) { return x; }
|
||||
|
||||
inline constexpr bool IsLittleEndian() { return false; }
|
||||
|
||||
#endif /* ENDIAN */
|
||||
|
||||
// Functions to do unaligned loads and stores in big-endian order.
|
||||
inline uint16_t Load16(const void *p) {
|
||||
return ToHost16(ABSL_INTERNAL_UNALIGNED_LOAD16(p));
|
||||
}
|
||||
|
||||
inline void Store16(void *p, uint16_t v) {
|
||||
ABSL_INTERNAL_UNALIGNED_STORE16(p, FromHost16(v));
|
||||
}
|
||||
|
||||
inline uint32_t Load32(const void *p) {
|
||||
return ToHost32(ABSL_INTERNAL_UNALIGNED_LOAD32(p));
|
||||
}
|
||||
|
||||
inline void Store32(void *p, uint32_t v) {
|
||||
ABSL_INTERNAL_UNALIGNED_STORE32(p, FromHost32(v));
|
||||
}
|
||||
|
||||
inline uint64_t Load64(const void *p) {
|
||||
return ToHost64(ABSL_INTERNAL_UNALIGNED_LOAD64(p));
|
||||
}
|
||||
|
||||
inline void Store64(void *p, uint64_t v) {
|
||||
ABSL_INTERNAL_UNALIGNED_STORE64(p, FromHost64(v));
|
||||
}
|
||||
|
||||
} // namespace big_endian
|
||||
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_ENDIAN_H_
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_ERRNO_SAVER_H_
|
||||
#define ABSL_BASE_INTERNAL_ERRNO_SAVER_H_
|
||||
|
||||
#include <cerrno>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// `ErrnoSaver` captures the value of `errno` upon construction and restores it
|
||||
// upon deletion. It is used in low-level code and must be super fast. Do not
|
||||
// add instrumentation, even in debug modes.
|
||||
class ErrnoSaver {
|
||||
public:
|
||||
ErrnoSaver() : saved_errno_(errno) {}
|
||||
~ErrnoSaver() { errno = saved_errno_; }
|
||||
int operator()() const { return saved_errno_; }
|
||||
|
||||
private:
|
||||
const int saved_errno_;
|
||||
};
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_ERRNO_SAVER_H_
|
||||
@@ -0,0 +1,93 @@
|
||||
// Copyright 2019 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/base/internal/exponential_biased.h"
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <atomic>
|
||||
#include <cmath>
|
||||
#include <limits>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/optimization.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// The algorithm generates a random number between 0 and 1 and applies the
|
||||
// inverse cumulative distribution function for an exponential. Specifically:
|
||||
// Let m be the inverse of the sample period, then the probability
|
||||
// distribution function is m*exp(-mx) so the CDF is
|
||||
// p = 1 - exp(-mx), so
|
||||
// q = 1 - p = exp(-mx)
|
||||
// log_e(q) = -mx
|
||||
// -log_e(q)/m = x
|
||||
// log_2(q) * (-log_e(2) * 1/m) = x
|
||||
// In the code, q is actually in the range 1 to 2**26, hence the -26 below
|
||||
int64_t ExponentialBiased::GetSkipCount(int64_t mean) {
|
||||
if (ABSL_PREDICT_FALSE(!initialized_)) {
|
||||
Initialize();
|
||||
}
|
||||
|
||||
uint64_t rng = NextRandom(rng_);
|
||||
rng_ = rng;
|
||||
|
||||
// Take the top 26 bits as the random number
|
||||
// (This plus the 1<<58 sampling bound give a max possible step of
|
||||
// 5194297183973780480 bytes.)
|
||||
// The uint32_t cast is to prevent a (hard-to-reproduce) NAN
|
||||
// under piii debug for some binaries.
|
||||
double q = static_cast<uint32_t>(rng >> (kPrngNumBits - 26)) + 1.0;
|
||||
// Put the computed p-value through the CDF of a geometric.
|
||||
double interval = bias_ + (std::log2(q) - 26) * (-std::log(2.0) * mean);
|
||||
// Very large values of interval overflow int64_t. To avoid that, we will
|
||||
// cheat and clamp any huge values to (int64_t max)/2. This is a potential
|
||||
// source of bias, but the mean would need to be such a large value that it's
|
||||
// not likely to come up. For example, with a mean of 1e18, the probability of
|
||||
// hitting this condition is about 1/1000. For a mean of 1e17, standard
|
||||
// calculators claim that this event won't happen.
|
||||
if (interval > static_cast<double>(std::numeric_limits<int64_t>::max() / 2)) {
|
||||
// Assume huge values are bias neutral, retain bias for next call.
|
||||
return std::numeric_limits<int64_t>::max() / 2;
|
||||
}
|
||||
double value = std::round(interval);
|
||||
bias_ = interval - value;
|
||||
return value;
|
||||
}
|
||||
|
||||
int64_t ExponentialBiased::GetStride(int64_t mean) {
|
||||
return GetSkipCount(mean - 1) + 1;
|
||||
}
|
||||
|
||||
void ExponentialBiased::Initialize() {
|
||||
// We don't get well distributed numbers from `this` so we call NextRandom() a
|
||||
// bunch to mush the bits around. We use a global_rand to handle the case
|
||||
// where the same thread (by memory address) gets created and destroyed
|
||||
// repeatedly.
|
||||
ABSL_CONST_INIT static std::atomic<uint32_t> global_rand(0);
|
||||
uint64_t r = reinterpret_cast<uint64_t>(this) +
|
||||
global_rand.fetch_add(1, std::memory_order_relaxed);
|
||||
for (int i = 0; i < 20; ++i) {
|
||||
r = NextRandom(r);
|
||||
}
|
||||
rng_ = r;
|
||||
initialized_ = true;
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
+130
@@ -0,0 +1,130 @@
|
||||
// Copyright 2019 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_EXPONENTIAL_BIASED_H_
|
||||
#define ABSL_BASE_INTERNAL_EXPONENTIAL_BIASED_H_
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
#include "absl/base/macros.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// ExponentialBiased provides a small and fast random number generator for a
|
||||
// rounded exponential distribution. This generator manages very little state,
|
||||
// and imposes no synchronization overhead. This makes it useful in specialized
|
||||
// scenarios requiring minimum overhead, such as stride based periodic sampling.
|
||||
//
|
||||
// ExponentialBiased provides two closely related functions, GetSkipCount() and
|
||||
// GetStride(), both returning a rounded integer defining a number of events
|
||||
// required before some event with a given mean probability occurs.
|
||||
//
|
||||
// The distribution is useful to generate a random wait time or some periodic
|
||||
// event with a given mean probability. For example, if an action is supposed to
|
||||
// happen on average once every 'N' events, then we can get a random 'stride'
|
||||
// counting down how long before the event to happen. For example, if we'd want
|
||||
// to sample one in every 1000 'Frobber' calls, our code could look like this:
|
||||
//
|
||||
// Frobber::Frobber() {
|
||||
// stride_ = exponential_biased_.GetStride(1000);
|
||||
// }
|
||||
//
|
||||
// void Frobber::Frob(int arg) {
|
||||
// if (--stride == 0) {
|
||||
// SampleFrob(arg);
|
||||
// stride_ = exponential_biased_.GetStride(1000);
|
||||
// }
|
||||
// ...
|
||||
// }
|
||||
//
|
||||
// The rounding of the return value creates a bias, especially for smaller means
|
||||
// where the distribution of the fraction is not evenly distributed. We correct
|
||||
// this bias by tracking the fraction we rounded up or down on each iteration,
|
||||
// effectively tracking the distance between the cumulative value, and the
|
||||
// rounded cumulative value. For example, given a mean of 2:
|
||||
//
|
||||
// raw = 1.63076, cumulative = 1.63076, rounded = 2, bias = -0.36923
|
||||
// raw = 0.14624, cumulative = 1.77701, rounded = 2, bias = 0.14624
|
||||
// raw = 4.93194, cumulative = 6.70895, rounded = 7, bias = -0.06805
|
||||
// raw = 0.24206, cumulative = 6.95101, rounded = 7, bias = 0.24206
|
||||
// etc...
|
||||
//
|
||||
// Adjusting with rounding bias is relatively trivial:
|
||||
//
|
||||
// double value = bias_ + exponential_distribution(mean)();
|
||||
// double rounded_value = std::round(value);
|
||||
// bias_ = value - rounded_value;
|
||||
// return rounded_value;
|
||||
//
|
||||
// This class is thread-compatible.
|
||||
class ExponentialBiased {
|
||||
public:
|
||||
// The number of bits set by NextRandom.
|
||||
static constexpr int kPrngNumBits = 48;
|
||||
|
||||
// `GetSkipCount()` returns the number of events to skip before some chosen
|
||||
// event happens. For example, randomly tossing a coin, we will on average
|
||||
// throw heads once before we get tails. We can simulate random coin tosses
|
||||
// using GetSkipCount() as:
|
||||
//
|
||||
// ExponentialBiased eb;
|
||||
// for (...) {
|
||||
// int number_of_heads_before_tail = eb.GetSkipCount(1);
|
||||
// for (int flips = 0; flips < number_of_heads_before_tail; ++flips) {
|
||||
// printf("head...");
|
||||
// }
|
||||
// printf("tail\n");
|
||||
// }
|
||||
//
|
||||
int64_t GetSkipCount(int64_t mean);
|
||||
|
||||
// GetStride() returns the number of events required for a specific event to
|
||||
// happen. See the class comments for a usage example. `GetStride()` is
|
||||
// equivalent to `GetSkipCount(mean - 1) + 1`. When to use `GetStride()` or
|
||||
// `GetSkipCount()` depends mostly on what best fits the use case.
|
||||
int64_t GetStride(int64_t mean);
|
||||
|
||||
// Computes a random number in the range [0, 1<<(kPrngNumBits+1) - 1]
|
||||
//
|
||||
// This is public to enable testing.
|
||||
static uint64_t NextRandom(uint64_t rnd);
|
||||
|
||||
private:
|
||||
void Initialize();
|
||||
|
||||
uint64_t rng_{0};
|
||||
double bias_{0};
|
||||
bool initialized_{false};
|
||||
};
|
||||
|
||||
// Returns the next prng value.
|
||||
// pRNG is: aX+b mod c with a = 0x5DEECE66D, b = 0xB, c = 1<<48
|
||||
// This is the lrand64 generator.
|
||||
inline uint64_t ExponentialBiased::NextRandom(uint64_t rnd) {
|
||||
const uint64_t prng_mult = uint64_t{0x5DEECE66D};
|
||||
const uint64_t prng_add = 0xB;
|
||||
const uint64_t prng_mod_power = 48;
|
||||
const uint64_t prng_mod_mask =
|
||||
~((~static_cast<uint64_t>(0)) << prng_mod_power);
|
||||
return (prng_mult * rnd + prng_add) & prng_mod_mask;
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_EXPONENTIAL_BIASED_H_
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_HIDE_PTR_H_
|
||||
#define ABSL_BASE_INTERNAL_HIDE_PTR_H_
|
||||
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// Arbitrary value with high bits set. Xor'ing with it is unlikely
|
||||
// to map one valid pointer to another valid pointer.
|
||||
constexpr uintptr_t HideMask() {
|
||||
return (uintptr_t{0xF03A5F7BU} << (sizeof(uintptr_t) - 4) * 8) | 0xF03A5F7BU;
|
||||
}
|
||||
|
||||
// Hide a pointer from the leak checker. For internal use only.
|
||||
// Differs from absl::IgnoreLeak(ptr) in that absl::IgnoreLeak(ptr) causes ptr
|
||||
// and all objects reachable from ptr to be ignored by the leak checker.
|
||||
template <class T>
|
||||
inline uintptr_t HidePtr(T* ptr) {
|
||||
return reinterpret_cast<uintptr_t>(ptr) ^ HideMask();
|
||||
}
|
||||
|
||||
// Return a pointer that has been hidden from the leak checker.
|
||||
// For internal use only.
|
||||
template <class T>
|
||||
inline T* UnhidePtr(uintptr_t hidden) {
|
||||
return reinterpret_cast<T*>(hidden ^ HideMask());
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_HIDE_PTR_H_
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_IDENTITY_H_
|
||||
#define ABSL_BASE_INTERNAL_IDENTITY_H_
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace internal {
|
||||
|
||||
template <typename T>
|
||||
struct identity {
|
||||
typedef T type;
|
||||
};
|
||||
|
||||
template <typename T>
|
||||
using identity_t = typename identity<T>::type;
|
||||
|
||||
} // namespace internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_IDENTITY_H_
|
||||
+107
@@ -0,0 +1,107 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_INLINE_VARIABLE_EMULATION_H_
|
||||
#define ABSL_BASE_INTERNAL_INLINE_VARIABLE_EMULATION_H_
|
||||
|
||||
#include <type_traits>
|
||||
|
||||
#include "absl/base/internal/identity.h"
|
||||
|
||||
// File:
|
||||
// This file define a macro that allows the creation of or emulation of C++17
|
||||
// inline variables based on whether or not the feature is supported.
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
// Macro: ABSL_INTERNAL_INLINE_CONSTEXPR(type, name, init)
|
||||
//
|
||||
// Description:
|
||||
// Expands to the equivalent of an inline constexpr instance of the specified
|
||||
// `type` and `name`, initialized to the value `init`. If the compiler being
|
||||
// used is detected as supporting actual inline variables as a language
|
||||
// feature, then the macro expands to an actual inline variable definition.
|
||||
//
|
||||
// Requires:
|
||||
// `type` is a type that is usable in an extern variable declaration.
|
||||
//
|
||||
// Requires: `name` is a valid identifier
|
||||
//
|
||||
// Requires:
|
||||
// `init` is an expression that can be used in the following definition:
|
||||
// constexpr type name = init;
|
||||
//
|
||||
// Usage:
|
||||
//
|
||||
// // Equivalent to: `inline constexpr size_t variant_npos = -1;`
|
||||
// ABSL_INTERNAL_INLINE_CONSTEXPR(size_t, variant_npos, -1);
|
||||
//
|
||||
// Differences in implementation:
|
||||
// For a direct, language-level inline variable, decltype(name) will be the
|
||||
// type that was specified along with const qualification, whereas for
|
||||
// emulated inline variables, decltype(name) may be different (in practice
|
||||
// it will likely be a reference type).
|
||||
////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
#ifdef __cpp_inline_variables
|
||||
|
||||
// Clang's -Wmissing-variable-declarations option erroneously warned that
|
||||
// inline constexpr objects need to be pre-declared. This has now been fixed,
|
||||
// but we will need to support this workaround for people building with older
|
||||
// versions of clang.
|
||||
//
|
||||
// Bug: https://bugs.llvm.org/show_bug.cgi?id=35862
|
||||
//
|
||||
// Note:
|
||||
// identity_t is used here so that the const and name are in the
|
||||
// appropriate place for pointer types, reference types, function pointer
|
||||
// types, etc..
|
||||
#if defined(__clang__)
|
||||
#define ABSL_INTERNAL_EXTERN_DECL(type, name) \
|
||||
extern const ::absl::internal::identity_t<type> name;
|
||||
#else // Otherwise, just define the macro to do nothing.
|
||||
#define ABSL_INTERNAL_EXTERN_DECL(type, name)
|
||||
#endif // defined(__clang__)
|
||||
|
||||
// See above comment at top of file for details.
|
||||
#define ABSL_INTERNAL_INLINE_CONSTEXPR(type, name, init) \
|
||||
ABSL_INTERNAL_EXTERN_DECL(type, name) \
|
||||
inline constexpr ::absl::internal::identity_t<type> name = init
|
||||
|
||||
#else
|
||||
|
||||
// See above comment at top of file for details.
|
||||
//
|
||||
// Note:
|
||||
// identity_t is used here so that the const and name are in the
|
||||
// appropriate place for pointer types, reference types, function pointer
|
||||
// types, etc..
|
||||
#define ABSL_INTERNAL_INLINE_CONSTEXPR(var_type, name, init) \
|
||||
template <class /*AbslInternalDummy*/ = void> \
|
||||
struct AbslInternalInlineVariableHolder##name { \
|
||||
static constexpr ::absl::internal::identity_t<var_type> kInstance = init; \
|
||||
}; \
|
||||
\
|
||||
template <class AbslInternalDummy> \
|
||||
constexpr ::absl::internal::identity_t<var_type> \
|
||||
AbslInternalInlineVariableHolder##name<AbslInternalDummy>::kInstance; \
|
||||
\
|
||||
static constexpr const ::absl::internal::identity_t<var_type>& \
|
||||
name = /* NOLINT */ \
|
||||
AbslInternalInlineVariableHolder##name<>::kInstance; \
|
||||
static_assert(sizeof(void (*)(decltype(name))) != 0, \
|
||||
"Silence unused variable warnings.")
|
||||
|
||||
#endif // __cpp_inline_variables
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_INLINE_VARIABLE_EMULATION_H_
|
||||
Generated
+187
@@ -0,0 +1,187 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// absl::base_internal::Invoke(f, args...) is an implementation of
|
||||
// INVOKE(f, args...) from section [func.require] of the C++ standard.
|
||||
//
|
||||
// [func.require]
|
||||
// Define INVOKE (f, t1, t2, ..., tN) as follows:
|
||||
// 1. (t1.*f)(t2, ..., tN) when f is a pointer to a member function of a class T
|
||||
// and t1 is an object of type T or a reference to an object of type T or a
|
||||
// reference to an object of a type derived from T;
|
||||
// 2. ((*t1).*f)(t2, ..., tN) when f is a pointer to a member function of a
|
||||
// class T and t1 is not one of the types described in the previous item;
|
||||
// 3. t1.*f when N == 1 and f is a pointer to member data of a class T and t1 is
|
||||
// an object of type T or a reference to an object of type T or a reference
|
||||
// to an object of a type derived from T;
|
||||
// 4. (*t1).*f when N == 1 and f is a pointer to member data of a class T and t1
|
||||
// is not one of the types described in the previous item;
|
||||
// 5. f(t1, t2, ..., tN) in all other cases.
|
||||
//
|
||||
// The implementation is SFINAE-friendly: substitution failure within Invoke()
|
||||
// isn't an error.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_INVOKE_H_
|
||||
#define ABSL_BASE_INTERNAL_INVOKE_H_
|
||||
|
||||
#include <algorithm>
|
||||
#include <type_traits>
|
||||
#include <utility>
|
||||
|
||||
#include "absl/meta/type_traits.h"
|
||||
|
||||
// The following code is internal implementation detail. See the comment at the
|
||||
// top of this file for the API documentation.
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// The five classes below each implement one of the clauses from the definition
|
||||
// of INVOKE. The inner class template Accept<F, Args...> checks whether the
|
||||
// clause is applicable; static function template Invoke(f, args...) does the
|
||||
// invocation.
|
||||
//
|
||||
// By separating the clause selection logic from invocation we make sure that
|
||||
// Invoke() does exactly what the standard says.
|
||||
|
||||
template <typename Derived>
|
||||
struct StrippedAccept {
|
||||
template <typename... Args>
|
||||
struct Accept : Derived::template AcceptImpl<typename std::remove_cv<
|
||||
typename std::remove_reference<Args>::type>::type...> {};
|
||||
};
|
||||
|
||||
// (t1.*f)(t2, ..., tN) when f is a pointer to a member function of a class T
|
||||
// and t1 is an object of type T or a reference to an object of type T or a
|
||||
// reference to an object of a type derived from T.
|
||||
struct MemFunAndRef : StrippedAccept<MemFunAndRef> {
|
||||
template <typename... Args>
|
||||
struct AcceptImpl : std::false_type {};
|
||||
|
||||
template <typename MemFunType, typename C, typename Obj, typename... Args>
|
||||
struct AcceptImpl<MemFunType C::*, Obj, Args...>
|
||||
: std::integral_constant<bool, std::is_base_of<C, Obj>::value &&
|
||||
absl::is_function<MemFunType>::value> {
|
||||
};
|
||||
|
||||
template <typename MemFun, typename Obj, typename... Args>
|
||||
static decltype((std::declval<Obj>().*
|
||||
std::declval<MemFun>())(std::declval<Args>()...))
|
||||
Invoke(MemFun&& mem_fun, Obj&& obj, Args&&... args) {
|
||||
return (std::forward<Obj>(obj).*
|
||||
std::forward<MemFun>(mem_fun))(std::forward<Args>(args)...);
|
||||
}
|
||||
};
|
||||
|
||||
// ((*t1).*f)(t2, ..., tN) when f is a pointer to a member function of a
|
||||
// class T and t1 is not one of the types described in the previous item.
|
||||
struct MemFunAndPtr : StrippedAccept<MemFunAndPtr> {
|
||||
template <typename... Args>
|
||||
struct AcceptImpl : std::false_type {};
|
||||
|
||||
template <typename MemFunType, typename C, typename Ptr, typename... Args>
|
||||
struct AcceptImpl<MemFunType C::*, Ptr, Args...>
|
||||
: std::integral_constant<bool, !std::is_base_of<C, Ptr>::value &&
|
||||
absl::is_function<MemFunType>::value> {
|
||||
};
|
||||
|
||||
template <typename MemFun, typename Ptr, typename... Args>
|
||||
static decltype(((*std::declval<Ptr>()).*
|
||||
std::declval<MemFun>())(std::declval<Args>()...))
|
||||
Invoke(MemFun&& mem_fun, Ptr&& ptr, Args&&... args) {
|
||||
return ((*std::forward<Ptr>(ptr)).*
|
||||
std::forward<MemFun>(mem_fun))(std::forward<Args>(args)...);
|
||||
}
|
||||
};
|
||||
|
||||
// t1.*f when N == 1 and f is a pointer to member data of a class T and t1 is
|
||||
// an object of type T or a reference to an object of type T or a reference
|
||||
// to an object of a type derived from T.
|
||||
struct DataMemAndRef : StrippedAccept<DataMemAndRef> {
|
||||
template <typename... Args>
|
||||
struct AcceptImpl : std::false_type {};
|
||||
|
||||
template <typename R, typename C, typename Obj>
|
||||
struct AcceptImpl<R C::*, Obj>
|
||||
: std::integral_constant<bool, std::is_base_of<C, Obj>::value &&
|
||||
!absl::is_function<R>::value> {};
|
||||
|
||||
template <typename DataMem, typename Ref>
|
||||
static decltype(std::declval<Ref>().*std::declval<DataMem>()) Invoke(
|
||||
DataMem&& data_mem, Ref&& ref) {
|
||||
return std::forward<Ref>(ref).*std::forward<DataMem>(data_mem);
|
||||
}
|
||||
};
|
||||
|
||||
// (*t1).*f when N == 1 and f is a pointer to member data of a class T and t1
|
||||
// is not one of the types described in the previous item.
|
||||
struct DataMemAndPtr : StrippedAccept<DataMemAndPtr> {
|
||||
template <typename... Args>
|
||||
struct AcceptImpl : std::false_type {};
|
||||
|
||||
template <typename R, typename C, typename Ptr>
|
||||
struct AcceptImpl<R C::*, Ptr>
|
||||
: std::integral_constant<bool, !std::is_base_of<C, Ptr>::value &&
|
||||
!absl::is_function<R>::value> {};
|
||||
|
||||
template <typename DataMem, typename Ptr>
|
||||
static decltype((*std::declval<Ptr>()).*std::declval<DataMem>()) Invoke(
|
||||
DataMem&& data_mem, Ptr&& ptr) {
|
||||
return (*std::forward<Ptr>(ptr)).*std::forward<DataMem>(data_mem);
|
||||
}
|
||||
};
|
||||
|
||||
// f(t1, t2, ..., tN) in all other cases.
|
||||
struct Callable {
|
||||
// Callable doesn't have Accept because it's the last clause that gets picked
|
||||
// when none of the previous clauses are applicable.
|
||||
template <typename F, typename... Args>
|
||||
static decltype(std::declval<F>()(std::declval<Args>()...)) Invoke(
|
||||
F&& f, Args&&... args) {
|
||||
return std::forward<F>(f)(std::forward<Args>(args)...);
|
||||
}
|
||||
};
|
||||
|
||||
// Resolves to the first matching clause.
|
||||
template <typename... Args>
|
||||
struct Invoker {
|
||||
typedef typename std::conditional<
|
||||
MemFunAndRef::Accept<Args...>::value, MemFunAndRef,
|
||||
typename std::conditional<
|
||||
MemFunAndPtr::Accept<Args...>::value, MemFunAndPtr,
|
||||
typename std::conditional<
|
||||
DataMemAndRef::Accept<Args...>::value, DataMemAndRef,
|
||||
typename std::conditional<DataMemAndPtr::Accept<Args...>::value,
|
||||
DataMemAndPtr, Callable>::type>::type>::
|
||||
type>::type type;
|
||||
};
|
||||
|
||||
// The result type of Invoke<F, Args...>.
|
||||
template <typename F, typename... Args>
|
||||
using InvokeT = decltype(Invoker<F, Args...>::type::Invoke(
|
||||
std::declval<F>(), std::declval<Args>()...));
|
||||
|
||||
// Invoke(f, args...) is an implementation of INVOKE(f, args...) from section
|
||||
// [func.require] of the C++ standard.
|
||||
template <typename F, typename... Args>
|
||||
InvokeT<F, Args...> Invoke(F&& f, Args&&... args) {
|
||||
return Invoker<F, Args...>::type::Invoke(std::forward<F>(f),
|
||||
std::forward<Args>(args)...);
|
||||
}
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_INVOKE_H_
|
||||
+620
@@ -0,0 +1,620 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// A low-level allocator that can be used by other low-level
|
||||
// modules without introducing dependency cycles.
|
||||
// This allocator is slow and wasteful of memory;
|
||||
// it should not be used when performance is key.
|
||||
|
||||
#include "absl/base/internal/low_level_alloc.h"
|
||||
|
||||
#include <type_traits>
|
||||
|
||||
#include "absl/base/call_once.h"
|
||||
#include "absl/base/config.h"
|
||||
#include "absl/base/internal/direct_mmap.h"
|
||||
#include "absl/base/internal/scheduling_mode.h"
|
||||
#include "absl/base/macros.h"
|
||||
#include "absl/base/thread_annotations.h"
|
||||
|
||||
// LowLevelAlloc requires that the platform support low-level
|
||||
// allocation of virtual memory. Platforms lacking this cannot use
|
||||
// LowLevelAlloc.
|
||||
#ifndef ABSL_LOW_LEVEL_ALLOC_MISSING
|
||||
|
||||
#ifndef _WIN32
|
||||
#include <pthread.h>
|
||||
#include <signal.h>
|
||||
#include <sys/mman.h>
|
||||
#include <unistd.h>
|
||||
#else
|
||||
#include <windows.h>
|
||||
#endif
|
||||
|
||||
#include <string.h>
|
||||
#include <algorithm>
|
||||
#include <atomic>
|
||||
#include <cerrno>
|
||||
#include <cstddef>
|
||||
#include <new> // for placement-new
|
||||
|
||||
#include "absl/base/dynamic_annotations.h"
|
||||
#include "absl/base/internal/raw_logging.h"
|
||||
#include "absl/base/internal/spinlock.h"
|
||||
|
||||
// MAP_ANONYMOUS
|
||||
#if defined(__APPLE__)
|
||||
// For mmap, Linux defines both MAP_ANONYMOUS and MAP_ANON and says MAP_ANON is
|
||||
// deprecated. In Darwin, MAP_ANON is all there is.
|
||||
#if !defined MAP_ANONYMOUS
|
||||
#define MAP_ANONYMOUS MAP_ANON
|
||||
#endif // !MAP_ANONYMOUS
|
||||
#endif // __APPLE__
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// A first-fit allocator with amortized logarithmic free() time.
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
static const int kMaxLevel = 30;
|
||||
|
||||
namespace {
|
||||
// This struct describes one allocated block, or one free block.
|
||||
struct AllocList {
|
||||
struct Header {
|
||||
// Size of entire region, including this field. Must be
|
||||
// first. Valid in both allocated and unallocated blocks.
|
||||
uintptr_t size;
|
||||
|
||||
// kMagicAllocated or kMagicUnallocated xor this.
|
||||
uintptr_t magic;
|
||||
|
||||
// Pointer to parent arena.
|
||||
LowLevelAlloc::Arena *arena;
|
||||
|
||||
// Aligns regions to 0 mod 2*sizeof(void*).
|
||||
void *dummy_for_alignment;
|
||||
} header;
|
||||
|
||||
// Next two fields: in unallocated blocks: freelist skiplist data
|
||||
// in allocated blocks: overlaps with client data
|
||||
|
||||
// Levels in skiplist used.
|
||||
int levels;
|
||||
|
||||
// Actually has levels elements. The AllocList node may not have room
|
||||
// for all kMaxLevel entries. See max_fit in LLA_SkiplistLevels().
|
||||
AllocList *next[kMaxLevel];
|
||||
};
|
||||
} // namespace
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// A trivial skiplist implementation. This is used to keep the freelist
|
||||
// in address order while taking only logarithmic time per insert and delete.
|
||||
|
||||
// An integer approximation of log2(size/base)
|
||||
// Requires size >= base.
|
||||
static int IntLog2(size_t size, size_t base) {
|
||||
int result = 0;
|
||||
for (size_t i = size; i > base; i >>= 1) { // i == floor(size/2**result)
|
||||
result++;
|
||||
}
|
||||
// floor(size / 2**result) <= base < floor(size / 2**(result-1))
|
||||
// => log2(size/(base+1)) <= result < 1+log2(size/base)
|
||||
// => result ~= log2(size/base)
|
||||
return result;
|
||||
}
|
||||
|
||||
// Return a random integer n: p(n)=1/(2**n) if 1 <= n; p(n)=0 if n < 1.
|
||||
static int Random(uint32_t *state) {
|
||||
uint32_t r = *state;
|
||||
int result = 1;
|
||||
while ((((r = r*1103515245 + 12345) >> 30) & 1) == 0) {
|
||||
result++;
|
||||
}
|
||||
*state = r;
|
||||
return result;
|
||||
}
|
||||
|
||||
// Return a number of skiplist levels for a node of size bytes, where
|
||||
// base is the minimum node size. Compute level=log2(size / base)+n
|
||||
// where n is 1 if random is false and otherwise a random number generated with
|
||||
// the standard distribution for a skiplist: See Random() above.
|
||||
// Bigger nodes tend to have more skiplist levels due to the log2(size / base)
|
||||
// term, so first-fit searches touch fewer nodes. "level" is clipped so
|
||||
// level<kMaxLevel and next[level-1] will fit in the node.
|
||||
// 0 < LLA_SkiplistLevels(x,y,false) <= LLA_SkiplistLevels(x,y,true) < kMaxLevel
|
||||
static int LLA_SkiplistLevels(size_t size, size_t base, uint32_t *random) {
|
||||
// max_fit is the maximum number of levels that will fit in a node for the
|
||||
// given size. We can't return more than max_fit, no matter what the
|
||||
// random number generator says.
|
||||
size_t max_fit = (size - offsetof(AllocList, next)) / sizeof(AllocList *);
|
||||
int level = IntLog2(size, base) + (random != nullptr ? Random(random) : 1);
|
||||
if (static_cast<size_t>(level) > max_fit) level = static_cast<int>(max_fit);
|
||||
if (level > kMaxLevel-1) level = kMaxLevel - 1;
|
||||
ABSL_RAW_CHECK(level >= 1, "block not big enough for even one level");
|
||||
return level;
|
||||
}
|
||||
|
||||
// Return "atleast", the first element of AllocList *head s.t. *atleast >= *e.
|
||||
// For 0 <= i < head->levels, set prev[i] to "no_greater", where no_greater
|
||||
// points to the last element at level i in the AllocList less than *e, or is
|
||||
// head if no such element exists.
|
||||
static AllocList *LLA_SkiplistSearch(AllocList *head,
|
||||
AllocList *e, AllocList **prev) {
|
||||
AllocList *p = head;
|
||||
for (int level = head->levels - 1; level >= 0; level--) {
|
||||
for (AllocList *n; (n = p->next[level]) != nullptr && n < e; p = n) {
|
||||
}
|
||||
prev[level] = p;
|
||||
}
|
||||
return (head->levels == 0) ? nullptr : prev[0]->next[0];
|
||||
}
|
||||
|
||||
// Insert element *e into AllocList *head. Set prev[] as LLA_SkiplistSearch.
|
||||
// Requires that e->levels be previously set by the caller (using
|
||||
// LLA_SkiplistLevels())
|
||||
static void LLA_SkiplistInsert(AllocList *head, AllocList *e,
|
||||
AllocList **prev) {
|
||||
LLA_SkiplistSearch(head, e, prev);
|
||||
for (; head->levels < e->levels; head->levels++) { // extend prev pointers
|
||||
prev[head->levels] = head; // to all *e's levels
|
||||
}
|
||||
for (int i = 0; i != e->levels; i++) { // add element to list
|
||||
e->next[i] = prev[i]->next[i];
|
||||
prev[i]->next[i] = e;
|
||||
}
|
||||
}
|
||||
|
||||
// Remove element *e from AllocList *head. Set prev[] as LLA_SkiplistSearch().
|
||||
// Requires that e->levels be previous set by the caller (using
|
||||
// LLA_SkiplistLevels())
|
||||
static void LLA_SkiplistDelete(AllocList *head, AllocList *e,
|
||||
AllocList **prev) {
|
||||
AllocList *found = LLA_SkiplistSearch(head, e, prev);
|
||||
ABSL_RAW_CHECK(e == found, "element not in freelist");
|
||||
for (int i = 0; i != e->levels && prev[i]->next[i] == e; i++) {
|
||||
prev[i]->next[i] = e->next[i];
|
||||
}
|
||||
while (head->levels > 0 && head->next[head->levels - 1] == nullptr) {
|
||||
head->levels--; // reduce head->levels if level unused
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Arena implementation
|
||||
|
||||
// Metadata for an LowLevelAlloc arena instance.
|
||||
struct LowLevelAlloc::Arena {
|
||||
// Constructs an arena with the given LowLevelAlloc flags.
|
||||
explicit Arena(uint32_t flags_value);
|
||||
|
||||
base_internal::SpinLock mu;
|
||||
// Head of free list, sorted by address
|
||||
AllocList freelist ABSL_GUARDED_BY(mu);
|
||||
// Count of allocated blocks
|
||||
int32_t allocation_count ABSL_GUARDED_BY(mu);
|
||||
// flags passed to NewArena
|
||||
const uint32_t flags;
|
||||
// Result of sysconf(_SC_PAGESIZE)
|
||||
const size_t pagesize;
|
||||
// Lowest power of two >= max(16, sizeof(AllocList))
|
||||
const size_t round_up;
|
||||
// Smallest allocation block size
|
||||
const size_t min_size;
|
||||
// PRNG state
|
||||
uint32_t random ABSL_GUARDED_BY(mu);
|
||||
};
|
||||
|
||||
namespace {
|
||||
// Static storage space for the lazily-constructed, default global arena
|
||||
// instances. We require this space because the whole point of LowLevelAlloc
|
||||
// is to avoid relying on malloc/new.
|
||||
alignas(LowLevelAlloc::Arena) unsigned char default_arena_storage[sizeof(
|
||||
LowLevelAlloc::Arena)];
|
||||
alignas(LowLevelAlloc::Arena) unsigned char unhooked_arena_storage[sizeof(
|
||||
LowLevelAlloc::Arena)];
|
||||
#ifndef ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
alignas(
|
||||
LowLevelAlloc::Arena) unsigned char unhooked_async_sig_safe_arena_storage
|
||||
[sizeof(LowLevelAlloc::Arena)];
|
||||
#endif
|
||||
|
||||
// We must use LowLevelCallOnce here to construct the global arenas, rather than
|
||||
// using function-level statics, to avoid recursively invoking the scheduler.
|
||||
absl::once_flag create_globals_once;
|
||||
|
||||
void CreateGlobalArenas() {
|
||||
new (&default_arena_storage)
|
||||
LowLevelAlloc::Arena(LowLevelAlloc::kCallMallocHook);
|
||||
new (&unhooked_arena_storage) LowLevelAlloc::Arena(0);
|
||||
#ifndef ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
new (&unhooked_async_sig_safe_arena_storage)
|
||||
LowLevelAlloc::Arena(LowLevelAlloc::kAsyncSignalSafe);
|
||||
#endif
|
||||
}
|
||||
|
||||
// Returns a global arena that does not call into hooks. Used by NewArena()
|
||||
// when kCallMallocHook is not set.
|
||||
LowLevelAlloc::Arena* UnhookedArena() {
|
||||
base_internal::LowLevelCallOnce(&create_globals_once, CreateGlobalArenas);
|
||||
return reinterpret_cast<LowLevelAlloc::Arena*>(&unhooked_arena_storage);
|
||||
}
|
||||
|
||||
#ifndef ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
// Returns a global arena that is async-signal safe. Used by NewArena() when
|
||||
// kAsyncSignalSafe is set.
|
||||
LowLevelAlloc::Arena *UnhookedAsyncSigSafeArena() {
|
||||
base_internal::LowLevelCallOnce(&create_globals_once, CreateGlobalArenas);
|
||||
return reinterpret_cast<LowLevelAlloc::Arena *>(
|
||||
&unhooked_async_sig_safe_arena_storage);
|
||||
}
|
||||
#endif
|
||||
|
||||
} // namespace
|
||||
|
||||
// Returns the default arena, as used by LowLevelAlloc::Alloc() and friends.
|
||||
LowLevelAlloc::Arena *LowLevelAlloc::DefaultArena() {
|
||||
base_internal::LowLevelCallOnce(&create_globals_once, CreateGlobalArenas);
|
||||
return reinterpret_cast<LowLevelAlloc::Arena*>(&default_arena_storage);
|
||||
}
|
||||
|
||||
// magic numbers to identify allocated and unallocated blocks
|
||||
static const uintptr_t kMagicAllocated = 0x4c833e95U;
|
||||
static const uintptr_t kMagicUnallocated = ~kMagicAllocated;
|
||||
|
||||
namespace {
|
||||
class ABSL_SCOPED_LOCKABLE ArenaLock {
|
||||
public:
|
||||
explicit ArenaLock(LowLevelAlloc::Arena *arena)
|
||||
ABSL_EXCLUSIVE_LOCK_FUNCTION(arena->mu)
|
||||
: arena_(arena) {
|
||||
#ifndef ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
if ((arena->flags & LowLevelAlloc::kAsyncSignalSafe) != 0) {
|
||||
sigset_t all;
|
||||
sigfillset(&all);
|
||||
mask_valid_ = pthread_sigmask(SIG_BLOCK, &all, &mask_) == 0;
|
||||
}
|
||||
#endif
|
||||
arena_->mu.Lock();
|
||||
}
|
||||
~ArenaLock() { ABSL_RAW_CHECK(left_, "haven't left Arena region"); }
|
||||
void Leave() ABSL_UNLOCK_FUNCTION() {
|
||||
arena_->mu.Unlock();
|
||||
#ifndef ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
if (mask_valid_) {
|
||||
const int err = pthread_sigmask(SIG_SETMASK, &mask_, nullptr);
|
||||
if (err != 0) {
|
||||
ABSL_RAW_LOG(FATAL, "pthread_sigmask failed: %d", err);
|
||||
}
|
||||
}
|
||||
#endif
|
||||
left_ = true;
|
||||
}
|
||||
|
||||
private:
|
||||
bool left_ = false; // whether left region
|
||||
#ifndef ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
bool mask_valid_ = false;
|
||||
sigset_t mask_; // old mask of blocked signals
|
||||
#endif
|
||||
LowLevelAlloc::Arena *arena_;
|
||||
ArenaLock(const ArenaLock &) = delete;
|
||||
ArenaLock &operator=(const ArenaLock &) = delete;
|
||||
};
|
||||
} // namespace
|
||||
|
||||
// create an appropriate magic number for an object at "ptr"
|
||||
// "magic" should be kMagicAllocated or kMagicUnallocated
|
||||
inline static uintptr_t Magic(uintptr_t magic, AllocList::Header *ptr) {
|
||||
return magic ^ reinterpret_cast<uintptr_t>(ptr);
|
||||
}
|
||||
|
||||
namespace {
|
||||
size_t GetPageSize() {
|
||||
#ifdef _WIN32
|
||||
SYSTEM_INFO system_info;
|
||||
GetSystemInfo(&system_info);
|
||||
return std::max(system_info.dwPageSize, system_info.dwAllocationGranularity);
|
||||
#elif defined(__wasm__) || defined(__asmjs__)
|
||||
return getpagesize();
|
||||
#else
|
||||
return sysconf(_SC_PAGESIZE);
|
||||
#endif
|
||||
}
|
||||
|
||||
size_t RoundedUpBlockSize() {
|
||||
// Round up block sizes to a power of two close to the header size.
|
||||
size_t round_up = 16;
|
||||
while (round_up < sizeof(AllocList::Header)) {
|
||||
round_up += round_up;
|
||||
}
|
||||
return round_up;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
LowLevelAlloc::Arena::Arena(uint32_t flags_value)
|
||||
: mu(base_internal::SCHEDULE_KERNEL_ONLY),
|
||||
allocation_count(0),
|
||||
flags(flags_value),
|
||||
pagesize(GetPageSize()),
|
||||
round_up(RoundedUpBlockSize()),
|
||||
min_size(2 * round_up),
|
||||
random(0) {
|
||||
freelist.header.size = 0;
|
||||
freelist.header.magic =
|
||||
Magic(kMagicUnallocated, &freelist.header);
|
||||
freelist.header.arena = this;
|
||||
freelist.levels = 0;
|
||||
memset(freelist.next, 0, sizeof(freelist.next));
|
||||
}
|
||||
|
||||
// L < meta_data_arena->mu
|
||||
LowLevelAlloc::Arena *LowLevelAlloc::NewArena(int32_t flags) {
|
||||
Arena *meta_data_arena = DefaultArena();
|
||||
#ifndef ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
if ((flags & LowLevelAlloc::kAsyncSignalSafe) != 0) {
|
||||
meta_data_arena = UnhookedAsyncSigSafeArena();
|
||||
} else // NOLINT(readability/braces)
|
||||
#endif
|
||||
if ((flags & LowLevelAlloc::kCallMallocHook) == 0) {
|
||||
meta_data_arena = UnhookedArena();
|
||||
}
|
||||
Arena *result =
|
||||
new (AllocWithArena(sizeof (*result), meta_data_arena)) Arena(flags);
|
||||
return result;
|
||||
}
|
||||
|
||||
// L < arena->mu, L < arena->arena->mu
|
||||
bool LowLevelAlloc::DeleteArena(Arena *arena) {
|
||||
ABSL_RAW_CHECK(
|
||||
arena != nullptr && arena != DefaultArena() && arena != UnhookedArena(),
|
||||
"may not delete default arena");
|
||||
ArenaLock section(arena);
|
||||
if (arena->allocation_count != 0) {
|
||||
section.Leave();
|
||||
return false;
|
||||
}
|
||||
while (arena->freelist.next[0] != nullptr) {
|
||||
AllocList *region = arena->freelist.next[0];
|
||||
size_t size = region->header.size;
|
||||
arena->freelist.next[0] = region->next[0];
|
||||
ABSL_RAW_CHECK(
|
||||
region->header.magic == Magic(kMagicUnallocated, ®ion->header),
|
||||
"bad magic number in DeleteArena()");
|
||||
ABSL_RAW_CHECK(region->header.arena == arena,
|
||||
"bad arena pointer in DeleteArena()");
|
||||
ABSL_RAW_CHECK(size % arena->pagesize == 0,
|
||||
"empty arena has non-page-aligned block size");
|
||||
ABSL_RAW_CHECK(reinterpret_cast<uintptr_t>(region) % arena->pagesize == 0,
|
||||
"empty arena has non-page-aligned block");
|
||||
int munmap_result;
|
||||
#ifdef _WIN32
|
||||
munmap_result = VirtualFree(region, 0, MEM_RELEASE);
|
||||
ABSL_RAW_CHECK(munmap_result != 0,
|
||||
"LowLevelAlloc::DeleteArena: VitualFree failed");
|
||||
#else
|
||||
#ifndef ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
if ((arena->flags & LowLevelAlloc::kAsyncSignalSafe) == 0) {
|
||||
munmap_result = munmap(region, size);
|
||||
} else {
|
||||
munmap_result = base_internal::DirectMunmap(region, size);
|
||||
}
|
||||
#else
|
||||
munmap_result = munmap(region, size);
|
||||
#endif // ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
if (munmap_result != 0) {
|
||||
ABSL_RAW_LOG(FATAL, "LowLevelAlloc::DeleteArena: munmap failed: %d",
|
||||
errno);
|
||||
}
|
||||
#endif // _WIN32
|
||||
}
|
||||
section.Leave();
|
||||
arena->~Arena();
|
||||
Free(arena);
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// Addition, checking for overflow. The intent is to die if an external client
|
||||
// manages to push through a request that would cause arithmetic to fail.
|
||||
static inline uintptr_t CheckedAdd(uintptr_t a, uintptr_t b) {
|
||||
uintptr_t sum = a + b;
|
||||
ABSL_RAW_CHECK(sum >= a, "LowLevelAlloc arithmetic overflow");
|
||||
return sum;
|
||||
}
|
||||
|
||||
// Return value rounded up to next multiple of align.
|
||||
// align must be a power of two.
|
||||
static inline uintptr_t RoundUp(uintptr_t addr, uintptr_t align) {
|
||||
return CheckedAdd(addr, align - 1) & ~(align - 1);
|
||||
}
|
||||
|
||||
// Equivalent to "return prev->next[i]" but with sanity checking
|
||||
// that the freelist is in the correct order, that it
|
||||
// consists of regions marked "unallocated", and that no two regions
|
||||
// are adjacent in memory (they should have been coalesced).
|
||||
// L >= arena->mu
|
||||
static AllocList *Next(int i, AllocList *prev, LowLevelAlloc::Arena *arena) {
|
||||
ABSL_RAW_CHECK(i < prev->levels, "too few levels in Next()");
|
||||
AllocList *next = prev->next[i];
|
||||
if (next != nullptr) {
|
||||
ABSL_RAW_CHECK(
|
||||
next->header.magic == Magic(kMagicUnallocated, &next->header),
|
||||
"bad magic number in Next()");
|
||||
ABSL_RAW_CHECK(next->header.arena == arena, "bad arena pointer in Next()");
|
||||
if (prev != &arena->freelist) {
|
||||
ABSL_RAW_CHECK(prev < next, "unordered freelist");
|
||||
ABSL_RAW_CHECK(reinterpret_cast<char *>(prev) + prev->header.size <
|
||||
reinterpret_cast<char *>(next),
|
||||
"malformed freelist");
|
||||
}
|
||||
}
|
||||
return next;
|
||||
}
|
||||
|
||||
// Coalesce list item "a" with its successor if they are adjacent.
|
||||
static void Coalesce(AllocList *a) {
|
||||
AllocList *n = a->next[0];
|
||||
if (n != nullptr && reinterpret_cast<char *>(a) + a->header.size ==
|
||||
reinterpret_cast<char *>(n)) {
|
||||
LowLevelAlloc::Arena *arena = a->header.arena;
|
||||
a->header.size += n->header.size;
|
||||
n->header.magic = 0;
|
||||
n->header.arena = nullptr;
|
||||
AllocList *prev[kMaxLevel];
|
||||
LLA_SkiplistDelete(&arena->freelist, n, prev);
|
||||
LLA_SkiplistDelete(&arena->freelist, a, prev);
|
||||
a->levels = LLA_SkiplistLevels(a->header.size, arena->min_size,
|
||||
&arena->random);
|
||||
LLA_SkiplistInsert(&arena->freelist, a, prev);
|
||||
}
|
||||
}
|
||||
|
||||
// Adds block at location "v" to the free list
|
||||
// L >= arena->mu
|
||||
static void AddToFreelist(void *v, LowLevelAlloc::Arena *arena) {
|
||||
AllocList *f = reinterpret_cast<AllocList *>(
|
||||
reinterpret_cast<char *>(v) - sizeof (f->header));
|
||||
ABSL_RAW_CHECK(f->header.magic == Magic(kMagicAllocated, &f->header),
|
||||
"bad magic number in AddToFreelist()");
|
||||
ABSL_RAW_CHECK(f->header.arena == arena,
|
||||
"bad arena pointer in AddToFreelist()");
|
||||
f->levels = LLA_SkiplistLevels(f->header.size, arena->min_size,
|
||||
&arena->random);
|
||||
AllocList *prev[kMaxLevel];
|
||||
LLA_SkiplistInsert(&arena->freelist, f, prev);
|
||||
f->header.magic = Magic(kMagicUnallocated, &f->header);
|
||||
Coalesce(f); // maybe coalesce with successor
|
||||
Coalesce(prev[0]); // maybe coalesce with predecessor
|
||||
}
|
||||
|
||||
// Frees storage allocated by LowLevelAlloc::Alloc().
|
||||
// L < arena->mu
|
||||
void LowLevelAlloc::Free(void *v) {
|
||||
if (v != nullptr) {
|
||||
AllocList *f = reinterpret_cast<AllocList *>(
|
||||
reinterpret_cast<char *>(v) - sizeof (f->header));
|
||||
LowLevelAlloc::Arena *arena = f->header.arena;
|
||||
ArenaLock section(arena);
|
||||
AddToFreelist(v, arena);
|
||||
ABSL_RAW_CHECK(arena->allocation_count > 0, "nothing in arena to free");
|
||||
arena->allocation_count--;
|
||||
section.Leave();
|
||||
}
|
||||
}
|
||||
|
||||
// allocates and returns a block of size bytes, to be freed with Free()
|
||||
// L < arena->mu
|
||||
static void *DoAllocWithArena(size_t request, LowLevelAlloc::Arena *arena) {
|
||||
void *result = nullptr;
|
||||
if (request != 0) {
|
||||
AllocList *s; // will point to region that satisfies request
|
||||
ArenaLock section(arena);
|
||||
// round up with header
|
||||
size_t req_rnd = RoundUp(CheckedAdd(request, sizeof (s->header)),
|
||||
arena->round_up);
|
||||
for (;;) { // loop until we find a suitable region
|
||||
// find the minimum levels that a block of this size must have
|
||||
int i = LLA_SkiplistLevels(req_rnd, arena->min_size, nullptr) - 1;
|
||||
if (i < arena->freelist.levels) { // potential blocks exist
|
||||
AllocList *before = &arena->freelist; // predecessor of s
|
||||
while ((s = Next(i, before, arena)) != nullptr &&
|
||||
s->header.size < req_rnd) {
|
||||
before = s;
|
||||
}
|
||||
if (s != nullptr) { // we found a region
|
||||
break;
|
||||
}
|
||||
}
|
||||
// we unlock before mmap() both because mmap() may call a callback hook,
|
||||
// and because it may be slow.
|
||||
arena->mu.Unlock();
|
||||
// mmap generous 64K chunks to decrease
|
||||
// the chances/impact of fragmentation:
|
||||
size_t new_pages_size = RoundUp(req_rnd, arena->pagesize * 16);
|
||||
void *new_pages;
|
||||
#ifdef _WIN32
|
||||
new_pages = VirtualAlloc(0, new_pages_size,
|
||||
MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE);
|
||||
ABSL_RAW_CHECK(new_pages != nullptr, "VirtualAlloc failed");
|
||||
#else
|
||||
#ifndef ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
if ((arena->flags & LowLevelAlloc::kAsyncSignalSafe) != 0) {
|
||||
new_pages = base_internal::DirectMmap(nullptr, new_pages_size,
|
||||
PROT_WRITE|PROT_READ, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
|
||||
} else {
|
||||
new_pages = mmap(nullptr, new_pages_size, PROT_WRITE | PROT_READ,
|
||||
MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
|
||||
}
|
||||
#else
|
||||
new_pages = mmap(nullptr, new_pages_size, PROT_WRITE | PROT_READ,
|
||||
MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
|
||||
#endif // ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
if (new_pages == MAP_FAILED) {
|
||||
ABSL_RAW_LOG(FATAL, "mmap error: %d", errno);
|
||||
}
|
||||
|
||||
#endif // _WIN32
|
||||
arena->mu.Lock();
|
||||
s = reinterpret_cast<AllocList *>(new_pages);
|
||||
s->header.size = new_pages_size;
|
||||
// Pretend the block is allocated; call AddToFreelist() to free it.
|
||||
s->header.magic = Magic(kMagicAllocated, &s->header);
|
||||
s->header.arena = arena;
|
||||
AddToFreelist(&s->levels, arena); // insert new region into free list
|
||||
}
|
||||
AllocList *prev[kMaxLevel];
|
||||
LLA_SkiplistDelete(&arena->freelist, s, prev); // remove from free list
|
||||
// s points to the first free region that's big enough
|
||||
if (CheckedAdd(req_rnd, arena->min_size) <= s->header.size) {
|
||||
// big enough to split
|
||||
AllocList *n = reinterpret_cast<AllocList *>
|
||||
(req_rnd + reinterpret_cast<char *>(s));
|
||||
n->header.size = s->header.size - req_rnd;
|
||||
n->header.magic = Magic(kMagicAllocated, &n->header);
|
||||
n->header.arena = arena;
|
||||
s->header.size = req_rnd;
|
||||
AddToFreelist(&n->levels, arena);
|
||||
}
|
||||
s->header.magic = Magic(kMagicAllocated, &s->header);
|
||||
ABSL_RAW_CHECK(s->header.arena == arena, "");
|
||||
arena->allocation_count++;
|
||||
section.Leave();
|
||||
result = &s->levels;
|
||||
}
|
||||
ANNOTATE_MEMORY_IS_UNINITIALIZED(result, request);
|
||||
return result;
|
||||
}
|
||||
|
||||
void *LowLevelAlloc::Alloc(size_t request) {
|
||||
void *result = DoAllocWithArena(request, DefaultArena());
|
||||
return result;
|
||||
}
|
||||
|
||||
void *LowLevelAlloc::AllocWithArena(size_t request, Arena *arena) {
|
||||
ABSL_RAW_CHECK(arena != nullptr, "must pass a valid arena");
|
||||
void *result = DoAllocWithArena(request, arena);
|
||||
return result;
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_LOW_LEVEL_ALLOC_MISSING
|
||||
+126
@@ -0,0 +1,126 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_LOW_LEVEL_ALLOC_H_
|
||||
#define ABSL_BASE_INTERNAL_LOW_LEVEL_ALLOC_H_
|
||||
|
||||
// A simple thread-safe memory allocator that does not depend on
|
||||
// mutexes or thread-specific data. It is intended to be used
|
||||
// sparingly, and only when malloc() would introduce an unwanted
|
||||
// dependency, such as inside the heap-checker, or the Mutex
|
||||
// implementation.
|
||||
|
||||
// IWYU pragma: private, include "base/low_level_alloc.h"
|
||||
|
||||
#include <sys/types.h>
|
||||
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/config.h"
|
||||
|
||||
// LowLevelAlloc requires that the platform support low-level
|
||||
// allocation of virtual memory. Platforms lacking this cannot use
|
||||
// LowLevelAlloc.
|
||||
#ifdef ABSL_LOW_LEVEL_ALLOC_MISSING
|
||||
#error ABSL_LOW_LEVEL_ALLOC_MISSING cannot be directly set
|
||||
#elif !defined(ABSL_HAVE_MMAP) && !defined(_WIN32)
|
||||
#define ABSL_LOW_LEVEL_ALLOC_MISSING 1
|
||||
#endif
|
||||
|
||||
// Using LowLevelAlloc with kAsyncSignalSafe isn't supported on Windows or
|
||||
// asm.js / WebAssembly.
|
||||
// See https://kripken.github.io/emscripten-site/docs/porting/pthreads.html
|
||||
// for more information.
|
||||
#ifdef ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
#error ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING cannot be directly set
|
||||
#elif defined(_WIN32) || defined(__asmjs__) || defined(__wasm__)
|
||||
#define ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING 1
|
||||
#endif
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
#include "absl/base/port.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
class LowLevelAlloc {
|
||||
public:
|
||||
struct Arena; // an arena from which memory may be allocated
|
||||
|
||||
// Returns a pointer to a block of at least "request" bytes
|
||||
// that have been newly allocated from the specific arena.
|
||||
// for Alloc() call the DefaultArena() is used.
|
||||
// Returns 0 if passed request==0.
|
||||
// Does not return 0 under other circumstances; it crashes if memory
|
||||
// is not available.
|
||||
static void *Alloc(size_t request) ABSL_ATTRIBUTE_SECTION(malloc_hook);
|
||||
static void *AllocWithArena(size_t request, Arena *arena)
|
||||
ABSL_ATTRIBUTE_SECTION(malloc_hook);
|
||||
|
||||
// Deallocates a region of memory that was previously allocated with
|
||||
// Alloc(). Does nothing if passed 0. "s" must be either 0,
|
||||
// or must have been returned from a call to Alloc() and not yet passed to
|
||||
// Free() since that call to Alloc(). The space is returned to the arena
|
||||
// from which it was allocated.
|
||||
static void Free(void *s) ABSL_ATTRIBUTE_SECTION(malloc_hook);
|
||||
|
||||
// ABSL_ATTRIBUTE_SECTION(malloc_hook) for Alloc* and Free
|
||||
// are to put all callers of MallocHook::Invoke* in this module
|
||||
// into special section,
|
||||
// so that MallocHook::GetCallerStackTrace can function accurately.
|
||||
|
||||
// Create a new arena.
|
||||
// The root metadata for the new arena is allocated in the
|
||||
// meta_data_arena; the DefaultArena() can be passed for meta_data_arena.
|
||||
// These values may be ored into flags:
|
||||
enum {
|
||||
// Report calls to Alloc() and Free() via the MallocHook interface.
|
||||
// Set in the DefaultArena.
|
||||
kCallMallocHook = 0x0001,
|
||||
|
||||
#ifndef ABSL_LOW_LEVEL_ALLOC_ASYNC_SIGNAL_SAFE_MISSING
|
||||
// Make calls to Alloc(), Free() be async-signal-safe. Not set in
|
||||
// DefaultArena(). Not supported on all platforms.
|
||||
kAsyncSignalSafe = 0x0002,
|
||||
#endif
|
||||
};
|
||||
// Construct a new arena. The allocation of the underlying metadata honors
|
||||
// the provided flags. For example, the call NewArena(kAsyncSignalSafe)
|
||||
// is itself async-signal-safe, as well as generatating an arena that provides
|
||||
// async-signal-safe Alloc/Free.
|
||||
static Arena *NewArena(int32_t flags);
|
||||
|
||||
// Destroys an arena allocated by NewArena and returns true,
|
||||
// provided no allocated blocks remain in the arena.
|
||||
// If allocated blocks remain in the arena, does nothing and
|
||||
// returns false.
|
||||
// It is illegal to attempt to destroy the DefaultArena().
|
||||
static bool DeleteArena(Arena *arena);
|
||||
|
||||
// The default arena that always exists.
|
||||
static Arena *DefaultArena();
|
||||
|
||||
private:
|
||||
LowLevelAlloc(); // no instances
|
||||
};
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_LOW_LEVEL_ALLOC_H_
|
||||
+107
@@ -0,0 +1,107 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Core interfaces and definitions used by by low-level interfaces such as
|
||||
// SpinLock.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_LOW_LEVEL_SCHEDULING_H_
|
||||
#define ABSL_BASE_INTERNAL_LOW_LEVEL_SCHEDULING_H_
|
||||
|
||||
#include "absl/base/internal/scheduling_mode.h"
|
||||
#include "absl/base/macros.h"
|
||||
|
||||
// The following two declarations exist so SchedulingGuard may friend them with
|
||||
// the appropriate language linkage. These callbacks allow libc internals, such
|
||||
// as function level statics, to schedule cooperatively when locking.
|
||||
extern "C" bool __google_disable_rescheduling(void);
|
||||
extern "C" void __google_enable_rescheduling(bool disable_result);
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
class SchedulingHelper; // To allow use of SchedulingGuard.
|
||||
class SpinLock; // To allow use of SchedulingGuard.
|
||||
|
||||
// SchedulingGuard
|
||||
// Provides guard semantics that may be used to disable cooperative rescheduling
|
||||
// of the calling thread within specific program blocks. This is used to
|
||||
// protect resources (e.g. low-level SpinLocks or Domain code) that cooperative
|
||||
// scheduling depends on.
|
||||
//
|
||||
// Domain implementations capable of rescheduling in reaction to involuntary
|
||||
// kernel thread actions (e.g blocking due to a pagefault or syscall) must
|
||||
// guarantee that an annotated thread is not allowed to (cooperatively)
|
||||
// reschedule until the annotated region is complete.
|
||||
//
|
||||
// It is an error to attempt to use a cooperatively scheduled resource (e.g.
|
||||
// Mutex) within a rescheduling-disabled region.
|
||||
//
|
||||
// All methods are async-signal safe.
|
||||
class SchedulingGuard {
|
||||
public:
|
||||
// Returns true iff the calling thread may be cooperatively rescheduled.
|
||||
static bool ReschedulingIsAllowed();
|
||||
|
||||
private:
|
||||
// Disable cooperative rescheduling of the calling thread. It may still
|
||||
// initiate scheduling operations (e.g. wake-ups), however, it may not itself
|
||||
// reschedule. Nestable. The returned result is opaque, clients should not
|
||||
// attempt to interpret it.
|
||||
// REQUIRES: Result must be passed to a pairing EnableScheduling().
|
||||
static bool DisableRescheduling();
|
||||
|
||||
// Marks the end of a rescheduling disabled region, previously started by
|
||||
// DisableRescheduling().
|
||||
// REQUIRES: Pairs with innermost call (and result) of DisableRescheduling().
|
||||
static void EnableRescheduling(bool disable_result);
|
||||
|
||||
// A scoped helper for {Disable, Enable}Rescheduling().
|
||||
// REQUIRES: destructor must run in same thread as constructor.
|
||||
struct ScopedDisable {
|
||||
ScopedDisable() { disabled = SchedulingGuard::DisableRescheduling(); }
|
||||
~ScopedDisable() { SchedulingGuard::EnableRescheduling(disabled); }
|
||||
|
||||
bool disabled;
|
||||
};
|
||||
|
||||
// Access to SchedulingGuard is explicitly white-listed.
|
||||
friend class SchedulingHelper;
|
||||
friend class SpinLock;
|
||||
|
||||
SchedulingGuard(const SchedulingGuard&) = delete;
|
||||
SchedulingGuard& operator=(const SchedulingGuard&) = delete;
|
||||
};
|
||||
|
||||
//------------------------------------------------------------------------------
|
||||
// End of public interfaces.
|
||||
//------------------------------------------------------------------------------
|
||||
|
||||
inline bool SchedulingGuard::ReschedulingIsAllowed() {
|
||||
return false;
|
||||
}
|
||||
|
||||
inline bool SchedulingGuard::DisableRescheduling() {
|
||||
return false;
|
||||
}
|
||||
|
||||
inline void SchedulingGuard::EnableRescheduling(bool /* disable_result */) {
|
||||
return;
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_LOW_LEVEL_SCHEDULING_H_
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_PER_THREAD_TLS_H_
|
||||
#define ABSL_BASE_INTERNAL_PER_THREAD_TLS_H_
|
||||
|
||||
// This header defines two macros:
|
||||
//
|
||||
// If the platform supports thread-local storage:
|
||||
//
|
||||
// * ABSL_PER_THREAD_TLS_KEYWORD is the C keyword needed to declare a
|
||||
// thread-local variable
|
||||
// * ABSL_PER_THREAD_TLS is 1
|
||||
//
|
||||
// Otherwise:
|
||||
//
|
||||
// * ABSL_PER_THREAD_TLS_KEYWORD is empty
|
||||
// * ABSL_PER_THREAD_TLS is 0
|
||||
//
|
||||
// Microsoft C supports thread-local storage.
|
||||
// GCC supports it if the appropriate version of glibc is available,
|
||||
// which the programmer can indicate by defining ABSL_HAVE_TLS
|
||||
|
||||
#include "absl/base/port.h" // For ABSL_HAVE_TLS
|
||||
|
||||
#if defined(ABSL_PER_THREAD_TLS)
|
||||
#error ABSL_PER_THREAD_TLS cannot be directly set
|
||||
#elif defined(ABSL_PER_THREAD_TLS_KEYWORD)
|
||||
#error ABSL_PER_THREAD_TLS_KEYWORD cannot be directly set
|
||||
#elif defined(ABSL_HAVE_TLS)
|
||||
#define ABSL_PER_THREAD_TLS_KEYWORD __thread
|
||||
#define ABSL_PER_THREAD_TLS 1
|
||||
#elif defined(_MSC_VER)
|
||||
#define ABSL_PER_THREAD_TLS_KEYWORD __declspec(thread)
|
||||
#define ABSL_PER_THREAD_TLS 1
|
||||
#else
|
||||
#define ABSL_PER_THREAD_TLS_KEYWORD
|
||||
#define ABSL_PER_THREAD_TLS 0
|
||||
#endif
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_PER_THREAD_TLS_H_
|
||||
+53
@@ -0,0 +1,53 @@
|
||||
// Copyright 2019 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/base/internal/periodic_sampler.h"
|
||||
|
||||
#include <atomic>
|
||||
|
||||
#include "absl/base/internal/exponential_biased.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
int64_t PeriodicSamplerBase::GetExponentialBiased(int period) noexcept {
|
||||
return rng_.GetStride(period);
|
||||
}
|
||||
|
||||
bool PeriodicSamplerBase::SubtleConfirmSample() noexcept {
|
||||
int current_period = period();
|
||||
|
||||
// Deal with period case 0 (always off) and 1 (always on)
|
||||
if (ABSL_PREDICT_FALSE(current_period < 2)) {
|
||||
stride_ = 0;
|
||||
return current_period == 1;
|
||||
}
|
||||
|
||||
// Check if this is the first call to Sample()
|
||||
if (ABSL_PREDICT_FALSE(stride_ == 1)) {
|
||||
stride_ = static_cast<uint64_t>(-GetExponentialBiased(current_period));
|
||||
if (static_cast<int64_t>(stride_) < -1) {
|
||||
++stride_;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
stride_ = static_cast<uint64_t>(-GetExponentialBiased(current_period));
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
+211
@@ -0,0 +1,211 @@
|
||||
// Copyright 2019 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_PERIODIC_SAMPLER_H_
|
||||
#define ABSL_BASE_INTERNAL_PERIODIC_SAMPLER_H_
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
#include <atomic>
|
||||
|
||||
#include "absl/base/internal/exponential_biased.h"
|
||||
#include "absl/base/optimization.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// PeriodicSamplerBase provides the basic period sampler implementation.
|
||||
//
|
||||
// This is the base class for the templated PeriodicSampler class, which holds
|
||||
// a global std::atomic value identified by a user defined tag, such that
|
||||
// each specific PeriodSampler implementation holds its own global period.
|
||||
//
|
||||
// PeriodicSamplerBase is thread-compatible except where stated otherwise.
|
||||
class PeriodicSamplerBase {
|
||||
public:
|
||||
// PeriodicSamplerBase is trivial / copyable / movable / destructible.
|
||||
PeriodicSamplerBase() = default;
|
||||
PeriodicSamplerBase(PeriodicSamplerBase&&) = default;
|
||||
PeriodicSamplerBase(const PeriodicSamplerBase&) = default;
|
||||
|
||||
// Returns true roughly once every `period` calls. This is established by a
|
||||
// randomly picked `stride` that is counted down on each call to `Sample`.
|
||||
// This stride is picked such that the probability of `Sample()` returning
|
||||
// true is 1 in `period`.
|
||||
inline bool Sample() noexcept;
|
||||
|
||||
// The below methods are intended for optimized use cases where the
|
||||
// size of the inlined fast path code is highly important. Applications
|
||||
// should use the `Sample()` method unless they have proof that their
|
||||
// specific use case requires the optimizations offered by these methods.
|
||||
//
|
||||
// An example of such a use case is SwissTable sampling. All sampling checks
|
||||
// are in inlined SwissTable methods, and the number of call sites is huge.
|
||||
// In this case, the inlined code size added to each translation unit calling
|
||||
// SwissTable methods is non-trivial.
|
||||
//
|
||||
// The `SubtleMaybeSample()` function spuriously returns true even if the
|
||||
// function should not be sampled, applications MUST match each call to
|
||||
// 'SubtleMaybeSample()' returning true with a `SubtleConfirmSample()` call,
|
||||
// and use the result of the latter as the sampling decision.
|
||||
// In other words: the code should logically be equivalent to:
|
||||
//
|
||||
// if (SubtleMaybeSample() && SubtleConfirmSample()) {
|
||||
// // Sample this call
|
||||
// }
|
||||
//
|
||||
// In the 'inline-size' optimized case, the `SubtleConfirmSample()` call can
|
||||
// be placed out of line, for example, the typical use case looks as follows:
|
||||
//
|
||||
// // --- frobber.h -----------
|
||||
// void FrobberSampled();
|
||||
//
|
||||
// inline void FrobberImpl() {
|
||||
// // ...
|
||||
// }
|
||||
//
|
||||
// inline void Frobber() {
|
||||
// if (ABSL_PREDICT_FALSE(sampler.SubtleMaybeSample())) {
|
||||
// FrobberSampled();
|
||||
// } else {
|
||||
// FrobberImpl();
|
||||
// }
|
||||
// }
|
||||
//
|
||||
// // --- frobber.cc -----------
|
||||
// void FrobberSampled() {
|
||||
// if (!sampler.SubtleConfirmSample())) {
|
||||
// // Spurious false positive
|
||||
// FrobberImpl();
|
||||
// return;
|
||||
// }
|
||||
//
|
||||
// // Sampled execution
|
||||
// // ...
|
||||
// }
|
||||
inline bool SubtleMaybeSample() noexcept;
|
||||
bool SubtleConfirmSample() noexcept;
|
||||
|
||||
protected:
|
||||
// We explicitly don't use a virtual destructor as this class is never
|
||||
// virtually destroyed, and it keeps the class trivial, which avoids TLS
|
||||
// prologue and epilogue code for our TLS instances.
|
||||
~PeriodicSamplerBase() = default;
|
||||
|
||||
// Returns the next stride for our sampler.
|
||||
// This function is virtual for testing purposes only.
|
||||
virtual int64_t GetExponentialBiased(int period) noexcept;
|
||||
|
||||
private:
|
||||
// Returns the current period of this sampler. Thread-safe.
|
||||
virtual int period() const noexcept = 0;
|
||||
|
||||
// Keep and decrement stride_ as an unsigned integer, but compare the value
|
||||
// to zero casted as a signed int. clang and msvc do not create optimum code
|
||||
// if we use signed for the combined decrement and sign comparison.
|
||||
//
|
||||
// Below 3 alternative options, all compiles generate the best code
|
||||
// using the unsigned increment <---> signed int comparison option.
|
||||
//
|
||||
// Option 1:
|
||||
// int64_t stride_;
|
||||
// if (ABSL_PREDICT_TRUE(++stride_ < 0)) { ... }
|
||||
//
|
||||
// GCC x64 (OK) : https://gcc.godbolt.org/z/R5MzzA
|
||||
// GCC ppc (OK) : https://gcc.godbolt.org/z/z7NZAt
|
||||
// Clang x64 (BAD): https://gcc.godbolt.org/z/t4gPsd
|
||||
// ICC x64 (OK) : https://gcc.godbolt.org/z/rE6s8W
|
||||
// MSVC x64 (OK) : https://gcc.godbolt.org/z/ARMXqS
|
||||
//
|
||||
// Option 2:
|
||||
// int64_t stride_ = 0;
|
||||
// if (ABSL_PREDICT_TRUE(--stride_ >= 0)) { ... }
|
||||
//
|
||||
// GCC x64 (OK) : https://gcc.godbolt.org/z/jSQxYK
|
||||
// GCC ppc (OK) : https://gcc.godbolt.org/z/VJdYaA
|
||||
// Clang x64 (BAD): https://gcc.godbolt.org/z/Xm4NjX
|
||||
// ICC x64 (OK) : https://gcc.godbolt.org/z/4snaFd
|
||||
// MSVC x64 (BAD): https://gcc.godbolt.org/z/BgnEKE
|
||||
//
|
||||
// Option 3:
|
||||
// uint64_t stride_;
|
||||
// if (ABSL_PREDICT_TRUE(static_cast<int64_t>(++stride_) < 0)) { ... }
|
||||
//
|
||||
// GCC x64 (OK) : https://gcc.godbolt.org/z/bFbfPy
|
||||
// GCC ppc (OK) : https://gcc.godbolt.org/z/S9KkUE
|
||||
// Clang x64 (OK) : https://gcc.godbolt.org/z/UYzRb4
|
||||
// ICC x64 (OK) : https://gcc.godbolt.org/z/ptTNfD
|
||||
// MSVC x64 (OK) : https://gcc.godbolt.org/z/76j4-5
|
||||
uint64_t stride_ = 0;
|
||||
ExponentialBiased rng_;
|
||||
};
|
||||
|
||||
inline bool PeriodicSamplerBase::SubtleMaybeSample() noexcept {
|
||||
// See comments on `stride_` for the unsigned increment / signed compare.
|
||||
if (ABSL_PREDICT_TRUE(static_cast<int64_t>(++stride_) < 0)) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
inline bool PeriodicSamplerBase::Sample() noexcept {
|
||||
return ABSL_PREDICT_FALSE(SubtleMaybeSample()) ? SubtleConfirmSample()
|
||||
: false;
|
||||
}
|
||||
|
||||
// PeriodicSampler is a concreted periodic sampler implementation.
|
||||
// The user provided Tag identifies the implementation, and is required to
|
||||
// isolate the global state of this instance from other instances.
|
||||
//
|
||||
// Typical use case:
|
||||
//
|
||||
// struct HashTablezTag {};
|
||||
// thread_local PeriodicSampler sampler;
|
||||
//
|
||||
// void HashTableSamplingLogic(...) {
|
||||
// if (sampler.Sample()) {
|
||||
// HashTableSlowSamplePath(...);
|
||||
// }
|
||||
// }
|
||||
//
|
||||
template <typename Tag, int default_period = 0>
|
||||
class PeriodicSampler final : public PeriodicSamplerBase {
|
||||
public:
|
||||
~PeriodicSampler() = default;
|
||||
|
||||
int period() const noexcept final {
|
||||
return period_.load(std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
// Sets the global period for this sampler. Thread-safe.
|
||||
// Setting a period of 0 disables the sampler, i.e., every call to Sample()
|
||||
// will return false. Setting a period of 1 puts the sampler in 'always on'
|
||||
// mode, i.e., every call to Sample() returns true.
|
||||
static void SetGlobalPeriod(int period) {
|
||||
period_.store(period, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
private:
|
||||
static std::atomic<int> period_;
|
||||
};
|
||||
|
||||
template <typename Tag, int default_period>
|
||||
std::atomic<int> PeriodicSampler<Tag, default_period>::period_(default_period);
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_PERIODIC_SAMPLER_H_
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_PRETTY_FUNCTION_H_
|
||||
#define ABSL_BASE_INTERNAL_PRETTY_FUNCTION_H_
|
||||
|
||||
// ABSL_PRETTY_FUNCTION
|
||||
//
|
||||
// In C++11, __func__ gives the undecorated name of the current function. That
|
||||
// is, "main", not "int main()". Various compilers give extra macros to get the
|
||||
// decorated function name, including return type and arguments, to
|
||||
// differentiate between overload sets. ABSL_PRETTY_FUNCTION is a portable
|
||||
// version of these macros which forwards to the correct macro on each compiler.
|
||||
#if defined(_MSC_VER)
|
||||
#define ABSL_PRETTY_FUNCTION __FUNCSIG__
|
||||
#elif defined(__GNUC__)
|
||||
#define ABSL_PRETTY_FUNCTION __PRETTY_FUNCTION__
|
||||
#else
|
||||
#error "Unsupported compiler"
|
||||
#endif
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_PRETTY_FUNCTION_H_
|
||||
+240
@@ -0,0 +1,240 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/base/internal/raw_logging.h"
|
||||
|
||||
#include <stddef.h>
|
||||
#include <cstdarg>
|
||||
#include <cstdio>
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/config.h"
|
||||
#include "absl/base/internal/atomic_hook.h"
|
||||
#include "absl/base/log_severity.h"
|
||||
|
||||
// We know how to perform low-level writes to stderr in POSIX and Windows. For
|
||||
// these platforms, we define the token ABSL_LOW_LEVEL_WRITE_SUPPORTED.
|
||||
// Much of raw_logging.cc becomes a no-op when we can't output messages,
|
||||
// although a FATAL ABSL_RAW_LOG message will still abort the process.
|
||||
|
||||
// ABSL_HAVE_POSIX_WRITE is defined when the platform provides posix write()
|
||||
// (as from unistd.h)
|
||||
//
|
||||
// This preprocessor token is also defined in raw_io.cc. If you need to copy
|
||||
// this, consider moving both to config.h instead.
|
||||
#if defined(__linux__) || defined(__APPLE__) || defined(__FreeBSD__) || \
|
||||
defined(__Fuchsia__) || defined(__native_client__) || \
|
||||
defined(__EMSCRIPTEN__) || defined(__ASYLO__)
|
||||
|
||||
#include <unistd.h>
|
||||
|
||||
#define ABSL_HAVE_POSIX_WRITE 1
|
||||
#define ABSL_LOW_LEVEL_WRITE_SUPPORTED 1
|
||||
#else
|
||||
#undef ABSL_HAVE_POSIX_WRITE
|
||||
#endif
|
||||
|
||||
// ABSL_HAVE_SYSCALL_WRITE is defined when the platform provides the syscall
|
||||
// syscall(SYS_write, /*int*/ fd, /*char* */ buf, /*size_t*/ len);
|
||||
// for low level operations that want to avoid libc.
|
||||
#if (defined(__linux__) || defined(__FreeBSD__)) && !defined(__ANDROID__)
|
||||
#include <sys/syscall.h>
|
||||
#define ABSL_HAVE_SYSCALL_WRITE 1
|
||||
#define ABSL_LOW_LEVEL_WRITE_SUPPORTED 1
|
||||
#else
|
||||
#undef ABSL_HAVE_SYSCALL_WRITE
|
||||
#endif
|
||||
|
||||
#ifdef _WIN32
|
||||
#include <io.h>
|
||||
|
||||
#define ABSL_HAVE_RAW_IO 1
|
||||
#define ABSL_LOW_LEVEL_WRITE_SUPPORTED 1
|
||||
#else
|
||||
#undef ABSL_HAVE_RAW_IO
|
||||
#endif
|
||||
|
||||
// TODO(gfalcon): We want raw-logging to work on as many platforms as possible.
|
||||
// Explicitly #error out when not ABSL_LOW_LEVEL_WRITE_SUPPORTED, except for a
|
||||
// whitelisted set of platforms for which we expect not to be able to raw log.
|
||||
|
||||
ABSL_INTERNAL_ATOMIC_HOOK_ATTRIBUTES static absl::base_internal::AtomicHook<
|
||||
absl::raw_logging_internal::LogPrefixHook>
|
||||
log_prefix_hook;
|
||||
ABSL_INTERNAL_ATOMIC_HOOK_ATTRIBUTES static absl::base_internal::AtomicHook<
|
||||
absl::raw_logging_internal::AbortHook>
|
||||
abort_hook;
|
||||
|
||||
#ifdef ABSL_LOW_LEVEL_WRITE_SUPPORTED
|
||||
static const char kTruncated[] = " ... (message truncated)\n";
|
||||
|
||||
// sprintf the format to the buffer, adjusting *buf and *size to reflect the
|
||||
// consumed bytes, and return whether the message fit without truncation. If
|
||||
// truncation occurred, if possible leave room in the buffer for the message
|
||||
// kTruncated[].
|
||||
inline static bool VADoRawLog(char** buf, int* size, const char* format,
|
||||
va_list ap) ABSL_PRINTF_ATTRIBUTE(3, 0);
|
||||
inline static bool VADoRawLog(char** buf, int* size,
|
||||
const char* format, va_list ap) {
|
||||
int n = vsnprintf(*buf, *size, format, ap);
|
||||
bool result = true;
|
||||
if (n < 0 || n > *size) {
|
||||
result = false;
|
||||
if (static_cast<size_t>(*size) > sizeof(kTruncated)) {
|
||||
n = *size - sizeof(kTruncated); // room for truncation message
|
||||
} else {
|
||||
n = 0; // no room for truncation message
|
||||
}
|
||||
}
|
||||
*size -= n;
|
||||
*buf += n;
|
||||
return result;
|
||||
}
|
||||
#endif // ABSL_LOW_LEVEL_WRITE_SUPPORTED
|
||||
|
||||
static constexpr int kLogBufSize = 3000;
|
||||
|
||||
namespace {
|
||||
|
||||
// CAVEAT: vsnprintf called from *DoRawLog below has some (exotic) code paths
|
||||
// that invoke malloc() and getenv() that might acquire some locks.
|
||||
|
||||
// Helper for RawLog below.
|
||||
// *DoRawLog writes to *buf of *size and move them past the written portion.
|
||||
// It returns true iff there was no overflow or error.
|
||||
bool DoRawLog(char** buf, int* size, const char* format, ...)
|
||||
ABSL_PRINTF_ATTRIBUTE(3, 4);
|
||||
bool DoRawLog(char** buf, int* size, const char* format, ...) {
|
||||
va_list ap;
|
||||
va_start(ap, format);
|
||||
int n = vsnprintf(*buf, *size, format, ap);
|
||||
va_end(ap);
|
||||
if (n < 0 || n > *size) return false;
|
||||
*size -= n;
|
||||
*buf += n;
|
||||
return true;
|
||||
}
|
||||
|
||||
void RawLogVA(absl::LogSeverity severity, const char* file, int line,
|
||||
const char* format, va_list ap) ABSL_PRINTF_ATTRIBUTE(4, 0);
|
||||
void RawLogVA(absl::LogSeverity severity, const char* file, int line,
|
||||
const char* format, va_list ap) {
|
||||
char buffer[kLogBufSize];
|
||||
char* buf = buffer;
|
||||
int size = sizeof(buffer);
|
||||
#ifdef ABSL_LOW_LEVEL_WRITE_SUPPORTED
|
||||
bool enabled = true;
|
||||
#else
|
||||
bool enabled = false;
|
||||
#endif
|
||||
|
||||
#ifdef ABSL_MIN_LOG_LEVEL
|
||||
if (severity < static_cast<absl::LogSeverity>(ABSL_MIN_LOG_LEVEL) &&
|
||||
severity < absl::LogSeverity::kFatal) {
|
||||
enabled = false;
|
||||
}
|
||||
#endif
|
||||
|
||||
auto log_prefix_hook_ptr = log_prefix_hook.Load();
|
||||
if (log_prefix_hook_ptr) {
|
||||
enabled = log_prefix_hook_ptr(severity, file, line, &buf, &size);
|
||||
} else {
|
||||
if (enabled) {
|
||||
DoRawLog(&buf, &size, "[%s : %d] RAW: ", file, line);
|
||||
}
|
||||
}
|
||||
const char* const prefix_end = buf;
|
||||
|
||||
#ifdef ABSL_LOW_LEVEL_WRITE_SUPPORTED
|
||||
if (enabled) {
|
||||
bool no_chop = VADoRawLog(&buf, &size, format, ap);
|
||||
if (no_chop) {
|
||||
DoRawLog(&buf, &size, "\n");
|
||||
} else {
|
||||
DoRawLog(&buf, &size, "%s", kTruncated);
|
||||
}
|
||||
absl::raw_logging_internal::SafeWriteToStderr(buffer, strlen(buffer));
|
||||
}
|
||||
#else
|
||||
static_cast<void>(format);
|
||||
static_cast<void>(ap);
|
||||
#endif
|
||||
|
||||
// Abort the process after logging a FATAL message, even if the output itself
|
||||
// was suppressed.
|
||||
if (severity == absl::LogSeverity::kFatal) {
|
||||
abort_hook(file, line, buffer, prefix_end, buffer + kLogBufSize);
|
||||
abort();
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace raw_logging_internal {
|
||||
void SafeWriteToStderr(const char *s, size_t len) {
|
||||
#if defined(ABSL_HAVE_SYSCALL_WRITE)
|
||||
syscall(SYS_write, STDERR_FILENO, s, len);
|
||||
#elif defined(ABSL_HAVE_POSIX_WRITE)
|
||||
write(STDERR_FILENO, s, len);
|
||||
#elif defined(ABSL_HAVE_RAW_IO)
|
||||
_write(/* stderr */ 2, s, len);
|
||||
#else
|
||||
// stderr logging unsupported on this platform
|
||||
(void) s;
|
||||
(void) len;
|
||||
#endif
|
||||
}
|
||||
|
||||
void RawLog(absl::LogSeverity severity, const char* file, int line,
|
||||
const char* format, ...) ABSL_PRINTF_ATTRIBUTE(4, 5);
|
||||
void RawLog(absl::LogSeverity severity, const char* file, int line,
|
||||
const char* format, ...) {
|
||||
va_list ap;
|
||||
va_start(ap, format);
|
||||
RawLogVA(severity, file, line, format, ap);
|
||||
va_end(ap);
|
||||
}
|
||||
|
||||
// Non-formatting version of RawLog().
|
||||
//
|
||||
// TODO(gfalcon): When string_view no longer depends on base, change this
|
||||
// interface to take its message as a string_view instead.
|
||||
static void DefaultInternalLog(absl::LogSeverity severity, const char* file,
|
||||
int line, const std::string& message) {
|
||||
RawLog(severity, file, line, "%s", message.c_str());
|
||||
}
|
||||
|
||||
bool RawLoggingFullySupported() {
|
||||
#ifdef ABSL_LOW_LEVEL_WRITE_SUPPORTED
|
||||
return true;
|
||||
#else // !ABSL_LOW_LEVEL_WRITE_SUPPORTED
|
||||
return false;
|
||||
#endif // !ABSL_LOW_LEVEL_WRITE_SUPPORTED
|
||||
}
|
||||
|
||||
ABSL_DLL ABSL_INTERNAL_ATOMIC_HOOK_ATTRIBUTES
|
||||
absl::base_internal::AtomicHook<InternalLogFunction>
|
||||
internal_log_function(DefaultInternalLog);
|
||||
|
||||
void RegisterInternalLogFunction(InternalLogFunction func) {
|
||||
internal_log_function.Store(func);
|
||||
}
|
||||
|
||||
} // namespace raw_logging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
+183
@@ -0,0 +1,183 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Thread-safe logging routines that do not allocate any memory or
|
||||
// acquire any locks, and can therefore be used by low-level memory
|
||||
// allocation, synchronization, and signal-handling code.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_RAW_LOGGING_H_
|
||||
#define ABSL_BASE_INTERNAL_RAW_LOGGING_H_
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/config.h"
|
||||
#include "absl/base/internal/atomic_hook.h"
|
||||
#include "absl/base/log_severity.h"
|
||||
#include "absl/base/macros.h"
|
||||
#include "absl/base/optimization.h"
|
||||
#include "absl/base/port.h"
|
||||
|
||||
// This is similar to LOG(severity) << format..., but
|
||||
// * it is to be used ONLY by low-level modules that can't use normal LOG()
|
||||
// * it is designed to be a low-level logger that does not allocate any
|
||||
// memory and does not need any locks, hence:
|
||||
// * it logs straight and ONLY to STDERR w/o buffering
|
||||
// * it uses an explicit printf-format and arguments list
|
||||
// * it will silently chop off really long message strings
|
||||
// Usage example:
|
||||
// ABSL_RAW_LOG(ERROR, "Failed foo with %i: %s", status, error);
|
||||
// This will print an almost standard log line like this to stderr only:
|
||||
// E0821 211317 file.cc:123] RAW: Failed foo with 22: bad_file
|
||||
|
||||
#define ABSL_RAW_LOG(severity, ...) \
|
||||
do { \
|
||||
constexpr const char* absl_raw_logging_internal_basename = \
|
||||
::absl::raw_logging_internal::Basename(__FILE__, \
|
||||
sizeof(__FILE__) - 1); \
|
||||
::absl::raw_logging_internal::RawLog(ABSL_RAW_LOGGING_INTERNAL_##severity, \
|
||||
absl_raw_logging_internal_basename, \
|
||||
__LINE__, __VA_ARGS__); \
|
||||
} while (0)
|
||||
|
||||
// Similar to CHECK(condition) << message, but for low-level modules:
|
||||
// we use only ABSL_RAW_LOG that does not allocate memory.
|
||||
// We do not want to provide args list here to encourage this usage:
|
||||
// if (!cond) ABSL_RAW_LOG(FATAL, "foo ...", hard_to_compute_args);
|
||||
// so that the args are not computed when not needed.
|
||||
#define ABSL_RAW_CHECK(condition, message) \
|
||||
do { \
|
||||
if (ABSL_PREDICT_FALSE(!(condition))) { \
|
||||
ABSL_RAW_LOG(FATAL, "Check %s failed: %s", #condition, message); \
|
||||
} \
|
||||
} while (0)
|
||||
|
||||
// ABSL_INTERNAL_LOG and ABSL_INTERNAL_CHECK work like the RAW variants above,
|
||||
// except that if the richer log library is linked into the binary, we dispatch
|
||||
// to that instead. This is potentially useful for internal logging and
|
||||
// assertions, where we are using RAW_LOG neither for its async-signal-safety
|
||||
// nor for its non-allocating nature, but rather because raw logging has very
|
||||
// few other dependencies.
|
||||
//
|
||||
// The API is a subset of the above: each macro only takes two arguments. Use
|
||||
// StrCat if you need to build a richer message.
|
||||
#define ABSL_INTERNAL_LOG(severity, message) \
|
||||
do { \
|
||||
::absl::raw_logging_internal::internal_log_function( \
|
||||
ABSL_RAW_LOGGING_INTERNAL_##severity, __FILE__, __LINE__, message); \
|
||||
} while (0)
|
||||
|
||||
#define ABSL_INTERNAL_CHECK(condition, message) \
|
||||
do { \
|
||||
if (ABSL_PREDICT_FALSE(!(condition))) { \
|
||||
std::string death_message = "Check " #condition " failed: "; \
|
||||
death_message += std::string(message); \
|
||||
ABSL_INTERNAL_LOG(FATAL, death_message); \
|
||||
} \
|
||||
} while (0)
|
||||
|
||||
#define ABSL_RAW_LOGGING_INTERNAL_INFO ::absl::LogSeverity::kInfo
|
||||
#define ABSL_RAW_LOGGING_INTERNAL_WARNING ::absl::LogSeverity::kWarning
|
||||
#define ABSL_RAW_LOGGING_INTERNAL_ERROR ::absl::LogSeverity::kError
|
||||
#define ABSL_RAW_LOGGING_INTERNAL_FATAL ::absl::LogSeverity::kFatal
|
||||
#define ABSL_RAW_LOGGING_INTERNAL_LEVEL(severity) \
|
||||
::absl::NormalizeLogSeverity(severity)
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace raw_logging_internal {
|
||||
|
||||
// Helper function to implement ABSL_RAW_LOG
|
||||
// Logs format... at "severity" level, reporting it
|
||||
// as called from file:line.
|
||||
// This does not allocate memory or acquire locks.
|
||||
void RawLog(absl::LogSeverity severity, const char* file, int line,
|
||||
const char* format, ...) ABSL_PRINTF_ATTRIBUTE(4, 5);
|
||||
|
||||
// Writes the provided buffer directly to stderr, in a safe, low-level manner.
|
||||
//
|
||||
// In POSIX this means calling write(), which is async-signal safe and does
|
||||
// not malloc. If the platform supports the SYS_write syscall, we invoke that
|
||||
// directly to side-step any libc interception.
|
||||
void SafeWriteToStderr(const char *s, size_t len);
|
||||
|
||||
// compile-time function to get the "base" filename, that is, the part of
|
||||
// a filename after the last "/" or "\" path separator. The search starts at
|
||||
// the end of the string; the second parameter is the length of the string.
|
||||
constexpr const char* Basename(const char* fname, int offset) {
|
||||
return offset == 0 || fname[offset - 1] == '/' || fname[offset - 1] == '\\'
|
||||
? fname + offset
|
||||
: Basename(fname, offset - 1);
|
||||
}
|
||||
|
||||
// For testing only.
|
||||
// Returns true if raw logging is fully supported. When it is not
|
||||
// fully supported, no messages will be emitted, but a log at FATAL
|
||||
// severity will cause an abort.
|
||||
//
|
||||
// TODO(gfalcon): Come up with a better name for this method.
|
||||
bool RawLoggingFullySupported();
|
||||
|
||||
// Function type for a raw_logging customization hook for suppressing messages
|
||||
// by severity, and for writing custom prefixes on non-suppressed messages.
|
||||
//
|
||||
// The installed hook is called for every raw log invocation. The message will
|
||||
// be logged to stderr only if the hook returns true. FATAL errors will cause
|
||||
// the process to abort, even if writing to stderr is suppressed. The hook is
|
||||
// also provided with an output buffer, where it can write a custom log message
|
||||
// prefix.
|
||||
//
|
||||
// The raw_logging system does not allocate memory or grab locks. User-provided
|
||||
// hooks must avoid these operations, and must not throw exceptions.
|
||||
//
|
||||
// 'severity' is the severity level of the message being written.
|
||||
// 'file' and 'line' are the file and line number where the ABSL_RAW_LOG macro
|
||||
// was located.
|
||||
// 'buffer' and 'buf_size' are pointers to the buffer and buffer size. If the
|
||||
// hook writes a prefix, it must increment *buffer and decrement *buf_size
|
||||
// accordingly.
|
||||
using LogPrefixHook = bool (*)(absl::LogSeverity severity, const char* file,
|
||||
int line, char** buffer, int* buf_size);
|
||||
|
||||
// Function type for a raw_logging customization hook called to abort a process
|
||||
// when a FATAL message is logged. If the provided AbortHook() returns, the
|
||||
// logging system will call abort().
|
||||
//
|
||||
// 'file' and 'line' are the file and line number where the ABSL_RAW_LOG macro
|
||||
// was located.
|
||||
// The NUL-terminated logged message lives in the buffer between 'buf_start'
|
||||
// and 'buf_end'. 'prefix_end' points to the first non-prefix character of the
|
||||
// buffer (as written by the LogPrefixHook.)
|
||||
using AbortHook = void (*)(const char* file, int line, const char* buf_start,
|
||||
const char* prefix_end, const char* buf_end);
|
||||
|
||||
// Internal logging function for ABSL_INTERNAL_LOG to dispatch to.
|
||||
//
|
||||
// TODO(gfalcon): When string_view no longer depends on base, change this
|
||||
// interface to take its message as a string_view instead.
|
||||
using InternalLogFunction = void (*)(absl::LogSeverity severity,
|
||||
const char* file, int line,
|
||||
const std::string& message);
|
||||
|
||||
ABSL_DLL ABSL_INTERNAL_ATOMIC_HOOK_ATTRIBUTES extern base_internal::AtomicHook<
|
||||
InternalLogFunction>
|
||||
internal_log_function;
|
||||
|
||||
void RegisterInternalLogFunction(InternalLogFunction func);
|
||||
|
||||
} // namespace raw_logging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_RAW_LOGGING_H_
|
||||
+58
@@ -0,0 +1,58 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Core interfaces and definitions used by by low-level interfaces such as
|
||||
// SpinLock.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_SCHEDULING_MODE_H_
|
||||
#define ABSL_BASE_INTERNAL_SCHEDULING_MODE_H_
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// Used to describe how a thread may be scheduled. Typically associated with
|
||||
// the declaration of a resource supporting synchronized access.
|
||||
//
|
||||
// SCHEDULE_COOPERATIVE_AND_KERNEL:
|
||||
// Specifies that when waiting, a cooperative thread (e.g. a Fiber) may
|
||||
// reschedule (using base::scheduling semantics); allowing other cooperative
|
||||
// threads to proceed.
|
||||
//
|
||||
// SCHEDULE_KERNEL_ONLY: (Also described as "non-cooperative")
|
||||
// Specifies that no cooperative scheduling semantics may be used, even if the
|
||||
// current thread is itself cooperatively scheduled. This means that
|
||||
// cooperative threads will NOT allow other cooperative threads to execute in
|
||||
// their place while waiting for a resource of this type. Host operating system
|
||||
// semantics (e.g. a futex) may still be used.
|
||||
//
|
||||
// When optional, clients should strongly prefer SCHEDULE_COOPERATIVE_AND_KERNEL
|
||||
// by default. SCHEDULE_KERNEL_ONLY should only be used for resources on which
|
||||
// base::scheduling (e.g. the implementation of a Scheduler) may depend.
|
||||
//
|
||||
// NOTE: Cooperative resources may not be nested below non-cooperative ones.
|
||||
// This means that it is invalid to to acquire a SCHEDULE_COOPERATIVE_AND_KERNEL
|
||||
// resource if a SCHEDULE_KERNEL_ONLY resource is already held.
|
||||
enum SchedulingMode {
|
||||
SCHEDULE_KERNEL_ONLY = 0, // Allow scheduling only the host OS.
|
||||
SCHEDULE_COOPERATIVE_AND_KERNEL, // Also allow cooperative scheduling.
|
||||
};
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_SCHEDULING_MODE_H_
|
||||
+233
@@ -0,0 +1,233 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/base/internal/spinlock.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <atomic>
|
||||
#include <limits>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/internal/atomic_hook.h"
|
||||
#include "absl/base/internal/cycleclock.h"
|
||||
#include "absl/base/internal/spinlock_wait.h"
|
||||
#include "absl/base/internal/sysinfo.h" /* For NumCPUs() */
|
||||
#include "absl/base/call_once.h"
|
||||
|
||||
// Description of lock-word:
|
||||
// 31..00: [............................3][2][1][0]
|
||||
//
|
||||
// [0]: kSpinLockHeld
|
||||
// [1]: kSpinLockCooperative
|
||||
// [2]: kSpinLockDisabledScheduling
|
||||
// [31..3]: ONLY kSpinLockSleeper OR
|
||||
// Wait time in cycles >> PROFILE_TIMESTAMP_SHIFT
|
||||
//
|
||||
// Detailed descriptions:
|
||||
//
|
||||
// Bit [0]: The lock is considered held iff kSpinLockHeld is set.
|
||||
//
|
||||
// Bit [1]: Eligible waiters (e.g. Fibers) may co-operatively reschedule when
|
||||
// contended iff kSpinLockCooperative is set.
|
||||
//
|
||||
// Bit [2]: This bit is exclusive from bit [1]. It is used only by a
|
||||
// non-cooperative lock. When set, indicates that scheduling was
|
||||
// successfully disabled when the lock was acquired. May be unset,
|
||||
// even if non-cooperative, if a ThreadIdentity did not yet exist at
|
||||
// time of acquisition.
|
||||
//
|
||||
// Bit [3]: If this is the only upper bit ([31..3]) set then this lock was
|
||||
// acquired without contention, however, at least one waiter exists.
|
||||
//
|
||||
// Otherwise, bits [31..3] represent the time spent by the current lock
|
||||
// holder to acquire the lock. There may be outstanding waiter(s).
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
ABSL_INTERNAL_ATOMIC_HOOK_ATTRIBUTES static base_internal::AtomicHook<void (*)(
|
||||
const void *lock, int64_t wait_cycles)>
|
||||
submit_profile_data;
|
||||
|
||||
void RegisterSpinLockProfiler(void (*fn)(const void *contendedlock,
|
||||
int64_t wait_cycles)) {
|
||||
submit_profile_data.Store(fn);
|
||||
}
|
||||
|
||||
// Uncommon constructors.
|
||||
SpinLock::SpinLock(base_internal::SchedulingMode mode)
|
||||
: lockword_(IsCooperative(mode) ? kSpinLockCooperative : 0) {
|
||||
ABSL_TSAN_MUTEX_CREATE(this, __tsan_mutex_not_static);
|
||||
}
|
||||
|
||||
SpinLock::SpinLock(base_internal::LinkerInitialized,
|
||||
base_internal::SchedulingMode mode) {
|
||||
ABSL_TSAN_MUTEX_CREATE(this, 0);
|
||||
if (IsCooperative(mode)) {
|
||||
InitLinkerInitializedAndCooperative();
|
||||
}
|
||||
// Otherwise, lockword_ is already initialized.
|
||||
}
|
||||
|
||||
// Static (linker initialized) spinlocks always start life as functional
|
||||
// non-cooperative locks. When their static constructor does run, it will call
|
||||
// this initializer to augment the lockword with the cooperative bit. By
|
||||
// actually taking the lock when we do this we avoid the need for an atomic
|
||||
// operation in the regular unlock path.
|
||||
//
|
||||
// SlowLock() must be careful to re-test for this bit so that any outstanding
|
||||
// waiters may be upgraded to cooperative status.
|
||||
void SpinLock::InitLinkerInitializedAndCooperative() {
|
||||
Lock();
|
||||
lockword_.fetch_or(kSpinLockCooperative, std::memory_order_relaxed);
|
||||
Unlock();
|
||||
}
|
||||
|
||||
// Monitor the lock to see if its value changes within some time period
|
||||
// (adaptive_spin_count loop iterations). The last value read from the lock
|
||||
// is returned from the method.
|
||||
uint32_t SpinLock::SpinLoop() {
|
||||
// We are already in the slow path of SpinLock, initialize the
|
||||
// adaptive_spin_count here.
|
||||
ABSL_CONST_INIT static absl::once_flag init_adaptive_spin_count;
|
||||
ABSL_CONST_INIT static int adaptive_spin_count = 0;
|
||||
base_internal::LowLevelCallOnce(&init_adaptive_spin_count, []() {
|
||||
adaptive_spin_count = base_internal::NumCPUs() > 1 ? 1000 : 1;
|
||||
});
|
||||
|
||||
int c = adaptive_spin_count;
|
||||
uint32_t lock_value;
|
||||
do {
|
||||
lock_value = lockword_.load(std::memory_order_relaxed);
|
||||
} while ((lock_value & kSpinLockHeld) != 0 && --c > 0);
|
||||
return lock_value;
|
||||
}
|
||||
|
||||
void SpinLock::SlowLock() {
|
||||
uint32_t lock_value = SpinLoop();
|
||||
lock_value = TryLockInternal(lock_value, 0);
|
||||
if ((lock_value & kSpinLockHeld) == 0) {
|
||||
return;
|
||||
}
|
||||
// The lock was not obtained initially, so this thread needs to wait for
|
||||
// it. Record the current timestamp in the local variable wait_start_time
|
||||
// so the total wait time can be stored in the lockword once this thread
|
||||
// obtains the lock.
|
||||
int64_t wait_start_time = CycleClock::Now();
|
||||
uint32_t wait_cycles = 0;
|
||||
int lock_wait_call_count = 0;
|
||||
while ((lock_value & kSpinLockHeld) != 0) {
|
||||
// If the lock is currently held, but not marked as having a sleeper, mark
|
||||
// it as having a sleeper.
|
||||
if ((lock_value & kWaitTimeMask) == 0) {
|
||||
// Here, just "mark" that the thread is going to sleep. Don't store the
|
||||
// lock wait time in the lock as that will cause the current lock
|
||||
// owner to think it experienced contention.
|
||||
if (lockword_.compare_exchange_strong(
|
||||
lock_value, lock_value | kSpinLockSleeper,
|
||||
std::memory_order_relaxed, std::memory_order_relaxed)) {
|
||||
// Successfully transitioned to kSpinLockSleeper. Pass
|
||||
// kSpinLockSleeper to the SpinLockWait routine to properly indicate
|
||||
// the last lock_value observed.
|
||||
lock_value |= kSpinLockSleeper;
|
||||
} else if ((lock_value & kSpinLockHeld) == 0) {
|
||||
// Lock is free again, so try and acquire it before sleeping. The
|
||||
// new lock state will be the number of cycles this thread waited if
|
||||
// this thread obtains the lock.
|
||||
lock_value = TryLockInternal(lock_value, wait_cycles);
|
||||
continue; // Skip the delay at the end of the loop.
|
||||
}
|
||||
}
|
||||
|
||||
base_internal::SchedulingMode scheduling_mode;
|
||||
if ((lock_value & kSpinLockCooperative) != 0) {
|
||||
scheduling_mode = base_internal::SCHEDULE_COOPERATIVE_AND_KERNEL;
|
||||
} else {
|
||||
scheduling_mode = base_internal::SCHEDULE_KERNEL_ONLY;
|
||||
}
|
||||
// SpinLockDelay() calls into fiber scheduler, we need to see
|
||||
// synchronization there to avoid false positives.
|
||||
ABSL_TSAN_MUTEX_PRE_DIVERT(this, 0);
|
||||
// Wait for an OS specific delay.
|
||||
base_internal::SpinLockDelay(&lockword_, lock_value, ++lock_wait_call_count,
|
||||
scheduling_mode);
|
||||
ABSL_TSAN_MUTEX_POST_DIVERT(this, 0);
|
||||
// Spin again after returning from the wait routine to give this thread
|
||||
// some chance of obtaining the lock.
|
||||
lock_value = SpinLoop();
|
||||
wait_cycles = EncodeWaitCycles(wait_start_time, CycleClock::Now());
|
||||
lock_value = TryLockInternal(lock_value, wait_cycles);
|
||||
}
|
||||
}
|
||||
|
||||
void SpinLock::SlowUnlock(uint32_t lock_value) {
|
||||
base_internal::SpinLockWake(&lockword_,
|
||||
false); // wake waiter if necessary
|
||||
|
||||
// If our acquisition was contended, collect contentionz profile info. We
|
||||
// reserve a unitary wait time to represent that a waiter exists without our
|
||||
// own acquisition having been contended.
|
||||
if ((lock_value & kWaitTimeMask) != kSpinLockSleeper) {
|
||||
const uint64_t wait_cycles = DecodeWaitCycles(lock_value);
|
||||
ABSL_TSAN_MUTEX_PRE_DIVERT(this, 0);
|
||||
submit_profile_data(this, wait_cycles);
|
||||
ABSL_TSAN_MUTEX_POST_DIVERT(this, 0);
|
||||
}
|
||||
}
|
||||
|
||||
// We use the upper 29 bits of the lock word to store the time spent waiting to
|
||||
// acquire this lock. This is reported by contentionz profiling. Since the
|
||||
// lower bits of the cycle counter wrap very quickly on high-frequency
|
||||
// processors we divide to reduce the granularity to 2^PROFILE_TIMESTAMP_SHIFT
|
||||
// sized units. On a 4Ghz machine this will lose track of wait times greater
|
||||
// than (2^29/4 Ghz)*128 =~ 17.2 seconds. Such waits should be extremely rare.
|
||||
enum { PROFILE_TIMESTAMP_SHIFT = 7 };
|
||||
enum { LOCKWORD_RESERVED_SHIFT = 3 }; // We currently reserve the lower 3 bits.
|
||||
|
||||
uint32_t SpinLock::EncodeWaitCycles(int64_t wait_start_time,
|
||||
int64_t wait_end_time) {
|
||||
static const int64_t kMaxWaitTime =
|
||||
std::numeric_limits<uint32_t>::max() >> LOCKWORD_RESERVED_SHIFT;
|
||||
int64_t scaled_wait_time =
|
||||
(wait_end_time - wait_start_time) >> PROFILE_TIMESTAMP_SHIFT;
|
||||
|
||||
// Return a representation of the time spent waiting that can be stored in
|
||||
// the lock word's upper bits.
|
||||
uint32_t clamped = static_cast<uint32_t>(
|
||||
std::min(scaled_wait_time, kMaxWaitTime) << LOCKWORD_RESERVED_SHIFT);
|
||||
|
||||
if (clamped == 0) {
|
||||
return kSpinLockSleeper; // Just wake waiters, but don't record contention.
|
||||
}
|
||||
// Bump up value if necessary to avoid returning kSpinLockSleeper.
|
||||
const uint32_t kMinWaitTime =
|
||||
kSpinLockSleeper + (1 << LOCKWORD_RESERVED_SHIFT);
|
||||
if (clamped == kSpinLockSleeper) {
|
||||
return kMinWaitTime;
|
||||
}
|
||||
return clamped;
|
||||
}
|
||||
|
||||
uint64_t SpinLock::DecodeWaitCycles(uint32_t lock_value) {
|
||||
// Cast to uint32_t first to ensure bits [63:32] are cleared.
|
||||
const uint64_t scaled_wait_time =
|
||||
static_cast<uint32_t>(lock_value & kWaitTimeMask);
|
||||
return scaled_wait_time
|
||||
<< (PROFILE_TIMESTAMP_SHIFT - LOCKWORD_RESERVED_SHIFT);
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
+243
@@ -0,0 +1,243 @@
|
||||
//
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
|
||||
// Most users requiring mutual exclusion should use Mutex.
|
||||
// SpinLock is provided for use in three situations:
|
||||
// - for use in code that Mutex itself depends on
|
||||
// - to get a faster fast-path release under low contention (without an
|
||||
// atomic read-modify-write) In return, SpinLock has worse behaviour under
|
||||
// contention, which is why Mutex is preferred in most situations.
|
||||
// - for async signal safety (see below)
|
||||
|
||||
// SpinLock is async signal safe. If a spinlock is used within a signal
|
||||
// handler, all code that acquires the lock must ensure that the signal cannot
|
||||
// arrive while they are holding the lock. Typically, this is done by blocking
|
||||
// the signal.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_SPINLOCK_H_
|
||||
#define ABSL_BASE_INTERNAL_SPINLOCK_H_
|
||||
|
||||
#include <stdint.h>
|
||||
#include <sys/types.h>
|
||||
|
||||
#include <atomic>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/dynamic_annotations.h"
|
||||
#include "absl/base/internal/low_level_scheduling.h"
|
||||
#include "absl/base/internal/raw_logging.h"
|
||||
#include "absl/base/internal/scheduling_mode.h"
|
||||
#include "absl/base/internal/tsan_mutex_interface.h"
|
||||
#include "absl/base/macros.h"
|
||||
#include "absl/base/port.h"
|
||||
#include "absl/base/thread_annotations.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
class ABSL_LOCKABLE SpinLock {
|
||||
public:
|
||||
SpinLock() : lockword_(kSpinLockCooperative) {
|
||||
ABSL_TSAN_MUTEX_CREATE(this, __tsan_mutex_not_static);
|
||||
}
|
||||
|
||||
// Special constructor for use with static SpinLock objects. E.g.,
|
||||
//
|
||||
// static SpinLock lock(base_internal::kLinkerInitialized);
|
||||
//
|
||||
// When initialized using this constructor, we depend on the fact
|
||||
// that the linker has already initialized the memory appropriately. The lock
|
||||
// is initialized in non-cooperative mode.
|
||||
//
|
||||
// A SpinLock constructed like this can be freely used from global
|
||||
// initializers without worrying about the order in which global
|
||||
// initializers run.
|
||||
explicit SpinLock(base_internal::LinkerInitialized) {
|
||||
// Does nothing; lockword_ is already initialized
|
||||
ABSL_TSAN_MUTEX_CREATE(this, 0);
|
||||
}
|
||||
|
||||
// Constructors that allow non-cooperative spinlocks to be created for use
|
||||
// inside thread schedulers. Normal clients should not use these.
|
||||
explicit SpinLock(base_internal::SchedulingMode mode);
|
||||
SpinLock(base_internal::LinkerInitialized,
|
||||
base_internal::SchedulingMode mode);
|
||||
|
||||
~SpinLock() { ABSL_TSAN_MUTEX_DESTROY(this, __tsan_mutex_not_static); }
|
||||
|
||||
// Acquire this SpinLock.
|
||||
inline void Lock() ABSL_EXCLUSIVE_LOCK_FUNCTION() {
|
||||
ABSL_TSAN_MUTEX_PRE_LOCK(this, 0);
|
||||
if (!TryLockImpl()) {
|
||||
SlowLock();
|
||||
}
|
||||
ABSL_TSAN_MUTEX_POST_LOCK(this, 0, 0);
|
||||
}
|
||||
|
||||
// Try to acquire this SpinLock without blocking and return true if the
|
||||
// acquisition was successful. If the lock was not acquired, false is
|
||||
// returned. If this SpinLock is free at the time of the call, TryLock
|
||||
// will return true with high probability.
|
||||
inline bool TryLock() ABSL_EXCLUSIVE_TRYLOCK_FUNCTION(true) {
|
||||
ABSL_TSAN_MUTEX_PRE_LOCK(this, __tsan_mutex_try_lock);
|
||||
bool res = TryLockImpl();
|
||||
ABSL_TSAN_MUTEX_POST_LOCK(
|
||||
this, __tsan_mutex_try_lock | (res ? 0 : __tsan_mutex_try_lock_failed),
|
||||
0);
|
||||
return res;
|
||||
}
|
||||
|
||||
// Release this SpinLock, which must be held by the calling thread.
|
||||
inline void Unlock() ABSL_UNLOCK_FUNCTION() {
|
||||
ABSL_TSAN_MUTEX_PRE_UNLOCK(this, 0);
|
||||
uint32_t lock_value = lockword_.load(std::memory_order_relaxed);
|
||||
lock_value = lockword_.exchange(lock_value & kSpinLockCooperative,
|
||||
std::memory_order_release);
|
||||
|
||||
if ((lock_value & kSpinLockDisabledScheduling) != 0) {
|
||||
base_internal::SchedulingGuard::EnableRescheduling(true);
|
||||
}
|
||||
if ((lock_value & kWaitTimeMask) != 0) {
|
||||
// Collect contentionz profile info, and speed the wakeup of any waiter.
|
||||
// The wait_cycles value indicates how long this thread spent waiting
|
||||
// for the lock.
|
||||
SlowUnlock(lock_value);
|
||||
}
|
||||
ABSL_TSAN_MUTEX_POST_UNLOCK(this, 0);
|
||||
}
|
||||
|
||||
// Determine if the lock is held. When the lock is held by the invoking
|
||||
// thread, true will always be returned. Intended to be used as
|
||||
// CHECK(lock.IsHeld()).
|
||||
inline bool IsHeld() const {
|
||||
return (lockword_.load(std::memory_order_relaxed) & kSpinLockHeld) != 0;
|
||||
}
|
||||
|
||||
protected:
|
||||
// These should not be exported except for testing.
|
||||
|
||||
// Store number of cycles between wait_start_time and wait_end_time in a
|
||||
// lock value.
|
||||
static uint32_t EncodeWaitCycles(int64_t wait_start_time,
|
||||
int64_t wait_end_time);
|
||||
|
||||
// Extract number of wait cycles in a lock value.
|
||||
static uint64_t DecodeWaitCycles(uint32_t lock_value);
|
||||
|
||||
// Provide access to protected method above. Use for testing only.
|
||||
friend struct SpinLockTest;
|
||||
|
||||
private:
|
||||
// lockword_ is used to store the following:
|
||||
//
|
||||
// bit[0] encodes whether a lock is being held.
|
||||
// bit[1] encodes whether a lock uses cooperative scheduling.
|
||||
// bit[2] encodes whether a lock disables scheduling.
|
||||
// bit[3:31] encodes time a lock spent on waiting as a 29-bit unsigned int.
|
||||
enum { kSpinLockHeld = 1 };
|
||||
enum { kSpinLockCooperative = 2 };
|
||||
enum { kSpinLockDisabledScheduling = 4 };
|
||||
enum { kSpinLockSleeper = 8 };
|
||||
enum { kWaitTimeMask = // Includes kSpinLockSleeper.
|
||||
~(kSpinLockHeld | kSpinLockCooperative | kSpinLockDisabledScheduling) };
|
||||
|
||||
// Returns true if the provided scheduling mode is cooperative.
|
||||
static constexpr bool IsCooperative(
|
||||
base_internal::SchedulingMode scheduling_mode) {
|
||||
return scheduling_mode == base_internal::SCHEDULE_COOPERATIVE_AND_KERNEL;
|
||||
}
|
||||
|
||||
uint32_t TryLockInternal(uint32_t lock_value, uint32_t wait_cycles);
|
||||
void InitLinkerInitializedAndCooperative();
|
||||
void SlowLock() ABSL_ATTRIBUTE_COLD;
|
||||
void SlowUnlock(uint32_t lock_value) ABSL_ATTRIBUTE_COLD;
|
||||
uint32_t SpinLoop();
|
||||
|
||||
inline bool TryLockImpl() {
|
||||
uint32_t lock_value = lockword_.load(std::memory_order_relaxed);
|
||||
return (TryLockInternal(lock_value, 0) & kSpinLockHeld) == 0;
|
||||
}
|
||||
|
||||
std::atomic<uint32_t> lockword_;
|
||||
|
||||
SpinLock(const SpinLock&) = delete;
|
||||
SpinLock& operator=(const SpinLock&) = delete;
|
||||
};
|
||||
|
||||
// Corresponding locker object that arranges to acquire a spinlock for
|
||||
// the duration of a C++ scope.
|
||||
class ABSL_SCOPED_LOCKABLE SpinLockHolder {
|
||||
public:
|
||||
inline explicit SpinLockHolder(SpinLock* l) ABSL_EXCLUSIVE_LOCK_FUNCTION(l)
|
||||
: lock_(l) {
|
||||
l->Lock();
|
||||
}
|
||||
inline ~SpinLockHolder() ABSL_UNLOCK_FUNCTION() { lock_->Unlock(); }
|
||||
|
||||
SpinLockHolder(const SpinLockHolder&) = delete;
|
||||
SpinLockHolder& operator=(const SpinLockHolder&) = delete;
|
||||
|
||||
private:
|
||||
SpinLock* lock_;
|
||||
};
|
||||
|
||||
// Register a hook for profiling support.
|
||||
//
|
||||
// The function pointer registered here will be called whenever a spinlock is
|
||||
// contended. The callback is given an opaque handle to the contended spinlock
|
||||
// and the number of wait cycles. This is thread-safe, but only a single
|
||||
// profiler can be registered. It is an error to call this function multiple
|
||||
// times with different arguments.
|
||||
void RegisterSpinLockProfiler(void (*fn)(const void* lock,
|
||||
int64_t wait_cycles));
|
||||
|
||||
//------------------------------------------------------------------------------
|
||||
// Public interface ends here.
|
||||
//------------------------------------------------------------------------------
|
||||
|
||||
// If (result & kSpinLockHeld) == 0, then *this was successfully locked.
|
||||
// Otherwise, returns last observed value for lockword_.
|
||||
inline uint32_t SpinLock::TryLockInternal(uint32_t lock_value,
|
||||
uint32_t wait_cycles) {
|
||||
if ((lock_value & kSpinLockHeld) != 0) {
|
||||
return lock_value;
|
||||
}
|
||||
|
||||
uint32_t sched_disabled_bit = 0;
|
||||
if ((lock_value & kSpinLockCooperative) == 0) {
|
||||
// For non-cooperative locks we must make sure we mark ourselves as
|
||||
// non-reschedulable before we attempt to CompareAndSwap.
|
||||
if (base_internal::SchedulingGuard::DisableRescheduling()) {
|
||||
sched_disabled_bit = kSpinLockDisabledScheduling;
|
||||
}
|
||||
}
|
||||
|
||||
if (!lockword_.compare_exchange_strong(
|
||||
lock_value,
|
||||
kSpinLockHeld | lock_value | wait_cycles | sched_disabled_bit,
|
||||
std::memory_order_acquire, std::memory_order_relaxed)) {
|
||||
base_internal::SchedulingGuard::EnableRescheduling(sched_disabled_bit != 0);
|
||||
}
|
||||
|
||||
return lock_value;
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_SPINLOCK_H_
|
||||
+35
@@ -0,0 +1,35 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// This file is an Akaros-specific part of spinlock_wait.cc
|
||||
|
||||
#include <atomic>
|
||||
|
||||
#include "absl/base/internal/scheduling_mode.h"
|
||||
|
||||
extern "C" {
|
||||
|
||||
ABSL_ATTRIBUTE_WEAK void AbslInternalSpinLockDelay(
|
||||
std::atomic<uint32_t>* /* lock_word */, uint32_t /* value */,
|
||||
int /* loop */, absl::base_internal::SchedulingMode /* mode */) {
|
||||
// In Akaros, one must take care not to call anything that could cause a
|
||||
// malloc(), a blocking system call, or a uthread_yield() while holding a
|
||||
// spinlock. Our callers assume will not call into libraries or other
|
||||
// arbitrary code.
|
||||
}
|
||||
|
||||
ABSL_ATTRIBUTE_WEAK void AbslInternalSpinLockWake(
|
||||
std::atomic<uint32_t>* /* lock_word */, bool /* all */) {}
|
||||
|
||||
} // extern "C"
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// This file is a Linux-specific part of spinlock_wait.cc
|
||||
|
||||
#include <linux/futex.h>
|
||||
#include <sys/syscall.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <atomic>
|
||||
#include <climits>
|
||||
#include <cstdint>
|
||||
#include <ctime>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/internal/errno_saver.h"
|
||||
|
||||
// The SpinLock lockword is `std::atomic<uint32_t>`. Here we assert that
|
||||
// `std::atomic<uint32_t>` is bitwise equivalent of the `int` expected
|
||||
// by SYS_futex. We also assume that reads/writes done to the lockword
|
||||
// by SYS_futex have rational semantics with regard to the
|
||||
// std::atomic<> API. C++ provides no guarantees of these assumptions,
|
||||
// but they are believed to hold in practice.
|
||||
static_assert(sizeof(std::atomic<uint32_t>) == sizeof(int),
|
||||
"SpinLock lockword has the wrong size for a futex");
|
||||
|
||||
// Some Android headers are missing these definitions even though they
|
||||
// support these futex operations.
|
||||
#ifdef __BIONIC__
|
||||
#ifndef SYS_futex
|
||||
#define SYS_futex __NR_futex
|
||||
#endif
|
||||
#ifndef FUTEX_PRIVATE_FLAG
|
||||
#define FUTEX_PRIVATE_FLAG 128
|
||||
#endif
|
||||
#endif
|
||||
|
||||
extern "C" {
|
||||
|
||||
ABSL_ATTRIBUTE_WEAK void AbslInternalSpinLockDelay(
|
||||
std::atomic<uint32_t> *w, uint32_t value, int loop,
|
||||
absl::base_internal::SchedulingMode) {
|
||||
absl::base_internal::ErrnoSaver errno_saver;
|
||||
struct timespec tm;
|
||||
tm.tv_sec = 0;
|
||||
tm.tv_nsec = absl::base_internal::SpinLockSuggestedDelayNS(loop);
|
||||
syscall(SYS_futex, w, FUTEX_WAIT | FUTEX_PRIVATE_FLAG, value, &tm);
|
||||
}
|
||||
|
||||
ABSL_ATTRIBUTE_WEAK void AbslInternalSpinLockWake(std::atomic<uint32_t> *w,
|
||||
bool all) {
|
||||
syscall(SYS_futex, w, FUTEX_WAKE | FUTEX_PRIVATE_FLAG, all ? INT_MAX : 1, 0);
|
||||
}
|
||||
|
||||
} // extern "C"
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// This file is a Posix-specific part of spinlock_wait.cc
|
||||
|
||||
#include <sched.h>
|
||||
|
||||
#include <atomic>
|
||||
#include <ctime>
|
||||
|
||||
#include "absl/base/internal/errno_saver.h"
|
||||
#include "absl/base/internal/scheduling_mode.h"
|
||||
#include "absl/base/port.h"
|
||||
|
||||
extern "C" {
|
||||
|
||||
ABSL_ATTRIBUTE_WEAK void AbslInternalSpinLockDelay(
|
||||
std::atomic<uint32_t>* /* lock_word */, uint32_t /* value */, int loop,
|
||||
absl::base_internal::SchedulingMode /* mode */) {
|
||||
absl::base_internal::ErrnoSaver errno_saver;
|
||||
if (loop == 0) {
|
||||
} else if (loop == 1) {
|
||||
sched_yield();
|
||||
} else {
|
||||
struct timespec tm;
|
||||
tm.tv_sec = 0;
|
||||
tm.tv_nsec = absl::base_internal::SpinLockSuggestedDelayNS(loop);
|
||||
nanosleep(&tm, nullptr);
|
||||
}
|
||||
}
|
||||
|
||||
ABSL_ATTRIBUTE_WEAK void AbslInternalSpinLockWake(
|
||||
std::atomic<uint32_t>* /* lock_word */, bool /* all */) {}
|
||||
|
||||
} // extern "C"
|
||||
+81
@@ -0,0 +1,81 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// The OS-specific header included below must provide two calls:
|
||||
// AbslInternalSpinLockDelay() and AbslInternalSpinLockWake().
|
||||
// See spinlock_wait.h for the specs.
|
||||
|
||||
#include <atomic>
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/base/internal/spinlock_wait.h"
|
||||
|
||||
#if defined(_WIN32)
|
||||
#include "absl/base/internal/spinlock_win32.inc"
|
||||
#elif defined(__linux__)
|
||||
#include "absl/base/internal/spinlock_linux.inc"
|
||||
#elif defined(__akaros__)
|
||||
#include "absl/base/internal/spinlock_akaros.inc"
|
||||
#else
|
||||
#include "absl/base/internal/spinlock_posix.inc"
|
||||
#endif
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// See spinlock_wait.h for spec.
|
||||
uint32_t SpinLockWait(std::atomic<uint32_t> *w, int n,
|
||||
const SpinLockWaitTransition trans[],
|
||||
base_internal::SchedulingMode scheduling_mode) {
|
||||
int loop = 0;
|
||||
for (;;) {
|
||||
uint32_t v = w->load(std::memory_order_acquire);
|
||||
int i;
|
||||
for (i = 0; i != n && v != trans[i].from; i++) {
|
||||
}
|
||||
if (i == n) {
|
||||
SpinLockDelay(w, v, ++loop, scheduling_mode); // no matching transition
|
||||
} else if (trans[i].to == v || // null transition
|
||||
w->compare_exchange_strong(v, trans[i].to,
|
||||
std::memory_order_acquire,
|
||||
std::memory_order_relaxed)) {
|
||||
if (trans[i].done) return v;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static std::atomic<uint64_t> delay_rand;
|
||||
|
||||
// Return a suggested delay in nanoseconds for iteration number "loop"
|
||||
int SpinLockSuggestedDelayNS(int loop) {
|
||||
// Weak pseudo-random number generator to get some spread between threads
|
||||
// when many are spinning.
|
||||
uint64_t r = delay_rand.load(std::memory_order_relaxed);
|
||||
r = 0x5deece66dLL * r + 0xb; // numbers from nrand48()
|
||||
delay_rand.store(r, std::memory_order_relaxed);
|
||||
|
||||
if (loop < 0 || loop > 32) { // limit loop to 0..32
|
||||
loop = 32;
|
||||
}
|
||||
const int kMinDelay = 128 << 10; // 128us
|
||||
// Double delay every 8 iterations, up to 16x (2ms).
|
||||
int delay = kMinDelay << (loop / 8);
|
||||
// Randomize in delay..2*delay range, for resulting 128us..4ms range.
|
||||
return delay | ((delay - 1) & static_cast<int>(r));
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
+93
@@ -0,0 +1,93 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_SPINLOCK_WAIT_H_
|
||||
#define ABSL_BASE_INTERNAL_SPINLOCK_WAIT_H_
|
||||
|
||||
// Operations to make atomic transitions on a word, and to allow
|
||||
// waiting for those transitions to become possible.
|
||||
|
||||
#include <stdint.h>
|
||||
#include <atomic>
|
||||
|
||||
#include "absl/base/internal/scheduling_mode.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// SpinLockWait() waits until it can perform one of several transitions from
|
||||
// "from" to "to". It returns when it performs a transition where done==true.
|
||||
struct SpinLockWaitTransition {
|
||||
uint32_t from;
|
||||
uint32_t to;
|
||||
bool done;
|
||||
};
|
||||
|
||||
// Wait until *w can transition from trans[i].from to trans[i].to for some i
|
||||
// satisfying 0<=i<n && trans[i].done, atomically make the transition,
|
||||
// then return the old value of *w. Make any other atomic transitions
|
||||
// where !trans[i].done, but continue waiting.
|
||||
uint32_t SpinLockWait(std::atomic<uint32_t> *w, int n,
|
||||
const SpinLockWaitTransition trans[],
|
||||
SchedulingMode scheduling_mode);
|
||||
|
||||
// If possible, wake some thread that has called SpinLockDelay(w, ...). If
|
||||
// "all" is true, wake all such threads. This call is a hint, and on some
|
||||
// systems it may be a no-op; threads calling SpinLockDelay() will always wake
|
||||
// eventually even if SpinLockWake() is never called.
|
||||
void SpinLockWake(std::atomic<uint32_t> *w, bool all);
|
||||
|
||||
// Wait for an appropriate spin delay on iteration "loop" of a
|
||||
// spin loop on location *w, whose previously observed value was "value".
|
||||
// SpinLockDelay() may do nothing, may yield the CPU, may sleep a clock tick,
|
||||
// or may wait for a delay that can be truncated by a call to SpinLockWake(w).
|
||||
// In all cases, it must return in bounded time even if SpinLockWake() is not
|
||||
// called.
|
||||
void SpinLockDelay(std::atomic<uint32_t> *w, uint32_t value, int loop,
|
||||
base_internal::SchedulingMode scheduling_mode);
|
||||
|
||||
// Helper used by AbslInternalSpinLockDelay.
|
||||
// Returns a suggested delay in nanoseconds for iteration number "loop".
|
||||
int SpinLockSuggestedDelayNS(int loop);
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
// In some build configurations we pass --detect-odr-violations to the
|
||||
// gold linker. This causes it to flag weak symbol overrides as ODR
|
||||
// violations. Because ODR only applies to C++ and not C,
|
||||
// --detect-odr-violations ignores symbols not mangled with C++ names.
|
||||
// By changing our extension points to be extern "C", we dodge this
|
||||
// check.
|
||||
extern "C" {
|
||||
void AbslInternalSpinLockWake(std::atomic<uint32_t> *w, bool all);
|
||||
void AbslInternalSpinLockDelay(
|
||||
std::atomic<uint32_t> *w, uint32_t value, int loop,
|
||||
absl::base_internal::SchedulingMode scheduling_mode);
|
||||
}
|
||||
|
||||
inline void absl::base_internal::SpinLockWake(std::atomic<uint32_t> *w,
|
||||
bool all) {
|
||||
AbslInternalSpinLockWake(w, all);
|
||||
}
|
||||
|
||||
inline void absl::base_internal::SpinLockDelay(
|
||||
std::atomic<uint32_t> *w, uint32_t value, int loop,
|
||||
absl::base_internal::SchedulingMode scheduling_mode) {
|
||||
AbslInternalSpinLockDelay(w, value, loop, scheduling_mode);
|
||||
}
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_SPINLOCK_WAIT_H_
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// This file is a Win32-specific part of spinlock_wait.cc
|
||||
|
||||
#include <windows.h>
|
||||
#include <atomic>
|
||||
#include "absl/base/internal/scheduling_mode.h"
|
||||
|
||||
extern "C" {
|
||||
|
||||
void AbslInternalSpinLockDelay(std::atomic<uint32_t>* /* lock_word */,
|
||||
uint32_t /* value */, int loop,
|
||||
absl::base_internal::SchedulingMode /* mode */) {
|
||||
if (loop == 0) {
|
||||
} else if (loop == 1) {
|
||||
Sleep(0);
|
||||
} else {
|
||||
Sleep(absl::base_internal::SpinLockSuggestedDelayNS(loop) / 1000000);
|
||||
}
|
||||
}
|
||||
|
||||
void AbslInternalSpinLockWake(std::atomic<uint32_t>* /* lock_word */,
|
||||
bool /* all */) {}
|
||||
|
||||
} // extern "C"
|
||||
+416
@@ -0,0 +1,416 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/base/internal/sysinfo.h"
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
|
||||
#ifdef _WIN32
|
||||
#include <windows.h>
|
||||
#else
|
||||
#include <fcntl.h>
|
||||
#include <pthread.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
#endif
|
||||
|
||||
#ifdef __linux__
|
||||
#include <sys/syscall.h>
|
||||
#endif
|
||||
|
||||
#if defined(__APPLE__) || defined(__FreeBSD__)
|
||||
#include <sys/sysctl.h>
|
||||
#endif
|
||||
|
||||
#if defined(__myriad2__)
|
||||
#include <rtems.h>
|
||||
#endif
|
||||
|
||||
#include <string.h>
|
||||
#include <cassert>
|
||||
#include <cstdint>
|
||||
#include <cstdio>
|
||||
#include <cstdlib>
|
||||
#include <ctime>
|
||||
#include <limits>
|
||||
#include <thread> // NOLINT(build/c++11)
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include "absl/base/call_once.h"
|
||||
#include "absl/base/internal/raw_logging.h"
|
||||
#include "absl/base/internal/spinlock.h"
|
||||
#include "absl/base/internal/unscaledcycleclock.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
static int GetNumCPUs() {
|
||||
#if defined(__myriad2__)
|
||||
return 1;
|
||||
#else
|
||||
// Other possibilities:
|
||||
// - Read /sys/devices/system/cpu/online and use cpumask_parse()
|
||||
// - sysconf(_SC_NPROCESSORS_ONLN)
|
||||
return std::thread::hardware_concurrency();
|
||||
#endif
|
||||
}
|
||||
|
||||
#if defined(_WIN32)
|
||||
|
||||
static double GetNominalCPUFrequency() {
|
||||
#pragma comment(lib, "advapi32.lib") // For Reg* functions.
|
||||
HKEY key;
|
||||
// Use the Reg* functions rather than the SH functions because shlwapi.dll
|
||||
// pulls in gdi32.dll which makes process destruction much more costly.
|
||||
if (RegOpenKeyExA(HKEY_LOCAL_MACHINE,
|
||||
"HARDWARE\\DESCRIPTION\\System\\CentralProcessor\\0", 0,
|
||||
KEY_READ, &key) == ERROR_SUCCESS) {
|
||||
DWORD type = 0;
|
||||
DWORD data = 0;
|
||||
DWORD data_size = sizeof(data);
|
||||
auto result = RegQueryValueExA(key, "~MHz", 0, &type,
|
||||
reinterpret_cast<LPBYTE>(&data), &data_size);
|
||||
RegCloseKey(key);
|
||||
if (result == ERROR_SUCCESS && type == REG_DWORD &&
|
||||
data_size == sizeof(data)) {
|
||||
return data * 1e6; // Value is MHz.
|
||||
}
|
||||
}
|
||||
return 1.0;
|
||||
}
|
||||
|
||||
#elif defined(CTL_HW) && defined(HW_CPU_FREQ)
|
||||
|
||||
static double GetNominalCPUFrequency() {
|
||||
unsigned freq;
|
||||
size_t size = sizeof(freq);
|
||||
int mib[2] = {CTL_HW, HW_CPU_FREQ};
|
||||
if (sysctl(mib, 2, &freq, &size, nullptr, 0) == 0) {
|
||||
return static_cast<double>(freq);
|
||||
}
|
||||
return 1.0;
|
||||
}
|
||||
|
||||
#else
|
||||
|
||||
// Helper function for reading a long from a file. Returns true if successful
|
||||
// and the memory location pointed to by value is set to the value read.
|
||||
static bool ReadLongFromFile(const char *file, long *value) {
|
||||
bool ret = false;
|
||||
int fd = open(file, O_RDONLY);
|
||||
if (fd != -1) {
|
||||
char line[1024];
|
||||
char *err;
|
||||
memset(line, '\0', sizeof(line));
|
||||
int len = read(fd, line, sizeof(line) - 1);
|
||||
if (len <= 0) {
|
||||
ret = false;
|
||||
} else {
|
||||
const long temp_value = strtol(line, &err, 10);
|
||||
if (line[0] != '\0' && (*err == '\n' || *err == '\0')) {
|
||||
*value = temp_value;
|
||||
ret = true;
|
||||
}
|
||||
}
|
||||
close(fd);
|
||||
}
|
||||
return ret;
|
||||
}
|
||||
|
||||
#if defined(ABSL_INTERNAL_UNSCALED_CYCLECLOCK_FREQUENCY_IS_CPU_FREQUENCY)
|
||||
|
||||
// Reads a monotonic time source and returns a value in
|
||||
// nanoseconds. The returned value uses an arbitrary epoch, not the
|
||||
// Unix epoch.
|
||||
static int64_t ReadMonotonicClockNanos() {
|
||||
struct timespec t;
|
||||
#ifdef CLOCK_MONOTONIC_RAW
|
||||
int rc = clock_gettime(CLOCK_MONOTONIC_RAW, &t);
|
||||
#else
|
||||
int rc = clock_gettime(CLOCK_MONOTONIC, &t);
|
||||
#endif
|
||||
if (rc != 0) {
|
||||
perror("clock_gettime() failed");
|
||||
abort();
|
||||
}
|
||||
return int64_t{t.tv_sec} * 1000000000 + t.tv_nsec;
|
||||
}
|
||||
|
||||
class UnscaledCycleClockWrapperForInitializeFrequency {
|
||||
public:
|
||||
static int64_t Now() { return base_internal::UnscaledCycleClock::Now(); }
|
||||
};
|
||||
|
||||
struct TimeTscPair {
|
||||
int64_t time; // From ReadMonotonicClockNanos().
|
||||
int64_t tsc; // From UnscaledCycleClock::Now().
|
||||
};
|
||||
|
||||
// Returns a pair of values (monotonic kernel time, TSC ticks) that
|
||||
// approximately correspond to each other. This is accomplished by
|
||||
// doing several reads and picking the reading with the lowest
|
||||
// latency. This approach is used to minimize the probability that
|
||||
// our thread was preempted between clock reads.
|
||||
static TimeTscPair GetTimeTscPair() {
|
||||
int64_t best_latency = std::numeric_limits<int64_t>::max();
|
||||
TimeTscPair best;
|
||||
for (int i = 0; i < 10; ++i) {
|
||||
int64_t t0 = ReadMonotonicClockNanos();
|
||||
int64_t tsc = UnscaledCycleClockWrapperForInitializeFrequency::Now();
|
||||
int64_t t1 = ReadMonotonicClockNanos();
|
||||
int64_t latency = t1 - t0;
|
||||
if (latency < best_latency) {
|
||||
best_latency = latency;
|
||||
best.time = t0;
|
||||
best.tsc = tsc;
|
||||
}
|
||||
}
|
||||
return best;
|
||||
}
|
||||
|
||||
// Measures and returns the TSC frequency by taking a pair of
|
||||
// measurements approximately `sleep_nanoseconds` apart.
|
||||
static double MeasureTscFrequencyWithSleep(int sleep_nanoseconds) {
|
||||
auto t0 = GetTimeTscPair();
|
||||
struct timespec ts;
|
||||
ts.tv_sec = 0;
|
||||
ts.tv_nsec = sleep_nanoseconds;
|
||||
while (nanosleep(&ts, &ts) != 0 && errno == EINTR) {}
|
||||
auto t1 = GetTimeTscPair();
|
||||
double elapsed_ticks = t1.tsc - t0.tsc;
|
||||
double elapsed_time = (t1.time - t0.time) * 1e-9;
|
||||
return elapsed_ticks / elapsed_time;
|
||||
}
|
||||
|
||||
// Measures and returns the TSC frequency by calling
|
||||
// MeasureTscFrequencyWithSleep(), doubling the sleep interval until the
|
||||
// frequency measurement stabilizes.
|
||||
static double MeasureTscFrequency() {
|
||||
double last_measurement = -1.0;
|
||||
int sleep_nanoseconds = 1000000; // 1 millisecond.
|
||||
for (int i = 0; i < 8; ++i) {
|
||||
double measurement = MeasureTscFrequencyWithSleep(sleep_nanoseconds);
|
||||
if (measurement * 0.99 < last_measurement &&
|
||||
last_measurement < measurement * 1.01) {
|
||||
// Use the current measurement if it is within 1% of the
|
||||
// previous measurement.
|
||||
return measurement;
|
||||
}
|
||||
last_measurement = measurement;
|
||||
sleep_nanoseconds *= 2;
|
||||
}
|
||||
return last_measurement;
|
||||
}
|
||||
|
||||
#endif // ABSL_INTERNAL_UNSCALED_CYCLECLOCK_FREQUENCY_IS_CPU_FREQUENCY
|
||||
|
||||
static double GetNominalCPUFrequency() {
|
||||
long freq = 0;
|
||||
|
||||
// Google's production kernel has a patch to export the TSC
|
||||
// frequency through sysfs. If the kernel is exporting the TSC
|
||||
// frequency use that. There are issues where cpuinfo_max_freq
|
||||
// cannot be relied on because the BIOS may be exporting an invalid
|
||||
// p-state (on x86) or p-states may be used to put the processor in
|
||||
// a new mode (turbo mode). Essentially, those frequencies cannot
|
||||
// always be relied upon. The same reasons apply to /proc/cpuinfo as
|
||||
// well.
|
||||
if (ReadLongFromFile("/sys/devices/system/cpu/cpu0/tsc_freq_khz", &freq)) {
|
||||
return freq * 1e3; // Value is kHz.
|
||||
}
|
||||
|
||||
#if defined(ABSL_INTERNAL_UNSCALED_CYCLECLOCK_FREQUENCY_IS_CPU_FREQUENCY)
|
||||
// On these platforms, the TSC frequency is the nominal CPU
|
||||
// frequency. But without having the kernel export it directly
|
||||
// though /sys/devices/system/cpu/cpu0/tsc_freq_khz, there is no
|
||||
// other way to reliably get the TSC frequency, so we have to
|
||||
// measure it ourselves. Some CPUs abuse cpuinfo_max_freq by
|
||||
// exporting "fake" frequencies for implementing new features. For
|
||||
// example, Intel's turbo mode is enabled by exposing a p-state
|
||||
// value with a higher frequency than that of the real TSC
|
||||
// rate. Because of this, we prefer to measure the TSC rate
|
||||
// ourselves on i386 and x86-64.
|
||||
return MeasureTscFrequency();
|
||||
#else
|
||||
|
||||
// If CPU scaling is in effect, we want to use the *maximum*
|
||||
// frequency, not whatever CPU speed some random processor happens
|
||||
// to be using now.
|
||||
if (ReadLongFromFile("/sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_max_freq",
|
||||
&freq)) {
|
||||
return freq * 1e3; // Value is kHz.
|
||||
}
|
||||
|
||||
return 1.0;
|
||||
#endif // !ABSL_INTERNAL_UNSCALED_CYCLECLOCK_FREQUENCY_IS_CPU_FREQUENCY
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
ABSL_CONST_INIT static once_flag init_num_cpus_once;
|
||||
ABSL_CONST_INIT static int num_cpus = 0;
|
||||
|
||||
// NumCPUs() may be called before main() and before malloc is properly
|
||||
// initialized, therefore this must not allocate memory.
|
||||
int NumCPUs() {
|
||||
base_internal::LowLevelCallOnce(
|
||||
&init_num_cpus_once, []() { num_cpus = GetNumCPUs(); });
|
||||
return num_cpus;
|
||||
}
|
||||
|
||||
// A default frequency of 0.0 might be dangerous if it is used in division.
|
||||
ABSL_CONST_INIT static once_flag init_nominal_cpu_frequency_once;
|
||||
ABSL_CONST_INIT static double nominal_cpu_frequency = 1.0;
|
||||
|
||||
// NominalCPUFrequency() may be called before main() and before malloc is
|
||||
// properly initialized, therefore this must not allocate memory.
|
||||
double NominalCPUFrequency() {
|
||||
base_internal::LowLevelCallOnce(
|
||||
&init_nominal_cpu_frequency_once,
|
||||
[]() { nominal_cpu_frequency = GetNominalCPUFrequency(); });
|
||||
return nominal_cpu_frequency;
|
||||
}
|
||||
|
||||
#if defined(_WIN32)
|
||||
|
||||
pid_t GetTID() {
|
||||
return pid_t{GetCurrentThreadId()};
|
||||
}
|
||||
|
||||
#elif defined(__linux__)
|
||||
|
||||
#ifndef SYS_gettid
|
||||
#define SYS_gettid __NR_gettid
|
||||
#endif
|
||||
|
||||
pid_t GetTID() {
|
||||
return syscall(SYS_gettid);
|
||||
}
|
||||
|
||||
#elif defined(__akaros__)
|
||||
|
||||
pid_t GetTID() {
|
||||
// Akaros has a concept of "vcore context", which is the state the program
|
||||
// is forced into when we need to make a user-level scheduling decision, or
|
||||
// run a signal handler. This is analogous to the interrupt context that a
|
||||
// CPU might enter if it encounters some kind of exception.
|
||||
//
|
||||
// There is no current thread context in vcore context, but we need to give
|
||||
// a reasonable answer if asked for a thread ID (e.g., in a signal handler).
|
||||
// Thread 0 always exists, so if we are in vcore context, we return that.
|
||||
//
|
||||
// Otherwise, we know (since we are using pthreads) that the uthread struct
|
||||
// current_uthread is pointing to is the first element of a
|
||||
// struct pthread_tcb, so we extract and return the thread ID from that.
|
||||
//
|
||||
// TODO(dcross): Akaros anticipates moving the thread ID to the uthread
|
||||
// structure at some point. We should modify this code to remove the cast
|
||||
// when that happens.
|
||||
if (in_vcore_context())
|
||||
return 0;
|
||||
return reinterpret_cast<struct pthread_tcb *>(current_uthread)->id;
|
||||
}
|
||||
|
||||
#elif defined(__myriad2__)
|
||||
|
||||
pid_t GetTID() {
|
||||
uint32_t tid;
|
||||
rtems_task_ident(RTEMS_SELF, 0, &tid);
|
||||
return tid;
|
||||
}
|
||||
|
||||
#else
|
||||
|
||||
// Fallback implementation of GetTID using pthread_getspecific.
|
||||
static once_flag tid_once;
|
||||
static pthread_key_t tid_key;
|
||||
static absl::base_internal::SpinLock tid_lock(
|
||||
absl::base_internal::kLinkerInitialized);
|
||||
|
||||
// We set a bit per thread in this array to indicate that an ID is in
|
||||
// use. ID 0 is unused because it is the default value returned by
|
||||
// pthread_getspecific().
|
||||
static std::vector<uint32_t>* tid_array GUARDED_BY(tid_lock) = nullptr;
|
||||
static constexpr int kBitsPerWord = 32; // tid_array is uint32_t.
|
||||
|
||||
// Returns the TID to tid_array.
|
||||
static void FreeTID(void *v) {
|
||||
intptr_t tid = reinterpret_cast<intptr_t>(v);
|
||||
int word = tid / kBitsPerWord;
|
||||
uint32_t mask = ~(1u << (tid % kBitsPerWord));
|
||||
absl::base_internal::SpinLockHolder lock(&tid_lock);
|
||||
assert(0 <= word && static_cast<size_t>(word) < tid_array->size());
|
||||
(*tid_array)[word] &= mask;
|
||||
}
|
||||
|
||||
static void InitGetTID() {
|
||||
if (pthread_key_create(&tid_key, FreeTID) != 0) {
|
||||
// The logging system calls GetTID() so it can't be used here.
|
||||
perror("pthread_key_create failed");
|
||||
abort();
|
||||
}
|
||||
|
||||
// Initialize tid_array.
|
||||
absl::base_internal::SpinLockHolder lock(&tid_lock);
|
||||
tid_array = new std::vector<uint32_t>(1);
|
||||
(*tid_array)[0] = 1; // ID 0 is never-allocated.
|
||||
}
|
||||
|
||||
// Return a per-thread small integer ID from pthread's thread-specific data.
|
||||
pid_t GetTID() {
|
||||
absl::call_once(tid_once, InitGetTID);
|
||||
|
||||
intptr_t tid = reinterpret_cast<intptr_t>(pthread_getspecific(tid_key));
|
||||
if (tid != 0) {
|
||||
return tid;
|
||||
}
|
||||
|
||||
int bit; // tid_array[word] = 1u << bit;
|
||||
size_t word;
|
||||
{
|
||||
// Search for the first unused ID.
|
||||
absl::base_internal::SpinLockHolder lock(&tid_lock);
|
||||
// First search for a word in the array that is not all ones.
|
||||
word = 0;
|
||||
while (word < tid_array->size() && ~(*tid_array)[word] == 0) {
|
||||
++word;
|
||||
}
|
||||
if (word == tid_array->size()) {
|
||||
tid_array->push_back(0); // No space left, add kBitsPerWord more IDs.
|
||||
}
|
||||
// Search for a zero bit in the word.
|
||||
bit = 0;
|
||||
while (bit < kBitsPerWord && (((*tid_array)[word] >> bit) & 1) != 0) {
|
||||
++bit;
|
||||
}
|
||||
tid = (word * kBitsPerWord) + bit;
|
||||
(*tid_array)[word] |= 1u << bit; // Mark the TID as allocated.
|
||||
}
|
||||
|
||||
if (pthread_setspecific(tid_key, reinterpret_cast<void *>(tid)) != 0) {
|
||||
perror("pthread_setspecific failed");
|
||||
abort();
|
||||
}
|
||||
|
||||
return static_cast<pid_t>(tid);
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
Generated
+66
@@ -0,0 +1,66 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// This file includes routines to find out characteristics
|
||||
// of the machine a program is running on. It is undoubtedly
|
||||
// system-dependent.
|
||||
|
||||
// Functions listed here that accept a pid_t as an argument act on the
|
||||
// current process if the pid_t argument is 0
|
||||
// All functions here are thread-hostile due to file caching unless
|
||||
// commented otherwise.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_SYSINFO_H_
|
||||
#define ABSL_BASE_INTERNAL_SYSINFO_H_
|
||||
|
||||
#ifndef _WIN32
|
||||
#include <sys/types.h>
|
||||
#endif
|
||||
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/base/port.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// Nominal core processor cycles per second of each processor. This is _not_
|
||||
// necessarily the frequency of the CycleClock counter (see cycleclock.h)
|
||||
// Thread-safe.
|
||||
double NominalCPUFrequency();
|
||||
|
||||
// Number of logical processors (hyperthreads) in system. Thread-safe.
|
||||
int NumCPUs();
|
||||
|
||||
// Return the thread id of the current thread, as told by the system.
|
||||
// No two currently-live threads implemented by the OS shall have the same ID.
|
||||
// Thread ids of exited threads may be reused. Multiple user-level threads
|
||||
// may have the same thread ID if multiplexed on the same OS thread.
|
||||
//
|
||||
// On Linux, you may send a signal to the resulting ID with kill(). However,
|
||||
// it is recommended for portability that you use pthread_kill() instead.
|
||||
#ifdef _WIN32
|
||||
// On Windows, process id and thread id are of the same type according to the
|
||||
// return types of GetProcessId() and GetThreadId() are both DWORD, an unsigned
|
||||
// 32-bit type.
|
||||
using pid_t = uint32_t;
|
||||
#endif
|
||||
pid_t GetTID();
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_SYSINFO_H_
|
||||
+271
@@ -0,0 +1,271 @@
|
||||
// Copyright 2019 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: thread_annotations.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// WARNING: This is a backwards compatible header and it will be removed after
|
||||
// the migration to prefixed thread annotations is finished; please include
|
||||
// "absl/base/thread_annotations.h".
|
||||
//
|
||||
// This header file contains macro definitions for thread safety annotations
|
||||
// that allow developers to document the locking policies of multi-threaded
|
||||
// code. The annotations can also help program analysis tools to identify
|
||||
// potential thread safety issues.
|
||||
//
|
||||
// These annotations are implemented using compiler attributes. Using the macros
|
||||
// defined here instead of raw attributes allow for portability and future
|
||||
// compatibility.
|
||||
//
|
||||
// When referring to mutexes in the arguments of the attributes, you should
|
||||
// use variable names or more complex expressions (e.g. my_object->mutex_)
|
||||
// that evaluate to a concrete mutex object whenever possible. If the mutex
|
||||
// you want to refer to is not in scope, you may use a member pointer
|
||||
// (e.g. &MyClass::mutex_) to refer to a mutex in some (unknown) object.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_THREAD_ANNOTATIONS_H_
|
||||
#define ABSL_BASE_INTERNAL_THREAD_ANNOTATIONS_H_
|
||||
|
||||
#if defined(__clang__)
|
||||
#define THREAD_ANNOTATION_ATTRIBUTE__(x) __attribute__((x))
|
||||
#else
|
||||
#define THREAD_ANNOTATION_ATTRIBUTE__(x) // no-op
|
||||
#endif
|
||||
|
||||
// GUARDED_BY()
|
||||
//
|
||||
// Documents if a shared field or global variable needs to be protected by a
|
||||
// mutex. GUARDED_BY() allows the user to specify a particular mutex that
|
||||
// should be held when accessing the annotated variable.
|
||||
//
|
||||
// Although this annotation (and PT_GUARDED_BY, below) cannot be applied to
|
||||
// local variables, a local variable and its associated mutex can often be
|
||||
// combined into a small class or struct, thereby allowing the annotation.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// class Foo {
|
||||
// Mutex mu_;
|
||||
// int p1_ GUARDED_BY(mu_);
|
||||
// ...
|
||||
// };
|
||||
#define GUARDED_BY(x) THREAD_ANNOTATION_ATTRIBUTE__(guarded_by(x))
|
||||
|
||||
// PT_GUARDED_BY()
|
||||
//
|
||||
// Documents if the memory location pointed to by a pointer should be guarded
|
||||
// by a mutex when dereferencing the pointer.
|
||||
//
|
||||
// Example:
|
||||
// class Foo {
|
||||
// Mutex mu_;
|
||||
// int *p1_ PT_GUARDED_BY(mu_);
|
||||
// ...
|
||||
// };
|
||||
//
|
||||
// Note that a pointer variable to a shared memory location could itself be a
|
||||
// shared variable.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// // `q_`, guarded by `mu1_`, points to a shared memory location that is
|
||||
// // guarded by `mu2_`:
|
||||
// int *q_ GUARDED_BY(mu1_) PT_GUARDED_BY(mu2_);
|
||||
#define PT_GUARDED_BY(x) THREAD_ANNOTATION_ATTRIBUTE__(pt_guarded_by(x))
|
||||
|
||||
// ACQUIRED_AFTER() / ACQUIRED_BEFORE()
|
||||
//
|
||||
// Documents the acquisition order between locks that can be held
|
||||
// simultaneously by a thread. For any two locks that need to be annotated
|
||||
// to establish an acquisition order, only one of them needs the annotation.
|
||||
// (i.e. You don't have to annotate both locks with both ACQUIRED_AFTER
|
||||
// and ACQUIRED_BEFORE.)
|
||||
//
|
||||
// As with GUARDED_BY, this is only applicable to mutexes that are shared
|
||||
// fields or global variables.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// Mutex m1_;
|
||||
// Mutex m2_ ACQUIRED_AFTER(m1_);
|
||||
#define ACQUIRED_AFTER(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(acquired_after(__VA_ARGS__))
|
||||
|
||||
#define ACQUIRED_BEFORE(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(acquired_before(__VA_ARGS__))
|
||||
|
||||
// EXCLUSIVE_LOCKS_REQUIRED() / SHARED_LOCKS_REQUIRED()
|
||||
//
|
||||
// Documents a function that expects a mutex to be held prior to entry.
|
||||
// The mutex is expected to be held both on entry to, and exit from, the
|
||||
// function.
|
||||
//
|
||||
// An exclusive lock allows read-write access to the guarded data member(s), and
|
||||
// only one thread can acquire a lock exclusively at any one time. A shared lock
|
||||
// allows read-only access, and any number of threads can acquire a shared lock
|
||||
// concurrently.
|
||||
//
|
||||
// Generally, non-const methods should be annotated with
|
||||
// EXCLUSIVE_LOCKS_REQUIRED, while const methods should be annotated with
|
||||
// SHARED_LOCKS_REQUIRED.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// Mutex mu1, mu2;
|
||||
// int a GUARDED_BY(mu1);
|
||||
// int b GUARDED_BY(mu2);
|
||||
//
|
||||
// void foo() EXCLUSIVE_LOCKS_REQUIRED(mu1, mu2) { ... }
|
||||
// void bar() const SHARED_LOCKS_REQUIRED(mu1, mu2) { ... }
|
||||
#define EXCLUSIVE_LOCKS_REQUIRED(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(exclusive_locks_required(__VA_ARGS__))
|
||||
|
||||
#define SHARED_LOCKS_REQUIRED(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(shared_locks_required(__VA_ARGS__))
|
||||
|
||||
// LOCKS_EXCLUDED()
|
||||
//
|
||||
// Documents the locks acquired in the body of the function. These locks
|
||||
// cannot be held when calling this function (as Abseil's `Mutex` locks are
|
||||
// non-reentrant).
|
||||
#define LOCKS_EXCLUDED(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(locks_excluded(__VA_ARGS__))
|
||||
|
||||
// LOCK_RETURNED()
|
||||
//
|
||||
// Documents a function that returns a mutex without acquiring it. For example,
|
||||
// a public getter method that returns a pointer to a private mutex should
|
||||
// be annotated with LOCK_RETURNED.
|
||||
#define LOCK_RETURNED(x) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(lock_returned(x))
|
||||
|
||||
// LOCKABLE
|
||||
//
|
||||
// Documents if a class/type is a lockable type (such as the `Mutex` class).
|
||||
#define LOCKABLE \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(lockable)
|
||||
|
||||
// SCOPED_LOCKABLE
|
||||
//
|
||||
// Documents if a class does RAII locking (such as the `MutexLock` class).
|
||||
// The constructor should use `LOCK_FUNCTION()` to specify the mutex that is
|
||||
// acquired, and the destructor should use `UNLOCK_FUNCTION()` with no
|
||||
// arguments; the analysis will assume that the destructor unlocks whatever the
|
||||
// constructor locked.
|
||||
#define SCOPED_LOCKABLE \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(scoped_lockable)
|
||||
|
||||
// EXCLUSIVE_LOCK_FUNCTION()
|
||||
//
|
||||
// Documents functions that acquire a lock in the body of a function, and do
|
||||
// not release it.
|
||||
#define EXCLUSIVE_LOCK_FUNCTION(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(exclusive_lock_function(__VA_ARGS__))
|
||||
|
||||
// SHARED_LOCK_FUNCTION()
|
||||
//
|
||||
// Documents functions that acquire a shared (reader) lock in the body of a
|
||||
// function, and do not release it.
|
||||
#define SHARED_LOCK_FUNCTION(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(shared_lock_function(__VA_ARGS__))
|
||||
|
||||
// UNLOCK_FUNCTION()
|
||||
//
|
||||
// Documents functions that expect a lock to be held on entry to the function,
|
||||
// and release it in the body of the function.
|
||||
#define UNLOCK_FUNCTION(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(unlock_function(__VA_ARGS__))
|
||||
|
||||
// EXCLUSIVE_TRYLOCK_FUNCTION() / SHARED_TRYLOCK_FUNCTION()
|
||||
//
|
||||
// Documents functions that try to acquire a lock, and return success or failure
|
||||
// (or a non-boolean value that can be interpreted as a boolean).
|
||||
// The first argument should be `true` for functions that return `true` on
|
||||
// success, or `false` for functions that return `false` on success. The second
|
||||
// argument specifies the mutex that is locked on success. If unspecified, this
|
||||
// mutex is assumed to be `this`.
|
||||
#define EXCLUSIVE_TRYLOCK_FUNCTION(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(exclusive_trylock_function(__VA_ARGS__))
|
||||
|
||||
#define SHARED_TRYLOCK_FUNCTION(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(shared_trylock_function(__VA_ARGS__))
|
||||
|
||||
// ASSERT_EXCLUSIVE_LOCK() / ASSERT_SHARED_LOCK()
|
||||
//
|
||||
// Documents functions that dynamically check to see if a lock is held, and fail
|
||||
// if it is not held.
|
||||
#define ASSERT_EXCLUSIVE_LOCK(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(assert_exclusive_lock(__VA_ARGS__))
|
||||
|
||||
#define ASSERT_SHARED_LOCK(...) \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(assert_shared_lock(__VA_ARGS__))
|
||||
|
||||
// NO_THREAD_SAFETY_ANALYSIS
|
||||
//
|
||||
// Turns off thread safety checking within the body of a particular function.
|
||||
// This annotation is used to mark functions that are known to be correct, but
|
||||
// the locking behavior is more complicated than the analyzer can handle.
|
||||
#define NO_THREAD_SAFETY_ANALYSIS \
|
||||
THREAD_ANNOTATION_ATTRIBUTE__(no_thread_safety_analysis)
|
||||
|
||||
//------------------------------------------------------------------------------
|
||||
// Tool-Supplied Annotations
|
||||
//------------------------------------------------------------------------------
|
||||
|
||||
// TS_UNCHECKED should be placed around lock expressions that are not valid
|
||||
// C++ syntax, but which are present for documentation purposes. These
|
||||
// annotations will be ignored by the analysis.
|
||||
#define TS_UNCHECKED(x) ""
|
||||
|
||||
// TS_FIXME is used to mark lock expressions that are not valid C++ syntax.
|
||||
// It is used by automated tools to mark and disable invalid expressions.
|
||||
// The annotation should either be fixed, or changed to TS_UNCHECKED.
|
||||
#define TS_FIXME(x) ""
|
||||
|
||||
// Like NO_THREAD_SAFETY_ANALYSIS, this turns off checking within the body of
|
||||
// a particular function. However, this attribute is used to mark functions
|
||||
// that are incorrect and need to be fixed. It is used by automated tools to
|
||||
// avoid breaking the build when the analysis is updated.
|
||||
// Code owners are expected to eventually fix the routine.
|
||||
#define NO_THREAD_SAFETY_ANALYSIS_FIXME NO_THREAD_SAFETY_ANALYSIS
|
||||
|
||||
// Similar to NO_THREAD_SAFETY_ANALYSIS_FIXME, this macro marks a GUARDED_BY
|
||||
// annotation that needs to be fixed, because it is producing thread safety
|
||||
// warning. It disables the GUARDED_BY.
|
||||
#define GUARDED_BY_FIXME(x)
|
||||
|
||||
// Disables warnings for a single read operation. This can be used to avoid
|
||||
// warnings when it is known that the read is not actually involved in a race,
|
||||
// but the compiler cannot confirm that.
|
||||
#define TS_UNCHECKED_READ(x) thread_safety_analysis::ts_unchecked_read(x)
|
||||
|
||||
|
||||
namespace thread_safety_analysis {
|
||||
|
||||
// Takes a reference to a guarded data member, and returns an unguarded
|
||||
// reference.
|
||||
template <typename T>
|
||||
inline const T& ts_unchecked_read(const T& v) NO_THREAD_SAFETY_ANALYSIS {
|
||||
return v;
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
inline T& ts_unchecked_read(T& v) NO_THREAD_SAFETY_ANALYSIS {
|
||||
return v;
|
||||
}
|
||||
|
||||
} // namespace thread_safety_analysis
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_THREAD_ANNOTATIONS_H_
|
||||
+152
@@ -0,0 +1,152 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/base/internal/thread_identity.h"
|
||||
|
||||
#ifndef _WIN32
|
||||
#include <pthread.h>
|
||||
#include <signal.h>
|
||||
#endif
|
||||
|
||||
#include <atomic>
|
||||
#include <cassert>
|
||||
#include <memory>
|
||||
|
||||
#include "absl/base/call_once.h"
|
||||
#include "absl/base/internal/raw_logging.h"
|
||||
#include "absl/base/internal/spinlock.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
#if ABSL_THREAD_IDENTITY_MODE != ABSL_THREAD_IDENTITY_MODE_USE_CPP11
|
||||
namespace {
|
||||
// Used to co-ordinate one-time creation of our pthread_key
|
||||
absl::once_flag init_thread_identity_key_once;
|
||||
pthread_key_t thread_identity_pthread_key;
|
||||
std::atomic<bool> pthread_key_initialized(false);
|
||||
|
||||
void AllocateThreadIdentityKey(ThreadIdentityReclaimerFunction reclaimer) {
|
||||
pthread_key_create(&thread_identity_pthread_key, reclaimer);
|
||||
pthread_key_initialized.store(true, std::memory_order_release);
|
||||
}
|
||||
} // namespace
|
||||
#endif
|
||||
|
||||
#if ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_TLS || \
|
||||
ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_CPP11
|
||||
// The actual TLS storage for a thread's currently associated ThreadIdentity.
|
||||
// This is referenced by inline accessors in the header.
|
||||
// "protected" visibility ensures that if multiple instances of Abseil code
|
||||
// exist within a process (via dlopen() or similar), references to
|
||||
// thread_identity_ptr from each instance of the code will refer to
|
||||
// *different* instances of this ptr.
|
||||
#ifdef __GNUC__
|
||||
__attribute__((visibility("protected")))
|
||||
#endif // __GNUC__
|
||||
#if ABSL_PER_THREAD_TLS
|
||||
// Prefer __thread to thread_local as benchmarks indicate it is a bit faster.
|
||||
ABSL_PER_THREAD_TLS_KEYWORD ThreadIdentity* thread_identity_ptr = nullptr;
|
||||
#elif defined(ABSL_HAVE_THREAD_LOCAL)
|
||||
thread_local ThreadIdentity* thread_identity_ptr = nullptr;
|
||||
#endif // ABSL_PER_THREAD_TLS
|
||||
#endif // TLS or CPP11
|
||||
|
||||
void SetCurrentThreadIdentity(
|
||||
ThreadIdentity* identity, ThreadIdentityReclaimerFunction reclaimer) {
|
||||
assert(CurrentThreadIdentityIfPresent() == nullptr);
|
||||
// Associate our destructor.
|
||||
// NOTE: This call to pthread_setspecific is currently the only immovable
|
||||
// barrier to CurrentThreadIdentity() always being async signal safe.
|
||||
#if ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_POSIX_SETSPECIFIC
|
||||
// NOTE: Not async-safe. But can be open-coded.
|
||||
absl::call_once(init_thread_identity_key_once, AllocateThreadIdentityKey,
|
||||
reclaimer);
|
||||
|
||||
#if defined(__EMSCRIPTEN__) || defined(__MINGW32__)
|
||||
// Emscripten and MinGW pthread implementations does not support signals.
|
||||
// See https://kripken.github.io/emscripten-site/docs/porting/pthreads.html
|
||||
// for more information.
|
||||
pthread_setspecific(thread_identity_pthread_key,
|
||||
reinterpret_cast<void*>(identity));
|
||||
#else
|
||||
// We must mask signals around the call to setspecific as with current glibc,
|
||||
// a concurrent getspecific (needed for GetCurrentThreadIdentityIfPresent())
|
||||
// may zero our value.
|
||||
//
|
||||
// While not officially async-signal safe, getspecific within a signal handler
|
||||
// is otherwise OK.
|
||||
sigset_t all_signals;
|
||||
sigset_t curr_signals;
|
||||
sigfillset(&all_signals);
|
||||
pthread_sigmask(SIG_SETMASK, &all_signals, &curr_signals);
|
||||
pthread_setspecific(thread_identity_pthread_key,
|
||||
reinterpret_cast<void*>(identity));
|
||||
pthread_sigmask(SIG_SETMASK, &curr_signals, nullptr);
|
||||
#endif // !__EMSCRIPTEN__ && !__MINGW32__
|
||||
|
||||
#elif ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_TLS
|
||||
// NOTE: Not async-safe. But can be open-coded.
|
||||
absl::call_once(init_thread_identity_key_once, AllocateThreadIdentityKey,
|
||||
reclaimer);
|
||||
pthread_setspecific(thread_identity_pthread_key,
|
||||
reinterpret_cast<void*>(identity));
|
||||
thread_identity_ptr = identity;
|
||||
#elif ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_CPP11
|
||||
thread_local std::unique_ptr<ThreadIdentity, ThreadIdentityReclaimerFunction>
|
||||
holder(identity, reclaimer);
|
||||
thread_identity_ptr = identity;
|
||||
#else
|
||||
#error Unimplemented ABSL_THREAD_IDENTITY_MODE
|
||||
#endif
|
||||
}
|
||||
|
||||
#if ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_TLS || \
|
||||
ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_CPP11
|
||||
|
||||
// Please see the comment on `CurrentThreadIdentityIfPresent` in
|
||||
// thread_identity.h. Because DLLs cannot expose thread_local variables in
|
||||
// headers, we opt for the correct-but-slower option of placing the definition
|
||||
// of this function only in a translation unit inside DLL.
|
||||
#if defined(ABSL_BUILD_DLL) || defined(ABSL_CONSUME_DLL)
|
||||
ThreadIdentity* CurrentThreadIdentityIfPresent() { return thread_identity_ptr; }
|
||||
#endif
|
||||
#endif
|
||||
|
||||
void ClearCurrentThreadIdentity() {
|
||||
#if ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_TLS || \
|
||||
ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_CPP11
|
||||
thread_identity_ptr = nullptr;
|
||||
#elif ABSL_THREAD_IDENTITY_MODE == \
|
||||
ABSL_THREAD_IDENTITY_MODE_USE_POSIX_SETSPECIFIC
|
||||
// pthread_setspecific expected to clear value on destruction
|
||||
assert(CurrentThreadIdentityIfPresent() == nullptr);
|
||||
#endif
|
||||
}
|
||||
|
||||
#if ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_POSIX_SETSPECIFIC
|
||||
ThreadIdentity* CurrentThreadIdentityIfPresent() {
|
||||
bool initialized = pthread_key_initialized.load(std::memory_order_acquire);
|
||||
if (!initialized) {
|
||||
return nullptr;
|
||||
}
|
||||
return reinterpret_cast<ThreadIdentity*>(
|
||||
pthread_getspecific(thread_identity_pthread_key));
|
||||
}
|
||||
#endif
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
+259
@@ -0,0 +1,259 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Each active thread has an ThreadIdentity that may represent the thread in
|
||||
// various level interfaces. ThreadIdentity objects are never deallocated.
|
||||
// When a thread terminates, its ThreadIdentity object may be reused for a
|
||||
// thread created later.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_THREAD_IDENTITY_H_
|
||||
#define ABSL_BASE_INTERNAL_THREAD_IDENTITY_H_
|
||||
|
||||
#ifndef _WIN32
|
||||
#include <pthread.h>
|
||||
// Defines __GOOGLE_GRTE_VERSION__ (via glibc-specific features.h) when
|
||||
// supported.
|
||||
#include <unistd.h>
|
||||
#endif
|
||||
|
||||
#include <atomic>
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
#include "absl/base/internal/per_thread_tls.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
|
||||
struct SynchLocksHeld;
|
||||
struct SynchWaitParams;
|
||||
|
||||
namespace base_internal {
|
||||
|
||||
class SpinLock;
|
||||
struct ThreadIdentity;
|
||||
|
||||
// Used by the implementation of absl::Mutex and absl::CondVar.
|
||||
struct PerThreadSynch {
|
||||
// The internal representation of absl::Mutex and absl::CondVar rely
|
||||
// on the alignment of PerThreadSynch. Both store the address of the
|
||||
// PerThreadSynch in the high-order bits of their internal state,
|
||||
// which means the low kLowZeroBits of the address of PerThreadSynch
|
||||
// must be zero.
|
||||
static constexpr int kLowZeroBits = 8;
|
||||
static constexpr int kAlignment = 1 << kLowZeroBits;
|
||||
|
||||
// Returns the associated ThreadIdentity.
|
||||
// This can be implemented as a cast because we guarantee
|
||||
// PerThreadSynch is the first element of ThreadIdentity.
|
||||
ThreadIdentity* thread_identity() {
|
||||
return reinterpret_cast<ThreadIdentity*>(this);
|
||||
}
|
||||
|
||||
PerThreadSynch *next; // Circular waiter queue; initialized to 0.
|
||||
PerThreadSynch *skip; // If non-zero, all entries in Mutex queue
|
||||
// up to and including "skip" have same
|
||||
// condition as this, and will be woken later
|
||||
bool may_skip; // if false while on mutex queue, a mutex unlocker
|
||||
// is using this PerThreadSynch as a terminator. Its
|
||||
// skip field must not be filled in because the loop
|
||||
// might then skip over the terminator.
|
||||
|
||||
// The wait parameters of the current wait. waitp is null if the
|
||||
// thread is not waiting. Transitions from null to non-null must
|
||||
// occur before the enqueue commit point (state = kQueued in
|
||||
// Enqueue() and CondVarEnqueue()). Transitions from non-null to
|
||||
// null must occur after the wait is finished (state = kAvailable in
|
||||
// Mutex::Block() and CondVar::WaitCommon()). This field may be
|
||||
// changed only by the thread that describes this PerThreadSynch. A
|
||||
// special case is Fer(), which calls Enqueue() on another thread,
|
||||
// but with an identical SynchWaitParams pointer, thus leaving the
|
||||
// pointer unchanged.
|
||||
SynchWaitParams *waitp;
|
||||
|
||||
bool suppress_fatal_errors; // If true, try to proceed even in the face of
|
||||
// broken invariants. This is used within fatal
|
||||
// signal handlers to improve the chances of
|
||||
// debug logging information being output
|
||||
// successfully.
|
||||
|
||||
intptr_t readers; // Number of readers in mutex.
|
||||
int priority; // Priority of thread (updated every so often).
|
||||
|
||||
// When priority will next be read (cycles).
|
||||
int64_t next_priority_read_cycles;
|
||||
|
||||
// State values:
|
||||
// kAvailable: This PerThreadSynch is available.
|
||||
// kQueued: This PerThreadSynch is unavailable, it's currently queued on a
|
||||
// Mutex or CondVar waistlist.
|
||||
//
|
||||
// Transitions from kQueued to kAvailable require a release
|
||||
// barrier. This is needed as a waiter may use "state" to
|
||||
// independently observe that it's no longer queued.
|
||||
//
|
||||
// Transitions from kAvailable to kQueued require no barrier, they
|
||||
// are externally ordered by the Mutex.
|
||||
enum State {
|
||||
kAvailable,
|
||||
kQueued
|
||||
};
|
||||
std::atomic<State> state;
|
||||
|
||||
bool maybe_unlocking; // Valid at head of Mutex waiter queue;
|
||||
// true if UnlockSlow could be searching
|
||||
// for a waiter to wake. Used for an optimization
|
||||
// in Enqueue(). true is always a valid value.
|
||||
// Can be reset to false when the unlocker or any
|
||||
// writer releases the lock, or a reader fully releases
|
||||
// the lock. It may not be set to false by a reader
|
||||
// that decrements the count to non-zero.
|
||||
// protected by mutex spinlock
|
||||
|
||||
bool wake; // This thread is to be woken from a Mutex.
|
||||
|
||||
// If "x" is on a waiter list for a mutex, "x->cond_waiter" is true iff the
|
||||
// waiter is waiting on the mutex as part of a CV Wait or Mutex Await.
|
||||
//
|
||||
// The value of "x->cond_waiter" is meaningless if "x" is not on a
|
||||
// Mutex waiter list.
|
||||
bool cond_waiter;
|
||||
|
||||
// Locks held; used during deadlock detection.
|
||||
// Allocated in Synch_GetAllLocks() and freed in ReclaimThreadIdentity().
|
||||
SynchLocksHeld *all_locks;
|
||||
};
|
||||
|
||||
struct ThreadIdentity {
|
||||
// Must be the first member. The Mutex implementation requires that
|
||||
// the PerThreadSynch object associated with each thread is
|
||||
// PerThreadSynch::kAlignment aligned. We provide this alignment on
|
||||
// ThreadIdentity itself.
|
||||
PerThreadSynch per_thread_synch;
|
||||
|
||||
// Private: Reserved for absl::synchronization_internal::Waiter.
|
||||
struct WaiterState {
|
||||
char data[128];
|
||||
} waiter_state;
|
||||
|
||||
// Used by PerThreadSem::{Get,Set}ThreadBlockedCounter().
|
||||
std::atomic<int>* blocked_count_ptr;
|
||||
|
||||
// The following variables are mostly read/written just by the
|
||||
// thread itself. The only exception is that these are read by
|
||||
// a ticker thread as a hint.
|
||||
std::atomic<int> ticker; // Tick counter, incremented once per second.
|
||||
std::atomic<int> wait_start; // Ticker value when thread started waiting.
|
||||
std::atomic<bool> is_idle; // Has thread become idle yet?
|
||||
|
||||
ThreadIdentity* next;
|
||||
};
|
||||
|
||||
// Returns the ThreadIdentity object representing the calling thread; guaranteed
|
||||
// to be unique for its lifetime. The returned object will remain valid for the
|
||||
// program's lifetime; although it may be re-assigned to a subsequent thread.
|
||||
// If one does not exist, return nullptr instead.
|
||||
//
|
||||
// Does not malloc(*), and is async-signal safe.
|
||||
// [*] Technically pthread_setspecific() does malloc on first use; however this
|
||||
// is handled internally within tcmalloc's initialization already.
|
||||
//
|
||||
// New ThreadIdentity objects can be constructed and associated with a thread
|
||||
// by calling GetOrCreateCurrentThreadIdentity() in per-thread-sem.h.
|
||||
ThreadIdentity* CurrentThreadIdentityIfPresent();
|
||||
|
||||
using ThreadIdentityReclaimerFunction = void (*)(void*);
|
||||
|
||||
// Sets the current thread identity to the given value. 'reclaimer' is a
|
||||
// pointer to the global function for cleaning up instances on thread
|
||||
// destruction.
|
||||
void SetCurrentThreadIdentity(ThreadIdentity* identity,
|
||||
ThreadIdentityReclaimerFunction reclaimer);
|
||||
|
||||
// Removes the currently associated ThreadIdentity from the running thread.
|
||||
// This must be called from inside the ThreadIdentityReclaimerFunction, and only
|
||||
// from that function.
|
||||
void ClearCurrentThreadIdentity();
|
||||
|
||||
// May be chosen at compile time via: -DABSL_FORCE_THREAD_IDENTITY_MODE=<mode
|
||||
// index>
|
||||
#ifdef ABSL_THREAD_IDENTITY_MODE_USE_POSIX_SETSPECIFIC
|
||||
#error ABSL_THREAD_IDENTITY_MODE_USE_POSIX_SETSPECIFIC cannot be direcly set
|
||||
#else
|
||||
#define ABSL_THREAD_IDENTITY_MODE_USE_POSIX_SETSPECIFIC 0
|
||||
#endif
|
||||
|
||||
#ifdef ABSL_THREAD_IDENTITY_MODE_USE_TLS
|
||||
#error ABSL_THREAD_IDENTITY_MODE_USE_TLS cannot be direcly set
|
||||
#else
|
||||
#define ABSL_THREAD_IDENTITY_MODE_USE_TLS 1
|
||||
#endif
|
||||
|
||||
#ifdef ABSL_THREAD_IDENTITY_MODE_USE_CPP11
|
||||
#error ABSL_THREAD_IDENTITY_MODE_USE_CPP11 cannot be direcly set
|
||||
#else
|
||||
#define ABSL_THREAD_IDENTITY_MODE_USE_CPP11 2
|
||||
#endif
|
||||
|
||||
#ifdef ABSL_THREAD_IDENTITY_MODE
|
||||
#error ABSL_THREAD_IDENTITY_MODE cannot be direcly set
|
||||
#elif defined(ABSL_FORCE_THREAD_IDENTITY_MODE)
|
||||
#define ABSL_THREAD_IDENTITY_MODE ABSL_FORCE_THREAD_IDENTITY_MODE
|
||||
#elif defined(_WIN32) && !defined(__MINGW32__)
|
||||
#define ABSL_THREAD_IDENTITY_MODE ABSL_THREAD_IDENTITY_MODE_USE_CPP11
|
||||
#elif ABSL_PER_THREAD_TLS && defined(__GOOGLE_GRTE_VERSION__) && \
|
||||
(__GOOGLE_GRTE_VERSION__ >= 20140228L)
|
||||
// Support for async-safe TLS was specifically added in GRTEv4. It's not
|
||||
// present in the upstream eglibc.
|
||||
// Note: Current default for production systems.
|
||||
#define ABSL_THREAD_IDENTITY_MODE ABSL_THREAD_IDENTITY_MODE_USE_TLS
|
||||
#else
|
||||
#define ABSL_THREAD_IDENTITY_MODE \
|
||||
ABSL_THREAD_IDENTITY_MODE_USE_POSIX_SETSPECIFIC
|
||||
#endif
|
||||
|
||||
#if ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_TLS || \
|
||||
ABSL_THREAD_IDENTITY_MODE == ABSL_THREAD_IDENTITY_MODE_USE_CPP11
|
||||
|
||||
#if ABSL_PER_THREAD_TLS
|
||||
ABSL_CONST_INIT extern ABSL_PER_THREAD_TLS_KEYWORD ThreadIdentity*
|
||||
thread_identity_ptr;
|
||||
#elif defined(ABSL_HAVE_THREAD_LOCAL)
|
||||
ABSL_CONST_INIT extern thread_local ThreadIdentity* thread_identity_ptr;
|
||||
#else
|
||||
#error Thread-local storage not detected on this platform
|
||||
#endif
|
||||
|
||||
// thread_local variables cannot be in headers exposed by DLLs. However, it is
|
||||
// important for performance reasons in general that
|
||||
// `CurrentThreadIdentityIfPresent` be inlined. This is not possible across a
|
||||
// DLL boundary so, with DLLs, we opt to have the function not be inlined. Note
|
||||
// that `CurrentThreadIdentityIfPresent` is declared above so we can exclude
|
||||
// this entire inline definition when compiling as a DLL.
|
||||
#if !defined(ABSL_BUILD_DLL) && !defined(ABSL_CONSUME_DLL)
|
||||
inline ThreadIdentity* CurrentThreadIdentityIfPresent() {
|
||||
return thread_identity_ptr;
|
||||
}
|
||||
#endif
|
||||
|
||||
#elif ABSL_THREAD_IDENTITY_MODE != \
|
||||
ABSL_THREAD_IDENTITY_MODE_USE_POSIX_SETSPECIFIC
|
||||
#error Unknown ABSL_THREAD_IDENTITY_MODE
|
||||
#endif
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_THREAD_IDENTITY_H_
|
||||
+108
@@ -0,0 +1,108 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/base/internal/throw_delegate.h"
|
||||
|
||||
#include <cstdlib>
|
||||
#include <functional>
|
||||
#include <new>
|
||||
#include <stdexcept>
|
||||
#include "absl/base/config.h"
|
||||
#include "absl/base/internal/raw_logging.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
namespace {
|
||||
template <typename T>
|
||||
[[noreturn]] void Throw(const T& error) {
|
||||
#ifdef ABSL_HAVE_EXCEPTIONS
|
||||
throw error;
|
||||
#else
|
||||
ABSL_RAW_LOG(FATAL, "%s", error.what());
|
||||
std::abort();
|
||||
#endif
|
||||
}
|
||||
} // namespace
|
||||
|
||||
void ThrowStdLogicError(const std::string& what_arg) {
|
||||
Throw(std::logic_error(what_arg));
|
||||
}
|
||||
void ThrowStdLogicError(const char* what_arg) {
|
||||
Throw(std::logic_error(what_arg));
|
||||
}
|
||||
void ThrowStdInvalidArgument(const std::string& what_arg) {
|
||||
Throw(std::invalid_argument(what_arg));
|
||||
}
|
||||
void ThrowStdInvalidArgument(const char* what_arg) {
|
||||
Throw(std::invalid_argument(what_arg));
|
||||
}
|
||||
|
||||
void ThrowStdDomainError(const std::string& what_arg) {
|
||||
Throw(std::domain_error(what_arg));
|
||||
}
|
||||
void ThrowStdDomainError(const char* what_arg) {
|
||||
Throw(std::domain_error(what_arg));
|
||||
}
|
||||
|
||||
void ThrowStdLengthError(const std::string& what_arg) {
|
||||
Throw(std::length_error(what_arg));
|
||||
}
|
||||
void ThrowStdLengthError(const char* what_arg) {
|
||||
Throw(std::length_error(what_arg));
|
||||
}
|
||||
|
||||
void ThrowStdOutOfRange(const std::string& what_arg) {
|
||||
Throw(std::out_of_range(what_arg));
|
||||
}
|
||||
void ThrowStdOutOfRange(const char* what_arg) {
|
||||
Throw(std::out_of_range(what_arg));
|
||||
}
|
||||
|
||||
void ThrowStdRuntimeError(const std::string& what_arg) {
|
||||
Throw(std::runtime_error(what_arg));
|
||||
}
|
||||
void ThrowStdRuntimeError(const char* what_arg) {
|
||||
Throw(std::runtime_error(what_arg));
|
||||
}
|
||||
|
||||
void ThrowStdRangeError(const std::string& what_arg) {
|
||||
Throw(std::range_error(what_arg));
|
||||
}
|
||||
void ThrowStdRangeError(const char* what_arg) {
|
||||
Throw(std::range_error(what_arg));
|
||||
}
|
||||
|
||||
void ThrowStdOverflowError(const std::string& what_arg) {
|
||||
Throw(std::overflow_error(what_arg));
|
||||
}
|
||||
void ThrowStdOverflowError(const char* what_arg) {
|
||||
Throw(std::overflow_error(what_arg));
|
||||
}
|
||||
|
||||
void ThrowStdUnderflowError(const std::string& what_arg) {
|
||||
Throw(std::underflow_error(what_arg));
|
||||
}
|
||||
void ThrowStdUnderflowError(const char* what_arg) {
|
||||
Throw(std::underflow_error(what_arg));
|
||||
}
|
||||
|
||||
void ThrowStdBadFunctionCall() { Throw(std::bad_function_call()); }
|
||||
|
||||
void ThrowStdBadAlloc() { Throw(std::bad_alloc()); }
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
+75
@@ -0,0 +1,75 @@
|
||||
//
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_THROW_DELEGATE_H_
|
||||
#define ABSL_BASE_INTERNAL_THROW_DELEGATE_H_
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// Helper functions that allow throwing exceptions consistently from anywhere.
|
||||
// The main use case is for header-based libraries (eg templates), as they will
|
||||
// be built by many different targets with their own compiler options.
|
||||
// In particular, this will allow a safe way to throw exceptions even if the
|
||||
// caller is compiled with -fno-exceptions. This is intended for implementing
|
||||
// things like map<>::at(), which the standard documents as throwing an
|
||||
// exception on error.
|
||||
//
|
||||
// Using other techniques like #if tricks could lead to ODR violations.
|
||||
//
|
||||
// You shouldn't use it unless you're writing code that you know will be built
|
||||
// both with and without exceptions and you need to conform to an interface
|
||||
// that uses exceptions.
|
||||
|
||||
[[noreturn]] void ThrowStdLogicError(const std::string& what_arg);
|
||||
[[noreturn]] void ThrowStdLogicError(const char* what_arg);
|
||||
[[noreturn]] void ThrowStdInvalidArgument(const std::string& what_arg);
|
||||
[[noreturn]] void ThrowStdInvalidArgument(const char* what_arg);
|
||||
[[noreturn]] void ThrowStdDomainError(const std::string& what_arg);
|
||||
[[noreturn]] void ThrowStdDomainError(const char* what_arg);
|
||||
[[noreturn]] void ThrowStdLengthError(const std::string& what_arg);
|
||||
[[noreturn]] void ThrowStdLengthError(const char* what_arg);
|
||||
[[noreturn]] void ThrowStdOutOfRange(const std::string& what_arg);
|
||||
[[noreturn]] void ThrowStdOutOfRange(const char* what_arg);
|
||||
[[noreturn]] void ThrowStdRuntimeError(const std::string& what_arg);
|
||||
[[noreturn]] void ThrowStdRuntimeError(const char* what_arg);
|
||||
[[noreturn]] void ThrowStdRangeError(const std::string& what_arg);
|
||||
[[noreturn]] void ThrowStdRangeError(const char* what_arg);
|
||||
[[noreturn]] void ThrowStdOverflowError(const std::string& what_arg);
|
||||
[[noreturn]] void ThrowStdOverflowError(const char* what_arg);
|
||||
[[noreturn]] void ThrowStdUnderflowError(const std::string& what_arg);
|
||||
[[noreturn]] void ThrowStdUnderflowError(const char* what_arg);
|
||||
|
||||
[[noreturn]] void ThrowStdBadFunctionCall();
|
||||
[[noreturn]] void ThrowStdBadAlloc();
|
||||
|
||||
// ThrowStdBadArrayNewLength() cannot be consistently supported because
|
||||
// std::bad_array_new_length is missing in libstdc++ until 4.9.0.
|
||||
// https://gcc.gnu.org/onlinedocs/gcc-4.8.3/libstdc++/api/a01379_source.html
|
||||
// https://gcc.gnu.org/onlinedocs/gcc-4.9.0/libstdc++/api/a01327_source.html
|
||||
// libcxx (as of 3.2) and msvc (as of 2015) both have it.
|
||||
// [[noreturn]] void ThrowStdBadArrayNewLength();
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_THROW_DELEGATE_H_
|
||||
@@ -0,0 +1,66 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// This file is intended solely for spinlock.h.
|
||||
// It provides ThreadSanitizer annotations for custom mutexes.
|
||||
// See <sanitizer/tsan_interface.h> for meaning of these annotations.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_TSAN_MUTEX_INTERFACE_H_
|
||||
#define ABSL_BASE_INTERNAL_TSAN_MUTEX_INTERFACE_H_
|
||||
|
||||
// ABSL_INTERNAL_HAVE_TSAN_INTERFACE
|
||||
// Macro intended only for internal use.
|
||||
//
|
||||
// Checks whether LLVM Thread Sanitizer interfaces are available.
|
||||
// First made available in LLVM 5.0 (Sep 2017).
|
||||
#ifdef ABSL_INTERNAL_HAVE_TSAN_INTERFACE
|
||||
#error "ABSL_INTERNAL_HAVE_TSAN_INTERFACE cannot be directly set."
|
||||
#endif
|
||||
|
||||
#if defined(THREAD_SANITIZER) && defined(__has_include)
|
||||
#if __has_include(<sanitizer/tsan_interface.h>)
|
||||
#define ABSL_INTERNAL_HAVE_TSAN_INTERFACE 1
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#ifdef ABSL_INTERNAL_HAVE_TSAN_INTERFACE
|
||||
#include <sanitizer/tsan_interface.h>
|
||||
|
||||
#define ABSL_TSAN_MUTEX_CREATE __tsan_mutex_create
|
||||
#define ABSL_TSAN_MUTEX_DESTROY __tsan_mutex_destroy
|
||||
#define ABSL_TSAN_MUTEX_PRE_LOCK __tsan_mutex_pre_lock
|
||||
#define ABSL_TSAN_MUTEX_POST_LOCK __tsan_mutex_post_lock
|
||||
#define ABSL_TSAN_MUTEX_PRE_UNLOCK __tsan_mutex_pre_unlock
|
||||
#define ABSL_TSAN_MUTEX_POST_UNLOCK __tsan_mutex_post_unlock
|
||||
#define ABSL_TSAN_MUTEX_PRE_SIGNAL __tsan_mutex_pre_signal
|
||||
#define ABSL_TSAN_MUTEX_POST_SIGNAL __tsan_mutex_post_signal
|
||||
#define ABSL_TSAN_MUTEX_PRE_DIVERT __tsan_mutex_pre_divert
|
||||
#define ABSL_TSAN_MUTEX_POST_DIVERT __tsan_mutex_post_divert
|
||||
|
||||
#else
|
||||
|
||||
#define ABSL_TSAN_MUTEX_CREATE(...)
|
||||
#define ABSL_TSAN_MUTEX_DESTROY(...)
|
||||
#define ABSL_TSAN_MUTEX_PRE_LOCK(...)
|
||||
#define ABSL_TSAN_MUTEX_POST_LOCK(...)
|
||||
#define ABSL_TSAN_MUTEX_PRE_UNLOCK(...)
|
||||
#define ABSL_TSAN_MUTEX_POST_UNLOCK(...)
|
||||
#define ABSL_TSAN_MUTEX_PRE_SIGNAL(...)
|
||||
#define ABSL_TSAN_MUTEX_POST_SIGNAL(...)
|
||||
#define ABSL_TSAN_MUTEX_PRE_DIVERT(...)
|
||||
#define ABSL_TSAN_MUTEX_POST_DIVERT(...)
|
||||
|
||||
#endif
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_TSAN_MUTEX_INTERFACE_H_
|
||||
+158
@@ -0,0 +1,158 @@
|
||||
//
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_UNALIGNED_ACCESS_H_
|
||||
#define ABSL_BASE_INTERNAL_UNALIGNED_ACCESS_H_
|
||||
|
||||
#include <string.h>
|
||||
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/config.h"
|
||||
|
||||
// unaligned APIs
|
||||
|
||||
// Portable handling of unaligned loads, stores, and copies.
|
||||
|
||||
// The unaligned API is C++ only. The declarations use C++ features
|
||||
// (namespaces, inline) which are absent or incompatible in C.
|
||||
#if defined(__cplusplus)
|
||||
|
||||
#if defined(ADDRESS_SANITIZER) || defined(THREAD_SANITIZER) ||\
|
||||
defined(MEMORY_SANITIZER)
|
||||
// Consider we have an unaligned load/store of 4 bytes from address 0x...05.
|
||||
// AddressSanitizer will treat it as a 3-byte access to the range 05:07 and
|
||||
// will miss a bug if 08 is the first unaddressable byte.
|
||||
// ThreadSanitizer will also treat this as a 3-byte access to 05:07 and will
|
||||
// miss a race between this access and some other accesses to 08.
|
||||
// MemorySanitizer will correctly propagate the shadow on unaligned stores
|
||||
// and correctly report bugs on unaligned loads, but it may not properly
|
||||
// update and report the origin of the uninitialized memory.
|
||||
// For all three tools, replacing an unaligned access with a tool-specific
|
||||
// callback solves the problem.
|
||||
|
||||
// Make sure uint16_t/uint32_t/uint64_t are defined.
|
||||
#include <stdint.h>
|
||||
|
||||
extern "C" {
|
||||
uint16_t __sanitizer_unaligned_load16(const void *p);
|
||||
uint32_t __sanitizer_unaligned_load32(const void *p);
|
||||
uint64_t __sanitizer_unaligned_load64(const void *p);
|
||||
void __sanitizer_unaligned_store16(void *p, uint16_t v);
|
||||
void __sanitizer_unaligned_store32(void *p, uint32_t v);
|
||||
void __sanitizer_unaligned_store64(void *p, uint64_t v);
|
||||
} // extern "C"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
inline uint16_t UnalignedLoad16(const void *p) {
|
||||
return __sanitizer_unaligned_load16(p);
|
||||
}
|
||||
|
||||
inline uint32_t UnalignedLoad32(const void *p) {
|
||||
return __sanitizer_unaligned_load32(p);
|
||||
}
|
||||
|
||||
inline uint64_t UnalignedLoad64(const void *p) {
|
||||
return __sanitizer_unaligned_load64(p);
|
||||
}
|
||||
|
||||
inline void UnalignedStore16(void *p, uint16_t v) {
|
||||
__sanitizer_unaligned_store16(p, v);
|
||||
}
|
||||
|
||||
inline void UnalignedStore32(void *p, uint32_t v) {
|
||||
__sanitizer_unaligned_store32(p, v);
|
||||
}
|
||||
|
||||
inline void UnalignedStore64(void *p, uint64_t v) {
|
||||
__sanitizer_unaligned_store64(p, v);
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#define ABSL_INTERNAL_UNALIGNED_LOAD16(_p) \
|
||||
(absl::base_internal::UnalignedLoad16(_p))
|
||||
#define ABSL_INTERNAL_UNALIGNED_LOAD32(_p) \
|
||||
(absl::base_internal::UnalignedLoad32(_p))
|
||||
#define ABSL_INTERNAL_UNALIGNED_LOAD64(_p) \
|
||||
(absl::base_internal::UnalignedLoad64(_p))
|
||||
|
||||
#define ABSL_INTERNAL_UNALIGNED_STORE16(_p, _val) \
|
||||
(absl::base_internal::UnalignedStore16(_p, _val))
|
||||
#define ABSL_INTERNAL_UNALIGNED_STORE32(_p, _val) \
|
||||
(absl::base_internal::UnalignedStore32(_p, _val))
|
||||
#define ABSL_INTERNAL_UNALIGNED_STORE64(_p, _val) \
|
||||
(absl::base_internal::UnalignedStore64(_p, _val))
|
||||
|
||||
#else
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
inline uint16_t UnalignedLoad16(const void *p) {
|
||||
uint16_t t;
|
||||
memcpy(&t, p, sizeof t);
|
||||
return t;
|
||||
}
|
||||
|
||||
inline uint32_t UnalignedLoad32(const void *p) {
|
||||
uint32_t t;
|
||||
memcpy(&t, p, sizeof t);
|
||||
return t;
|
||||
}
|
||||
|
||||
inline uint64_t UnalignedLoad64(const void *p) {
|
||||
uint64_t t;
|
||||
memcpy(&t, p, sizeof t);
|
||||
return t;
|
||||
}
|
||||
|
||||
inline void UnalignedStore16(void *p, uint16_t v) { memcpy(p, &v, sizeof v); }
|
||||
|
||||
inline void UnalignedStore32(void *p, uint32_t v) { memcpy(p, &v, sizeof v); }
|
||||
|
||||
inline void UnalignedStore64(void *p, uint64_t v) { memcpy(p, &v, sizeof v); }
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#define ABSL_INTERNAL_UNALIGNED_LOAD16(_p) \
|
||||
(absl::base_internal::UnalignedLoad16(_p))
|
||||
#define ABSL_INTERNAL_UNALIGNED_LOAD32(_p) \
|
||||
(absl::base_internal::UnalignedLoad32(_p))
|
||||
#define ABSL_INTERNAL_UNALIGNED_LOAD64(_p) \
|
||||
(absl::base_internal::UnalignedLoad64(_p))
|
||||
|
||||
#define ABSL_INTERNAL_UNALIGNED_STORE16(_p, _val) \
|
||||
(absl::base_internal::UnalignedStore16(_p, _val))
|
||||
#define ABSL_INTERNAL_UNALIGNED_STORE32(_p, _val) \
|
||||
(absl::base_internal::UnalignedStore32(_p, _val))
|
||||
#define ABSL_INTERNAL_UNALIGNED_STORE64(_p, _val) \
|
||||
(absl::base_internal::UnalignedStore64(_p, _val))
|
||||
|
||||
#endif
|
||||
|
||||
#endif // defined(__cplusplus), end of unaligned API
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_UNALIGNED_ACCESS_H_
|
||||
+140
@@ -0,0 +1,140 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/base/internal/unscaledcycleclock.h"
|
||||
|
||||
#if ABSL_USE_UNSCALED_CYCLECLOCK
|
||||
|
||||
#if defined(_WIN32)
|
||||
#include <intrin.h>
|
||||
#endif
|
||||
|
||||
#if defined(__powerpc__) || defined(__ppc__)
|
||||
#ifdef __GLIBC__
|
||||
#include <sys/platform/ppc.h>
|
||||
#elif defined(__FreeBSD__)
|
||||
#include <sys/sysctl.h>
|
||||
#include <sys/types.h>
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#include "absl/base/internal/sysinfo.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
#if defined(__i386__)
|
||||
|
||||
int64_t UnscaledCycleClock::Now() {
|
||||
int64_t ret;
|
||||
__asm__ volatile("rdtsc" : "=A"(ret));
|
||||
return ret;
|
||||
}
|
||||
|
||||
double UnscaledCycleClock::Frequency() {
|
||||
return base_internal::NominalCPUFrequency();
|
||||
}
|
||||
|
||||
#elif defined(__x86_64__)
|
||||
|
||||
int64_t UnscaledCycleClock::Now() {
|
||||
uint64_t low, high;
|
||||
__asm__ volatile("rdtsc" : "=a"(low), "=d"(high));
|
||||
return (high << 32) | low;
|
||||
}
|
||||
|
||||
double UnscaledCycleClock::Frequency() {
|
||||
return base_internal::NominalCPUFrequency();
|
||||
}
|
||||
|
||||
#elif defined(__powerpc__) || defined(__ppc__)
|
||||
|
||||
int64_t UnscaledCycleClock::Now() {
|
||||
#ifdef __GLIBC__
|
||||
return __ppc_get_timebase();
|
||||
#else
|
||||
#ifdef __powerpc64__
|
||||
int64_t tbr;
|
||||
asm volatile("mfspr %0, 268" : "=r"(tbr));
|
||||
return tbr;
|
||||
#else
|
||||
int32_t tbu, tbl, tmp;
|
||||
asm volatile(
|
||||
"0:\n"
|
||||
"mftbu %[hi32]\n"
|
||||
"mftb %[lo32]\n"
|
||||
"mftbu %[tmp]\n"
|
||||
"cmpw %[tmp],%[hi32]\n"
|
||||
"bne 0b\n"
|
||||
: [ hi32 ] "=r"(tbu), [ lo32 ] "=r"(tbl), [ tmp ] "=r"(tmp));
|
||||
return (static_cast<int64_t>(tbu) << 32) | tbl;
|
||||
#endif
|
||||
#endif
|
||||
}
|
||||
|
||||
double UnscaledCycleClock::Frequency() {
|
||||
#ifdef __GLIBC__
|
||||
return __ppc_get_timebase_freq();
|
||||
#elif defined(__FreeBSD__)
|
||||
static once_flag init_timebase_frequency_once;
|
||||
static double timebase_frequency = 0.0;
|
||||
base_internal::LowLevelCallOnce(&init_timebase_frequency_once, [&]() {
|
||||
size_t length = sizeof(timebase_frequency);
|
||||
sysctlbyname("kern.timecounter.tc.timebase.frequency", &timebase_frequency,
|
||||
&length, nullptr, 0);
|
||||
});
|
||||
return timebase_frequency;
|
||||
#else
|
||||
#error Must implement UnscaledCycleClock::Frequency()
|
||||
#endif
|
||||
}
|
||||
|
||||
#elif defined(__aarch64__)
|
||||
|
||||
// System timer of ARMv8 runs at a different frequency than the CPU's.
|
||||
// The frequency is fixed, typically in the range 1-50MHz. It can be
|
||||
// read at CNTFRQ special register. We assume the OS has set up
|
||||
// the virtual timer properly.
|
||||
int64_t UnscaledCycleClock::Now() {
|
||||
int64_t virtual_timer_value;
|
||||
asm volatile("mrs %0, cntvct_el0" : "=r"(virtual_timer_value));
|
||||
return virtual_timer_value;
|
||||
}
|
||||
|
||||
double UnscaledCycleClock::Frequency() {
|
||||
uint64_t aarch64_timer_frequency;
|
||||
asm volatile("mrs %0, cntfrq_el0" : "=r"(aarch64_timer_frequency));
|
||||
return aarch64_timer_frequency;
|
||||
}
|
||||
|
||||
#elif defined(_M_IX86) || defined(_M_X64)
|
||||
|
||||
#pragma intrinsic(__rdtsc)
|
||||
|
||||
int64_t UnscaledCycleClock::Now() {
|
||||
return __rdtsc();
|
||||
}
|
||||
|
||||
double UnscaledCycleClock::Frequency() {
|
||||
return base_internal::NominalCPUFrequency();
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_USE_UNSCALED_CYCLECLOCK
|
||||
+124
@@ -0,0 +1,124 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// UnscaledCycleClock
|
||||
// An UnscaledCycleClock yields the value and frequency of a cycle counter
|
||||
// that increments at a rate that is approximately constant.
|
||||
// This class is for internal / whitelisted use only, you should consider
|
||||
// using CycleClock instead.
|
||||
//
|
||||
// Notes:
|
||||
// The cycle counter frequency is not necessarily the core clock frequency.
|
||||
// That is, CycleCounter cycles are not necessarily "CPU cycles".
|
||||
//
|
||||
// An arbitrary offset may have been added to the counter at power on.
|
||||
//
|
||||
// On some platforms, the rate and offset of the counter may differ
|
||||
// slightly when read from different CPUs of a multiprocessor. Usually,
|
||||
// we try to ensure that the operating system adjusts values periodically
|
||||
// so that values agree approximately. If you need stronger guarantees,
|
||||
// consider using alternate interfaces.
|
||||
//
|
||||
// The CPU is not required to maintain the ordering of a cycle counter read
|
||||
// with respect to surrounding instructions.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_UNSCALEDCYCLECLOCK_H_
|
||||
#define ABSL_BASE_INTERNAL_UNSCALEDCYCLECLOCK_H_
|
||||
|
||||
#include <cstdint>
|
||||
|
||||
#if defined(__APPLE__)
|
||||
#include <TargetConditionals.h>
|
||||
#endif
|
||||
|
||||
#include "absl/base/port.h"
|
||||
|
||||
// The following platforms have an implementation of a hardware counter.
|
||||
#if defined(__i386__) || defined(__x86_64__) || defined(__aarch64__) || \
|
||||
defined(__powerpc__) || defined(__ppc__) || \
|
||||
defined(_M_IX86) || defined(_M_X64)
|
||||
#define ABSL_HAVE_UNSCALED_CYCLECLOCK_IMPLEMENTATION 1
|
||||
#else
|
||||
#define ABSL_HAVE_UNSCALED_CYCLECLOCK_IMPLEMENTATION 0
|
||||
#endif
|
||||
|
||||
// The following platforms often disable access to the hardware
|
||||
// counter (through a sandbox) even if the underlying hardware has a
|
||||
// usable counter. The CycleTimer interface also requires a *scaled*
|
||||
// CycleClock that runs at atleast 1 MHz. We've found some Android
|
||||
// ARM64 devices where this is not the case, so we disable it by
|
||||
// default on Android ARM64.
|
||||
#if defined(__native_client__) || \
|
||||
(defined(TARGET_OS_IPHONE) && TARGET_OS_IPHONE) || \
|
||||
(defined(__ANDROID__) && defined(__aarch64__))
|
||||
#define ABSL_USE_UNSCALED_CYCLECLOCK_DEFAULT 0
|
||||
#else
|
||||
#define ABSL_USE_UNSCALED_CYCLECLOCK_DEFAULT 1
|
||||
#endif
|
||||
|
||||
// UnscaledCycleClock is an optional internal feature.
|
||||
// Use "#if ABSL_USE_UNSCALED_CYCLECLOCK" to test for its presence.
|
||||
// Can be overridden at compile-time via -DABSL_USE_UNSCALED_CYCLECLOCK=0|1
|
||||
#if !defined(ABSL_USE_UNSCALED_CYCLECLOCK)
|
||||
#define ABSL_USE_UNSCALED_CYCLECLOCK \
|
||||
(ABSL_HAVE_UNSCALED_CYCLECLOCK_IMPLEMENTATION && \
|
||||
ABSL_USE_UNSCALED_CYCLECLOCK_DEFAULT)
|
||||
#endif
|
||||
|
||||
#if ABSL_USE_UNSCALED_CYCLECLOCK
|
||||
|
||||
// This macro can be used to test if UnscaledCycleClock::Frequency()
|
||||
// is NominalCPUFrequency() on a particular platform.
|
||||
#if (defined(__i386__) || defined(__x86_64__) || \
|
||||
defined(_M_IX86) || defined(_M_X64))
|
||||
#define ABSL_INTERNAL_UNSCALED_CYCLECLOCK_FREQUENCY_IS_CPU_FREQUENCY
|
||||
#endif
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace time_internal {
|
||||
class UnscaledCycleClockWrapperForGetCurrentTime;
|
||||
} // namespace time_internal
|
||||
|
||||
namespace base_internal {
|
||||
class CycleClock;
|
||||
class UnscaledCycleClockWrapperForInitializeFrequency;
|
||||
|
||||
class UnscaledCycleClock {
|
||||
private:
|
||||
UnscaledCycleClock() = delete;
|
||||
|
||||
// Return the value of a cycle counter that counts at a rate that is
|
||||
// approximately constant.
|
||||
static int64_t Now();
|
||||
|
||||
// Return the how much UnscaledCycleClock::Now() increases per second.
|
||||
// This is not necessarily the core CPU clock frequency.
|
||||
// It may be the nominal value report by the kernel, rather than a measured
|
||||
// value.
|
||||
static double Frequency();
|
||||
|
||||
// Whitelisted friends.
|
||||
friend class base_internal::CycleClock;
|
||||
friend class time_internal::UnscaledCycleClockWrapperForGetCurrentTime;
|
||||
friend class base_internal::UnscaledCycleClockWrapperForInitializeFrequency;
|
||||
};
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_USE_UNSCALED_CYCLECLOCK
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_UNSCALEDCYCLECLOCK_H_
|
||||
Generated
+27
@@ -0,0 +1,27 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/base/log_severity.h"
|
||||
|
||||
#include <ostream>
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
|
||||
std::ostream& operator<<(std::ostream& os, absl::LogSeverity s) {
|
||||
if (s == absl::NormalizeLogSeverity(s)) return os << absl::LogSeverityName(s);
|
||||
return os << "absl::LogSeverity(" << static_cast<int>(s) << ")";
|
||||
}
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
Generated
+121
@@ -0,0 +1,121 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_BASE_INTERNAL_LOG_SEVERITY_H_
|
||||
#define ABSL_BASE_INTERNAL_LOG_SEVERITY_H_
|
||||
|
||||
#include <array>
|
||||
#include <ostream>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
|
||||
// absl::LogSeverity
|
||||
//
|
||||
// Four severity levels are defined. Logging APIs should terminate the program
|
||||
// when a message is logged at severity `kFatal`; the other levels have no
|
||||
// special semantics.
|
||||
//
|
||||
// Values other than the four defined levels (e.g. produced by `static_cast`)
|
||||
// are valid, but their semantics when passed to a function, macro, or flag
|
||||
// depend on the function, macro, or flag. The usual behavior is to normalize
|
||||
// such values to a defined severity level, however in some cases values other
|
||||
// than the defined levels are useful for comparison.
|
||||
//
|
||||
// Exmaple:
|
||||
//
|
||||
// // Effectively disables all logging:
|
||||
// SetMinLogLevel(static_cast<absl::LogSeverity>(100));
|
||||
//
|
||||
// Abseil flags may be defined with type `LogSeverity`. Dependency layering
|
||||
// constraints require that the `AbslParseFlag()` overload be declared and
|
||||
// defined in the flags library itself rather than here. The `AbslUnparseFlag()`
|
||||
// overload is defined there as well for consistency.
|
||||
//
|
||||
// absl::LogSeverity Flag String Representation
|
||||
//
|
||||
// An `absl::LogSeverity` has a string representation used for parsing
|
||||
// command-line flags based on the enumerator name (e.g. `kFatal`) or
|
||||
// its unprefixed name (without the `k`) in any case-insensitive form. (E.g.
|
||||
// "FATAL", "fatal" or "Fatal" are all valid.) Unparsing such flags produces an
|
||||
// unprefixed string representation in all caps (e.g. "FATAL") or an integer.
|
||||
//
|
||||
// Additionally, the parser accepts arbitrary integers (as if the type were
|
||||
// `int`).
|
||||
//
|
||||
// Examples:
|
||||
//
|
||||
// --my_log_level=kInfo
|
||||
// --my_log_level=INFO
|
||||
// --my_log_level=info
|
||||
// --my_log_level=0
|
||||
//
|
||||
// Unparsing a flag produces the same result as `absl::LogSeverityName()` for
|
||||
// the standard levels and a base-ten integer otherwise.
|
||||
enum class LogSeverity : int {
|
||||
kInfo = 0,
|
||||
kWarning = 1,
|
||||
kError = 2,
|
||||
kFatal = 3,
|
||||
};
|
||||
|
||||
// LogSeverities()
|
||||
//
|
||||
// Returns an iterable of all standard `absl::LogSeverity` values, ordered from
|
||||
// least to most severe.
|
||||
constexpr std::array<absl::LogSeverity, 4> LogSeverities() {
|
||||
return {{absl::LogSeverity::kInfo, absl::LogSeverity::kWarning,
|
||||
absl::LogSeverity::kError, absl::LogSeverity::kFatal}};
|
||||
}
|
||||
|
||||
// LogSeverityName()
|
||||
//
|
||||
// Returns the all-caps string representation (e.g. "INFO") of the specified
|
||||
// severity level if it is one of the standard levels and "UNKNOWN" otherwise.
|
||||
constexpr const char* LogSeverityName(absl::LogSeverity s) {
|
||||
return s == absl::LogSeverity::kInfo
|
||||
? "INFO"
|
||||
: s == absl::LogSeverity::kWarning
|
||||
? "WARNING"
|
||||
: s == absl::LogSeverity::kError
|
||||
? "ERROR"
|
||||
: s == absl::LogSeverity::kFatal ? "FATAL" : "UNKNOWN";
|
||||
}
|
||||
|
||||
// NormalizeLogSeverity()
|
||||
//
|
||||
// Values less than `kInfo` normalize to `kInfo`; values greater than `kFatal`
|
||||
// normalize to `kError` (**NOT** `kFatal`).
|
||||
constexpr absl::LogSeverity NormalizeLogSeverity(absl::LogSeverity s) {
|
||||
return s < absl::LogSeverity::kInfo
|
||||
? absl::LogSeverity::kInfo
|
||||
: s > absl::LogSeverity::kFatal ? absl::LogSeverity::kError : s;
|
||||
}
|
||||
constexpr absl::LogSeverity NormalizeLogSeverity(int s) {
|
||||
return absl::NormalizeLogSeverity(static_cast<absl::LogSeverity>(s));
|
||||
}
|
||||
|
||||
// operator<<
|
||||
//
|
||||
// The exact representation of a streamed `absl::LogSeverity` is deliberately
|
||||
// unspecified; do not rely on it.
|
||||
std::ostream& operator<<(std::ostream& os, absl::LogSeverity s);
|
||||
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_INTERNAL_LOG_SEVERITY_H_
|
||||
Generated
+220
@@ -0,0 +1,220 @@
|
||||
//
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: macros.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This header file defines the set of language macros used within Abseil code.
|
||||
// For the set of macros used to determine supported compilers and platforms,
|
||||
// see absl/base/config.h instead.
|
||||
//
|
||||
// This code is compiled directly on many platforms, including client
|
||||
// platforms like Windows, Mac, and embedded systems. Before making
|
||||
// any changes here, make sure that you're not breaking any platforms.
|
||||
|
||||
#ifndef ABSL_BASE_MACROS_H_
|
||||
#define ABSL_BASE_MACROS_H_
|
||||
|
||||
#include <cassert>
|
||||
#include <cstddef>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/optimization.h"
|
||||
#include "absl/base/port.h"
|
||||
|
||||
// ABSL_ARRAYSIZE()
|
||||
//
|
||||
// Returns the number of elements in an array as a compile-time constant, which
|
||||
// can be used in defining new arrays. If you use this macro on a pointer by
|
||||
// mistake, you will get a compile-time error.
|
||||
#define ABSL_ARRAYSIZE(array) \
|
||||
(sizeof(::absl::macros_internal::ArraySizeHelper(array)))
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace macros_internal {
|
||||
// Note: this internal template function declaration is used by ABSL_ARRAYSIZE.
|
||||
// The function doesn't need a definition, as we only use its type.
|
||||
template <typename T, size_t N>
|
||||
auto ArraySizeHelper(const T (&array)[N]) -> char (&)[N];
|
||||
} // namespace macros_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
// kLinkerInitialized
|
||||
//
|
||||
// An enum used only as a constructor argument to indicate that a variable has
|
||||
// static storage duration, and that the constructor should do nothing to its
|
||||
// state. Use of this macro indicates to the reader that it is legal to
|
||||
// declare a static instance of the class, provided the constructor is given
|
||||
// the absl::base_internal::kLinkerInitialized argument.
|
||||
//
|
||||
// Normally, it is unsafe to declare a static variable that has a constructor or
|
||||
// a destructor because invocation order is undefined. However, if the type can
|
||||
// be zero-initialized (which the loader does for static variables) into a valid
|
||||
// state and the type's destructor does not affect storage, then a constructor
|
||||
// for static initialization can be declared.
|
||||
//
|
||||
// Example:
|
||||
// // Declaration
|
||||
// explicit MyClass(absl::base_internal:LinkerInitialized x) {}
|
||||
//
|
||||
// // Invocation
|
||||
// static MyClass my_global(absl::base_internal::kLinkerInitialized);
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
enum LinkerInitialized {
|
||||
kLinkerInitialized = 0,
|
||||
};
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
// ABSL_FALLTHROUGH_INTENDED
|
||||
//
|
||||
// Annotates implicit fall-through between switch labels, allowing a case to
|
||||
// indicate intentional fallthrough and turn off warnings about any lack of a
|
||||
// `break` statement. The ABSL_FALLTHROUGH_INTENDED macro should be followed by
|
||||
// a semicolon and can be used in most places where `break` can, provided that
|
||||
// no statements exist between it and the next switch label.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// switch (x) {
|
||||
// case 40:
|
||||
// case 41:
|
||||
// if (truth_is_out_there) {
|
||||
// ++x;
|
||||
// ABSL_FALLTHROUGH_INTENDED; // Use instead of/along with annotations
|
||||
// // in comments
|
||||
// } else {
|
||||
// return x;
|
||||
// }
|
||||
// case 42:
|
||||
// ...
|
||||
//
|
||||
// Notes: when compiled with clang in C++11 mode, the ABSL_FALLTHROUGH_INTENDED
|
||||
// macro is expanded to the [[clang::fallthrough]] attribute, which is analysed
|
||||
// when performing switch labels fall-through diagnostic
|
||||
// (`-Wimplicit-fallthrough`). See clang documentation on language extensions
|
||||
// for details:
|
||||
// https://clang.llvm.org/docs/AttributeReference.html#fallthrough-clang-fallthrough
|
||||
//
|
||||
// When used with unsupported compilers, the ABSL_FALLTHROUGH_INTENDED macro
|
||||
// has no effect on diagnostics. In any case this macro has no effect on runtime
|
||||
// behavior and performance of code.
|
||||
#ifdef ABSL_FALLTHROUGH_INTENDED
|
||||
#error "ABSL_FALLTHROUGH_INTENDED should not be defined."
|
||||
#endif
|
||||
|
||||
// TODO(zhangxy): Use c++17 standard [[fallthrough]] macro, when supported.
|
||||
#if defined(__clang__) && defined(__has_warning)
|
||||
#if __has_feature(cxx_attributes) && __has_warning("-Wimplicit-fallthrough")
|
||||
#define ABSL_FALLTHROUGH_INTENDED [[clang::fallthrough]]
|
||||
#endif
|
||||
#elif defined(__GNUC__) && __GNUC__ >= 7
|
||||
#define ABSL_FALLTHROUGH_INTENDED [[gnu::fallthrough]]
|
||||
#endif
|
||||
|
||||
#ifndef ABSL_FALLTHROUGH_INTENDED
|
||||
#define ABSL_FALLTHROUGH_INTENDED \
|
||||
do { \
|
||||
} while (0)
|
||||
#endif
|
||||
|
||||
// ABSL_DEPRECATED()
|
||||
//
|
||||
// Marks a deprecated class, struct, enum, function, method and variable
|
||||
// declarations. The macro argument is used as a custom diagnostic message (e.g.
|
||||
// suggestion of a better alternative).
|
||||
//
|
||||
// Examples:
|
||||
//
|
||||
// class ABSL_DEPRECATED("Use Bar instead") Foo {...};
|
||||
//
|
||||
// ABSL_DEPRECATED("Use Baz() instead") void Bar() {...}
|
||||
//
|
||||
// template <typename T>
|
||||
// ABSL_DEPRECATED("Use DoThat() instead")
|
||||
// void DoThis();
|
||||
//
|
||||
// Every usage of a deprecated entity will trigger a warning when compiled with
|
||||
// clang's `-Wdeprecated-declarations` option. This option is turned off by
|
||||
// default, but the warnings will be reported by clang-tidy.
|
||||
#if defined(__clang__) && __cplusplus >= 201103L
|
||||
#define ABSL_DEPRECATED(message) __attribute__((deprecated(message)))
|
||||
#endif
|
||||
|
||||
#ifndef ABSL_DEPRECATED
|
||||
#define ABSL_DEPRECATED(message)
|
||||
#endif
|
||||
|
||||
// ABSL_BAD_CALL_IF()
|
||||
//
|
||||
// Used on a function overload to trap bad calls: any call that matches the
|
||||
// overload will cause a compile-time error. This macro uses a clang-specific
|
||||
// "enable_if" attribute, as described at
|
||||
// https://clang.llvm.org/docs/AttributeReference.html#enable-if
|
||||
//
|
||||
// Overloads which use this macro should be bracketed by
|
||||
// `#ifdef ABSL_BAD_CALL_IF`.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// int isdigit(int c);
|
||||
// #ifdef ABSL_BAD_CALL_IF
|
||||
// int isdigit(int c)
|
||||
// ABSL_BAD_CALL_IF(c <= -1 || c > 255,
|
||||
// "'c' must have the value of an unsigned char or EOF");
|
||||
// #endif // ABSL_BAD_CALL_IF
|
||||
#if ABSL_HAVE_ATTRIBUTE(enable_if)
|
||||
#define ABSL_BAD_CALL_IF(expr, msg) \
|
||||
__attribute__((enable_if(expr, "Bad call trap"), unavailable(msg)))
|
||||
#endif
|
||||
|
||||
// ABSL_ASSERT()
|
||||
//
|
||||
// In C++11, `assert` can't be used portably within constexpr functions.
|
||||
// ABSL_ASSERT functions as a runtime assert but works in C++11 constexpr
|
||||
// functions. Example:
|
||||
//
|
||||
// constexpr double Divide(double a, double b) {
|
||||
// return ABSL_ASSERT(b != 0), a / b;
|
||||
// }
|
||||
//
|
||||
// This macro is inspired by
|
||||
// https://akrzemi1.wordpress.com/2017/05/18/asserts-in-constexpr-functions/
|
||||
#if defined(NDEBUG)
|
||||
#define ABSL_ASSERT(expr) \
|
||||
(false ? static_cast<void>(expr) : static_cast<void>(0))
|
||||
#else
|
||||
#define ABSL_ASSERT(expr) \
|
||||
(ABSL_PREDICT_TRUE((expr)) ? static_cast<void>(0) \
|
||||
: [] { assert(false && #expr); }()) // NOLINT
|
||||
#endif
|
||||
|
||||
#ifdef ABSL_HAVE_EXCEPTIONS
|
||||
#define ABSL_INTERNAL_TRY try
|
||||
#define ABSL_INTERNAL_CATCH_ANY catch (...)
|
||||
#define ABSL_INTERNAL_RETHROW do { throw; } while (false)
|
||||
#else // ABSL_HAVE_EXCEPTIONS
|
||||
#define ABSL_INTERNAL_TRY if (true)
|
||||
#define ABSL_INTERNAL_CATCH_ANY else if (false)
|
||||
#define ABSL_INTERNAL_RETHROW do {} while (false)
|
||||
#endif // ABSL_HAVE_EXCEPTIONS
|
||||
|
||||
#endif // ABSL_BASE_MACROS_H_
|
||||
Generated
+181
@@ -0,0 +1,181 @@
|
||||
//
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: optimization.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This header file defines portable macros for performance optimization.
|
||||
|
||||
#ifndef ABSL_BASE_OPTIMIZATION_H_
|
||||
#define ABSL_BASE_OPTIMIZATION_H_
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
// ABSL_BLOCK_TAIL_CALL_OPTIMIZATION
|
||||
//
|
||||
// Instructs the compiler to avoid optimizing tail-call recursion. Use of this
|
||||
// macro is useful when you wish to preserve the existing function order within
|
||||
// a stack trace for logging, debugging, or profiling purposes.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// int f() {
|
||||
// int result = g();
|
||||
// ABSL_BLOCK_TAIL_CALL_OPTIMIZATION();
|
||||
// return result;
|
||||
// }
|
||||
#if defined(__pnacl__)
|
||||
#define ABSL_BLOCK_TAIL_CALL_OPTIMIZATION() if (volatile int x = 0) { (void)x; }
|
||||
#elif defined(__clang__)
|
||||
// Clang will not tail call given inline volatile assembly.
|
||||
#define ABSL_BLOCK_TAIL_CALL_OPTIMIZATION() __asm__ __volatile__("")
|
||||
#elif defined(__GNUC__)
|
||||
// GCC will not tail call given inline volatile assembly.
|
||||
#define ABSL_BLOCK_TAIL_CALL_OPTIMIZATION() __asm__ __volatile__("")
|
||||
#elif defined(_MSC_VER)
|
||||
#include <intrin.h>
|
||||
// The __nop() intrinsic blocks the optimisation.
|
||||
#define ABSL_BLOCK_TAIL_CALL_OPTIMIZATION() __nop()
|
||||
#else
|
||||
#define ABSL_BLOCK_TAIL_CALL_OPTIMIZATION() if (volatile int x = 0) { (void)x; }
|
||||
#endif
|
||||
|
||||
// ABSL_CACHELINE_SIZE
|
||||
//
|
||||
// Explicitly defines the size of the L1 cache for purposes of alignment.
|
||||
// Setting the cacheline size allows you to specify that certain objects be
|
||||
// aligned on a cacheline boundary with `ABSL_CACHELINE_ALIGNED` declarations.
|
||||
// (See below.)
|
||||
//
|
||||
// NOTE: this macro should be replaced with the following C++17 features, when
|
||||
// those are generally available:
|
||||
//
|
||||
// * `std::hardware_constructive_interference_size`
|
||||
// * `std::hardware_destructive_interference_size`
|
||||
//
|
||||
// See http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0154r1.html
|
||||
// for more information.
|
||||
#if defined(__GNUC__)
|
||||
// Cache line alignment
|
||||
#if defined(__i386__) || defined(__x86_64__)
|
||||
#define ABSL_CACHELINE_SIZE 64
|
||||
#elif defined(__powerpc64__)
|
||||
#define ABSL_CACHELINE_SIZE 128
|
||||
#elif defined(__aarch64__)
|
||||
// We would need to read special register ctr_el0 to find out L1 dcache size.
|
||||
// This value is a good estimate based on a real aarch64 machine.
|
||||
#define ABSL_CACHELINE_SIZE 64
|
||||
#elif defined(__arm__)
|
||||
// Cache line sizes for ARM: These values are not strictly correct since
|
||||
// cache line sizes depend on implementations, not architectures. There
|
||||
// are even implementations with cache line sizes configurable at boot
|
||||
// time.
|
||||
#if defined(__ARM_ARCH_5T__)
|
||||
#define ABSL_CACHELINE_SIZE 32
|
||||
#elif defined(__ARM_ARCH_7A__)
|
||||
#define ABSL_CACHELINE_SIZE 64
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#ifndef ABSL_CACHELINE_SIZE
|
||||
// A reasonable default guess. Note that overestimates tend to waste more
|
||||
// space, while underestimates tend to waste more time.
|
||||
#define ABSL_CACHELINE_SIZE 64
|
||||
#endif
|
||||
|
||||
// ABSL_CACHELINE_ALIGNED
|
||||
//
|
||||
// Indicates that the declared object be cache aligned using
|
||||
// `ABSL_CACHELINE_SIZE` (see above). Cacheline aligning objects allows you to
|
||||
// load a set of related objects in the L1 cache for performance improvements.
|
||||
// Cacheline aligning objects properly allows constructive memory sharing and
|
||||
// prevents destructive (or "false") memory sharing.
|
||||
//
|
||||
// NOTE: this macro should be replaced with usage of `alignas()` using
|
||||
// `std::hardware_constructive_interference_size` and/or
|
||||
// `std::hardware_destructive_interference_size` when available within C++17.
|
||||
//
|
||||
// See http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0154r1.html
|
||||
// for more information.
|
||||
//
|
||||
// On some compilers, `ABSL_CACHELINE_ALIGNED` expands to an `__attribute__`
|
||||
// or `__declspec` attribute. For compilers where this is not known to work,
|
||||
// the macro expands to nothing.
|
||||
//
|
||||
// No further guarantees are made here. The result of applying the macro
|
||||
// to variables and types is always implementation-defined.
|
||||
//
|
||||
// WARNING: It is easy to use this attribute incorrectly, even to the point
|
||||
// of causing bugs that are difficult to diagnose, crash, etc. It does not
|
||||
// of itself guarantee that objects are aligned to a cache line.
|
||||
//
|
||||
// NOTE: Some compilers are picky about the locations of annotations such as
|
||||
// this attribute, so prefer to put it at the beginning of your declaration.
|
||||
// For example,
|
||||
//
|
||||
// ABSL_CACHELINE_ALIGNED static Foo* foo = ...
|
||||
//
|
||||
// class ABSL_CACHELINE_ALIGNED Bar { ...
|
||||
//
|
||||
// Recommendations:
|
||||
//
|
||||
// 1) Consult compiler documentation; this comment is not kept in sync as
|
||||
// toolchains evolve.
|
||||
// 2) Verify your use has the intended effect. This often requires inspecting
|
||||
// the generated machine code.
|
||||
// 3) Prefer applying this attribute to individual variables. Avoid
|
||||
// applying it to types. This tends to localize the effect.
|
||||
#define ABSL_CACHELINE_ALIGNED __attribute__((aligned(ABSL_CACHELINE_SIZE)))
|
||||
#elif defined(_MSC_VER)
|
||||
#define ABSL_CACHELINE_SIZE 64
|
||||
#define ABSL_CACHELINE_ALIGNED __declspec(align(ABSL_CACHELINE_SIZE))
|
||||
#else
|
||||
#define ABSL_CACHELINE_SIZE 64
|
||||
#define ABSL_CACHELINE_ALIGNED
|
||||
#endif
|
||||
|
||||
// ABSL_PREDICT_TRUE, ABSL_PREDICT_FALSE
|
||||
//
|
||||
// Enables the compiler to prioritize compilation using static analysis for
|
||||
// likely paths within a boolean branch.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// if (ABSL_PREDICT_TRUE(expression)) {
|
||||
// return result; // Faster if more likely
|
||||
// } else {
|
||||
// return 0;
|
||||
// }
|
||||
//
|
||||
// Compilers can use the information that a certain branch is not likely to be
|
||||
// taken (for instance, a CHECK failure) to optimize for the common case in
|
||||
// the absence of better information (ie. compiling gcc with `-fprofile-arcs`).
|
||||
//
|
||||
// Recommendation: Modern CPUs dynamically predict branch execution paths,
|
||||
// typically with accuracy greater than 97%. As a result, annotating every
|
||||
// branch in a codebase is likely counterproductive; however, annotating
|
||||
// specific branches that are both hot and consistently mispredicted is likely
|
||||
// to yield performance improvements.
|
||||
#if ABSL_HAVE_BUILTIN(__builtin_expect) || \
|
||||
(defined(__GNUC__) && !defined(__clang__))
|
||||
#define ABSL_PREDICT_FALSE(x) (__builtin_expect(x, 0))
|
||||
#define ABSL_PREDICT_TRUE(x) (__builtin_expect(false || (x), true))
|
||||
#else
|
||||
#define ABSL_PREDICT_FALSE(x) (x)
|
||||
#define ABSL_PREDICT_TRUE(x) (x)
|
||||
#endif
|
||||
|
||||
#endif // ABSL_BASE_OPTIMIZATION_H_
|
||||
Generated
+211
@@ -0,0 +1,211 @@
|
||||
#ifndef ABSL_BASE_OPTIONS_H_
|
||||
#define ABSL_BASE_OPTIONS_H_
|
||||
|
||||
// Copyright 2019 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: options.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This file contains Abseil configuration options for setting specific
|
||||
// implementations instead of letting Abseil determine which implementation to
|
||||
// use at compile-time. Setting these options may be useful for package or build
|
||||
// managers who wish to guarantee ABI stability within binary builds (which are
|
||||
// otherwise difficult to enforce).
|
||||
//
|
||||
// *** IMPORTANT NOTICE FOR PACKAGE MANAGERS: It is important that
|
||||
// maintainers of package managers who wish to package Abseil read and
|
||||
// understand this file! ***
|
||||
//
|
||||
// Abseil contains a number of possible configuration endpoints, based on
|
||||
// parameters such as the detected platform, language version, or command-line
|
||||
// flags used to invoke the underlying binary. As is the case with all
|
||||
// libraries, binaries which contain Abseil code must ensure that separate
|
||||
// packages use the same compiled copy of Abseil to avoid a diamond dependency
|
||||
// problem, which can occur if two packages built with different Abseil
|
||||
// configuration settings are linked together. Diamond dependency problems in
|
||||
// C++ may manifest as violations to the One Definition Rule (ODR) (resulting in
|
||||
// linker errors), or undefined behavior (resulting in crashes).
|
||||
//
|
||||
// Diamond dependency problems can be avoided if all packages utilize the same
|
||||
// exact version of Abseil. Building from source code with the same compilation
|
||||
// parameters is the easiest way to avoid such dependency problems. However, for
|
||||
// package managers who cannot control such compilation parameters, we are
|
||||
// providing the file to allow you to inject ABI (Application Binary Interface)
|
||||
// stability across builds. Settings options in this file will neither change
|
||||
// API nor ABI, providing a stable copy of Abseil between packages.
|
||||
//
|
||||
// Care must be taken to keep options within these configurations isolated
|
||||
// from any other dynamic settings, such as command-line flags which could alter
|
||||
// these options. This file is provided specifically to help build and package
|
||||
// managers provide a stable copy of Abseil within their libraries and binaries;
|
||||
// other developers should not have need to alter the contents of this file.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// Usage
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// For any particular package release, set the appropriate definitions within
|
||||
// this file to whatever value makes the most sense for your package(s). Note
|
||||
// that, by default, most of these options, at the moment, affect the
|
||||
// implementation of types; future options may affect other implementation
|
||||
// details.
|
||||
//
|
||||
// NOTE: the defaults within this file all assume that Abseil can select the
|
||||
// proper Abseil implementation at compile-time, which will not be sufficient
|
||||
// to guarantee ABI stability to package managers.
|
||||
|
||||
// Include a standard library header to allow configuration based on the
|
||||
// standard library in use.
|
||||
#ifdef __cplusplus
|
||||
#include <ciso646>
|
||||
#endif
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// Type Compatibility Options
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// ABSL_OPTION_USE_STD_ANY
|
||||
//
|
||||
// This option controls whether absl::any is implemented as an alias to
|
||||
// std::any, or as an independent implementation.
|
||||
//
|
||||
// A value of 0 means to use Abseil's implementation. This requires only C++11
|
||||
// support, and is expected to work on every toolchain we support.
|
||||
//
|
||||
// A value of 1 means to use an alias to std::any. This requires that all code
|
||||
// using Abseil is built in C++17 mode or later.
|
||||
//
|
||||
// A value of 2 means to detect the C++ version being used to compile Abseil,
|
||||
// and use an alias only if a working std::any is available. This option is
|
||||
// useful when you are building your entire program, including all of its
|
||||
// dependencies, from source. It should not be used otherwise -- for example,
|
||||
// if you are distributing Abseil in a binary package manager -- since in
|
||||
// mode 2, absl::any will name a different type, with a different mangled name
|
||||
// and binary layout, depending on the compiler flags passed by the end user.
|
||||
// For more info, see https://abseil.io/about/design/dropin-types.
|
||||
//
|
||||
// User code should not inspect this macro. To check in the preprocessor if
|
||||
// absl::any is a typedef of std::any, use the feature macro ABSL_USES_STD_ANY.
|
||||
|
||||
#define ABSL_OPTION_USE_STD_ANY 2
|
||||
|
||||
|
||||
// ABSL_OPTION_USE_STD_OPTIONAL
|
||||
//
|
||||
// This option controls whether absl::optional is implemented as an alias to
|
||||
// std::optional, or as an independent implementation.
|
||||
//
|
||||
// A value of 0 means to use Abseil's implementation. This requires only C++11
|
||||
// support, and is expected to work on every toolchain we support.
|
||||
//
|
||||
// A value of 1 means to use an alias to std::optional. This requires that all
|
||||
// code using Abseil is built in C++17 mode or later.
|
||||
//
|
||||
// A value of 2 means to detect the C++ version being used to compile Abseil,
|
||||
// and use an alias only if a working std::optional is available. This option
|
||||
// is useful when you are building your program from source. It should not be
|
||||
// used otherwise -- for example, if you are distributing Abseil in a binary
|
||||
// package manager -- since in mode 2, absl::optional will name a different
|
||||
// type, with a different mangled name and binary layout, depending on the
|
||||
// compiler flags passed by the end user. For more info, see
|
||||
// https://abseil.io/about/design/dropin-types.
|
||||
|
||||
// User code should not inspect this macro. To check in the preprocessor if
|
||||
// absl::optional is a typedef of std::optional, use the feature macro
|
||||
// ABSL_USES_STD_OPTIONAL.
|
||||
|
||||
#define ABSL_OPTION_USE_STD_OPTIONAL 2
|
||||
|
||||
|
||||
// ABSL_OPTION_USE_STD_STRING_VIEW
|
||||
//
|
||||
// This option controls whether absl::string_view is implemented as an alias to
|
||||
// std::string_view, or as an independent implementation.
|
||||
//
|
||||
// A value of 0 means to use Abseil's implementation. This requires only C++11
|
||||
// support, and is expected to work on every toolchain we support.
|
||||
//
|
||||
// A value of 1 means to use an alias to std::string_view. This requires that
|
||||
// all code using Abseil is built in C++17 mode or later.
|
||||
//
|
||||
// A value of 2 means to detect the C++ version being used to compile Abseil,
|
||||
// and use an alias only if a working std::string_view is available. This
|
||||
// option is useful when you are building your program from source. It should
|
||||
// not be used otherwise -- for example, if you are distributing Abseil in a
|
||||
// binary package manager -- since in mode 2, absl::string_view will name a
|
||||
// different type, with a different mangled name and binary layout, depending on
|
||||
// the compiler flags passed by the end user. For more info, see
|
||||
// https://abseil.io/about/design/dropin-types.
|
||||
//
|
||||
// User code should not inspect this macro. To check in the preprocessor if
|
||||
// absl::string_view is a typedef of std::string_view, use the feature macro
|
||||
// ABSL_USES_STD_STRING_VIEW.
|
||||
|
||||
#define ABSL_OPTION_USE_STD_STRING_VIEW 2
|
||||
|
||||
// ABSL_OPTION_USE_STD_VARIANT
|
||||
//
|
||||
// This option controls whether absl::variant is implemented as an alias to
|
||||
// std::variant, or as an independent implementation.
|
||||
//
|
||||
// A value of 0 means to use Abseil's implementation. This requires only C++11
|
||||
// support, and is expected to work on every toolchain we support.
|
||||
//
|
||||
// A value of 1 means to use an alias to std::variant. This requires that all
|
||||
// code using Abseil is built in C++17 mode or later.
|
||||
//
|
||||
// A value of 2 means to detect the C++ version being used to compile Abseil,
|
||||
// and use an alias only if a working std::variant is available. This option
|
||||
// is useful when you are building your program from source. It should not be
|
||||
// used otherwise -- for example, if you are distributing Abseil in a binary
|
||||
// package manager -- since in mode 2, absl::variant will name a different
|
||||
// type, with a different mangled name and binary layout, depending on the
|
||||
// compiler flags passed by the end user. For more info, see
|
||||
// https://abseil.io/about/design/dropin-types.
|
||||
//
|
||||
// User code should not inspect this macro. To check in the preprocessor if
|
||||
// absl::variant is a typedef of std::variant, use the feature macro
|
||||
// ABSL_USES_STD_VARIANT.
|
||||
|
||||
#define ABSL_OPTION_USE_STD_VARIANT 2
|
||||
|
||||
|
||||
// ABSL_OPTION_USE_INLINE_NAMESPACE
|
||||
// ABSL_OPTION_INLINE_NAMESPACE_NAME
|
||||
//
|
||||
// These options controls whether all entities in the absl namespace are
|
||||
// contained within an inner inline namespace. This does not affect the
|
||||
// user-visible API of Abseil, but it changes the mangled names of all symbols.
|
||||
//
|
||||
// This can be useful as a version tag if you are distributing Abseil in
|
||||
// precompiled form. This will prevent a binary library build of Abseil with
|
||||
// one inline namespace being used with headers configured with a different
|
||||
// inline namespace name. Binary packagers are reminded that Abseil does not
|
||||
// guarantee any ABI stability in Abseil, so any update of Abseil or
|
||||
// configuration change in such a binary package should be combined with a
|
||||
// new, unique value for the inline namespace name.
|
||||
//
|
||||
// A value of 0 means not to use inline namespaces.
|
||||
//
|
||||
// A value of 1 means to use an inline namespace with the given name inside
|
||||
// namespace absl. If this is set, ABSL_OPTION_INLINE_NAMESPACE_NAME must also
|
||||
// be changed to a new, unique identifier name. In particular "head" is not
|
||||
// allowed.
|
||||
|
||||
#define ABSL_OPTION_USE_INLINE_NAMESPACE 1
|
||||
#define ABSL_OPTION_INLINE_NAMESPACE_NAME lts_2020_02_25
|
||||
|
||||
#endif // ABSL_BASE_OPTIONS_H_
|
||||
Generated
+111
@@ -0,0 +1,111 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: policy_checks.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This header enforces a minimum set of policies at build time, such as the
|
||||
// supported compiler and library versions. Unsupported configurations are
|
||||
// reported with `#error`. This enforcement is best effort, so successfully
|
||||
// compiling this header does not guarantee a supported configuration.
|
||||
|
||||
#ifndef ABSL_BASE_POLICY_CHECKS_H_
|
||||
#define ABSL_BASE_POLICY_CHECKS_H_
|
||||
|
||||
// Included for the __GLIBC_PREREQ macro used below.
|
||||
#include <limits.h>
|
||||
|
||||
// Included for the _STLPORT_VERSION macro used below.
|
||||
#if defined(__cplusplus)
|
||||
#include <cstddef>
|
||||
#endif
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// Operating System Check
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
#if defined(__CYGWIN__)
|
||||
#error "Cygwin is not supported."
|
||||
#endif
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// Compiler Check
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
// We support MSVC++ 14.0 update 2 and later.
|
||||
// This minimum will go up.
|
||||
#if defined(_MSC_FULL_VER) && _MSC_FULL_VER < 190023918 && !defined(__clang__)
|
||||
#error "This package requires Visual Studio 2015 Update 2 or higher."
|
||||
#endif
|
||||
|
||||
// We support gcc 4.7 and later.
|
||||
// This minimum will go up.
|
||||
#if defined(__GNUC__) && !defined(__clang__)
|
||||
#if __GNUC__ < 4 || (__GNUC__ == 4 && __GNUC_MINOR__ < 7)
|
||||
#error "This package requires gcc 4.7 or higher."
|
||||
#endif
|
||||
#endif
|
||||
|
||||
// We support Apple Xcode clang 4.2.1 (version 421.11.65) and later.
|
||||
// This corresponds to Apple Xcode version 4.5.
|
||||
// This minimum will go up.
|
||||
#if defined(__apple_build_version__) && __apple_build_version__ < 4211165
|
||||
#error "This package requires __apple_build_version__ of 4211165 or higher."
|
||||
#endif
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// C++ Version Check
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
// Enforce C++11 as the minimum. Note that Visual Studio has not
|
||||
// advanced __cplusplus despite being good enough for our purposes, so
|
||||
// so we exempt it from the check.
|
||||
#if defined(__cplusplus) && !defined(_MSC_VER)
|
||||
#if __cplusplus < 201103L
|
||||
#error "C++ versions less than C++11 are not supported."
|
||||
#endif
|
||||
#endif
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// Standard Library Check
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
#if defined(_STLPORT_VERSION)
|
||||
#error "STLPort is not supported."
|
||||
#endif
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// `char` Size Check
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
// Abseil currently assumes CHAR_BIT == 8. If you would like to use Abseil on a
|
||||
// platform where this is not the case, please provide us with the details about
|
||||
// your platform so we can consider relaxing this requirement.
|
||||
#if CHAR_BIT != 8
|
||||
#error "Abseil assumes CHAR_BIT == 8."
|
||||
#endif
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// `int` Size Check
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
// Abseil currently assumes that an int is 4 bytes. If you would like to use
|
||||
// Abseil on a platform where this is not the case, please provide us with the
|
||||
// details about your platform so we can consider relaxing this requirement.
|
||||
#if INT_MAX < 2147483647
|
||||
#error "Abseil assumes that int is at least 4 bytes. "
|
||||
#endif
|
||||
|
||||
#endif // ABSL_BASE_POLICY_CHECKS_H_
|
||||
Generated
+26
@@ -0,0 +1,26 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// This files is a forwarding header for other headers containing various
|
||||
// portability macros and functions.
|
||||
// This file is used for both C and C++!
|
||||
|
||||
#ifndef ABSL_BASE_PORT_H_
|
||||
#define ABSL_BASE_PORT_H_
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/config.h"
|
||||
#include "absl/base/optimization.h"
|
||||
|
||||
#endif // ABSL_BASE_PORT_H_
|
||||
+280
@@ -0,0 +1,280 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: thread_annotations.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This header file contains macro definitions for thread safety annotations
|
||||
// that allow developers to document the locking policies of multi-threaded
|
||||
// code. The annotations can also help program analysis tools to identify
|
||||
// potential thread safety issues.
|
||||
//
|
||||
// These annotations are implemented using compiler attributes. Using the macros
|
||||
// defined here instead of raw attributes allow for portability and future
|
||||
// compatibility.
|
||||
//
|
||||
// When referring to mutexes in the arguments of the attributes, you should
|
||||
// use variable names or more complex expressions (e.g. my_object->mutex_)
|
||||
// that evaluate to a concrete mutex object whenever possible. If the mutex
|
||||
// you want to refer to is not in scope, you may use a member pointer
|
||||
// (e.g. &MyClass::mutex_) to refer to a mutex in some (unknown) object.
|
||||
|
||||
#ifndef ABSL_BASE_THREAD_ANNOTATIONS_H_
|
||||
#define ABSL_BASE_THREAD_ANNOTATIONS_H_
|
||||
|
||||
#include "absl/base/config.h"
|
||||
// TODO(mbonadei): Remove after the backward compatibility period.
|
||||
#include "absl/base/internal/thread_annotations.h" // IWYU pragma: export
|
||||
|
||||
#if defined(__clang__)
|
||||
#define ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(x) __attribute__((x))
|
||||
#else
|
||||
#define ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(x) // no-op
|
||||
#endif
|
||||
|
||||
// ABSL_GUARDED_BY()
|
||||
//
|
||||
// Documents if a shared field or global variable needs to be protected by a
|
||||
// mutex. ABSL_GUARDED_BY() allows the user to specify a particular mutex that
|
||||
// should be held when accessing the annotated variable.
|
||||
//
|
||||
// Although this annotation (and ABSL_PT_GUARDED_BY, below) cannot be applied to
|
||||
// local variables, a local variable and its associated mutex can often be
|
||||
// combined into a small class or struct, thereby allowing the annotation.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// class Foo {
|
||||
// Mutex mu_;
|
||||
// int p1_ ABSL_GUARDED_BY(mu_);
|
||||
// ...
|
||||
// };
|
||||
#define ABSL_GUARDED_BY(x) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(guarded_by(x))
|
||||
|
||||
// ABSL_PT_GUARDED_BY()
|
||||
//
|
||||
// Documents if the memory location pointed to by a pointer should be guarded
|
||||
// by a mutex when dereferencing the pointer.
|
||||
//
|
||||
// Example:
|
||||
// class Foo {
|
||||
// Mutex mu_;
|
||||
// int *p1_ ABSL_PT_GUARDED_BY(mu_);
|
||||
// ...
|
||||
// };
|
||||
//
|
||||
// Note that a pointer variable to a shared memory location could itself be a
|
||||
// shared variable.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// // `q_`, guarded by `mu1_`, points to a shared memory location that is
|
||||
// // guarded by `mu2_`:
|
||||
// int *q_ ABSL_GUARDED_BY(mu1_) ABSL_PT_GUARDED_BY(mu2_);
|
||||
#define ABSL_PT_GUARDED_BY(x) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(pt_guarded_by(x))
|
||||
|
||||
// ABSL_ACQUIRED_AFTER() / ABSL_ACQUIRED_BEFORE()
|
||||
//
|
||||
// Documents the acquisition order between locks that can be held
|
||||
// simultaneously by a thread. For any two locks that need to be annotated
|
||||
// to establish an acquisition order, only one of them needs the annotation.
|
||||
// (i.e. You don't have to annotate both locks with both ABSL_ACQUIRED_AFTER
|
||||
// and ABSL_ACQUIRED_BEFORE.)
|
||||
//
|
||||
// As with ABSL_GUARDED_BY, this is only applicable to mutexes that are shared
|
||||
// fields or global variables.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// Mutex m1_;
|
||||
// Mutex m2_ ABSL_ACQUIRED_AFTER(m1_);
|
||||
#define ABSL_ACQUIRED_AFTER(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(acquired_after(__VA_ARGS__))
|
||||
|
||||
#define ABSL_ACQUIRED_BEFORE(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(acquired_before(__VA_ARGS__))
|
||||
|
||||
// ABSL_EXCLUSIVE_LOCKS_REQUIRED() / ABSL_SHARED_LOCKS_REQUIRED()
|
||||
//
|
||||
// Documents a function that expects a mutex to be held prior to entry.
|
||||
// The mutex is expected to be held both on entry to, and exit from, the
|
||||
// function.
|
||||
//
|
||||
// An exclusive lock allows read-write access to the guarded data member(s), and
|
||||
// only one thread can acquire a lock exclusively at any one time. A shared lock
|
||||
// allows read-only access, and any number of threads can acquire a shared lock
|
||||
// concurrently.
|
||||
//
|
||||
// Generally, non-const methods should be annotated with
|
||||
// ABSL_EXCLUSIVE_LOCKS_REQUIRED, while const methods should be annotated with
|
||||
// ABSL_SHARED_LOCKS_REQUIRED.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// Mutex mu1, mu2;
|
||||
// int a ABSL_GUARDED_BY(mu1);
|
||||
// int b ABSL_GUARDED_BY(mu2);
|
||||
//
|
||||
// void foo() ABSL_EXCLUSIVE_LOCKS_REQUIRED(mu1, mu2) { ... }
|
||||
// void bar() const ABSL_SHARED_LOCKS_REQUIRED(mu1, mu2) { ... }
|
||||
#define ABSL_EXCLUSIVE_LOCKS_REQUIRED(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE( \
|
||||
exclusive_locks_required(__VA_ARGS__))
|
||||
|
||||
#define ABSL_SHARED_LOCKS_REQUIRED(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(shared_locks_required(__VA_ARGS__))
|
||||
|
||||
// ABSL_LOCKS_EXCLUDED()
|
||||
//
|
||||
// Documents the locks acquired in the body of the function. These locks
|
||||
// cannot be held when calling this function (as Abseil's `Mutex` locks are
|
||||
// non-reentrant).
|
||||
#define ABSL_LOCKS_EXCLUDED(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(locks_excluded(__VA_ARGS__))
|
||||
|
||||
// ABSL_LOCK_RETURNED()
|
||||
//
|
||||
// Documents a function that returns a mutex without acquiring it. For example,
|
||||
// a public getter method that returns a pointer to a private mutex should
|
||||
// be annotated with ABSL_LOCK_RETURNED.
|
||||
#define ABSL_LOCK_RETURNED(x) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(lock_returned(x))
|
||||
|
||||
// ABSL_LOCKABLE
|
||||
//
|
||||
// Documents if a class/type is a lockable type (such as the `Mutex` class).
|
||||
#define ABSL_LOCKABLE ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(lockable)
|
||||
|
||||
// ABSL_SCOPED_LOCKABLE
|
||||
//
|
||||
// Documents if a class does RAII locking (such as the `MutexLock` class).
|
||||
// The constructor should use `LOCK_FUNCTION()` to specify the mutex that is
|
||||
// acquired, and the destructor should use `UNLOCK_FUNCTION()` with no
|
||||
// arguments; the analysis will assume that the destructor unlocks whatever the
|
||||
// constructor locked.
|
||||
#define ABSL_SCOPED_LOCKABLE \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(scoped_lockable)
|
||||
|
||||
// ABSL_EXCLUSIVE_LOCK_FUNCTION()
|
||||
//
|
||||
// Documents functions that acquire a lock in the body of a function, and do
|
||||
// not release it.
|
||||
#define ABSL_EXCLUSIVE_LOCK_FUNCTION(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE( \
|
||||
exclusive_lock_function(__VA_ARGS__))
|
||||
|
||||
// ABSL_SHARED_LOCK_FUNCTION()
|
||||
//
|
||||
// Documents functions that acquire a shared (reader) lock in the body of a
|
||||
// function, and do not release it.
|
||||
#define ABSL_SHARED_LOCK_FUNCTION(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(shared_lock_function(__VA_ARGS__))
|
||||
|
||||
// ABSL_UNLOCK_FUNCTION()
|
||||
//
|
||||
// Documents functions that expect a lock to be held on entry to the function,
|
||||
// and release it in the body of the function.
|
||||
#define ABSL_UNLOCK_FUNCTION(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(unlock_function(__VA_ARGS__))
|
||||
|
||||
// ABSL_EXCLUSIVE_TRYLOCK_FUNCTION() / ABSL_SHARED_TRYLOCK_FUNCTION()
|
||||
//
|
||||
// Documents functions that try to acquire a lock, and return success or failure
|
||||
// (or a non-boolean value that can be interpreted as a boolean).
|
||||
// The first argument should be `true` for functions that return `true` on
|
||||
// success, or `false` for functions that return `false` on success. The second
|
||||
// argument specifies the mutex that is locked on success. If unspecified, this
|
||||
// mutex is assumed to be `this`.
|
||||
#define ABSL_EXCLUSIVE_TRYLOCK_FUNCTION(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE( \
|
||||
exclusive_trylock_function(__VA_ARGS__))
|
||||
|
||||
#define ABSL_SHARED_TRYLOCK_FUNCTION(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE( \
|
||||
shared_trylock_function(__VA_ARGS__))
|
||||
|
||||
// ABSL_ASSERT_EXCLUSIVE_LOCK() / ABSL_ASSERT_SHARED_LOCK()
|
||||
//
|
||||
// Documents functions that dynamically check to see if a lock is held, and fail
|
||||
// if it is not held.
|
||||
#define ABSL_ASSERT_EXCLUSIVE_LOCK(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(assert_exclusive_lock(__VA_ARGS__))
|
||||
|
||||
#define ABSL_ASSERT_SHARED_LOCK(...) \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(assert_shared_lock(__VA_ARGS__))
|
||||
|
||||
// ABSL_NO_THREAD_SAFETY_ANALYSIS
|
||||
//
|
||||
// Turns off thread safety checking within the body of a particular function.
|
||||
// This annotation is used to mark functions that are known to be correct, but
|
||||
// the locking behavior is more complicated than the analyzer can handle.
|
||||
#define ABSL_NO_THREAD_SAFETY_ANALYSIS \
|
||||
ABSL_INTERNAL_THREAD_ANNOTATION_ATTRIBUTE(no_thread_safety_analysis)
|
||||
|
||||
//------------------------------------------------------------------------------
|
||||
// Tool-Supplied Annotations
|
||||
//------------------------------------------------------------------------------
|
||||
|
||||
// ABSL_TS_UNCHECKED should be placed around lock expressions that are not valid
|
||||
// C++ syntax, but which are present for documentation purposes. These
|
||||
// annotations will be ignored by the analysis.
|
||||
#define ABSL_TS_UNCHECKED(x) ""
|
||||
|
||||
// ABSL_TS_FIXME is used to mark lock expressions that are not valid C++ syntax.
|
||||
// It is used by automated tools to mark and disable invalid expressions.
|
||||
// The annotation should either be fixed, or changed to ABSL_TS_UNCHECKED.
|
||||
#define ABSL_TS_FIXME(x) ""
|
||||
|
||||
// Like ABSL_NO_THREAD_SAFETY_ANALYSIS, this turns off checking within the body
|
||||
// of a particular function. However, this attribute is used to mark functions
|
||||
// that are incorrect and need to be fixed. It is used by automated tools to
|
||||
// avoid breaking the build when the analysis is updated.
|
||||
// Code owners are expected to eventually fix the routine.
|
||||
#define ABSL_NO_THREAD_SAFETY_ANALYSIS_FIXME ABSL_NO_THREAD_SAFETY_ANALYSIS
|
||||
|
||||
// Similar to ABSL_NO_THREAD_SAFETY_ANALYSIS_FIXME, this macro marks a
|
||||
// ABSL_GUARDED_BY annotation that needs to be fixed, because it is producing
|
||||
// thread safety warning. It disables the ABSL_GUARDED_BY.
|
||||
#define ABSL_GUARDED_BY_FIXME(x)
|
||||
|
||||
// Disables warnings for a single read operation. This can be used to avoid
|
||||
// warnings when it is known that the read is not actually involved in a race,
|
||||
// but the compiler cannot confirm that.
|
||||
#define ABSL_TS_UNCHECKED_READ(x) absl::base_internal::ts_unchecked_read(x)
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace base_internal {
|
||||
|
||||
// Takes a reference to a guarded data member, and returns an unguarded
|
||||
// reference.
|
||||
// Do not used this function directly, use ABSL_TS_UNCHECKED_READ instead.
|
||||
template <typename T>
|
||||
inline const T& ts_unchecked_read(const T& v) ABSL_NO_THREAD_SAFETY_ANALYSIS {
|
||||
return v;
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
inline T& ts_unchecked_read(T& v) ABSL_NO_THREAD_SAFETY_ANALYSIS {
|
||||
return v;
|
||||
}
|
||||
|
||||
} // namespace base_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_BASE_THREAD_ANNOTATIONS_H_
|
||||
Generated
+515
@@ -0,0 +1,515 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: fixed_array.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// A `FixedArray<T>` represents a non-resizable array of `T` where the length of
|
||||
// the array can be determined at run-time. It is a good replacement for
|
||||
// non-standard and deprecated uses of `alloca()` and variable length arrays
|
||||
// within the GCC extension. (See
|
||||
// https://gcc.gnu.org/onlinedocs/gcc/Variable-Length.html).
|
||||
//
|
||||
// `FixedArray` allocates small arrays inline, keeping performance fast by
|
||||
// avoiding heap operations. It also helps reduce the chances of
|
||||
// accidentally overflowing your stack if large input is passed to
|
||||
// your function.
|
||||
|
||||
#ifndef ABSL_CONTAINER_FIXED_ARRAY_H_
|
||||
#define ABSL_CONTAINER_FIXED_ARRAY_H_
|
||||
|
||||
#include <algorithm>
|
||||
#include <cassert>
|
||||
#include <cstddef>
|
||||
#include <initializer_list>
|
||||
#include <iterator>
|
||||
#include <limits>
|
||||
#include <memory>
|
||||
#include <new>
|
||||
#include <type_traits>
|
||||
|
||||
#include "absl/algorithm/algorithm.h"
|
||||
#include "absl/base/dynamic_annotations.h"
|
||||
#include "absl/base/internal/throw_delegate.h"
|
||||
#include "absl/base/macros.h"
|
||||
#include "absl/base/optimization.h"
|
||||
#include "absl/base/port.h"
|
||||
#include "absl/container/internal/compressed_tuple.h"
|
||||
#include "absl/memory/memory.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
|
||||
constexpr static auto kFixedArrayUseDefault = static_cast<size_t>(-1);
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// FixedArray
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// A `FixedArray` provides a run-time fixed-size array, allocating a small array
|
||||
// inline for efficiency.
|
||||
//
|
||||
// Most users should not specify an `inline_elements` argument and let
|
||||
// `FixedArray` automatically determine the number of elements
|
||||
// to store inline based on `sizeof(T)`. If `inline_elements` is specified, the
|
||||
// `FixedArray` implementation will use inline storage for arrays with a
|
||||
// length <= `inline_elements`.
|
||||
//
|
||||
// Note that a `FixedArray` constructed with a `size_type` argument will
|
||||
// default-initialize its values by leaving trivially constructible types
|
||||
// uninitialized (e.g. int, int[4], double), and others default-constructed.
|
||||
// This matches the behavior of c-style arrays and `std::array`, but not
|
||||
// `std::vector`.
|
||||
//
|
||||
// Note that `FixedArray` does not provide a public allocator; if it requires a
|
||||
// heap allocation, it will do so with global `::operator new[]()` and
|
||||
// `::operator delete[]()`, even if T provides class-scope overrides for these
|
||||
// operators.
|
||||
template <typename T, size_t N = kFixedArrayUseDefault,
|
||||
typename A = std::allocator<T>>
|
||||
class FixedArray {
|
||||
static_assert(!std::is_array<T>::value || std::extent<T>::value > 0,
|
||||
"Arrays with unknown bounds cannot be used with FixedArray.");
|
||||
|
||||
static constexpr size_t kInlineBytesDefault = 256;
|
||||
|
||||
using AllocatorTraits = std::allocator_traits<A>;
|
||||
// std::iterator_traits isn't guaranteed to be SFINAE-friendly until C++17,
|
||||
// but this seems to be mostly pedantic.
|
||||
template <typename Iterator>
|
||||
using EnableIfForwardIterator = absl::enable_if_t<std::is_convertible<
|
||||
typename std::iterator_traits<Iterator>::iterator_category,
|
||||
std::forward_iterator_tag>::value>;
|
||||
static constexpr bool NoexceptCopyable() {
|
||||
return std::is_nothrow_copy_constructible<StorageElement>::value &&
|
||||
absl::allocator_is_nothrow<allocator_type>::value;
|
||||
}
|
||||
static constexpr bool NoexceptMovable() {
|
||||
return std::is_nothrow_move_constructible<StorageElement>::value &&
|
||||
absl::allocator_is_nothrow<allocator_type>::value;
|
||||
}
|
||||
static constexpr bool DefaultConstructorIsNonTrivial() {
|
||||
return !absl::is_trivially_default_constructible<StorageElement>::value;
|
||||
}
|
||||
|
||||
public:
|
||||
using allocator_type = typename AllocatorTraits::allocator_type;
|
||||
using value_type = typename allocator_type::value_type;
|
||||
using pointer = typename allocator_type::pointer;
|
||||
using const_pointer = typename allocator_type::const_pointer;
|
||||
using reference = typename allocator_type::reference;
|
||||
using const_reference = typename allocator_type::const_reference;
|
||||
using size_type = typename allocator_type::size_type;
|
||||
using difference_type = typename allocator_type::difference_type;
|
||||
using iterator = pointer;
|
||||
using const_iterator = const_pointer;
|
||||
using reverse_iterator = std::reverse_iterator<iterator>;
|
||||
using const_reverse_iterator = std::reverse_iterator<const_iterator>;
|
||||
|
||||
static constexpr size_type inline_elements =
|
||||
(N == kFixedArrayUseDefault ? kInlineBytesDefault / sizeof(value_type)
|
||||
: static_cast<size_type>(N));
|
||||
|
||||
FixedArray(
|
||||
const FixedArray& other,
|
||||
const allocator_type& a = allocator_type()) noexcept(NoexceptCopyable())
|
||||
: FixedArray(other.begin(), other.end(), a) {}
|
||||
|
||||
FixedArray(
|
||||
FixedArray&& other,
|
||||
const allocator_type& a = allocator_type()) noexcept(NoexceptMovable())
|
||||
: FixedArray(std::make_move_iterator(other.begin()),
|
||||
std::make_move_iterator(other.end()), a) {}
|
||||
|
||||
// Creates an array object that can store `n` elements.
|
||||
// Note that trivially constructible elements will be uninitialized.
|
||||
explicit FixedArray(size_type n, const allocator_type& a = allocator_type())
|
||||
: storage_(n, a) {
|
||||
if (DefaultConstructorIsNonTrivial()) {
|
||||
memory_internal::ConstructRange(storage_.alloc(), storage_.begin(),
|
||||
storage_.end());
|
||||
}
|
||||
}
|
||||
|
||||
// Creates an array initialized with `n` copies of `val`.
|
||||
FixedArray(size_type n, const value_type& val,
|
||||
const allocator_type& a = allocator_type())
|
||||
: storage_(n, a) {
|
||||
memory_internal::ConstructRange(storage_.alloc(), storage_.begin(),
|
||||
storage_.end(), val);
|
||||
}
|
||||
|
||||
// Creates an array initialized with the size and contents of `init_list`.
|
||||
FixedArray(std::initializer_list<value_type> init_list,
|
||||
const allocator_type& a = allocator_type())
|
||||
: FixedArray(init_list.begin(), init_list.end(), a) {}
|
||||
|
||||
// Creates an array initialized with the elements from the input
|
||||
// range. The array's size will always be `std::distance(first, last)`.
|
||||
// REQUIRES: Iterator must be a forward_iterator or better.
|
||||
template <typename Iterator, EnableIfForwardIterator<Iterator>* = nullptr>
|
||||
FixedArray(Iterator first, Iterator last,
|
||||
const allocator_type& a = allocator_type())
|
||||
: storage_(std::distance(first, last), a) {
|
||||
memory_internal::CopyRange(storage_.alloc(), storage_.begin(), first, last);
|
||||
}
|
||||
|
||||
~FixedArray() noexcept {
|
||||
for (auto* cur = storage_.begin(); cur != storage_.end(); ++cur) {
|
||||
AllocatorTraits::destroy(storage_.alloc(), cur);
|
||||
}
|
||||
}
|
||||
|
||||
// Assignments are deleted because they break the invariant that the size of a
|
||||
// `FixedArray` never changes.
|
||||
void operator=(FixedArray&&) = delete;
|
||||
void operator=(const FixedArray&) = delete;
|
||||
|
||||
// FixedArray::size()
|
||||
//
|
||||
// Returns the length of the fixed array.
|
||||
size_type size() const { return storage_.size(); }
|
||||
|
||||
// FixedArray::max_size()
|
||||
//
|
||||
// Returns the largest possible value of `std::distance(begin(), end())` for a
|
||||
// `FixedArray<T>`. This is equivalent to the most possible addressable bytes
|
||||
// over the number of bytes taken by T.
|
||||
constexpr size_type max_size() const {
|
||||
return (std::numeric_limits<difference_type>::max)() / sizeof(value_type);
|
||||
}
|
||||
|
||||
// FixedArray::empty()
|
||||
//
|
||||
// Returns whether or not the fixed array is empty.
|
||||
bool empty() const { return size() == 0; }
|
||||
|
||||
// FixedArray::memsize()
|
||||
//
|
||||
// Returns the memory size of the fixed array in bytes.
|
||||
size_t memsize() const { return size() * sizeof(value_type); }
|
||||
|
||||
// FixedArray::data()
|
||||
//
|
||||
// Returns a const T* pointer to elements of the `FixedArray`. This pointer
|
||||
// can be used to access (but not modify) the contained elements.
|
||||
const_pointer data() const { return AsValueType(storage_.begin()); }
|
||||
|
||||
// Overload of FixedArray::data() to return a T* pointer to elements of the
|
||||
// fixed array. This pointer can be used to access and modify the contained
|
||||
// elements.
|
||||
pointer data() { return AsValueType(storage_.begin()); }
|
||||
|
||||
// FixedArray::operator[]
|
||||
//
|
||||
// Returns a reference the ith element of the fixed array.
|
||||
// REQUIRES: 0 <= i < size()
|
||||
reference operator[](size_type i) {
|
||||
assert(i < size());
|
||||
return data()[i];
|
||||
}
|
||||
|
||||
// Overload of FixedArray::operator()[] to return a const reference to the
|
||||
// ith element of the fixed array.
|
||||
// REQUIRES: 0 <= i < size()
|
||||
const_reference operator[](size_type i) const {
|
||||
assert(i < size());
|
||||
return data()[i];
|
||||
}
|
||||
|
||||
// FixedArray::at
|
||||
//
|
||||
// Bounds-checked access. Returns a reference to the ith element of the
|
||||
// fiexed array, or throws std::out_of_range
|
||||
reference at(size_type i) {
|
||||
if (ABSL_PREDICT_FALSE(i >= size())) {
|
||||
base_internal::ThrowStdOutOfRange("FixedArray::at failed bounds check");
|
||||
}
|
||||
return data()[i];
|
||||
}
|
||||
|
||||
// Overload of FixedArray::at() to return a const reference to the ith element
|
||||
// of the fixed array.
|
||||
const_reference at(size_type i) const {
|
||||
if (ABSL_PREDICT_FALSE(i >= size())) {
|
||||
base_internal::ThrowStdOutOfRange("FixedArray::at failed bounds check");
|
||||
}
|
||||
return data()[i];
|
||||
}
|
||||
|
||||
// FixedArray::front()
|
||||
//
|
||||
// Returns a reference to the first element of the fixed array.
|
||||
reference front() { return *begin(); }
|
||||
|
||||
// Overload of FixedArray::front() to return a reference to the first element
|
||||
// of a fixed array of const values.
|
||||
const_reference front() const { return *begin(); }
|
||||
|
||||
// FixedArray::back()
|
||||
//
|
||||
// Returns a reference to the last element of the fixed array.
|
||||
reference back() { return *(end() - 1); }
|
||||
|
||||
// Overload of FixedArray::back() to return a reference to the last element
|
||||
// of a fixed array of const values.
|
||||
const_reference back() const { return *(end() - 1); }
|
||||
|
||||
// FixedArray::begin()
|
||||
//
|
||||
// Returns an iterator to the beginning of the fixed array.
|
||||
iterator begin() { return data(); }
|
||||
|
||||
// Overload of FixedArray::begin() to return a const iterator to the
|
||||
// beginning of the fixed array.
|
||||
const_iterator begin() const { return data(); }
|
||||
|
||||
// FixedArray::cbegin()
|
||||
//
|
||||
// Returns a const iterator to the beginning of the fixed array.
|
||||
const_iterator cbegin() const { return begin(); }
|
||||
|
||||
// FixedArray::end()
|
||||
//
|
||||
// Returns an iterator to the end of the fixed array.
|
||||
iterator end() { return data() + size(); }
|
||||
|
||||
// Overload of FixedArray::end() to return a const iterator to the end of the
|
||||
// fixed array.
|
||||
const_iterator end() const { return data() + size(); }
|
||||
|
||||
// FixedArray::cend()
|
||||
//
|
||||
// Returns a const iterator to the end of the fixed array.
|
||||
const_iterator cend() const { return end(); }
|
||||
|
||||
// FixedArray::rbegin()
|
||||
//
|
||||
// Returns a reverse iterator from the end of the fixed array.
|
||||
reverse_iterator rbegin() { return reverse_iterator(end()); }
|
||||
|
||||
// Overload of FixedArray::rbegin() to return a const reverse iterator from
|
||||
// the end of the fixed array.
|
||||
const_reverse_iterator rbegin() const {
|
||||
return const_reverse_iterator(end());
|
||||
}
|
||||
|
||||
// FixedArray::crbegin()
|
||||
//
|
||||
// Returns a const reverse iterator from the end of the fixed array.
|
||||
const_reverse_iterator crbegin() const { return rbegin(); }
|
||||
|
||||
// FixedArray::rend()
|
||||
//
|
||||
// Returns a reverse iterator from the beginning of the fixed array.
|
||||
reverse_iterator rend() { return reverse_iterator(begin()); }
|
||||
|
||||
// Overload of FixedArray::rend() for returning a const reverse iterator
|
||||
// from the beginning of the fixed array.
|
||||
const_reverse_iterator rend() const {
|
||||
return const_reverse_iterator(begin());
|
||||
}
|
||||
|
||||
// FixedArray::crend()
|
||||
//
|
||||
// Returns a reverse iterator from the beginning of the fixed array.
|
||||
const_reverse_iterator crend() const { return rend(); }
|
||||
|
||||
// FixedArray::fill()
|
||||
//
|
||||
// Assigns the given `value` to all elements in the fixed array.
|
||||
void fill(const value_type& val) { std::fill(begin(), end(), val); }
|
||||
|
||||
// Relational operators. Equality operators are elementwise using
|
||||
// `operator==`, while order operators order FixedArrays lexicographically.
|
||||
friend bool operator==(const FixedArray& lhs, const FixedArray& rhs) {
|
||||
return absl::equal(lhs.begin(), lhs.end(), rhs.begin(), rhs.end());
|
||||
}
|
||||
|
||||
friend bool operator!=(const FixedArray& lhs, const FixedArray& rhs) {
|
||||
return !(lhs == rhs);
|
||||
}
|
||||
|
||||
friend bool operator<(const FixedArray& lhs, const FixedArray& rhs) {
|
||||
return std::lexicographical_compare(lhs.begin(), lhs.end(), rhs.begin(),
|
||||
rhs.end());
|
||||
}
|
||||
|
||||
friend bool operator>(const FixedArray& lhs, const FixedArray& rhs) {
|
||||
return rhs < lhs;
|
||||
}
|
||||
|
||||
friend bool operator<=(const FixedArray& lhs, const FixedArray& rhs) {
|
||||
return !(rhs < lhs);
|
||||
}
|
||||
|
||||
friend bool operator>=(const FixedArray& lhs, const FixedArray& rhs) {
|
||||
return !(lhs < rhs);
|
||||
}
|
||||
|
||||
template <typename H>
|
||||
friend H AbslHashValue(H h, const FixedArray& v) {
|
||||
return H::combine(H::combine_contiguous(std::move(h), v.data(), v.size()),
|
||||
v.size());
|
||||
}
|
||||
|
||||
private:
|
||||
// StorageElement
|
||||
//
|
||||
// For FixedArrays with a C-style-array value_type, StorageElement is a POD
|
||||
// wrapper struct called StorageElementWrapper that holds the value_type
|
||||
// instance inside. This is needed for construction and destruction of the
|
||||
// entire array regardless of how many dimensions it has. For all other cases,
|
||||
// StorageElement is just an alias of value_type.
|
||||
//
|
||||
// Maintainer's Note: The simpler solution would be to simply wrap value_type
|
||||
// in a struct whether it's an array or not. That causes some paranoid
|
||||
// diagnostics to misfire, believing that 'data()' returns a pointer to a
|
||||
// single element, rather than the packed array that it really is.
|
||||
// e.g.:
|
||||
//
|
||||
// FixedArray<char> buf(1);
|
||||
// sprintf(buf.data(), "foo");
|
||||
//
|
||||
// error: call to int __builtin___sprintf_chk(etc...)
|
||||
// will always overflow destination buffer [-Werror]
|
||||
//
|
||||
template <typename OuterT, typename InnerT = absl::remove_extent_t<OuterT>,
|
||||
size_t InnerN = std::extent<OuterT>::value>
|
||||
struct StorageElementWrapper {
|
||||
InnerT array[InnerN];
|
||||
};
|
||||
|
||||
using StorageElement =
|
||||
absl::conditional_t<std::is_array<value_type>::value,
|
||||
StorageElementWrapper<value_type>, value_type>;
|
||||
|
||||
static pointer AsValueType(pointer ptr) { return ptr; }
|
||||
static pointer AsValueType(StorageElementWrapper<value_type>* ptr) {
|
||||
return std::addressof(ptr->array);
|
||||
}
|
||||
|
||||
static_assert(sizeof(StorageElement) == sizeof(value_type), "");
|
||||
static_assert(alignof(StorageElement) == alignof(value_type), "");
|
||||
|
||||
class NonEmptyInlinedStorage {
|
||||
public:
|
||||
StorageElement* data() { return reinterpret_cast<StorageElement*>(buff_); }
|
||||
void AnnotateConstruct(size_type n);
|
||||
void AnnotateDestruct(size_type n);
|
||||
|
||||
#ifdef ADDRESS_SANITIZER
|
||||
void* RedzoneBegin() { return &redzone_begin_; }
|
||||
void* RedzoneEnd() { return &redzone_end_ + 1; }
|
||||
#endif // ADDRESS_SANITIZER
|
||||
|
||||
private:
|
||||
ADDRESS_SANITIZER_REDZONE(redzone_begin_);
|
||||
alignas(StorageElement) char buff_[sizeof(StorageElement[inline_elements])];
|
||||
ADDRESS_SANITIZER_REDZONE(redzone_end_);
|
||||
};
|
||||
|
||||
class EmptyInlinedStorage {
|
||||
public:
|
||||
StorageElement* data() { return nullptr; }
|
||||
void AnnotateConstruct(size_type) {}
|
||||
void AnnotateDestruct(size_type) {}
|
||||
};
|
||||
|
||||
using InlinedStorage =
|
||||
absl::conditional_t<inline_elements == 0, EmptyInlinedStorage,
|
||||
NonEmptyInlinedStorage>;
|
||||
|
||||
// Storage
|
||||
//
|
||||
// An instance of Storage manages the inline and out-of-line memory for
|
||||
// instances of FixedArray. This guarantees that even when construction of
|
||||
// individual elements fails in the FixedArray constructor body, the
|
||||
// destructor for Storage will still be called and out-of-line memory will be
|
||||
// properly deallocated.
|
||||
//
|
||||
class Storage : public InlinedStorage {
|
||||
public:
|
||||
Storage(size_type n, const allocator_type& a)
|
||||
: size_alloc_(n, a), data_(InitializeData()) {}
|
||||
|
||||
~Storage() noexcept {
|
||||
if (UsingInlinedStorage(size())) {
|
||||
InlinedStorage::AnnotateDestruct(size());
|
||||
} else {
|
||||
AllocatorTraits::deallocate(alloc(), AsValueType(begin()), size());
|
||||
}
|
||||
}
|
||||
|
||||
size_type size() const { return size_alloc_.template get<0>(); }
|
||||
StorageElement* begin() const { return data_; }
|
||||
StorageElement* end() const { return begin() + size(); }
|
||||
allocator_type& alloc() { return size_alloc_.template get<1>(); }
|
||||
|
||||
private:
|
||||
static bool UsingInlinedStorage(size_type n) {
|
||||
return n <= inline_elements;
|
||||
}
|
||||
|
||||
StorageElement* InitializeData() {
|
||||
if (UsingInlinedStorage(size())) {
|
||||
InlinedStorage::AnnotateConstruct(size());
|
||||
return InlinedStorage::data();
|
||||
} else {
|
||||
return reinterpret_cast<StorageElement*>(
|
||||
AllocatorTraits::allocate(alloc(), size()));
|
||||
}
|
||||
}
|
||||
|
||||
// `CompressedTuple` takes advantage of EBCO for stateless `allocator_type`s
|
||||
container_internal::CompressedTuple<size_type, allocator_type> size_alloc_;
|
||||
StorageElement* data_;
|
||||
};
|
||||
|
||||
Storage storage_;
|
||||
};
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
constexpr size_t FixedArray<T, N, A>::kInlineBytesDefault;
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
constexpr typename FixedArray<T, N, A>::size_type
|
||||
FixedArray<T, N, A>::inline_elements;
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
void FixedArray<T, N, A>::NonEmptyInlinedStorage::AnnotateConstruct(
|
||||
typename FixedArray<T, N, A>::size_type n) {
|
||||
#ifdef ADDRESS_SANITIZER
|
||||
if (!n) return;
|
||||
ANNOTATE_CONTIGUOUS_CONTAINER(data(), RedzoneEnd(), RedzoneEnd(), data() + n);
|
||||
ANNOTATE_CONTIGUOUS_CONTAINER(RedzoneBegin(), data(), data(), RedzoneBegin());
|
||||
#endif // ADDRESS_SANITIZER
|
||||
static_cast<void>(n); // Mark used when not in asan mode
|
||||
}
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
void FixedArray<T, N, A>::NonEmptyInlinedStorage::AnnotateDestruct(
|
||||
typename FixedArray<T, N, A>::size_type n) {
|
||||
#ifdef ADDRESS_SANITIZER
|
||||
if (!n) return;
|
||||
ANNOTATE_CONTIGUOUS_CONTAINER(data(), RedzoneEnd(), data() + n, RedzoneEnd());
|
||||
ANNOTATE_CONTIGUOUS_CONTAINER(RedzoneBegin(), data(), RedzoneBegin(), data());
|
||||
#endif // ADDRESS_SANITIZER
|
||||
static_cast<void>(n); // Mark used when not in asan mode
|
||||
}
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_FIXED_ARRAY_H_
|
||||
+600
@@ -0,0 +1,600 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: flat_hash_map.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// An `absl::flat_hash_map<K, V>` is an unordered associative container of
|
||||
// unique keys and associated values designed to be a more efficient replacement
|
||||
// for `std::unordered_map`. Like `unordered_map`, search, insertion, and
|
||||
// deletion of map elements can be done as an `O(1)` operation. However,
|
||||
// `flat_hash_map` (and other unordered associative containers known as the
|
||||
// collection of Abseil "Swiss tables") contain other optimizations that result
|
||||
// in both memory and computation advantages.
|
||||
//
|
||||
// In most cases, your default choice for a hash map should be a map of type
|
||||
// `flat_hash_map`.
|
||||
|
||||
#ifndef ABSL_CONTAINER_FLAT_HASH_MAP_H_
|
||||
#define ABSL_CONTAINER_FLAT_HASH_MAP_H_
|
||||
|
||||
#include <cstddef>
|
||||
#include <new>
|
||||
#include <type_traits>
|
||||
#include <utility>
|
||||
|
||||
#include "absl/algorithm/container.h"
|
||||
#include "absl/container/internal/container_memory.h"
|
||||
#include "absl/container/internal/hash_function_defaults.h" // IWYU pragma: export
|
||||
#include "absl/container/internal/raw_hash_map.h" // IWYU pragma: export
|
||||
#include "absl/memory/memory.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
template <class K, class V>
|
||||
struct FlatHashMapPolicy;
|
||||
} // namespace container_internal
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// absl::flat_hash_map
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// An `absl::flat_hash_map<K, V>` is an unordered associative container which
|
||||
// has been optimized for both speed and memory footprint in most common use
|
||||
// cases. Its interface is similar to that of `std::unordered_map<K, V>` with
|
||||
// the following notable differences:
|
||||
//
|
||||
// * Requires keys that are CopyConstructible
|
||||
// * Requires values that are MoveConstructible
|
||||
// * Supports heterogeneous lookup, through `find()`, `operator[]()` and
|
||||
// `insert()`, provided that the map is provided a compatible heterogeneous
|
||||
// hashing function and equality operator.
|
||||
// * Invalidates any references and pointers to elements within the table after
|
||||
// `rehash()`.
|
||||
// * Contains a `capacity()` member function indicating the number of element
|
||||
// slots (open, deleted, and empty) within the hash map.
|
||||
// * Returns `void` from the `erase(iterator)` overload.
|
||||
//
|
||||
// By default, `flat_hash_map` uses the `absl::Hash` hashing framework.
|
||||
// All fundamental and Abseil types that support the `absl::Hash` framework have
|
||||
// a compatible equality operator for comparing insertions into `flat_hash_map`.
|
||||
// If your type is not yet supported by the `absl::Hash` framework, see
|
||||
// absl/hash/hash.h for information on extending Abseil hashing to user-defined
|
||||
// types.
|
||||
//
|
||||
// NOTE: A `flat_hash_map` stores its value types directly inside its
|
||||
// implementation array to avoid memory indirection. Because a `flat_hash_map`
|
||||
// is designed to move data when rehashed, map values will not retain pointer
|
||||
// stability. If you require pointer stability, or if your values are large,
|
||||
// consider using `absl::flat_hash_map<Key, std::unique_ptr<Value>>` instead.
|
||||
// If your types are not moveable or you require pointer stability for keys,
|
||||
// consider `absl::node_hash_map`.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// // Create a flat hash map of three strings (that map to strings)
|
||||
// absl::flat_hash_map<std::string, std::string> ducks =
|
||||
// {{"a", "huey"}, {"b", "dewey"}, {"c", "louie"}};
|
||||
//
|
||||
// // Insert a new element into the flat hash map
|
||||
// ducks.insert({"d", "donald"});
|
||||
//
|
||||
// // Force a rehash of the flat hash map
|
||||
// ducks.rehash(0);
|
||||
//
|
||||
// // Find the element with the key "b"
|
||||
// std::string search_key = "b";
|
||||
// auto result = ducks.find(search_key);
|
||||
// if (result != ducks.end()) {
|
||||
// std::cout << "Result: " << result->second << std::endl;
|
||||
// }
|
||||
template <class K, class V,
|
||||
class Hash = absl::container_internal::hash_default_hash<K>,
|
||||
class Eq = absl::container_internal::hash_default_eq<K>,
|
||||
class Allocator = std::allocator<std::pair<const K, V>>>
|
||||
class flat_hash_map : public absl::container_internal::raw_hash_map<
|
||||
absl::container_internal::FlatHashMapPolicy<K, V>,
|
||||
Hash, Eq, Allocator> {
|
||||
using Base = typename flat_hash_map::raw_hash_map;
|
||||
|
||||
public:
|
||||
// Constructors and Assignment Operators
|
||||
//
|
||||
// A flat_hash_map supports the same overload set as `std::unordered_map`
|
||||
// for construction and assignment:
|
||||
//
|
||||
// * Default constructor
|
||||
//
|
||||
// // No allocation for the table's elements is made.
|
||||
// absl::flat_hash_map<int, std::string> map1;
|
||||
//
|
||||
// * Initializer List constructor
|
||||
//
|
||||
// absl::flat_hash_map<int, std::string> map2 =
|
||||
// {{1, "huey"}, {2, "dewey"}, {3, "louie"},};
|
||||
//
|
||||
// * Copy constructor
|
||||
//
|
||||
// absl::flat_hash_map<int, std::string> map3(map2);
|
||||
//
|
||||
// * Copy assignment operator
|
||||
//
|
||||
// // Hash functor and Comparator are copied as well
|
||||
// absl::flat_hash_map<int, std::string> map4;
|
||||
// map4 = map3;
|
||||
//
|
||||
// * Move constructor
|
||||
//
|
||||
// // Move is guaranteed efficient
|
||||
// absl::flat_hash_map<int, std::string> map5(std::move(map4));
|
||||
//
|
||||
// * Move assignment operator
|
||||
//
|
||||
// // May be efficient if allocators are compatible
|
||||
// absl::flat_hash_map<int, std::string> map6;
|
||||
// map6 = std::move(map5);
|
||||
//
|
||||
// * Range constructor
|
||||
//
|
||||
// std::vector<std::pair<int, std::string>> v = {{1, "a"}, {2, "b"}};
|
||||
// absl::flat_hash_map<int, std::string> map7(v.begin(), v.end());
|
||||
flat_hash_map() {}
|
||||
using Base::Base;
|
||||
|
||||
// flat_hash_map::begin()
|
||||
//
|
||||
// Returns an iterator to the beginning of the `flat_hash_map`.
|
||||
using Base::begin;
|
||||
|
||||
// flat_hash_map::cbegin()
|
||||
//
|
||||
// Returns a const iterator to the beginning of the `flat_hash_map`.
|
||||
using Base::cbegin;
|
||||
|
||||
// flat_hash_map::cend()
|
||||
//
|
||||
// Returns a const iterator to the end of the `flat_hash_map`.
|
||||
using Base::cend;
|
||||
|
||||
// flat_hash_map::end()
|
||||
//
|
||||
// Returns an iterator to the end of the `flat_hash_map`.
|
||||
using Base::end;
|
||||
|
||||
// flat_hash_map::capacity()
|
||||
//
|
||||
// Returns the number of element slots (assigned, deleted, and empty)
|
||||
// available within the `flat_hash_map`.
|
||||
//
|
||||
// NOTE: this member function is particular to `absl::flat_hash_map` and is
|
||||
// not provided in the `std::unordered_map` API.
|
||||
using Base::capacity;
|
||||
|
||||
// flat_hash_map::empty()
|
||||
//
|
||||
// Returns whether or not the `flat_hash_map` is empty.
|
||||
using Base::empty;
|
||||
|
||||
// flat_hash_map::max_size()
|
||||
//
|
||||
// Returns the largest theoretical possible number of elements within a
|
||||
// `flat_hash_map` under current memory constraints. This value can be thought
|
||||
// of the largest value of `std::distance(begin(), end())` for a
|
||||
// `flat_hash_map<K, V>`.
|
||||
using Base::max_size;
|
||||
|
||||
// flat_hash_map::size()
|
||||
//
|
||||
// Returns the number of elements currently within the `flat_hash_map`.
|
||||
using Base::size;
|
||||
|
||||
// flat_hash_map::clear()
|
||||
//
|
||||
// Removes all elements from the `flat_hash_map`. Invalidates any references,
|
||||
// pointers, or iterators referring to contained elements.
|
||||
//
|
||||
// NOTE: this operation may shrink the underlying buffer. To avoid shrinking
|
||||
// the underlying buffer call `erase(begin(), end())`.
|
||||
using Base::clear;
|
||||
|
||||
// flat_hash_map::erase()
|
||||
//
|
||||
// Erases elements within the `flat_hash_map`. Erasing does not trigger a
|
||||
// rehash. Overloads are listed below.
|
||||
//
|
||||
// void erase(const_iterator pos):
|
||||
//
|
||||
// Erases the element at `position` of the `flat_hash_map`, returning
|
||||
// `void`.
|
||||
//
|
||||
// NOTE: returning `void` in this case is different than that of STL
|
||||
// containers in general and `std::unordered_map` in particular (which
|
||||
// return an iterator to the element following the erased element). If that
|
||||
// iterator is needed, simply post increment the iterator:
|
||||
//
|
||||
// map.erase(it++);
|
||||
//
|
||||
// iterator erase(const_iterator first, const_iterator last):
|
||||
//
|
||||
// Erases the elements in the open interval [`first`, `last`), returning an
|
||||
// iterator pointing to `last`.
|
||||
//
|
||||
// size_type erase(const key_type& key):
|
||||
//
|
||||
// Erases the element with the matching key, if it exists.
|
||||
using Base::erase;
|
||||
|
||||
// flat_hash_map::insert()
|
||||
//
|
||||
// Inserts an element of the specified value into the `flat_hash_map`,
|
||||
// returning an iterator pointing to the newly inserted element, provided that
|
||||
// an element with the given key does not already exist. If rehashing occurs
|
||||
// due to the insertion, all iterators are invalidated. Overloads are listed
|
||||
// below.
|
||||
//
|
||||
// std::pair<iterator,bool> insert(const init_type& value):
|
||||
//
|
||||
// Inserts a value into the `flat_hash_map`. Returns a pair consisting of an
|
||||
// iterator to the inserted element (or to the element that prevented the
|
||||
// insertion) and a bool denoting whether the insertion took place.
|
||||
//
|
||||
// std::pair<iterator,bool> insert(T&& value):
|
||||
// std::pair<iterator,bool> insert(init_type&& value):
|
||||
//
|
||||
// Inserts a moveable value into the `flat_hash_map`. Returns a pair
|
||||
// consisting of an iterator to the inserted element (or to the element that
|
||||
// prevented the insertion) and a bool denoting whether the insertion took
|
||||
// place.
|
||||
//
|
||||
// iterator insert(const_iterator hint, const init_type& value):
|
||||
// iterator insert(const_iterator hint, T&& value):
|
||||
// iterator insert(const_iterator hint, init_type&& value);
|
||||
//
|
||||
// Inserts a value, using the position of `hint` as a non-binding suggestion
|
||||
// for where to begin the insertion search. Returns an iterator to the
|
||||
// inserted element, or to the existing element that prevented the
|
||||
// insertion.
|
||||
//
|
||||
// void insert(InputIterator first, InputIterator last):
|
||||
//
|
||||
// Inserts a range of values [`first`, `last`).
|
||||
//
|
||||
// NOTE: Although the STL does not specify which element may be inserted if
|
||||
// multiple keys compare equivalently, for `flat_hash_map` we guarantee the
|
||||
// first match is inserted.
|
||||
//
|
||||
// void insert(std::initializer_list<init_type> ilist):
|
||||
//
|
||||
// Inserts the elements within the initializer list `ilist`.
|
||||
//
|
||||
// NOTE: Although the STL does not specify which element may be inserted if
|
||||
// multiple keys compare equivalently within the initializer list, for
|
||||
// `flat_hash_map` we guarantee the first match is inserted.
|
||||
using Base::insert;
|
||||
|
||||
// flat_hash_map::insert_or_assign()
|
||||
//
|
||||
// Inserts an element of the specified value into the `flat_hash_map` provided
|
||||
// that a value with the given key does not already exist, or replaces it with
|
||||
// the element value if a key for that value already exists, returning an
|
||||
// iterator pointing to the newly inserted element. If rehashing occurs due
|
||||
// to the insertion, all existing iterators are invalidated. Overloads are
|
||||
// listed below.
|
||||
//
|
||||
// pair<iterator, bool> insert_or_assign(const init_type& k, T&& obj):
|
||||
// pair<iterator, bool> insert_or_assign(init_type&& k, T&& obj):
|
||||
//
|
||||
// Inserts/Assigns (or moves) the element of the specified key into the
|
||||
// `flat_hash_map`.
|
||||
//
|
||||
// iterator insert_or_assign(const_iterator hint,
|
||||
// const init_type& k, T&& obj):
|
||||
// iterator insert_or_assign(const_iterator hint, init_type&& k, T&& obj):
|
||||
//
|
||||
// Inserts/Assigns (or moves) the element of the specified key into the
|
||||
// `flat_hash_map` using the position of `hint` as a non-binding suggestion
|
||||
// for where to begin the insertion search.
|
||||
using Base::insert_or_assign;
|
||||
|
||||
// flat_hash_map::emplace()
|
||||
//
|
||||
// Inserts an element of the specified value by constructing it in-place
|
||||
// within the `flat_hash_map`, provided that no element with the given key
|
||||
// already exists.
|
||||
//
|
||||
// The element may be constructed even if there already is an element with the
|
||||
// key in the container, in which case the newly constructed element will be
|
||||
// destroyed immediately. Prefer `try_emplace()` unless your key is not
|
||||
// copyable or moveable.
|
||||
//
|
||||
// If rehashing occurs due to the insertion, all iterators are invalidated.
|
||||
using Base::emplace;
|
||||
|
||||
// flat_hash_map::emplace_hint()
|
||||
//
|
||||
// Inserts an element of the specified value by constructing it in-place
|
||||
// within the `flat_hash_map`, using the position of `hint` as a non-binding
|
||||
// suggestion for where to begin the insertion search, and only inserts
|
||||
// provided that no element with the given key already exists.
|
||||
//
|
||||
// The element may be constructed even if there already is an element with the
|
||||
// key in the container, in which case the newly constructed element will be
|
||||
// destroyed immediately. Prefer `try_emplace()` unless your key is not
|
||||
// copyable or moveable.
|
||||
//
|
||||
// If rehashing occurs due to the insertion, all iterators are invalidated.
|
||||
using Base::emplace_hint;
|
||||
|
||||
// flat_hash_map::try_emplace()
|
||||
//
|
||||
// Inserts an element of the specified value by constructing it in-place
|
||||
// within the `flat_hash_map`, provided that no element with the given key
|
||||
// already exists. Unlike `emplace()`, if an element with the given key
|
||||
// already exists, we guarantee that no element is constructed.
|
||||
//
|
||||
// If rehashing occurs due to the insertion, all iterators are invalidated.
|
||||
// Overloads are listed below.
|
||||
//
|
||||
// pair<iterator, bool> try_emplace(const key_type& k, Args&&... args):
|
||||
// pair<iterator, bool> try_emplace(key_type&& k, Args&&... args):
|
||||
//
|
||||
// Inserts (via copy or move) the element of the specified key into the
|
||||
// `flat_hash_map`.
|
||||
//
|
||||
// iterator try_emplace(const_iterator hint,
|
||||
// const init_type& k, Args&&... args):
|
||||
// iterator try_emplace(const_iterator hint, init_type&& k, Args&&... args):
|
||||
//
|
||||
// Inserts (via copy or move) the element of the specified key into the
|
||||
// `flat_hash_map` using the position of `hint` as a non-binding suggestion
|
||||
// for where to begin the insertion search.
|
||||
//
|
||||
// All `try_emplace()` overloads make the same guarantees regarding rvalue
|
||||
// arguments as `std::unordered_map::try_emplace()`, namely that these
|
||||
// functions will not move from rvalue arguments if insertions do not happen.
|
||||
using Base::try_emplace;
|
||||
|
||||
// flat_hash_map::extract()
|
||||
//
|
||||
// Extracts the indicated element, erasing it in the process, and returns it
|
||||
// as a C++17-compatible node handle. Overloads are listed below.
|
||||
//
|
||||
// node_type extract(const_iterator position):
|
||||
//
|
||||
// Extracts the key,value pair of the element at the indicated position and
|
||||
// returns a node handle owning that extracted data.
|
||||
//
|
||||
// node_type extract(const key_type& x):
|
||||
//
|
||||
// Extracts the key,value pair of the element with a key matching the passed
|
||||
// key value and returns a node handle owning that extracted data. If the
|
||||
// `flat_hash_map` does not contain an element with a matching key, this
|
||||
// function returns an empty node handle.
|
||||
using Base::extract;
|
||||
|
||||
// flat_hash_map::merge()
|
||||
//
|
||||
// Extracts elements from a given `source` flat hash map into this
|
||||
// `flat_hash_map`. If the destination `flat_hash_map` already contains an
|
||||
// element with an equivalent key, that element is not extracted.
|
||||
using Base::merge;
|
||||
|
||||
// flat_hash_map::swap(flat_hash_map& other)
|
||||
//
|
||||
// Exchanges the contents of this `flat_hash_map` with those of the `other`
|
||||
// flat hash map, avoiding invocation of any move, copy, or swap operations on
|
||||
// individual elements.
|
||||
//
|
||||
// All iterators and references on the `flat_hash_map` remain valid, excepting
|
||||
// for the past-the-end iterator, which is invalidated.
|
||||
//
|
||||
// `swap()` requires that the flat hash map's hashing and key equivalence
|
||||
// functions be Swappable, and are exchanged using unqualified calls to
|
||||
// non-member `swap()`. If the map's allocator has
|
||||
// `std::allocator_traits<allocator_type>::propagate_on_container_swap::value`
|
||||
// set to `true`, the allocators are also exchanged using an unqualified call
|
||||
// to non-member `swap()`; otherwise, the allocators are not swapped.
|
||||
using Base::swap;
|
||||
|
||||
// flat_hash_map::rehash(count)
|
||||
//
|
||||
// Rehashes the `flat_hash_map`, setting the number of slots to be at least
|
||||
// the passed value. If the new number of slots increases the load factor more
|
||||
// than the current maximum load factor
|
||||
// (`count` < `size()` / `max_load_factor()`), then the new number of slots
|
||||
// will be at least `size()` / `max_load_factor()`.
|
||||
//
|
||||
// To force a rehash, pass rehash(0).
|
||||
//
|
||||
// NOTE: unlike behavior in `std::unordered_map`, references are also
|
||||
// invalidated upon a `rehash()`.
|
||||
using Base::rehash;
|
||||
|
||||
// flat_hash_map::reserve(count)
|
||||
//
|
||||
// Sets the number of slots in the `flat_hash_map` to the number needed to
|
||||
// accommodate at least `count` total elements without exceeding the current
|
||||
// maximum load factor, and may rehash the container if needed.
|
||||
using Base::reserve;
|
||||
|
||||
// flat_hash_map::at()
|
||||
//
|
||||
// Returns a reference to the mapped value of the element with key equivalent
|
||||
// to the passed key.
|
||||
using Base::at;
|
||||
|
||||
// flat_hash_map::contains()
|
||||
//
|
||||
// Determines whether an element with a key comparing equal to the given `key`
|
||||
// exists within the `flat_hash_map`, returning `true` if so or `false`
|
||||
// otherwise.
|
||||
using Base::contains;
|
||||
|
||||
// flat_hash_map::count(const Key& key) const
|
||||
//
|
||||
// Returns the number of elements with a key comparing equal to the given
|
||||
// `key` within the `flat_hash_map`. note that this function will return
|
||||
// either `1` or `0` since duplicate keys are not allowed within a
|
||||
// `flat_hash_map`.
|
||||
using Base::count;
|
||||
|
||||
// flat_hash_map::equal_range()
|
||||
//
|
||||
// Returns a closed range [first, last], defined by a `std::pair` of two
|
||||
// iterators, containing all elements with the passed key in the
|
||||
// `flat_hash_map`.
|
||||
using Base::equal_range;
|
||||
|
||||
// flat_hash_map::find()
|
||||
//
|
||||
// Finds an element with the passed `key` within the `flat_hash_map`.
|
||||
using Base::find;
|
||||
|
||||
// flat_hash_map::operator[]()
|
||||
//
|
||||
// Returns a reference to the value mapped to the passed key within the
|
||||
// `flat_hash_map`, performing an `insert()` if the key does not already
|
||||
// exist.
|
||||
//
|
||||
// If an insertion occurs and results in a rehashing of the container, all
|
||||
// iterators are invalidated. Otherwise iterators are not affected and
|
||||
// references are not invalidated. Overloads are listed below.
|
||||
//
|
||||
// T& operator[](const Key& key):
|
||||
//
|
||||
// Inserts an init_type object constructed in-place if the element with the
|
||||
// given key does not exist.
|
||||
//
|
||||
// T& operator[](Key&& key):
|
||||
//
|
||||
// Inserts an init_type object constructed in-place provided that an element
|
||||
// with the given key does not exist.
|
||||
using Base::operator[];
|
||||
|
||||
// flat_hash_map::bucket_count()
|
||||
//
|
||||
// Returns the number of "buckets" within the `flat_hash_map`. Note that
|
||||
// because a flat hash map contains all elements within its internal storage,
|
||||
// this value simply equals the current capacity of the `flat_hash_map`.
|
||||
using Base::bucket_count;
|
||||
|
||||
// flat_hash_map::load_factor()
|
||||
//
|
||||
// Returns the current load factor of the `flat_hash_map` (the average number
|
||||
// of slots occupied with a value within the hash map).
|
||||
using Base::load_factor;
|
||||
|
||||
// flat_hash_map::max_load_factor()
|
||||
//
|
||||
// Manages the maximum load factor of the `flat_hash_map`. Overloads are
|
||||
// listed below.
|
||||
//
|
||||
// float flat_hash_map::max_load_factor()
|
||||
//
|
||||
// Returns the current maximum load factor of the `flat_hash_map`.
|
||||
//
|
||||
// void flat_hash_map::max_load_factor(float ml)
|
||||
//
|
||||
// Sets the maximum load factor of the `flat_hash_map` to the passed value.
|
||||
//
|
||||
// NOTE: This overload is provided only for API compatibility with the STL;
|
||||
// `flat_hash_map` will ignore any set load factor and manage its rehashing
|
||||
// internally as an implementation detail.
|
||||
using Base::max_load_factor;
|
||||
|
||||
// flat_hash_map::get_allocator()
|
||||
//
|
||||
// Returns the allocator function associated with this `flat_hash_map`.
|
||||
using Base::get_allocator;
|
||||
|
||||
// flat_hash_map::hash_function()
|
||||
//
|
||||
// Returns the hashing function used to hash the keys within this
|
||||
// `flat_hash_map`.
|
||||
using Base::hash_function;
|
||||
|
||||
// flat_hash_map::key_eq()
|
||||
//
|
||||
// Returns the function used for comparing keys equality.
|
||||
using Base::key_eq;
|
||||
};
|
||||
|
||||
// erase_if(flat_hash_map<>, Pred)
|
||||
//
|
||||
// Erases all elements that satisfy the predicate `pred` from the container `c`.
|
||||
template <typename K, typename V, typename H, typename E, typename A,
|
||||
typename Predicate>
|
||||
void erase_if(flat_hash_map<K, V, H, E, A>& c, Predicate pred) {
|
||||
container_internal::EraseIf(pred, &c);
|
||||
}
|
||||
|
||||
namespace container_internal {
|
||||
|
||||
template <class K, class V>
|
||||
struct FlatHashMapPolicy {
|
||||
using slot_policy = container_internal::map_slot_policy<K, V>;
|
||||
using slot_type = typename slot_policy::slot_type;
|
||||
using key_type = K;
|
||||
using mapped_type = V;
|
||||
using init_type = std::pair</*non const*/ key_type, mapped_type>;
|
||||
|
||||
template <class Allocator, class... Args>
|
||||
static void construct(Allocator* alloc, slot_type* slot, Args&&... args) {
|
||||
slot_policy::construct(alloc, slot, std::forward<Args>(args)...);
|
||||
}
|
||||
|
||||
template <class Allocator>
|
||||
static void destroy(Allocator* alloc, slot_type* slot) {
|
||||
slot_policy::destroy(alloc, slot);
|
||||
}
|
||||
|
||||
template <class Allocator>
|
||||
static void transfer(Allocator* alloc, slot_type* new_slot,
|
||||
slot_type* old_slot) {
|
||||
slot_policy::transfer(alloc, new_slot, old_slot);
|
||||
}
|
||||
|
||||
template <class F, class... Args>
|
||||
static decltype(absl::container_internal::DecomposePair(
|
||||
std::declval<F>(), std::declval<Args>()...))
|
||||
apply(F&& f, Args&&... args) {
|
||||
return absl::container_internal::DecomposePair(std::forward<F>(f),
|
||||
std::forward<Args>(args)...);
|
||||
}
|
||||
|
||||
static size_t space_used(const slot_type*) { return 0; }
|
||||
|
||||
static std::pair<const K, V>& element(slot_type* slot) { return slot->value; }
|
||||
|
||||
static V& value(std::pair<const K, V>* kv) { return kv->second; }
|
||||
static const V& value(const std::pair<const K, V>* kv) { return kv->second; }
|
||||
};
|
||||
|
||||
} // namespace container_internal
|
||||
|
||||
namespace container_algorithm_internal {
|
||||
|
||||
// Specialization of trait in absl/algorithm/container.h
|
||||
template <class Key, class T, class Hash, class KeyEqual, class Allocator>
|
||||
struct IsUnorderedContainer<
|
||||
absl::flat_hash_map<Key, T, Hash, KeyEqual, Allocator>> : std::true_type {};
|
||||
|
||||
} // namespace container_algorithm_internal
|
||||
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_FLAT_HASH_MAP_H_
|
||||
+848
@@ -0,0 +1,848 @@
|
||||
// Copyright 2019 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: inlined_vector.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This header file contains the declaration and definition of an "inlined
|
||||
// vector" which behaves in an equivalent fashion to a `std::vector`, except
|
||||
// that storage for small sequences of the vector are provided inline without
|
||||
// requiring any heap allocation.
|
||||
//
|
||||
// An `absl::InlinedVector<T, N>` specifies the default capacity `N` as one of
|
||||
// its template parameters. Instances where `size() <= N` hold contained
|
||||
// elements in inline space. Typically `N` is very small so that sequences that
|
||||
// are expected to be short do not require allocations.
|
||||
//
|
||||
// An `absl::InlinedVector` does not usually require a specific allocator. If
|
||||
// the inlined vector grows beyond its initial constraints, it will need to
|
||||
// allocate (as any normal `std::vector` would). This is usually performed with
|
||||
// the default allocator (defined as `std::allocator<T>`). Optionally, a custom
|
||||
// allocator type may be specified as `A` in `absl::InlinedVector<T, N, A>`.
|
||||
|
||||
#ifndef ABSL_CONTAINER_INLINED_VECTOR_H_
|
||||
#define ABSL_CONTAINER_INLINED_VECTOR_H_
|
||||
|
||||
#include <algorithm>
|
||||
#include <cassert>
|
||||
#include <cstddef>
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
#include <initializer_list>
|
||||
#include <iterator>
|
||||
#include <memory>
|
||||
#include <type_traits>
|
||||
#include <utility>
|
||||
|
||||
#include "absl/algorithm/algorithm.h"
|
||||
#include "absl/base/internal/throw_delegate.h"
|
||||
#include "absl/base/optimization.h"
|
||||
#include "absl/base/port.h"
|
||||
#include "absl/container/internal/inlined_vector.h"
|
||||
#include "absl/memory/memory.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
// -----------------------------------------------------------------------------
|
||||
// InlinedVector
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// An `absl::InlinedVector` is designed to be a drop-in replacement for
|
||||
// `std::vector` for use cases where the vector's size is sufficiently small
|
||||
// that it can be inlined. If the inlined vector does grow beyond its estimated
|
||||
// capacity, it will trigger an initial allocation on the heap, and will behave
|
||||
// as a `std:vector`. The API of the `absl::InlinedVector` within this file is
|
||||
// designed to cover the same API footprint as covered by `std::vector`.
|
||||
template <typename T, size_t N, typename A = std::allocator<T>>
|
||||
class InlinedVector {
|
||||
static_assert(N > 0, "`absl::InlinedVector` requires an inlined capacity.");
|
||||
|
||||
using Storage = inlined_vector_internal::Storage<T, N, A>;
|
||||
|
||||
using AllocatorTraits = typename Storage::AllocatorTraits;
|
||||
using RValueReference = typename Storage::RValueReference;
|
||||
using MoveIterator = typename Storage::MoveIterator;
|
||||
using IsMemcpyOk = typename Storage::IsMemcpyOk;
|
||||
|
||||
template <typename Iterator>
|
||||
using IteratorValueAdapter =
|
||||
typename Storage::template IteratorValueAdapter<Iterator>;
|
||||
using CopyValueAdapter = typename Storage::CopyValueAdapter;
|
||||
using DefaultValueAdapter = typename Storage::DefaultValueAdapter;
|
||||
|
||||
template <typename Iterator>
|
||||
using EnableIfAtLeastForwardIterator = absl::enable_if_t<
|
||||
inlined_vector_internal::IsAtLeastForwardIterator<Iterator>::value>;
|
||||
template <typename Iterator>
|
||||
using DisableIfAtLeastForwardIterator = absl::enable_if_t<
|
||||
!inlined_vector_internal::IsAtLeastForwardIterator<Iterator>::value>;
|
||||
|
||||
public:
|
||||
using allocator_type = typename Storage::allocator_type;
|
||||
using value_type = typename Storage::value_type;
|
||||
using pointer = typename Storage::pointer;
|
||||
using const_pointer = typename Storage::const_pointer;
|
||||
using size_type = typename Storage::size_type;
|
||||
using difference_type = typename Storage::difference_type;
|
||||
using reference = typename Storage::reference;
|
||||
using const_reference = typename Storage::const_reference;
|
||||
using iterator = typename Storage::iterator;
|
||||
using const_iterator = typename Storage::const_iterator;
|
||||
using reverse_iterator = typename Storage::reverse_iterator;
|
||||
using const_reverse_iterator = typename Storage::const_reverse_iterator;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// InlinedVector Constructors and Destructor
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// Creates an empty inlined vector with a value-initialized allocator.
|
||||
InlinedVector() noexcept(noexcept(allocator_type())) : storage_() {}
|
||||
|
||||
// Creates an empty inlined vector with a copy of `alloc`.
|
||||
explicit InlinedVector(const allocator_type& alloc) noexcept
|
||||
: storage_(alloc) {}
|
||||
|
||||
// Creates an inlined vector with `n` copies of `value_type()`.
|
||||
explicit InlinedVector(size_type n,
|
||||
const allocator_type& alloc = allocator_type())
|
||||
: storage_(alloc) {
|
||||
storage_.Initialize(DefaultValueAdapter(), n);
|
||||
}
|
||||
|
||||
// Creates an inlined vector with `n` copies of `v`.
|
||||
InlinedVector(size_type n, const_reference v,
|
||||
const allocator_type& alloc = allocator_type())
|
||||
: storage_(alloc) {
|
||||
storage_.Initialize(CopyValueAdapter(v), n);
|
||||
}
|
||||
|
||||
// Creates an inlined vector with copies of the elements of `list`.
|
||||
InlinedVector(std::initializer_list<value_type> list,
|
||||
const allocator_type& alloc = allocator_type())
|
||||
: InlinedVector(list.begin(), list.end(), alloc) {}
|
||||
|
||||
// Creates an inlined vector with elements constructed from the provided
|
||||
// forward iterator range [`first`, `last`).
|
||||
//
|
||||
// NOTE: the `enable_if` prevents ambiguous interpretation between a call to
|
||||
// this constructor with two integral arguments and a call to the above
|
||||
// `InlinedVector(size_type, const_reference)` constructor.
|
||||
template <typename ForwardIterator,
|
||||
EnableIfAtLeastForwardIterator<ForwardIterator>* = nullptr>
|
||||
InlinedVector(ForwardIterator first, ForwardIterator last,
|
||||
const allocator_type& alloc = allocator_type())
|
||||
: storage_(alloc) {
|
||||
storage_.Initialize(IteratorValueAdapter<ForwardIterator>(first),
|
||||
std::distance(first, last));
|
||||
}
|
||||
|
||||
// Creates an inlined vector with elements constructed from the provided input
|
||||
// iterator range [`first`, `last`).
|
||||
template <typename InputIterator,
|
||||
DisableIfAtLeastForwardIterator<InputIterator>* = nullptr>
|
||||
InlinedVector(InputIterator first, InputIterator last,
|
||||
const allocator_type& alloc = allocator_type())
|
||||
: storage_(alloc) {
|
||||
std::copy(first, last, std::back_inserter(*this));
|
||||
}
|
||||
|
||||
// Creates an inlined vector by copying the contents of `other` using
|
||||
// `other`'s allocator.
|
||||
InlinedVector(const InlinedVector& other)
|
||||
: InlinedVector(other, *other.storage_.GetAllocPtr()) {}
|
||||
|
||||
// Creates an inlined vector by copying the contents of `other` using `alloc`.
|
||||
InlinedVector(const InlinedVector& other, const allocator_type& alloc)
|
||||
: storage_(alloc) {
|
||||
if (IsMemcpyOk::value && !other.storage_.GetIsAllocated()) {
|
||||
storage_.MemcpyFrom(other.storage_);
|
||||
} else {
|
||||
storage_.Initialize(IteratorValueAdapter<const_pointer>(other.data()),
|
||||
other.size());
|
||||
}
|
||||
}
|
||||
|
||||
// Creates an inlined vector by moving in the contents of `other` without
|
||||
// allocating. If `other` contains allocated memory, the newly-created inlined
|
||||
// vector will take ownership of that memory. However, if `other` does not
|
||||
// contain allocated memory, the newly-created inlined vector will perform
|
||||
// element-wise move construction of the contents of `other`.
|
||||
//
|
||||
// NOTE: since no allocation is performed for the inlined vector in either
|
||||
// case, the `noexcept(...)` specification depends on whether moving the
|
||||
// underlying objects can throw. It is assumed assumed that...
|
||||
// a) move constructors should only throw due to allocation failure.
|
||||
// b) if `value_type`'s move constructor allocates, it uses the same
|
||||
// allocation function as the inlined vector's allocator.
|
||||
// Thus, the move constructor is non-throwing if the allocator is non-throwing
|
||||
// or `value_type`'s move constructor is specified as `noexcept`.
|
||||
InlinedVector(InlinedVector&& other) noexcept(
|
||||
absl::allocator_is_nothrow<allocator_type>::value ||
|
||||
std::is_nothrow_move_constructible<value_type>::value)
|
||||
: storage_(*other.storage_.GetAllocPtr()) {
|
||||
if (IsMemcpyOk::value) {
|
||||
storage_.MemcpyFrom(other.storage_);
|
||||
|
||||
other.storage_.SetInlinedSize(0);
|
||||
} else if (other.storage_.GetIsAllocated()) {
|
||||
storage_.SetAllocatedData(other.storage_.GetAllocatedData(),
|
||||
other.storage_.GetAllocatedCapacity());
|
||||
storage_.SetAllocatedSize(other.storage_.GetSize());
|
||||
|
||||
other.storage_.SetInlinedSize(0);
|
||||
} else {
|
||||
IteratorValueAdapter<MoveIterator> other_values(
|
||||
MoveIterator(other.storage_.GetInlinedData()));
|
||||
|
||||
inlined_vector_internal::ConstructElements(
|
||||
storage_.GetAllocPtr(), storage_.GetInlinedData(), &other_values,
|
||||
other.storage_.GetSize());
|
||||
|
||||
storage_.SetInlinedSize(other.storage_.GetSize());
|
||||
}
|
||||
}
|
||||
|
||||
// Creates an inlined vector by moving in the contents of `other` with a copy
|
||||
// of `alloc`.
|
||||
//
|
||||
// NOTE: if `other`'s allocator is not equal to `alloc`, even if `other`
|
||||
// contains allocated memory, this move constructor will still allocate. Since
|
||||
// allocation is performed, this constructor can only be `noexcept` if the
|
||||
// specified allocator is also `noexcept`.
|
||||
InlinedVector(InlinedVector&& other, const allocator_type& alloc) noexcept(
|
||||
absl::allocator_is_nothrow<allocator_type>::value)
|
||||
: storage_(alloc) {
|
||||
if (IsMemcpyOk::value) {
|
||||
storage_.MemcpyFrom(other.storage_);
|
||||
|
||||
other.storage_.SetInlinedSize(0);
|
||||
} else if ((*storage_.GetAllocPtr() == *other.storage_.GetAllocPtr()) &&
|
||||
other.storage_.GetIsAllocated()) {
|
||||
storage_.SetAllocatedData(other.storage_.GetAllocatedData(),
|
||||
other.storage_.GetAllocatedCapacity());
|
||||
storage_.SetAllocatedSize(other.storage_.GetSize());
|
||||
|
||||
other.storage_.SetInlinedSize(0);
|
||||
} else {
|
||||
storage_.Initialize(
|
||||
IteratorValueAdapter<MoveIterator>(MoveIterator(other.data())),
|
||||
other.size());
|
||||
}
|
||||
}
|
||||
|
||||
~InlinedVector() {}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// InlinedVector Member Accessors
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// `InlinedVector::empty()`
|
||||
//
|
||||
// Returns whether the inlined vector contains no elements.
|
||||
bool empty() const noexcept { return !size(); }
|
||||
|
||||
// `InlinedVector::size()`
|
||||
//
|
||||
// Returns the number of elements in the inlined vector.
|
||||
size_type size() const noexcept { return storage_.GetSize(); }
|
||||
|
||||
// `InlinedVector::max_size()`
|
||||
//
|
||||
// Returns the maximum number of elements the inlined vector can hold.
|
||||
size_type max_size() const noexcept {
|
||||
// One bit of the size storage is used to indicate whether the inlined
|
||||
// vector contains allocated memory. As a result, the maximum size that the
|
||||
// inlined vector can express is half of the max for `size_type`.
|
||||
return (std::numeric_limits<size_type>::max)() / 2;
|
||||
}
|
||||
|
||||
// `InlinedVector::capacity()`
|
||||
//
|
||||
// Returns the number of elements that could be stored in the inlined vector
|
||||
// without requiring a reallocation.
|
||||
//
|
||||
// NOTE: for most inlined vectors, `capacity()` should be equal to the
|
||||
// template parameter `N`. For inlined vectors which exceed this capacity,
|
||||
// they will no longer be inlined and `capacity()` will equal the capactity of
|
||||
// the allocated memory.
|
||||
size_type capacity() const noexcept {
|
||||
return storage_.GetIsAllocated() ? storage_.GetAllocatedCapacity()
|
||||
: storage_.GetInlinedCapacity();
|
||||
}
|
||||
|
||||
// `InlinedVector::data()`
|
||||
//
|
||||
// Returns a `pointer` to the elements of the inlined vector. This pointer
|
||||
// can be used to access and modify the contained elements.
|
||||
//
|
||||
// NOTE: only elements within [`data()`, `data() + size()`) are valid.
|
||||
pointer data() noexcept {
|
||||
return storage_.GetIsAllocated() ? storage_.GetAllocatedData()
|
||||
: storage_.GetInlinedData();
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::data()` that returns a `const_pointer` to the
|
||||
// elements of the inlined vector. This pointer can be used to access but not
|
||||
// modify the contained elements.
|
||||
//
|
||||
// NOTE: only elements within [`data()`, `data() + size()`) are valid.
|
||||
const_pointer data() const noexcept {
|
||||
return storage_.GetIsAllocated() ? storage_.GetAllocatedData()
|
||||
: storage_.GetInlinedData();
|
||||
}
|
||||
|
||||
// `InlinedVector::operator[](...)`
|
||||
//
|
||||
// Returns a `reference` to the `i`th element of the inlined vector.
|
||||
reference operator[](size_type i) {
|
||||
assert(i < size());
|
||||
|
||||
return data()[i];
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::operator[](...)` that returns a
|
||||
// `const_reference` to the `i`th element of the inlined vector.
|
||||
const_reference operator[](size_type i) const {
|
||||
assert(i < size());
|
||||
|
||||
return data()[i];
|
||||
}
|
||||
|
||||
// `InlinedVector::at(...)`
|
||||
//
|
||||
// Returns a `reference` to the `i`th element of the inlined vector.
|
||||
//
|
||||
// NOTE: if `i` is not within the required range of `InlinedVector::at(...)`,
|
||||
// in both debug and non-debug builds, `std::out_of_range` will be thrown.
|
||||
reference at(size_type i) {
|
||||
if (ABSL_PREDICT_FALSE(i >= size())) {
|
||||
base_internal::ThrowStdOutOfRange(
|
||||
"`InlinedVector::at(size_type)` failed bounds check");
|
||||
}
|
||||
|
||||
return data()[i];
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::at(...)` that returns a `const_reference` to
|
||||
// the `i`th element of the inlined vector.
|
||||
//
|
||||
// NOTE: if `i` is not within the required range of `InlinedVector::at(...)`,
|
||||
// in both debug and non-debug builds, `std::out_of_range` will be thrown.
|
||||
const_reference at(size_type i) const {
|
||||
if (ABSL_PREDICT_FALSE(i >= size())) {
|
||||
base_internal::ThrowStdOutOfRange(
|
||||
"`InlinedVector::at(size_type) const` failed bounds check");
|
||||
}
|
||||
|
||||
return data()[i];
|
||||
}
|
||||
|
||||
// `InlinedVector::front()`
|
||||
//
|
||||
// Returns a `reference` to the first element of the inlined vector.
|
||||
reference front() {
|
||||
assert(!empty());
|
||||
|
||||
return at(0);
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::front()` that returns a `const_reference` to
|
||||
// the first element of the inlined vector.
|
||||
const_reference front() const {
|
||||
assert(!empty());
|
||||
|
||||
return at(0);
|
||||
}
|
||||
|
||||
// `InlinedVector::back()`
|
||||
//
|
||||
// Returns a `reference` to the last element of the inlined vector.
|
||||
reference back() {
|
||||
assert(!empty());
|
||||
|
||||
return at(size() - 1);
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::back()` that returns a `const_reference` to the
|
||||
// last element of the inlined vector.
|
||||
const_reference back() const {
|
||||
assert(!empty());
|
||||
|
||||
return at(size() - 1);
|
||||
}
|
||||
|
||||
// `InlinedVector::begin()`
|
||||
//
|
||||
// Returns an `iterator` to the beginning of the inlined vector.
|
||||
iterator begin() noexcept { return data(); }
|
||||
|
||||
// Overload of `InlinedVector::begin()` that returns a `const_iterator` to
|
||||
// the beginning of the inlined vector.
|
||||
const_iterator begin() const noexcept { return data(); }
|
||||
|
||||
// `InlinedVector::end()`
|
||||
//
|
||||
// Returns an `iterator` to the end of the inlined vector.
|
||||
iterator end() noexcept { return data() + size(); }
|
||||
|
||||
// Overload of `InlinedVector::end()` that returns a `const_iterator` to the
|
||||
// end of the inlined vector.
|
||||
const_iterator end() const noexcept { return data() + size(); }
|
||||
|
||||
// `InlinedVector::cbegin()`
|
||||
//
|
||||
// Returns a `const_iterator` to the beginning of the inlined vector.
|
||||
const_iterator cbegin() const noexcept { return begin(); }
|
||||
|
||||
// `InlinedVector::cend()`
|
||||
//
|
||||
// Returns a `const_iterator` to the end of the inlined vector.
|
||||
const_iterator cend() const noexcept { return end(); }
|
||||
|
||||
// `InlinedVector::rbegin()`
|
||||
//
|
||||
// Returns a `reverse_iterator` from the end of the inlined vector.
|
||||
reverse_iterator rbegin() noexcept { return reverse_iterator(end()); }
|
||||
|
||||
// Overload of `InlinedVector::rbegin()` that returns a
|
||||
// `const_reverse_iterator` from the end of the inlined vector.
|
||||
const_reverse_iterator rbegin() const noexcept {
|
||||
return const_reverse_iterator(end());
|
||||
}
|
||||
|
||||
// `InlinedVector::rend()`
|
||||
//
|
||||
// Returns a `reverse_iterator` from the beginning of the inlined vector.
|
||||
reverse_iterator rend() noexcept { return reverse_iterator(begin()); }
|
||||
|
||||
// Overload of `InlinedVector::rend()` that returns a `const_reverse_iterator`
|
||||
// from the beginning of the inlined vector.
|
||||
const_reverse_iterator rend() const noexcept {
|
||||
return const_reverse_iterator(begin());
|
||||
}
|
||||
|
||||
// `InlinedVector::crbegin()`
|
||||
//
|
||||
// Returns a `const_reverse_iterator` from the end of the inlined vector.
|
||||
const_reverse_iterator crbegin() const noexcept { return rbegin(); }
|
||||
|
||||
// `InlinedVector::crend()`
|
||||
//
|
||||
// Returns a `const_reverse_iterator` from the beginning of the inlined
|
||||
// vector.
|
||||
const_reverse_iterator crend() const noexcept { return rend(); }
|
||||
|
||||
// `InlinedVector::get_allocator()`
|
||||
//
|
||||
// Returns a copy of the inlined vector's allocator.
|
||||
allocator_type get_allocator() const { return *storage_.GetAllocPtr(); }
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// InlinedVector Member Mutators
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// `InlinedVector::operator=(...)`
|
||||
//
|
||||
// Replaces the elements of the inlined vector with copies of the elements of
|
||||
// `list`.
|
||||
InlinedVector& operator=(std::initializer_list<value_type> list) {
|
||||
assign(list.begin(), list.end());
|
||||
|
||||
return *this;
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::operator=(...)` that replaces the elements of
|
||||
// the inlined vector with copies of the elements of `other`.
|
||||
InlinedVector& operator=(const InlinedVector& other) {
|
||||
if (ABSL_PREDICT_TRUE(this != std::addressof(other))) {
|
||||
const_pointer other_data = other.data();
|
||||
assign(other_data, other_data + other.size());
|
||||
}
|
||||
|
||||
return *this;
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::operator=(...)` that moves the elements of
|
||||
// `other` into the inlined vector.
|
||||
//
|
||||
// NOTE: as a result of calling this overload, `other` is left in a valid but
|
||||
// unspecified state.
|
||||
InlinedVector& operator=(InlinedVector&& other) {
|
||||
if (ABSL_PREDICT_TRUE(this != std::addressof(other))) {
|
||||
if (IsMemcpyOk::value || other.storage_.GetIsAllocated()) {
|
||||
inlined_vector_internal::DestroyElements(storage_.GetAllocPtr(), data(),
|
||||
size());
|
||||
storage_.DeallocateIfAllocated();
|
||||
storage_.MemcpyFrom(other.storage_);
|
||||
|
||||
other.storage_.SetInlinedSize(0);
|
||||
} else {
|
||||
storage_.Assign(IteratorValueAdapter<MoveIterator>(
|
||||
MoveIterator(other.storage_.GetInlinedData())),
|
||||
other.size());
|
||||
}
|
||||
}
|
||||
|
||||
return *this;
|
||||
}
|
||||
|
||||
// `InlinedVector::assign(...)`
|
||||
//
|
||||
// Replaces the contents of the inlined vector with `n` copies of `v`.
|
||||
void assign(size_type n, const_reference v) {
|
||||
storage_.Assign(CopyValueAdapter(v), n);
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::assign(...)` that replaces the contents of the
|
||||
// inlined vector with copies of the elements of `list`.
|
||||
void assign(std::initializer_list<value_type> list) {
|
||||
assign(list.begin(), list.end());
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::assign(...)` to replace the contents of the
|
||||
// inlined vector with the range [`first`, `last`).
|
||||
//
|
||||
// NOTE: this overload is for iterators that are "forward" category or better.
|
||||
template <typename ForwardIterator,
|
||||
EnableIfAtLeastForwardIterator<ForwardIterator>* = nullptr>
|
||||
void assign(ForwardIterator first, ForwardIterator last) {
|
||||
storage_.Assign(IteratorValueAdapter<ForwardIterator>(first),
|
||||
std::distance(first, last));
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::assign(...)` to replace the contents of the
|
||||
// inlined vector with the range [`first`, `last`).
|
||||
//
|
||||
// NOTE: this overload is for iterators that are "input" category.
|
||||
template <typename InputIterator,
|
||||
DisableIfAtLeastForwardIterator<InputIterator>* = nullptr>
|
||||
void assign(InputIterator first, InputIterator last) {
|
||||
size_type i = 0;
|
||||
for (; i < size() && first != last; ++i, static_cast<void>(++first)) {
|
||||
at(i) = *first;
|
||||
}
|
||||
|
||||
erase(data() + i, data() + size());
|
||||
std::copy(first, last, std::back_inserter(*this));
|
||||
}
|
||||
|
||||
// `InlinedVector::resize(...)`
|
||||
//
|
||||
// Resizes the inlined vector to contain `n` elements.
|
||||
//
|
||||
// NOTE: if `n` is smaller than `size()`, extra elements are destroyed. If `n`
|
||||
// is larger than `size()`, new elements are value-initialized.
|
||||
void resize(size_type n) { storage_.Resize(DefaultValueAdapter(), n); }
|
||||
|
||||
// Overload of `InlinedVector::resize(...)` that resizes the inlined vector to
|
||||
// contain `n` elements.
|
||||
//
|
||||
// NOTE: if `n` is smaller than `size()`, extra elements are destroyed. If `n`
|
||||
// is larger than `size()`, new elements are copied-constructed from `v`.
|
||||
void resize(size_type n, const_reference v) {
|
||||
storage_.Resize(CopyValueAdapter(v), n);
|
||||
}
|
||||
|
||||
// `InlinedVector::insert(...)`
|
||||
//
|
||||
// Inserts a copy of `v` at `pos`, returning an `iterator` to the newly
|
||||
// inserted element.
|
||||
iterator insert(const_iterator pos, const_reference v) {
|
||||
return emplace(pos, v);
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::insert(...)` that inserts `v` at `pos` using
|
||||
// move semantics, returning an `iterator` to the newly inserted element.
|
||||
iterator insert(const_iterator pos, RValueReference v) {
|
||||
return emplace(pos, std::move(v));
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::insert(...)` that inserts `n` contiguous copies
|
||||
// of `v` starting at `pos`, returning an `iterator` pointing to the first of
|
||||
// the newly inserted elements.
|
||||
iterator insert(const_iterator pos, size_type n, const_reference v) {
|
||||
assert(pos >= begin());
|
||||
assert(pos <= end());
|
||||
|
||||
if (ABSL_PREDICT_TRUE(n != 0)) {
|
||||
value_type dealias = v;
|
||||
return storage_.Insert(pos, CopyValueAdapter(dealias), n);
|
||||
} else {
|
||||
return const_cast<iterator>(pos);
|
||||
}
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::insert(...)` that inserts copies of the
|
||||
// elements of `list` starting at `pos`, returning an `iterator` pointing to
|
||||
// the first of the newly inserted elements.
|
||||
iterator insert(const_iterator pos, std::initializer_list<value_type> list) {
|
||||
return insert(pos, list.begin(), list.end());
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::insert(...)` that inserts the range [`first`,
|
||||
// `last`) starting at `pos`, returning an `iterator` pointing to the first
|
||||
// of the newly inserted elements.
|
||||
//
|
||||
// NOTE: this overload is for iterators that are "forward" category or better.
|
||||
template <typename ForwardIterator,
|
||||
EnableIfAtLeastForwardIterator<ForwardIterator>* = nullptr>
|
||||
iterator insert(const_iterator pos, ForwardIterator first,
|
||||
ForwardIterator last) {
|
||||
assert(pos >= begin());
|
||||
assert(pos <= end());
|
||||
|
||||
if (ABSL_PREDICT_TRUE(first != last)) {
|
||||
return storage_.Insert(pos, IteratorValueAdapter<ForwardIterator>(first),
|
||||
std::distance(first, last));
|
||||
} else {
|
||||
return const_cast<iterator>(pos);
|
||||
}
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::insert(...)` that inserts the range [`first`,
|
||||
// `last`) starting at `pos`, returning an `iterator` pointing to the first
|
||||
// of the newly inserted elements.
|
||||
//
|
||||
// NOTE: this overload is for iterators that are "input" category.
|
||||
template <typename InputIterator,
|
||||
DisableIfAtLeastForwardIterator<InputIterator>* = nullptr>
|
||||
iterator insert(const_iterator pos, InputIterator first, InputIterator last) {
|
||||
assert(pos >= begin());
|
||||
assert(pos <= end());
|
||||
|
||||
size_type index = std::distance(cbegin(), pos);
|
||||
for (size_type i = index; first != last; ++i, static_cast<void>(++first)) {
|
||||
insert(data() + i, *first);
|
||||
}
|
||||
|
||||
return iterator(data() + index);
|
||||
}
|
||||
|
||||
// `InlinedVector::emplace(...)`
|
||||
//
|
||||
// Constructs and inserts an element using `args...` in the inlined vector at
|
||||
// `pos`, returning an `iterator` pointing to the newly emplaced element.
|
||||
template <typename... Args>
|
||||
iterator emplace(const_iterator pos, Args&&... args) {
|
||||
assert(pos >= begin());
|
||||
assert(pos <= end());
|
||||
|
||||
value_type dealias(std::forward<Args>(args)...);
|
||||
return storage_.Insert(pos,
|
||||
IteratorValueAdapter<MoveIterator>(
|
||||
MoveIterator(std::addressof(dealias))),
|
||||
1);
|
||||
}
|
||||
|
||||
// `InlinedVector::emplace_back(...)`
|
||||
//
|
||||
// Constructs and inserts an element using `args...` in the inlined vector at
|
||||
// `end()`, returning a `reference` to the newly emplaced element.
|
||||
template <typename... Args>
|
||||
reference emplace_back(Args&&... args) {
|
||||
return storage_.EmplaceBack(std::forward<Args>(args)...);
|
||||
}
|
||||
|
||||
// `InlinedVector::push_back(...)`
|
||||
//
|
||||
// Inserts a copy of `v` in the inlined vector at `end()`.
|
||||
void push_back(const_reference v) { static_cast<void>(emplace_back(v)); }
|
||||
|
||||
// Overload of `InlinedVector::push_back(...)` for inserting `v` at `end()`
|
||||
// using move semantics.
|
||||
void push_back(RValueReference v) {
|
||||
static_cast<void>(emplace_back(std::move(v)));
|
||||
}
|
||||
|
||||
// `InlinedVector::pop_back()`
|
||||
//
|
||||
// Destroys the element at `back()`, reducing the size by `1`.
|
||||
void pop_back() noexcept {
|
||||
assert(!empty());
|
||||
|
||||
AllocatorTraits::destroy(*storage_.GetAllocPtr(), data() + (size() - 1));
|
||||
storage_.SubtractSize(1);
|
||||
}
|
||||
|
||||
// `InlinedVector::erase(...)`
|
||||
//
|
||||
// Erases the element at `pos`, returning an `iterator` pointing to where the
|
||||
// erased element was located.
|
||||
//
|
||||
// NOTE: may return `end()`, which is not dereferencable.
|
||||
iterator erase(const_iterator pos) {
|
||||
assert(pos >= begin());
|
||||
assert(pos < end());
|
||||
|
||||
return storage_.Erase(pos, pos + 1);
|
||||
}
|
||||
|
||||
// Overload of `InlinedVector::erase(...)` that erases every element in the
|
||||
// range [`from`, `to`), returning an `iterator` pointing to where the first
|
||||
// erased element was located.
|
||||
//
|
||||
// NOTE: may return `end()`, which is not dereferencable.
|
||||
iterator erase(const_iterator from, const_iterator to) {
|
||||
assert(from >= begin());
|
||||
assert(from <= to);
|
||||
assert(to <= end());
|
||||
|
||||
if (ABSL_PREDICT_TRUE(from != to)) {
|
||||
return storage_.Erase(from, to);
|
||||
} else {
|
||||
return const_cast<iterator>(from);
|
||||
}
|
||||
}
|
||||
|
||||
// `InlinedVector::clear()`
|
||||
//
|
||||
// Destroys all elements in the inlined vector, setting the size to `0` and
|
||||
// deallocating any held memory.
|
||||
void clear() noexcept {
|
||||
inlined_vector_internal::DestroyElements(storage_.GetAllocPtr(), data(),
|
||||
size());
|
||||
storage_.DeallocateIfAllocated();
|
||||
|
||||
storage_.SetInlinedSize(0);
|
||||
}
|
||||
|
||||
// `InlinedVector::reserve(...)`
|
||||
//
|
||||
// Ensures that there is enough room for at least `n` elements.
|
||||
void reserve(size_type n) { storage_.Reserve(n); }
|
||||
|
||||
// `InlinedVector::shrink_to_fit()`
|
||||
//
|
||||
// Reduces memory usage by freeing unused memory. After being called, calls to
|
||||
// `capacity()` will be equal to `max(N, size())`.
|
||||
//
|
||||
// If `size() <= N` and the inlined vector contains allocated memory, the
|
||||
// elements will all be moved to the inlined space and the allocated memory
|
||||
// will be deallocated.
|
||||
//
|
||||
// If `size() > N` and `size() < capacity()`, the elements will be moved to a
|
||||
// smaller allocation.
|
||||
void shrink_to_fit() {
|
||||
if (storage_.GetIsAllocated()) {
|
||||
storage_.ShrinkToFit();
|
||||
}
|
||||
}
|
||||
|
||||
// `InlinedVector::swap(...)`
|
||||
//
|
||||
// Swaps the contents of the inlined vector with `other`.
|
||||
void swap(InlinedVector& other) {
|
||||
if (ABSL_PREDICT_TRUE(this != std::addressof(other))) {
|
||||
storage_.Swap(std::addressof(other.storage_));
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
template <typename H, typename TheT, size_t TheN, typename TheA>
|
||||
friend H AbslHashValue(H h, const absl::InlinedVector<TheT, TheN, TheA>& a);
|
||||
|
||||
Storage storage_;
|
||||
};
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// InlinedVector Non-Member Functions
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
// `swap(...)`
|
||||
//
|
||||
// Swaps the contents of two inlined vectors.
|
||||
template <typename T, size_t N, typename A>
|
||||
void swap(absl::InlinedVector<T, N, A>& a,
|
||||
absl::InlinedVector<T, N, A>& b) noexcept(noexcept(a.swap(b))) {
|
||||
a.swap(b);
|
||||
}
|
||||
|
||||
// `operator==(...)`
|
||||
//
|
||||
// Tests for value-equality of two inlined vectors.
|
||||
template <typename T, size_t N, typename A>
|
||||
bool operator==(const absl::InlinedVector<T, N, A>& a,
|
||||
const absl::InlinedVector<T, N, A>& b) {
|
||||
auto a_data = a.data();
|
||||
auto b_data = b.data();
|
||||
return absl::equal(a_data, a_data + a.size(), b_data, b_data + b.size());
|
||||
}
|
||||
|
||||
// `operator!=(...)`
|
||||
//
|
||||
// Tests for value-inequality of two inlined vectors.
|
||||
template <typename T, size_t N, typename A>
|
||||
bool operator!=(const absl::InlinedVector<T, N, A>& a,
|
||||
const absl::InlinedVector<T, N, A>& b) {
|
||||
return !(a == b);
|
||||
}
|
||||
|
||||
// `operator<(...)`
|
||||
//
|
||||
// Tests whether the value of an inlined vector is less than the value of
|
||||
// another inlined vector using a lexicographical comparison algorithm.
|
||||
template <typename T, size_t N, typename A>
|
||||
bool operator<(const absl::InlinedVector<T, N, A>& a,
|
||||
const absl::InlinedVector<T, N, A>& b) {
|
||||
auto a_data = a.data();
|
||||
auto b_data = b.data();
|
||||
return std::lexicographical_compare(a_data, a_data + a.size(), b_data,
|
||||
b_data + b.size());
|
||||
}
|
||||
|
||||
// `operator>(...)`
|
||||
//
|
||||
// Tests whether the value of an inlined vector is greater than the value of
|
||||
// another inlined vector using a lexicographical comparison algorithm.
|
||||
template <typename T, size_t N, typename A>
|
||||
bool operator>(const absl::InlinedVector<T, N, A>& a,
|
||||
const absl::InlinedVector<T, N, A>& b) {
|
||||
return b < a;
|
||||
}
|
||||
|
||||
// `operator<=(...)`
|
||||
//
|
||||
// Tests whether the value of an inlined vector is less than or equal to the
|
||||
// value of another inlined vector using a lexicographical comparison algorithm.
|
||||
template <typename T, size_t N, typename A>
|
||||
bool operator<=(const absl::InlinedVector<T, N, A>& a,
|
||||
const absl::InlinedVector<T, N, A>& b) {
|
||||
return !(b < a);
|
||||
}
|
||||
|
||||
// `operator>=(...)`
|
||||
//
|
||||
// Tests whether the value of an inlined vector is greater than or equal to the
|
||||
// value of another inlined vector using a lexicographical comparison algorithm.
|
||||
template <typename T, size_t N, typename A>
|
||||
bool operator>=(const absl::InlinedVector<T, N, A>& a,
|
||||
const absl::InlinedVector<T, N, A>& b) {
|
||||
return !(a < b);
|
||||
}
|
||||
|
||||
// `AbslHashValue(...)`
|
||||
//
|
||||
// Provides `absl::Hash` support for `absl::InlinedVector`. It is uncommon to
|
||||
// call this directly.
|
||||
template <typename H, typename T, size_t N, typename A>
|
||||
H AbslHashValue(H h, const absl::InlinedVector<T, N, A>& a) {
|
||||
auto size = a.size();
|
||||
return H::combine(H::combine_contiguous(std::move(h), a.data(), size), size);
|
||||
}
|
||||
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_INLINED_VECTOR_H_
|
||||
+202
@@ -0,0 +1,202 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_CONTAINER_INTERNAL_CONTAINER_H_
|
||||
#define ABSL_CONTAINER_INTERNAL_CONTAINER_H_
|
||||
|
||||
#include <cassert>
|
||||
#include <type_traits>
|
||||
|
||||
#include "absl/meta/type_traits.h"
|
||||
#include "absl/types/optional.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
|
||||
template <class, class = void>
|
||||
struct IsTransparent : std::false_type {};
|
||||
template <class T>
|
||||
struct IsTransparent<T, absl::void_t<typename T::is_transparent>>
|
||||
: std::true_type {};
|
||||
|
||||
template <bool is_transparent>
|
||||
struct KeyArg {
|
||||
// Transparent. Forward `K`.
|
||||
template <typename K, typename key_type>
|
||||
using type = K;
|
||||
};
|
||||
|
||||
template <>
|
||||
struct KeyArg<false> {
|
||||
// Not transparent. Always use `key_type`.
|
||||
template <typename K, typename key_type>
|
||||
using type = key_type;
|
||||
};
|
||||
|
||||
// The node_handle concept from C++17.
|
||||
// We specialize node_handle for sets and maps. node_handle_base holds the
|
||||
// common API of both.
|
||||
template <typename PolicyTraits, typename Alloc>
|
||||
class node_handle_base {
|
||||
protected:
|
||||
using slot_type = typename PolicyTraits::slot_type;
|
||||
|
||||
public:
|
||||
using allocator_type = Alloc;
|
||||
|
||||
constexpr node_handle_base() = default;
|
||||
node_handle_base(node_handle_base&& other) noexcept {
|
||||
*this = std::move(other);
|
||||
}
|
||||
~node_handle_base() { destroy(); }
|
||||
node_handle_base& operator=(node_handle_base&& other) noexcept {
|
||||
destroy();
|
||||
if (!other.empty()) {
|
||||
alloc_ = other.alloc_;
|
||||
PolicyTraits::transfer(alloc(), slot(), other.slot());
|
||||
other.reset();
|
||||
}
|
||||
return *this;
|
||||
}
|
||||
|
||||
bool empty() const noexcept { return !alloc_; }
|
||||
explicit operator bool() const noexcept { return !empty(); }
|
||||
allocator_type get_allocator() const { return *alloc_; }
|
||||
|
||||
protected:
|
||||
friend struct CommonAccess;
|
||||
|
||||
struct transfer_tag_t {};
|
||||
node_handle_base(transfer_tag_t, const allocator_type& a, slot_type* s)
|
||||
: alloc_(a) {
|
||||
PolicyTraits::transfer(alloc(), slot(), s);
|
||||
}
|
||||
|
||||
struct move_tag_t {};
|
||||
node_handle_base(move_tag_t, const allocator_type& a, slot_type* s)
|
||||
: alloc_(a) {
|
||||
PolicyTraits::construct(alloc(), slot(), s);
|
||||
}
|
||||
|
||||
void destroy() {
|
||||
if (!empty()) {
|
||||
PolicyTraits::destroy(alloc(), slot());
|
||||
reset();
|
||||
}
|
||||
}
|
||||
|
||||
void reset() {
|
||||
assert(alloc_.has_value());
|
||||
alloc_ = absl::nullopt;
|
||||
}
|
||||
|
||||
slot_type* slot() const {
|
||||
assert(!empty());
|
||||
return reinterpret_cast<slot_type*>(std::addressof(slot_space_));
|
||||
}
|
||||
allocator_type* alloc() { return std::addressof(*alloc_); }
|
||||
|
||||
private:
|
||||
absl::optional<allocator_type> alloc_ = {};
|
||||
alignas(slot_type) mutable unsigned char slot_space_[sizeof(slot_type)] = {};
|
||||
};
|
||||
|
||||
// For sets.
|
||||
template <typename Policy, typename PolicyTraits, typename Alloc,
|
||||
typename = void>
|
||||
class node_handle : public node_handle_base<PolicyTraits, Alloc> {
|
||||
using Base = node_handle_base<PolicyTraits, Alloc>;
|
||||
|
||||
public:
|
||||
using value_type = typename PolicyTraits::value_type;
|
||||
|
||||
constexpr node_handle() {}
|
||||
|
||||
value_type& value() const { return PolicyTraits::element(this->slot()); }
|
||||
|
||||
private:
|
||||
friend struct CommonAccess;
|
||||
|
||||
using Base::Base;
|
||||
};
|
||||
|
||||
// For maps.
|
||||
template <typename Policy, typename PolicyTraits, typename Alloc>
|
||||
class node_handle<Policy, PolicyTraits, Alloc,
|
||||
absl::void_t<typename Policy::mapped_type>>
|
||||
: public node_handle_base<PolicyTraits, Alloc> {
|
||||
using Base = node_handle_base<PolicyTraits, Alloc>;
|
||||
|
||||
public:
|
||||
using key_type = typename Policy::key_type;
|
||||
using mapped_type = typename Policy::mapped_type;
|
||||
|
||||
constexpr node_handle() {}
|
||||
|
||||
auto key() const -> decltype(PolicyTraits::key(this->slot())) {
|
||||
return PolicyTraits::key(this->slot());
|
||||
}
|
||||
|
||||
mapped_type& mapped() const {
|
||||
return PolicyTraits::value(&PolicyTraits::element(this->slot()));
|
||||
}
|
||||
|
||||
private:
|
||||
friend struct CommonAccess;
|
||||
|
||||
using Base::Base;
|
||||
};
|
||||
|
||||
// Provide access to non-public node-handle functions.
|
||||
struct CommonAccess {
|
||||
template <typename Node>
|
||||
static auto GetSlot(const Node& node) -> decltype(node.slot()) {
|
||||
return node.slot();
|
||||
}
|
||||
|
||||
template <typename Node>
|
||||
static void Destroy(Node* node) {
|
||||
node->destroy();
|
||||
}
|
||||
|
||||
template <typename Node>
|
||||
static void Reset(Node* node) {
|
||||
node->reset();
|
||||
}
|
||||
|
||||
template <typename T, typename... Args>
|
||||
static T Transfer(Args&&... args) {
|
||||
return T(typename T::transfer_tag_t{}, std::forward<Args>(args)...);
|
||||
}
|
||||
|
||||
template <typename T, typename... Args>
|
||||
static T Move(Args&&... args) {
|
||||
return T(typename T::move_tag_t{}, std::forward<Args>(args)...);
|
||||
}
|
||||
};
|
||||
|
||||
// Implement the insert_return_type<> concept of C++17.
|
||||
template <class Iterator, class NodeType>
|
||||
struct InsertReturnType {
|
||||
Iterator position;
|
||||
bool inserted;
|
||||
NodeType node;
|
||||
};
|
||||
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_INTERNAL_CONTAINER_H_
|
||||
+265
@@ -0,0 +1,265 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Helper class to perform the Empty Base Optimization.
|
||||
// Ts can contain classes and non-classes, empty or not. For the ones that
|
||||
// are empty classes, we perform the optimization. If all types in Ts are empty
|
||||
// classes, then CompressedTuple<Ts...> is itself an empty class.
|
||||
//
|
||||
// To access the members, use member get<N>() function.
|
||||
//
|
||||
// Eg:
|
||||
// absl::container_internal::CompressedTuple<int, T1, T2, T3> value(7, t1, t2,
|
||||
// t3);
|
||||
// assert(value.get<0>() == 7);
|
||||
// T1& t1 = value.get<1>();
|
||||
// const T2& t2 = value.get<2>();
|
||||
// ...
|
||||
//
|
||||
// https://en.cppreference.com/w/cpp/language/ebo
|
||||
|
||||
#ifndef ABSL_CONTAINER_INTERNAL_COMPRESSED_TUPLE_H_
|
||||
#define ABSL_CONTAINER_INTERNAL_COMPRESSED_TUPLE_H_
|
||||
|
||||
#include <initializer_list>
|
||||
#include <tuple>
|
||||
#include <type_traits>
|
||||
#include <utility>
|
||||
|
||||
#include "absl/utility/utility.h"
|
||||
|
||||
#if defined(_MSC_VER) && !defined(__NVCC__)
|
||||
// We need to mark these classes with this declspec to ensure that
|
||||
// CompressedTuple happens.
|
||||
#define ABSL_INTERNAL_COMPRESSED_TUPLE_DECLSPEC __declspec(empty_bases)
|
||||
#else
|
||||
#define ABSL_INTERNAL_COMPRESSED_TUPLE_DECLSPEC
|
||||
#endif
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
|
||||
template <typename... Ts>
|
||||
class CompressedTuple;
|
||||
|
||||
namespace internal_compressed_tuple {
|
||||
|
||||
template <typename D, size_t I>
|
||||
struct Elem;
|
||||
template <typename... B, size_t I>
|
||||
struct Elem<CompressedTuple<B...>, I>
|
||||
: std::tuple_element<I, std::tuple<B...>> {};
|
||||
template <typename D, size_t I>
|
||||
using ElemT = typename Elem<D, I>::type;
|
||||
|
||||
// Use the __is_final intrinsic if available. Where it's not available, classes
|
||||
// declared with the 'final' specifier cannot be used as CompressedTuple
|
||||
// elements.
|
||||
// TODO(sbenza): Replace this with std::is_final in C++14.
|
||||
template <typename T>
|
||||
constexpr bool IsFinal() {
|
||||
#if defined(__clang__) || defined(__GNUC__)
|
||||
return __is_final(T);
|
||||
#else
|
||||
return false;
|
||||
#endif
|
||||
}
|
||||
|
||||
// We can't use EBCO on other CompressedTuples because that would mean that we
|
||||
// derive from multiple Storage<> instantiations with the same I parameter,
|
||||
// and potentially from multiple identical Storage<> instantiations. So anytime
|
||||
// we use type inheritance rather than encapsulation, we mark
|
||||
// CompressedTupleImpl, to make this easy to detect.
|
||||
struct uses_inheritance {};
|
||||
|
||||
template <typename T>
|
||||
constexpr bool ShouldUseBase() {
|
||||
return std::is_class<T>::value && std::is_empty<T>::value && !IsFinal<T>() &&
|
||||
!std::is_base_of<uses_inheritance, T>::value;
|
||||
}
|
||||
|
||||
// The storage class provides two specializations:
|
||||
// - For empty classes, it stores T as a base class.
|
||||
// - For everything else, it stores T as a member.
|
||||
template <typename T, size_t I,
|
||||
#if defined(_MSC_VER)
|
||||
bool UseBase =
|
||||
ShouldUseBase<typename std::enable_if<true, T>::type>()>
|
||||
#else
|
||||
bool UseBase = ShouldUseBase<T>()>
|
||||
#endif
|
||||
struct Storage {
|
||||
T value;
|
||||
constexpr Storage() = default;
|
||||
template <typename V>
|
||||
explicit constexpr Storage(absl::in_place_t, V&& v)
|
||||
: value(absl::forward<V>(v)) {}
|
||||
constexpr const T& get() const& { return value; }
|
||||
T& get() & { return value; }
|
||||
constexpr const T&& get() const&& { return absl::move(*this).value; }
|
||||
T&& get() && { return std::move(*this).value; }
|
||||
};
|
||||
|
||||
template <typename T, size_t I>
|
||||
struct ABSL_INTERNAL_COMPRESSED_TUPLE_DECLSPEC Storage<T, I, true> : T {
|
||||
constexpr Storage() = default;
|
||||
|
||||
template <typename V>
|
||||
explicit constexpr Storage(absl::in_place_t, V&& v)
|
||||
: T(absl::forward<V>(v)) {}
|
||||
|
||||
constexpr const T& get() const& { return *this; }
|
||||
T& get() & { return *this; }
|
||||
constexpr const T&& get() const&& { return absl::move(*this); }
|
||||
T&& get() && { return std::move(*this); }
|
||||
};
|
||||
|
||||
template <typename D, typename I, bool ShouldAnyUseBase>
|
||||
struct ABSL_INTERNAL_COMPRESSED_TUPLE_DECLSPEC CompressedTupleImpl;
|
||||
|
||||
template <typename... Ts, size_t... I, bool ShouldAnyUseBase>
|
||||
struct ABSL_INTERNAL_COMPRESSED_TUPLE_DECLSPEC CompressedTupleImpl<
|
||||
CompressedTuple<Ts...>, absl::index_sequence<I...>, ShouldAnyUseBase>
|
||||
// We use the dummy identity function through std::integral_constant to
|
||||
// convince MSVC of accepting and expanding I in that context. Without it
|
||||
// you would get:
|
||||
// error C3548: 'I': parameter pack cannot be used in this context
|
||||
: uses_inheritance,
|
||||
Storage<Ts, std::integral_constant<size_t, I>::value>... {
|
||||
constexpr CompressedTupleImpl() = default;
|
||||
template <typename... Vs>
|
||||
explicit constexpr CompressedTupleImpl(absl::in_place_t, Vs&&... args)
|
||||
: Storage<Ts, I>(absl::in_place, absl::forward<Vs>(args))... {}
|
||||
friend CompressedTuple<Ts...>;
|
||||
};
|
||||
|
||||
template <typename... Ts, size_t... I>
|
||||
struct ABSL_INTERNAL_COMPRESSED_TUPLE_DECLSPEC CompressedTupleImpl<
|
||||
CompressedTuple<Ts...>, absl::index_sequence<I...>, false>
|
||||
// We use the dummy identity function as above...
|
||||
: Storage<Ts, std::integral_constant<size_t, I>::value, false>... {
|
||||
constexpr CompressedTupleImpl() = default;
|
||||
template <typename... Vs>
|
||||
explicit constexpr CompressedTupleImpl(absl::in_place_t, Vs&&... args)
|
||||
: Storage<Ts, I, false>(absl::in_place, absl::forward<Vs>(args))... {}
|
||||
friend CompressedTuple<Ts...>;
|
||||
};
|
||||
|
||||
std::false_type Or(std::initializer_list<std::false_type>);
|
||||
std::true_type Or(std::initializer_list<bool>);
|
||||
|
||||
// MSVC requires this to be done separately rather than within the declaration
|
||||
// of CompressedTuple below.
|
||||
template <typename... Ts>
|
||||
constexpr bool ShouldAnyUseBase() {
|
||||
return decltype(
|
||||
Or({std::integral_constant<bool, ShouldUseBase<Ts>()>()...})){};
|
||||
}
|
||||
|
||||
template <typename T, typename V>
|
||||
using TupleMoveConstructible = typename std::conditional<
|
||||
std::is_reference<T>::value, std::is_convertible<V, T>,
|
||||
std::is_constructible<T, V&&>>::type;
|
||||
|
||||
} // namespace internal_compressed_tuple
|
||||
|
||||
// Helper class to perform the Empty Base Class Optimization.
|
||||
// Ts can contain classes and non-classes, empty or not. For the ones that
|
||||
// are empty classes, we perform the CompressedTuple. If all types in Ts are
|
||||
// empty classes, then CompressedTuple<Ts...> is itself an empty class. (This
|
||||
// does not apply when one or more of those empty classes is itself an empty
|
||||
// CompressedTuple.)
|
||||
//
|
||||
// To access the members, use member .get<N>() function.
|
||||
//
|
||||
// Eg:
|
||||
// absl::container_internal::CompressedTuple<int, T1, T2, T3> value(7, t1, t2,
|
||||
// t3);
|
||||
// assert(value.get<0>() == 7);
|
||||
// T1& t1 = value.get<1>();
|
||||
// const T2& t2 = value.get<2>();
|
||||
// ...
|
||||
//
|
||||
// https://en.cppreference.com/w/cpp/language/ebo
|
||||
template <typename... Ts>
|
||||
class ABSL_INTERNAL_COMPRESSED_TUPLE_DECLSPEC CompressedTuple
|
||||
: private internal_compressed_tuple::CompressedTupleImpl<
|
||||
CompressedTuple<Ts...>, absl::index_sequence_for<Ts...>,
|
||||
internal_compressed_tuple::ShouldAnyUseBase<Ts...>()> {
|
||||
private:
|
||||
template <int I>
|
||||
using ElemT = internal_compressed_tuple::ElemT<CompressedTuple, I>;
|
||||
|
||||
template <int I>
|
||||
using StorageT = internal_compressed_tuple::Storage<ElemT<I>, I>;
|
||||
|
||||
public:
|
||||
// There seems to be a bug in MSVC dealing in which using '=default' here will
|
||||
// cause the compiler to ignore the body of other constructors. The work-
|
||||
// around is to explicitly implement the default constructor.
|
||||
#if defined(_MSC_VER)
|
||||
constexpr CompressedTuple() : CompressedTuple::CompressedTupleImpl() {}
|
||||
#else
|
||||
constexpr CompressedTuple() = default;
|
||||
#endif
|
||||
explicit constexpr CompressedTuple(const Ts&... base)
|
||||
: CompressedTuple::CompressedTupleImpl(absl::in_place, base...) {}
|
||||
|
||||
template <typename... Vs,
|
||||
absl::enable_if_t<
|
||||
absl::conjunction<
|
||||
// Ensure we are not hiding default copy/move constructors.
|
||||
absl::negation<std::is_same<void(CompressedTuple),
|
||||
void(absl::decay_t<Vs>...)>>,
|
||||
internal_compressed_tuple::TupleMoveConstructible<
|
||||
Ts, Vs&&>...>::value,
|
||||
bool> = true>
|
||||
explicit constexpr CompressedTuple(Vs&&... base)
|
||||
: CompressedTuple::CompressedTupleImpl(absl::in_place,
|
||||
absl::forward<Vs>(base)...) {}
|
||||
|
||||
template <int I>
|
||||
ElemT<I>& get() & {
|
||||
return internal_compressed_tuple::Storage<ElemT<I>, I>::get();
|
||||
}
|
||||
|
||||
template <int I>
|
||||
constexpr const ElemT<I>& get() const& {
|
||||
return StorageT<I>::get();
|
||||
}
|
||||
|
||||
template <int I>
|
||||
ElemT<I>&& get() && {
|
||||
return std::move(*this).StorageT<I>::get();
|
||||
}
|
||||
|
||||
template <int I>
|
||||
constexpr const ElemT<I>&& get() const&& {
|
||||
return absl::move(*this).StorageT<I>::get();
|
||||
}
|
||||
};
|
||||
|
||||
// Explicit specialization for a zero-element tuple
|
||||
// (needed to avoid ambiguous overloads for the default constructor).
|
||||
template <>
|
||||
class ABSL_INTERNAL_COMPRESSED_TUPLE_DECLSPEC CompressedTuple<> {};
|
||||
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#undef ABSL_INTERNAL_COMPRESSED_TUPLE_DECLSPEC
|
||||
|
||||
#endif // ABSL_CONTAINER_INTERNAL_COMPRESSED_TUPLE_H_
|
||||
+440
@@ -0,0 +1,440 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_CONTAINER_INTERNAL_CONTAINER_MEMORY_H_
|
||||
#define ABSL_CONTAINER_INTERNAL_CONTAINER_MEMORY_H_
|
||||
|
||||
#ifdef ADDRESS_SANITIZER
|
||||
#include <sanitizer/asan_interface.h>
|
||||
#endif
|
||||
|
||||
#ifdef MEMORY_SANITIZER
|
||||
#include <sanitizer/msan_interface.h>
|
||||
#endif
|
||||
|
||||
#include <cassert>
|
||||
#include <cstddef>
|
||||
#include <memory>
|
||||
#include <tuple>
|
||||
#include <type_traits>
|
||||
#include <utility>
|
||||
|
||||
#include "absl/memory/memory.h"
|
||||
#include "absl/utility/utility.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
|
||||
// Allocates at least n bytes aligned to the specified alignment.
|
||||
// Alignment must be a power of 2. It must be positive.
|
||||
//
|
||||
// Note that many allocators don't honor alignment requirements above certain
|
||||
// threshold (usually either alignof(std::max_align_t) or alignof(void*)).
|
||||
// Allocate() doesn't apply alignment corrections. If the underlying allocator
|
||||
// returns insufficiently alignment pointer, that's what you are going to get.
|
||||
template <size_t Alignment, class Alloc>
|
||||
void* Allocate(Alloc* alloc, size_t n) {
|
||||
static_assert(Alignment > 0, "");
|
||||
assert(n && "n must be positive");
|
||||
struct alignas(Alignment) M {};
|
||||
using A = typename absl::allocator_traits<Alloc>::template rebind_alloc<M>;
|
||||
using AT = typename absl::allocator_traits<Alloc>::template rebind_traits<M>;
|
||||
A mem_alloc(*alloc);
|
||||
void* p = AT::allocate(mem_alloc, (n + sizeof(M) - 1) / sizeof(M));
|
||||
assert(reinterpret_cast<uintptr_t>(p) % Alignment == 0 &&
|
||||
"allocator does not respect alignment");
|
||||
return p;
|
||||
}
|
||||
|
||||
// The pointer must have been previously obtained by calling
|
||||
// Allocate<Alignment>(alloc, n).
|
||||
template <size_t Alignment, class Alloc>
|
||||
void Deallocate(Alloc* alloc, void* p, size_t n) {
|
||||
static_assert(Alignment > 0, "");
|
||||
assert(n && "n must be positive");
|
||||
struct alignas(Alignment) M {};
|
||||
using A = typename absl::allocator_traits<Alloc>::template rebind_alloc<M>;
|
||||
using AT = typename absl::allocator_traits<Alloc>::template rebind_traits<M>;
|
||||
A mem_alloc(*alloc);
|
||||
AT::deallocate(mem_alloc, static_cast<M*>(p),
|
||||
(n + sizeof(M) - 1) / sizeof(M));
|
||||
}
|
||||
|
||||
namespace memory_internal {
|
||||
|
||||
// Constructs T into uninitialized storage pointed by `ptr` using the args
|
||||
// specified in the tuple.
|
||||
template <class Alloc, class T, class Tuple, size_t... I>
|
||||
void ConstructFromTupleImpl(Alloc* alloc, T* ptr, Tuple&& t,
|
||||
absl::index_sequence<I...>) {
|
||||
absl::allocator_traits<Alloc>::construct(
|
||||
*alloc, ptr, std::get<I>(std::forward<Tuple>(t))...);
|
||||
}
|
||||
|
||||
template <class T, class F>
|
||||
struct WithConstructedImplF {
|
||||
template <class... Args>
|
||||
decltype(std::declval<F>()(std::declval<T>())) operator()(
|
||||
Args&&... args) const {
|
||||
return std::forward<F>(f)(T(std::forward<Args>(args)...));
|
||||
}
|
||||
F&& f;
|
||||
};
|
||||
|
||||
template <class T, class Tuple, size_t... Is, class F>
|
||||
decltype(std::declval<F>()(std::declval<T>())) WithConstructedImpl(
|
||||
Tuple&& t, absl::index_sequence<Is...>, F&& f) {
|
||||
return WithConstructedImplF<T, F>{std::forward<F>(f)}(
|
||||
std::get<Is>(std::forward<Tuple>(t))...);
|
||||
}
|
||||
|
||||
template <class T, size_t... Is>
|
||||
auto TupleRefImpl(T&& t, absl::index_sequence<Is...>)
|
||||
-> decltype(std::forward_as_tuple(std::get<Is>(std::forward<T>(t))...)) {
|
||||
return std::forward_as_tuple(std::get<Is>(std::forward<T>(t))...);
|
||||
}
|
||||
|
||||
// Returns a tuple of references to the elements of the input tuple. T must be a
|
||||
// tuple.
|
||||
template <class T>
|
||||
auto TupleRef(T&& t) -> decltype(
|
||||
TupleRefImpl(std::forward<T>(t),
|
||||
absl::make_index_sequence<
|
||||
std::tuple_size<typename std::decay<T>::type>::value>())) {
|
||||
return TupleRefImpl(
|
||||
std::forward<T>(t),
|
||||
absl::make_index_sequence<
|
||||
std::tuple_size<typename std::decay<T>::type>::value>());
|
||||
}
|
||||
|
||||
template <class F, class K, class V>
|
||||
decltype(std::declval<F>()(std::declval<const K&>(), std::piecewise_construct,
|
||||
std::declval<std::tuple<K>>(), std::declval<V>()))
|
||||
DecomposePairImpl(F&& f, std::pair<std::tuple<K>, V> p) {
|
||||
const auto& key = std::get<0>(p.first);
|
||||
return std::forward<F>(f)(key, std::piecewise_construct, std::move(p.first),
|
||||
std::move(p.second));
|
||||
}
|
||||
|
||||
} // namespace memory_internal
|
||||
|
||||
// Constructs T into uninitialized storage pointed by `ptr` using the args
|
||||
// specified in the tuple.
|
||||
template <class Alloc, class T, class Tuple>
|
||||
void ConstructFromTuple(Alloc* alloc, T* ptr, Tuple&& t) {
|
||||
memory_internal::ConstructFromTupleImpl(
|
||||
alloc, ptr, std::forward<Tuple>(t),
|
||||
absl::make_index_sequence<
|
||||
std::tuple_size<typename std::decay<Tuple>::type>::value>());
|
||||
}
|
||||
|
||||
// Constructs T using the args specified in the tuple and calls F with the
|
||||
// constructed value.
|
||||
template <class T, class Tuple, class F>
|
||||
decltype(std::declval<F>()(std::declval<T>())) WithConstructed(
|
||||
Tuple&& t, F&& f) {
|
||||
return memory_internal::WithConstructedImpl<T>(
|
||||
std::forward<Tuple>(t),
|
||||
absl::make_index_sequence<
|
||||
std::tuple_size<typename std::decay<Tuple>::type>::value>(),
|
||||
std::forward<F>(f));
|
||||
}
|
||||
|
||||
// Given arguments of an std::pair's consructor, PairArgs() returns a pair of
|
||||
// tuples with references to the passed arguments. The tuples contain
|
||||
// constructor arguments for the first and the second elements of the pair.
|
||||
//
|
||||
// The following two snippets are equivalent.
|
||||
//
|
||||
// 1. std::pair<F, S> p(args...);
|
||||
//
|
||||
// 2. auto a = PairArgs(args...);
|
||||
// std::pair<F, S> p(std::piecewise_construct,
|
||||
// std::move(p.first), std::move(p.second));
|
||||
inline std::pair<std::tuple<>, std::tuple<>> PairArgs() { return {}; }
|
||||
template <class F, class S>
|
||||
std::pair<std::tuple<F&&>, std::tuple<S&&>> PairArgs(F&& f, S&& s) {
|
||||
return {std::piecewise_construct, std::forward_as_tuple(std::forward<F>(f)),
|
||||
std::forward_as_tuple(std::forward<S>(s))};
|
||||
}
|
||||
template <class F, class S>
|
||||
std::pair<std::tuple<const F&>, std::tuple<const S&>> PairArgs(
|
||||
const std::pair<F, S>& p) {
|
||||
return PairArgs(p.first, p.second);
|
||||
}
|
||||
template <class F, class S>
|
||||
std::pair<std::tuple<F&&>, std::tuple<S&&>> PairArgs(std::pair<F, S>&& p) {
|
||||
return PairArgs(std::forward<F>(p.first), std::forward<S>(p.second));
|
||||
}
|
||||
template <class F, class S>
|
||||
auto PairArgs(std::piecewise_construct_t, F&& f, S&& s)
|
||||
-> decltype(std::make_pair(memory_internal::TupleRef(std::forward<F>(f)),
|
||||
memory_internal::TupleRef(std::forward<S>(s)))) {
|
||||
return std::make_pair(memory_internal::TupleRef(std::forward<F>(f)),
|
||||
memory_internal::TupleRef(std::forward<S>(s)));
|
||||
}
|
||||
|
||||
// A helper function for implementing apply() in map policies.
|
||||
template <class F, class... Args>
|
||||
auto DecomposePair(F&& f, Args&&... args)
|
||||
-> decltype(memory_internal::DecomposePairImpl(
|
||||
std::forward<F>(f), PairArgs(std::forward<Args>(args)...))) {
|
||||
return memory_internal::DecomposePairImpl(
|
||||
std::forward<F>(f), PairArgs(std::forward<Args>(args)...));
|
||||
}
|
||||
|
||||
// A helper function for implementing apply() in set policies.
|
||||
template <class F, class Arg>
|
||||
decltype(std::declval<F>()(std::declval<const Arg&>(), std::declval<Arg>()))
|
||||
DecomposeValue(F&& f, Arg&& arg) {
|
||||
const auto& key = arg;
|
||||
return std::forward<F>(f)(key, std::forward<Arg>(arg));
|
||||
}
|
||||
|
||||
// Helper functions for asan and msan.
|
||||
inline void SanitizerPoisonMemoryRegion(const void* m, size_t s) {
|
||||
#ifdef ADDRESS_SANITIZER
|
||||
ASAN_POISON_MEMORY_REGION(m, s);
|
||||
#endif
|
||||
#ifdef MEMORY_SANITIZER
|
||||
__msan_poison(m, s);
|
||||
#endif
|
||||
(void)m;
|
||||
(void)s;
|
||||
}
|
||||
|
||||
inline void SanitizerUnpoisonMemoryRegion(const void* m, size_t s) {
|
||||
#ifdef ADDRESS_SANITIZER
|
||||
ASAN_UNPOISON_MEMORY_REGION(m, s);
|
||||
#endif
|
||||
#ifdef MEMORY_SANITIZER
|
||||
__msan_unpoison(m, s);
|
||||
#endif
|
||||
(void)m;
|
||||
(void)s;
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
inline void SanitizerPoisonObject(const T* object) {
|
||||
SanitizerPoisonMemoryRegion(object, sizeof(T));
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
inline void SanitizerUnpoisonObject(const T* object) {
|
||||
SanitizerUnpoisonMemoryRegion(object, sizeof(T));
|
||||
}
|
||||
|
||||
namespace memory_internal {
|
||||
|
||||
// If Pair is a standard-layout type, OffsetOf<Pair>::kFirst and
|
||||
// OffsetOf<Pair>::kSecond are equivalent to offsetof(Pair, first) and
|
||||
// offsetof(Pair, second) respectively. Otherwise they are -1.
|
||||
//
|
||||
// The purpose of OffsetOf is to avoid calling offsetof() on non-standard-layout
|
||||
// type, which is non-portable.
|
||||
template <class Pair, class = std::true_type>
|
||||
struct OffsetOf {
|
||||
static constexpr size_t kFirst = -1;
|
||||
static constexpr size_t kSecond = -1;
|
||||
};
|
||||
|
||||
template <class Pair>
|
||||
struct OffsetOf<Pair, typename std::is_standard_layout<Pair>::type> {
|
||||
static constexpr size_t kFirst = offsetof(Pair, first);
|
||||
static constexpr size_t kSecond = offsetof(Pair, second);
|
||||
};
|
||||
|
||||
template <class K, class V>
|
||||
struct IsLayoutCompatible {
|
||||
private:
|
||||
struct Pair {
|
||||
K first;
|
||||
V second;
|
||||
};
|
||||
|
||||
// Is P layout-compatible with Pair?
|
||||
template <class P>
|
||||
static constexpr bool LayoutCompatible() {
|
||||
return std::is_standard_layout<P>() && sizeof(P) == sizeof(Pair) &&
|
||||
alignof(P) == alignof(Pair) &&
|
||||
memory_internal::OffsetOf<P>::kFirst ==
|
||||
memory_internal::OffsetOf<Pair>::kFirst &&
|
||||
memory_internal::OffsetOf<P>::kSecond ==
|
||||
memory_internal::OffsetOf<Pair>::kSecond;
|
||||
}
|
||||
|
||||
public:
|
||||
// Whether pair<const K, V> and pair<K, V> are layout-compatible. If they are,
|
||||
// then it is safe to store them in a union and read from either.
|
||||
static constexpr bool value = std::is_standard_layout<K>() &&
|
||||
std::is_standard_layout<Pair>() &&
|
||||
memory_internal::OffsetOf<Pair>::kFirst == 0 &&
|
||||
LayoutCompatible<std::pair<K, V>>() &&
|
||||
LayoutCompatible<std::pair<const K, V>>();
|
||||
};
|
||||
|
||||
} // namespace memory_internal
|
||||
|
||||
// The internal storage type for key-value containers like flat_hash_map.
|
||||
//
|
||||
// It is convenient for the value_type of a flat_hash_map<K, V> to be
|
||||
// pair<const K, V>; the "const K" prevents accidental modification of the key
|
||||
// when dealing with the reference returned from find() and similar methods.
|
||||
// However, this creates other problems; we want to be able to emplace(K, V)
|
||||
// efficiently with move operations, and similarly be able to move a
|
||||
// pair<K, V> in insert().
|
||||
//
|
||||
// The solution is this union, which aliases the const and non-const versions
|
||||
// of the pair. This also allows flat_hash_map<const K, V> to work, even though
|
||||
// that has the same efficiency issues with move in emplace() and insert() -
|
||||
// but people do it anyway.
|
||||
//
|
||||
// If kMutableKeys is false, only the value member can be accessed.
|
||||
//
|
||||
// If kMutableKeys is true, key can be accessed through all slots while value
|
||||
// and mutable_value must be accessed only via INITIALIZED slots. Slots are
|
||||
// created and destroyed via mutable_value so that the key can be moved later.
|
||||
//
|
||||
// Accessing one of the union fields while the other is active is safe as
|
||||
// long as they are layout-compatible, which is guaranteed by the definition of
|
||||
// kMutableKeys. For C++11, the relevant section of the standard is
|
||||
// https://timsong-cpp.github.io/cppwp/n3337/class.mem#19 (9.2.19)
|
||||
template <class K, class V>
|
||||
union map_slot_type {
|
||||
map_slot_type() {}
|
||||
~map_slot_type() = delete;
|
||||
using value_type = std::pair<const K, V>;
|
||||
using mutable_value_type = std::pair<K, V>;
|
||||
|
||||
value_type value;
|
||||
mutable_value_type mutable_value;
|
||||
K key;
|
||||
};
|
||||
|
||||
template <class K, class V>
|
||||
struct map_slot_policy {
|
||||
using slot_type = map_slot_type<K, V>;
|
||||
using value_type = std::pair<const K, V>;
|
||||
using mutable_value_type = std::pair<K, V>;
|
||||
|
||||
private:
|
||||
static void emplace(slot_type* slot) {
|
||||
// The construction of union doesn't do anything at runtime but it allows us
|
||||
// to access its members without violating aliasing rules.
|
||||
new (slot) slot_type;
|
||||
}
|
||||
// If pair<const K, V> and pair<K, V> are layout-compatible, we can accept one
|
||||
// or the other via slot_type. We are also free to access the key via
|
||||
// slot_type::key in this case.
|
||||
using kMutableKeys = memory_internal::IsLayoutCompatible<K, V>;
|
||||
|
||||
public:
|
||||
static value_type& element(slot_type* slot) { return slot->value; }
|
||||
static const value_type& element(const slot_type* slot) {
|
||||
return slot->value;
|
||||
}
|
||||
|
||||
static const K& key(const slot_type* slot) {
|
||||
return kMutableKeys::value ? slot->key : slot->value.first;
|
||||
}
|
||||
|
||||
template <class Allocator, class... Args>
|
||||
static void construct(Allocator* alloc, slot_type* slot, Args&&... args) {
|
||||
emplace(slot);
|
||||
if (kMutableKeys::value) {
|
||||
absl::allocator_traits<Allocator>::construct(*alloc, &slot->mutable_value,
|
||||
std::forward<Args>(args)...);
|
||||
} else {
|
||||
absl::allocator_traits<Allocator>::construct(*alloc, &slot->value,
|
||||
std::forward<Args>(args)...);
|
||||
}
|
||||
}
|
||||
|
||||
// Construct this slot by moving from another slot.
|
||||
template <class Allocator>
|
||||
static void construct(Allocator* alloc, slot_type* slot, slot_type* other) {
|
||||
emplace(slot);
|
||||
if (kMutableKeys::value) {
|
||||
absl::allocator_traits<Allocator>::construct(
|
||||
*alloc, &slot->mutable_value, std::move(other->mutable_value));
|
||||
} else {
|
||||
absl::allocator_traits<Allocator>::construct(*alloc, &slot->value,
|
||||
std::move(other->value));
|
||||
}
|
||||
}
|
||||
|
||||
template <class Allocator>
|
||||
static void destroy(Allocator* alloc, slot_type* slot) {
|
||||
if (kMutableKeys::value) {
|
||||
absl::allocator_traits<Allocator>::destroy(*alloc, &slot->mutable_value);
|
||||
} else {
|
||||
absl::allocator_traits<Allocator>::destroy(*alloc, &slot->value);
|
||||
}
|
||||
}
|
||||
|
||||
template <class Allocator>
|
||||
static void transfer(Allocator* alloc, slot_type* new_slot,
|
||||
slot_type* old_slot) {
|
||||
emplace(new_slot);
|
||||
if (kMutableKeys::value) {
|
||||
absl::allocator_traits<Allocator>::construct(
|
||||
*alloc, &new_slot->mutable_value, std::move(old_slot->mutable_value));
|
||||
} else {
|
||||
absl::allocator_traits<Allocator>::construct(*alloc, &new_slot->value,
|
||||
std::move(old_slot->value));
|
||||
}
|
||||
destroy(alloc, old_slot);
|
||||
}
|
||||
|
||||
template <class Allocator>
|
||||
static void swap(Allocator* alloc, slot_type* a, slot_type* b) {
|
||||
if (kMutableKeys::value) {
|
||||
using std::swap;
|
||||
swap(a->mutable_value, b->mutable_value);
|
||||
} else {
|
||||
value_type tmp = std::move(a->value);
|
||||
absl::allocator_traits<Allocator>::destroy(*alloc, &a->value);
|
||||
absl::allocator_traits<Allocator>::construct(*alloc, &a->value,
|
||||
std::move(b->value));
|
||||
absl::allocator_traits<Allocator>::destroy(*alloc, &b->value);
|
||||
absl::allocator_traits<Allocator>::construct(*alloc, &b->value,
|
||||
std::move(tmp));
|
||||
}
|
||||
}
|
||||
|
||||
template <class Allocator>
|
||||
static void move(Allocator* alloc, slot_type* src, slot_type* dest) {
|
||||
if (kMutableKeys::value) {
|
||||
dest->mutable_value = std::move(src->mutable_value);
|
||||
} else {
|
||||
absl::allocator_traits<Allocator>::destroy(*alloc, &dest->value);
|
||||
absl::allocator_traits<Allocator>::construct(*alloc, &dest->value,
|
||||
std::move(src->value));
|
||||
}
|
||||
}
|
||||
|
||||
template <class Allocator>
|
||||
static void move(Allocator* alloc, slot_type* first, slot_type* last,
|
||||
slot_type* result) {
|
||||
for (slot_type *src = first, *dest = result; src != last; ++src, ++dest)
|
||||
move(alloc, src, dest);
|
||||
}
|
||||
};
|
||||
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_INTERNAL_CONTAINER_MEMORY_H_
|
||||
@@ -0,0 +1,146 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Define the default Hash and Eq functions for SwissTable containers.
|
||||
//
|
||||
// std::hash<T> and std::equal_to<T> are not appropriate hash and equal
|
||||
// functions for SwissTable containers. There are two reasons for this.
|
||||
//
|
||||
// SwissTable containers are power of 2 sized containers:
|
||||
//
|
||||
// This means they use the lower bits of the hash value to find the slot for
|
||||
// each entry. The typical hash function for integral types is the identity.
|
||||
// This is a very weak hash function for SwissTable and any power of 2 sized
|
||||
// hashtable implementation which will lead to excessive collisions. For
|
||||
// SwissTable we use murmur3 style mixing to reduce collisions to a minimum.
|
||||
//
|
||||
// SwissTable containers support heterogeneous lookup:
|
||||
//
|
||||
// In order to make heterogeneous lookup work, hash and equal functions must be
|
||||
// polymorphic. At the same time they have to satisfy the same requirements the
|
||||
// C++ standard imposes on hash functions and equality operators. That is:
|
||||
//
|
||||
// if hash_default_eq<T>(a, b) returns true for any a and b of type T, then
|
||||
// hash_default_hash<T>(a) must equal hash_default_hash<T>(b)
|
||||
//
|
||||
// For SwissTable containers this requirement is relaxed to allow a and b of
|
||||
// any and possibly different types. Note that like the standard the hash and
|
||||
// equal functions are still bound to T. This is important because some type U
|
||||
// can be hashed by/tested for equality differently depending on T. A notable
|
||||
// example is `const char*`. `const char*` is treated as a c-style string when
|
||||
// the hash function is hash<std::string> but as a pointer when the hash
|
||||
// function is hash<void*>.
|
||||
//
|
||||
#ifndef ABSL_CONTAINER_INTERNAL_HASH_FUNCTION_DEFAULTS_H_
|
||||
#define ABSL_CONTAINER_INTERNAL_HASH_FUNCTION_DEFAULTS_H_
|
||||
|
||||
#include <stdint.h>
|
||||
#include <cstddef>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <type_traits>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
#include "absl/hash/hash.h"
|
||||
#include "absl/strings/string_view.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
|
||||
// The hash of an object of type T is computed by using absl::Hash.
|
||||
template <class T, class E = void>
|
||||
struct HashEq {
|
||||
using Hash = absl::Hash<T>;
|
||||
using Eq = std::equal_to<T>;
|
||||
};
|
||||
|
||||
struct StringHash {
|
||||
using is_transparent = void;
|
||||
|
||||
size_t operator()(absl::string_view v) const {
|
||||
return absl::Hash<absl::string_view>{}(v);
|
||||
}
|
||||
};
|
||||
|
||||
// Supports heterogeneous lookup for string-like elements.
|
||||
struct StringHashEq {
|
||||
using Hash = StringHash;
|
||||
struct Eq {
|
||||
using is_transparent = void;
|
||||
bool operator()(absl::string_view lhs, absl::string_view rhs) const {
|
||||
return lhs == rhs;
|
||||
}
|
||||
};
|
||||
};
|
||||
|
||||
template <>
|
||||
struct HashEq<std::string> : StringHashEq {};
|
||||
template <>
|
||||
struct HashEq<absl::string_view> : StringHashEq {};
|
||||
|
||||
// Supports heterogeneous lookup for pointers and smart pointers.
|
||||
template <class T>
|
||||
struct HashEq<T*> {
|
||||
struct Hash {
|
||||
using is_transparent = void;
|
||||
template <class U>
|
||||
size_t operator()(const U& ptr) const {
|
||||
return absl::Hash<const T*>{}(HashEq::ToPtr(ptr));
|
||||
}
|
||||
};
|
||||
struct Eq {
|
||||
using is_transparent = void;
|
||||
template <class A, class B>
|
||||
bool operator()(const A& a, const B& b) const {
|
||||
return HashEq::ToPtr(a) == HashEq::ToPtr(b);
|
||||
}
|
||||
};
|
||||
|
||||
private:
|
||||
static const T* ToPtr(const T* ptr) { return ptr; }
|
||||
template <class U, class D>
|
||||
static const T* ToPtr(const std::unique_ptr<U, D>& ptr) {
|
||||
return ptr.get();
|
||||
}
|
||||
template <class U>
|
||||
static const T* ToPtr(const std::shared_ptr<U>& ptr) {
|
||||
return ptr.get();
|
||||
}
|
||||
};
|
||||
|
||||
template <class T, class D>
|
||||
struct HashEq<std::unique_ptr<T, D>> : HashEq<T*> {};
|
||||
template <class T>
|
||||
struct HashEq<std::shared_ptr<T>> : HashEq<T*> {};
|
||||
|
||||
// This header's visibility is restricted. If you need to access the default
|
||||
// hasher please use the container's ::hasher alias instead.
|
||||
//
|
||||
// Example: typename Hash = typename absl::flat_hash_map<K, V>::hasher
|
||||
template <class T>
|
||||
using hash_default_hash = typename container_internal::HashEq<T>::Hash;
|
||||
|
||||
// This header's visibility is restricted. If you need to access the default
|
||||
// key equal please use the container's ::key_equal alias instead.
|
||||
//
|
||||
// Example: typename Eq = typename absl::flat_hash_map<K, V, Hash>::key_equal
|
||||
template <class T>
|
||||
using hash_default_eq = typename container_internal::HashEq<T>::Eq;
|
||||
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_INTERNAL_HASH_FUNCTION_DEFAULTS_H_
|
||||
@@ -0,0 +1,191 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_CONTAINER_INTERNAL_HASH_POLICY_TRAITS_H_
|
||||
#define ABSL_CONTAINER_INTERNAL_HASH_POLICY_TRAITS_H_
|
||||
|
||||
#include <cstddef>
|
||||
#include <memory>
|
||||
#include <type_traits>
|
||||
#include <utility>
|
||||
|
||||
#include "absl/meta/type_traits.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
|
||||
// Defines how slots are initialized/destroyed/moved.
|
||||
template <class Policy, class = void>
|
||||
struct hash_policy_traits {
|
||||
private:
|
||||
struct ReturnKey {
|
||||
// We return `Key` here.
|
||||
// When Key=T&, we forward the lvalue reference.
|
||||
// When Key=T, we return by value to avoid a dangling reference.
|
||||
// eg, for string_hash_map.
|
||||
template <class Key, class... Args>
|
||||
Key operator()(Key&& k, const Args&...) const {
|
||||
return std::forward<Key>(k);
|
||||
}
|
||||
};
|
||||
|
||||
template <class P = Policy, class = void>
|
||||
struct ConstantIteratorsImpl : std::false_type {};
|
||||
|
||||
template <class P>
|
||||
struct ConstantIteratorsImpl<P, absl::void_t<typename P::constant_iterators>>
|
||||
: P::constant_iterators {};
|
||||
|
||||
public:
|
||||
// The actual object stored in the hash table.
|
||||
using slot_type = typename Policy::slot_type;
|
||||
|
||||
// The type of the keys stored in the hashtable.
|
||||
using key_type = typename Policy::key_type;
|
||||
|
||||
// The argument type for insertions into the hashtable. This is different
|
||||
// from value_type for increased performance. See initializer_list constructor
|
||||
// and insert() member functions for more details.
|
||||
using init_type = typename Policy::init_type;
|
||||
|
||||
using reference = decltype(Policy::element(std::declval<slot_type*>()));
|
||||
using pointer = typename std::remove_reference<reference>::type*;
|
||||
using value_type = typename std::remove_reference<reference>::type;
|
||||
|
||||
// Policies can set this variable to tell raw_hash_set that all iterators
|
||||
// should be constant, even `iterator`. This is useful for set-like
|
||||
// containers.
|
||||
// Defaults to false if not provided by the policy.
|
||||
using constant_iterators = ConstantIteratorsImpl<>;
|
||||
|
||||
// PRECONDITION: `slot` is UNINITIALIZED
|
||||
// POSTCONDITION: `slot` is INITIALIZED
|
||||
template <class Alloc, class... Args>
|
||||
static void construct(Alloc* alloc, slot_type* slot, Args&&... args) {
|
||||
Policy::construct(alloc, slot, std::forward<Args>(args)...);
|
||||
}
|
||||
|
||||
// PRECONDITION: `slot` is INITIALIZED
|
||||
// POSTCONDITION: `slot` is UNINITIALIZED
|
||||
template <class Alloc>
|
||||
static void destroy(Alloc* alloc, slot_type* slot) {
|
||||
Policy::destroy(alloc, slot);
|
||||
}
|
||||
|
||||
// Transfers the `old_slot` to `new_slot`. Any memory allocated by the
|
||||
// allocator inside `old_slot` to `new_slot` can be transferred.
|
||||
//
|
||||
// OPTIONAL: defaults to:
|
||||
//
|
||||
// clone(new_slot, std::move(*old_slot));
|
||||
// destroy(old_slot);
|
||||
//
|
||||
// PRECONDITION: `new_slot` is UNINITIALIZED and `old_slot` is INITIALIZED
|
||||
// POSTCONDITION: `new_slot` is INITIALIZED and `old_slot` is
|
||||
// UNINITIALIZED
|
||||
template <class Alloc>
|
||||
static void transfer(Alloc* alloc, slot_type* new_slot, slot_type* old_slot) {
|
||||
transfer_impl(alloc, new_slot, old_slot, 0);
|
||||
}
|
||||
|
||||
// PRECONDITION: `slot` is INITIALIZED
|
||||
// POSTCONDITION: `slot` is INITIALIZED
|
||||
template <class P = Policy>
|
||||
static auto element(slot_type* slot) -> decltype(P::element(slot)) {
|
||||
return P::element(slot);
|
||||
}
|
||||
|
||||
// Returns the amount of memory owned by `slot`, exclusive of `sizeof(*slot)`.
|
||||
//
|
||||
// If `slot` is nullptr, returns the constant amount of memory owned by any
|
||||
// full slot or -1 if slots own variable amounts of memory.
|
||||
//
|
||||
// PRECONDITION: `slot` is INITIALIZED or nullptr
|
||||
template <class P = Policy>
|
||||
static size_t space_used(const slot_type* slot) {
|
||||
return P::space_used(slot);
|
||||
}
|
||||
|
||||
// Provides generalized access to the key for elements, both for elements in
|
||||
// the table and for elements that have not yet been inserted (or even
|
||||
// constructed). We would like an API that allows us to say: `key(args...)`
|
||||
// but we cannot do that for all cases, so we use this more general API that
|
||||
// can be used for many things, including the following:
|
||||
//
|
||||
// - Given an element in a table, get its key.
|
||||
// - Given an element initializer, get its key.
|
||||
// - Given `emplace()` arguments, get the element key.
|
||||
//
|
||||
// Implementations of this must adhere to a very strict technical
|
||||
// specification around aliasing and consuming arguments:
|
||||
//
|
||||
// Let `value_type` be the result type of `element()` without ref- and
|
||||
// cv-qualifiers. The first argument is a functor, the rest are constructor
|
||||
// arguments for `value_type`. Returns `std::forward<F>(f)(k, xs...)`, where
|
||||
// `k` is the element key, and `xs...` are the new constructor arguments for
|
||||
// `value_type`. It's allowed for `k` to alias `xs...`, and for both to alias
|
||||
// `ts...`. The key won't be touched once `xs...` are used to construct an
|
||||
// element; `ts...` won't be touched at all, which allows `apply()` to consume
|
||||
// any rvalues among them.
|
||||
//
|
||||
// If `value_type` is constructible from `Ts&&...`, `Policy::apply()` must not
|
||||
// trigger a hard compile error unless it originates from `f`. In other words,
|
||||
// `Policy::apply()` must be SFINAE-friendly. If `value_type` is not
|
||||
// constructible from `Ts&&...`, either SFINAE or a hard compile error is OK.
|
||||
//
|
||||
// If `Ts...` is `[cv] value_type[&]` or `[cv] init_type[&]`,
|
||||
// `Policy::apply()` must work. A compile error is not allowed, SFINAE or not.
|
||||
template <class F, class... Ts, class P = Policy>
|
||||
static auto apply(F&& f, Ts&&... ts)
|
||||
-> decltype(P::apply(std::forward<F>(f), std::forward<Ts>(ts)...)) {
|
||||
return P::apply(std::forward<F>(f), std::forward<Ts>(ts)...);
|
||||
}
|
||||
|
||||
// Returns the "key" portion of the slot.
|
||||
// Used for node handle manipulation.
|
||||
template <class P = Policy>
|
||||
static auto key(slot_type* slot)
|
||||
-> decltype(P::apply(ReturnKey(), element(slot))) {
|
||||
return P::apply(ReturnKey(), element(slot));
|
||||
}
|
||||
|
||||
// Returns the "value" (as opposed to the "key") portion of the element. Used
|
||||
// by maps to implement `operator[]`, `at()` and `insert_or_assign()`.
|
||||
template <class T, class P = Policy>
|
||||
static auto value(T* elem) -> decltype(P::value(elem)) {
|
||||
return P::value(elem);
|
||||
}
|
||||
|
||||
private:
|
||||
// Use auto -> decltype as an enabler.
|
||||
template <class Alloc, class P = Policy>
|
||||
static auto transfer_impl(Alloc* alloc, slot_type* new_slot,
|
||||
slot_type* old_slot, int)
|
||||
-> decltype((void)P::transfer(alloc, new_slot, old_slot)) {
|
||||
P::transfer(alloc, new_slot, old_slot);
|
||||
}
|
||||
template <class Alloc>
|
||||
static void transfer_impl(Alloc* alloc, slot_type* new_slot,
|
||||
slot_type* old_slot, char) {
|
||||
construct(alloc, new_slot, std::move(element(old_slot)));
|
||||
destroy(alloc, old_slot);
|
||||
}
|
||||
};
|
||||
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_INTERNAL_HASH_POLICY_TRAITS_H_
|
||||
@@ -0,0 +1,85 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Provides the internal API for hashtable_debug.h.
|
||||
|
||||
#ifndef ABSL_CONTAINER_INTERNAL_HASHTABLE_DEBUG_HOOKS_H_
|
||||
#define ABSL_CONTAINER_INTERNAL_HASHTABLE_DEBUG_HOOKS_H_
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
#include <algorithm>
|
||||
#include <type_traits>
|
||||
#include <vector>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
namespace hashtable_debug_internal {
|
||||
|
||||
// If it is a map, call get<0>().
|
||||
using std::get;
|
||||
template <typename T, typename = typename T::mapped_type>
|
||||
auto GetKey(const typename T::value_type& pair, int) -> decltype(get<0>(pair)) {
|
||||
return get<0>(pair);
|
||||
}
|
||||
|
||||
// If it is not a map, return the value directly.
|
||||
template <typename T>
|
||||
const typename T::key_type& GetKey(const typename T::key_type& key, char) {
|
||||
return key;
|
||||
}
|
||||
|
||||
// Containers should specialize this to provide debug information for that
|
||||
// container.
|
||||
template <class Container, typename Enabler = void>
|
||||
struct HashtableDebugAccess {
|
||||
// Returns the number of probes required to find `key` in `c`. The "number of
|
||||
// probes" is a concept that can vary by container. Implementations should
|
||||
// return 0 when `key` was found in the minimum number of operations and
|
||||
// should increment the result for each non-trivial operation required to find
|
||||
// `key`.
|
||||
//
|
||||
// The default implementation uses the bucket api from the standard and thus
|
||||
// works for `std::unordered_*` containers.
|
||||
static size_t GetNumProbes(const Container& c,
|
||||
const typename Container::key_type& key) {
|
||||
if (!c.bucket_count()) return {};
|
||||
size_t num_probes = 0;
|
||||
size_t bucket = c.bucket(key);
|
||||
for (auto it = c.begin(bucket), e = c.end(bucket);; ++it, ++num_probes) {
|
||||
if (it == e) return num_probes;
|
||||
if (c.key_eq()(key, GetKey<Container>(*it, 0))) return num_probes;
|
||||
}
|
||||
}
|
||||
|
||||
// Returns the number of bytes requested from the allocator by the container
|
||||
// and not freed.
|
||||
//
|
||||
// static size_t AllocatedByteSize(const Container& c);
|
||||
|
||||
// Returns a tight lower bound for AllocatedByteSize(c) where `c` is of type
|
||||
// `Container` and `c.size()` is equal to `num_elements`.
|
||||
//
|
||||
// static size_t LowerBoundAllocatedByteSize(size_t num_elements);
|
||||
};
|
||||
|
||||
} // namespace hashtable_debug_internal
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_INTERNAL_HASHTABLE_DEBUG_HOOKS_H_
|
||||
@@ -0,0 +1,269 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/container/internal/hashtablez_sampler.h"
|
||||
|
||||
#include <atomic>
|
||||
#include <cassert>
|
||||
#include <cmath>
|
||||
#include <functional>
|
||||
#include <limits>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/internal/exponential_biased.h"
|
||||
#include "absl/container/internal/have_sse.h"
|
||||
#include "absl/debugging/stacktrace.h"
|
||||
#include "absl/memory/memory.h"
|
||||
#include "absl/synchronization/mutex.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
constexpr int HashtablezInfo::kMaxStackDepth;
|
||||
|
||||
namespace {
|
||||
ABSL_CONST_INIT std::atomic<bool> g_hashtablez_enabled{
|
||||
false
|
||||
};
|
||||
ABSL_CONST_INIT std::atomic<int32_t> g_hashtablez_sample_parameter{1 << 10};
|
||||
ABSL_CONST_INIT std::atomic<int32_t> g_hashtablez_max_samples{1 << 20};
|
||||
|
||||
#if defined(ABSL_INTERNAL_HASHTABLEZ_SAMPLE)
|
||||
ABSL_PER_THREAD_TLS_KEYWORD absl::base_internal::ExponentialBiased
|
||||
g_exponential_biased_generator;
|
||||
#endif
|
||||
|
||||
} // namespace
|
||||
|
||||
#if defined(ABSL_INTERNAL_HASHTABLEZ_SAMPLE)
|
||||
ABSL_PER_THREAD_TLS_KEYWORD int64_t global_next_sample = 0;
|
||||
#endif // defined(ABSL_INTERNAL_HASHTABLEZ_SAMPLE)
|
||||
|
||||
HashtablezSampler& HashtablezSampler::Global() {
|
||||
static auto* sampler = new HashtablezSampler();
|
||||
return *sampler;
|
||||
}
|
||||
|
||||
HashtablezSampler::DisposeCallback HashtablezSampler::SetDisposeCallback(
|
||||
DisposeCallback f) {
|
||||
return dispose_.exchange(f, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
HashtablezInfo::HashtablezInfo() { PrepareForSampling(); }
|
||||
HashtablezInfo::~HashtablezInfo() = default;
|
||||
|
||||
void HashtablezInfo::PrepareForSampling() {
|
||||
capacity.store(0, std::memory_order_relaxed);
|
||||
size.store(0, std::memory_order_relaxed);
|
||||
num_erases.store(0, std::memory_order_relaxed);
|
||||
max_probe_length.store(0, std::memory_order_relaxed);
|
||||
total_probe_length.store(0, std::memory_order_relaxed);
|
||||
hashes_bitwise_or.store(0, std::memory_order_relaxed);
|
||||
hashes_bitwise_and.store(~size_t{}, std::memory_order_relaxed);
|
||||
|
||||
create_time = absl::Now();
|
||||
// The inliner makes hardcoded skip_count difficult (especially when combined
|
||||
// with LTO). We use the ability to exclude stacks by regex when encoding
|
||||
// instead.
|
||||
depth = absl::GetStackTrace(stack, HashtablezInfo::kMaxStackDepth,
|
||||
/* skip_count= */ 0);
|
||||
dead = nullptr;
|
||||
}
|
||||
|
||||
HashtablezSampler::HashtablezSampler()
|
||||
: dropped_samples_(0), size_estimate_(0), all_(nullptr), dispose_(nullptr) {
|
||||
absl::MutexLock l(&graveyard_.init_mu);
|
||||
graveyard_.dead = &graveyard_;
|
||||
}
|
||||
|
||||
HashtablezSampler::~HashtablezSampler() {
|
||||
HashtablezInfo* s = all_.load(std::memory_order_acquire);
|
||||
while (s != nullptr) {
|
||||
HashtablezInfo* next = s->next;
|
||||
delete s;
|
||||
s = next;
|
||||
}
|
||||
}
|
||||
|
||||
void HashtablezSampler::PushNew(HashtablezInfo* sample) {
|
||||
sample->next = all_.load(std::memory_order_relaxed);
|
||||
while (!all_.compare_exchange_weak(sample->next, sample,
|
||||
std::memory_order_release,
|
||||
std::memory_order_relaxed)) {
|
||||
}
|
||||
}
|
||||
|
||||
void HashtablezSampler::PushDead(HashtablezInfo* sample) {
|
||||
if (auto* dispose = dispose_.load(std::memory_order_relaxed)) {
|
||||
dispose(*sample);
|
||||
}
|
||||
|
||||
absl::MutexLock graveyard_lock(&graveyard_.init_mu);
|
||||
absl::MutexLock sample_lock(&sample->init_mu);
|
||||
sample->dead = graveyard_.dead;
|
||||
graveyard_.dead = sample;
|
||||
}
|
||||
|
||||
HashtablezInfo* HashtablezSampler::PopDead() {
|
||||
absl::MutexLock graveyard_lock(&graveyard_.init_mu);
|
||||
|
||||
// The list is circular, so eventually it collapses down to
|
||||
// graveyard_.dead == &graveyard_
|
||||
// when it is empty.
|
||||
HashtablezInfo* sample = graveyard_.dead;
|
||||
if (sample == &graveyard_) return nullptr;
|
||||
|
||||
absl::MutexLock sample_lock(&sample->init_mu);
|
||||
graveyard_.dead = sample->dead;
|
||||
sample->PrepareForSampling();
|
||||
return sample;
|
||||
}
|
||||
|
||||
HashtablezInfo* HashtablezSampler::Register() {
|
||||
int64_t size = size_estimate_.fetch_add(1, std::memory_order_relaxed);
|
||||
if (size > g_hashtablez_max_samples.load(std::memory_order_relaxed)) {
|
||||
size_estimate_.fetch_sub(1, std::memory_order_relaxed);
|
||||
dropped_samples_.fetch_add(1, std::memory_order_relaxed);
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
HashtablezInfo* sample = PopDead();
|
||||
if (sample == nullptr) {
|
||||
// Resurrection failed. Hire a new warlock.
|
||||
sample = new HashtablezInfo();
|
||||
PushNew(sample);
|
||||
}
|
||||
|
||||
return sample;
|
||||
}
|
||||
|
||||
void HashtablezSampler::Unregister(HashtablezInfo* sample) {
|
||||
PushDead(sample);
|
||||
size_estimate_.fetch_sub(1, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
int64_t HashtablezSampler::Iterate(
|
||||
const std::function<void(const HashtablezInfo& stack)>& f) {
|
||||
HashtablezInfo* s = all_.load(std::memory_order_acquire);
|
||||
while (s != nullptr) {
|
||||
absl::MutexLock l(&s->init_mu);
|
||||
if (s->dead == nullptr) {
|
||||
f(*s);
|
||||
}
|
||||
s = s->next;
|
||||
}
|
||||
|
||||
return dropped_samples_.load(std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
static bool ShouldForceSampling() {
|
||||
enum ForceState {
|
||||
kDontForce,
|
||||
kForce,
|
||||
kUninitialized
|
||||
};
|
||||
ABSL_CONST_INIT static std::atomic<ForceState> global_state{
|
||||
kUninitialized};
|
||||
ForceState state = global_state.load(std::memory_order_relaxed);
|
||||
if (ABSL_PREDICT_TRUE(state == kDontForce)) return false;
|
||||
|
||||
if (state == kUninitialized) {
|
||||
state = AbslContainerInternalSampleEverything() ? kForce : kDontForce;
|
||||
global_state.store(state, std::memory_order_relaxed);
|
||||
}
|
||||
return state == kForce;
|
||||
}
|
||||
|
||||
HashtablezInfo* SampleSlow(int64_t* next_sample) {
|
||||
if (ABSL_PREDICT_FALSE(ShouldForceSampling())) {
|
||||
*next_sample = 1;
|
||||
return HashtablezSampler::Global().Register();
|
||||
}
|
||||
|
||||
#if !defined(ABSL_INTERNAL_HASHTABLEZ_SAMPLE)
|
||||
*next_sample = std::numeric_limits<int64_t>::max();
|
||||
return nullptr;
|
||||
#else
|
||||
bool first = *next_sample < 0;
|
||||
*next_sample = g_exponential_biased_generator.GetStride(
|
||||
g_hashtablez_sample_parameter.load(std::memory_order_relaxed));
|
||||
// Small values of interval are equivalent to just sampling next time.
|
||||
ABSL_ASSERT(*next_sample >= 1);
|
||||
|
||||
// g_hashtablez_enabled can be dynamically flipped, we need to set a threshold
|
||||
// low enough that we will start sampling in a reasonable time, so we just use
|
||||
// the default sampling rate.
|
||||
if (!g_hashtablez_enabled.load(std::memory_order_relaxed)) return nullptr;
|
||||
|
||||
// We will only be negative on our first count, so we should just retry in
|
||||
// that case.
|
||||
if (first) {
|
||||
if (ABSL_PREDICT_TRUE(--*next_sample > 0)) return nullptr;
|
||||
return SampleSlow(next_sample);
|
||||
}
|
||||
|
||||
return HashtablezSampler::Global().Register();
|
||||
#endif
|
||||
}
|
||||
|
||||
void UnsampleSlow(HashtablezInfo* info) {
|
||||
HashtablezSampler::Global().Unregister(info);
|
||||
}
|
||||
|
||||
void RecordInsertSlow(HashtablezInfo* info, size_t hash,
|
||||
size_t distance_from_desired) {
|
||||
// SwissTables probe in groups of 16, so scale this to count items probes and
|
||||
// not offset from desired.
|
||||
size_t probe_length = distance_from_desired;
|
||||
#if SWISSTABLE_HAVE_SSE2
|
||||
probe_length /= 16;
|
||||
#else
|
||||
probe_length /= 8;
|
||||
#endif
|
||||
|
||||
info->hashes_bitwise_and.fetch_and(hash, std::memory_order_relaxed);
|
||||
info->hashes_bitwise_or.fetch_or(hash, std::memory_order_relaxed);
|
||||
info->max_probe_length.store(
|
||||
std::max(info->max_probe_length.load(std::memory_order_relaxed),
|
||||
probe_length),
|
||||
std::memory_order_relaxed);
|
||||
info->total_probe_length.fetch_add(probe_length, std::memory_order_relaxed);
|
||||
info->size.fetch_add(1, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
void SetHashtablezEnabled(bool enabled) {
|
||||
g_hashtablez_enabled.store(enabled, std::memory_order_release);
|
||||
}
|
||||
|
||||
void SetHashtablezSampleParameter(int32_t rate) {
|
||||
if (rate > 0) {
|
||||
g_hashtablez_sample_parameter.store(rate, std::memory_order_release);
|
||||
} else {
|
||||
ABSL_RAW_LOG(ERROR, "Invalid hashtablez sample rate: %lld",
|
||||
static_cast<long long>(rate)); // NOLINT(runtime/int)
|
||||
}
|
||||
}
|
||||
|
||||
void SetHashtablezMaxSamples(int32_t max) {
|
||||
if (max > 0) {
|
||||
g_hashtablez_max_samples.store(max, std::memory_order_release);
|
||||
} else {
|
||||
ABSL_RAW_LOG(ERROR, "Invalid hashtablez max samples: %lld",
|
||||
static_cast<long long>(max)); // NOLINT(runtime/int)
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
@@ -0,0 +1,297 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: hashtablez_sampler.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This header file defines the API for a low level library to sample hashtables
|
||||
// and collect runtime statistics about them.
|
||||
//
|
||||
// `HashtablezSampler` controls the lifecycle of `HashtablezInfo` objects which
|
||||
// store information about a single sample.
|
||||
//
|
||||
// `Record*` methods store information into samples.
|
||||
// `Sample()` and `Unsample()` make use of a single global sampler with
|
||||
// properties controlled by the flags hashtablez_enabled,
|
||||
// hashtablez_sample_rate, and hashtablez_max_samples.
|
||||
//
|
||||
// WARNING
|
||||
//
|
||||
// Using this sampling API may cause sampled Swiss tables to use the global
|
||||
// allocator (operator `new`) in addition to any custom allocator. If you
|
||||
// are using a table in an unusual circumstance where allocation or calling a
|
||||
// linux syscall is unacceptable, this could interfere.
|
||||
//
|
||||
// This utility is internal-only. Use at your own risk.
|
||||
|
||||
#ifndef ABSL_CONTAINER_INTERNAL_HASHTABLEZ_SAMPLER_H_
|
||||
#define ABSL_CONTAINER_INTERNAL_HASHTABLEZ_SAMPLER_H_
|
||||
|
||||
#include <atomic>
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
#include <vector>
|
||||
|
||||
#include "absl/base/internal/per_thread_tls.h"
|
||||
#include "absl/base/optimization.h"
|
||||
#include "absl/container/internal/have_sse.h"
|
||||
#include "absl/synchronization/mutex.h"
|
||||
#include "absl/utility/utility.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
|
||||
// Stores information about a sampled hashtable. All mutations to this *must*
|
||||
// be made through `Record*` functions below. All reads from this *must* only
|
||||
// occur in the callback to `HashtablezSampler::Iterate`.
|
||||
struct HashtablezInfo {
|
||||
// Constructs the object but does not fill in any fields.
|
||||
HashtablezInfo();
|
||||
~HashtablezInfo();
|
||||
HashtablezInfo(const HashtablezInfo&) = delete;
|
||||
HashtablezInfo& operator=(const HashtablezInfo&) = delete;
|
||||
|
||||
// Puts the object into a clean state, fills in the logically `const` members,
|
||||
// blocking for any readers that are currently sampling the object.
|
||||
void PrepareForSampling() ABSL_EXCLUSIVE_LOCKS_REQUIRED(init_mu);
|
||||
|
||||
// These fields are mutated by the various Record* APIs and need to be
|
||||
// thread-safe.
|
||||
std::atomic<size_t> capacity;
|
||||
std::atomic<size_t> size;
|
||||
std::atomic<size_t> num_erases;
|
||||
std::atomic<size_t> max_probe_length;
|
||||
std::atomic<size_t> total_probe_length;
|
||||
std::atomic<size_t> hashes_bitwise_or;
|
||||
std::atomic<size_t> hashes_bitwise_and;
|
||||
|
||||
// `HashtablezSampler` maintains intrusive linked lists for all samples. See
|
||||
// comments on `HashtablezSampler::all_` for details on these. `init_mu`
|
||||
// guards the ability to restore the sample to a pristine state. This
|
||||
// prevents races with sampling and resurrecting an object.
|
||||
absl::Mutex init_mu;
|
||||
HashtablezInfo* next;
|
||||
HashtablezInfo* dead ABSL_GUARDED_BY(init_mu);
|
||||
|
||||
// All of the fields below are set by `PrepareForSampling`, they must not be
|
||||
// mutated in `Record*` functions. They are logically `const` in that sense.
|
||||
// These are guarded by init_mu, but that is not externalized to clients, who
|
||||
// can only read them during `HashtablezSampler::Iterate` which will hold the
|
||||
// lock.
|
||||
static constexpr int kMaxStackDepth = 64;
|
||||
absl::Time create_time;
|
||||
int32_t depth;
|
||||
void* stack[kMaxStackDepth];
|
||||
};
|
||||
|
||||
inline void RecordRehashSlow(HashtablezInfo* info, size_t total_probe_length) {
|
||||
#if SWISSTABLE_HAVE_SSE2
|
||||
total_probe_length /= 16;
|
||||
#else
|
||||
total_probe_length /= 8;
|
||||
#endif
|
||||
info->total_probe_length.store(total_probe_length, std::memory_order_relaxed);
|
||||
info->num_erases.store(0, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
inline void RecordStorageChangedSlow(HashtablezInfo* info, size_t size,
|
||||
size_t capacity) {
|
||||
info->size.store(size, std::memory_order_relaxed);
|
||||
info->capacity.store(capacity, std::memory_order_relaxed);
|
||||
if (size == 0) {
|
||||
// This is a clear, reset the total/num_erases too.
|
||||
RecordRehashSlow(info, 0);
|
||||
}
|
||||
}
|
||||
|
||||
void RecordInsertSlow(HashtablezInfo* info, size_t hash,
|
||||
size_t distance_from_desired);
|
||||
|
||||
inline void RecordEraseSlow(HashtablezInfo* info) {
|
||||
info->size.fetch_sub(1, std::memory_order_relaxed);
|
||||
info->num_erases.fetch_add(1, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
HashtablezInfo* SampleSlow(int64_t* next_sample);
|
||||
void UnsampleSlow(HashtablezInfo* info);
|
||||
|
||||
class HashtablezInfoHandle {
|
||||
public:
|
||||
explicit HashtablezInfoHandle() : info_(nullptr) {}
|
||||
explicit HashtablezInfoHandle(HashtablezInfo* info) : info_(info) {}
|
||||
~HashtablezInfoHandle() {
|
||||
if (ABSL_PREDICT_TRUE(info_ == nullptr)) return;
|
||||
UnsampleSlow(info_);
|
||||
}
|
||||
|
||||
HashtablezInfoHandle(const HashtablezInfoHandle&) = delete;
|
||||
HashtablezInfoHandle& operator=(const HashtablezInfoHandle&) = delete;
|
||||
|
||||
HashtablezInfoHandle(HashtablezInfoHandle&& o) noexcept
|
||||
: info_(absl::exchange(o.info_, nullptr)) {}
|
||||
HashtablezInfoHandle& operator=(HashtablezInfoHandle&& o) noexcept {
|
||||
if (ABSL_PREDICT_FALSE(info_ != nullptr)) {
|
||||
UnsampleSlow(info_);
|
||||
}
|
||||
info_ = absl::exchange(o.info_, nullptr);
|
||||
return *this;
|
||||
}
|
||||
|
||||
inline void RecordStorageChanged(size_t size, size_t capacity) {
|
||||
if (ABSL_PREDICT_TRUE(info_ == nullptr)) return;
|
||||
RecordStorageChangedSlow(info_, size, capacity);
|
||||
}
|
||||
|
||||
inline void RecordRehash(size_t total_probe_length) {
|
||||
if (ABSL_PREDICT_TRUE(info_ == nullptr)) return;
|
||||
RecordRehashSlow(info_, total_probe_length);
|
||||
}
|
||||
|
||||
inline void RecordInsert(size_t hash, size_t distance_from_desired) {
|
||||
if (ABSL_PREDICT_TRUE(info_ == nullptr)) return;
|
||||
RecordInsertSlow(info_, hash, distance_from_desired);
|
||||
}
|
||||
|
||||
inline void RecordErase() {
|
||||
if (ABSL_PREDICT_TRUE(info_ == nullptr)) return;
|
||||
RecordEraseSlow(info_);
|
||||
}
|
||||
|
||||
friend inline void swap(HashtablezInfoHandle& lhs,
|
||||
HashtablezInfoHandle& rhs) {
|
||||
std::swap(lhs.info_, rhs.info_);
|
||||
}
|
||||
|
||||
private:
|
||||
friend class HashtablezInfoHandlePeer;
|
||||
HashtablezInfo* info_;
|
||||
};
|
||||
|
||||
#if defined(ABSL_INTERNAL_HASHTABLEZ_SAMPLE)
|
||||
#error ABSL_INTERNAL_HASHTABLEZ_SAMPLE cannot be directly set
|
||||
#endif // defined(ABSL_INTERNAL_HASHTABLEZ_SAMPLE)
|
||||
|
||||
#if (ABSL_PER_THREAD_TLS == 1) && !defined(ABSL_BUILD_DLL) && \
|
||||
!defined(ABSL_CONSUME_DLL)
|
||||
#define ABSL_INTERNAL_HASHTABLEZ_SAMPLE
|
||||
#endif
|
||||
|
||||
#if defined(ABSL_INTERNAL_HASHTABLEZ_SAMPLE)
|
||||
extern ABSL_PER_THREAD_TLS_KEYWORD int64_t global_next_sample;
|
||||
#endif // ABSL_PER_THREAD_TLS
|
||||
|
||||
// Returns an RAII sampling handle that manages registration and unregistation
|
||||
// with the global sampler.
|
||||
inline HashtablezInfoHandle Sample() {
|
||||
#if defined(ABSL_INTERNAL_HASHTABLEZ_SAMPLE)
|
||||
if (ABSL_PREDICT_TRUE(--global_next_sample > 0)) {
|
||||
return HashtablezInfoHandle(nullptr);
|
||||
}
|
||||
return HashtablezInfoHandle(SampleSlow(&global_next_sample));
|
||||
#else
|
||||
return HashtablezInfoHandle(nullptr);
|
||||
#endif // !ABSL_PER_THREAD_TLS
|
||||
}
|
||||
|
||||
// Holds samples and their associated stack traces with a soft limit of
|
||||
// `SetHashtablezMaxSamples()`.
|
||||
//
|
||||
// Thread safe.
|
||||
class HashtablezSampler {
|
||||
public:
|
||||
// Returns a global Sampler.
|
||||
static HashtablezSampler& Global();
|
||||
|
||||
HashtablezSampler();
|
||||
~HashtablezSampler();
|
||||
|
||||
// Registers for sampling. Returns an opaque registration info.
|
||||
HashtablezInfo* Register();
|
||||
|
||||
// Unregisters the sample.
|
||||
void Unregister(HashtablezInfo* sample);
|
||||
|
||||
// The dispose callback will be called on all samples the moment they are
|
||||
// being unregistered. Only affects samples that are unregistered after the
|
||||
// callback has been set.
|
||||
// Returns the previous callback.
|
||||
using DisposeCallback = void (*)(const HashtablezInfo&);
|
||||
DisposeCallback SetDisposeCallback(DisposeCallback f);
|
||||
|
||||
// Iterates over all the registered `StackInfo`s. Returning the number of
|
||||
// samples that have been dropped.
|
||||
int64_t Iterate(const std::function<void(const HashtablezInfo& stack)>& f);
|
||||
|
||||
private:
|
||||
void PushNew(HashtablezInfo* sample);
|
||||
void PushDead(HashtablezInfo* sample);
|
||||
HashtablezInfo* PopDead();
|
||||
|
||||
std::atomic<size_t> dropped_samples_;
|
||||
std::atomic<size_t> size_estimate_;
|
||||
|
||||
// Intrusive lock free linked lists for tracking samples.
|
||||
//
|
||||
// `all_` records all samples (they are never removed from this list) and is
|
||||
// terminated with a `nullptr`.
|
||||
//
|
||||
// `graveyard_.dead` is a circular linked list. When it is empty,
|
||||
// `graveyard_.dead == &graveyard`. The list is circular so that
|
||||
// every item on it (even the last) has a non-null dead pointer. This allows
|
||||
// `Iterate` to determine if a given sample is live or dead using only
|
||||
// information on the sample itself.
|
||||
//
|
||||
// For example, nodes [A, B, C, D, E] with [A, C, E] alive and [B, D] dead
|
||||
// looks like this (G is the Graveyard):
|
||||
//
|
||||
// +---+ +---+ +---+ +---+ +---+
|
||||
// all -->| A |--->| B |--->| C |--->| D |--->| E |
|
||||
// | | | | | | | | | |
|
||||
// +---+ | | +->| |-+ | | +->| |-+ | |
|
||||
// | G | +---+ | +---+ | +---+ | +---+ | +---+
|
||||
// | | | | | |
|
||||
// | | --------+ +--------+ |
|
||||
// +---+ |
|
||||
// ^ |
|
||||
// +--------------------------------------+
|
||||
//
|
||||
std::atomic<HashtablezInfo*> all_;
|
||||
HashtablezInfo graveyard_;
|
||||
|
||||
std::atomic<DisposeCallback> dispose_;
|
||||
};
|
||||
|
||||
// Enables or disables sampling for Swiss tables.
|
||||
void SetHashtablezEnabled(bool enabled);
|
||||
|
||||
// Sets the rate at which Swiss tables will be sampled.
|
||||
void SetHashtablezSampleParameter(int32_t rate);
|
||||
|
||||
// Sets a soft max for the number of samples that will be kept.
|
||||
void SetHashtablezMaxSamples(int32_t max);
|
||||
|
||||
// Configuration override.
|
||||
// This allows process-wide sampling without depending on order of
|
||||
// initialization of static storage duration objects.
|
||||
// The definition of this constant is weak, which allows us to inject a
|
||||
// different value for it at link time.
|
||||
extern "C" bool AbslContainerInternalSampleEverything();
|
||||
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_INTERNAL_HASHTABLEZ_SAMPLER_H_
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/container/internal/hashtablez_sampler.h"
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
|
||||
// See hashtablez_sampler.h for details.
|
||||
extern "C" ABSL_ATTRIBUTE_WEAK bool AbslContainerInternalSampleEverything() {
|
||||
return false;
|
||||
}
|
||||
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Shared config probing for SSE instructions used in Swiss tables.
|
||||
#ifndef ABSL_CONTAINER_INTERNAL_HAVE_SSE_H_
|
||||
#define ABSL_CONTAINER_INTERNAL_HAVE_SSE_H_
|
||||
|
||||
#ifndef SWISSTABLE_HAVE_SSE2
|
||||
#if defined(__SSE2__) || \
|
||||
(defined(_MSC_VER) && \
|
||||
(defined(_M_X64) || (defined(_M_IX86) && _M_IX86_FP >= 2)))
|
||||
#define SWISSTABLE_HAVE_SSE2 1
|
||||
#else
|
||||
#define SWISSTABLE_HAVE_SSE2 0
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#ifndef SWISSTABLE_HAVE_SSSE3
|
||||
#ifdef __SSSE3__
|
||||
#define SWISSTABLE_HAVE_SSSE3 1
|
||||
#else
|
||||
#define SWISSTABLE_HAVE_SSSE3 0
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#if SWISSTABLE_HAVE_SSSE3 && !SWISSTABLE_HAVE_SSE2
|
||||
#error "Bad configuration!"
|
||||
#endif
|
||||
|
||||
#if SWISSTABLE_HAVE_SSE2
|
||||
#include <emmintrin.h>
|
||||
#endif
|
||||
|
||||
#if SWISSTABLE_HAVE_SSSE3
|
||||
#include <tmmintrin.h>
|
||||
#endif
|
||||
|
||||
#endif // ABSL_CONTAINER_INTERNAL_HAVE_SSE_H_
|
||||
+892
@@ -0,0 +1,892 @@
|
||||
// Copyright 2019 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_CONTAINER_INTERNAL_INLINED_VECTOR_INTERNAL_H_
|
||||
#define ABSL_CONTAINER_INTERNAL_INLINED_VECTOR_INTERNAL_H_
|
||||
|
||||
#include <algorithm>
|
||||
#include <cstddef>
|
||||
#include <cstring>
|
||||
#include <iterator>
|
||||
#include <limits>
|
||||
#include <memory>
|
||||
#include <utility>
|
||||
|
||||
#include "absl/base/macros.h"
|
||||
#include "absl/container/internal/compressed_tuple.h"
|
||||
#include "absl/memory/memory.h"
|
||||
#include "absl/meta/type_traits.h"
|
||||
#include "absl/types/span.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace inlined_vector_internal {
|
||||
|
||||
template <typename Iterator>
|
||||
using IsAtLeastForwardIterator = std::is_convertible<
|
||||
typename std::iterator_traits<Iterator>::iterator_category,
|
||||
std::forward_iterator_tag>;
|
||||
|
||||
template <typename AllocatorType,
|
||||
typename ValueType =
|
||||
typename absl::allocator_traits<AllocatorType>::value_type>
|
||||
using IsMemcpyOk =
|
||||
absl::conjunction<std::is_same<AllocatorType, std::allocator<ValueType>>,
|
||||
absl::is_trivially_copy_constructible<ValueType>,
|
||||
absl::is_trivially_copy_assignable<ValueType>,
|
||||
absl::is_trivially_destructible<ValueType>>;
|
||||
|
||||
template <typename AllocatorType, typename Pointer, typename SizeType>
|
||||
void DestroyElements(AllocatorType* alloc_ptr, Pointer destroy_first,
|
||||
SizeType destroy_size) {
|
||||
using AllocatorTraits = absl::allocator_traits<AllocatorType>;
|
||||
|
||||
if (destroy_first != nullptr) {
|
||||
for (auto i = destroy_size; i != 0;) {
|
||||
--i;
|
||||
AllocatorTraits::destroy(*alloc_ptr, destroy_first + i);
|
||||
}
|
||||
|
||||
#if !defined(NDEBUG)
|
||||
{
|
||||
using ValueType = typename AllocatorTraits::value_type;
|
||||
|
||||
// Overwrite unused memory with `0xab` so we can catch uninitialized
|
||||
// usage.
|
||||
//
|
||||
// Cast to `void*` to tell the compiler that we don't care that we might
|
||||
// be scribbling on a vtable pointer.
|
||||
void* memory_ptr = destroy_first;
|
||||
auto memory_size = destroy_size * sizeof(ValueType);
|
||||
std::memset(memory_ptr, 0xab, memory_size);
|
||||
}
|
||||
#endif // !defined(NDEBUG)
|
||||
}
|
||||
}
|
||||
|
||||
template <typename AllocatorType, typename Pointer, typename ValueAdapter,
|
||||
typename SizeType>
|
||||
void ConstructElements(AllocatorType* alloc_ptr, Pointer construct_first,
|
||||
ValueAdapter* values_ptr, SizeType construct_size) {
|
||||
for (SizeType i = 0; i < construct_size; ++i) {
|
||||
ABSL_INTERNAL_TRY {
|
||||
values_ptr->ConstructNext(alloc_ptr, construct_first + i);
|
||||
}
|
||||
ABSL_INTERNAL_CATCH_ANY {
|
||||
inlined_vector_internal::DestroyElements(alloc_ptr, construct_first, i);
|
||||
ABSL_INTERNAL_RETHROW;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
template <typename Pointer, typename ValueAdapter, typename SizeType>
|
||||
void AssignElements(Pointer assign_first, ValueAdapter* values_ptr,
|
||||
SizeType assign_size) {
|
||||
for (SizeType i = 0; i < assign_size; ++i) {
|
||||
values_ptr->AssignNext(assign_first + i);
|
||||
}
|
||||
}
|
||||
|
||||
template <typename AllocatorType>
|
||||
struct StorageView {
|
||||
using AllocatorTraits = absl::allocator_traits<AllocatorType>;
|
||||
using Pointer = typename AllocatorTraits::pointer;
|
||||
using SizeType = typename AllocatorTraits::size_type;
|
||||
|
||||
Pointer data;
|
||||
SizeType size;
|
||||
SizeType capacity;
|
||||
};
|
||||
|
||||
template <typename AllocatorType, typename Iterator>
|
||||
class IteratorValueAdapter {
|
||||
using AllocatorTraits = absl::allocator_traits<AllocatorType>;
|
||||
using Pointer = typename AllocatorTraits::pointer;
|
||||
|
||||
public:
|
||||
explicit IteratorValueAdapter(const Iterator& it) : it_(it) {}
|
||||
|
||||
void ConstructNext(AllocatorType* alloc_ptr, Pointer construct_at) {
|
||||
AllocatorTraits::construct(*alloc_ptr, construct_at, *it_);
|
||||
++it_;
|
||||
}
|
||||
|
||||
void AssignNext(Pointer assign_at) {
|
||||
*assign_at = *it_;
|
||||
++it_;
|
||||
}
|
||||
|
||||
private:
|
||||
Iterator it_;
|
||||
};
|
||||
|
||||
template <typename AllocatorType>
|
||||
class CopyValueAdapter {
|
||||
using AllocatorTraits = absl::allocator_traits<AllocatorType>;
|
||||
using ValueType = typename AllocatorTraits::value_type;
|
||||
using Pointer = typename AllocatorTraits::pointer;
|
||||
using ConstPointer = typename AllocatorTraits::const_pointer;
|
||||
|
||||
public:
|
||||
explicit CopyValueAdapter(const ValueType& v) : ptr_(std::addressof(v)) {}
|
||||
|
||||
void ConstructNext(AllocatorType* alloc_ptr, Pointer construct_at) {
|
||||
AllocatorTraits::construct(*alloc_ptr, construct_at, *ptr_);
|
||||
}
|
||||
|
||||
void AssignNext(Pointer assign_at) { *assign_at = *ptr_; }
|
||||
|
||||
private:
|
||||
ConstPointer ptr_;
|
||||
};
|
||||
|
||||
template <typename AllocatorType>
|
||||
class DefaultValueAdapter {
|
||||
using AllocatorTraits = absl::allocator_traits<AllocatorType>;
|
||||
using ValueType = typename AllocatorTraits::value_type;
|
||||
using Pointer = typename AllocatorTraits::pointer;
|
||||
|
||||
public:
|
||||
explicit DefaultValueAdapter() {}
|
||||
|
||||
void ConstructNext(AllocatorType* alloc_ptr, Pointer construct_at) {
|
||||
AllocatorTraits::construct(*alloc_ptr, construct_at);
|
||||
}
|
||||
|
||||
void AssignNext(Pointer assign_at) { *assign_at = ValueType(); }
|
||||
};
|
||||
|
||||
template <typename AllocatorType>
|
||||
class AllocationTransaction {
|
||||
using AllocatorTraits = absl::allocator_traits<AllocatorType>;
|
||||
using Pointer = typename AllocatorTraits::pointer;
|
||||
using SizeType = typename AllocatorTraits::size_type;
|
||||
|
||||
public:
|
||||
explicit AllocationTransaction(AllocatorType* alloc_ptr)
|
||||
: alloc_data_(*alloc_ptr, nullptr) {}
|
||||
|
||||
~AllocationTransaction() {
|
||||
if (DidAllocate()) {
|
||||
AllocatorTraits::deallocate(GetAllocator(), GetData(), GetCapacity());
|
||||
}
|
||||
}
|
||||
|
||||
AllocationTransaction(const AllocationTransaction&) = delete;
|
||||
void operator=(const AllocationTransaction&) = delete;
|
||||
|
||||
AllocatorType& GetAllocator() { return alloc_data_.template get<0>(); }
|
||||
Pointer& GetData() { return alloc_data_.template get<1>(); }
|
||||
SizeType& GetCapacity() { return capacity_; }
|
||||
|
||||
bool DidAllocate() { return GetData() != nullptr; }
|
||||
Pointer Allocate(SizeType capacity) {
|
||||
GetData() = AllocatorTraits::allocate(GetAllocator(), capacity);
|
||||
GetCapacity() = capacity;
|
||||
return GetData();
|
||||
}
|
||||
|
||||
void Reset() {
|
||||
GetData() = nullptr;
|
||||
GetCapacity() = 0;
|
||||
}
|
||||
|
||||
private:
|
||||
container_internal::CompressedTuple<AllocatorType, Pointer> alloc_data_;
|
||||
SizeType capacity_ = 0;
|
||||
};
|
||||
|
||||
template <typename AllocatorType>
|
||||
class ConstructionTransaction {
|
||||
using AllocatorTraits = absl::allocator_traits<AllocatorType>;
|
||||
using Pointer = typename AllocatorTraits::pointer;
|
||||
using SizeType = typename AllocatorTraits::size_type;
|
||||
|
||||
public:
|
||||
explicit ConstructionTransaction(AllocatorType* alloc_ptr)
|
||||
: alloc_data_(*alloc_ptr, nullptr) {}
|
||||
|
||||
~ConstructionTransaction() {
|
||||
if (DidConstruct()) {
|
||||
inlined_vector_internal::DestroyElements(std::addressof(GetAllocator()),
|
||||
GetData(), GetSize());
|
||||
}
|
||||
}
|
||||
|
||||
ConstructionTransaction(const ConstructionTransaction&) = delete;
|
||||
void operator=(const ConstructionTransaction&) = delete;
|
||||
|
||||
AllocatorType& GetAllocator() { return alloc_data_.template get<0>(); }
|
||||
Pointer& GetData() { return alloc_data_.template get<1>(); }
|
||||
SizeType& GetSize() { return size_; }
|
||||
|
||||
bool DidConstruct() { return GetData() != nullptr; }
|
||||
template <typename ValueAdapter>
|
||||
void Construct(Pointer data, ValueAdapter* values_ptr, SizeType size) {
|
||||
inlined_vector_internal::ConstructElements(std::addressof(GetAllocator()),
|
||||
data, values_ptr, size);
|
||||
GetData() = data;
|
||||
GetSize() = size;
|
||||
}
|
||||
void Commit() {
|
||||
GetData() = nullptr;
|
||||
GetSize() = 0;
|
||||
}
|
||||
|
||||
private:
|
||||
container_internal::CompressedTuple<AllocatorType, Pointer> alloc_data_;
|
||||
SizeType size_ = 0;
|
||||
};
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
class Storage {
|
||||
public:
|
||||
using AllocatorTraits = absl::allocator_traits<A>;
|
||||
using allocator_type = typename AllocatorTraits::allocator_type;
|
||||
using value_type = typename AllocatorTraits::value_type;
|
||||
using pointer = typename AllocatorTraits::pointer;
|
||||
using const_pointer = typename AllocatorTraits::const_pointer;
|
||||
using size_type = typename AllocatorTraits::size_type;
|
||||
using difference_type = typename AllocatorTraits::difference_type;
|
||||
|
||||
using reference = value_type&;
|
||||
using const_reference = const value_type&;
|
||||
using RValueReference = value_type&&;
|
||||
using iterator = pointer;
|
||||
using const_iterator = const_pointer;
|
||||
using reverse_iterator = std::reverse_iterator<iterator>;
|
||||
using const_reverse_iterator = std::reverse_iterator<const_iterator>;
|
||||
using MoveIterator = std::move_iterator<iterator>;
|
||||
using IsMemcpyOk = inlined_vector_internal::IsMemcpyOk<allocator_type>;
|
||||
|
||||
using StorageView = inlined_vector_internal::StorageView<allocator_type>;
|
||||
|
||||
template <typename Iterator>
|
||||
using IteratorValueAdapter =
|
||||
inlined_vector_internal::IteratorValueAdapter<allocator_type, Iterator>;
|
||||
using CopyValueAdapter =
|
||||
inlined_vector_internal::CopyValueAdapter<allocator_type>;
|
||||
using DefaultValueAdapter =
|
||||
inlined_vector_internal::DefaultValueAdapter<allocator_type>;
|
||||
|
||||
using AllocationTransaction =
|
||||
inlined_vector_internal::AllocationTransaction<allocator_type>;
|
||||
using ConstructionTransaction =
|
||||
inlined_vector_internal::ConstructionTransaction<allocator_type>;
|
||||
|
||||
static size_type NextCapacity(size_type current_capacity) {
|
||||
return current_capacity * 2;
|
||||
}
|
||||
|
||||
static size_type ComputeCapacity(size_type current_capacity,
|
||||
size_type requested_capacity) {
|
||||
return (std::max)(NextCapacity(current_capacity), requested_capacity);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Storage Constructors and Destructor
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
Storage() : metadata_() {}
|
||||
|
||||
explicit Storage(const allocator_type& alloc) : metadata_(alloc, {}) {}
|
||||
|
||||
~Storage() {
|
||||
pointer data = GetIsAllocated() ? GetAllocatedData() : GetInlinedData();
|
||||
inlined_vector_internal::DestroyElements(GetAllocPtr(), data, GetSize());
|
||||
DeallocateIfAllocated();
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Storage Member Accessors
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
size_type& GetSizeAndIsAllocated() { return metadata_.template get<1>(); }
|
||||
|
||||
const size_type& GetSizeAndIsAllocated() const {
|
||||
return metadata_.template get<1>();
|
||||
}
|
||||
|
||||
size_type GetSize() const { return GetSizeAndIsAllocated() >> 1; }
|
||||
|
||||
bool GetIsAllocated() const { return GetSizeAndIsAllocated() & 1; }
|
||||
|
||||
pointer GetAllocatedData() { return data_.allocated.allocated_data; }
|
||||
|
||||
const_pointer GetAllocatedData() const {
|
||||
return data_.allocated.allocated_data;
|
||||
}
|
||||
|
||||
pointer GetInlinedData() {
|
||||
return reinterpret_cast<pointer>(
|
||||
std::addressof(data_.inlined.inlined_data[0]));
|
||||
}
|
||||
|
||||
const_pointer GetInlinedData() const {
|
||||
return reinterpret_cast<const_pointer>(
|
||||
std::addressof(data_.inlined.inlined_data[0]));
|
||||
}
|
||||
|
||||
size_type GetAllocatedCapacity() const {
|
||||
return data_.allocated.allocated_capacity;
|
||||
}
|
||||
|
||||
size_type GetInlinedCapacity() const { return static_cast<size_type>(N); }
|
||||
|
||||
StorageView MakeStorageView() {
|
||||
return GetIsAllocated()
|
||||
? StorageView{GetAllocatedData(), GetSize(),
|
||||
GetAllocatedCapacity()}
|
||||
: StorageView{GetInlinedData(), GetSize(), GetInlinedCapacity()};
|
||||
}
|
||||
|
||||
allocator_type* GetAllocPtr() {
|
||||
return std::addressof(metadata_.template get<0>());
|
||||
}
|
||||
|
||||
const allocator_type* GetAllocPtr() const {
|
||||
return std::addressof(metadata_.template get<0>());
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Storage Member Mutators
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
template <typename ValueAdapter>
|
||||
void Initialize(ValueAdapter values, size_type new_size);
|
||||
|
||||
template <typename ValueAdapter>
|
||||
void Assign(ValueAdapter values, size_type new_size);
|
||||
|
||||
template <typename ValueAdapter>
|
||||
void Resize(ValueAdapter values, size_type new_size);
|
||||
|
||||
template <typename ValueAdapter>
|
||||
iterator Insert(const_iterator pos, ValueAdapter values,
|
||||
size_type insert_count);
|
||||
|
||||
template <typename... Args>
|
||||
reference EmplaceBack(Args&&... args);
|
||||
|
||||
iterator Erase(const_iterator from, const_iterator to);
|
||||
|
||||
void Reserve(size_type requested_capacity);
|
||||
|
||||
void ShrinkToFit();
|
||||
|
||||
void Swap(Storage* other_storage_ptr);
|
||||
|
||||
void SetIsAllocated() {
|
||||
GetSizeAndIsAllocated() |= static_cast<size_type>(1);
|
||||
}
|
||||
|
||||
void UnsetIsAllocated() {
|
||||
GetSizeAndIsAllocated() &= ((std::numeric_limits<size_type>::max)() - 1);
|
||||
}
|
||||
|
||||
void SetSize(size_type size) {
|
||||
GetSizeAndIsAllocated() =
|
||||
(size << 1) | static_cast<size_type>(GetIsAllocated());
|
||||
}
|
||||
|
||||
void SetAllocatedSize(size_type size) {
|
||||
GetSizeAndIsAllocated() = (size << 1) | static_cast<size_type>(1);
|
||||
}
|
||||
|
||||
void SetInlinedSize(size_type size) {
|
||||
GetSizeAndIsAllocated() = size << static_cast<size_type>(1);
|
||||
}
|
||||
|
||||
void AddSize(size_type count) {
|
||||
GetSizeAndIsAllocated() += count << static_cast<size_type>(1);
|
||||
}
|
||||
|
||||
void SubtractSize(size_type count) {
|
||||
assert(count <= GetSize());
|
||||
|
||||
GetSizeAndIsAllocated() -= count << static_cast<size_type>(1);
|
||||
}
|
||||
|
||||
void SetAllocatedData(pointer data, size_type capacity) {
|
||||
data_.allocated.allocated_data = data;
|
||||
data_.allocated.allocated_capacity = capacity;
|
||||
}
|
||||
|
||||
void AcquireAllocatedData(AllocationTransaction* allocation_tx_ptr) {
|
||||
SetAllocatedData(allocation_tx_ptr->GetData(),
|
||||
allocation_tx_ptr->GetCapacity());
|
||||
|
||||
allocation_tx_ptr->Reset();
|
||||
}
|
||||
|
||||
void MemcpyFrom(const Storage& other_storage) {
|
||||
assert(IsMemcpyOk::value || other_storage.GetIsAllocated());
|
||||
|
||||
GetSizeAndIsAllocated() = other_storage.GetSizeAndIsAllocated();
|
||||
data_ = other_storage.data_;
|
||||
}
|
||||
|
||||
void DeallocateIfAllocated() {
|
||||
if (GetIsAllocated()) {
|
||||
AllocatorTraits::deallocate(*GetAllocPtr(), GetAllocatedData(),
|
||||
GetAllocatedCapacity());
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
using Metadata =
|
||||
container_internal::CompressedTuple<allocator_type, size_type>;
|
||||
|
||||
struct Allocated {
|
||||
pointer allocated_data;
|
||||
size_type allocated_capacity;
|
||||
};
|
||||
|
||||
struct Inlined {
|
||||
alignas(value_type) char inlined_data[sizeof(value_type[N])];
|
||||
};
|
||||
|
||||
union Data {
|
||||
Allocated allocated;
|
||||
Inlined inlined;
|
||||
};
|
||||
|
||||
Metadata metadata_;
|
||||
Data data_;
|
||||
};
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
template <typename ValueAdapter>
|
||||
auto Storage<T, N, A>::Initialize(ValueAdapter values, size_type new_size)
|
||||
-> void {
|
||||
// Only callable from constructors!
|
||||
assert(!GetIsAllocated());
|
||||
assert(GetSize() == 0);
|
||||
|
||||
pointer construct_data;
|
||||
if (new_size > GetInlinedCapacity()) {
|
||||
// Because this is only called from the `InlinedVector` constructors, it's
|
||||
// safe to take on the allocation with size `0`. If `ConstructElements(...)`
|
||||
// throws, deallocation will be automatically handled by `~Storage()`.
|
||||
size_type new_capacity = ComputeCapacity(GetInlinedCapacity(), new_size);
|
||||
construct_data = AllocatorTraits::allocate(*GetAllocPtr(), new_capacity);
|
||||
SetAllocatedData(construct_data, new_capacity);
|
||||
SetIsAllocated();
|
||||
} else {
|
||||
construct_data = GetInlinedData();
|
||||
}
|
||||
|
||||
inlined_vector_internal::ConstructElements(GetAllocPtr(), construct_data,
|
||||
&values, new_size);
|
||||
|
||||
// Since the initial size was guaranteed to be `0` and the allocated bit is
|
||||
// already correct for either case, *adding* `new_size` gives us the correct
|
||||
// result faster than setting it directly.
|
||||
AddSize(new_size);
|
||||
}
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
template <typename ValueAdapter>
|
||||
auto Storage<T, N, A>::Assign(ValueAdapter values, size_type new_size) -> void {
|
||||
StorageView storage_view = MakeStorageView();
|
||||
|
||||
AllocationTransaction allocation_tx(GetAllocPtr());
|
||||
|
||||
absl::Span<value_type> assign_loop;
|
||||
absl::Span<value_type> construct_loop;
|
||||
absl::Span<value_type> destroy_loop;
|
||||
|
||||
if (new_size > storage_view.capacity) {
|
||||
size_type new_capacity = ComputeCapacity(storage_view.capacity, new_size);
|
||||
construct_loop = {allocation_tx.Allocate(new_capacity), new_size};
|
||||
destroy_loop = {storage_view.data, storage_view.size};
|
||||
} else if (new_size > storage_view.size) {
|
||||
assign_loop = {storage_view.data, storage_view.size};
|
||||
construct_loop = {storage_view.data + storage_view.size,
|
||||
new_size - storage_view.size};
|
||||
} else {
|
||||
assign_loop = {storage_view.data, new_size};
|
||||
destroy_loop = {storage_view.data + new_size, storage_view.size - new_size};
|
||||
}
|
||||
|
||||
inlined_vector_internal::AssignElements(assign_loop.data(), &values,
|
||||
assign_loop.size());
|
||||
|
||||
inlined_vector_internal::ConstructElements(
|
||||
GetAllocPtr(), construct_loop.data(), &values, construct_loop.size());
|
||||
|
||||
inlined_vector_internal::DestroyElements(GetAllocPtr(), destroy_loop.data(),
|
||||
destroy_loop.size());
|
||||
|
||||
if (allocation_tx.DidAllocate()) {
|
||||
DeallocateIfAllocated();
|
||||
AcquireAllocatedData(&allocation_tx);
|
||||
SetIsAllocated();
|
||||
}
|
||||
|
||||
SetSize(new_size);
|
||||
}
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
template <typename ValueAdapter>
|
||||
auto Storage<T, N, A>::Resize(ValueAdapter values, size_type new_size) -> void {
|
||||
StorageView storage_view = MakeStorageView();
|
||||
|
||||
IteratorValueAdapter<MoveIterator> move_values(
|
||||
MoveIterator(storage_view.data));
|
||||
|
||||
AllocationTransaction allocation_tx(GetAllocPtr());
|
||||
ConstructionTransaction construction_tx(GetAllocPtr());
|
||||
|
||||
absl::Span<value_type> construct_loop;
|
||||
absl::Span<value_type> move_construct_loop;
|
||||
absl::Span<value_type> destroy_loop;
|
||||
|
||||
if (new_size > storage_view.capacity) {
|
||||
size_type new_capacity = ComputeCapacity(storage_view.capacity, new_size);
|
||||
pointer new_data = allocation_tx.Allocate(new_capacity);
|
||||
construct_loop = {new_data + storage_view.size,
|
||||
new_size - storage_view.size};
|
||||
move_construct_loop = {new_data, storage_view.size};
|
||||
destroy_loop = {storage_view.data, storage_view.size};
|
||||
} else if (new_size > storage_view.size) {
|
||||
construct_loop = {storage_view.data + storage_view.size,
|
||||
new_size - storage_view.size};
|
||||
} else {
|
||||
destroy_loop = {storage_view.data + new_size, storage_view.size - new_size};
|
||||
}
|
||||
|
||||
construction_tx.Construct(construct_loop.data(), &values,
|
||||
construct_loop.size());
|
||||
|
||||
inlined_vector_internal::ConstructElements(
|
||||
GetAllocPtr(), move_construct_loop.data(), &move_values,
|
||||
move_construct_loop.size());
|
||||
|
||||
inlined_vector_internal::DestroyElements(GetAllocPtr(), destroy_loop.data(),
|
||||
destroy_loop.size());
|
||||
|
||||
construction_tx.Commit();
|
||||
if (allocation_tx.DidAllocate()) {
|
||||
DeallocateIfAllocated();
|
||||
AcquireAllocatedData(&allocation_tx);
|
||||
SetIsAllocated();
|
||||
}
|
||||
|
||||
SetSize(new_size);
|
||||
}
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
template <typename ValueAdapter>
|
||||
auto Storage<T, N, A>::Insert(const_iterator pos, ValueAdapter values,
|
||||
size_type insert_count) -> iterator {
|
||||
StorageView storage_view = MakeStorageView();
|
||||
|
||||
size_type insert_index =
|
||||
std::distance(const_iterator(storage_view.data), pos);
|
||||
size_type insert_end_index = insert_index + insert_count;
|
||||
size_type new_size = storage_view.size + insert_count;
|
||||
|
||||
if (new_size > storage_view.capacity) {
|
||||
AllocationTransaction allocation_tx(GetAllocPtr());
|
||||
ConstructionTransaction construction_tx(GetAllocPtr());
|
||||
ConstructionTransaction move_construciton_tx(GetAllocPtr());
|
||||
|
||||
IteratorValueAdapter<MoveIterator> move_values(
|
||||
MoveIterator(storage_view.data));
|
||||
|
||||
size_type new_capacity = ComputeCapacity(storage_view.capacity, new_size);
|
||||
pointer new_data = allocation_tx.Allocate(new_capacity);
|
||||
|
||||
construction_tx.Construct(new_data + insert_index, &values, insert_count);
|
||||
|
||||
move_construciton_tx.Construct(new_data, &move_values, insert_index);
|
||||
|
||||
inlined_vector_internal::ConstructElements(
|
||||
GetAllocPtr(), new_data + insert_end_index, &move_values,
|
||||
storage_view.size - insert_index);
|
||||
|
||||
inlined_vector_internal::DestroyElements(GetAllocPtr(), storage_view.data,
|
||||
storage_view.size);
|
||||
|
||||
construction_tx.Commit();
|
||||
move_construciton_tx.Commit();
|
||||
DeallocateIfAllocated();
|
||||
AcquireAllocatedData(&allocation_tx);
|
||||
|
||||
SetAllocatedSize(new_size);
|
||||
return iterator(new_data + insert_index);
|
||||
} else {
|
||||
size_type move_construction_destination_index =
|
||||
(std::max)(insert_end_index, storage_view.size);
|
||||
|
||||
ConstructionTransaction move_construction_tx(GetAllocPtr());
|
||||
|
||||
IteratorValueAdapter<MoveIterator> move_construction_values(
|
||||
MoveIterator(storage_view.data +
|
||||
(move_construction_destination_index - insert_count)));
|
||||
absl::Span<value_type> move_construction = {
|
||||
storage_view.data + move_construction_destination_index,
|
||||
new_size - move_construction_destination_index};
|
||||
|
||||
pointer move_assignment_values = storage_view.data + insert_index;
|
||||
absl::Span<value_type> move_assignment = {
|
||||
storage_view.data + insert_end_index,
|
||||
move_construction_destination_index - insert_end_index};
|
||||
|
||||
absl::Span<value_type> insert_assignment = {move_assignment_values,
|
||||
move_construction.size()};
|
||||
|
||||
absl::Span<value_type> insert_construction = {
|
||||
insert_assignment.data() + insert_assignment.size(),
|
||||
insert_count - insert_assignment.size()};
|
||||
|
||||
move_construction_tx.Construct(move_construction.data(),
|
||||
&move_construction_values,
|
||||
move_construction.size());
|
||||
|
||||
for (pointer destination = move_assignment.data() + move_assignment.size(),
|
||||
last_destination = move_assignment.data(),
|
||||
source = move_assignment_values + move_assignment.size();
|
||||
;) {
|
||||
--destination;
|
||||
--source;
|
||||
if (destination < last_destination) break;
|
||||
*destination = std::move(*source);
|
||||
}
|
||||
|
||||
inlined_vector_internal::AssignElements(insert_assignment.data(), &values,
|
||||
insert_assignment.size());
|
||||
|
||||
inlined_vector_internal::ConstructElements(
|
||||
GetAllocPtr(), insert_construction.data(), &values,
|
||||
insert_construction.size());
|
||||
|
||||
move_construction_tx.Commit();
|
||||
|
||||
AddSize(insert_count);
|
||||
return iterator(storage_view.data + insert_index);
|
||||
}
|
||||
}
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
template <typename... Args>
|
||||
auto Storage<T, N, A>::EmplaceBack(Args&&... args) -> reference {
|
||||
StorageView storage_view = MakeStorageView();
|
||||
|
||||
AllocationTransaction allocation_tx(GetAllocPtr());
|
||||
|
||||
IteratorValueAdapter<MoveIterator> move_values(
|
||||
MoveIterator(storage_view.data));
|
||||
|
||||
pointer construct_data;
|
||||
if (storage_view.size == storage_view.capacity) {
|
||||
size_type new_capacity = NextCapacity(storage_view.capacity);
|
||||
construct_data = allocation_tx.Allocate(new_capacity);
|
||||
} else {
|
||||
construct_data = storage_view.data;
|
||||
}
|
||||
|
||||
pointer last_ptr = construct_data + storage_view.size;
|
||||
|
||||
AllocatorTraits::construct(*GetAllocPtr(), last_ptr,
|
||||
std::forward<Args>(args)...);
|
||||
|
||||
if (allocation_tx.DidAllocate()) {
|
||||
ABSL_INTERNAL_TRY {
|
||||
inlined_vector_internal::ConstructElements(
|
||||
GetAllocPtr(), allocation_tx.GetData(), &move_values,
|
||||
storage_view.size);
|
||||
}
|
||||
ABSL_INTERNAL_CATCH_ANY {
|
||||
AllocatorTraits::destroy(*GetAllocPtr(), last_ptr);
|
||||
ABSL_INTERNAL_RETHROW;
|
||||
}
|
||||
|
||||
inlined_vector_internal::DestroyElements(GetAllocPtr(), storage_view.data,
|
||||
storage_view.size);
|
||||
|
||||
DeallocateIfAllocated();
|
||||
AcquireAllocatedData(&allocation_tx);
|
||||
SetIsAllocated();
|
||||
}
|
||||
|
||||
AddSize(1);
|
||||
return *last_ptr;
|
||||
}
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
auto Storage<T, N, A>::Erase(const_iterator from, const_iterator to)
|
||||
-> iterator {
|
||||
StorageView storage_view = MakeStorageView();
|
||||
|
||||
size_type erase_size = std::distance(from, to);
|
||||
size_type erase_index =
|
||||
std::distance(const_iterator(storage_view.data), from);
|
||||
size_type erase_end_index = erase_index + erase_size;
|
||||
|
||||
IteratorValueAdapter<MoveIterator> move_values(
|
||||
MoveIterator(storage_view.data + erase_end_index));
|
||||
|
||||
inlined_vector_internal::AssignElements(storage_view.data + erase_index,
|
||||
&move_values,
|
||||
storage_view.size - erase_end_index);
|
||||
|
||||
inlined_vector_internal::DestroyElements(
|
||||
GetAllocPtr(), storage_view.data + (storage_view.size - erase_size),
|
||||
erase_size);
|
||||
|
||||
SubtractSize(erase_size);
|
||||
return iterator(storage_view.data + erase_index);
|
||||
}
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
auto Storage<T, N, A>::Reserve(size_type requested_capacity) -> void {
|
||||
StorageView storage_view = MakeStorageView();
|
||||
|
||||
if (ABSL_PREDICT_FALSE(requested_capacity <= storage_view.capacity)) return;
|
||||
|
||||
AllocationTransaction allocation_tx(GetAllocPtr());
|
||||
|
||||
IteratorValueAdapter<MoveIterator> move_values(
|
||||
MoveIterator(storage_view.data));
|
||||
|
||||
size_type new_capacity =
|
||||
ComputeCapacity(storage_view.capacity, requested_capacity);
|
||||
pointer new_data = allocation_tx.Allocate(new_capacity);
|
||||
|
||||
inlined_vector_internal::ConstructElements(GetAllocPtr(), new_data,
|
||||
&move_values, storage_view.size);
|
||||
|
||||
inlined_vector_internal::DestroyElements(GetAllocPtr(), storage_view.data,
|
||||
storage_view.size);
|
||||
|
||||
DeallocateIfAllocated();
|
||||
AcquireAllocatedData(&allocation_tx);
|
||||
SetIsAllocated();
|
||||
}
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
auto Storage<T, N, A>::ShrinkToFit() -> void {
|
||||
// May only be called on allocated instances!
|
||||
assert(GetIsAllocated());
|
||||
|
||||
StorageView storage_view{GetAllocatedData(), GetSize(),
|
||||
GetAllocatedCapacity()};
|
||||
|
||||
if (ABSL_PREDICT_FALSE(storage_view.size == storage_view.capacity)) return;
|
||||
|
||||
AllocationTransaction allocation_tx(GetAllocPtr());
|
||||
|
||||
IteratorValueAdapter<MoveIterator> move_values(
|
||||
MoveIterator(storage_view.data));
|
||||
|
||||
pointer construct_data;
|
||||
if (storage_view.size > GetInlinedCapacity()) {
|
||||
size_type new_capacity = storage_view.size;
|
||||
construct_data = allocation_tx.Allocate(new_capacity);
|
||||
} else {
|
||||
construct_data = GetInlinedData();
|
||||
}
|
||||
|
||||
ABSL_INTERNAL_TRY {
|
||||
inlined_vector_internal::ConstructElements(GetAllocPtr(), construct_data,
|
||||
&move_values, storage_view.size);
|
||||
}
|
||||
ABSL_INTERNAL_CATCH_ANY {
|
||||
SetAllocatedData(storage_view.data, storage_view.capacity);
|
||||
ABSL_INTERNAL_RETHROW;
|
||||
}
|
||||
|
||||
inlined_vector_internal::DestroyElements(GetAllocPtr(), storage_view.data,
|
||||
storage_view.size);
|
||||
|
||||
AllocatorTraits::deallocate(*GetAllocPtr(), storage_view.data,
|
||||
storage_view.capacity);
|
||||
|
||||
if (allocation_tx.DidAllocate()) {
|
||||
AcquireAllocatedData(&allocation_tx);
|
||||
} else {
|
||||
UnsetIsAllocated();
|
||||
}
|
||||
}
|
||||
|
||||
template <typename T, size_t N, typename A>
|
||||
auto Storage<T, N, A>::Swap(Storage* other_storage_ptr) -> void {
|
||||
using std::swap;
|
||||
assert(this != other_storage_ptr);
|
||||
|
||||
if (GetIsAllocated() && other_storage_ptr->GetIsAllocated()) {
|
||||
swap(data_.allocated, other_storage_ptr->data_.allocated);
|
||||
} else if (!GetIsAllocated() && !other_storage_ptr->GetIsAllocated()) {
|
||||
Storage* small_ptr = this;
|
||||
Storage* large_ptr = other_storage_ptr;
|
||||
if (small_ptr->GetSize() > large_ptr->GetSize()) swap(small_ptr, large_ptr);
|
||||
|
||||
for (size_type i = 0; i < small_ptr->GetSize(); ++i) {
|
||||
swap(small_ptr->GetInlinedData()[i], large_ptr->GetInlinedData()[i]);
|
||||
}
|
||||
|
||||
IteratorValueAdapter<MoveIterator> move_values(
|
||||
MoveIterator(large_ptr->GetInlinedData() + small_ptr->GetSize()));
|
||||
|
||||
inlined_vector_internal::ConstructElements(
|
||||
large_ptr->GetAllocPtr(),
|
||||
small_ptr->GetInlinedData() + small_ptr->GetSize(), &move_values,
|
||||
large_ptr->GetSize() - small_ptr->GetSize());
|
||||
|
||||
inlined_vector_internal::DestroyElements(
|
||||
large_ptr->GetAllocPtr(),
|
||||
large_ptr->GetInlinedData() + small_ptr->GetSize(),
|
||||
large_ptr->GetSize() - small_ptr->GetSize());
|
||||
} else {
|
||||
Storage* allocated_ptr = this;
|
||||
Storage* inlined_ptr = other_storage_ptr;
|
||||
if (!allocated_ptr->GetIsAllocated()) swap(allocated_ptr, inlined_ptr);
|
||||
|
||||
StorageView allocated_storage_view{allocated_ptr->GetAllocatedData(),
|
||||
allocated_ptr->GetSize(),
|
||||
allocated_ptr->GetAllocatedCapacity()};
|
||||
|
||||
IteratorValueAdapter<MoveIterator> move_values(
|
||||
MoveIterator(inlined_ptr->GetInlinedData()));
|
||||
|
||||
ABSL_INTERNAL_TRY {
|
||||
inlined_vector_internal::ConstructElements(
|
||||
inlined_ptr->GetAllocPtr(), allocated_ptr->GetInlinedData(),
|
||||
&move_values, inlined_ptr->GetSize());
|
||||
}
|
||||
ABSL_INTERNAL_CATCH_ANY {
|
||||
allocated_ptr->SetAllocatedData(allocated_storage_view.data,
|
||||
allocated_storage_view.capacity);
|
||||
ABSL_INTERNAL_RETHROW;
|
||||
}
|
||||
|
||||
inlined_vector_internal::DestroyElements(inlined_ptr->GetAllocPtr(),
|
||||
inlined_ptr->GetInlinedData(),
|
||||
inlined_ptr->GetSize());
|
||||
|
||||
inlined_ptr->SetAllocatedData(allocated_storage_view.data,
|
||||
allocated_storage_view.capacity);
|
||||
}
|
||||
|
||||
swap(GetSizeAndIsAllocated(), other_storage_ptr->GetSizeAndIsAllocated());
|
||||
swap(*GetAllocPtr(), *other_storage_ptr->GetAllocPtr());
|
||||
}
|
||||
|
||||
} // namespace inlined_vector_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_INTERNAL_INLINED_VECTOR_INTERNAL_H_
|
||||
+741
@@ -0,0 +1,741 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// MOTIVATION AND TUTORIAL
|
||||
//
|
||||
// If you want to put in a single heap allocation N doubles followed by M ints,
|
||||
// it's easy if N and M are known at compile time.
|
||||
//
|
||||
// struct S {
|
||||
// double a[N];
|
||||
// int b[M];
|
||||
// };
|
||||
//
|
||||
// S* p = new S;
|
||||
//
|
||||
// But what if N and M are known only in run time? Class template Layout to the
|
||||
// rescue! It's a portable generalization of the technique known as struct hack.
|
||||
//
|
||||
// // This object will tell us everything we need to know about the memory
|
||||
// // layout of double[N] followed by int[M]. It's structurally identical to
|
||||
// // size_t[2] that stores N and M. It's very cheap to create.
|
||||
// const Layout<double, int> layout(N, M);
|
||||
//
|
||||
// // Allocate enough memory for both arrays. `AllocSize()` tells us how much
|
||||
// // memory is needed. We are free to use any allocation function we want as
|
||||
// // long as it returns aligned memory.
|
||||
// std::unique_ptr<unsigned char[]> p(new unsigned char[layout.AllocSize()]);
|
||||
//
|
||||
// // Obtain the pointer to the array of doubles.
|
||||
// // Equivalent to `reinterpret_cast<double*>(p.get())`.
|
||||
// //
|
||||
// // We could have written layout.Pointer<0>(p) instead. If all the types are
|
||||
// // unique you can use either form, but if some types are repeated you must
|
||||
// // use the index form.
|
||||
// double* a = layout.Pointer<double>(p.get());
|
||||
//
|
||||
// // Obtain the pointer to the array of ints.
|
||||
// // Equivalent to `reinterpret_cast<int*>(p.get() + N * 8)`.
|
||||
// int* b = layout.Pointer<int>(p);
|
||||
//
|
||||
// If we are unable to specify sizes of all fields, we can pass as many sizes as
|
||||
// we can to `Partial()`. In return, it'll allow us to access the fields whose
|
||||
// locations and sizes can be computed from the provided information.
|
||||
// `Partial()` comes in handy when the array sizes are embedded into the
|
||||
// allocation.
|
||||
//
|
||||
// // size_t[1] containing N, size_t[1] containing M, double[N], int[M].
|
||||
// using L = Layout<size_t, size_t, double, int>;
|
||||
//
|
||||
// unsigned char* Allocate(size_t n, size_t m) {
|
||||
// const L layout(1, 1, n, m);
|
||||
// unsigned char* p = new unsigned char[layout.AllocSize()];
|
||||
// *layout.Pointer<0>(p) = n;
|
||||
// *layout.Pointer<1>(p) = m;
|
||||
// return p;
|
||||
// }
|
||||
//
|
||||
// void Use(unsigned char* p) {
|
||||
// // First, extract N and M.
|
||||
// // Specify that the first array has only one element. Using `prefix` we
|
||||
// // can access the first two arrays but not more.
|
||||
// constexpr auto prefix = L::Partial(1);
|
||||
// size_t n = *prefix.Pointer<0>(p);
|
||||
// size_t m = *prefix.Pointer<1>(p);
|
||||
//
|
||||
// // Now we can get pointers to the payload.
|
||||
// const L layout(1, 1, n, m);
|
||||
// double* a = layout.Pointer<double>(p);
|
||||
// int* b = layout.Pointer<int>(p);
|
||||
// }
|
||||
//
|
||||
// The layout we used above combines fixed-size with dynamically-sized fields.
|
||||
// This is quite common. Layout is optimized for this use case and generates
|
||||
// optimal code. All computations that can be performed at compile time are
|
||||
// indeed performed at compile time.
|
||||
//
|
||||
// Efficiency tip: The order of fields matters. In `Layout<T1, ..., TN>` try to
|
||||
// ensure that `alignof(T1) >= ... >= alignof(TN)`. This way you'll have no
|
||||
// padding in between arrays.
|
||||
//
|
||||
// You can manually override the alignment of an array by wrapping the type in
|
||||
// `Aligned<T, N>`. `Layout<..., Aligned<T, N>, ...>` has exactly the same API
|
||||
// and behavior as `Layout<..., T, ...>` except that the first element of the
|
||||
// array of `T` is aligned to `N` (the rest of the elements follow without
|
||||
// padding). `N` cannot be less than `alignof(T)`.
|
||||
//
|
||||
// `AllocSize()` and `Pointer()` are the most basic methods for dealing with
|
||||
// memory layouts. Check out the reference or code below to discover more.
|
||||
//
|
||||
// EXAMPLE
|
||||
//
|
||||
// // Immutable move-only string with sizeof equal to sizeof(void*). The
|
||||
// // string size and the characters are kept in the same heap allocation.
|
||||
// class CompactString {
|
||||
// public:
|
||||
// CompactString(const char* s = "") {
|
||||
// const size_t size = strlen(s);
|
||||
// // size_t[1] followed by char[size + 1].
|
||||
// const L layout(1, size + 1);
|
||||
// p_.reset(new unsigned char[layout.AllocSize()]);
|
||||
// // If running under ASAN, mark the padding bytes, if any, to catch
|
||||
// // memory errors.
|
||||
// layout.PoisonPadding(p_.get());
|
||||
// // Store the size in the allocation.
|
||||
// *layout.Pointer<size_t>(p_.get()) = size;
|
||||
// // Store the characters in the allocation.
|
||||
// memcpy(layout.Pointer<char>(p_.get()), s, size + 1);
|
||||
// }
|
||||
//
|
||||
// size_t size() const {
|
||||
// // Equivalent to reinterpret_cast<size_t&>(*p).
|
||||
// return *L::Partial().Pointer<size_t>(p_.get());
|
||||
// }
|
||||
//
|
||||
// const char* c_str() const {
|
||||
// // Equivalent to reinterpret_cast<char*>(p.get() + sizeof(size_t)).
|
||||
// // The argument in Partial(1) specifies that we have size_t[1] in front
|
||||
// // of the characters.
|
||||
// return L::Partial(1).Pointer<char>(p_.get());
|
||||
// }
|
||||
//
|
||||
// private:
|
||||
// // Our heap allocation contains a size_t followed by an array of chars.
|
||||
// using L = Layout<size_t, char>;
|
||||
// std::unique_ptr<unsigned char[]> p_;
|
||||
// };
|
||||
//
|
||||
// int main() {
|
||||
// CompactString s = "hello";
|
||||
// assert(s.size() == 5);
|
||||
// assert(strcmp(s.c_str(), "hello") == 0);
|
||||
// }
|
||||
//
|
||||
// DOCUMENTATION
|
||||
//
|
||||
// The interface exported by this file consists of:
|
||||
// - class `Layout<>` and its public members.
|
||||
// - The public members of class `internal_layout::LayoutImpl<>`. That class
|
||||
// isn't intended to be used directly, and its name and template parameter
|
||||
// list are internal implementation details, but the class itself provides
|
||||
// most of the functionality in this file. See comments on its members for
|
||||
// detailed documentation.
|
||||
//
|
||||
// `Layout<T1,... Tn>::Partial(count1,..., countm)` (where `m` <= `n`) returns a
|
||||
// `LayoutImpl<>` object. `Layout<T1,..., Tn> layout(count1,..., countn)`
|
||||
// creates a `Layout` object, which exposes the same functionality by inheriting
|
||||
// from `LayoutImpl<>`.
|
||||
|
||||
#ifndef ABSL_CONTAINER_INTERNAL_LAYOUT_H_
|
||||
#define ABSL_CONTAINER_INTERNAL_LAYOUT_H_
|
||||
|
||||
#include <assert.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
#include <ostream>
|
||||
#include <string>
|
||||
#include <tuple>
|
||||
#include <type_traits>
|
||||
#include <typeinfo>
|
||||
#include <utility>
|
||||
|
||||
#ifdef ADDRESS_SANITIZER
|
||||
#include <sanitizer/asan_interface.h>
|
||||
#endif
|
||||
|
||||
#include "absl/meta/type_traits.h"
|
||||
#include "absl/strings/str_cat.h"
|
||||
#include "absl/types/span.h"
|
||||
#include "absl/utility/utility.h"
|
||||
|
||||
#if defined(__GXX_RTTI)
|
||||
#define ABSL_INTERNAL_HAS_CXA_DEMANGLE
|
||||
#endif
|
||||
|
||||
#ifdef ABSL_INTERNAL_HAS_CXA_DEMANGLE
|
||||
#include <cxxabi.h>
|
||||
#endif
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
|
||||
// A type wrapper that instructs `Layout` to use the specific alignment for the
|
||||
// array. `Layout<..., Aligned<T, N>, ...>` has exactly the same API
|
||||
// and behavior as `Layout<..., T, ...>` except that the first element of the
|
||||
// array of `T` is aligned to `N` (the rest of the elements follow without
|
||||
// padding).
|
||||
//
|
||||
// Requires: `N >= alignof(T)` and `N` is a power of 2.
|
||||
template <class T, size_t N>
|
||||
struct Aligned;
|
||||
|
||||
namespace internal_layout {
|
||||
|
||||
template <class T>
|
||||
struct NotAligned {};
|
||||
|
||||
template <class T, size_t N>
|
||||
struct NotAligned<const Aligned<T, N>> {
|
||||
static_assert(sizeof(T) == 0, "Aligned<T, N> cannot be const-qualified");
|
||||
};
|
||||
|
||||
template <size_t>
|
||||
using IntToSize = size_t;
|
||||
|
||||
template <class>
|
||||
using TypeToSize = size_t;
|
||||
|
||||
template <class T>
|
||||
struct Type : NotAligned<T> {
|
||||
using type = T;
|
||||
};
|
||||
|
||||
template <class T, size_t N>
|
||||
struct Type<Aligned<T, N>> {
|
||||
using type = T;
|
||||
};
|
||||
|
||||
template <class T>
|
||||
struct SizeOf : NotAligned<T>, std::integral_constant<size_t, sizeof(T)> {};
|
||||
|
||||
template <class T, size_t N>
|
||||
struct SizeOf<Aligned<T, N>> : std::integral_constant<size_t, sizeof(T)> {};
|
||||
|
||||
// Note: workaround for https://gcc.gnu.org/PR88115
|
||||
template <class T>
|
||||
struct AlignOf : NotAligned<T> {
|
||||
static constexpr size_t value = alignof(T);
|
||||
};
|
||||
|
||||
template <class T, size_t N>
|
||||
struct AlignOf<Aligned<T, N>> {
|
||||
static_assert(N % alignof(T) == 0,
|
||||
"Custom alignment can't be lower than the type's alignment");
|
||||
static constexpr size_t value = N;
|
||||
};
|
||||
|
||||
// Does `Ts...` contain `T`?
|
||||
template <class T, class... Ts>
|
||||
using Contains = absl::disjunction<std::is_same<T, Ts>...>;
|
||||
|
||||
template <class From, class To>
|
||||
using CopyConst =
|
||||
typename std::conditional<std::is_const<From>::value, const To, To>::type;
|
||||
|
||||
// Note: We're not qualifying this with absl:: because it doesn't compile under
|
||||
// MSVC.
|
||||
template <class T>
|
||||
using SliceType = Span<T>;
|
||||
|
||||
// This namespace contains no types. It prevents functions defined in it from
|
||||
// being found by ADL.
|
||||
namespace adl_barrier {
|
||||
|
||||
template <class Needle, class... Ts>
|
||||
constexpr size_t Find(Needle, Needle, Ts...) {
|
||||
static_assert(!Contains<Needle, Ts...>(), "Duplicate element type");
|
||||
return 0;
|
||||
}
|
||||
|
||||
template <class Needle, class T, class... Ts>
|
||||
constexpr size_t Find(Needle, T, Ts...) {
|
||||
return adl_barrier::Find(Needle(), Ts()...) + 1;
|
||||
}
|
||||
|
||||
constexpr bool IsPow2(size_t n) { return !(n & (n - 1)); }
|
||||
|
||||
// Returns `q * m` for the smallest `q` such that `q * m >= n`.
|
||||
// Requires: `m` is a power of two. It's enforced by IsLegalElementType below.
|
||||
constexpr size_t Align(size_t n, size_t m) { return (n + m - 1) & ~(m - 1); }
|
||||
|
||||
constexpr size_t Min(size_t a, size_t b) { return b < a ? b : a; }
|
||||
|
||||
constexpr size_t Max(size_t a) { return a; }
|
||||
|
||||
template <class... Ts>
|
||||
constexpr size_t Max(size_t a, size_t b, Ts... rest) {
|
||||
return adl_barrier::Max(b < a ? a : b, rest...);
|
||||
}
|
||||
|
||||
template <class T>
|
||||
std::string TypeName() {
|
||||
std::string out;
|
||||
int status = 0;
|
||||
char* demangled = nullptr;
|
||||
#ifdef ABSL_INTERNAL_HAS_CXA_DEMANGLE
|
||||
demangled = abi::__cxa_demangle(typeid(T).name(), nullptr, nullptr, &status);
|
||||
#endif
|
||||
if (status == 0 && demangled != nullptr) { // Demangling succeeded.
|
||||
absl::StrAppend(&out, "<", demangled, ">");
|
||||
free(demangled);
|
||||
} else {
|
||||
#if defined(__GXX_RTTI) || defined(_CPPRTTI)
|
||||
absl::StrAppend(&out, "<", typeid(T).name(), ">");
|
||||
#endif
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace adl_barrier
|
||||
|
||||
template <bool C>
|
||||
using EnableIf = typename std::enable_if<C, int>::type;
|
||||
|
||||
// Can `T` be a template argument of `Layout`?
|
||||
template <class T>
|
||||
using IsLegalElementType = std::integral_constant<
|
||||
bool, !std::is_reference<T>::value && !std::is_volatile<T>::value &&
|
||||
!std::is_reference<typename Type<T>::type>::value &&
|
||||
!std::is_volatile<typename Type<T>::type>::value &&
|
||||
adl_barrier::IsPow2(AlignOf<T>::value)>;
|
||||
|
||||
template <class Elements, class SizeSeq, class OffsetSeq>
|
||||
class LayoutImpl;
|
||||
|
||||
// Public base class of `Layout` and the result type of `Layout::Partial()`.
|
||||
//
|
||||
// `Elements...` contains all template arguments of `Layout` that created this
|
||||
// instance.
|
||||
//
|
||||
// `SizeSeq...` is `[0, NumSizes)` where `NumSizes` is the number of arguments
|
||||
// passed to `Layout::Partial()` or `Layout::Layout()`.
|
||||
//
|
||||
// `OffsetSeq...` is `[0, NumOffsets)` where `NumOffsets` is
|
||||
// `Min(sizeof...(Elements), NumSizes + 1)` (the number of arrays for which we
|
||||
// can compute offsets).
|
||||
template <class... Elements, size_t... SizeSeq, size_t... OffsetSeq>
|
||||
class LayoutImpl<std::tuple<Elements...>, absl::index_sequence<SizeSeq...>,
|
||||
absl::index_sequence<OffsetSeq...>> {
|
||||
private:
|
||||
static_assert(sizeof...(Elements) > 0, "At least one field is required");
|
||||
static_assert(absl::conjunction<IsLegalElementType<Elements>...>::value,
|
||||
"Invalid element type (see IsLegalElementType)");
|
||||
|
||||
enum {
|
||||
NumTypes = sizeof...(Elements),
|
||||
NumSizes = sizeof...(SizeSeq),
|
||||
NumOffsets = sizeof...(OffsetSeq),
|
||||
};
|
||||
|
||||
// These are guaranteed by `Layout`.
|
||||
static_assert(NumOffsets == adl_barrier::Min(NumTypes, NumSizes + 1),
|
||||
"Internal error");
|
||||
static_assert(NumTypes > 0, "Internal error");
|
||||
|
||||
// Returns the index of `T` in `Elements...`. Results in a compilation error
|
||||
// if `Elements...` doesn't contain exactly one instance of `T`.
|
||||
template <class T>
|
||||
static constexpr size_t ElementIndex() {
|
||||
static_assert(Contains<Type<T>, Type<typename Type<Elements>::type>...>(),
|
||||
"Type not found");
|
||||
return adl_barrier::Find(Type<T>(),
|
||||
Type<typename Type<Elements>::type>()...);
|
||||
}
|
||||
|
||||
template <size_t N>
|
||||
using ElementAlignment =
|
||||
AlignOf<typename std::tuple_element<N, std::tuple<Elements...>>::type>;
|
||||
|
||||
public:
|
||||
// Element types of all arrays packed in a tuple.
|
||||
using ElementTypes = std::tuple<typename Type<Elements>::type...>;
|
||||
|
||||
// Element type of the Nth array.
|
||||
template <size_t N>
|
||||
using ElementType = typename std::tuple_element<N, ElementTypes>::type;
|
||||
|
||||
constexpr explicit LayoutImpl(IntToSize<SizeSeq>... sizes)
|
||||
: size_{sizes...} {}
|
||||
|
||||
// Alignment of the layout, equal to the strictest alignment of all elements.
|
||||
// All pointers passed to the methods of layout must be aligned to this value.
|
||||
static constexpr size_t Alignment() {
|
||||
return adl_barrier::Max(AlignOf<Elements>::value...);
|
||||
}
|
||||
|
||||
// Offset in bytes of the Nth array.
|
||||
//
|
||||
// // int[3], 4 bytes of padding, double[4].
|
||||
// Layout<int, double> x(3, 4);
|
||||
// assert(x.Offset<0>() == 0); // The ints starts from 0.
|
||||
// assert(x.Offset<1>() == 16); // The doubles starts from 16.
|
||||
//
|
||||
// Requires: `N <= NumSizes && N < sizeof...(Ts)`.
|
||||
template <size_t N, EnableIf<N == 0> = 0>
|
||||
constexpr size_t Offset() const {
|
||||
return 0;
|
||||
}
|
||||
|
||||
template <size_t N, EnableIf<N != 0> = 0>
|
||||
constexpr size_t Offset() const {
|
||||
static_assert(N < NumOffsets, "Index out of bounds");
|
||||
return adl_barrier::Align(
|
||||
Offset<N - 1>() + SizeOf<ElementType<N - 1>>() * size_[N - 1],
|
||||
ElementAlignment<N>::value);
|
||||
}
|
||||
|
||||
// Offset in bytes of the array with the specified element type. There must
|
||||
// be exactly one such array and its zero-based index must be at most
|
||||
// `NumSizes`.
|
||||
//
|
||||
// // int[3], 4 bytes of padding, double[4].
|
||||
// Layout<int, double> x(3, 4);
|
||||
// assert(x.Offset<int>() == 0); // The ints starts from 0.
|
||||
// assert(x.Offset<double>() == 16); // The doubles starts from 16.
|
||||
template <class T>
|
||||
constexpr size_t Offset() const {
|
||||
return Offset<ElementIndex<T>()>();
|
||||
}
|
||||
|
||||
// Offsets in bytes of all arrays for which the offsets are known.
|
||||
constexpr std::array<size_t, NumOffsets> Offsets() const {
|
||||
return {{Offset<OffsetSeq>()...}};
|
||||
}
|
||||
|
||||
// The number of elements in the Nth array. This is the Nth argument of
|
||||
// `Layout::Partial()` or `Layout::Layout()` (zero-based).
|
||||
//
|
||||
// // int[3], 4 bytes of padding, double[4].
|
||||
// Layout<int, double> x(3, 4);
|
||||
// assert(x.Size<0>() == 3);
|
||||
// assert(x.Size<1>() == 4);
|
||||
//
|
||||
// Requires: `N < NumSizes`.
|
||||
template <size_t N>
|
||||
constexpr size_t Size() const {
|
||||
static_assert(N < NumSizes, "Index out of bounds");
|
||||
return size_[N];
|
||||
}
|
||||
|
||||
// The number of elements in the array with the specified element type.
|
||||
// There must be exactly one such array and its zero-based index must be
|
||||
// at most `NumSizes`.
|
||||
//
|
||||
// // int[3], 4 bytes of padding, double[4].
|
||||
// Layout<int, double> x(3, 4);
|
||||
// assert(x.Size<int>() == 3);
|
||||
// assert(x.Size<double>() == 4);
|
||||
template <class T>
|
||||
constexpr size_t Size() const {
|
||||
return Size<ElementIndex<T>()>();
|
||||
}
|
||||
|
||||
// The number of elements of all arrays for which they are known.
|
||||
constexpr std::array<size_t, NumSizes> Sizes() const {
|
||||
return {{Size<SizeSeq>()...}};
|
||||
}
|
||||
|
||||
// Pointer to the beginning of the Nth array.
|
||||
//
|
||||
// `Char` must be `[const] [signed|unsigned] char`.
|
||||
//
|
||||
// // int[3], 4 bytes of padding, double[4].
|
||||
// Layout<int, double> x(3, 4);
|
||||
// unsigned char* p = new unsigned char[x.AllocSize()];
|
||||
// int* ints = x.Pointer<0>(p);
|
||||
// double* doubles = x.Pointer<1>(p);
|
||||
//
|
||||
// Requires: `N <= NumSizes && N < sizeof...(Ts)`.
|
||||
// Requires: `p` is aligned to `Alignment()`.
|
||||
template <size_t N, class Char>
|
||||
CopyConst<Char, ElementType<N>>* Pointer(Char* p) const {
|
||||
using C = typename std::remove_const<Char>::type;
|
||||
static_assert(
|
||||
std::is_same<C, char>() || std::is_same<C, unsigned char>() ||
|
||||
std::is_same<C, signed char>(),
|
||||
"The argument must be a pointer to [const] [signed|unsigned] char");
|
||||
constexpr size_t alignment = Alignment();
|
||||
(void)alignment;
|
||||
assert(reinterpret_cast<uintptr_t>(p) % alignment == 0);
|
||||
return reinterpret_cast<CopyConst<Char, ElementType<N>>*>(p + Offset<N>());
|
||||
}
|
||||
|
||||
// Pointer to the beginning of the array with the specified element type.
|
||||
// There must be exactly one such array and its zero-based index must be at
|
||||
// most `NumSizes`.
|
||||
//
|
||||
// `Char` must be `[const] [signed|unsigned] char`.
|
||||
//
|
||||
// // int[3], 4 bytes of padding, double[4].
|
||||
// Layout<int, double> x(3, 4);
|
||||
// unsigned char* p = new unsigned char[x.AllocSize()];
|
||||
// int* ints = x.Pointer<int>(p);
|
||||
// double* doubles = x.Pointer<double>(p);
|
||||
//
|
||||
// Requires: `p` is aligned to `Alignment()`.
|
||||
template <class T, class Char>
|
||||
CopyConst<Char, T>* Pointer(Char* p) const {
|
||||
return Pointer<ElementIndex<T>()>(p);
|
||||
}
|
||||
|
||||
// Pointers to all arrays for which pointers are known.
|
||||
//
|
||||
// `Char` must be `[const] [signed|unsigned] char`.
|
||||
//
|
||||
// // int[3], 4 bytes of padding, double[4].
|
||||
// Layout<int, double> x(3, 4);
|
||||
// unsigned char* p = new unsigned char[x.AllocSize()];
|
||||
//
|
||||
// int* ints;
|
||||
// double* doubles;
|
||||
// std::tie(ints, doubles) = x.Pointers(p);
|
||||
//
|
||||
// Requires: `p` is aligned to `Alignment()`.
|
||||
//
|
||||
// Note: We're not using ElementType alias here because it does not compile
|
||||
// under MSVC.
|
||||
template <class Char>
|
||||
std::tuple<CopyConst<
|
||||
Char, typename std::tuple_element<OffsetSeq, ElementTypes>::type>*...>
|
||||
Pointers(Char* p) const {
|
||||
return std::tuple<CopyConst<Char, ElementType<OffsetSeq>>*...>(
|
||||
Pointer<OffsetSeq>(p)...);
|
||||
}
|
||||
|
||||
// The Nth array.
|
||||
//
|
||||
// `Char` must be `[const] [signed|unsigned] char`.
|
||||
//
|
||||
// // int[3], 4 bytes of padding, double[4].
|
||||
// Layout<int, double> x(3, 4);
|
||||
// unsigned char* p = new unsigned char[x.AllocSize()];
|
||||
// Span<int> ints = x.Slice<0>(p);
|
||||
// Span<double> doubles = x.Slice<1>(p);
|
||||
//
|
||||
// Requires: `N < NumSizes`.
|
||||
// Requires: `p` is aligned to `Alignment()`.
|
||||
template <size_t N, class Char>
|
||||
SliceType<CopyConst<Char, ElementType<N>>> Slice(Char* p) const {
|
||||
return SliceType<CopyConst<Char, ElementType<N>>>(Pointer<N>(p), Size<N>());
|
||||
}
|
||||
|
||||
// The array with the specified element type. There must be exactly one
|
||||
// such array and its zero-based index must be less than `NumSizes`.
|
||||
//
|
||||
// `Char` must be `[const] [signed|unsigned] char`.
|
||||
//
|
||||
// // int[3], 4 bytes of padding, double[4].
|
||||
// Layout<int, double> x(3, 4);
|
||||
// unsigned char* p = new unsigned char[x.AllocSize()];
|
||||
// Span<int> ints = x.Slice<int>(p);
|
||||
// Span<double> doubles = x.Slice<double>(p);
|
||||
//
|
||||
// Requires: `p` is aligned to `Alignment()`.
|
||||
template <class T, class Char>
|
||||
SliceType<CopyConst<Char, T>> Slice(Char* p) const {
|
||||
return Slice<ElementIndex<T>()>(p);
|
||||
}
|
||||
|
||||
// All arrays with known sizes.
|
||||
//
|
||||
// `Char` must be `[const] [signed|unsigned] char`.
|
||||
//
|
||||
// // int[3], 4 bytes of padding, double[4].
|
||||
// Layout<int, double> x(3, 4);
|
||||
// unsigned char* p = new unsigned char[x.AllocSize()];
|
||||
//
|
||||
// Span<int> ints;
|
||||
// Span<double> doubles;
|
||||
// std::tie(ints, doubles) = x.Slices(p);
|
||||
//
|
||||
// Requires: `p` is aligned to `Alignment()`.
|
||||
//
|
||||
// Note: We're not using ElementType alias here because it does not compile
|
||||
// under MSVC.
|
||||
template <class Char>
|
||||
std::tuple<SliceType<CopyConst<
|
||||
Char, typename std::tuple_element<SizeSeq, ElementTypes>::type>>...>
|
||||
Slices(Char* p) const {
|
||||
// Workaround for https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63875 (fixed
|
||||
// in 6.1).
|
||||
(void)p;
|
||||
return std::tuple<SliceType<CopyConst<Char, ElementType<SizeSeq>>>...>(
|
||||
Slice<SizeSeq>(p)...);
|
||||
}
|
||||
|
||||
// The size of the allocation that fits all arrays.
|
||||
//
|
||||
// // int[3], 4 bytes of padding, double[4].
|
||||
// Layout<int, double> x(3, 4);
|
||||
// unsigned char* p = new unsigned char[x.AllocSize()]; // 48 bytes
|
||||
//
|
||||
// Requires: `NumSizes == sizeof...(Ts)`.
|
||||
constexpr size_t AllocSize() const {
|
||||
static_assert(NumTypes == NumSizes, "You must specify sizes of all fields");
|
||||
return Offset<NumTypes - 1>() +
|
||||
SizeOf<ElementType<NumTypes - 1>>() * size_[NumTypes - 1];
|
||||
}
|
||||
|
||||
// If built with --config=asan, poisons padding bytes (if any) in the
|
||||
// allocation. The pointer must point to a memory block at least
|
||||
// `AllocSize()` bytes in length.
|
||||
//
|
||||
// `Char` must be `[const] [signed|unsigned] char`.
|
||||
//
|
||||
// Requires: `p` is aligned to `Alignment()`.
|
||||
template <class Char, size_t N = NumOffsets - 1, EnableIf<N == 0> = 0>
|
||||
void PoisonPadding(const Char* p) const {
|
||||
Pointer<0>(p); // verify the requirements on `Char` and `p`
|
||||
}
|
||||
|
||||
template <class Char, size_t N = NumOffsets - 1, EnableIf<N != 0> = 0>
|
||||
void PoisonPadding(const Char* p) const {
|
||||
static_assert(N < NumOffsets, "Index out of bounds");
|
||||
(void)p;
|
||||
#ifdef ADDRESS_SANITIZER
|
||||
PoisonPadding<Char, N - 1>(p);
|
||||
// The `if` is an optimization. It doesn't affect the observable behaviour.
|
||||
if (ElementAlignment<N - 1>::value % ElementAlignment<N>::value) {
|
||||
size_t start =
|
||||
Offset<N - 1>() + SizeOf<ElementType<N - 1>>() * size_[N - 1];
|
||||
ASAN_POISON_MEMORY_REGION(p + start, Offset<N>() - start);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
// Human-readable description of the memory layout. Useful for debugging.
|
||||
// Slow.
|
||||
//
|
||||
// // char[5], 3 bytes of padding, int[3], 4 bytes of padding, followed
|
||||
// // by an unknown number of doubles.
|
||||
// auto x = Layout<char, int, double>::Partial(5, 3);
|
||||
// assert(x.DebugString() ==
|
||||
// "@0<char>(1)[5]; @8<int>(4)[3]; @24<double>(8)");
|
||||
//
|
||||
// Each field is in the following format: @offset<type>(sizeof)[size] (<type>
|
||||
// may be missing depending on the target platform). For example,
|
||||
// @8<int>(4)[3] means that at offset 8 we have an array of ints, where each
|
||||
// int is 4 bytes, and we have 3 of those ints. The size of the last field may
|
||||
// be missing (as in the example above). Only fields with known offsets are
|
||||
// described. Type names may differ across platforms: one compiler might
|
||||
// produce "unsigned*" where another produces "unsigned int *".
|
||||
std::string DebugString() const {
|
||||
const auto offsets = Offsets();
|
||||
const size_t sizes[] = {SizeOf<ElementType<OffsetSeq>>()...};
|
||||
const std::string types[] = {
|
||||
adl_barrier::TypeName<ElementType<OffsetSeq>>()...};
|
||||
std::string res = absl::StrCat("@0", types[0], "(", sizes[0], ")");
|
||||
for (size_t i = 0; i != NumOffsets - 1; ++i) {
|
||||
absl::StrAppend(&res, "[", size_[i], "]; @", offsets[i + 1], types[i + 1],
|
||||
"(", sizes[i + 1], ")");
|
||||
}
|
||||
// NumSizes is a constant that may be zero. Some compilers cannot see that
|
||||
// inside the if statement "size_[NumSizes - 1]" must be valid.
|
||||
int last = static_cast<int>(NumSizes) - 1;
|
||||
if (NumTypes == NumSizes && last >= 0) {
|
||||
absl::StrAppend(&res, "[", size_[last], "]");
|
||||
}
|
||||
return res;
|
||||
}
|
||||
|
||||
private:
|
||||
// Arguments of `Layout::Partial()` or `Layout::Layout()`.
|
||||
size_t size_[NumSizes > 0 ? NumSizes : 1];
|
||||
};
|
||||
|
||||
template <size_t NumSizes, class... Ts>
|
||||
using LayoutType = LayoutImpl<
|
||||
std::tuple<Ts...>, absl::make_index_sequence<NumSizes>,
|
||||
absl::make_index_sequence<adl_barrier::Min(sizeof...(Ts), NumSizes + 1)>>;
|
||||
|
||||
} // namespace internal_layout
|
||||
|
||||
// Descriptor of arrays of various types and sizes laid out in memory one after
|
||||
// another. See the top of the file for documentation.
|
||||
//
|
||||
// Check out the public API of internal_layout::LayoutImpl above. The type is
|
||||
// internal to the library but its methods are public, and they are inherited
|
||||
// by `Layout`.
|
||||
template <class... Ts>
|
||||
class Layout : public internal_layout::LayoutType<sizeof...(Ts), Ts...> {
|
||||
public:
|
||||
static_assert(sizeof...(Ts) > 0, "At least one field is required");
|
||||
static_assert(
|
||||
absl::conjunction<internal_layout::IsLegalElementType<Ts>...>::value,
|
||||
"Invalid element type (see IsLegalElementType)");
|
||||
|
||||
// The result type of `Partial()` with `NumSizes` arguments.
|
||||
template <size_t NumSizes>
|
||||
using PartialType = internal_layout::LayoutType<NumSizes, Ts...>;
|
||||
|
||||
// `Layout` knows the element types of the arrays we want to lay out in
|
||||
// memory but not the number of elements in each array.
|
||||
// `Partial(size1, ..., sizeN)` allows us to specify the latter. The
|
||||
// resulting immutable object can be used to obtain pointers to the
|
||||
// individual arrays.
|
||||
//
|
||||
// It's allowed to pass fewer array sizes than the number of arrays. E.g.,
|
||||
// if all you need is to the offset of the second array, you only need to
|
||||
// pass one argument -- the number of elements in the first array.
|
||||
//
|
||||
// // int[3] followed by 4 bytes of padding and an unknown number of
|
||||
// // doubles.
|
||||
// auto x = Layout<int, double>::Partial(3);
|
||||
// // doubles start at byte 16.
|
||||
// assert(x.Offset<1>() == 16);
|
||||
//
|
||||
// If you know the number of elements in all arrays, you can still call
|
||||
// `Partial()` but it's more convenient to use the constructor of `Layout`.
|
||||
//
|
||||
// Layout<int, double> x(3, 5);
|
||||
//
|
||||
// Note: The sizes of the arrays must be specified in number of elements,
|
||||
// not in bytes.
|
||||
//
|
||||
// Requires: `sizeof...(Sizes) <= sizeof...(Ts)`.
|
||||
// Requires: all arguments are convertible to `size_t`.
|
||||
template <class... Sizes>
|
||||
static constexpr PartialType<sizeof...(Sizes)> Partial(Sizes&&... sizes) {
|
||||
static_assert(sizeof...(Sizes) <= sizeof...(Ts), "");
|
||||
return PartialType<sizeof...(Sizes)>(absl::forward<Sizes>(sizes)...);
|
||||
}
|
||||
|
||||
// Creates a layout with the sizes of all arrays specified. If you know
|
||||
// only the sizes of the first N arrays (where N can be zero), you can use
|
||||
// `Partial()` defined above. The constructor is essentially equivalent to
|
||||
// calling `Partial()` and passing in all array sizes; the constructor is
|
||||
// provided as a convenient abbreviation.
|
||||
//
|
||||
// Note: The sizes of the arrays must be specified in number of elements,
|
||||
// not in bytes.
|
||||
constexpr explicit Layout(internal_layout::TypeToSize<Ts>... sizes)
|
||||
: internal_layout::LayoutType<sizeof...(Ts), Ts...>(sizes...) {}
|
||||
};
|
||||
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_INTERNAL_LAYOUT_H_
|
||||
+197
@@ -0,0 +1,197 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_CONTAINER_INTERNAL_RAW_HASH_MAP_H_
|
||||
#define ABSL_CONTAINER_INTERNAL_RAW_HASH_MAP_H_
|
||||
|
||||
#include <tuple>
|
||||
#include <type_traits>
|
||||
#include <utility>
|
||||
|
||||
#include "absl/base/internal/throw_delegate.h"
|
||||
#include "absl/container/internal/container_memory.h"
|
||||
#include "absl/container/internal/raw_hash_set.h" // IWYU pragma: export
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
|
||||
template <class Policy, class Hash, class Eq, class Alloc>
|
||||
class raw_hash_map : public raw_hash_set<Policy, Hash, Eq, Alloc> {
|
||||
// P is Policy. It's passed as a template argument to support maps that have
|
||||
// incomplete types as values, as in unordered_map<K, IncompleteType>.
|
||||
// MappedReference<> may be a non-reference type.
|
||||
template <class P>
|
||||
using MappedReference = decltype(P::value(
|
||||
std::addressof(std::declval<typename raw_hash_map::reference>())));
|
||||
|
||||
// MappedConstReference<> may be a non-reference type.
|
||||
template <class P>
|
||||
using MappedConstReference = decltype(P::value(
|
||||
std::addressof(std::declval<typename raw_hash_map::const_reference>())));
|
||||
|
||||
using KeyArgImpl =
|
||||
KeyArg<IsTransparent<Eq>::value && IsTransparent<Hash>::value>;
|
||||
|
||||
public:
|
||||
using key_type = typename Policy::key_type;
|
||||
using mapped_type = typename Policy::mapped_type;
|
||||
template <class K>
|
||||
using key_arg = typename KeyArgImpl::template type<K, key_type>;
|
||||
|
||||
static_assert(!std::is_reference<key_type>::value, "");
|
||||
// TODO(alkis): remove this assertion and verify that reference mapped_type is
|
||||
// supported.
|
||||
static_assert(!std::is_reference<mapped_type>::value, "");
|
||||
|
||||
using iterator = typename raw_hash_map::raw_hash_set::iterator;
|
||||
using const_iterator = typename raw_hash_map::raw_hash_set::const_iterator;
|
||||
|
||||
raw_hash_map() {}
|
||||
using raw_hash_map::raw_hash_set::raw_hash_set;
|
||||
|
||||
// The last two template parameters ensure that both arguments are rvalues
|
||||
// (lvalue arguments are handled by the overloads below). This is necessary
|
||||
// for supporting bitfield arguments.
|
||||
//
|
||||
// union { int n : 1; };
|
||||
// flat_hash_map<int, int> m;
|
||||
// m.insert_or_assign(n, n);
|
||||
template <class K = key_type, class V = mapped_type, K* = nullptr,
|
||||
V* = nullptr>
|
||||
std::pair<iterator, bool> insert_or_assign(key_arg<K>&& k, V&& v) {
|
||||
return insert_or_assign_impl(std::forward<K>(k), std::forward<V>(v));
|
||||
}
|
||||
|
||||
template <class K = key_type, class V = mapped_type, K* = nullptr>
|
||||
std::pair<iterator, bool> insert_or_assign(key_arg<K>&& k, const V& v) {
|
||||
return insert_or_assign_impl(std::forward<K>(k), v);
|
||||
}
|
||||
|
||||
template <class K = key_type, class V = mapped_type, V* = nullptr>
|
||||
std::pair<iterator, bool> insert_or_assign(const key_arg<K>& k, V&& v) {
|
||||
return insert_or_assign_impl(k, std::forward<V>(v));
|
||||
}
|
||||
|
||||
template <class K = key_type, class V = mapped_type>
|
||||
std::pair<iterator, bool> insert_or_assign(const key_arg<K>& k, const V& v) {
|
||||
return insert_or_assign_impl(k, v);
|
||||
}
|
||||
|
||||
template <class K = key_type, class V = mapped_type, K* = nullptr,
|
||||
V* = nullptr>
|
||||
iterator insert_or_assign(const_iterator, key_arg<K>&& k, V&& v) {
|
||||
return insert_or_assign(std::forward<K>(k), std::forward<V>(v)).first;
|
||||
}
|
||||
|
||||
template <class K = key_type, class V = mapped_type, K* = nullptr>
|
||||
iterator insert_or_assign(const_iterator, key_arg<K>&& k, const V& v) {
|
||||
return insert_or_assign(std::forward<K>(k), v).first;
|
||||
}
|
||||
|
||||
template <class K = key_type, class V = mapped_type, V* = nullptr>
|
||||
iterator insert_or_assign(const_iterator, const key_arg<K>& k, V&& v) {
|
||||
return insert_or_assign(k, std::forward<V>(v)).first;
|
||||
}
|
||||
|
||||
template <class K = key_type, class V = mapped_type>
|
||||
iterator insert_or_assign(const_iterator, const key_arg<K>& k, const V& v) {
|
||||
return insert_or_assign(k, v).first;
|
||||
}
|
||||
|
||||
// All `try_emplace()` overloads make the same guarantees regarding rvalue
|
||||
// arguments as `std::unordered_map::try_emplace()`, namely that these
|
||||
// functions will not move from rvalue arguments if insertions do not happen.
|
||||
template <class K = key_type, class... Args,
|
||||
typename std::enable_if<
|
||||
!std::is_convertible<K, const_iterator>::value, int>::type = 0,
|
||||
K* = nullptr>
|
||||
std::pair<iterator, bool> try_emplace(key_arg<K>&& k, Args&&... args) {
|
||||
return try_emplace_impl(std::forward<K>(k), std::forward<Args>(args)...);
|
||||
}
|
||||
|
||||
template <class K = key_type, class... Args,
|
||||
typename std::enable_if<
|
||||
!std::is_convertible<K, const_iterator>::value, int>::type = 0>
|
||||
std::pair<iterator, bool> try_emplace(const key_arg<K>& k, Args&&... args) {
|
||||
return try_emplace_impl(k, std::forward<Args>(args)...);
|
||||
}
|
||||
|
||||
template <class K = key_type, class... Args, K* = nullptr>
|
||||
iterator try_emplace(const_iterator, key_arg<K>&& k, Args&&... args) {
|
||||
return try_emplace(std::forward<K>(k), std::forward<Args>(args)...).first;
|
||||
}
|
||||
|
||||
template <class K = key_type, class... Args>
|
||||
iterator try_emplace(const_iterator, const key_arg<K>& k, Args&&... args) {
|
||||
return try_emplace(k, std::forward<Args>(args)...).first;
|
||||
}
|
||||
|
||||
template <class K = key_type, class P = Policy>
|
||||
MappedReference<P> at(const key_arg<K>& key) {
|
||||
auto it = this->find(key);
|
||||
if (it == this->end()) {
|
||||
base_internal::ThrowStdOutOfRange(
|
||||
"absl::container_internal::raw_hash_map<>::at");
|
||||
}
|
||||
return Policy::value(&*it);
|
||||
}
|
||||
|
||||
template <class K = key_type, class P = Policy>
|
||||
MappedConstReference<P> at(const key_arg<K>& key) const {
|
||||
auto it = this->find(key);
|
||||
if (it == this->end()) {
|
||||
base_internal::ThrowStdOutOfRange(
|
||||
"absl::container_internal::raw_hash_map<>::at");
|
||||
}
|
||||
return Policy::value(&*it);
|
||||
}
|
||||
|
||||
template <class K = key_type, class P = Policy, K* = nullptr>
|
||||
MappedReference<P> operator[](key_arg<K>&& key) {
|
||||
return Policy::value(&*try_emplace(std::forward<K>(key)).first);
|
||||
}
|
||||
|
||||
template <class K = key_type, class P = Policy>
|
||||
MappedReference<P> operator[](const key_arg<K>& key) {
|
||||
return Policy::value(&*try_emplace(key).first);
|
||||
}
|
||||
|
||||
private:
|
||||
template <class K, class V>
|
||||
std::pair<iterator, bool> insert_or_assign_impl(K&& k, V&& v) {
|
||||
auto res = this->find_or_prepare_insert(k);
|
||||
if (res.second)
|
||||
this->emplace_at(res.first, std::forward<K>(k), std::forward<V>(v));
|
||||
else
|
||||
Policy::value(&*this->iterator_at(res.first)) = std::forward<V>(v);
|
||||
return {this->iterator_at(res.first), res.second};
|
||||
}
|
||||
|
||||
template <class K = key_type, class... Args>
|
||||
std::pair<iterator, bool> try_emplace_impl(K&& k, Args&&... args) {
|
||||
auto res = this->find_or_prepare_insert(k);
|
||||
if (res.second)
|
||||
this->emplace_at(res.first, std::piecewise_construct,
|
||||
std::forward_as_tuple(std::forward<K>(k)),
|
||||
std::forward_as_tuple(std::forward<Args>(args)...));
|
||||
return {this->iterator_at(res.first), res.second};
|
||||
}
|
||||
};
|
||||
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_CONTAINER_INTERNAL_RAW_HASH_MAP_H_
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/container/internal/raw_hash_set.h"
|
||||
|
||||
#include <atomic>
|
||||
#include <cstddef>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace container_internal {
|
||||
|
||||
constexpr size_t Group::kWidth;
|
||||
|
||||
// Returns "random" seed.
|
||||
inline size_t RandomSeed() {
|
||||
#if ABSL_HAVE_THREAD_LOCAL
|
||||
static thread_local size_t counter = 0;
|
||||
size_t value = ++counter;
|
||||
#else // ABSL_HAVE_THREAD_LOCAL
|
||||
static std::atomic<size_t> counter(0);
|
||||
size_t value = counter.fetch_add(1, std::memory_order_relaxed);
|
||||
#endif // ABSL_HAVE_THREAD_LOCAL
|
||||
return value ^ static_cast<size_t>(reinterpret_cast<uintptr_t>(&counter));
|
||||
}
|
||||
|
||||
bool ShouldInsertBackwards(size_t hash, ctrl_t* ctrl) {
|
||||
// To avoid problems with weak hashes and single bit tests, we use % 13.
|
||||
// TODO(kfm,sbenza): revisit after we do unconditional mixing
|
||||
return (H1(hash, ctrl) ^ RandomSeed()) % 13 > 6;
|
||||
}
|
||||
|
||||
} // namespace container_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
+1882
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,138 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// base::AddressIsReadable() probes an address to see whether it is readable,
|
||||
// without faulting.
|
||||
|
||||
#include "absl/debugging/internal/address_is_readable.h"
|
||||
|
||||
#if !defined(__linux__) || defined(__ANDROID__)
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
|
||||
// On platforms other than Linux, just return true.
|
||||
bool AddressIsReadable(const void* /* addr */) { return true; }
|
||||
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#else
|
||||
|
||||
#include <fcntl.h>
|
||||
#include <sys/syscall.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <atomic>
|
||||
#include <cerrno>
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/base/internal/errno_saver.h"
|
||||
#include "absl/base/internal/raw_logging.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
|
||||
// Pack a pid and two file descriptors into a 64-bit word,
|
||||
// using 16, 24, and 24 bits for each respectively.
|
||||
static uint64_t Pack(uint64_t pid, uint64_t read_fd, uint64_t write_fd) {
|
||||
ABSL_RAW_CHECK((read_fd >> 24) == 0 && (write_fd >> 24) == 0,
|
||||
"fd out of range");
|
||||
return (pid << 48) | ((read_fd & 0xffffff) << 24) | (write_fd & 0xffffff);
|
||||
}
|
||||
|
||||
// Unpack x into a pid and two file descriptors, where x was created with
|
||||
// Pack().
|
||||
static void Unpack(uint64_t x, int *pid, int *read_fd, int *write_fd) {
|
||||
*pid = x >> 48;
|
||||
*read_fd = (x >> 24) & 0xffffff;
|
||||
*write_fd = x & 0xffffff;
|
||||
}
|
||||
|
||||
// Return whether the byte at *addr is readable, without faulting.
|
||||
// Save and restores errno. Returns true on systems where
|
||||
// unimplemented.
|
||||
// This is a namespace-scoped variable for correct zero-initialization.
|
||||
static std::atomic<uint64_t> pid_and_fds; // initially 0, an invalid pid.
|
||||
bool AddressIsReadable(const void *addr) {
|
||||
absl::base_internal::ErrnoSaver errno_saver;
|
||||
// We test whether a byte is readable by using write(). Normally, this would
|
||||
// be done via a cached file descriptor to /dev/null, but linux fails to
|
||||
// check whether the byte is readable when the destination is /dev/null, so
|
||||
// we use a cached pipe. We store the pid of the process that created the
|
||||
// pipe to handle the case where a process forks, and the child closes all
|
||||
// the file descriptors and then calls this routine. This is not perfect:
|
||||
// the child could use the routine, then close all file descriptors and then
|
||||
// use this routine again. But the likely use of this routine is when
|
||||
// crashing, to test the validity of pages when dumping the stack. Beware
|
||||
// that we may leak file descriptors, but we're unlikely to leak many.
|
||||
int bytes_written;
|
||||
int current_pid = getpid() & 0xffff; // we use only the low order 16 bits
|
||||
do { // until we do not get EBADF trying to use file descriptors
|
||||
int pid;
|
||||
int read_fd;
|
||||
int write_fd;
|
||||
uint64_t local_pid_and_fds = pid_and_fds.load(std::memory_order_relaxed);
|
||||
Unpack(local_pid_and_fds, &pid, &read_fd, &write_fd);
|
||||
while (current_pid != pid) {
|
||||
int p[2];
|
||||
// new pipe
|
||||
if (pipe(p) != 0) {
|
||||
ABSL_RAW_LOG(FATAL, "Failed to create pipe, errno=%d", errno);
|
||||
}
|
||||
fcntl(p[0], F_SETFD, FD_CLOEXEC);
|
||||
fcntl(p[1], F_SETFD, FD_CLOEXEC);
|
||||
uint64_t new_pid_and_fds = Pack(current_pid, p[0], p[1]);
|
||||
if (pid_and_fds.compare_exchange_strong(
|
||||
local_pid_and_fds, new_pid_and_fds, std::memory_order_relaxed,
|
||||
std::memory_order_relaxed)) {
|
||||
local_pid_and_fds = new_pid_and_fds; // fds exposed to other threads
|
||||
} else { // fds not exposed to other threads; we can close them.
|
||||
close(p[0]);
|
||||
close(p[1]);
|
||||
local_pid_and_fds = pid_and_fds.load(std::memory_order_relaxed);
|
||||
}
|
||||
Unpack(local_pid_and_fds, &pid, &read_fd, &write_fd);
|
||||
}
|
||||
errno = 0;
|
||||
// Use syscall(SYS_write, ...) instead of write() to prevent ASAN
|
||||
// and other checkers from complaining about accesses to arbitrary
|
||||
// memory.
|
||||
do {
|
||||
bytes_written = syscall(SYS_write, write_fd, addr, 1);
|
||||
} while (bytes_written == -1 && errno == EINTR);
|
||||
if (bytes_written == 1) { // remove the byte from the pipe
|
||||
char c;
|
||||
while (read(read_fd, &c, 1) == -1 && errno == EINTR) {
|
||||
}
|
||||
}
|
||||
if (errno == EBADF) { // Descriptors invalid.
|
||||
// If pid_and_fds contains the problematic file descriptors we just used,
|
||||
// this call will forget them, and the loop will try again.
|
||||
pid_and_fds.compare_exchange_strong(local_pid_and_fds, 0,
|
||||
std::memory_order_relaxed,
|
||||
std::memory_order_relaxed);
|
||||
}
|
||||
} while (errno == EBADF);
|
||||
return bytes_written == 1;
|
||||
}
|
||||
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif
|
||||
@@ -0,0 +1,32 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_ADDRESS_IS_READABLE_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_ADDRESS_IS_READABLE_H_
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
|
||||
// Return whether the byte at *addr is readable, without faulting.
|
||||
// Save and restores errno.
|
||||
bool AddressIsReadable(const void *addr);
|
||||
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_ADDRESS_IS_READABLE_H_
|
||||
+1895
File diff suppressed because it is too large
Load Diff
+71
@@ -0,0 +1,71 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// An async-signal-safe and thread-safe demangler for Itanium C++ ABI
|
||||
// (aka G++ V3 ABI).
|
||||
//
|
||||
// The demangler is implemented to be used in async signal handlers to
|
||||
// symbolize stack traces. We cannot use libstdc++'s
|
||||
// abi::__cxa_demangle() in such signal handlers since it's not async
|
||||
// signal safe (it uses malloc() internally).
|
||||
//
|
||||
// Note that this demangler doesn't support full demangling. More
|
||||
// specifically, it doesn't print types of function parameters and
|
||||
// types of template arguments. It just skips them. However, it's
|
||||
// still very useful to extract basic information such as class,
|
||||
// function, constructor, destructor, and operator names.
|
||||
//
|
||||
// See the implementation note in demangle.cc if you are interested.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// | Mangled Name | The Demangler | abi::__cxa_demangle()
|
||||
// |---------------|---------------|-----------------------
|
||||
// | _Z1fv | f() | f()
|
||||
// | _Z1fi | f() | f(int)
|
||||
// | _Z3foo3bar | foo() | foo(bar)
|
||||
// | _Z1fIiEvi | f<>() | void f<int>(int)
|
||||
// | _ZN1N1fE | N::f | N::f
|
||||
// | _ZN3Foo3BarEv | Foo::Bar() | Foo::Bar()
|
||||
// | _Zrm1XS_" | operator%() | operator%(X, X)
|
||||
// | _ZN3FooC1Ev | Foo::Foo() | Foo::Foo()
|
||||
// | _Z1fSs | f() | f(std::basic_string<char,
|
||||
// | | | std::char_traits<char>,
|
||||
// | | | std::allocator<char> >)
|
||||
//
|
||||
// See the unit test for more examples.
|
||||
//
|
||||
// Note: we might want to write demanglers for ABIs other than Itanium
|
||||
// C++ ABI in the future.
|
||||
//
|
||||
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_DEMANGLE_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_DEMANGLE_H_
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
|
||||
// Demangle `mangled`. On success, return true and write the
|
||||
// demangled symbol name to `out`. Otherwise, return false.
|
||||
// `out` is modified even if demangling is unsuccessful.
|
||||
bool Demangle(const char *mangled, char *out, int out_size);
|
||||
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_DEMANGLE_H_
|
||||
+382
@@ -0,0 +1,382 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// Allow dynamic symbol lookup in an in-memory Elf image.
|
||||
//
|
||||
|
||||
#include "absl/debugging/internal/elf_mem_image.h"
|
||||
|
||||
#ifdef ABSL_HAVE_ELF_MEM_IMAGE // defined in elf_mem_image.h
|
||||
|
||||
#include <string.h>
|
||||
#include <cassert>
|
||||
#include <cstddef>
|
||||
#include "absl/base/internal/raw_logging.h"
|
||||
|
||||
// From binutils/include/elf/common.h (this doesn't appear to be documented
|
||||
// anywhere else).
|
||||
//
|
||||
// /* This flag appears in a Versym structure. It means that the symbol
|
||||
// is hidden, and is only visible with an explicit version number.
|
||||
// This is a GNU extension. */
|
||||
// #define VERSYM_HIDDEN 0x8000
|
||||
//
|
||||
// /* This is the mask for the rest of the Versym information. */
|
||||
// #define VERSYM_VERSION 0x7fff
|
||||
|
||||
#define VERSYM_VERSION 0x7fff
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
|
||||
namespace {
|
||||
|
||||
#if __WORDSIZE == 32
|
||||
const int kElfClass = ELFCLASS32;
|
||||
int ElfBind(const ElfW(Sym) *symbol) { return ELF32_ST_BIND(symbol->st_info); }
|
||||
int ElfType(const ElfW(Sym) *symbol) { return ELF32_ST_TYPE(symbol->st_info); }
|
||||
#elif __WORDSIZE == 64
|
||||
const int kElfClass = ELFCLASS64;
|
||||
int ElfBind(const ElfW(Sym) *symbol) { return ELF64_ST_BIND(symbol->st_info); }
|
||||
int ElfType(const ElfW(Sym) *symbol) { return ELF64_ST_TYPE(symbol->st_info); }
|
||||
#else
|
||||
const int kElfClass = -1;
|
||||
int ElfBind(const ElfW(Sym) *) {
|
||||
ABSL_RAW_LOG(FATAL, "Unexpected word size");
|
||||
return 0;
|
||||
}
|
||||
int ElfType(const ElfW(Sym) *) {
|
||||
ABSL_RAW_LOG(FATAL, "Unexpected word size");
|
||||
return 0;
|
||||
}
|
||||
#endif
|
||||
|
||||
// Extract an element from one of the ELF tables, cast it to desired type.
|
||||
// This is just a simple arithmetic and a glorified cast.
|
||||
// Callers are responsible for bounds checking.
|
||||
template <typename T>
|
||||
const T *GetTableElement(const ElfW(Ehdr) * ehdr, ElfW(Off) table_offset,
|
||||
ElfW(Word) element_size, size_t index) {
|
||||
return reinterpret_cast<const T*>(reinterpret_cast<const char *>(ehdr)
|
||||
+ table_offset
|
||||
+ index * element_size);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
// The value of this variable doesn't matter; it's used only for its
|
||||
// unique address.
|
||||
const int ElfMemImage::kInvalidBaseSentinel = 0;
|
||||
|
||||
ElfMemImage::ElfMemImage(const void *base) {
|
||||
ABSL_RAW_CHECK(base != kInvalidBase, "bad pointer");
|
||||
Init(base);
|
||||
}
|
||||
|
||||
int ElfMemImage::GetNumSymbols() const {
|
||||
if (!hash_) {
|
||||
return 0;
|
||||
}
|
||||
// See http://www.caldera.com/developers/gabi/latest/ch5.dynamic.html#hash
|
||||
return hash_[1];
|
||||
}
|
||||
|
||||
const ElfW(Sym) *ElfMemImage::GetDynsym(int index) const {
|
||||
ABSL_RAW_CHECK(index < GetNumSymbols(), "index out of range");
|
||||
return dynsym_ + index;
|
||||
}
|
||||
|
||||
const ElfW(Versym) *ElfMemImage::GetVersym(int index) const {
|
||||
ABSL_RAW_CHECK(index < GetNumSymbols(), "index out of range");
|
||||
return versym_ + index;
|
||||
}
|
||||
|
||||
const ElfW(Phdr) *ElfMemImage::GetPhdr(int index) const {
|
||||
ABSL_RAW_CHECK(index < ehdr_->e_phnum, "index out of range");
|
||||
return GetTableElement<ElfW(Phdr)>(ehdr_,
|
||||
ehdr_->e_phoff,
|
||||
ehdr_->e_phentsize,
|
||||
index);
|
||||
}
|
||||
|
||||
const char *ElfMemImage::GetDynstr(ElfW(Word) offset) const {
|
||||
ABSL_RAW_CHECK(offset < strsize_, "offset out of range");
|
||||
return dynstr_ + offset;
|
||||
}
|
||||
|
||||
const void *ElfMemImage::GetSymAddr(const ElfW(Sym) *sym) const {
|
||||
if (sym->st_shndx == SHN_UNDEF || sym->st_shndx >= SHN_LORESERVE) {
|
||||
// Symbol corresponds to "special" (e.g. SHN_ABS) section.
|
||||
return reinterpret_cast<const void *>(sym->st_value);
|
||||
}
|
||||
ABSL_RAW_CHECK(link_base_ < sym->st_value, "symbol out of range");
|
||||
return GetTableElement<char>(ehdr_, 0, 1, sym->st_value - link_base_);
|
||||
}
|
||||
|
||||
const ElfW(Verdef) *ElfMemImage::GetVerdef(int index) const {
|
||||
ABSL_RAW_CHECK(0 <= index && static_cast<size_t>(index) <= verdefnum_,
|
||||
"index out of range");
|
||||
const ElfW(Verdef) *version_definition = verdef_;
|
||||
while (version_definition->vd_ndx < index && version_definition->vd_next) {
|
||||
const char *const version_definition_as_char =
|
||||
reinterpret_cast<const char *>(version_definition);
|
||||
version_definition =
|
||||
reinterpret_cast<const ElfW(Verdef) *>(version_definition_as_char +
|
||||
version_definition->vd_next);
|
||||
}
|
||||
return version_definition->vd_ndx == index ? version_definition : nullptr;
|
||||
}
|
||||
|
||||
const ElfW(Verdaux) *ElfMemImage::GetVerdefAux(
|
||||
const ElfW(Verdef) *verdef) const {
|
||||
return reinterpret_cast<const ElfW(Verdaux) *>(verdef+1);
|
||||
}
|
||||
|
||||
const char *ElfMemImage::GetVerstr(ElfW(Word) offset) const {
|
||||
ABSL_RAW_CHECK(offset < strsize_, "offset out of range");
|
||||
return dynstr_ + offset;
|
||||
}
|
||||
|
||||
void ElfMemImage::Init(const void *base) {
|
||||
ehdr_ = nullptr;
|
||||
dynsym_ = nullptr;
|
||||
dynstr_ = nullptr;
|
||||
versym_ = nullptr;
|
||||
verdef_ = nullptr;
|
||||
hash_ = nullptr;
|
||||
strsize_ = 0;
|
||||
verdefnum_ = 0;
|
||||
link_base_ = ~0L; // Sentinel: PT_LOAD .p_vaddr can't possibly be this.
|
||||
if (!base) {
|
||||
return;
|
||||
}
|
||||
const char *const base_as_char = reinterpret_cast<const char *>(base);
|
||||
if (base_as_char[EI_MAG0] != ELFMAG0 || base_as_char[EI_MAG1] != ELFMAG1 ||
|
||||
base_as_char[EI_MAG2] != ELFMAG2 || base_as_char[EI_MAG3] != ELFMAG3) {
|
||||
assert(false);
|
||||
return;
|
||||
}
|
||||
int elf_class = base_as_char[EI_CLASS];
|
||||
if (elf_class != kElfClass) {
|
||||
assert(false);
|
||||
return;
|
||||
}
|
||||
switch (base_as_char[EI_DATA]) {
|
||||
case ELFDATA2LSB: {
|
||||
if (__LITTLE_ENDIAN != __BYTE_ORDER) {
|
||||
assert(false);
|
||||
return;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case ELFDATA2MSB: {
|
||||
if (__BIG_ENDIAN != __BYTE_ORDER) {
|
||||
assert(false);
|
||||
return;
|
||||
}
|
||||
break;
|
||||
}
|
||||
default: {
|
||||
assert(false);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
ehdr_ = reinterpret_cast<const ElfW(Ehdr) *>(base);
|
||||
const ElfW(Phdr) *dynamic_program_header = nullptr;
|
||||
for (int i = 0; i < ehdr_->e_phnum; ++i) {
|
||||
const ElfW(Phdr) *const program_header = GetPhdr(i);
|
||||
switch (program_header->p_type) {
|
||||
case PT_LOAD:
|
||||
if (!~link_base_) {
|
||||
link_base_ = program_header->p_vaddr;
|
||||
}
|
||||
break;
|
||||
case PT_DYNAMIC:
|
||||
dynamic_program_header = program_header;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!~link_base_ || !dynamic_program_header) {
|
||||
assert(false);
|
||||
// Mark this image as not present. Can not recur infinitely.
|
||||
Init(nullptr);
|
||||
return;
|
||||
}
|
||||
ptrdiff_t relocation =
|
||||
base_as_char - reinterpret_cast<const char *>(link_base_);
|
||||
ElfW(Dyn) *dynamic_entry =
|
||||
reinterpret_cast<ElfW(Dyn) *>(dynamic_program_header->p_vaddr +
|
||||
relocation);
|
||||
for (; dynamic_entry->d_tag != DT_NULL; ++dynamic_entry) {
|
||||
const ElfW(Xword) value = dynamic_entry->d_un.d_val + relocation;
|
||||
switch (dynamic_entry->d_tag) {
|
||||
case DT_HASH:
|
||||
hash_ = reinterpret_cast<ElfW(Word) *>(value);
|
||||
break;
|
||||
case DT_SYMTAB:
|
||||
dynsym_ = reinterpret_cast<ElfW(Sym) *>(value);
|
||||
break;
|
||||
case DT_STRTAB:
|
||||
dynstr_ = reinterpret_cast<const char *>(value);
|
||||
break;
|
||||
case DT_VERSYM:
|
||||
versym_ = reinterpret_cast<ElfW(Versym) *>(value);
|
||||
break;
|
||||
case DT_VERDEF:
|
||||
verdef_ = reinterpret_cast<ElfW(Verdef) *>(value);
|
||||
break;
|
||||
case DT_VERDEFNUM:
|
||||
verdefnum_ = dynamic_entry->d_un.d_val;
|
||||
break;
|
||||
case DT_STRSZ:
|
||||
strsize_ = dynamic_entry->d_un.d_val;
|
||||
break;
|
||||
default:
|
||||
// Unrecognized entries explicitly ignored.
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!hash_ || !dynsym_ || !dynstr_ || !versym_ ||
|
||||
!verdef_ || !verdefnum_ || !strsize_) {
|
||||
assert(false); // invalid VDSO
|
||||
// Mark this image as not present. Can not recur infinitely.
|
||||
Init(nullptr);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
bool ElfMemImage::LookupSymbol(const char *name,
|
||||
const char *version,
|
||||
int type,
|
||||
SymbolInfo *info_out) const {
|
||||
for (const SymbolInfo& info : *this) {
|
||||
if (strcmp(info.name, name) == 0 && strcmp(info.version, version) == 0 &&
|
||||
ElfType(info.symbol) == type) {
|
||||
if (info_out) {
|
||||
*info_out = info;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
bool ElfMemImage::LookupSymbolByAddress(const void *address,
|
||||
SymbolInfo *info_out) const {
|
||||
for (const SymbolInfo& info : *this) {
|
||||
const char *const symbol_start =
|
||||
reinterpret_cast<const char *>(info.address);
|
||||
const char *const symbol_end = symbol_start + info.symbol->st_size;
|
||||
if (symbol_start <= address && address < symbol_end) {
|
||||
if (info_out) {
|
||||
// Client wants to know details for that symbol (the usual case).
|
||||
if (ElfBind(info.symbol) == STB_GLOBAL) {
|
||||
// Strong symbol; just return it.
|
||||
*info_out = info;
|
||||
return true;
|
||||
} else {
|
||||
// Weak or local. Record it, but keep looking for a strong one.
|
||||
*info_out = info;
|
||||
}
|
||||
} else {
|
||||
// Client only cares if there is an overlapping symbol.
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
ElfMemImage::SymbolIterator::SymbolIterator(const void *const image, int index)
|
||||
: index_(index), image_(image) {
|
||||
}
|
||||
|
||||
const ElfMemImage::SymbolInfo *ElfMemImage::SymbolIterator::operator->() const {
|
||||
return &info_;
|
||||
}
|
||||
|
||||
const ElfMemImage::SymbolInfo& ElfMemImage::SymbolIterator::operator*() const {
|
||||
return info_;
|
||||
}
|
||||
|
||||
bool ElfMemImage::SymbolIterator::operator==(const SymbolIterator &rhs) const {
|
||||
return this->image_ == rhs.image_ && this->index_ == rhs.index_;
|
||||
}
|
||||
|
||||
bool ElfMemImage::SymbolIterator::operator!=(const SymbolIterator &rhs) const {
|
||||
return !(*this == rhs);
|
||||
}
|
||||
|
||||
ElfMemImage::SymbolIterator &ElfMemImage::SymbolIterator::operator++() {
|
||||
this->Update(1);
|
||||
return *this;
|
||||
}
|
||||
|
||||
ElfMemImage::SymbolIterator ElfMemImage::begin() const {
|
||||
SymbolIterator it(this, 0);
|
||||
it.Update(0);
|
||||
return it;
|
||||
}
|
||||
|
||||
ElfMemImage::SymbolIterator ElfMemImage::end() const {
|
||||
return SymbolIterator(this, GetNumSymbols());
|
||||
}
|
||||
|
||||
void ElfMemImage::SymbolIterator::Update(int increment) {
|
||||
const ElfMemImage *image = reinterpret_cast<const ElfMemImage *>(image_);
|
||||
ABSL_RAW_CHECK(image->IsPresent() || increment == 0, "");
|
||||
if (!image->IsPresent()) {
|
||||
return;
|
||||
}
|
||||
index_ += increment;
|
||||
if (index_ >= image->GetNumSymbols()) {
|
||||
index_ = image->GetNumSymbols();
|
||||
return;
|
||||
}
|
||||
const ElfW(Sym) *symbol = image->GetDynsym(index_);
|
||||
const ElfW(Versym) *version_symbol = image->GetVersym(index_);
|
||||
ABSL_RAW_CHECK(symbol && version_symbol, "");
|
||||
const char *const symbol_name = image->GetDynstr(symbol->st_name);
|
||||
const ElfW(Versym) version_index = version_symbol[0] & VERSYM_VERSION;
|
||||
const ElfW(Verdef) *version_definition = nullptr;
|
||||
const char *version_name = "";
|
||||
if (symbol->st_shndx == SHN_UNDEF) {
|
||||
// Undefined symbols reference DT_VERNEED, not DT_VERDEF, and
|
||||
// version_index could well be greater than verdefnum_, so calling
|
||||
// GetVerdef(version_index) may trigger assertion.
|
||||
} else {
|
||||
version_definition = image->GetVerdef(version_index);
|
||||
}
|
||||
if (version_definition) {
|
||||
// I am expecting 1 or 2 auxiliary entries: 1 for the version itself,
|
||||
// optional 2nd if the version has a parent.
|
||||
ABSL_RAW_CHECK(
|
||||
version_definition->vd_cnt == 1 || version_definition->vd_cnt == 2,
|
||||
"wrong number of entries");
|
||||
const ElfW(Verdaux) *version_aux = image->GetVerdefAux(version_definition);
|
||||
version_name = image->GetVerstr(version_aux->vda_name);
|
||||
}
|
||||
info_.name = symbol_name;
|
||||
info_.version = version_name;
|
||||
info_.address = image->GetSymAddr(symbol);
|
||||
info_.symbol = symbol;
|
||||
}
|
||||
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_HAVE_ELF_MEM_IMAGE
|
||||
+134
@@ -0,0 +1,134 @@
|
||||
/*
|
||||
* Copyright 2017 The Abseil Authors.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
// Allow dynamic symbol lookup for in-memory Elf images.
|
||||
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_ELF_MEM_IMAGE_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_ELF_MEM_IMAGE_H_
|
||||
|
||||
// Including this will define the __GLIBC__ macro if glibc is being
|
||||
// used.
|
||||
#include <climits>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
// Maybe one day we can rewrite this file not to require the elf
|
||||
// symbol extensions in glibc, but for right now we need them.
|
||||
#ifdef ABSL_HAVE_ELF_MEM_IMAGE
|
||||
#error ABSL_HAVE_ELF_MEM_IMAGE cannot be directly set
|
||||
#endif
|
||||
|
||||
#if defined(__ELF__) && defined(__GLIBC__) && !defined(__native_client__) && \
|
||||
!defined(__asmjs__) && !defined(__wasm__)
|
||||
#define ABSL_HAVE_ELF_MEM_IMAGE 1
|
||||
#endif
|
||||
|
||||
#ifdef ABSL_HAVE_ELF_MEM_IMAGE
|
||||
|
||||
#include <link.h> // for ElfW
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
|
||||
// An in-memory ELF image (may not exist on disk).
|
||||
class ElfMemImage {
|
||||
private:
|
||||
// Sentinel: there could never be an elf image at &kInvalidBaseSentinel.
|
||||
static const int kInvalidBaseSentinel;
|
||||
|
||||
public:
|
||||
// Sentinel: there could never be an elf image at this address.
|
||||
static constexpr const void *const kInvalidBase =
|
||||
static_cast<const void*>(&kInvalidBaseSentinel);
|
||||
|
||||
// Information about a single vdso symbol.
|
||||
// All pointers are into .dynsym, .dynstr, or .text of the VDSO.
|
||||
// Do not free() them or modify through them.
|
||||
struct SymbolInfo {
|
||||
const char *name; // E.g. "__vdso_getcpu"
|
||||
const char *version; // E.g. "LINUX_2.6", could be ""
|
||||
// for unversioned symbol.
|
||||
const void *address; // Relocated symbol address.
|
||||
const ElfW(Sym) *symbol; // Symbol in the dynamic symbol table.
|
||||
};
|
||||
|
||||
// Supports iteration over all dynamic symbols.
|
||||
class SymbolIterator {
|
||||
public:
|
||||
friend class ElfMemImage;
|
||||
const SymbolInfo *operator->() const;
|
||||
const SymbolInfo &operator*() const;
|
||||
SymbolIterator& operator++();
|
||||
bool operator!=(const SymbolIterator &rhs) const;
|
||||
bool operator==(const SymbolIterator &rhs) const;
|
||||
private:
|
||||
SymbolIterator(const void *const image, int index);
|
||||
void Update(int incr);
|
||||
SymbolInfo info_;
|
||||
int index_;
|
||||
const void *const image_;
|
||||
};
|
||||
|
||||
|
||||
explicit ElfMemImage(const void *base);
|
||||
void Init(const void *base);
|
||||
bool IsPresent() const { return ehdr_ != nullptr; }
|
||||
const ElfW(Phdr)* GetPhdr(int index) const;
|
||||
const ElfW(Sym)* GetDynsym(int index) const;
|
||||
const ElfW(Versym)* GetVersym(int index) const;
|
||||
const ElfW(Verdef)* GetVerdef(int index) const;
|
||||
const ElfW(Verdaux)* GetVerdefAux(const ElfW(Verdef) *verdef) const;
|
||||
const char* GetDynstr(ElfW(Word) offset) const;
|
||||
const void* GetSymAddr(const ElfW(Sym) *sym) const;
|
||||
const char* GetVerstr(ElfW(Word) offset) const;
|
||||
int GetNumSymbols() const;
|
||||
|
||||
SymbolIterator begin() const;
|
||||
SymbolIterator end() const;
|
||||
|
||||
// Look up versioned dynamic symbol in the image.
|
||||
// Returns false if image is not present, or doesn't contain given
|
||||
// symbol/version/type combination.
|
||||
// If info_out is non-null, additional details are filled in.
|
||||
bool LookupSymbol(const char *name, const char *version,
|
||||
int symbol_type, SymbolInfo *info_out) const;
|
||||
|
||||
// Find info about symbol (if any) which overlaps given address.
|
||||
// Returns true if symbol was found; false if image isn't present
|
||||
// or doesn't have a symbol overlapping given address.
|
||||
// If info_out is non-null, additional details are filled in.
|
||||
bool LookupSymbolByAddress(const void *address, SymbolInfo *info_out) const;
|
||||
|
||||
private:
|
||||
const ElfW(Ehdr) *ehdr_;
|
||||
const ElfW(Sym) *dynsym_;
|
||||
const ElfW(Versym) *versym_;
|
||||
const ElfW(Verdef) *verdef_;
|
||||
const ElfW(Word) *hash_;
|
||||
const char *dynstr_;
|
||||
size_t strsize_;
|
||||
size_t verdefnum_;
|
||||
ElfW(Addr) link_base_; // Link-time base (p_vaddr of first PT_LOAD).
|
||||
};
|
||||
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_HAVE_ELF_MEM_IMAGE
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_ELF_MEM_IMAGE_H_
|
||||
@@ -0,0 +1,192 @@
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_STACKTRACE_AARCH64_INL_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_STACKTRACE_AARCH64_INL_H_
|
||||
|
||||
// Generate stack tracer for aarch64
|
||||
|
||||
#if defined(__linux__)
|
||||
#include <sys/mman.h>
|
||||
#include <ucontext.h>
|
||||
#include <unistd.h>
|
||||
#endif
|
||||
|
||||
#include <atomic>
|
||||
#include <cassert>
|
||||
#include <cstdint>
|
||||
#include <iostream>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/debugging/internal/address_is_readable.h"
|
||||
#include "absl/debugging/internal/vdso_support.h" // a no-op on non-elf or non-glibc systems
|
||||
#include "absl/debugging/stacktrace.h"
|
||||
|
||||
static const uintptr_t kUnknownFrameSize = 0;
|
||||
|
||||
#if defined(__linux__)
|
||||
// Returns the address of the VDSO __kernel_rt_sigreturn function, if present.
|
||||
static const unsigned char* GetKernelRtSigreturnAddress() {
|
||||
constexpr uintptr_t kImpossibleAddress = 1;
|
||||
ABSL_CONST_INIT static std::atomic<uintptr_t> memoized{kImpossibleAddress};
|
||||
uintptr_t address = memoized.load(std::memory_order_relaxed);
|
||||
if (address != kImpossibleAddress) {
|
||||
return reinterpret_cast<const unsigned char*>(address);
|
||||
}
|
||||
|
||||
address = reinterpret_cast<uintptr_t>(nullptr);
|
||||
|
||||
#ifdef ABSL_HAVE_VDSO_SUPPORT
|
||||
absl::debugging_internal::VDSOSupport vdso;
|
||||
if (vdso.IsPresent()) {
|
||||
absl::debugging_internal::VDSOSupport::SymbolInfo symbol_info;
|
||||
if (!vdso.LookupSymbol("__kernel_rt_sigreturn", "LINUX_2.6.39", STT_FUNC,
|
||||
&symbol_info) ||
|
||||
symbol_info.address == nullptr) {
|
||||
// Unexpected: VDSO is present, yet the expected symbol is missing
|
||||
// or null.
|
||||
assert(false && "VDSO is present, but doesn't have expected symbol");
|
||||
} else {
|
||||
if (reinterpret_cast<uintptr_t>(symbol_info.address) !=
|
||||
kImpossibleAddress) {
|
||||
address = reinterpret_cast<uintptr_t>(symbol_info.address);
|
||||
} else {
|
||||
assert(false && "VDSO returned invalid address");
|
||||
}
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
memoized.store(address, std::memory_order_relaxed);
|
||||
return reinterpret_cast<const unsigned char*>(address);
|
||||
}
|
||||
#endif // __linux__
|
||||
|
||||
// Compute the size of a stack frame in [low..high). We assume that
|
||||
// low < high. Return size of kUnknownFrameSize.
|
||||
template<typename T>
|
||||
static inline uintptr_t ComputeStackFrameSize(const T* low,
|
||||
const T* high) {
|
||||
const char* low_char_ptr = reinterpret_cast<const char *>(low);
|
||||
const char* high_char_ptr = reinterpret_cast<const char *>(high);
|
||||
return low < high ? high_char_ptr - low_char_ptr : kUnknownFrameSize;
|
||||
}
|
||||
|
||||
// Given a pointer to a stack frame, locate and return the calling
|
||||
// stackframe, or return null if no stackframe can be found. Perform sanity
|
||||
// checks (the strictness of which is controlled by the boolean parameter
|
||||
// "STRICT_UNWINDING") to reduce the chance that a bad pointer is returned.
|
||||
template<bool STRICT_UNWINDING, bool WITH_CONTEXT>
|
||||
static void **NextStackFrame(void **old_frame_pointer, const void *uc) {
|
||||
void **new_frame_pointer = reinterpret_cast<void**>(*old_frame_pointer);
|
||||
bool check_frame_size = true;
|
||||
|
||||
#if defined(__linux__)
|
||||
if (WITH_CONTEXT && uc != nullptr) {
|
||||
// Check to see if next frame's return address is __kernel_rt_sigreturn.
|
||||
if (old_frame_pointer[1] == GetKernelRtSigreturnAddress()) {
|
||||
const ucontext_t *ucv = static_cast<const ucontext_t *>(uc);
|
||||
// old_frame_pointer[0] is not suitable for unwinding, look at
|
||||
// ucontext to discover frame pointer before signal.
|
||||
void **const pre_signal_frame_pointer =
|
||||
reinterpret_cast<void **>(ucv->uc_mcontext.regs[29]);
|
||||
|
||||
// Check that alleged frame pointer is actually readable. This is to
|
||||
// prevent "double fault" in case we hit the first fault due to e.g.
|
||||
// stack corruption.
|
||||
if (!absl::debugging_internal::AddressIsReadable(
|
||||
pre_signal_frame_pointer))
|
||||
return nullptr;
|
||||
|
||||
// Alleged frame pointer is readable, use it for further unwinding.
|
||||
new_frame_pointer = pre_signal_frame_pointer;
|
||||
|
||||
// Skip frame size check if we return from a signal. We may be using a
|
||||
// an alternate stack for signals.
|
||||
check_frame_size = false;
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
// aarch64 ABI requires stack pointer to be 16-byte-aligned.
|
||||
if ((reinterpret_cast<uintptr_t>(new_frame_pointer) & 15) != 0)
|
||||
return nullptr;
|
||||
|
||||
// Check frame size. In strict mode, we assume frames to be under
|
||||
// 100,000 bytes. In non-strict mode, we relax the limit to 1MB.
|
||||
if (check_frame_size) {
|
||||
const uintptr_t max_size = STRICT_UNWINDING ? 100000 : 1000000;
|
||||
const uintptr_t frame_size =
|
||||
ComputeStackFrameSize(old_frame_pointer, new_frame_pointer);
|
||||
if (frame_size == kUnknownFrameSize || frame_size > max_size)
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
return new_frame_pointer;
|
||||
}
|
||||
|
||||
template <bool IS_STACK_FRAMES, bool IS_WITH_CONTEXT>
|
||||
static int UnwindImpl(void** result, int* sizes, int max_depth, int skip_count,
|
||||
const void *ucp, int *min_dropped_frames) {
|
||||
#ifdef __GNUC__
|
||||
void **frame_pointer = reinterpret_cast<void**>(__builtin_frame_address(0));
|
||||
#else
|
||||
# error reading stack point not yet supported on this platform.
|
||||
#endif
|
||||
|
||||
skip_count++; // Skip the frame for this function.
|
||||
int n = 0;
|
||||
|
||||
// The frame pointer points to low address of a frame. The first 64-bit
|
||||
// word of a frame points to the next frame up the call chain, which normally
|
||||
// is just after the high address of the current frame. The second word of
|
||||
// a frame contains return adress of to the caller. To find a pc value
|
||||
// associated with the current frame, we need to go down a level in the call
|
||||
// chain. So we remember return the address of the last frame seen. This
|
||||
// does not work for the first stack frame, which belongs to UnwindImp() but
|
||||
// we skip the frame for UnwindImp() anyway.
|
||||
void* prev_return_address = nullptr;
|
||||
|
||||
while (frame_pointer && n < max_depth) {
|
||||
// The absl::GetStackFrames routine is called when we are in some
|
||||
// informational context (the failure signal handler for example).
|
||||
// Use the non-strict unwinding rules to produce a stack trace
|
||||
// that is as complete as possible (even if it contains a few bogus
|
||||
// entries in some rare cases).
|
||||
void **next_frame_pointer =
|
||||
NextStackFrame<!IS_STACK_FRAMES, IS_WITH_CONTEXT>(frame_pointer, ucp);
|
||||
|
||||
if (skip_count > 0) {
|
||||
skip_count--;
|
||||
} else {
|
||||
result[n] = prev_return_address;
|
||||
if (IS_STACK_FRAMES) {
|
||||
sizes[n] = ComputeStackFrameSize(frame_pointer, next_frame_pointer);
|
||||
}
|
||||
n++;
|
||||
}
|
||||
prev_return_address = frame_pointer[1];
|
||||
frame_pointer = next_frame_pointer;
|
||||
}
|
||||
if (min_dropped_frames != nullptr) {
|
||||
// Implementation detail: we clamp the max of frames we are willing to
|
||||
// count, so as not to spend too much time in the loop below.
|
||||
const int kMaxUnwind = 200;
|
||||
int j = 0;
|
||||
for (; frame_pointer != nullptr && j < kMaxUnwind; j++) {
|
||||
frame_pointer =
|
||||
NextStackFrame<!IS_STACK_FRAMES, IS_WITH_CONTEXT>(frame_pointer, ucp);
|
||||
}
|
||||
*min_dropped_frames = j;
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
bool StackTraceWorksForTest() {
|
||||
return true;
|
||||
}
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_STACKTRACE_AARCH64_INL_H_
|
||||
@@ -0,0 +1,125 @@
|
||||
// Copyright 2011 and onwards Google Inc.
|
||||
// All rights reserved.
|
||||
//
|
||||
// Author: Doug Kwan
|
||||
// This is inspired by Craig Silverstein's PowerPC stacktrace code.
|
||||
//
|
||||
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_STACKTRACE_ARM_INL_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_STACKTRACE_ARM_INL_H_
|
||||
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/debugging/stacktrace.h"
|
||||
|
||||
// WARNING:
|
||||
// This only works if all your code is in either ARM or THUMB mode. With
|
||||
// interworking, the frame pointer of the caller can either be in r11 (ARM
|
||||
// mode) or r7 (THUMB mode). A callee only saves the frame pointer of its
|
||||
// mode in a fixed location on its stack frame. If the caller is a different
|
||||
// mode, there is no easy way to find the frame pointer. It can either be
|
||||
// still in the designated register or saved on stack along with other callee
|
||||
// saved registers.
|
||||
|
||||
// Given a pointer to a stack frame, locate and return the calling
|
||||
// stackframe, or return nullptr if no stackframe can be found. Perform sanity
|
||||
// checks (the strictness of which is controlled by the boolean parameter
|
||||
// "STRICT_UNWINDING") to reduce the chance that a bad pointer is returned.
|
||||
template<bool STRICT_UNWINDING>
|
||||
static void **NextStackFrame(void **old_sp) {
|
||||
void **new_sp = (void**) old_sp[-1];
|
||||
|
||||
// Check that the transition from frame pointer old_sp to frame
|
||||
// pointer new_sp isn't clearly bogus
|
||||
if (STRICT_UNWINDING) {
|
||||
// With the stack growing downwards, older stack frame must be
|
||||
// at a greater address that the current one.
|
||||
if (new_sp <= old_sp) return nullptr;
|
||||
// Assume stack frames larger than 100,000 bytes are bogus.
|
||||
if ((uintptr_t)new_sp - (uintptr_t)old_sp > 100000) return nullptr;
|
||||
} else {
|
||||
// In the non-strict mode, allow discontiguous stack frames.
|
||||
// (alternate-signal-stacks for example).
|
||||
if (new_sp == old_sp) return nullptr;
|
||||
// And allow frames upto about 1MB.
|
||||
if ((new_sp > old_sp)
|
||||
&& ((uintptr_t)new_sp - (uintptr_t)old_sp > 1000000)) return nullptr;
|
||||
}
|
||||
if ((uintptr_t)new_sp & (sizeof(void *) - 1)) return nullptr;
|
||||
return new_sp;
|
||||
}
|
||||
|
||||
// This ensures that absl::GetStackTrace sets up the Link Register properly.
|
||||
#ifdef __GNUC__
|
||||
void StacktraceArmDummyFunction() __attribute__((noinline));
|
||||
void StacktraceArmDummyFunction() { __asm__ volatile(""); }
|
||||
#else
|
||||
# error StacktraceArmDummyFunction() needs to be ported to this platform.
|
||||
#endif
|
||||
|
||||
template <bool IS_STACK_FRAMES, bool IS_WITH_CONTEXT>
|
||||
static int UnwindImpl(void** result, int* sizes, int max_depth, int skip_count,
|
||||
const void * /* ucp */, int *min_dropped_frames) {
|
||||
#ifdef __GNUC__
|
||||
void **sp = reinterpret_cast<void**>(__builtin_frame_address(0));
|
||||
#else
|
||||
# error reading stack point not yet supported on this platform.
|
||||
#endif
|
||||
|
||||
// On ARM, the return address is stored in the link register (r14).
|
||||
// This is not saved on the stack frame of a leaf function. To
|
||||
// simplify code that reads return addresses, we call a dummy
|
||||
// function so that the return address of this function is also
|
||||
// stored in the stack frame. This works at least for gcc.
|
||||
StacktraceArmDummyFunction();
|
||||
|
||||
int n = 0;
|
||||
while (sp && n < max_depth) {
|
||||
// The absl::GetStackFrames routine is called when we are in some
|
||||
// informational context (the failure signal handler for example).
|
||||
// Use the non-strict unwinding rules to produce a stack trace
|
||||
// that is as complete as possible (even if it contains a few bogus
|
||||
// entries in some rare cases).
|
||||
void **next_sp = NextStackFrame<!IS_STACK_FRAMES>(sp);
|
||||
|
||||
if (skip_count > 0) {
|
||||
skip_count--;
|
||||
} else {
|
||||
result[n] = *sp;
|
||||
|
||||
if (IS_STACK_FRAMES) {
|
||||
if (next_sp > sp) {
|
||||
sizes[n] = (uintptr_t)next_sp - (uintptr_t)sp;
|
||||
} else {
|
||||
// A frame-size of 0 is used to indicate unknown frame size.
|
||||
sizes[n] = 0;
|
||||
}
|
||||
}
|
||||
n++;
|
||||
}
|
||||
sp = next_sp;
|
||||
}
|
||||
if (min_dropped_frames != nullptr) {
|
||||
// Implementation detail: we clamp the max of frames we are willing to
|
||||
// count, so as not to spend too much time in the loop below.
|
||||
const int kMaxUnwind = 200;
|
||||
int j = 0;
|
||||
for (; sp != nullptr && j < kMaxUnwind; j++) {
|
||||
sp = NextStackFrame<!IS_STACK_FRAMES>(sp);
|
||||
}
|
||||
*min_dropped_frames = j;
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
bool StackTraceWorksForTest() {
|
||||
return false;
|
||||
}
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_STACKTRACE_ARM_INL_H_
|
||||
@@ -0,0 +1,70 @@
|
||||
/*
|
||||
* Copyright 2017 The Abseil Authors.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
|
||||
* Defines ABSL_STACKTRACE_INL_HEADER to the *-inl.h containing
|
||||
* actual unwinder implementation.
|
||||
* This header is "private" to stacktrace.cc.
|
||||
* DO NOT include it into any other files.
|
||||
*/
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_STACKTRACE_CONFIG_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_STACKTRACE_CONFIG_H_
|
||||
|
||||
#if defined(ABSL_STACKTRACE_INL_HEADER)
|
||||
#error ABSL_STACKTRACE_INL_HEADER cannot be directly set
|
||||
|
||||
#elif defined(_WIN32)
|
||||
#define ABSL_STACKTRACE_INL_HEADER \
|
||||
"absl/debugging/internal/stacktrace_win32-inl.inc"
|
||||
|
||||
#elif defined(__linux__) && !defined(__ANDROID__)
|
||||
|
||||
#if !defined(NO_FRAME_POINTER)
|
||||
# if defined(__i386__) || defined(__x86_64__)
|
||||
#define ABSL_STACKTRACE_INL_HEADER \
|
||||
"absl/debugging/internal/stacktrace_x86-inl.inc"
|
||||
# elif defined(__ppc__) || defined(__PPC__)
|
||||
#define ABSL_STACKTRACE_INL_HEADER \
|
||||
"absl/debugging/internal/stacktrace_powerpc-inl.inc"
|
||||
# elif defined(__aarch64__)
|
||||
#define ABSL_STACKTRACE_INL_HEADER \
|
||||
"absl/debugging/internal/stacktrace_aarch64-inl.inc"
|
||||
# elif defined(__arm__)
|
||||
// Note: When using glibc this may require -funwind-tables to function properly.
|
||||
#define ABSL_STACKTRACE_INL_HEADER \
|
||||
"absl/debugging/internal/stacktrace_generic-inl.inc"
|
||||
# else
|
||||
#define ABSL_STACKTRACE_INL_HEADER \
|
||||
"absl/debugging/internal/stacktrace_unimplemented-inl.inc"
|
||||
# endif
|
||||
#else // defined(NO_FRAME_POINTER)
|
||||
# if defined(__i386__) || defined(__x86_64__) || defined(__aarch64__)
|
||||
#define ABSL_STACKTRACE_INL_HEADER \
|
||||
"absl/debugging/internal/stacktrace_generic-inl.inc"
|
||||
# elif defined(__ppc__) || defined(__PPC__)
|
||||
#define ABSL_STACKTRACE_INL_HEADER \
|
||||
"absl/debugging/internal/stacktrace_generic-inl.inc"
|
||||
# else
|
||||
#define ABSL_STACKTRACE_INL_HEADER \
|
||||
"absl/debugging/internal/stacktrace_unimplemented-inl.inc"
|
||||
# endif
|
||||
#endif // NO_FRAME_POINTER
|
||||
|
||||
#else
|
||||
#define ABSL_STACKTRACE_INL_HEADER \
|
||||
"absl/debugging/internal/stacktrace_unimplemented-inl.inc"
|
||||
|
||||
#endif
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_STACKTRACE_CONFIG_H_
|
||||
@@ -0,0 +1,99 @@
|
||||
// Copyright 2000 - 2007 Google Inc.
|
||||
// All rights reserved.
|
||||
//
|
||||
// Author: Sanjay Ghemawat
|
||||
//
|
||||
// Portable implementation - just use glibc
|
||||
//
|
||||
// Note: The glibc implementation may cause a call to malloc.
|
||||
// This can cause a deadlock in HeapProfiler.
|
||||
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_STACKTRACE_GENERIC_INL_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_STACKTRACE_GENERIC_INL_H_
|
||||
|
||||
#include <execinfo.h>
|
||||
#include <atomic>
|
||||
#include <cstring>
|
||||
|
||||
#include "absl/debugging/stacktrace.h"
|
||||
#include "absl/base/attributes.h"
|
||||
|
||||
// Sometimes, we can try to get a stack trace from within a stack
|
||||
// trace, because we don't block signals inside this code (which would be too
|
||||
// expensive: the two extra system calls per stack trace do matter here).
|
||||
// That can cause a self-deadlock.
|
||||
// Protect against such reentrant call by failing to get a stack trace.
|
||||
//
|
||||
// We use __thread here because the code here is extremely low level -- it is
|
||||
// called while collecting stack traces from within malloc and mmap, and thus
|
||||
// can not call anything which might call malloc or mmap itself.
|
||||
static __thread int recursive = 0;
|
||||
|
||||
// The stack trace function might be invoked very early in the program's
|
||||
// execution (e.g. from the very first malloc if using tcmalloc). Also, the
|
||||
// glibc implementation itself will trigger malloc the first time it is called.
|
||||
// As such, we suppress usage of backtrace during this early stage of execution.
|
||||
static std::atomic<bool> disable_stacktraces(true); // Disabled until healthy.
|
||||
// Waiting until static initializers run seems to be late enough.
|
||||
// This file is included into stacktrace.cc so this will only run once.
|
||||
ABSL_ATTRIBUTE_UNUSED static int stacktraces_enabler = []() {
|
||||
void* unused_stack[1];
|
||||
// Force the first backtrace to happen early to get the one-time shared lib
|
||||
// loading (allocation) out of the way. After the first call it is much safer
|
||||
// to use backtrace from a signal handler if we crash somewhere later.
|
||||
backtrace(unused_stack, 1);
|
||||
disable_stacktraces.store(false, std::memory_order_relaxed);
|
||||
return 0;
|
||||
}();
|
||||
|
||||
template <bool IS_STACK_FRAMES, bool IS_WITH_CONTEXT>
|
||||
static int UnwindImpl(void** result, int* sizes, int max_depth, int skip_count,
|
||||
const void *ucp, int *min_dropped_frames) {
|
||||
if (recursive || disable_stacktraces.load(std::memory_order_relaxed)) {
|
||||
return 0;
|
||||
}
|
||||
++recursive;
|
||||
|
||||
static_cast<void>(ucp); // Unused.
|
||||
static const int kStackLength = 64;
|
||||
void * stack[kStackLength];
|
||||
int size;
|
||||
|
||||
size = backtrace(stack, kStackLength);
|
||||
skip_count++; // we want to skip the current frame as well
|
||||
int result_count = size - skip_count;
|
||||
if (result_count < 0)
|
||||
result_count = 0;
|
||||
if (result_count > max_depth)
|
||||
result_count = max_depth;
|
||||
for (int i = 0; i < result_count; i++)
|
||||
result[i] = stack[i + skip_count];
|
||||
|
||||
if (IS_STACK_FRAMES) {
|
||||
// No implementation for finding out the stack frame sizes yet.
|
||||
memset(sizes, 0, sizeof(*sizes) * result_count);
|
||||
}
|
||||
if (min_dropped_frames != nullptr) {
|
||||
if (size - skip_count - max_depth > 0) {
|
||||
*min_dropped_frames = size - skip_count - max_depth;
|
||||
} else {
|
||||
*min_dropped_frames = 0;
|
||||
}
|
||||
}
|
||||
|
||||
--recursive;
|
||||
|
||||
return result_count;
|
||||
}
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
bool StackTraceWorksForTest() {
|
||||
return true;
|
||||
}
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_STACKTRACE_GENERIC_INL_H_
|
||||
@@ -0,0 +1,248 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Produce stack trace. I'm guessing (hoping!) the code is much like
|
||||
// for x86. For apple machines, at least, it seems to be; see
|
||||
// https://developer.apple.com/documentation/mac/runtimehtml/RTArch-59.html
|
||||
// https://www.linux-foundation.org/spec/ELF/ppc64/PPC-elf64abi-1.9.html#STACK
|
||||
// Linux has similar code: http://patchwork.ozlabs.org/linuxppc/patch?id=8882
|
||||
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_STACKTRACE_POWERPC_INL_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_STACKTRACE_POWERPC_INL_H_
|
||||
|
||||
#if defined(__linux__)
|
||||
#include <asm/ptrace.h> // for PT_NIP.
|
||||
#include <ucontext.h> // for ucontext_t
|
||||
#endif
|
||||
|
||||
#include <unistd.h>
|
||||
#include <cassert>
|
||||
#include <cstdint>
|
||||
#include <cstdio>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/optimization.h"
|
||||
#include "absl/base/port.h"
|
||||
#include "absl/debugging/stacktrace.h"
|
||||
#include "absl/debugging/internal/address_is_readable.h"
|
||||
#include "absl/debugging/internal/vdso_support.h" // a no-op on non-elf or non-glibc systems
|
||||
|
||||
// Given a stack pointer, return the saved link register value.
|
||||
// Note that this is the link register for a callee.
|
||||
static inline void *StacktracePowerPCGetLR(void **sp) {
|
||||
// PowerPC has 3 main ABIs, which say where in the stack the
|
||||
// Link Register is. For DARWIN and AIX (used by apple and
|
||||
// linux ppc64), it's in sp[2]. For SYSV (used by linux ppc),
|
||||
// it's in sp[1].
|
||||
#if defined(_CALL_AIX) || defined(_CALL_DARWIN)
|
||||
return *(sp+2);
|
||||
#elif defined(_CALL_SYSV)
|
||||
return *(sp+1);
|
||||
#elif defined(__APPLE__) || defined(__FreeBSD__) || \
|
||||
(defined(__linux__) && defined(__PPC64__))
|
||||
// This check is in case the compiler doesn't define _CALL_AIX/etc.
|
||||
return *(sp+2);
|
||||
#elif defined(__linux)
|
||||
// This check is in case the compiler doesn't define _CALL_SYSV.
|
||||
return *(sp+1);
|
||||
#else
|
||||
#error Need to specify the PPC ABI for your archiecture.
|
||||
#endif
|
||||
}
|
||||
|
||||
// Given a pointer to a stack frame, locate and return the calling
|
||||
// stackframe, or return null if no stackframe can be found. Perform sanity
|
||||
// checks (the strictness of which is controlled by the boolean parameter
|
||||
// "STRICT_UNWINDING") to reduce the chance that a bad pointer is returned.
|
||||
template<bool STRICT_UNWINDING, bool IS_WITH_CONTEXT>
|
||||
ABSL_ATTRIBUTE_NO_SANITIZE_ADDRESS // May read random elements from stack.
|
||||
ABSL_ATTRIBUTE_NO_SANITIZE_MEMORY // May read random elements from stack.
|
||||
static void **NextStackFrame(void **old_sp, const void *uc) {
|
||||
void **new_sp = (void **) *old_sp;
|
||||
enum { kStackAlignment = 16 };
|
||||
|
||||
// Check that the transition from frame pointer old_sp to frame
|
||||
// pointer new_sp isn't clearly bogus
|
||||
if (STRICT_UNWINDING) {
|
||||
// With the stack growing downwards, older stack frame must be
|
||||
// at a greater address that the current one.
|
||||
if (new_sp <= old_sp) return nullptr;
|
||||
// Assume stack frames larger than 100,000 bytes are bogus.
|
||||
if ((uintptr_t)new_sp - (uintptr_t)old_sp > 100000) return nullptr;
|
||||
} else {
|
||||
// In the non-strict mode, allow discontiguous stack frames.
|
||||
// (alternate-signal-stacks for example).
|
||||
if (new_sp == old_sp) return nullptr;
|
||||
// And allow frames upto about 1MB.
|
||||
if ((new_sp > old_sp)
|
||||
&& ((uintptr_t)new_sp - (uintptr_t)old_sp > 1000000)) return nullptr;
|
||||
}
|
||||
if ((uintptr_t)new_sp % kStackAlignment != 0) return nullptr;
|
||||
|
||||
#if defined(__linux__)
|
||||
enum StackTraceKernelSymbolStatus {
|
||||
kNotInitialized = 0, kAddressValid, kAddressInvalid };
|
||||
|
||||
if (IS_WITH_CONTEXT && uc != nullptr) {
|
||||
static StackTraceKernelSymbolStatus kernel_symbol_status =
|
||||
kNotInitialized; // Sentinel: not computed yet.
|
||||
// Initialize with sentinel value: __kernel_rt_sigtramp_rt64 can not
|
||||
// possibly be there.
|
||||
static const unsigned char *kernel_sigtramp_rt64_address = nullptr;
|
||||
if (kernel_symbol_status == kNotInitialized) {
|
||||
absl::debugging_internal::VDSOSupport vdso;
|
||||
if (vdso.IsPresent()) {
|
||||
absl::debugging_internal::VDSOSupport::SymbolInfo
|
||||
sigtramp_rt64_symbol_info;
|
||||
if (!vdso.LookupSymbol(
|
||||
"__kernel_sigtramp_rt64", "LINUX_2.6.15",
|
||||
absl::debugging_internal::VDSOSupport::kVDSOSymbolType,
|
||||
&sigtramp_rt64_symbol_info) ||
|
||||
sigtramp_rt64_symbol_info.address == nullptr) {
|
||||
// Unexpected: VDSO is present, yet the expected symbol is missing
|
||||
// or null.
|
||||
assert(false && "VDSO is present, but doesn't have expected symbol");
|
||||
kernel_symbol_status = kAddressInvalid;
|
||||
} else {
|
||||
kernel_sigtramp_rt64_address =
|
||||
reinterpret_cast<const unsigned char *>(
|
||||
sigtramp_rt64_symbol_info.address);
|
||||
kernel_symbol_status = kAddressValid;
|
||||
}
|
||||
} else {
|
||||
kernel_symbol_status = kAddressInvalid;
|
||||
}
|
||||
}
|
||||
|
||||
if (new_sp != nullptr &&
|
||||
kernel_symbol_status == kAddressValid &&
|
||||
StacktracePowerPCGetLR(new_sp) == kernel_sigtramp_rt64_address) {
|
||||
const ucontext_t* signal_context =
|
||||
reinterpret_cast<const ucontext_t*>(uc);
|
||||
void **const sp_before_signal =
|
||||
reinterpret_cast<void**>(signal_context->uc_mcontext.gp_regs[PT_R1]);
|
||||
// Check that alleged sp before signal is nonnull and is reasonably
|
||||
// aligned.
|
||||
if (sp_before_signal != nullptr &&
|
||||
((uintptr_t)sp_before_signal % kStackAlignment) == 0) {
|
||||
// Check that alleged stack pointer is actually readable. This is to
|
||||
// prevent a "double fault" in case we hit the first fault due to e.g.
|
||||
// a stack corruption.
|
||||
if (absl::debugging_internal::AddressIsReadable(sp_before_signal)) {
|
||||
// Alleged stack pointer is readable, use it for further unwinding.
|
||||
new_sp = sp_before_signal;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
return new_sp;
|
||||
}
|
||||
|
||||
// This ensures that absl::GetStackTrace sets up the Link Register properly.
|
||||
ABSL_ATTRIBUTE_NOINLINE static void AbslStacktracePowerPCDummyFunction() {
|
||||
ABSL_BLOCK_TAIL_CALL_OPTIMIZATION();
|
||||
}
|
||||
|
||||
template <bool IS_STACK_FRAMES, bool IS_WITH_CONTEXT>
|
||||
ABSL_ATTRIBUTE_NO_SANITIZE_ADDRESS // May read random elements from stack.
|
||||
ABSL_ATTRIBUTE_NO_SANITIZE_MEMORY // May read random elements from stack.
|
||||
static int UnwindImpl(void** result, int* sizes, int max_depth, int skip_count,
|
||||
const void *ucp, int *min_dropped_frames) {
|
||||
void **sp;
|
||||
// Apple macOS uses an old version of gnu as -- both Darwin 7.9.0 (Panther)
|
||||
// and Darwin 8.8.1 (Tiger) use as 1.38. This means we have to use a
|
||||
// different asm syntax. I don't know quite the best way to discriminate
|
||||
// systems using the old as from the new one; I've gone with __APPLE__.
|
||||
#ifdef __APPLE__
|
||||
__asm__ volatile ("mr %0,r1" : "=r" (sp));
|
||||
#else
|
||||
__asm__ volatile ("mr %0,1" : "=r" (sp));
|
||||
#endif
|
||||
|
||||
// On PowerPC, the "Link Register" or "Link Record" (LR), is a stack
|
||||
// entry that holds the return address of the subroutine call (what
|
||||
// instruction we run after our function finishes). This is the
|
||||
// same as the stack-pointer of our parent routine, which is what we
|
||||
// want here. While the compiler will always(?) set up LR for
|
||||
// subroutine calls, it may not for leaf functions (such as this one).
|
||||
// This routine forces the compiler (at least gcc) to push it anyway.
|
||||
AbslStacktracePowerPCDummyFunction();
|
||||
|
||||
// The LR save area is used by the callee, so the top entry is bogus.
|
||||
skip_count++;
|
||||
|
||||
int n = 0;
|
||||
|
||||
// Unlike ABIs of X86 and ARM, PowerPC ABIs say that return address (in
|
||||
// the link register) of a function call is stored in the caller's stack
|
||||
// frame instead of the callee's. When we look for the return address
|
||||
// associated with a stack frame, we need to make sure that there is a
|
||||
// caller frame before it. So we call NextStackFrame before entering the
|
||||
// loop below and check next_sp instead of sp for loop termination.
|
||||
// The outermost frame is set up by runtimes and it does not have a
|
||||
// caller frame, so it is skipped.
|
||||
|
||||
// The absl::GetStackFrames routine is called when we are in some
|
||||
// informational context (the failure signal handler for example).
|
||||
// Use the non-strict unwinding rules to produce a stack trace
|
||||
// that is as complete as possible (even if it contains a few
|
||||
// bogus entries in some rare cases).
|
||||
void **next_sp = NextStackFrame<!IS_STACK_FRAMES, IS_WITH_CONTEXT>(sp, ucp);
|
||||
|
||||
while (next_sp && n < max_depth) {
|
||||
if (skip_count > 0) {
|
||||
skip_count--;
|
||||
} else {
|
||||
result[n] = StacktracePowerPCGetLR(sp);
|
||||
if (IS_STACK_FRAMES) {
|
||||
if (next_sp > sp) {
|
||||
sizes[n] = (uintptr_t)next_sp - (uintptr_t)sp;
|
||||
} else {
|
||||
// A frame-size of 0 is used to indicate unknown frame size.
|
||||
sizes[n] = 0;
|
||||
}
|
||||
}
|
||||
n++;
|
||||
}
|
||||
|
||||
sp = next_sp;
|
||||
next_sp = NextStackFrame<!IS_STACK_FRAMES, IS_WITH_CONTEXT>(sp, ucp);
|
||||
}
|
||||
|
||||
if (min_dropped_frames != nullptr) {
|
||||
// Implementation detail: we clamp the max of frames we are willing to
|
||||
// count, so as not to spend too much time in the loop below.
|
||||
const int kMaxUnwind = 1000;
|
||||
int j = 0;
|
||||
for (; next_sp != nullptr && j < kMaxUnwind; j++) {
|
||||
next_sp = NextStackFrame<!IS_STACK_FRAMES, IS_WITH_CONTEXT>(next_sp, ucp);
|
||||
}
|
||||
*min_dropped_frames = j;
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
bool StackTraceWorksForTest() {
|
||||
return true;
|
||||
}
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_STACKTRACE_POWERPC_INL_H_
|
||||
@@ -0,0 +1,24 @@
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_STACKTRACE_UNIMPLEMENTED_INL_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_STACKTRACE_UNIMPLEMENTED_INL_H_
|
||||
|
||||
template <bool IS_STACK_FRAMES, bool IS_WITH_CONTEXT>
|
||||
static int UnwindImpl(void** /* result */, int* /* sizes */,
|
||||
int /* max_depth */, int /* skip_count */,
|
||||
const void* /* ucp */, int *min_dropped_frames) {
|
||||
if (min_dropped_frames != nullptr) {
|
||||
*min_dropped_frames = 0;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
bool StackTraceWorksForTest() {
|
||||
return false;
|
||||
}
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_STACKTRACE_UNIMPLEMENTED_INL_H_
|
||||
@@ -0,0 +1,85 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Produces a stack trace for Windows. Normally, one could use
|
||||
// stacktrace_x86-inl.h or stacktrace_x86_64-inl.h -- and indeed, that
|
||||
// should work for binaries compiled using MSVC in "debug" mode.
|
||||
// However, in "release" mode, Windows uses frame-pointer
|
||||
// optimization, which makes getting a stack trace very difficult.
|
||||
//
|
||||
// There are several approaches one can take. One is to use Windows
|
||||
// intrinsics like StackWalk64. These can work, but have restrictions
|
||||
// on how successful they can be. Another attempt is to write a
|
||||
// version of stacktrace_x86-inl.h that has heuristic support for
|
||||
// dealing with FPO, similar to what WinDbg does (see
|
||||
// http://www.nynaeve.net/?p=97). There are (non-working) examples of
|
||||
// these approaches, complete with TODOs, in stacktrace_win32-inl.h#1
|
||||
//
|
||||
// The solution we've ended up doing is to call the undocumented
|
||||
// windows function RtlCaptureStackBackTrace, which probably doesn't
|
||||
// work with FPO but at least is fast, and doesn't require a symbol
|
||||
// server.
|
||||
//
|
||||
// This code is inspired by a patch from David Vitek:
|
||||
// https://code.google.com/p/google-perftools/issues/detail?id=83
|
||||
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_STACKTRACE_WIN32_INL_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_STACKTRACE_WIN32_INL_H_
|
||||
|
||||
#include <windows.h> // for GetProcAddress and GetModuleHandle
|
||||
#include <cassert>
|
||||
|
||||
typedef USHORT NTAPI RtlCaptureStackBackTrace_Function(
|
||||
IN ULONG frames_to_skip,
|
||||
IN ULONG frames_to_capture,
|
||||
OUT PVOID *backtrace,
|
||||
OUT PULONG backtrace_hash);
|
||||
|
||||
// Load the function we need at static init time, where we don't have
|
||||
// to worry about someone else holding the loader's lock.
|
||||
static RtlCaptureStackBackTrace_Function* const RtlCaptureStackBackTrace_fn =
|
||||
(RtlCaptureStackBackTrace_Function*)
|
||||
GetProcAddress(GetModuleHandleA("ntdll.dll"), "RtlCaptureStackBackTrace");
|
||||
|
||||
template <bool IS_STACK_FRAMES, bool IS_WITH_CONTEXT>
|
||||
static int UnwindImpl(void** result, int* sizes, int max_depth, int skip_count,
|
||||
const void*, int* min_dropped_frames) {
|
||||
int n = 0;
|
||||
if (!RtlCaptureStackBackTrace_fn) {
|
||||
// can't find a stacktrace with no function to call
|
||||
} else {
|
||||
n = (int)RtlCaptureStackBackTrace_fn(skip_count + 2, max_depth, result, 0);
|
||||
}
|
||||
if (IS_STACK_FRAMES) {
|
||||
// No implementation for finding out the stack frame sizes yet.
|
||||
memset(sizes, 0, sizeof(*sizes) * n);
|
||||
}
|
||||
if (min_dropped_frames != nullptr) {
|
||||
// Not implemented.
|
||||
*min_dropped_frames = 0;
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
bool StackTraceWorksForTest() {
|
||||
return false;
|
||||
}
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_STACKTRACE_WIN32_INL_H_
|
||||
@@ -0,0 +1,346 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Produce stack trace
|
||||
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_STACKTRACE_X86_INL_INC_
|
||||
#define ABSL_DEBUGGING_INTERNAL_STACKTRACE_X86_INL_INC_
|
||||
|
||||
#if defined(__linux__) && (defined(__i386__) || defined(__x86_64__))
|
||||
#include <ucontext.h> // for ucontext_t
|
||||
#endif
|
||||
|
||||
#if !defined(_WIN32)
|
||||
#include <unistd.h>
|
||||
#endif
|
||||
|
||||
#include <cassert>
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/base/macros.h"
|
||||
#include "absl/base/port.h"
|
||||
#include "absl/debugging/internal/address_is_readable.h"
|
||||
#include "absl/debugging/internal/vdso_support.h" // a no-op on non-elf or non-glibc systems
|
||||
#include "absl/debugging/stacktrace.h"
|
||||
|
||||
#include "absl/base/internal/raw_logging.h"
|
||||
|
||||
using absl::debugging_internal::AddressIsReadable;
|
||||
|
||||
#if defined(__linux__) && defined(__i386__)
|
||||
// Count "push %reg" instructions in VDSO __kernel_vsyscall(),
|
||||
// preceeding "syscall" or "sysenter".
|
||||
// If __kernel_vsyscall uses frame pointer, answer 0.
|
||||
//
|
||||
// kMaxBytes tells how many instruction bytes of __kernel_vsyscall
|
||||
// to analyze before giving up. Up to kMaxBytes+1 bytes of
|
||||
// instructions could be accessed.
|
||||
//
|
||||
// Here are known __kernel_vsyscall instruction sequences:
|
||||
//
|
||||
// SYSENTER (linux-2.6.26/arch/x86/vdso/vdso32/sysenter.S).
|
||||
// Used on Intel.
|
||||
// 0xffffe400 <__kernel_vsyscall+0>: push %ecx
|
||||
// 0xffffe401 <__kernel_vsyscall+1>: push %edx
|
||||
// 0xffffe402 <__kernel_vsyscall+2>: push %ebp
|
||||
// 0xffffe403 <__kernel_vsyscall+3>: mov %esp,%ebp
|
||||
// 0xffffe405 <__kernel_vsyscall+5>: sysenter
|
||||
//
|
||||
// SYSCALL (see linux-2.6.26/arch/x86/vdso/vdso32/syscall.S).
|
||||
// Used on AMD.
|
||||
// 0xffffe400 <__kernel_vsyscall+0>: push %ebp
|
||||
// 0xffffe401 <__kernel_vsyscall+1>: mov %ecx,%ebp
|
||||
// 0xffffe403 <__kernel_vsyscall+3>: syscall
|
||||
//
|
||||
|
||||
// The sequence below isn't actually expected in Google fleet,
|
||||
// here only for completeness. Remove this comment from OSS release.
|
||||
|
||||
// i386 (see linux-2.6.26/arch/x86/vdso/vdso32/int80.S)
|
||||
// 0xffffe400 <__kernel_vsyscall+0>: int $0x80
|
||||
// 0xffffe401 <__kernel_vsyscall+1>: ret
|
||||
//
|
||||
static const int kMaxBytes = 10;
|
||||
|
||||
// We use assert()s instead of DCHECK()s -- this is too low level
|
||||
// for DCHECK().
|
||||
|
||||
static int CountPushInstructions(const unsigned char *const addr) {
|
||||
int result = 0;
|
||||
for (int i = 0; i < kMaxBytes; ++i) {
|
||||
if (addr[i] == 0x89) {
|
||||
// "mov reg,reg"
|
||||
if (addr[i + 1] == 0xE5) {
|
||||
// Found "mov %esp,%ebp".
|
||||
return 0;
|
||||
}
|
||||
++i; // Skip register encoding byte.
|
||||
} else if (addr[i] == 0x0F &&
|
||||
(addr[i + 1] == 0x34 || addr[i + 1] == 0x05)) {
|
||||
// Found "sysenter" or "syscall".
|
||||
return result;
|
||||
} else if ((addr[i] & 0xF0) == 0x50) {
|
||||
// Found "push %reg".
|
||||
++result;
|
||||
} else if (addr[i] == 0xCD && addr[i + 1] == 0x80) {
|
||||
// Found "int $0x80"
|
||||
assert(result == 0);
|
||||
return 0;
|
||||
} else {
|
||||
// Unexpected instruction.
|
||||
assert(false && "unexpected instruction in __kernel_vsyscall");
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
// Unexpected: didn't find SYSENTER or SYSCALL in
|
||||
// [__kernel_vsyscall, __kernel_vsyscall + kMaxBytes) interval.
|
||||
assert(false && "did not find SYSENTER or SYSCALL in __kernel_vsyscall");
|
||||
return 0;
|
||||
}
|
||||
#endif
|
||||
|
||||
// Assume stack frames larger than 100,000 bytes are bogus.
|
||||
static const int kMaxFrameBytes = 100000;
|
||||
|
||||
// Returns the stack frame pointer from signal context, 0 if unknown.
|
||||
// vuc is a ucontext_t *. We use void* to avoid the use
|
||||
// of ucontext_t on non-POSIX systems.
|
||||
static uintptr_t GetFP(const void *vuc) {
|
||||
#if !defined(__linux__)
|
||||
static_cast<void>(vuc); // Avoid an unused argument compiler warning.
|
||||
#else
|
||||
if (vuc != nullptr) {
|
||||
auto *uc = reinterpret_cast<const ucontext_t *>(vuc);
|
||||
#if defined(__i386__)
|
||||
const auto bp = uc->uc_mcontext.gregs[REG_EBP];
|
||||
const auto sp = uc->uc_mcontext.gregs[REG_ESP];
|
||||
#elif defined(__x86_64__)
|
||||
const auto bp = uc->uc_mcontext.gregs[REG_RBP];
|
||||
const auto sp = uc->uc_mcontext.gregs[REG_RSP];
|
||||
#else
|
||||
const uintptr_t bp = 0;
|
||||
const uintptr_t sp = 0;
|
||||
#endif
|
||||
// Sanity-check that the base pointer is valid. It should be as long as
|
||||
// SHRINK_WRAP_FRAME_POINTER is not set, but it's possible that some code in
|
||||
// the process is compiled with --copt=-fomit-frame-pointer or
|
||||
// --copt=-momit-leaf-frame-pointer.
|
||||
//
|
||||
// TODO(bcmills): -momit-leaf-frame-pointer is currently the default
|
||||
// behavior when building with clang. Talk to the C++ toolchain team about
|
||||
// fixing that.
|
||||
if (bp >= sp && bp - sp <= kMaxFrameBytes) return bp;
|
||||
|
||||
// If bp isn't a plausible frame pointer, return the stack pointer instead.
|
||||
// If we're lucky, it points to the start of a stack frame; otherwise, we'll
|
||||
// get one frame of garbage in the stack trace and fail the sanity check on
|
||||
// the next iteration.
|
||||
return sp;
|
||||
}
|
||||
#endif
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Given a pointer to a stack frame, locate and return the calling
|
||||
// stackframe, or return null if no stackframe can be found. Perform sanity
|
||||
// checks (the strictness of which is controlled by the boolean parameter
|
||||
// "STRICT_UNWINDING") to reduce the chance that a bad pointer is returned.
|
||||
template <bool STRICT_UNWINDING, bool WITH_CONTEXT>
|
||||
ABSL_ATTRIBUTE_NO_SANITIZE_ADDRESS // May read random elements from stack.
|
||||
ABSL_ATTRIBUTE_NO_SANITIZE_MEMORY // May read random elements from stack.
|
||||
static void **NextStackFrame(void **old_fp, const void *uc) {
|
||||
void **new_fp = (void **)*old_fp;
|
||||
|
||||
#if defined(__linux__) && defined(__i386__)
|
||||
if (WITH_CONTEXT && uc != nullptr) {
|
||||
// How many "push %reg" instructions are there at __kernel_vsyscall?
|
||||
// This is constant for a given kernel and processor, so compute
|
||||
// it only once.
|
||||
static int num_push_instructions = -1; // Sentinel: not computed yet.
|
||||
// Initialize with sentinel value: __kernel_rt_sigreturn can not possibly
|
||||
// be there.
|
||||
static const unsigned char *kernel_rt_sigreturn_address = nullptr;
|
||||
static const unsigned char *kernel_vsyscall_address = nullptr;
|
||||
if (num_push_instructions == -1) {
|
||||
#ifdef ABSL_HAVE_VDSO_SUPPORT
|
||||
absl::debugging_internal::VDSOSupport vdso;
|
||||
if (vdso.IsPresent()) {
|
||||
absl::debugging_internal::VDSOSupport::SymbolInfo
|
||||
rt_sigreturn_symbol_info;
|
||||
absl::debugging_internal::VDSOSupport::SymbolInfo vsyscall_symbol_info;
|
||||
if (!vdso.LookupSymbol("__kernel_rt_sigreturn", "LINUX_2.5", STT_FUNC,
|
||||
&rt_sigreturn_symbol_info) ||
|
||||
!vdso.LookupSymbol("__kernel_vsyscall", "LINUX_2.5", STT_FUNC,
|
||||
&vsyscall_symbol_info) ||
|
||||
rt_sigreturn_symbol_info.address == nullptr ||
|
||||
vsyscall_symbol_info.address == nullptr) {
|
||||
// Unexpected: 32-bit VDSO is present, yet one of the expected
|
||||
// symbols is missing or null.
|
||||
assert(false && "VDSO is present, but doesn't have expected symbols");
|
||||
num_push_instructions = 0;
|
||||
} else {
|
||||
kernel_rt_sigreturn_address =
|
||||
reinterpret_cast<const unsigned char *>(
|
||||
rt_sigreturn_symbol_info.address);
|
||||
kernel_vsyscall_address =
|
||||
reinterpret_cast<const unsigned char *>(
|
||||
vsyscall_symbol_info.address);
|
||||
num_push_instructions =
|
||||
CountPushInstructions(kernel_vsyscall_address);
|
||||
}
|
||||
} else {
|
||||
num_push_instructions = 0;
|
||||
}
|
||||
#else // ABSL_HAVE_VDSO_SUPPORT
|
||||
num_push_instructions = 0;
|
||||
#endif // ABSL_HAVE_VDSO_SUPPORT
|
||||
}
|
||||
if (num_push_instructions != 0 && kernel_rt_sigreturn_address != nullptr &&
|
||||
old_fp[1] == kernel_rt_sigreturn_address) {
|
||||
const ucontext_t *ucv = static_cast<const ucontext_t *>(uc);
|
||||
// This kernel does not use frame pointer in its VDSO code,
|
||||
// and so %ebp is not suitable for unwinding.
|
||||
void **const reg_ebp =
|
||||
reinterpret_cast<void **>(ucv->uc_mcontext.gregs[REG_EBP]);
|
||||
const unsigned char *const reg_eip =
|
||||
reinterpret_cast<unsigned char *>(ucv->uc_mcontext.gregs[REG_EIP]);
|
||||
if (new_fp == reg_ebp && kernel_vsyscall_address <= reg_eip &&
|
||||
reg_eip - kernel_vsyscall_address < kMaxBytes) {
|
||||
// We "stepped up" to __kernel_vsyscall, but %ebp is not usable.
|
||||
// Restore from 'ucv' instead.
|
||||
void **const reg_esp =
|
||||
reinterpret_cast<void **>(ucv->uc_mcontext.gregs[REG_ESP]);
|
||||
// Check that alleged %esp is not null and is reasonably aligned.
|
||||
if (reg_esp &&
|
||||
((uintptr_t)reg_esp & (sizeof(reg_esp) - 1)) == 0) {
|
||||
// Check that alleged %esp is actually readable. This is to prevent
|
||||
// "double fault" in case we hit the first fault due to e.g. stack
|
||||
// corruption.
|
||||
void *const reg_esp2 = reg_esp[num_push_instructions - 1];
|
||||
if (AddressIsReadable(reg_esp2)) {
|
||||
// Alleged %esp is readable, use it for further unwinding.
|
||||
new_fp = reinterpret_cast<void **>(reg_esp2);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
const uintptr_t old_fp_u = reinterpret_cast<uintptr_t>(old_fp);
|
||||
const uintptr_t new_fp_u = reinterpret_cast<uintptr_t>(new_fp);
|
||||
|
||||
// Check that the transition from frame pointer old_fp to frame
|
||||
// pointer new_fp isn't clearly bogus. Skip the checks if new_fp
|
||||
// matches the signal context, so that we don't skip out early when
|
||||
// using an alternate signal stack.
|
||||
//
|
||||
// TODO(bcmills): The GetFP call should be completely unnecessary when
|
||||
// SHRINK_WRAP_FRAME_POINTER is set (because we should be back in the thread's
|
||||
// stack by this point), but it is empirically still needed (e.g. when the
|
||||
// stack includes a call to abort). unw_get_reg returns UNW_EBADREG for some
|
||||
// frames. Figure out why GetValidFrameAddr and/or libunwind isn't doing what
|
||||
// it's supposed to.
|
||||
if (STRICT_UNWINDING &&
|
||||
(!WITH_CONTEXT || uc == nullptr || new_fp_u != GetFP(uc))) {
|
||||
// With the stack growing downwards, older stack frame must be
|
||||
// at a greater address that the current one.
|
||||
if (new_fp_u <= old_fp_u) return nullptr;
|
||||
if (new_fp_u - old_fp_u > kMaxFrameBytes) return nullptr;
|
||||
} else {
|
||||
if (new_fp == nullptr) return nullptr; // skip AddressIsReadable() below
|
||||
// In the non-strict mode, allow discontiguous stack frames.
|
||||
// (alternate-signal-stacks for example).
|
||||
if (new_fp == old_fp) return nullptr;
|
||||
}
|
||||
|
||||
if (new_fp_u & (sizeof(void *) - 1)) return nullptr;
|
||||
#ifdef __i386__
|
||||
// On 32-bit machines, the stack pointer can be very close to
|
||||
// 0xffffffff, so we explicitly check for a pointer into the
|
||||
// last two pages in the address space
|
||||
if (new_fp_u >= 0xffffe000) return nullptr;
|
||||
#endif
|
||||
#if !defined(_WIN32)
|
||||
if (!STRICT_UNWINDING) {
|
||||
// Lax sanity checks cause a crash in 32-bit tcmalloc/crash_reason_test
|
||||
// on AMD-based machines with VDSO-enabled kernels.
|
||||
// Make an extra sanity check to insure new_fp is readable.
|
||||
// Note: NextStackFrame<false>() is only called while the program
|
||||
// is already on its last leg, so it's ok to be slow here.
|
||||
|
||||
if (!AddressIsReadable(new_fp)) {
|
||||
return nullptr;
|
||||
}
|
||||
}
|
||||
#endif
|
||||
return new_fp;
|
||||
}
|
||||
|
||||
template <bool IS_STACK_FRAMES, bool IS_WITH_CONTEXT>
|
||||
ABSL_ATTRIBUTE_NO_SANITIZE_ADDRESS // May read random elements from stack.
|
||||
ABSL_ATTRIBUTE_NO_SANITIZE_MEMORY // May read random elements from stack.
|
||||
ABSL_ATTRIBUTE_NOINLINE
|
||||
static int UnwindImpl(void **result, int *sizes, int max_depth, int skip_count,
|
||||
const void *ucp, int *min_dropped_frames) {
|
||||
int n = 0;
|
||||
void **fp = reinterpret_cast<void **>(__builtin_frame_address(0));
|
||||
|
||||
while (fp && n < max_depth) {
|
||||
if (*(fp + 1) == reinterpret_cast<void *>(0)) {
|
||||
// In 64-bit code, we often see a frame that
|
||||
// points to itself and has a return address of 0.
|
||||
break;
|
||||
}
|
||||
void **next_fp = NextStackFrame<!IS_STACK_FRAMES, IS_WITH_CONTEXT>(fp, ucp);
|
||||
if (skip_count > 0) {
|
||||
skip_count--;
|
||||
} else {
|
||||
result[n] = *(fp + 1);
|
||||
if (IS_STACK_FRAMES) {
|
||||
if (next_fp > fp) {
|
||||
sizes[n] = (uintptr_t)next_fp - (uintptr_t)fp;
|
||||
} else {
|
||||
// A frame-size of 0 is used to indicate unknown frame size.
|
||||
sizes[n] = 0;
|
||||
}
|
||||
}
|
||||
n++;
|
||||
}
|
||||
fp = next_fp;
|
||||
}
|
||||
if (min_dropped_frames != nullptr) {
|
||||
// Implementation detail: we clamp the max of frames we are willing to
|
||||
// count, so as not to spend too much time in the loop below.
|
||||
const int kMaxUnwind = 1000;
|
||||
int j = 0;
|
||||
for (; fp != nullptr && j < kMaxUnwind; j++) {
|
||||
fp = NextStackFrame<!IS_STACK_FRAMES, IS_WITH_CONTEXT>(fp, ucp);
|
||||
}
|
||||
*min_dropped_frames = j;
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
bool StackTraceWorksForTest() {
|
||||
return true;
|
||||
}
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_STACKTRACE_X86_INL_INC_
|
||||
+128
@@ -0,0 +1,128 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// This file contains internal parts of the Abseil symbolizer.
|
||||
// Do not depend on the anything in this file, it may change at anytime.
|
||||
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_SYMBOLIZE_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_SYMBOLIZE_H_
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
#ifdef ABSL_INTERNAL_HAVE_ELF_SYMBOLIZE
|
||||
#error ABSL_INTERNAL_HAVE_ELF_SYMBOLIZE cannot be directly set
|
||||
#elif defined(__ELF__) && defined(__GLIBC__) && !defined(__native_client__) && \
|
||||
!defined(__asmjs__) && !defined(__wasm__)
|
||||
#define ABSL_INTERNAL_HAVE_ELF_SYMBOLIZE 1
|
||||
|
||||
#include <elf.h>
|
||||
#include <link.h> // For ElfW() macro.
|
||||
#include <functional>
|
||||
#include <string>
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
|
||||
// Iterates over all sections, invoking callback on each with the section name
|
||||
// and the section header.
|
||||
//
|
||||
// Returns true on success; otherwise returns false in case of errors.
|
||||
//
|
||||
// This is not async-signal-safe.
|
||||
bool ForEachSection(int fd,
|
||||
const std::function<bool(const std::string& name,
|
||||
const ElfW(Shdr) &)>& callback);
|
||||
|
||||
// Gets the section header for the given name, if it exists. Returns true on
|
||||
// success. Otherwise, returns false.
|
||||
bool GetSectionHeaderByName(int fd, const char *name, size_t name_len,
|
||||
ElfW(Shdr) *out);
|
||||
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_INTERNAL_HAVE_ELF_SYMBOLIZE
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
|
||||
struct SymbolDecoratorArgs {
|
||||
// The program counter we are getting symbolic name for.
|
||||
const void *pc;
|
||||
// 0 for main executable, load address for shared libraries.
|
||||
ptrdiff_t relocation;
|
||||
// Read-only file descriptor for ELF image covering "pc",
|
||||
// or -1 if no such ELF image exists in /proc/self/maps.
|
||||
int fd;
|
||||
// Output buffer, size.
|
||||
// Note: the buffer may not be empty -- default symbolizer may have already
|
||||
// produced some output, and earlier decorators may have adorned it in
|
||||
// some way. You are free to replace or augment the contents (within the
|
||||
// symbol_buf_size limit).
|
||||
char *const symbol_buf;
|
||||
size_t symbol_buf_size;
|
||||
// Temporary scratch space, size.
|
||||
// Use that space in preference to allocating your own stack buffer to
|
||||
// conserve stack.
|
||||
char *const tmp_buf;
|
||||
size_t tmp_buf_size;
|
||||
// User-provided argument
|
||||
void* arg;
|
||||
};
|
||||
using SymbolDecorator = void (*)(const SymbolDecoratorArgs *);
|
||||
|
||||
// Installs a function-pointer as a decorator. Returns a value less than zero
|
||||
// if the system cannot install the decorator. Otherwise, returns a unique
|
||||
// identifier corresponding to the decorator. This identifier can be used to
|
||||
// uninstall the decorator - See RemoveSymbolDecorator() below.
|
||||
int InstallSymbolDecorator(SymbolDecorator decorator, void* arg);
|
||||
|
||||
// Removes a previously installed function-pointer decorator. Parameter "ticket"
|
||||
// is the return-value from calling InstallSymbolDecorator().
|
||||
bool RemoveSymbolDecorator(int ticket);
|
||||
|
||||
// Remove all installed decorators. Returns true if successful, false if
|
||||
// symbolization is currently in progress.
|
||||
bool RemoveAllSymbolDecorators(void);
|
||||
|
||||
// Registers an address range to a file mapping.
|
||||
//
|
||||
// Preconditions:
|
||||
// start <= end
|
||||
// filename != nullptr
|
||||
//
|
||||
// Returns true if the file was successfully registered.
|
||||
bool RegisterFileMappingHint(
|
||||
const void* start, const void* end, uint64_t offset, const char* filename);
|
||||
|
||||
// Looks up the file mapping registered by RegisterFileMappingHint for an
|
||||
// address range. If there is one, the file name is stored in *filename and
|
||||
// *start and *end are modified to reflect the registered mapping. Returns
|
||||
// whether any hint was found.
|
||||
bool GetFileMappingHint(const void** start,
|
||||
const void** end,
|
||||
uint64_t * offset,
|
||||
const char** filename);
|
||||
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_SYMBOLIZE_H_
|
||||
+194
@@ -0,0 +1,194 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// Allow dynamic symbol lookup in the kernel VDSO page.
|
||||
//
|
||||
// VDSOSupport -- a class representing kernel VDSO (if present).
|
||||
|
||||
#include "absl/debugging/internal/vdso_support.h"
|
||||
|
||||
#ifdef ABSL_HAVE_VDSO_SUPPORT // defined in vdso_support.h
|
||||
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <sys/syscall.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#if __GLIBC_PREREQ(2, 16) // GLIBC-2.16 implements getauxval.
|
||||
#include <sys/auxv.h>
|
||||
#endif
|
||||
|
||||
#include "absl/base/dynamic_annotations.h"
|
||||
#include "absl/base/internal/raw_logging.h"
|
||||
#include "absl/base/port.h"
|
||||
|
||||
#ifndef AT_SYSINFO_EHDR
|
||||
#define AT_SYSINFO_EHDR 33 // for crosstoolv10
|
||||
#endif
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
|
||||
ABSL_CONST_INIT
|
||||
std::atomic<const void *> VDSOSupport::vdso_base_(
|
||||
debugging_internal::ElfMemImage::kInvalidBase);
|
||||
|
||||
std::atomic<VDSOSupport::GetCpuFn> VDSOSupport::getcpu_fn_(&InitAndGetCPU);
|
||||
VDSOSupport::VDSOSupport()
|
||||
// If vdso_base_ is still set to kInvalidBase, we got here
|
||||
// before VDSOSupport::Init has been called. Call it now.
|
||||
: image_(vdso_base_.load(std::memory_order_relaxed) ==
|
||||
debugging_internal::ElfMemImage::kInvalidBase
|
||||
? Init()
|
||||
: vdso_base_.load(std::memory_order_relaxed)) {}
|
||||
|
||||
// NOTE: we can't use GoogleOnceInit() below, because we can be
|
||||
// called by tcmalloc, and none of the *once* stuff may be functional yet.
|
||||
//
|
||||
// In addition, we hope that the VDSOSupportHelper constructor
|
||||
// causes this code to run before there are any threads, and before
|
||||
// InitGoogle() has executed any chroot or setuid calls.
|
||||
//
|
||||
// Finally, even if there is a race here, it is harmless, because
|
||||
// the operation should be idempotent.
|
||||
const void *VDSOSupport::Init() {
|
||||
const auto kInvalidBase = debugging_internal::ElfMemImage::kInvalidBase;
|
||||
#if __GLIBC_PREREQ(2, 16)
|
||||
if (vdso_base_.load(std::memory_order_relaxed) == kInvalidBase) {
|
||||
errno = 0;
|
||||
const void *const sysinfo_ehdr =
|
||||
reinterpret_cast<const void *>(getauxval(AT_SYSINFO_EHDR));
|
||||
if (errno == 0) {
|
||||
vdso_base_.store(sysinfo_ehdr, std::memory_order_relaxed);
|
||||
}
|
||||
}
|
||||
#endif // __GLIBC_PREREQ(2, 16)
|
||||
if (vdso_base_.load(std::memory_order_relaxed) == kInvalidBase) {
|
||||
// Valgrind zaps AT_SYSINFO_EHDR and friends from the auxv[]
|
||||
// on stack, and so glibc works as if VDSO was not present.
|
||||
// But going directly to kernel via /proc/self/auxv below bypasses
|
||||
// Valgrind zapping. So we check for Valgrind separately.
|
||||
if (RunningOnValgrind()) {
|
||||
vdso_base_.store(nullptr, std::memory_order_relaxed);
|
||||
getcpu_fn_.store(&GetCPUViaSyscall, std::memory_order_relaxed);
|
||||
return nullptr;
|
||||
}
|
||||
int fd = open("/proc/self/auxv", O_RDONLY);
|
||||
if (fd == -1) {
|
||||
// Kernel too old to have a VDSO.
|
||||
vdso_base_.store(nullptr, std::memory_order_relaxed);
|
||||
getcpu_fn_.store(&GetCPUViaSyscall, std::memory_order_relaxed);
|
||||
return nullptr;
|
||||
}
|
||||
ElfW(auxv_t) aux;
|
||||
while (read(fd, &aux, sizeof(aux)) == sizeof(aux)) {
|
||||
if (aux.a_type == AT_SYSINFO_EHDR) {
|
||||
vdso_base_.store(reinterpret_cast<void *>(aux.a_un.a_val),
|
||||
std::memory_order_relaxed);
|
||||
break;
|
||||
}
|
||||
}
|
||||
close(fd);
|
||||
if (vdso_base_.load(std::memory_order_relaxed) == kInvalidBase) {
|
||||
// Didn't find AT_SYSINFO_EHDR in auxv[].
|
||||
vdso_base_.store(nullptr, std::memory_order_relaxed);
|
||||
}
|
||||
}
|
||||
GetCpuFn fn = &GetCPUViaSyscall; // default if VDSO not present.
|
||||
if (vdso_base_.load(std::memory_order_relaxed)) {
|
||||
VDSOSupport vdso;
|
||||
SymbolInfo info;
|
||||
if (vdso.LookupSymbol("__vdso_getcpu", "LINUX_2.6", STT_FUNC, &info)) {
|
||||
fn = reinterpret_cast<GetCpuFn>(const_cast<void *>(info.address));
|
||||
}
|
||||
}
|
||||
// Subtle: this code runs outside of any locks; prevent compiler
|
||||
// from assigning to getcpu_fn_ more than once.
|
||||
getcpu_fn_.store(fn, std::memory_order_relaxed);
|
||||
return vdso_base_.load(std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
const void *VDSOSupport::SetBase(const void *base) {
|
||||
ABSL_RAW_CHECK(base != debugging_internal::ElfMemImage::kInvalidBase,
|
||||
"internal error");
|
||||
const void *old_base = vdso_base_.load(std::memory_order_relaxed);
|
||||
vdso_base_.store(base, std::memory_order_relaxed);
|
||||
image_.Init(base);
|
||||
// Also reset getcpu_fn_, so GetCPU could be tested with simulated VDSO.
|
||||
getcpu_fn_.store(&InitAndGetCPU, std::memory_order_relaxed);
|
||||
return old_base;
|
||||
}
|
||||
|
||||
bool VDSOSupport::LookupSymbol(const char *name,
|
||||
const char *version,
|
||||
int type,
|
||||
SymbolInfo *info) const {
|
||||
return image_.LookupSymbol(name, version, type, info);
|
||||
}
|
||||
|
||||
bool VDSOSupport::LookupSymbolByAddress(const void *address,
|
||||
SymbolInfo *info_out) const {
|
||||
return image_.LookupSymbolByAddress(address, info_out);
|
||||
}
|
||||
|
||||
// NOLINT on 'long' because this routine mimics kernel api.
|
||||
long VDSOSupport::GetCPUViaSyscall(unsigned *cpu, // NOLINT(runtime/int)
|
||||
void *, void *) {
|
||||
#ifdef SYS_getcpu
|
||||
return syscall(SYS_getcpu, cpu, nullptr, nullptr);
|
||||
#else
|
||||
// x86_64 never implemented sys_getcpu(), except as a VDSO call.
|
||||
static_cast<void>(cpu); // Avoid an unused argument compiler warning.
|
||||
errno = ENOSYS;
|
||||
return -1;
|
||||
#endif
|
||||
}
|
||||
|
||||
// Use fast __vdso_getcpu if available.
|
||||
long VDSOSupport::InitAndGetCPU(unsigned *cpu, // NOLINT(runtime/int)
|
||||
void *x, void *y) {
|
||||
Init();
|
||||
GetCpuFn fn = getcpu_fn_.load(std::memory_order_relaxed);
|
||||
ABSL_RAW_CHECK(fn != &InitAndGetCPU, "Init() did not set getcpu_fn_");
|
||||
return (*fn)(cpu, x, y);
|
||||
}
|
||||
|
||||
// This function must be very fast, and may be called from very
|
||||
// low level (e.g. tcmalloc). Hence I avoid things like
|
||||
// GoogleOnceInit() and ::operator new.
|
||||
ABSL_ATTRIBUTE_NO_SANITIZE_MEMORY
|
||||
int GetCPU() {
|
||||
unsigned cpu;
|
||||
int ret_code = (*VDSOSupport::getcpu_fn_)(&cpu, nullptr, nullptr);
|
||||
return ret_code == 0 ? cpu : ret_code;
|
||||
}
|
||||
|
||||
// We need to make sure VDSOSupport::Init() is called before
|
||||
// InitGoogle() does any setuid or chroot calls. If VDSOSupport
|
||||
// is used in any global constructor, this will happen, since
|
||||
// VDSOSupport's constructor calls Init. But if not, we need to
|
||||
// ensure it here, with a global constructor of our own. This
|
||||
// is an allowed exception to the normal rule against non-trivial
|
||||
// global constructors.
|
||||
static class VDSOInitHelper {
|
||||
public:
|
||||
VDSOInitHelper() { VDSOSupport::Init(); }
|
||||
} vdso_init_helper;
|
||||
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_HAVE_VDSO_SUPPORT
|
||||
+158
@@ -0,0 +1,158 @@
|
||||
//
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
|
||||
// Allow dynamic symbol lookup in the kernel VDSO page.
|
||||
//
|
||||
// VDSO stands for "Virtual Dynamic Shared Object" -- a page of
|
||||
// executable code, which looks like a shared library, but doesn't
|
||||
// necessarily exist anywhere on disk, and which gets mmap()ed into
|
||||
// every process by kernels which support VDSO, such as 2.6.x for 32-bit
|
||||
// executables, and 2.6.24 and above for 64-bit executables.
|
||||
//
|
||||
// More details could be found here:
|
||||
// http://www.trilithium.com/johan/2005/08/linux-gate/
|
||||
//
|
||||
// VDSOSupport -- a class representing kernel VDSO (if present).
|
||||
//
|
||||
// Example usage:
|
||||
// VDSOSupport vdso;
|
||||
// VDSOSupport::SymbolInfo info;
|
||||
// typedef (*FN)(unsigned *, void *, void *);
|
||||
// FN fn = nullptr;
|
||||
// if (vdso.LookupSymbol("__vdso_getcpu", "LINUX_2.6", STT_FUNC, &info)) {
|
||||
// fn = reinterpret_cast<FN>(info.address);
|
||||
// }
|
||||
|
||||
#ifndef ABSL_DEBUGGING_INTERNAL_VDSO_SUPPORT_H_
|
||||
#define ABSL_DEBUGGING_INTERNAL_VDSO_SUPPORT_H_
|
||||
|
||||
#include <atomic>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/debugging/internal/elf_mem_image.h"
|
||||
|
||||
#ifdef ABSL_HAVE_ELF_MEM_IMAGE
|
||||
|
||||
#ifdef ABSL_HAVE_VDSO_SUPPORT
|
||||
#error ABSL_HAVE_VDSO_SUPPORT cannot be directly set
|
||||
#else
|
||||
#define ABSL_HAVE_VDSO_SUPPORT 1
|
||||
#endif
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace debugging_internal {
|
||||
|
||||
// NOTE: this class may be used from within tcmalloc, and can not
|
||||
// use any memory allocation routines.
|
||||
class VDSOSupport {
|
||||
public:
|
||||
VDSOSupport();
|
||||
|
||||
typedef ElfMemImage::SymbolInfo SymbolInfo;
|
||||
typedef ElfMemImage::SymbolIterator SymbolIterator;
|
||||
|
||||
// On PowerPC64 VDSO symbols can either be of type STT_FUNC or STT_NOTYPE
|
||||
// depending on how the kernel is built. The kernel is normally built with
|
||||
// STT_NOTYPE type VDSO symbols. Let's make things simpler first by using a
|
||||
// compile-time constant.
|
||||
#ifdef __powerpc64__
|
||||
enum { kVDSOSymbolType = STT_NOTYPE };
|
||||
#else
|
||||
enum { kVDSOSymbolType = STT_FUNC };
|
||||
#endif
|
||||
|
||||
// Answers whether we have a vdso at all.
|
||||
bool IsPresent() const { return image_.IsPresent(); }
|
||||
|
||||
// Allow to iterate over all VDSO symbols.
|
||||
SymbolIterator begin() const { return image_.begin(); }
|
||||
SymbolIterator end() const { return image_.end(); }
|
||||
|
||||
// Look up versioned dynamic symbol in the kernel VDSO.
|
||||
// Returns false if VDSO is not present, or doesn't contain given
|
||||
// symbol/version/type combination.
|
||||
// If info_out != nullptr, additional details are filled in.
|
||||
bool LookupSymbol(const char *name, const char *version,
|
||||
int symbol_type, SymbolInfo *info_out) const;
|
||||
|
||||
// Find info about symbol (if any) which overlaps given address.
|
||||
// Returns true if symbol was found; false if VDSO isn't present
|
||||
// or doesn't have a symbol overlapping given address.
|
||||
// If info_out != nullptr, additional details are filled in.
|
||||
bool LookupSymbolByAddress(const void *address, SymbolInfo *info_out) const;
|
||||
|
||||
// Used only for testing. Replace real VDSO base with a mock.
|
||||
// Returns previous value of vdso_base_. After you are done testing,
|
||||
// you are expected to call SetBase() with previous value, in order to
|
||||
// reset state to the way it was.
|
||||
const void *SetBase(const void *s);
|
||||
|
||||
// Computes vdso_base_ and returns it. Should be called as early as
|
||||
// possible; before any thread creation, chroot or setuid.
|
||||
static const void *Init();
|
||||
|
||||
private:
|
||||
// image_ represents VDSO ELF image in memory.
|
||||
// image_.ehdr_ == nullptr implies there is no VDSO.
|
||||
ElfMemImage image_;
|
||||
|
||||
// Cached value of auxv AT_SYSINFO_EHDR, computed once.
|
||||
// This is a tri-state:
|
||||
// kInvalidBase => value hasn't been determined yet.
|
||||
// 0 => there is no VDSO.
|
||||
// else => vma of VDSO Elf{32,64}_Ehdr.
|
||||
//
|
||||
// When testing with mock VDSO, low bit is set.
|
||||
// The low bit is always available because vdso_base_ is
|
||||
// page-aligned.
|
||||
static std::atomic<const void *> vdso_base_;
|
||||
|
||||
// NOLINT on 'long' because these routines mimic kernel api.
|
||||
// The 'cache' parameter may be used by some versions of the kernel,
|
||||
// and should be nullptr or point to a static buffer containing at
|
||||
// least two 'long's.
|
||||
static long InitAndGetCPU(unsigned *cpu, void *cache, // NOLINT 'long'.
|
||||
void *unused);
|
||||
static long GetCPUViaSyscall(unsigned *cpu, void *cache, // NOLINT 'long'.
|
||||
void *unused);
|
||||
typedef long (*GetCpuFn)(unsigned *cpu, void *cache, // NOLINT 'long'.
|
||||
void *unused);
|
||||
|
||||
// This function pointer may point to InitAndGetCPU,
|
||||
// GetCPUViaSyscall, or __vdso_getcpu at different stages of initialization.
|
||||
ABSL_CONST_INIT static std::atomic<GetCpuFn> getcpu_fn_;
|
||||
|
||||
friend int GetCPU(void); // Needs access to getcpu_fn_.
|
||||
|
||||
VDSOSupport(const VDSOSupport&) = delete;
|
||||
VDSOSupport& operator=(const VDSOSupport&) = delete;
|
||||
};
|
||||
|
||||
// Same as sched_getcpu() on later glibc versions.
|
||||
// Return current CPU, using (fast) __vdso_getcpu@LINUX_2.6 if present,
|
||||
// otherwise use syscall(SYS_getcpu,...).
|
||||
// May return -1 with errno == ENOSYS if the kernel doesn't
|
||||
// support SYS_getcpu.
|
||||
int GetCPU();
|
||||
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_HAVE_ELF_MEM_IMAGE
|
||||
|
||||
#endif // ABSL_DEBUGGING_INTERNAL_VDSO_SUPPORT_H_
|
||||
Generated
+140
@@ -0,0 +1,140 @@
|
||||
// Copyright 2017 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// Produce stack trace.
|
||||
//
|
||||
// There are three different ways we can try to get the stack trace:
|
||||
//
|
||||
// 1) Our hand-coded stack-unwinder. This depends on a certain stack
|
||||
// layout, which is used by gcc (and those systems using a
|
||||
// gcc-compatible ABI) on x86 systems, at least since gcc 2.95.
|
||||
// It uses the frame pointer to do its work.
|
||||
//
|
||||
// 2) The libunwind library. This is still in development, and as a
|
||||
// separate library adds a new dependency, but doesn't need a frame
|
||||
// pointer. It also doesn't call malloc.
|
||||
//
|
||||
// 3) The gdb unwinder -- also the one used by the c++ exception code.
|
||||
// It's obviously well-tested, but has a fatal flaw: it can call
|
||||
// malloc() from the unwinder. This is a problem because we're
|
||||
// trying to use the unwinder to instrument malloc().
|
||||
//
|
||||
// Note: if you add a new implementation here, make sure it works
|
||||
// correctly when absl::GetStackTrace() is called with max_depth == 0.
|
||||
// Some code may do that.
|
||||
|
||||
#include "absl/debugging/stacktrace.h"
|
||||
|
||||
#include <atomic>
|
||||
|
||||
#include "absl/base/attributes.h"
|
||||
#include "absl/base/port.h"
|
||||
#include "absl/debugging/internal/stacktrace_config.h"
|
||||
|
||||
#if defined(ABSL_STACKTRACE_INL_HEADER)
|
||||
#include ABSL_STACKTRACE_INL_HEADER
|
||||
#else
|
||||
# error Cannot calculate stack trace: will need to write for your environment
|
||||
|
||||
# include "absl/debugging/internal/stacktrace_aarch64-inl.inc"
|
||||
# include "absl/debugging/internal/stacktrace_arm-inl.inc"
|
||||
# include "absl/debugging/internal/stacktrace_generic-inl.inc"
|
||||
# include "absl/debugging/internal/stacktrace_powerpc-inl.inc"
|
||||
# include "absl/debugging/internal/stacktrace_unimplemented-inl.inc"
|
||||
# include "absl/debugging/internal/stacktrace_win32-inl.inc"
|
||||
# include "absl/debugging/internal/stacktrace_x86-inl.inc"
|
||||
#endif
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
namespace {
|
||||
|
||||
typedef int (*Unwinder)(void**, int*, int, int, const void*, int*);
|
||||
std::atomic<Unwinder> custom;
|
||||
|
||||
template <bool IS_STACK_FRAMES, bool IS_WITH_CONTEXT>
|
||||
ABSL_ATTRIBUTE_ALWAYS_INLINE inline int Unwind(void** result, int* sizes,
|
||||
int max_depth, int skip_count,
|
||||
const void* uc,
|
||||
int* min_dropped_frames) {
|
||||
Unwinder f = &UnwindImpl<IS_STACK_FRAMES, IS_WITH_CONTEXT>;
|
||||
Unwinder g = custom.load(std::memory_order_acquire);
|
||||
if (g != nullptr) f = g;
|
||||
|
||||
// Add 1 to skip count for the unwinder function itself
|
||||
int size = (*f)(result, sizes, max_depth, skip_count + 1, uc,
|
||||
min_dropped_frames);
|
||||
// To disable tail call to (*f)(...)
|
||||
ABSL_BLOCK_TAIL_CALL_OPTIMIZATION();
|
||||
return size;
|
||||
}
|
||||
|
||||
} // anonymous namespace
|
||||
|
||||
ABSL_ATTRIBUTE_NOINLINE ABSL_ATTRIBUTE_NO_TAIL_CALL int GetStackFrames(
|
||||
void** result, int* sizes, int max_depth, int skip_count) {
|
||||
return Unwind<true, false>(result, sizes, max_depth, skip_count, nullptr,
|
||||
nullptr);
|
||||
}
|
||||
|
||||
ABSL_ATTRIBUTE_NOINLINE ABSL_ATTRIBUTE_NO_TAIL_CALL int
|
||||
GetStackFramesWithContext(void** result, int* sizes, int max_depth,
|
||||
int skip_count, const void* uc,
|
||||
int* min_dropped_frames) {
|
||||
return Unwind<true, true>(result, sizes, max_depth, skip_count, uc,
|
||||
min_dropped_frames);
|
||||
}
|
||||
|
||||
ABSL_ATTRIBUTE_NOINLINE ABSL_ATTRIBUTE_NO_TAIL_CALL int GetStackTrace(
|
||||
void** result, int max_depth, int skip_count) {
|
||||
return Unwind<false, false>(result, nullptr, max_depth, skip_count, nullptr,
|
||||
nullptr);
|
||||
}
|
||||
|
||||
ABSL_ATTRIBUTE_NOINLINE ABSL_ATTRIBUTE_NO_TAIL_CALL int
|
||||
GetStackTraceWithContext(void** result, int max_depth, int skip_count,
|
||||
const void* uc, int* min_dropped_frames) {
|
||||
return Unwind<false, true>(result, nullptr, max_depth, skip_count, uc,
|
||||
min_dropped_frames);
|
||||
}
|
||||
|
||||
void SetStackUnwinder(Unwinder w) {
|
||||
custom.store(w, std::memory_order_release);
|
||||
}
|
||||
|
||||
int DefaultStackUnwinder(void** pcs, int* sizes, int depth, int skip,
|
||||
const void* uc, int* min_dropped_frames) {
|
||||
skip++; // For this function
|
||||
Unwinder f = nullptr;
|
||||
if (sizes == nullptr) {
|
||||
if (uc == nullptr) {
|
||||
f = &UnwindImpl<false, false>;
|
||||
} else {
|
||||
f = &UnwindImpl<false, true>;
|
||||
}
|
||||
} else {
|
||||
if (uc == nullptr) {
|
||||
f = &UnwindImpl<true, false>;
|
||||
} else {
|
||||
f = &UnwindImpl<true, true>;
|
||||
}
|
||||
}
|
||||
volatile int x = 0;
|
||||
int n = (*f)(pcs, sizes, depth, skip, uc, min_dropped_frames);
|
||||
x = 1; (void) x; // To disable tail call to (*f)(...)
|
||||
return n;
|
||||
}
|
||||
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
Generated
+231
@@ -0,0 +1,231 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: stacktrace.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This file contains routines to extract the current stack trace and associated
|
||||
// stack frames. These functions are thread-safe and async-signal-safe.
|
||||
//
|
||||
// Note that stack trace functionality is platform dependent and requires
|
||||
// additional support from the compiler/build system in most cases. (That is,
|
||||
// this functionality generally only works on platforms/builds that have been
|
||||
// specifically configured to support it.)
|
||||
//
|
||||
// Note: stack traces in Abseil that do not utilize a symbolizer will result in
|
||||
// frames consisting of function addresses rather than human-readable function
|
||||
// names. (See symbolize.h for information on symbolizing these values.)
|
||||
|
||||
#ifndef ABSL_DEBUGGING_STACKTRACE_H_
|
||||
#define ABSL_DEBUGGING_STACKTRACE_H_
|
||||
|
||||
#include "absl/base/config.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
|
||||
// GetStackFrames()
|
||||
//
|
||||
// Records program counter values for up to `max_depth` frames, skipping the
|
||||
// most recent `skip_count` stack frames, stores their corresponding values
|
||||
// and sizes in `results` and `sizes` buffers, and returns the number of frames
|
||||
// stored. (Note that the frame generated for the `absl::GetStackFrames()`
|
||||
// routine itself is also skipped.)
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// main() { foo(); }
|
||||
// foo() { bar(); }
|
||||
// bar() {
|
||||
// void* result[10];
|
||||
// int sizes[10];
|
||||
// int depth = absl::GetStackFrames(result, sizes, 10, 1);
|
||||
// }
|
||||
//
|
||||
// The current stack frame would consist of three function calls: `bar()`,
|
||||
// `foo()`, and then `main()`; however, since the `GetStackFrames()` call sets
|
||||
// `skip_count` to `1`, it will skip the frame for `bar()`, the most recently
|
||||
// invoked function call. It will therefore return 2 and fill `result` with
|
||||
// program counters within the following functions:
|
||||
//
|
||||
// result[0] foo()
|
||||
// result[1] main()
|
||||
//
|
||||
// (Note: in practice, a few more entries after `main()` may be added to account
|
||||
// for startup processes.)
|
||||
//
|
||||
// Corresponding stack frame sizes will also be recorded:
|
||||
//
|
||||
// sizes[0] 16
|
||||
// sizes[1] 16
|
||||
//
|
||||
// (Stack frame sizes of `16` above are just for illustration purposes.)
|
||||
//
|
||||
// Stack frame sizes of 0 or less indicate that those frame sizes couldn't
|
||||
// be identified.
|
||||
//
|
||||
// This routine may return fewer stack frame entries than are
|
||||
// available. Also note that `result` and `sizes` must both be non-null.
|
||||
extern int GetStackFrames(void** result, int* sizes, int max_depth,
|
||||
int skip_count);
|
||||
|
||||
// GetStackFramesWithContext()
|
||||
//
|
||||
// Records program counter values obtained from a signal handler. Records
|
||||
// program counter values for up to `max_depth` frames, skipping the most recent
|
||||
// `skip_count` stack frames, stores their corresponding values and sizes in
|
||||
// `results` and `sizes` buffers, and returns the number of frames stored. (Note
|
||||
// that the frame generated for the `absl::GetStackFramesWithContext()` routine
|
||||
// itself is also skipped.)
|
||||
//
|
||||
// The `uc` parameter, if non-null, should be a pointer to a `ucontext_t` value
|
||||
// passed to a signal handler registered via the `sa_sigaction` field of a
|
||||
// `sigaction` struct. (See
|
||||
// http://man7.org/linux/man-pages/man2/sigaction.2.html.) The `uc` value may
|
||||
// help a stack unwinder to provide a better stack trace under certain
|
||||
// conditions. `uc` may safely be null.
|
||||
//
|
||||
// The `min_dropped_frames` output parameter, if non-null, points to the
|
||||
// location to note any dropped stack frames, if any, due to buffer limitations
|
||||
// or other reasons. (This value will be set to `0` if no frames were dropped.)
|
||||
// The number of total stack frames is guaranteed to be >= skip_count +
|
||||
// max_depth + *min_dropped_frames.
|
||||
extern int GetStackFramesWithContext(void** result, int* sizes, int max_depth,
|
||||
int skip_count, const void* uc,
|
||||
int* min_dropped_frames);
|
||||
|
||||
// GetStackTrace()
|
||||
//
|
||||
// Records program counter values for up to `max_depth` frames, skipping the
|
||||
// most recent `skip_count` stack frames, stores their corresponding values
|
||||
// in `results`, and returns the number of frames
|
||||
// stored. Note that this function is similar to `absl::GetStackFrames()`
|
||||
// except that it returns the stack trace only, and not stack frame sizes.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// main() { foo(); }
|
||||
// foo() { bar(); }
|
||||
// bar() {
|
||||
// void* result[10];
|
||||
// int depth = absl::GetStackTrace(result, 10, 1);
|
||||
// }
|
||||
//
|
||||
// This produces:
|
||||
//
|
||||
// result[0] foo
|
||||
// result[1] main
|
||||
// .... ...
|
||||
//
|
||||
// `result` must not be null.
|
||||
extern int GetStackTrace(void** result, int max_depth, int skip_count);
|
||||
|
||||
// GetStackTraceWithContext()
|
||||
//
|
||||
// Records program counter values obtained from a signal handler. Records
|
||||
// program counter values for up to `max_depth` frames, skipping the most recent
|
||||
// `skip_count` stack frames, stores their corresponding values in `results`,
|
||||
// and returns the number of frames stored. (Note that the frame generated for
|
||||
// the `absl::GetStackFramesWithContext()` routine itself is also skipped.)
|
||||
//
|
||||
// The `uc` parameter, if non-null, should be a pointer to a `ucontext_t` value
|
||||
// passed to a signal handler registered via the `sa_sigaction` field of a
|
||||
// `sigaction` struct. (See
|
||||
// http://man7.org/linux/man-pages/man2/sigaction.2.html.) The `uc` value may
|
||||
// help a stack unwinder to provide a better stack trace under certain
|
||||
// conditions. `uc` may safely be null.
|
||||
//
|
||||
// The `min_dropped_frames` output parameter, if non-null, points to the
|
||||
// location to note any dropped stack frames, if any, due to buffer limitations
|
||||
// or other reasons. (This value will be set to `0` if no frames were dropped.)
|
||||
// The number of total stack frames is guaranteed to be >= skip_count +
|
||||
// max_depth + *min_dropped_frames.
|
||||
extern int GetStackTraceWithContext(void** result, int max_depth,
|
||||
int skip_count, const void* uc,
|
||||
int* min_dropped_frames);
|
||||
|
||||
// SetStackUnwinder()
|
||||
//
|
||||
// Provides a custom function for unwinding stack frames that will be used in
|
||||
// place of the default stack unwinder when invoking the static
|
||||
// GetStack{Frames,Trace}{,WithContext}() functions above.
|
||||
//
|
||||
// The arguments passed to the unwinder function will match the
|
||||
// arguments passed to `absl::GetStackFramesWithContext()` except that sizes
|
||||
// will be non-null iff the caller is interested in frame sizes.
|
||||
//
|
||||
// If unwinder is set to null, we revert to the default stack-tracing behavior.
|
||||
//
|
||||
// *****************************************************************************
|
||||
// WARNING
|
||||
// *****************************************************************************
|
||||
//
|
||||
// absl::SetStackUnwinder is not suitable for general purpose use. It is
|
||||
// provided for custom runtimes.
|
||||
// Some things to watch out for when calling `absl::SetStackUnwinder()`:
|
||||
//
|
||||
// (a) The unwinder may be called from within signal handlers and
|
||||
// therefore must be async-signal-safe.
|
||||
//
|
||||
// (b) Even after a custom stack unwinder has been unregistered, other
|
||||
// threads may still be in the process of using that unwinder.
|
||||
// Therefore do not clean up any state that may be needed by an old
|
||||
// unwinder.
|
||||
// *****************************************************************************
|
||||
extern void SetStackUnwinder(int (*unwinder)(void** pcs, int* sizes,
|
||||
int max_depth, int skip_count,
|
||||
const void* uc,
|
||||
int* min_dropped_frames));
|
||||
|
||||
// DefaultStackUnwinder()
|
||||
//
|
||||
// Records program counter values of up to `max_depth` frames, skipping the most
|
||||
// recent `skip_count` stack frames, and stores their corresponding values in
|
||||
// `pcs`. (Note that the frame generated for this call itself is also skipped.)
|
||||
// This function acts as a generic stack-unwinder; prefer usage of the more
|
||||
// specific `GetStack{Trace,Frames}{,WithContext}()` functions above.
|
||||
//
|
||||
// If you have set your own stack unwinder (with the `SetStackUnwinder()`
|
||||
// function above, you can still get the default stack unwinder by calling
|
||||
// `DefaultStackUnwinder()`, which will ignore any previously set stack unwinder
|
||||
// and use the default one instead.
|
||||
//
|
||||
// Because this function is generic, only `pcs` is guaranteed to be non-null
|
||||
// upon return. It is legal for `sizes`, `uc`, and `min_dropped_frames` to all
|
||||
// be null when called.
|
||||
//
|
||||
// The semantics are the same as the corresponding `GetStack*()` function in the
|
||||
// case where `absl::SetStackUnwinder()` was never called. Equivalents are:
|
||||
//
|
||||
// null sizes | non-nullptr sizes
|
||||
// |==========================================================|
|
||||
// null uc | GetStackTrace() | GetStackFrames() |
|
||||
// non-null uc | GetStackTraceWithContext() | GetStackFramesWithContext() |
|
||||
// |==========================================================|
|
||||
extern int DefaultStackUnwinder(void** pcs, int* sizes, int max_depth,
|
||||
int skip_count, const void* uc,
|
||||
int* min_dropped_frames);
|
||||
|
||||
namespace debugging_internal {
|
||||
// Returns true for platforms which are expected to have functioning stack trace
|
||||
// implementations. Intended to be used for tests which want to exclude
|
||||
// verification of logic known to be broken because stack traces are not
|
||||
// working.
|
||||
extern bool StackTraceWorksForTest();
|
||||
} // namespace debugging_internal
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_STACKTRACE_H_
|
||||
Generated
+25
@@ -0,0 +1,25 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
#include "absl/debugging/symbolize.h"
|
||||
|
||||
#if defined(ABSL_INTERNAL_HAVE_ELF_SYMBOLIZE)
|
||||
#include "absl/debugging/symbolize_elf.inc"
|
||||
#elif defined(_WIN32)
|
||||
// The Windows Symbolizer only works if PDB files containing the debug info
|
||||
// are available to the program at runtime.
|
||||
#include "absl/debugging/symbolize_win32.inc"
|
||||
#else
|
||||
#include "absl/debugging/symbolize_unimplemented.inc"
|
||||
#endif
|
||||
Generated
+99
@@ -0,0 +1,99 @@
|
||||
// Copyright 2018 The Abseil Authors.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// https://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// -----------------------------------------------------------------------------
|
||||
// File: symbolize.h
|
||||
// -----------------------------------------------------------------------------
|
||||
//
|
||||
// This file configures the Abseil symbolizer for use in converting instruction
|
||||
// pointer addresses (program counters) into human-readable names (function
|
||||
// calls, etc.) within Abseil code.
|
||||
//
|
||||
// The symbolizer may be invoked from several sources:
|
||||
//
|
||||
// * Implicitly, through the installation of an Abseil failure signal handler.
|
||||
// (See failure_signal_handler.h for more information.)
|
||||
// * By calling `Symbolize()` directly on a program counter you obtain through
|
||||
// `absl::GetStackTrace()` or `absl::GetStackFrames()`. (See stacktrace.h
|
||||
// for more information.
|
||||
// * By calling `Symbolize()` directly on a program counter you obtain through
|
||||
// other means (which would be platform-dependent).
|
||||
//
|
||||
// In all of the above cases, the symbolizer must first be initialized before
|
||||
// any program counter values can be symbolized. If you are installing a failure
|
||||
// signal handler, initialize the symbolizer before you do so.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// int main(int argc, char** argv) {
|
||||
// // Initialize the Symbolizer before installing the failure signal handler
|
||||
// absl::InitializeSymbolizer(argv[0]);
|
||||
//
|
||||
// // Now you may install the failure signal handler
|
||||
// absl::FailureSignalHandlerOptions options;
|
||||
// absl::InstallFailureSignalHandler(options);
|
||||
//
|
||||
// // Start running your main program
|
||||
// ...
|
||||
// return 0;
|
||||
// }
|
||||
//
|
||||
#ifndef ABSL_DEBUGGING_SYMBOLIZE_H_
|
||||
#define ABSL_DEBUGGING_SYMBOLIZE_H_
|
||||
|
||||
#include "absl/debugging/internal/symbolize.h"
|
||||
|
||||
namespace absl {
|
||||
ABSL_NAMESPACE_BEGIN
|
||||
|
||||
// InitializeSymbolizer()
|
||||
//
|
||||
// Initializes the program counter symbolizer, given the path of the program
|
||||
// (typically obtained through `main()`s `argv[0]`). The Abseil symbolizer
|
||||
// allows you to read program counters (instruction pointer values) using their
|
||||
// human-readable names within output such as stack traces.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// int main(int argc, char *argv[]) {
|
||||
// absl::InitializeSymbolizer(argv[0]);
|
||||
// // Now you can use the symbolizer
|
||||
// }
|
||||
void InitializeSymbolizer(const char* argv0);
|
||||
//
|
||||
// Symbolize()
|
||||
//
|
||||
// Symbolizes a program counter (instruction pointer value) `pc` and, on
|
||||
// success, writes the name to `out`. The symbol name is demangled, if possible.
|
||||
// Note that the symbolized name may be truncated and will be NUL-terminated.
|
||||
// Demangling is supported for symbols generated by GCC 3.x or newer). Returns
|
||||
// `false` on failure.
|
||||
//
|
||||
// Example:
|
||||
//
|
||||
// // Print a program counter and its symbol name.
|
||||
// static void DumpPCAndSymbol(void *pc) {
|
||||
// char tmp[1024];
|
||||
// const char *symbol = "(unknown)";
|
||||
// if (absl::Symbolize(pc, tmp, sizeof(tmp))) {
|
||||
// symbol = tmp;
|
||||
// }
|
||||
// absl::PrintF("%p %s\n", pc, symbol);
|
||||
// }
|
||||
bool Symbolize(const void *pc, char *out, int out_size);
|
||||
|
||||
ABSL_NAMESPACE_END
|
||||
} // namespace absl
|
||||
|
||||
#endif // ABSL_DEBUGGING_SYMBOLIZE_H_
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user