Telling stories is as basic to human beings as eating. More so, in fact, for while food makes us live, stories are what make our lives worth living - Richard Kearney
tools
CommonJS
AMD
UMD
ES6 (Harmony)
Best Practices
Tree shaking: import , export
Webpack vs Rollup vs Babel?
webpack:
code splitting,
async loading of bundles,
tree shaking,
UI Libraries have a dist folder that has the bundled and minified version for ES and UMD/CJS module systems as a target. There is a lib folder that has the transpiled version of the library.
Core Packages have just one folder which has the bundled and minified version for CJS or UMD module system as a target.
At it’s root, what you have there is an assignment statement, and according to the Google Javascript Style Guide, all statements should end in a ;. Especially assignment statements. JSLint requires ; or else it won’t pass.
Frequency Hertz
Amplitude
Wavelength
Speed
(1.4) Musical Instrument Tone
There is a huge variety of musical instruments and sounds, as you would already know from your experience with music. Even two instruments playing the same note can sound very different.
This is because a musical instrument produces a sound wave which is a combination of different but related frequencies (known as harmonics) which all mix together to create the distinctive tone or voice of the instrument.
The lowest frequency is usually dominant, and you perceive this one as the pitch. The combination of the other harmonics provides the distinctive shape of the waveform, and thereby the distinctive tone of the instrument.
String instrument frequency determined by the length of the string.
Guitar
http://hyperphysics.phy-astr.gsu.edu/hbase/Waves/string.html
Music21 is a set of tools for helping scholars and other active listeners answer questions about music quickly and simply. If you’ve ever asked yourself a question like, “I wonder how often Bach does that” or “I wish I knew which band was the first to use these chords in this order,” or “I’ll bet we’d know more about Renaissance counterpoint (or Indian ragas or post-tonal pitch structures or the form of minuets) if I could write a program to automatically write more of them,” then music21 can help you with your work.
Updating UI in dedicated thread. Single Thread UI / main thread
epoll
Windows
Android
AsyncTask
Vue
vm._events[event] stores an array of listener functions: (vm._events[event] || (vm._events[event] = [])).push(fn)
vm.$emit
vm.$on
vm.$once( event, callback )
vm.$off( [event, callback] )
// node_modules/vue/src/core/instance/events.jsexportfunctioneventsMixin(Vue:Class<Component>){consthookRE=/^hook:/Vue.prototype.$on=function(event:string|Array<string>,fn:Function):Component{constvm:Component=thisif(Array.isArray(event)){for(leti=0,l=event.length;i<l;i++){this.$on(event[i],fn)}}else{(vm._events[event]||(vm._events[event]=[])).push(fn)// optimize hook:event cost by using a boolean flag marked at registration// instead of a hash lookupif(hookRE.test(event)){vm._hasHookEvent=true}}returnvm}
https://publicobject.com/2016/01/20/strict-naming-conventions-are-a-liability/
Gson has built-in magic to convert snakes into camels with its LOWER_CASE_WITH_UNDERSCORES FieldNamingPolicy. We make a global configuration change to our Gson instance and the problem is solved.
8.4.3. Property Expressions
Property expressions can refer only to a direct property of the managed entity, as shown in the preceding example. At query creation time, you already make sure that the parsed property is a property of the managed domain class. However, you can also define constraints by traversing nested properties. Consider the following method signature:
List<Person> findByAddressZipCode(ZipCode zipCode);
Assume a Person has an Address with a ZipCode. In that case, the method creates the property traversal x.address.zipCode. The resolution algorithm starts by interpreting the entire part (AddressZipCode) as the property and checks the domain class for a property with that name (uncapitalized). If the algorithm succeeds, it uses that property. If not, the algorithm splits up the source at the camel case parts from the right side into a head and a tail and tries to find the corresponding property — in our example, AddressZip and Code. If the algorithm finds a property with that head, it takes the tail and continues building the tree down from there, splitting the tail up in the way just described. If the first split does not match, the algorithm moves the split point to the left (Address, ZipCode) and continues.
Although this should work for most cases, it is possible for the algorithm to select the wrong property. Suppose the Person class has an addressZip property as well. The algorithm would match in the first split round already, choose the wrong property, and fail (as the type of addressZip probably has no code property).
To resolve this ambiguity you can use \_ inside your method name to manually define traversal points. So our method name would be as follows:
List<Person> findByAddress_ZipCode(ZipCode zipCode);
Because we treat the underscore character as a reserved character, we strongly advise following standard Java naming conventions (that is, not using underscores in property names but using camel case instead).
Mongo References:
manual references
DBRef
Manual references are named after the referenced collection. Use the document type (collection name in singular) followed by _id ( _id ). This is the only case where you can use underscore in the middle.
For example, in a dogs collection, a document would have manual references to external documents named like this:
{
name: 'fido',
owner_id: '5358e4249611f4a65e3068ab',
race_id: '5358ee549611f4a65e3068ac',
colour: 'yellow'
...
}
Mongo follows javascript naming convention using camelCase
Mongo has no preference on camelCase v.s. underscore_style
just stay consistent with the language.
But also be consistent with on-wire json/xml format, also consider for Android/ iOS / Javascript web consumer sides.
different to relational database,
in mongoldb, key names are stored in every document, so avoid using long names.
db.collection.stats()
using shorter keys can potentially save a lot of space and even increase performance (after all, more of a smaller database can fit into memory).
WiredTiger engine in Mongo 3.0 supports compression, the point is rather moot.
Use tabs to indent. This applies to all MongoDB-specific code (chained functions) and objects used by MongoDB (queries, projections, documents).
Always have a space after a : colon.
Comma-last.
If you divide the components of an object/array into various lines, use one line for each component. The } closing curly brace should follow the last component (except for aggregation).
GENERAL GUIDELINES
No _ underscores in the middle of names (database, collection, fields) except for manual references.
Collections database, variables, properties and function names should uselowerCamelCase. They should also be descriptive. Single character variables and uncommon abbreviations should generally be avoided.
Place spaces between nested parentheticals and elements in JavaScript examples. For example, prefer { [ a, a, a ] } over {[a,a,a]}.
COLLECTION NAMES
The name should be a plural of the types of documents to be saved.
Use camelCase. Normally you shouldn’t have to because the collection name will be one word (plural of the types of documents saved).
A collection with “” empty string is not a valid collection name.
A collection name should not contain the null character because this defines the end of collection name.
Collection name should not start with the prefix “system.” as this is reserved for internal collections.
It would be good to not contain the character “$” in the collection name as various drivers available for database do not support “$” in collection name.
DATABASE NAMES
Try to have the database named after the project and one database per project.
Use camelCase.
A database with “” empty string is not a valid database name.
Database name cannot be more than 64 bytes.
Database name are case-sensitive, even on non-case-sensitive file systems. Thus it is good to keep name in lower case.
A database name cannot contain any of these characters “/, \, ., “, *, <, >, :, |, ?, $,”. It also cannot contain a single space or null character.
FIELD NAMES
Use camelCase.
Don’t use _ underscore as the starting character of the field name. The only field with _ undescore should be _id.
Field names cannot contain . dots or null characters and must not start with a $ dollar sign.
Manual references are named after the referenced collection. Use the document type (collection name in singular) followed by _id ( _id ). This is the only case where you can use underscore in the middle.
For example, in a dogs collection, a document would have manual references to external documents named like this:
{
name: 'fido',
owner_id: '5358e4249611f4a65e3068ab',
race_id: '5358ee549611f4a65e3068ac',
colour: 'yellow'
...
}
FUNCTIONS
One method per line should be used when chaining methods.
You should also indent these methods with a tab so it’s easier to tell they are part of the same chain.
If you need to use a long query, projection or options object for a function, assign it to a var and use the var in the function call to improve readability.
vue create hello_mockjs
cd hello_mockjs
npm install mockjs
npm run serve
数据模板定义规范 DTD
` ‘name|rule’: value `
random
Random is a useful nodejs commandline / library for generating random objects, like url, email, etc.
npm install random -g
➜ hello_mockjs git:(master) ✗ random
Usage: random [options] [command]
Options:
-V, --version output the version number
-h, --help output usage information
Commands:
boolean Random.boolean( min, max, cur )
bool Random.bool( min, max, cur )
natural Random.natural( min, max )
integer Random.integer( min, max )
int Random.int( min, max )
float Random.float( min, max, dmin, dmax )
character Random.character( pool )
char Random.char( pool )
string Random.string( pool, min, max )
str Random.str()
range Random.range( start, stop, step )date Random.date( format )time Random.time( format )
datetime Random.datetime( format )
now Random.now( unit, format )
image Random.image( size, background, foreground, format, text )
img Random.img()
color Random.color( name )
hex Random.hex()
rgb Random.rgb()
rgba Random.rgba()
hsl Random.hsl()
paragraph Random.paragraph( min, max )
cparagraph Random.cparagraph( min, max )
sentence Random.sentence( min, max )
csentence Random.csentence( min, max )
word Random.word( min, max )
cword Random.cword( pool, min, max )
title Random.title( min, max )
ctitle Random.ctitle( min, max )
first Random.first()
last Random.last()
name Random.name( middle )
cfirst Random.cfirst()
clast Random.clast()
cname Random.cname()
url Random.url( protocol, host )
protocol Random.protocol()
domain Random.domain( tld )
tld Random.tld()
email Random.email( domain )
ip Random.ip()
region Random.region()
province Random.province()
city Random.city( prefix )
county Random.county( prefix )
zip Random.zip( len )
d4 Random.d4()
d6 Random.d6()
d8 Random.d8()
d12 Random.d12()
d20 Random.d20()
d100 Random.d100()
guid Random.guid()
uuid Random.uuid()id Random.id()
Examples:
$ random date yyyy-MM-dd
$ random time HH:mm:ss
Node.js can create, open, read, write, delete, and close files on the server
Node.js can collect form data
Node.js can add, delete, modify data in your database
Modules
What is a Module in Node.js?
Consider modules to be the same as JavaScript libraries.A set of functions you want to include in your application.
built-in modules
Module
Description
assert
Provides a set of assertion tests
buffer
To handle binary data
child_process
To run a child process
cluster
To split a single Node process into multiple processes
crypto
To handle OpenSSL cryptographic functions
dgram
Provides implementation of UDP datagram sockets
dns
To do DNS lookups and name resolution functions
domain
Deprecated. To handle unhandled errors
events
To handle events
fs
To handle the file system
http
To make Node.js act as an HTTP server
https
To make Node.js act as an HTTPS server.
net
To create servers and clients
os
Provides information about the operation system
path
To handle file paths
punycode
Deprecated. A character encoding scheme
querystring
To handle URL query strings
readline
To handle readable streams one line at the time
stream
To handle streaming data
string_decoder
To decode buffer objects into strings
timers
To execute a function after a given number of milliseconds
tls
To implement TLS and SSL protocols
tty
Provides classes used by a text terminal
url
To parse URL strings
util
To access utility functions
v8
To access information about V8 (the JavaScript engine)
vm
To compile JavaScript code in a virtual machine
zlib
To compress or decompress files
Include Modules
To include a module, use the require() function with the name of the module:
varhttp=require('http');
Create Your Own Modules
// save as mymodule.j* exports.myDateTime=function(){returnDate();};
// in another main.jsvarmymodule=require('./mymodule')vardatetime=mymodule.myDateTime()
HTTP Module
varhttp=require('http');//create a server object:http.createServer(function(req,res){res.write('Hello World!');//write a response to the clientres.end();//end the response}).listen(8080);
nodemailer module
var nodemailer = require('nodemailer');
multiple receivers
html
Events module
addListener()
defaultMaxListeners
emit()
eventNames()
getMaxListeners()
listenerCount()
listeners()
on()
once()
prependListener()
prependOnceListener()
removeAllListeners()
removeListener()
setMaxListeners()
varevents=require('events');vareventEmitter=newevents.EventEmitter();eventEmitter.on('scream',function(){console.log('A scream is detected!');});eventEmitter.emit('scream');
The package.json file is core to the Node.js ecosystem and is a basic part of understanding and working with Node.js, npm, and even modern JavaScript. The package.json is used as what equates to a manifest about applications, modules, packages, and more - it’s a tool to that’s used to make modern development streamlined, modular, and efficient.
Instead, a component’s data option must be a function, so that each instance can maintain an independent copy of the returned data object:
data:function(){return{count:0}}
Organizing components
registration:
global
Vue.component('name', opts)
Globally registered components can be used in the template of any root Vue instance (new Vue) created afterwards – and even inside all subcomponents of that Vue instance’s component tree.
local
Props
props is like interface to class, and data is used to maitain interal state, as private fields to class.
The answer is short and simple: Scope pertains to the visibility of variables, and context refers to the object within which a function is executed.
javascript context/ scope/ this
context/ this: function
context means this keyword
Context is most often determined by how a function is invoked. When a function is called as a method of an object, this is set to the object the method is called on:
So for a global function, this refers to ‘window’ in browser
The answer is short and simple: Scope pertains to the visibility of variables, and context refers to the object within which a function is executed.
Vue-cli
https://cli.vuejs.org/
new 3.x is @vue/cli, the old is vue-cli
instant prototyping
vue serve
Create a project
vue create
Plugins
Presets
CLI service
UI
UI template
vue ui
vue-cli-service
➜ hello-vue git:(master) ✗ ./node_modules/.bin/vue-cli-service
Usage: vue-cli-service <command>[options]
Commands:
serve start development server
build build for production
inspect inspect internal webpack config
lint lint and fix source files
run vue-cli-service help[command]for usage of a specific command.
Using the Binary
vue-cli-service serve
vue-cli-service build
vue-cli-service inspect
Checking All Available Commands
Caching and Parallelization
Git Hooks
Configuration without Ejecting
webpack-dev-server
MISCS
UMD
AMD
CommonJS is a project with the goal of specifying an ecosystem for JavaScript outside the browser (for example, on the server or for native desktop applications).
// filename: foo.jsdefine(['jquery','underscore'],function($,_){// methodsfunctiona(){};// private because it's not returned (see below)functionb(){};// public because it's returnedfunctionc(){};// public because it's returned// exposed public methodsreturn{b:b,c:c}});
CommonJS <- Browserify
// filename: foo.jsvar$=require('jquery');var_=require('underscore');// methodsfunctiona(){};// private because it's omitted from module.exports (see below)functionb(){};// public because it's defined in module.exportsfunctionc(){};// public because it's defined in module.exports// exposed public methodsmodule.exports={b:b,c:c};
UMD: The pattern is admittedly ugly, but is both AMD and CommonJS compatible, as well as supporting the old-style “global” variable definition:
(function(root,factory){if(typeofdefine==='function'&&define.amd){// AMDdefine(['jquery'],factory);}elseif(typeofexports==='object'){// Node, CommonJS-likemodule.exports=factory(require('jquery'));}else{// Browser globals (root is window)root.returnExports=factory(root.jQuery);}}(this,function($){// methodsfunctionmyFunc(){};// exposed public methodreturnmyFunc;}));
MISC
js === v.s. ==
##
vuex
vuex 集中管理应用中的状态。 类似 Java Spring 中的配置中心。
一种管理的哲学思想
commit 的方式相当于 会隐藏了 setOnListener 的调用, 类似 set 方法， 而不是直接访问状态字段
state
mutations
触发变化 store.commit('increment')
调用状态: 计算属性
computed:{count(){returnstore.state.count}}
单一状态树， 唯一数据源 SSOT, 状态快照
So actions/mutations allow you to:
centrally define allowed changes to the state, and by that,
how those changes are connected and logically depend on one another (i.e. by reading an action that commits a series of mutations)
track these changes in the devtools very easily, as you see all the Mutation names you just read in the list there.
writing expressive code that, to an extend, documents your business logic. You can understand a lot about an app’s business logic by just reading its store actions & mutations, without digging through hundreds of components looking where some piece of state is altered under which circumstances.
.config Spring XML namespace configuration for MongoDB specific repositories.
.core MongoDB core support.
.core.aggregation Support for the MongoDB aggregation framework.
.core.convert Spring Data MongoDB specific converter infrastructure.
.core.geo Support for MongoDB geo-spatial queries.
.core.index Support for MongoDB document indexing.
.core.mapping Infrastructure for the MongoDB document-to-object mapping subsystem.
.core.mapping.event Mapping event callback infrastructure for the MongoDB document-to-object mapping subsystem.
.core.mapreduce Support for MongoDB map-reduce operations.
.core.query MongoDB specific query and update support.
.core.script Abstraction classes javascript function execution within MongoDB Server.
.core.spel Support classes to transform SpEL expressions into MongoDB expressions.
.crossstore Infrastructure for Spring Data’s MongoDB cross store support.
.gridfs Support for MongoDB GridFS feature.
.monitor MongoDB specific JMX monitoring support.
.repository MongoDB specific repository implementation.
.repository.cdi CDI support for MongoDB specific repository implementation.
.repository.config Support infrastructure for the configuration of MongoDB specific repositories.
.repository.query Query derivation mechanism for MongoDB specific repositories.
.repository.support Support infrastructure for query derivation of MongoDB specific repositories.
.util MongoDB driver-specific utility classes for Bson and DBObject interaction.
修改错误
{"timestamp":"2018-06-08T06:47:54.222+0000","status":500,"error":"Internal Server Error","message":"Could not extract response: no suitable HttpMessageConverter found for response type [class com.google.gson.JsonObject] and content type [text/plain]","path":"/auth/login/wechat"}
root@open2:/etc/nginx/sites-enabled# ls
city.opentown.cn opentown.cn
root@open2:/etc/nginx/sites-enabled# git diff city.opentown.cn
root@open2:/etc/nginx/sites-enabled# vim city.opentown.cn
root@open2:/etc/nginx/sites-enabled# certbot --nginx
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Which names would you like to activate HTTPS for?
-------------------------------------------------------------------------------
1: opentown.cn
2: adunk.opentown.cn
3: api.opentown.cn
4: cityapi.opentown.cn
5: m.opentown.cn
6: www.opentown.cn
-------------------------------------------------------------------------------
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 4
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for cityapi.opentown.cn
Waiting for verification...
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/city.opentown.cn
Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
-------------------------------------------------------------------------------
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
-------------------------------------------------------------------------------
Select the appropriate number [1-2] then[enter] (press 'c' to cancel): 1
-------------------------------------------------------------------------------
Congratulations! You have successfully enabled https://cityapi.opentown.cn
You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=cityapi.opentown.cn
-------------------------------------------------------------------------------
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/cityapi.opentown.cn/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/cityapi.opentown.cn/privkey.pem
Your cert will expire on 2018-09-04. To obtain a new or tweaked
version of this certificate in the future, simply run certbot again
with the "certonly" option. To non-interactively renew *all* of
your certificates, run "certbot renew"
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
Complex member variables for contracts including arbitrarily hierarchical mappings and structs
Contracts support inheritance, including multiple inheritance with C3 linearization.
An application binary interface (ABI) facilitating multiple type-safe functions within a single contract was also introduced (and later supported by Serpent).
A documentation system for specifying a user-centric description of the ramifications of a method-call was also included in the proposal, known as “Natural Language Specification”.
on EVM
compile to bytecode
designed around ECMAScript
Blockchain Basics
Transactions
Blocks
EVM
Accounts: external accounts controlled by public private key pairs
contract accounts controlled by the code stored together with the account.
Every account has a persistent key-value store mapping 256-bit words to 256-bit words called storage.
Storage:a key-value store that maps 256-bit words to 256-bit words. It is not possible to enumerate storage from within a contract and it is comparatively costly to read and even more so, to modify storage. A contract can neither read nor write to any storage apart from its own.
Memory: a contract obtains a freshly cleared instance for each message call. Memory is linear and can be addressed at byte level, but reads are limited to a width of 256 bits, while writes can be either 8 bits or 256 bits wide. Memory is expanded by a word (256-bit), when accessing (either reading or writing) a previously untouched memory word (ie. any offset within a word). At the time of expansion, the cost in gas must be paid. Memory is more costly the larger it grows (it scales quadratically).
Stack
Instruction Set
Message Calls
Delegatecall / Callcode and Libraries
Logs
Create
self-destruct
Installing the Solidity Compiler
Solidity by Example
Solidity in Depth
Security Considerations
Using the compiler
Contract Metadata
Application Binary Interface Specification
Joyfully Universal Language for (Inline) Assembly
Style Guide
Common Patterns
List of Known Bugs
Contributing
Frequently Asked Questions
##
Gloabal variable
Ether Units
wei,
finney,
szabo or
ether
Time Units
1 == 1 seconds
1 minutes == 60 seconds
1 hours == 60 minutes
1 days == 24 hours
1 weeks == 7 days
1 years == 365 days
Globals
block.blockhash(uint blockNumber) returns (bytes32): hash of the given block - only works for 256 most recent blocks excluding current
block.coinbase (address): current block miner’s address
block.difficulty (uint): current block difficulty
block.gaslimit (uint): current block gaslimit
block.number (uint): current block number
block.timestamp (uint): current block timestamp as seconds since unix epoch
gasleft() returns (uint256): remaining gas
msg.data (bytes): complete calldata
msg.gas (uint): remaining gas - deprecated in version 0.4.21 and to be replaced by gasleft()
msg.sender (address): sender of the message (current call)
msg.sig (bytes4): first four bytes of the calldata (i.e. function identifier)
msg.value (uint): number of wei sent with the message
now (uint): current block timestamp (alias for block.timestamp)
tx.gasprice (uint): gas price of the transaction
tx.origin (address): sender of the transaction (full call chain)
Error Handling
assert(bool condition): throws if the condition is not met - to be used for internal errors.
require(bool condition): throws if the condition is not met - to be used for errors in inputs or external components.
revert(): abort execution and revert state changes
All identifiers (contract names, function names and variable names) are restricted to the ASCII character set. It is possible to store UTF-8 encoded data in string variables.
public: The keyword public automatically generates a function that allows you to access the current value of the state variable from outside of the contract. Without this keyword, other contracts have no way to access the variable.
address: 160 bit value
mapping: Mappings can be seen as hash tables which are virtually initialized such that every possible key exists and is mapped to a value whose byte-representation is all zeros.
Spring Security中进行身份验证的是AuthenticationManager接口，ProviderManager是它的一个默认实现，但它并不用来处理身份认证，而是委托给配置好的AuthenticationProvider，每个AuthenticationProvider会轮流检查身份认证。检查后或者返回Authentication对象或者抛出异常。
Threads are executed in their own system-level thread (e.g., a POSIX thread or Windows threads) that is fully managed by the host operating system. Once started, threads run independently until the target function returns.
Due to a global interpreter lock (GIL), Python threads are restricted to an execution model that only allows one thread to execute in the interpreter at any given time. For this reason, Python threads should generally not be used for computationally intensive tasks where you are trying to achieve parallelism on multiple CPUs. They are much better suited for I/O handling and handling concurrent execution in code that performs blocking operations (e.g., waiting for I/O, waiting for results from a database, etc.).
An RLock or re-entrant lock object is a lock that can be acquired multiple times by the same thread. It is primarily used to implement code based locking or synchronization based on a construct known as a “monitor.” With this kind of locking, only one thread is allowed to use an entire function or the methods of a class while the lock is held.
threading.local()
Queue
Perhaps the safest way to send data from one thread to another is to use a Queue from the queue library. To do this, you create a Queue instance that is shared by the threads. Threads then use put() or get() operations to add or remove items from the queue.
In a nutshell, an actor is a concurrently executing task that simply acts upon messages sent to it. In response to these messages, it may decide to send further messages to other actors. Communication with actors is one way and asynchronous. Thus, the sender of a message does not know when a message actually gets delivered, nor does it receive a response or acknowledgment that the message has been processed.
classPool(object):Process=Processdef__init__(self,processes=None,initializer=None,initargs=(),maxtasksperchild=None):self._setup_queues()self._taskqueue=Queue.Queue()self._cache={}...# stuff we don't care about
self._worker_handler=threading.Thread(target=Pool._handle_workers,args=(self,))self._worker_handler.daemon=Trueself._worker_handler._state=RUNself._worker_handler.start()self._task_handler=threading.Thread(target=Pool._handle_tasks,args=(self._taskqueue,self._quick_put,self._outqueue,self._pool,self._cache))self._task_handler.daemon=Trueself._task_handler._state=RUNself._task_handler.start()self._result_handler=threading.Thread(target=Pool._handle_results,args=(self._outqueue,self._quick_get,self._cache))self._result_handler.daemon=Trueself._result_handler._state=RUNself._result_handler.start()
queues:
self._taskqueue = Queue.Queue(): cache all submited tasks
self._inqueue = SimpleQueue(): _task_handler fetch task from _taskqueue and add to _inqueue for process to work on, this is shared among main process and worker process.
self._outqueue = SimpleQueue(): store the result.
threads:
_worker_handler = _handle_workers(pool): maintain the internal state
_task_handler = _handle_tasks(taskqueue, put, outqueue, pool, cache): fetch tasks and send to worker process
_result_handler = _handle_results(outqueue, get, cache): process result, callback etc.
Note: callback is called in _result_handler thread in caller process.
"""Implements ProcessPoolExecutor.
The follow diagram and text describe the data-flow through the system:
|======================= In-process =====================|== Out-of-process ==|
+----------+ +----------+ +--------+ +-----------+ +---------+
| | => | Work Ids | => | | => | Call Q | => | |
| | +----------+ | | +-----------+ | |
| | | ... | | | | ... | | |
| | | 6 | | | | 5, call() | | |
| | | 7 | | | | ... | | |
| Process | | ... | | Local | +-----------+ | Process |
| Pool | +----------+ | Worker | | #1..n |
| Executor | | Thread | | |
| | +----------- + | | +-----------+ | |
| | <=> | Work Items | <=> | | <= | Result Q | <= | |
| | +------------+ | | +-----------+ | |
| | | 6: call() | | | | ... | | |
| | | future | | | | 4, result | | |
| | | ... | | | | 3, except | | |
+----------+ +------------+ +--------+ +-----------+ +---------+
Executor.submit() called:
- creates a uniquely numbered _WorkItem and adds it to the "Work Items" dict
- adds the id of the _WorkItem to the "Work Ids" queue
Local worker thread:
- reads work ids from the "Work Ids" queue and looks up the corresponding
WorkItem from the "Work Items" dict: if the work item has been cancelled then
it is simply removed from the dict, otherwise it is repackaged as a
_CallItem and put in the "Call Q". New _CallItems are put in the "Call Q"
until "Call Q" is full. NOTE: the size of the "Call Q" is kept small because
calls placed in the "Call Q" can no longer be cancelled with Future.cancel().
- reads _ResultItems from "Result Q", updates the future stored in the
"Work Items" dict and deletes the dict entry
Process #1..n:
- reads _CallItems from "Call Q", executes the calls, and puts the resulting
_ResultItems in "Request Q"
"""
Future
Future.cancel()
Future.cancelled()
Future.running()
Future.result(timeout=None)
Future.exception(timeout=None)
Future.add_done_callback(fn)
Internal methods, meant for use in unit tests and Executor implementations.
The main difference between the aforementioned map method with as_completed is that map returns the results in the order in which we pass the iterables. That is the first result from the map method is the result for the first item. On the other hand, the first result from the as_completed function is from whichever future completed first.
#Launching a Daemon Process on Unix
Debugging Python Multi-processes Program
Take aways
Another subtle aspect of pools is that mixing threads and process pools together can be a good way to make your head explode. If you are going to use both of these features together, it is often best to create the process pool as a singleton at program startup, prior to the creation of any threads. Threads will then use the same process pool for all of their computationally intensive work.
Many programmers, when faced with thread performance problems, are quick to blame the GIL for all of their ills. However, doing so is shortsighted and naive. Just as a real-world example, mysterious “stalls” in a multithreaded network program might be caused by something entirely different (e.g., a stalled DNS lookup) rather than anything related to the GIL. The bottom line is that you really need to study your code to know if the GIL is an issue or not. Again, realize that the GIL is mostly concerned with CPU-bound processing, not I/O.
Great care should be made when combining process pools and programs that use threads. In particular, you should probably create and launch process pools prior to the creation of any threads (e.g., create the pool in the main thread at program startup).
Warning: Visual Studio for Mac Community Edition does not support TypeScript natively.
So if on Mac, use Visual Studio Code instead.
or Use Visual Studio on Windows for better IDE support.
The curated list contains all the spring modules that you can use with Spring Boot as well as a refined list of third party libraries. The list is available as a standard Bills of Materials (spring-boot-dependencies) that can be used with both Maven and Gradle.
Binary Search: Using a loop or recursion to divide search space in half after each comparison
Post-processing: Determine viable candidates in the remaining space.
3 Templates for Binary Search
Template #1 is used to search for an element or condition which can be determined by accessing a single index in the array.
Search Condition can be determined without comparing to the element’s neighbors (or use specific elements around it)
No post-processing required because at each step, you are checking to see if the element has been found. If you reach the end, then you know the element is not found
Initial Condition: left = 0, right = length-1
Termination: left > right
intbinarySearch(int[]nums,inttarget){if(nums==null||nums.length==0)return-1;intleft=0,right=nums.length-1;while(left<=right){// Prevent (left + right) overflowintmid=left+(right-left)/2;if(nums[mid]==target){returnmid;}elseif(nums[mid]<target){left=mid+1;}else{right=mid-1;}}// End Condition: left > rightreturn-1;}
It is used to search for an element or condition which requires accessing the current index and its immediate right neighbor’s index in the array.
An advanced way to implement Binary Search.
Search Condition needs to access element’s immediate right neighbor
Use element’s right neighbor to determine if condition is met and decide whether to go left or right
Gurantees Search Space is at least 2 in size at each step
Post-processing required. Loop/Recursion ends when you have 1 element left. Need to assess if the remaining element meets the condition.
Initial Condition: left = 0, right = length
Termination: left == right
intbinarySearch(int[]nums,inttarget){if(nums==null||nums.length==0)return-1;intleft=0,right=nums.length;while(left<right){// Prevent (left + right) overflowintmid=left+(right-left)/2;if(nums[mid]==target){returnmid;}elseif(nums[mid]<target){left=mid+1;}else{right=mid;}}// Post-processing:// End Condition: left == rightif(left!=nums.length&&nums[left]==target)returnleft;return-1;}
Template #3 is another unique form of Binary Search. It is used to search for an element or condition which requires accessing the current index and its immediate left and right neighbor’s index in the array.
An alternative way to implement Binary Search
Search Condition needs to access element’s immediate left and right neighbors
Use element’s neighbors to determine if condition is met and decide whether to go left or right
Gurantees Search Space is at least 3 in size at each step
Post-processing required. Loop/Recursion ends when you have 2 elements left. Need to assess if the remaining elements meet the condition.
Initial Condition: left = 0, right = length-1
Termination: left + 1 == right
intbinarySearch(int[]nums,inttarget){if(nums==null||nums.length==0)return-1;intleft=0,right=nums.length-1;while(left+1<right){// Prevent (left + right) overflowintmid=left+(right-left)/2;if(nums[mid]==target){returnmid;}elseif(nums[mid]<target){left=mid;}else{right=mid;}}// Post-processing:// End Condition: left + 1 == rightif(nums[left]==target)returnleft;if(nums[right]==target)returnright;return-1;}
e.g. median of two sorted array a and b
/**
* Find all pairs (i, j) in sorted array `arr` such that `arr[i] + arr[j] > s`
*/publicintcountPairs(int[]arr,ints)intl=0;r=arr.length-1;intcount=0;while(l<r){intt=arr[l]+arr[r];if(t>s){count+=r-l;r--;}elsel++;}returncount;
/**
* find two numbers with sum equal to s in a sorted array.
*/publicint[]countPairs(int[]arr,ints)intl=0;r=arr.length-1;intcount=0;int[]result={-1,-1};while(l<r){intt=arr[l]+arr[r];if(t==s){result[0]=l;result[1]=r;returnresult;}elseif(t<s){l++;}elseif(t>s){r--;}}returnresult;
Dynamic Programming
Steps
建模
定义子问题
迭代、递归公式
边界条件
– Two eggs problem:
You have two identical eggs and you are given access to a 100 story building. You would like to know the highest floor for the egg not to break when dropped. The problem is the egg might break on the 1st floor, or even the 100th floor, you just don’t know. Find the maximum number of trials you need to conduct in order to find the answer.
– You have 5 identical jars. 4 of the jars contain balls with identical size weighing 10g each, while 1 of the jar contains balls weighing 9g each. You are given a digital scale, find out the 1 jar that has the 9g balls with only 1 weighing.
Two Pointers
Recursion
To iterate is human, to recurse divine (Anonymous)
solve( problem p )
if ( p is simple )
Solve the problem p directly
else
Divide p into new sub-problems p1, p2, ...
r1 = solve(p1)
r2 = solve(p2)
Reassemble r1, r2, ... to solve p
end
end
状态是相关变量的组合状态，一个变量相当于状态的一个分量
一次循环就是一轮状态转移（state transition）
初始状态 -> 按某种规则转移，有些状态属于终止状态，然后状态的某个变量就是所求
当前状态下，各个变量间互动，迁移，达到另一个状态，状态机的一般形式是 while (true) { //转移规则}， for (int i = 0; i < n; ++i ) {}这种是说，i这个状态分量每一轮的转移是固定的（++i），并且i == n 是触发状态机stop的方式之一。 状态机停止的一般形式是在转移的规则里的break或者 return
A tail call [tail recursion] is a kind of goto dressed as a call. A tail call happens when a function calls another as its last action, so it has nothing else to do. For instance, in the following code, the call to g is a tail call:
functionf(x)returng(x)end
After f calls g, it has nothing else to do. In such situations, the program does not need to return to the calling function when the called function ends. Therefore, after the tail call, the program does not need to keep any information about the calling function in the stack. …
Because a proper tail call uses no stack space, there is no limit on the number of “nested” tail calls that a program can make. For instance, we can call the following function with any number as argument; it will never overflow the stack:
functionfoo(n)ifn>0thenreturnfoo(n-1)endend
As I said earlier, a tail call is a kind of goto. As such, a quite useful application of proper tail calls in Lua is for programming state machines. Such applications can represent each state by a function; to change state is to go to (or to call) a specific function.
In traditional recursion, the typical model is that you perform your recursive calls first, and then you take the return value of the recursive call and calculate the result. In this manner, you don’t get the result of your calculation until you have returned from every recursive call.
In tail recursion, you perform your calculations first, and then you execute the recursive call, passing the results of your current step to the next recursive step. This results in the last statement being in the form of “(return (recursive-function params))” (I think that’s the syntax for Lisp). Basically, the return value of any given recursive step is the same as the return value of the next recursive call.
The consequence of this is that once you are ready to perform your next recursive step, you don’t need the current stack frame any more. This allows for some optimization. In fact, with an appropriately written compiler, you should never have a stack overflow snicker with a tail recursive call. Simply reuse the current stack frame for the next recursive step. I’m pretty sure Lisp does this.
Tail Recursion vs. Non-Tail Recursion
In general, (non-tail) recursive function calls put parameters on the stack
Every call grows the stack
On return, the parameters are needed to compute the result (together with the partial result returned)
In tail recursive functions, the parameters from the call before are not needed anymore
Instead, the result is directly handed to the parent
Hence, no parameters need to be put on the stack
Languages that use tail recursion optimization realize this and don’t grow the stack
The languages we cover next are optimized in this way So they are much more efficient when using recursion
➜ /tmp cat /usr/local/bin/sbt
#!/bin/shif[-f"$HOME/.sbtconfig"];then
echo"Use of ~/.sbtconfig is deprecated, please migrate global settings to /usr/local/etc/sbtopts">&2
."$HOME/.sbtconfig"fi
exec"/usr/local/Cellar/sbt/0.13.15/libexec/bin/sbt""$@"
While /usr/local/Cellar/sbt/0.13.15/libexec/bin/sbt is also a shell script wrapper.
The real sbt exe and options are configured in file /usr/local/etc/sbtopts.
➜ /tmp cat /usr/local/etc/sbtopts
# ------------------------------------------------ ## The SBT Configuration file. ## ------------------------------------------------ ## Disable ANSI color codes##-no-colors# Starts sbt even if the current directory contains no sbt project.#-sbt-create# Path to global settings/plugins directory (default: ~/.sbt)##-sbt-dir /etc/sbt# Path to shared boot directory (default: ~/.sbt/boot in 0.11 series)##-sbt-boot ~/.sbt/boot # Path to local Ivy repository (default: ~/.ivy2)##-ivy ~/.ivy2# set memory options##-mem <integer> # Use local caches for projects, no sharing.##-no-share# Put SBT in offline mode.##-offline# Sets the SBT version to use.#-sbt-version 0.11.3# Scala version (default: latest release)##-scala-home <path> #-scala-version <version># java version (default: java from PATH, currently $(java -version |& grep version))##-java-home <path>-J-Xmx2G-J-XX:+CMSClassUnloadingEnabled
The real sbt jar/ bin are under path: /usr/local/Cellar/sbt
There are many components of a play-framework project, you should take care of when configuring the versions.
play-framework
scala
sbt
sbt-plugin for play-framework
application-releated
Firstly, you choose a play-framework verison, usally, the latest one. (the current latest is 2.6.11).
Since play-framework is written in Scala, and you also need to write your application code in Scala, you have to choose a Scala version. The latest is 2.12.3. Configure it in build.sbt file.
➜ opencity-server git:(master) ✗ cat build.sbt
name := """opencity-server"""
organization := "cn.opencity"
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayScala)
scalaVersion := "2.12.3"
libraryDependencies += guice
libraryDependencies += "org.scalatestplus.play" %% "scalatestplus-play" % "3.1.2" % Test
// Adds additional packages into Twirl
//TwirlKeys.templateImports += "cn.opencity.controllers._"
// Adds additional packages into conf/routes
// play.sbt.routes.RoutesKeys.routesImport += "cn.opencity.binders._"
build-tool versions
sbt and sbt-plugin for play-framework are used to build the project.
There versions are specified under ./project/* dir configuring files.
Mill is a new build tool for Scala: it compiles your Scala code, packages it, runs it, and caches things to avoid doing unnecessary work. Mill aims to be better than Scala’s venerable old SBT build tool, learning from it’s mistakes and building upon ideas from functional programming to come up with a build tool that is fast, flexible, and easy to understand and use. This post will explore what makes Mill interesting to a Scala developer who is likely already using SBT
Eth: The C++ implementation is simply called Eth. If you want added security by running two different implementations in parallel or are serious about GPU mining, then the C++ “Eth” client is for you.
Pyethapp: The Python implementation is called Pyethapp. If you are interested in understanding how Ethereum works and how to extend it, the code base of this client is probably the most readable and has a great contract tester library with fast development cycles. It is not meant for high-end usage as performance in this client is not as high priority as clarity and reliability. If you are a Python developer that wants to build decentralized apps or are interested in Ethereum for research or an academic purpose, this is a great client: we invite you to take a look and contribute to it.
Parity a Rust implementation by Parity Technologies
A Haskell implementation developed by Blockapps
If you are interested in developing a light application that will run entirely in a web browser, then we recommend using EthereumJS as a basis.
If you want to create a small hardware project, look into the implementation for the Raspberry Pi
If you want to install geth for non-ubuntu linux then we recommend you look into building from source
If you want more flexibility on the Mac, try Homebrew
Solidity
Solidity is the major language to develop smart contracts on Ethereum blockchain.
The documentation is at https://solidity.readthedocs.io/en/develop/
##
Editors & IDEs
Remix: Browser-based IDE with integrated compiler and Solidity runtime environment without server-side components.
IntelliJ IDEA plugin: Solidity plugin for IntelliJ IDEA (and all other JetBrains IDEs)
SWARM
decentralized storage service, like ipfs.
The swarm of Swarm is the collection of nodes of the devp2p network each of which run the bzz protocol on the same network id.
ABI
solc compiler
Events
misc
Whisper: decentralized messaging protocol
geth: command line tools
geth is listening on UDP/TCP port 30303
ps aux |grep geth
lucas 19806 35.5 8.0 558645868 1348204 s001 S+ 4:24PM 1:45.52 geth console
lucas 20355 0.0 0.0 4267768 892 s004 S+ 4:30PM 0:00.00 grep geth
lsof -i-a-p 19806
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
geth 19806 lucas 13u IPv6 0x9295f2087a46ab9 0t0 UDP *:30303
geth 19806 lucas 16u IPv4 0x9295f208ea01d21 0t0 TCP 192.168.1.116:64563->39.107.26.140:61910 (ESTABLISHED)
geth 19806 lucas 21u IPv6 0x9295f2082d4f599 0t0 TCP *:30303 (LISTEN)
geth 19806 lucas 28u IPv4 0x9295f208f6f6101 0t0 TCP 192.168.1.116:63606->ns3066492.ip-79-137-70.eu:30303 (ESTABLIS
HED)
geth 19806 lucas 39u IPv4 0x9295f208ec0e101 0t0 TCP 192.168.1.116:64180->hst-46-166-161-114.balticservers.eu:30303
(ESTABLISHED)
geth 19806 lucas 66u IPv4 0x9295f2082332101 0t0 TCP 192.168.1.116:64552->47.74.5.209:30821 (SYN_SENT)
geth 19806 lucas 75u IPv4 0x9295f20823fed21 0t0 TCP 192.168.1.116:64070->47.104.15.188:30303 (ESTABLISHED)
geth 19806 lucas 88u IPv4 0x9295f20a183d681 0t0 TCP 192.168.1.116:64559->101.207.224.48:40145 (SYN_SENT)
Ethereum Network Visualization
http://ethviewer.live/
https://ethstats.net/
0x
mempools
Operations on orders:
generating,
signing,
filling and
cancelling ,
verifying an orders signature,
setting or checking a users ERC20 token balance/allowance
After upgrading to Android Studio 3.0, many configs changed.
Please check https://developer.android.com/studio/build/gradle-plugin-3-0-0-migration.html for details.
Versions
Android Studio as an IDE often breaks because of the incompatiblity of these three components:
Android Studio
Gradle
Android Plugin for Gradle
Make sure they are compatible.
In latest version as 2018/02/23,
Android Studio v3.0.1
Gradle v4.1
Android Plugin for Gradle v3.0.1
The version for Android Studio and ` Android Plugin for Gradle` should be always the same.
Naming
com.android.feature
implementation, api, compileOnly, and runtimeOnly.
Miners are required to do a Proof of Work (PoW) computation to find a x such that
SHA256(digest+x) == 000102340... with specified number of leading zeros.
This is a less strict requirement than birthday attack which requires to find x and y such that
SHA256(x) == SHA256(Y).
I guess by inspecting the internal state of SHA256 algorithm when doing a hash, one can dramatically reduce the work to find a x to satisfy PoW, which means he can mine the block much faster than others, which could be big economic incentive to optimize mining process, or, in other words, break the current bitcoin PoW mechanism.
By Using a neural network model to quick find x, can be a promising approach.
➜ module2 l /usr/local/lib/python2.7/site-packages
total 4392
drwxr-xr-x 95 lucas admin 3.0K Nov 24 18:00 .
drwxr-xr-x 4 lucas admin 128B Nov 1 20:26 ..
drwxr-xr-x 9 lucas admin 288B Nov 3 23:25 astroid-1.5.3.dist-info
drwxr-xr-x 7 lucas admin 224B Nov 3 23:25 backports
drwxr-xr-x 9 lucas admin 288B Nov 3 23:25 backports.functools_lru_cache-1.4.dist-info
Install
$ [sudo] pip install virtualenv
也可以安装源代码到本地，这样不需要 sudo 权限
$ python virtualenv.py myVE
$ ls-lah
total 256K
drwxr-xr-x 10 lucas lucas 4.0K 2017-11-27 16:01 .
drwxr-xr-x 7 lucas lucas 4.0K 2017-11-27 16:01 ..
-rw-r--r-- 1 lucas lucas 1.2K 2016-11-16 11:39 AUTHORS.txt
drwxr-xr-x 2 lucas lucas 4.0K 2016-11-16 11:39 bin
drwxr-xr-x 2 lucas lucas 4.0K 2016-11-16 11:39 docs
-rw-r--r-- 1 lucas lucas 1.2K 2016-11-16 11:39 LICENSE.txt
-rw-r--r-- 1 lucas lucas 345 2016-11-16 11:39 MANIFEST.in
drwxr-xr-x 5 lucas lucas 4.0K 2017-11-27 16:01 myVE
-rw-r--r-- 1 lucas lucas 3.4K 2016-11-16 11:39 PKG-INFO
-rw-r--r-- 1 lucas lucas 1.2K 2016-11-16 11:39 README.rst
drwxr-xr-x 2 lucas lucas 4.0K 2016-11-16 11:39 scripts
-rw-r--r-- 1 lucas lucas 88 2016-11-16 11:39 setup.cfg
-rw-r--r-- 1 lucas lucas 4.0K 2016-11-16 11:39 setup.py
drwxr-xr-x 2 lucas lucas 4.0K 2016-11-16 11:39 tests
drwxr-xr-x 2 lucas lucas 4.0K 2016-11-16 11:39 virtualenv.egg-info
drwxr-xr-x 2 lucas lucas 4.0K 2016-11-16 11:39 virtualenv_embedded
-rwxr-xr-x 1 lucas lucas 97K 2016-11-16 11:39 virtualenv.py
-rw-r--r-- 1 lucas lucas 86K 2017-11-27 16:01 virtualenv.pyc
drwxr-xr-x 2 lucas lucas 4.0K 2016-11-16 11:39 virtualenv_support
ENV/lib/ and ENV/include/ are created, containing supporting library files for a new virtualenv python. Packages installed in this environment will live under ENV/lib/pythonX.X/site-packages/.
ENV/bin is created, where executables live - noticeably a new python. Thus running a script with #! /path/to/ENV/bin/python would run that script under this virtualenv’s python.
The crucial packages pip and setuptools are installed, which allow other packages to be easily installed to the environment. This associated pip can be run from ENV/bin/pip.
G = <T,N, S,R>
• T is set of terminals (lexicon)
• N is set of non-terminals For NLP, we usually distinguish out a set
P N of preterminals which always rewrite as terminals.
• S is start symbol (one of the nonterminals)
• R is rules/productions of the form X !
, where X is a nonterminal
and
is a sequence of terminals and nonterminals (may be empty).
• A grammar G generates a language L.
生成新格式其中一个需求是针对的某些符合需求的日志做替换。
如果日志中的 IP 不在给定的 IP 网段集合中， 则需要使用备用库中的 IP 替换原始 IP。
这涉及到一个判定 IP 在不在给定 IP 网段集合的算法。
IP 网段的形式是 (1.1.1.0/24).约有3000个网段。
v1: 直接循环判定
使用 python ipaddress library:
forlineinlines:subnet=ipaddress.ip_network(line)self.localnet.append(subnet)defip_in_localnet_raw(self,ip):''' deprecated raw method, which is slow. use ip_in_localnet
'''address=ipaddress.ip_address(unicode(ip))forsubnetinself.localnet:ifaddressinsubnet:returnTruereturnFalse
这个算法当中， 每次 IP 都需要循环3000/2次判定在不在给定 IP 网段集合中。时间非常慢。
v2: bloomfilter
对于快速判定某元素在不在指定集合中， bloomfilter 是非常好的结构。
直接使用 bloomfilter， 将 IP 网段集合转换成 IP 集合。
但是3000个网段，对应的 IP 数量非常庞大，bloom filter 也非常大。
v3: bitarray
IP 数据有很好的特征。 IPV4 实际是32bit unsigned integer. 如果直接将一个 IP 在不在用一个 bit 表示，则需要2^32bit =512MB 的内存。 对于服务器来说， 可以接受。 当然可以直接使用一个 int array 。 但 Python 有现成的 library 来使用: bitarray
上面这个函数，将所有的 IP 网段转换成 IP 集合， 存到 ips[] bitarray 当中。
加载需要24s 左右。
v4: 加载速度优化
# for x in range(start, end):
# self.ips[x] = 1
# This is far more faster than add one by one.
self.ips[start:end]=1
GroupLog loading 2128 localnets done, takes 0:00:24.227503 (set one by one)
GroupLog loading 2128 localnets done, takes 0:00:00.214876 (set by [start:end])
benchmark: call 300000 times in 0:00:02.242983
经过多次优化， 加载和判定速度得到极大优化。
Python 多线程并不能利用多核
Threads are useful when the code is blocked by non bytecode execution, such as I/O or external process execution (C code, system calls, etc). If byte code execution is holding things up, the ProcessPool starts multiple interpreters that can execute in parallel. However, there is more overhead in spinning up these interpreters and in them communicating with the main thread through serialized representations (basically pickle or json over a socket if I understand correctly).
A module is a file containing Python definitions and statements. The file name is the module name with the suffix ‘.py’ appended. Within a module, the module’s name (as a string) is available as the value of the global variable __name__.
A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only the first time the module is imported somewhere.
When a module named ‘spam’ is imported, the interpreter searches for a file named ‘spam.py’ in the current directory, and then in the list of directories specified by the environment variable ‘PYTHONPATH’. This has the same syntax as the shell variable ‘PATH’, that is, a list of directory names. When ‘PYTHONPATH’ is not set, or when the file is not found there, the search continues in an installation-dependent default path; on UNIX, this is usually ‘.:/usr/local/lib/python’.
sys.path = the directory containing the input script (or the current directory), + ‘PYTHONPATH’ + the installation-dependent default.
Packages are a way of structuring Python’s module namespace by using “dotted module names”.
The __init__.py files are required to make Python treat the directories as containing packages;
` all = [“echo”, “surround”, “reverse”]`
Intra-package References
__path__. This is initialized to be a list containing the name of the directory holding the package’s __init__.py before the code in that file is executed.
__del__: Called when an instance is about to be destroyed, which lets you do any clean-up e.g. closing file handles or database connections
__repr__ and __str__: Both return a string representation of the object, but __repr__ should return a Python expression that can be used to re-create the object. The more commonly used one is __str__, which can return anything.
__cmp__: Called to compare the object with another object. Note that this is only used with Python 2.x. In Python 3.x, only rich comparison methods are used. Such as __lt__.
__lt__, __le__, __eq__, __ne__, __gt__ and __ge__: Called to compare the object with another object. These will be called if defined, otherwise Python will fall-back to using __cmp__.
__hash__: Called to calculate a hash for the object, which is used for placing objects in data structures such as sets and dictionaries.
__call__: Lets an object be “called” e.g. so that you can write things like this: obj(arg1,arg2,…).
Python also lets you define methods that let an object act like an array (so you can write things like this: obj[2] = "foo"), or like a numeric type (so you write things like this: print(obj1 + 3*obj2).
__dict__ − Dictionary containing the class’s namespace.
__doc__ − Class documentation string or none, if undefined.
__name__ − Class name.
__module__ − Module name in which the class is defined. This attribute is “__main__” in interactive mode.
__bases__ − A possibly empty tuple containing the base classes, in the order of their occurrence in the base class list.
In Python, methods and member variables are always public i.e. anyone can access them.
private methods and variables:
Class methods and variable names that start with two underscores are considered to be private to that class, However, this is only a convention
classEmployee:'Common base class for all employees'empCount=0def__init__(self,name,salary):self.name=nameself.salary=salaryEmployee.empCount+=1defdisplayCount(self):print"Total Employee %d"%Employee.empCountdefdisplayEmployee(self):print"Name : ",self.name,", Salary: ",self.salaryclass_suite
The class has a documentation string, which can be accessed via ClassName.doc.
The getattr(obj, name[, default]) − to access the attribute of object.
The hasattr(obj,name) − to check if an attribute exists or not.
The setattr(obj,name,value) − to set an attribute. If attribute does not exist, then it would be created.
sys.path is populated using the current working directory, followed by directories listed in your PYTHONPATH environment variable, followed by installation-dependent default paths, which are controlled by the site module.
The site module is automatically imported when you start Python, you can read more about how it manipulates your sys.path in the Python docs.
Loggers expose the interface that application code directly uses.
Handlers send the log records (created by loggers) to the appropriate destination.
Filters provide a finer grained facility for determining which log records to output.
Formatters specify the layout of log records in the final output.
` LogRecord`
syslog
Rsyslog
Python Logging
importlogginglogger=logging.getLogger(__name__)logger.setLevel(logging.INFO)# create a file handler
handler=logging.FileHandler('hello.log')handler.setLevel(logging.INFO)# create a logging format
formatter=logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')handler.setFormatter(formatter)# add the handlers to the logger
logger.addHandler(handler)logger.info('Hello baby')
Logger
logger names track the package/module hierarchy, and it’s intuitively obvious where events are logged just from the logger name.
Logger.setLevel() specifies the lowest-severity log message a logger will handle, where debug is the lowest built-in severity level and critical is the highest built-in severity. For example, if the severity level is INFO, the logger will handle only INFO, WARNING, ERROR, and CRITICAL messages and will ignore DEBUG messages.
Logger.addHandler() and Logger.removeHandler() add and remove handler objects from the logger object. Handlers are covered in more detail in Handlers.
Logger.addFilter() and Logger.removeFilter() add and remove filter objects from the logger object. Filters are covered in more detail in Filter Objects.
Config
logging.ini
three ways:
Creating loggers, handlers, and formatters explicitly using Python code that calls the configuration methods listed above.
Creating a logging config file and reading it using the fileConfig() function.
logging.config.fileConfig('logging.conf')
Creating a dictionary of configuration information and passing it to the dictConfig() function.
Use rotating file handler
Level/ Severity
debug(), info(), warning(), error() and critical().
Handler
class Handler()
class logging.StreamHandler(stream=None)
class logging.NullHandler
class logging.FileHandler(filename, mode='a', encoding=None, delay=False)
class logging.handlers.WatchedFileHandler(filename[, mode[, encoding[, delay]]])
class logging.handlers.RotatingFileHandler(filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=0)
doRollover()
class logging.handlers.TimedRotatingFileHandler(filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False)
class logging.handlers.SocketHandler(host, port)
class logging.handlers.DatagramHandler(host, port)
class logging.handlers.SysLogHandler(address=('localhost', SYSLOG_UDP_PORT), facility=LOG_USER, socktype=socket.SOCK_DGRAM)
class logging.handlers.NTEventLogHandler(appname, dllname=None, logtype='Application')
class logging.handlers.SMTPHandler(mailhost, fromaddr, toaddrs, subject, credentials=None, secure=None)
class logging.handlers.BufferingHandler(capacity)
class logging.handlers.MemoryHandler(capacity, flushLevel=ERROR, target=None)
class logging.handlers.HTTPHandler(host, url, method='GET')
misc
Logging variable data:
logging.warning('%s before you %s','Look','leap!')str.format()string.TemplatefromstringimportTemplates=Template('$who likes $what')s.substitute(who='tim',what='kung pao')
TODO
what is python module and __name__ variable?
if the python interpreter is running that module (the source file) as the main program, it sets the special __name__ variable to have a value “__main__”. If this file is being imported from another module, __name__ will be set to the module’s name.
A module’s __name__ is set equal to ‘__main__’ when read from standard input, a script, or from an interactive prompt.
A module can discover whether or not it is running in the main scope by checking its own __name__, which allows a common idiom for conditionally executing code in a module when it is run as a script or with python -m but not when it is imported:
For a package, the same effect can be achieved by including a __main__.py module, the contents of which will be executed when the module is run with -m.
parser: parse the webpage html into structured data
spider: define how crawler and parser work
scraper: extract data from within a webpage.
A crawler gets web pages – i.e., given a starting address (or set of starting addresses) and some conditions (e.g., how many links deep to go, types of files to ignore) it downloads whatever is linked to from the starting point(s).
` A scraper ` takes pages that have been downloaded [Edit: or, in a more general sense, data that’s formatted for display], and (attempts to) extract data from those pages, so that it can (for example) be stored in a database and manipulated as desired.
parse(self, response)
return
extract dict {}
Item
Request with parse() defined.
Selectors
grammar: response.css(selector) or response.xpath(selector)
property: a ::text
child level: div.pre_post > a
class: div.pre_post
id: div#id
functions:
extract_first() css selection results are array, return the first element of the results, even though there are only one element in the results.
props: A list/hash of attributes that are exposed to accept data from the parent component.
propsData: initial / default data for props
computed:
Questions
what is javascript prototype?
fdf
what is javascript context this?
The answer is short and simple: Scope pertains to the visibility of variables, and context refers to the object within which a function is executed.
javascript context/ scope/ this
context/ this: function
context means this keyword
Context is most often determined by how a function is invoked. When a function is called as a method of an object, this is set to the object the method is called on:
So for a global function, this refers to ‘window’ in browser
<!--PuttingjQueryintono-conflictmode.--><scriptsrc="prototype.js"></script>
<scriptsrc="jquery.js"></script>
<script>var$j=jQuery.noConflict();// $j is now an alias to the jQuery function; creating the new alias is optional.$j(document).ready(function(){$j("div").hide();});// The $ variable now has the prototype meaning, which is a shortcut for// document.getElementById(). mainDiv below is a DOM element, not a jQuery object.window.onload=function(){varmainDiv=$("main");}</script>
Attributes
setter: .attr(key, value)
getter: .attr(key)
Selecting Elements
Selecting Elements by ID
$( "#myId" ); // Note IDs must be unique per page.
* Selecting Elements by Class Name
$( “.myClass” );
link Selecting Elements by Attribute
1
$( “input[name=’first_name’]” );
link Selecting Elements by Compound CSS Selector
1
$( “#contents ul.people li” );
link Selecting Elements with a Comma-separated List of Selectors
1
$( “div.myClass, ul.people” );
The jQuery object and the native DOM element.
jQuery element is like a wrapper on a native DOM element, so you can use a lot of convinience methods and properties provided by jQuery. This is a bit like design pattern facet.
I really cannot bear the awkward of the Mac Pro Keyboard for long time. Yesterday I bought this the Rapoo V500s Mechanical Keyboard. It is pretty shinning and easy to use. The most importantly, it is cost-effectve and simple to perform.
The sound of percussion the keyboard is crisp and rhythmic.
One day I updated my MacOS to Sierra, it broke a lot of toolchain on my system.
And the most importantly, VIM is broken.
➜ _posts git:(master) ✗ vim
dyld: Library not loaded: /usr/local/opt/ruby/lib/libruby.2.3.0.dylib
Referenced from: /usr/local/bin/vim
Reason: image not found
[1] 6993 abort vim
Run brew doctor to pinpoint the issue.
➜ _posts git:(master) ✗ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry and just ignore them. Thanks!
...
.
.
.
Warning: Some installed formula are missing dependencies.
You should `brew install` the missing dependencies:
brew install libev libsodium udns
Run `brew missing` for more details.
➜ _posts git:(master) ✗ brew install libev libsodium udns
Updating Homebrew...
...
.
.
.
➜ _posts git:(master) ✗ brew upgrade vim
==> Upgrading 1 outdated package, with result:
vim 8.0.0495
...
.
.
.
➜ _posts git:(master) ✗ which vim
/usr/local/bin/vim
➜ _posts git:(master) ✗ vim
➜ _posts git:(master) ✗
You can’t modify anything in /System, /bin, /sbin, or /usr (except /usr/local); or any of the built-in apps and utilities. Only Installer and software update can modify these areas, and even they only do it when installing Apple-signed packages. But since normal OS X-style customizations go in /Library (or ~/Library, or /Applications), and unix-style customizations (e.g. Homebrew) go in /usr/local (or sometimes /etc or /opt), this shouldn’t be a big deal. It also prevents block-level writes to the startup disk, so you can’t bypass it that way.
The full list of restricted directories (and exceptions like /usr/local and a few others) is in /System/Library/Sandbox/rootless.conf. Of course, this file is itself in a restricted area.
When you upgrade to El Capitan, it moves any “unauthorized” files from restricted areas to /Library/SystemMigration/History/Migration-(some UUID)/QuarantineRoot/.
You can’t attach to system processes (e.g. those running from those system locations) for things like debugging (or changing what dynamic libraries they load, or some other things). Again, not too much of a big deal; developers can still debug their own programs.
This does block some significant things like injecting code into the built-in Apple apps (notably the Finder). It also means that dtrace-based tools for system monitoring (e.g. opensnoop) will not be able to monitor & report on many system processes.
You can’t load kernel extensions (kexts) unless they’re properly signed (i.e. by Apple or an Apple-approved developer). Note that this replaces the old system for enforcing kext signing (and the old ways of bypassing it). But since v10.10.4 Apple has had a way to enable trim support for third-party SSDs, the #1 reason to use unsigned kexts has gone away.
This post tells you how to “steal” data and code from other Android applications at Runtime.
At the core of the trick is the method called “Context.createPackageContext”. Use this method to create a context for the applications you want to retrieve data from.
After building up a Context for the other application, get the “Resources” object by “context.getResources”, then with the new Resources object, you can do whatever you want. Check the following example:
This example shows how to retrieve all string values from the famous app “WeChat”.
18906 android.lucas.swip.. I classloader=dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/android.lucas.swipeback.demo-2/base.apk"],nativeLibraryDirectories=[/vendor/lib, /system/lib]]]
18906 android.lucas.swip.. I classloader=dalvik.system.PathClassLoader[DexPathList[[zip file "/system/framework/com.google.android.maps.jar", zip file "/data/app/com.facebook.katana-1/base.apk"],nativeLibraryDirectories=[/data/app/com.facebook.katana-1/lib/arm, /vendor/lib, /system/lib]]]
As the above log shows, 2 classloaders were built, one for the current application, and the other for “WeChat” application.
String Resources
The following log prints out all string resources contained in “WeChat” package. Similarly, you can list all other major types of resources, such as layouts, drawables, raw assets, etc.
28058 android.lucas.swip.. I com.tencent.mm:string/cje, public static final int com.tencent.mm.a$m.dtP=0x7f091187, value=Reminder: You can only link your own bank cards.
28058 android.lucas.swip.. I com.tencent.mm:string/ci0, public static final int com.tencent.mm.a$m.dtQ=0x7f091153, value=Last 4 digits
28058 android.lucas.swip.. I com.tencent.mm:string/cjd, public static final int com.tencent.mm.a$m.dtR=0x7f091186, value=Your information will be encrypted
28058 android.lucas.swip.. I com.tencent.mm:string/ciz, public static final int com.tencent.mm.a$m.dtS=0x7f091177, value=Add Now
28058 android.lucas.swip.. I com.tencent.mm:string/cjt, public static final int com.tencent.mm.a$m.dtT=0x7f091196, value=You've already linked this card on WeChat Payment
28058 android.lucas.swip.. I com.tencent.mm:string/ciy, public static final int com.tencent.mm.a$m.dtU=0x7f091176, value=Open Now
28058 android.lucas.swip.. I com.tencent.mm:string/ci3, public static final int com.tencent.mm.a$m.dtV=0x7f091156, value=Registered with bank
28058 android.lucas.swip.. I com.tencent.mm:string/cjg, public static final int com.tencent.mm.a$m.dtW=0x7f091189, value=Cardholder Description
28058 android.lucas.swip.. I com.tencent.mm:string/cht, public static final int com.tencent.mm.a$m.dtX=0x7f09114c, value=Enter bank card number
28058 android.lucas.swip.. I com.tencent.mm:string/cjk, public static final int com.tencent.mm.a$m.dtY=0x7f09118d, value=Phone Number
28058 android.lucas.swip.. I com.tencent.mm:string/cjo, public static final int com.tencent.mm.a$m.dtZ=0x7f091191, value=Pay via both payment password and SMS verification
28058 android.lucas.swip.. I com.tencent.mm:string/crn, public static final int com.tencent.mm.a$m.dta=0x7f0912b7, value=Details
28058 android.lucas.swip.. I com.tencent.mm:string/cro, public static final int com.tencent.mm.a$m.dtb=0x7f0912b8, value=Top-up successful
28058 android.lucas.swip.. I com.tencent.mm:string/crm, public static final int com.tencent.mm.a$m.dtc=0x7f0912b6, value=Details
28058 android.lucas.swip.. I com.tencent.mm:string/cra, public static final int com.tencent.mm.a$m.dtd=0x7f0912aa, value=Amount (%s)
28058 android.lucas.swip.. I com.tencent.mm:string/crc, public static final int com.tencent.mm.a$m.dte=0x7f0912ac, value=Input a correct amount
28058 android.lucas.swip.. I com.tencent.mm:string/cr_, public static final int com.tencent.mm.a$m.dtf=0x7f0912a9, value=Balance Top-up
28058 android.lucas.swip.. I com.tencent.mm:string/crv, public static final int com.tencent.mm.a$m.dtg=0x7f0912bf, value=Select a bank card
28058 android.lucas.swip.. I com.tencent.mm:string/cmm, public static final int com.tencent.mm.a$m.dth=0x7f0911fe, value=The bank's system is currently undergoing maintena
Like you, as an Android developer that has benefitted from Github open source libraries a lot for long time, I start thinking how to repay this community. The best way, of course, is to share my contribution by publishing android library code. This post tells what to follow when you want to publish your Android library project on Github.
Github & Git
The first of the first, you should take some time to get familiar with Git and Github.
Gradle & Android Stuido
Familiarize yourself with Gradle and Android Studio.
Choose a License
Choose a license for your project. If you do not know how to, check this page:
http://choosealicense.com/
“Apache License 2.0” is common to Android open source libraries on Github.
Add a readme
Add “readme.md” file to tell people what the project is about and how to use it.
Publish
When you are done with all the library code and Android Studio / gradle project code stuff, it is time to share it.
Android now jcenter uses by default. It is a good idea to share your library by jcenter too so that other Android developers and use your library by just adding gradle dependency, such as:
Give it a dedicated website or webpage makes your open source library project look professional.
Github provides “Github Pages” feature so that you can create a page for your project very easily.
It has been a widely used pattern by swiping from the left edge of the screen to return to the previous page or screen. iOS Safari presents such a feature. This navigation pattern is natural and convient, and it would be great to Android developers and users if it can be ported to Android platform.
Take a look for the illustration.
The target of this project is to achieve the effect shown by iOS Safari, on Android Activity transition, like native Activity transition animation shown. Fragment transition animation is easy to implement by using ViewPager etc. and thus not discussed here.
Android implementation
how to:
If the user swipes into the screen from the outside of the left edge, the previous page slides in from the left edge while the current page slides out to the right edge of the screen.
It looks like this:
Illusiveness
To make it more illusive, we can slide the previous page from 75% X-axis, that is, the previous page has already slided in 75% percent when the finder touches the left edge, while the content covered by the current page.
The pace of the previous page is 25% of the current page. So when the current page slides from the left to the right by 100%, the underlying previous page just 25%, which makes it 100% filling the screen perfectly.
This illusiveness is what Safari on iOS does.
To implement the effect on Android, we have to solve the following obstacles:
Gesture detection
To show part of the Activity / Page on the screen.
To show the animation, we can use the native animation support for the activity but controls it manually.
The most difficult part is to show 2 Activities simultaneously on the screen.
The solutions I found online are all workarounds by using Fragment.
The open source github projects sockeqwe/SwipeBack
and ikew0ng/SwipeBackLayout
replaces the decor view and show the swipe gesture, but cannot show the previous Activity correctly.
Steps
Use GestureDetector to detect the touch event on Activity and set the root decorView of the Activity to the moved position.
override Activity’s onTouchEvent method.
Set the Activity theme to Translucent. ‘’
API Design
call android.lucas.swipeback.SwipeBack.enable()
Creating a new Activity is a bad idea because users may use different types of Activities and this also obeys the rule of “Prefer composition to inheritence”.
setBackgroundDrawable method takes effect only the parameter is TRANSPARENT. Setting the drawable to any other drawable does not work.
Add a translucent theme to the activity
<!-- Base application theme. --><stylename="AppTheme"parent="Theme.AppCompat.Light.DarkActionBar"><itemname="android:windowIsTranslucent">true</item><itemname="android:windowBackground">@android:color/transparent</item><!-- Customize your theme here. --></style>
Another Idea:
create a layout and insert it into decorview as a child and parent of LinearLayout. Intercept touch event on this layout.
// Top-level build file where you can add configuration options common to all sub-projects/modules.buildscript{repositories{jcenter()}dependencies{classpath'com.android.tools.build:gradle:0.12.+'// NOTE: Do not place your application dependencies here; they belong// in the individual module build.gradle files}}allprojects{repositories{jcenter()}}
buildscript{repositories{mavenCentral()}dependencies{classpath'com.android.tools.build:gradle:0.12.+'}}applyplugin:'android'dependencies{compile"com.android.support:support-v4:20.+"compile"com.android.support:support-v13:20.+"}// The sample build uses multiple directories to// keep boilerplate and common code separate from// the main sample code.List<String>dirs=['main',// main sample code; look here for the interesting stuff.'common',// components that are reused by multiple samples'template']// boilerplate code that is generated by the sample template processandroid{compileSdkVersion"android-L"buildToolsVersion"20.0.0"sourceSets{main{dirs.each{dir->java.srcDirs"src/${dir}/java"res.srcDirs"src/${dir}/res"}}androidTest.setRoot('tests')androidTest.java.srcDirs=['tests/src']}}
The ‘aar’ bundle is the binary distribution of an Android Library Project.
The file extension is .aar, and the maven artifact type should be aar as well, but the file itself a simple zip file with the following entries:
/AndroidManifest.xml (mandatory)
/classes.jar (mandatory)
/res/ (mandatory)
/R.txt (mandatory)
/assets/ (optional)
/libs/*.jar (optional)
/jni//*.so (optional)
/proguard.txt (optional)
/lint.jar (optional)
These entries are directly at the root of the zip file.
The R.txt file is the output of aapt with –output-text-symbols.
Genymotion is based on VirtualBox, most issues come from VirtualBox. If you cannot start a Genymotion emulator, try reinstall VirtualBox, or repair VirtualBox.
Using ./extract-files to extract vendor drivers from real device.
/media/android/cm/system/device/lge/mako$ ll extract-files.sh
-rwxrwxr-x 1 lucas lucas 354 Nov 18 23:19 extract-files.sh*
This pulls data from device’s /system directory to /vendor/lge/mako/proprietary directory.’
lucas@lucas-ubuntu:/media/android/cm/system/vendor/lge/mako/proprietary$ ll
total 24
drwxr-xr-x 6 lucas lucas 4096 Nov 19 11:29 ./
drwxr-xr-x 3 lucas lucas 4096 Nov 19 10:17 ../
drwxr-xr-x 2 lucas lucas 4096 Nov 19 11:29 bin/
drwxr-xr-x 3 lucas lucas 4096 Nov 19 11:29 etc/
drwxr-xr-x 5 lucas lucas 4096 Nov 19 11:29 lib/
drwxr-xr-x 4 lucas lucas 4096 Nov 19 11:29 vendor/
These includes drivers for CPU, NFC fromBroadcom, Camera, Sensors, Audio, DRM, Cryptography from LG, and imgs, GSM, Camera, GPS, Wi-Fi, Bluetooth, Sensors, Media, DRM, DSP, USB from Qualcomm. All these drivers are proprietary.
Kernel
The next thing is to build the kernel source
use breakfast mako (code name for Nexus 4) to download kernel source code from github
device, board support packages and configurations for a particular device. device//, /device/]
asus/grouper, device/lge/mako, where to put customized stuff for each device. `breakfast grouper && repo sync`
docs: android open source website
external, non-android utilities, with Android.mk to work with android build system
frameworks, Android core
hardware, platform/hardware specific libs
kernel, /kernel//
ndk
out, make clobber
packages, Android standalone apps, like Settings, Browser, Launcher, etc
prebuilt(s)
system: linux shell, etc.
vendor: cm, lge, htc, etc
For Nexus 4:
vendor:cm
hardware: ti
kernel: google/msm
device: lge/mako
Repo manifest format
top level ./.repo/default.xml XML file.
manifest: the root element of the file.
remote: git url shared by one or more projects and the gerrit review server those projects upload changes through.
name: short git remote name
alias: override name if specified in each project’’s .git/config
fetch: git url prefix for all projects which use this remote
review: gerrit review server used by repo update
default: used when a project does not specify remtoe and revision.
manifest-server
project: single git repo.
annotation
remove-project
include
Local Manifests
.repo/local_manifests/*.xml
/device/lge/mako/extract-files.sh
adb pull /system/* to /vendor/lge/mako/proprietary
The problems encountered during downloading the cm source
Since the source repo is huge, the downloading process can be intermittant
3. build
breakfast
brunch
lunch
build files
.mk
Makefiles
/build
each module/ subproject has a Android.mk file, telling the build system how to build the module, and where to put the output in the Android directory.
The built files are put in /out/target/project/CODENAME, with zipped flashable recovery*.zip and fastboot*.img files.
## $OUT directory
$OUT= /out/target/project/CODENAME
* kernel
* /system, the /system folder on Android
* /root, contains the files that will be turned into ramdisk loaded and run by the kernel, including init, init.rc, init.CODENAME.rc
* /recovery/root, ramdisk contains the recovery mode
```sh
mm: make modules, mm -B
adb sync system
adb remount= adb shell mount -o rw,remount /system
Build Success
/media/android/cm/system/out/target/product/mako
➜ mako ls -la
total 803408
drwxrwxr-x 10 lucas lucas 4096 Nov 19 21:15 .
drwxrwxr-x 3 lucas lucas 4096 Nov 19 18:33 ..
-rw-rw-r-- 1 lucas lucas 19 Nov 19 18:34 android-info.txt
-rw-r--r-- 1 lucas lucas 6436864 Nov 19 21:02 boot.img
-rw-rw-r-- 1 lucas lucas 34552 Nov 19 18:33 clean_steps.mk
-rw-rw-r-- 2 lucas lucas 189761342 Nov 19 21:15 cm-10.2-20131119-UNOFFICIAL-mako.zip
-rw-rw-r-- 1 lucas lucas 120 Nov 19 21:15 cm-10.2-20131119-UNOFFICIAL-mako.zip.md5sum
-rw-rw-r-- 2 lucas lucas 189761342 Nov 19 21:15 cm_mako-ota-3f402468d6.zip
drwxrwxr-x 2 lucas lucas 4096 Nov 19 20:46 data
drwxrwxr-x 4 lucas lucas 4096 Nov 19 18:39 external
drwxrwxr-x 2 lucas lucas 4096 Nov 19 21:10 fake_packages
-rw-rw-r-- 1 lucas lucas 71456 Nov 19 21:13 installed-files.txt
-rwxrwxr-x 1 lucas lucas 5997088 Nov 19 20:36 kernel
drwxrwxr-x 18 lucas lucas 4096 Nov 19 21:13 obj
-rw-rw-r-- 1 lucas lucas 49 Nov 19 21:14 ota_script_path
-rw-rw-r-- 1 lucas lucas 669 Nov 19 18:33 previous_build_config.mk
-rw-rw-r-- 1 lucas lucas 434701 Nov 19 21:01 ramdisk.img
-rw-rw-r-- 1 lucas lucas 3845120 Nov 19 21:03 ramdisk-recovery.cpio
-rw-rw-r-- 1 lucas lucas 2510001 Nov 19 21:04 ramdisk-recovery.img
drwxrwxr-x 3 lucas lucas 4096 Nov 19 21:02 recovery
-rw-r--r-- 1 lucas lucas 8511488 Nov 19 21:04 recovery.img
drwxrwxr-x 9 lucas lucas 4096 Nov 19 21:01 root
drwxrwxr-x 5 lucas lucas 4096 Nov 19 20:59 symbols
drwxrwxr-x 15 lucas lucas 4096 Nov 19 21:01 system
-rw-r--r-- 1 lucas lucas 314466532 Nov 19 21:14 system.img
-rw-r--r-- 1 lucas lucas 100775392 Nov 19 20:46 userdata.img
➜ mako
Qura: [What are good interview questions to ask JAVA developers?] (http://www.quora.com/Java-programming-language/What-are-good-interview-questions-to-ask-JAVA-developers)
Java puzzlers by Joshua Bloch. RECOMMENDED FOR SENIOR JAVA DEVELOPERS.
Effective Java by Joshua Bloch. RECOMMENDED FOR ADVANCED JUNIOR JAVA DEVELOPERS.
Java Concurrency in Practice by Tim Peierls, Brian Goetz, Joshua Bloch, Joseph Bowbeer, Doug Lea, David Holmes. RECOMMENDED FOR SERVER DEVELOPERS.
I mostly agree with John Kurlak’s answer. Personally I feel Cracking the Coding Interview: 150 Programming Questions and Solutions is a good choice if you only know Java. Programming Interviews Exposed is a little elementary and the materials actually need to be updated. Elements of Programming Interviews: 300 Questions and Solutions seems for people who actually want to pass all interviews including the hardest one after concluding all reviews on Amazon, and it is mainly written in C++ which makes this as a great choice for people who can understand on C++.
The following shuffle method purports to shuffle its input array fairly. In other words, it purports to generate all permutations with equal likelihood, assuming that the underlying pseudorandom number generator is fair. Does it make good on its promise? If not, how do you fix it?
Thispuzzletestsyourknowledgeofrecursion.Whatdoesthisprogramdo?publicclassWorkout{publicstaticvoidmain(String[]args){workHard();System.out.println("It's nap time.");}privatestaticvoidworkHard(){try{workHard();}finally{workHard();}}}
Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.
Clock puzzles - find the angle between hr,min,sec handles, etc
From Twitter http://qandwhat.apps.runkite.com/i-failed-a-twitter-interview/
"Consider the following picture:"
"In this picture we have walls of different heights. This picture is represented by an array of integers, where the value at each index is the height of the wall. The picture above is represented with an array as [2,5,1,2,3,4,7,7,6]."
"Now imagine it rains. How much water is going to be accumulated in puddles between walls?"
"We count volume in square blocks of 1X1. So in the picture above, everything to the left of index 1 spills out. Water to the right of index 7 also spills out. We are left with a puddle between 1 and 6 and the volume is 10."
Online Tests
Test For Geeks, including java and android tests. Extremely challenging.
Once I found this interview question: calculate Pi.
Where, all of us have learned calculating Pi in our primary math class by using regular polygon to approach a circle.
Today I am gonna show you a probability way.
Method
Check the following graph. I hope it is self-explainatory.
classPi{publicstaticvoidmain(String[]args){if(args.length<1)System.exit(0);longiter=Long.parseLong(args[0]);doublepi=calculate(iter);System.out.println(String.format("after %d iterations, pi is %f",iter,pi));}publicstaticdoublecalculate(longsimNum){longi=0L;longcount=0L;while(i++<simNum){doublex=Math.random();doubley=Math.random();doubleresult=x*x+y*y;if(result<1.0D){count++;}}return4.0D*count/simNum;}}
Experiment
lucas@lucas-ubuntu:~/dev/workspace/pi$ java Pi 1000
after 1000 iterations, pi is 3.032000
lucas@lucas-ubuntu:~/dev/workspace/pi$ java Pi 10000
after 10000 iterations, pi is 3.137200
lucas@lucas-ubuntu:~/dev/workspace/pi$ java Pi 100000
after 100000 iterations, pi is 3.140240
lucas@lucas-ubuntu:~/dev/workspace/pi$ java Pi 1000000
after 1000000 iterations, pi is 3.144108
lucas@lucas-ubuntu:~/dev/workspace/pi$ java Pi 10000000
after 10000000 iterations, pi is 3.141199
lucas@lucas-ubuntu:~/dev/workspace/pi$ java Pi 100000000
after 100000000 iterations, pi is 3.141486
lucas@lucas-ubuntu:~/dev/workspace/pi$
Conclusion
When iteration is set to 100,000,000, it takes about half a minute on my i7 Core Mac Pro with precision only to 3 digits.
This algorithm is unacceptable.
The precision cannot be guranteed as it is using a probabilistic way.
ListView is one of the most fundamental and important Android UI components. Due to the long rectangle shape of mobile device, ListView is widely used as the first level navigation to the more detailed content. Popular using scenarios include, but not limited to, news reader, reading list, e-commere product list, things to do list, ranking list, etc. If your applicaiton intends to providing updatable contents to users, you cannot avoid using this super powerful widgets.
ListView can be very simple and also extremely complex. There are many traps and tricks for novice developers. In this article, the author will summerize some typical traps developers may occur during their use of ListView and try to provide solutions or suggestions. In more general, the author will summerize typical advanced usages of ListView, such as pull to refresh ListView, HTTP cache, data binding, loader, etc.
Simple Example
Android ApiDemos Sample provide a bundle of examples for using ListView (under android-sdk-macosx/samples/android-17/ApiDemos/src/com/example/android/apis/view).
From the above simple example, we find 3 components for the ListView: the standard MVC pattern. Model, View, and Controller.
View
ListView is embeded in Activity in layout XML.
Data
To provide data for each item in the ListView, we need a Adapter. Here we use ListAdapter, constructed from a static String array.
Controller
If you want users to interact with your app, you can add Listeners for user actions, for example, click event, touch event.
This can be done by setting setOnItemClickListener.
Framework API
Official Tutorials
Official introductory tutorial can be found at this link.
Occasionally, you get the IllegalStateException error, stating
“The content of the adapter has changed but “
+ “ListView did not receive a notification. Make sure the content of “
+ “your adapter is not modified from a background thread, but only from “
+ “the UI thread. Make sure your adapter calls notifyDataSetChanged() “
+ “when its content changes. “
By checking the source code, we find that when the ListView’s ‘mItemCount’ is different from the bound adapter’s count, framework will throw the exception.
Here ‘mItemCount’ is written when the adapter is first time set or when ‘onMeasure’ is called (refresh).
<img src=”imgs/listview_itemcount_write.png”.
// Handle the empty set by removing all views that are visible// and calling it a dayif(mItemCount==0){resetList();invokeOnItemScrollListener();return;}elseif(mItemCount!=mAdapter.getCount()){thrownewIllegalStateException("The content of the adapter has changed but "+"ListView did not receive a notification. Make sure the content of "+"your adapter is not modified from a background thread, but only from "+"the UI thread. Make sure your adapter calls notifyDataSetChanged() "+"when its content changes. [in ListView("+getId()+", "+getClass()+") with Adapter("+mAdapter.getClass()+")]");}
[Google I/O 2010 - The world of ListView]<iframe width="560" height="315" src="//www.youtube.com/embed/wDBM6wVEO70" frameborder="0" allowfullscreen></iframe>
Vim provides a great tutorial tool called vim tutor. You can open it by using vim tutor or vimtutor.
By following the tutorial, you wil get a basic idea how to use vim more efficiently in 20-30 minutes.
Baiscally, vim command has the grammar:
operator times motion
Here, operator means the operation you want to do: like delete, and times is a integer that represents how many times your want to repeat the operation. motion is the sementics you want the operation does, like w, which represent a word, and e, end of a word.
so d5w mean delete the following 5 words.
If operator is missing, it means moving by default. So 5w means moving the cursor by 5 words. There are 2 special motions: $, representing the end of the line and 0, representing the beginning of the line. Typing $ will take your cursor to the end of the line and d$ will delete from the cursor to the end of the line.
Due to the high frequency of deleting the whole line when editing in vim, deleting the whole line is simply dd.
To delete multiple lines, use 5dd. You can use d5d, but that takes you longer time to type.
operater:
delete: d dw de
change: c ce
replace: r rx
Go to to line: 78 G, uppercase G
Search: /abc n for next N for previous
find matching parenthesis: %
visualize the block: v
v, then move the cursor to hightlight the selected text.
# This file is automatically generated by Android Tools.# Do not modify this file -- YOUR CHANGES WILL BE ERASED!## This file must *NOT* be checked into Version Control Systems,# as it contains information specific to your local configuration.# location of the SDK. This is only used by Ant# For customization when using a Version Control System, please read the# header note.
sdk.dir=/Users/lucas/dev/android/android-sdk-macosx
ant.properties
The ant.properties file can be created by you. It is only edited by the
'android' tool to add properties to it.
This is the place to change some Ant specific build properties.
Here are some properties you may want to change/update:
source.dir
The name of the source directory. Default is 'src'.
out.dir
The name of the output directory. Default is 'bin'.
For other overridable properties, look at the beginning of the rules
files in the SDK, at tools/ant/build.xml
Properties related to the SDK location or the project target should
be updated using the 'android' tool with the 'update' action.
This file is an integral part of the build system for your
application and should be checked into Version Control Systems.
# This file is automatically generated by Android Tools.# Do not modify this file -- YOUR CHANGES WILL BE ERASED!## This file must be checked in Version Control Systems.## To customize properties used by the Ant build system edit# "ant.properties", and override values to adapt the script to your# project structure.## To enable ProGuard to shrink and obfuscate your code, uncomment this (available properties: sdk.dir, user.home):#proguard.config=${sdk.dir}/tools/proguard/proguard-android.txt:proguard-project.txt# Project target.target=android-14
android.library=true
Default build.xml generated by android tool
lucas-ma:apf-framework lucas$ cat build.xml
<?xml version="1.0" encoding="UTF-8"?><projectname="apf-framework"default="help"><!-- The local.properties file is created and updated by the 'android' tool.
It contains the path to the SDK. It should *NOT* be checked into
Version Control Systems. --><propertyfile="local.properties"/><!-- The ant.properties file can be created by you. It is only edited by the
'android' tool to add properties to it.
This is the place to change some Ant specific build properties.
Here are some properties you may want to change/update:
source.dir
The name of the source directory. Default is 'src'.
out.dir
The name of the output directory. Default is 'bin'.
For other overridable properties, look at the beginning of the rules
files in the SDK, at tools/ant/build.xml
Properties related to the SDK location or the project target should
be updated using the 'android' tool with the 'update' action.
This file is an integral part of the build system for your
application and should be checked into Version Control Systems.
--><propertyfile="ant.properties"/><!-- if sdk.dir was not set from one of the property file, then
get it from the ANDROID_HOME env var.
This must be done before we load project.properties since
the proguard config can use sdk.dir --><propertyenvironment="env"/><conditionproperty="sdk.dir"value="${env.ANDROID_HOME}"><issetproperty="env.ANDROID_HOME"/></condition><!-- The project.properties file is created and updated by the 'android'
tool, as well as ADT.
This contains project specific properties such as project target, and library
dependencies. Lower level build properties are stored in ant.properties
(or in .classpath for Eclipse projects).
This file is an integral part of the build system for your
application and should be checked into Version Control Systems. --><loadpropertiessrcFile="project.properties"/><!-- quick check on sdk.dir --><failmessage="sdk.dir is missing. Make sure to generate local.properties using 'android update project' or to inject it through the ANDROID_HOME environment variable."unless="sdk.dir"/><!--
Import per project custom build rules if present at the root of the project.
This is the place to put custom intermediary targets such as:
-pre-build
-pre-compile
-post-compile (This is typically used for code obfuscation.
Compiled code location: ${out.classes.absolute.dir}
If this is not done in place, override ${out.dex.input.absolute.dir})
-post-package
-post-build
-pre-clean
--><importfile="custom_rules.xml"optional="true"/><!-- Import the actual build file.
To customize existing targets, there are two options:
- Customize only one target:
- copy/paste the target into this file, *before* the
<import> task.
- customize it to your needs.
- Customize the whole content of build.xml
- copy/paste the content of the rules files (minus the top node)
into this file, replacing the <import> task.
- customize to your needs.
***********************
****** IMPORTANT ******
***********************
In all cases you must update the value of version-tag below to read 'custom' instead of an integer,
in order to avoid having your file be overridden by tools such as "android update project"
--><!-- version-tag: 1 --><importfile="${sdk.dir}/tools/ant/build.xml"/></project>
To build the latest source in a Mac OS environment, you will need an Intel/x86 machine running MacOS 10.6 (Snow Leopard) or MacOS 10.7 (Lion), along with Xcode 4.2 (Apple’s Developer Tools). Although Lion does not come with a JDK, it should install automatically when you attempt to build the source.
‘
add the following to your ~/.bash_profile to mount the image when you execute “mountAndroid”:
# mount the android file imagefunction mountAndroid { hdiutil attach ~/android.dmg -mountpoint /Volumes/android;}
mountAndroid
Setting a file descriptor limit
On MacOS the default limit on the number of simultaneous file descriptors open is too low and a highly parallel build process may exceed this limit.
To increase the cap, add the following lines to your ~/.bash_profile:
# set the number of open files to be 1024ulimit-S-n 1024
</img>
http://up.2cto.com/kf/201103/20110314161540730.png
I really cannot bear the awkward of the Mac Pro Keyboard for long time. Yesterday I bought this the Rapoo V500s Mechanical Keyboard. It is pretty shinning and easy to use. The most importantly, it is cost-effectve and simple to perform.
The sound of percussion the keyboard is crisp and rhythmic.
